[GitHub] XD-DENG commented on a change in pull request #3688: [AIRFLOW-2843] ExternalTaskSensor-check if external task exists
XD-DENG commented on a change in pull request #3688: [AIRFLOW-2843] ExternalTaskSensor-check if external task exists
File path: airflow/sensors/external_task_sensor.py
@@ -70,9 +76,24 @@ def __init__(self,
self.execution_date_fn = execution_date_fn
self.external_dag_id = external_dag_id
self.external_task_id = external_task_id
+ self.check_existence = check_existence
def poke(self, context, session=None):
+ TI = TaskInstance
+ if self.check_existence:
+ existence = session.query(TI).filter(
+ TI.dag_id == self.external_dag_id,
+ TI.task_id == self.external_task_id,
+ if existence == 0:
+ raise AirflowException('The external task "' +
There may be a few cases:
- The external DAG ID specified is wrong (due to reasons like typo);
- The external task specified doesn't exist in the corresponding DAG (similar reason).
Starting on time or not doesn't matter much here, since only DAG ID& task ID are used here for querying.
I consider this feature as a "guard" to prevent from entering wrong DAG ID and task ID. Without guard, it may be hard to find this type of errors (the sensor will keep waiting and eventually leaves an impression that it fails only because the external task was not executed/finished yet).
Given the default value of this new argument is `False`, it will not change the current behavior.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
Apache Git Services