git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[GitHub] XD-DENG commented on a change in pull request #3688: [AIRFLOW-2843] ExternalTaskSensor-check if external task exists


XD-DENG commented on a change in pull request #3688: [AIRFLOW-2843] ExternalTaskSensor-check if external task exists
URL: https://github.com/apache/incubator-airflow/pull/3688#discussion_r207610690
 
 

 ##########
 File path: airflow/sensors/external_task_sensor.py
 ##########
 @@ -70,9 +76,24 @@ def __init__(self,
         self.execution_date_fn = execution_date_fn
         self.external_dag_id = external_dag_id
         self.external_task_id = external_task_id
+        self.check_existence = check_existence
 
     @provide_session
     def poke(self, context, session=None):
+        TI = TaskInstance
+
+        if self.check_existence:
+            existence = session.query(TI).filter(
+                TI.dag_id == self.external_dag_id,
+                TI.task_id == self.external_task_id,
+            ).count()
+            session.commit()
+            if existence == 0:
+                raise AirflowException('The external task "' +
 
 Review comment:
   But I do agree this feature may be too specific and only applicable to limited use cases (like some I'm dealing with).
   
   Please share your thoughts, and feel free to close this PR should you find pro < con. Thanks.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@xxxxxxxxxxxxxxxx


With regards,
Apache Git Services



( ! ) Warning: include(msgfooter.php): failed to open stream: No such file or directory in /var/www/git/apache-airflow-development/msg04073.html on line 112
Call Stack
#TimeMemoryFunctionLocation
10.0013364760{main}( ).../msg04073.html:0

( ! ) Warning: include(): Failed opening 'msgfooter.php' for inclusion (include_path='.:/var/www/git') in /var/www/git/apache-airflow-development/msg04073.html on line 112
Call Stack
#TimeMemoryFunctionLocation
10.0013364760{main}( ).../msg04073.html:0