git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: TriggerDagRunOperator sub tasks are scheduled to run after few hours


HI All,

 

Could anyone please address this issue ? We are really stuck here and not able to proceed further

 

Thanks & Regards

Goutam Sahoo

 

From: Goutam Kumar Sahoo
Sent: Friday, September 07, 2018 4:51 PM
To: 'dev@xxxxxxxxxxxxxxxxxxxxxxxxxxxx' <dev@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>; 'dev@xxxxxxxxxxxxxxxxxx' <dev@xxxxxxxxxxxxxxxxxx>
Subject: TriggerDagRunOperator sub tasks are scheduled to run after few hours

 

HI Experts

 

In our project, we are trying to replicate the existing  job scheduling implemented in Microsoft Orchestrater to new scheduler called “Apache Airflow” .

 

During this replication process we have an requirement to create Master DAG which in turn will call Other child DAGs based on different condition. We referred to the existing example (example_trigger_controller_dag, example_trigger_target_dag) available in Airflow and tried running them manually.  

 

When we triggered the controller DAG manually , it is got triggered immediately with execution time created in local timezone (PDT) whereas the child DAG is scheduled to run few hours later. Even though child DAGRUN is showing as running, it is doing nothing and waiting for the time to satisfy.  Our requirement is to trigger the Target/Child DAG run as soon it is triggered by controller DAG

 

 

If you see here , the trigger time and execution time is having a 7 hours GAP.

 

We have tried to solve this many different ways but couldn’t succeed. We have asked for help in many forums but didn’t get any satisfactory answer form any of them

 

 

Please help us ASAP as this issue is one of the show stopper for us .