git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Bit confused about start_date and schedule_interval related to daily/weekly DAG


I'm a bit confused with how the scheduler catches up in relation to
start_date and schedule_interval. I have one dag that runs hourly:

dag = DAG(
    dag_id='hourly_dag',
    start_date=days_ago(1),
    schedule_interval='@hourly',
    default_args=ARGS)

When I start this DAG fresh it will catch up 24 hours + however many hours
have passed in the current day all the way up to the most recent hour. This
makes sense to me.

Now if I have a daily DAG:

dag = DAG(
    dag_id='daily_dag',
    start_date=days_ago(1),
    schedule_interval='0 5 * * *',
    default_args=ARGS)

Starting this DAG fresh will run yesterday's execution. This is fine since
I use the execution_date (ds_nodash) to have the task be lagged by one day.
What I can't seem to wrap my head around is how I would get this DAG to run
for the current day. I've tried passing is days_ago(0) but the tasks never
seem to start?

In addition to all that, I have a weekly DAG that must also use the
execution_date, but it needs the current weeks execution_date.

*How do I get a DAG that is not hourly to have an execution_date of the
current day or week?*


( ! ) Warning: include(msgfooter.php): failed to open stream: No such file or directory in /var/www/git/apache-airflow-development/msg03154.html on line 100
Call Stack
#TimeMemoryFunctionLocation
10.0005363704{main}( ).../msg03154.html:0

( ! ) Warning: include(): Failed opening 'msgfooter.php' for inclusion (include_path='.:/var/www/git') in /var/www/git/apache-airflow-development/msg03154.html on line 100
Call Stack
#TimeMemoryFunctionLocation
10.0005363704{main}( ).../msg03154.html:0