git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: About how to pause the running task


There are no semantics or concept of pause within a task, though you can
clear/kill tasks which is essentially sending a poison pill to kill the
task subprocess.

Even if BaseOperator (common to all tasks) was to implement pausing
semantics, they probably wouldn't be implemented for most operators. Is it
even possible to pause a bash script externally (through SIGTSTP
<https://en.wikipedia.org/wiki/SIGTSTP>, but does that always work)? a
python script? Hive job? A Spark job? Personally I don't think that feature
would get much use. You end up freezing and not releasing a worker slot.

You can pause a DAG though. FYI when a DAG is paused, it only means it will
stop scheduling new tasks.

Max

On Thu, Apr 26, 2018 at 10:14 AM, Song Liu <songliu@xxxxxxxxxxx> wrote:

> Hi,
>
> A DAG is composed of many tasks, when this DAG is started, how to pause
> the current running task ?
>
> Thanks,
> Song
>


( ! ) Warning: include(msgfooter.php): failed to open stream: No such file or directory in /var/www/git/apache-airflow-development/msg03207.html on line 94
Call Stack
#TimeMemoryFunctionLocation
10.0007363640{main}( ).../msg03207.html:0

( ! ) Warning: include(): Failed opening 'msgfooter.php' for inclusion (include_path='.:/var/www/git') in /var/www/git/apache-airflow-development/msg03207.html on line 94
Call Stack
#TimeMemoryFunctionLocation
10.0007363640{main}( ).../msg03207.html:0