git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Airflow - YARN as an executor?


Happy to have it as a contrib executor. However, I personally think yarn is a dead end. It has a lot of catching up to do and all the momentum is with kubernetes. 

B.

Verstuurd vanaf mijn iPad

> Op 24 apr. 2018 om 22:13 heeft Ruslan Dautkhanov <dautkhanov@xxxxxxxxx> het volgende geschreven:
> 
> With Hadoop 3's Docker on YARN support, I think YARN becomes
> somewhat a competitor for Kubernetes.
> 
> Great job on adding k8s support to Airflow.
> 
> Very similarly I see Airflow could integrate with YARN and use
> its infrastructure as an "executor" .. have anyone explored feasibility of
> this approach?
> 
> 
> Thanks!
> Ruslan Dautkhanov



( ! ) Warning: include(msgfooter.php): failed to open stream: No such file or directory in /var/www/git/apache-airflow-development/msg03184.html on line 89
Call Stack
#TimeMemoryFunctionLocation
10.0019363128{main}( ).../msg03184.html:0

( ! ) Warning: include(): Failed opening 'msgfooter.php' for inclusion (include_path='.:/var/www/git') in /var/www/git/apache-airflow-development/msg03184.html on line 89
Call Stack
#TimeMemoryFunctionLocation
10.0019363128{main}( ).../msg03184.html:0