git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Ability to discover custom plugins, operators, sensors, etc. from various locations


Hi,

I was trying to find if its possible to create custom plugins without
dropping them into `$AIRFLOW_HOME/plugins` (or the directory defined in
`airflow.cfg`).
We can define one location in `airflow.cfg` but I have multiple projects
which will have their own workflows so, ideally I would want to implement
custom plugins (along with operators, sensors, etc.) in those repositories
itself and let them get discovered from there.
Checking through documentation and browsing code I concluded that the only
way to get this done is by ensuring that `$PYTHONPATH` is set to include
directories where these custom modules are stored.

So, I want to ask if there is a better way to handle this?
If not, can `plugins_manager` (AirflowPlugin) be improved to suit this
use-case?


Kind Regards,
Ritesh


( ! ) Warning: include(msgfooter.php): failed to open stream: No such file or directory in /var/www/git/apache-airflow-development/msg03487.html on line 88
Call Stack
#TimeMemoryFunctionLocation
10.0001363192{main}( ).../msg03487.html:0

( ! ) Warning: include(): Failed opening 'msgfooter.php' for inclusion (include_path='.:/var/www/git') in /var/www/git/apache-airflow-development/msg03487.html on line 88
Call Stack
#TimeMemoryFunctionLocation
10.0001363192{main}( ).../msg03487.html:0