Using environment connection URL in operators
I am trying to use: MySqlToHiveTransfer Operator.
From the base_hook code, I see that there is a way to pass connection URL via environment variables. The variable needs to be pre-fixed with: AIRFLOW_CONN_ followed by the connection ID.
I have tried exporting this variable but the behaviour seems to be random. In some instances of the DAG Run, the variable is present in os.environ but in some of the runs it is not. Is this an issue?
We are on version 1.8.2 with Celery Executor.