git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Solved: suppress PendingDeprecationWarning messages in airflow logs


Hello,

I’ve been looking for a way to suppress the PendingDeprecationWarning messages cluttering our airflow logs and I have a working solution which I thought I would share.

In order to do this, you first need to configure airflow for custom logging using steps 1-4 here:

https://airflow.readthedocs.io/en/stable/howto/write-logs.html#writing-logs-to-azure-blob-storage

(note that although the document is for Azure remote logging you don’t actually need azure for this)

Next, modify the log_config.py script created in the step above as follows:


  1.  Import logging
  2.  Define the filter class:



class DeprecationWarningFilter(logging.Filter):

    def filter(self, record):

        allow = 'DeprecationWarning' not in record.msg

        return allow


  1.  Add a “filters” section to the LOGGING_CONFIG beneath “formatters:



'filters': {

        'noDepWarn': {

            '()': DeprecationWarningFilter,

        }

    },


  1.  For each of the handlers where you want to suppress the warnings (console, task, processor, or any of the remote log handlers you may be using) add the following line to its configuration:



'filters': ['noDepWarn'],

Restart airflow and your logs should be clean.


Sean Carey