git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: KubernetesPodOperator: Invalid arguments were passed to BaseOperator


Oh, OK, I just saw this in example_kubernetes_operator.py:

try:
    # Kubernetes is optional, so not available in vanilla Airflow
    # pip install airflow[gcp]
    from airflow.contrib.operators.kubernetes_pod_operator import
KubernetesPodOperator
except ImportError:
    # Just import the BaseOperator as the KubernetesPodOperator
    logging.warn("Could not import KubernetesPodOperator")
    from airflow.models import BaseOperator as KubernetesPodOperator

That code is just bad and confusing for an example!!

--
Craig

On Wed, May 30, 2018 at 12:39 PM Chris Palmer <chris@xxxxxxxxxxxx> wrote:

> In the example
> <
> https://github.com/apache/incubator-airflow/blob/master/airflow/example_dags/example_kubernetes_operator.py#L24
> >
> it imports BaseOperator as KubernetesPodOperator, when the kubernetes
> modules can't be found.
>
> On Wed, May 30, 2018 at 3:34 PM, Craig Rodrigues <rodrigc@xxxxxxxxxxxxxx>
> wrote:
>
> > For this particular DeprecationWarning, this problem is not caused
> > by dependencies on kubernetes stuff.
> >
> > On this line:
> > https://github.com/apache/incubator-airflow/blob/master/
> > airflow/contrib/operators/kubernetes_pod_operator.py#L137
> >
> > The __init__() method for KubernetesPodOperator is calling the __init__()
> > method
> > for BaseOperator, and passing in values for kwargs that BaseOperator
> > doesn't accept:
> >
> > name': 'airflow-test-pod',
> > 'image': 'ubuntu:16.04',
> > 'labels': {'foo': 'bar'},
> > 'namespace': 'default',
> > 'cmds': ['bash', '-cx'],
> > 'arguments': ['echo', '10'],
> > 'in_cluster': False,
> > 'get_logs': True
> >
> > I don't understand how these things are being passed via kwargs?
> >
> > --
> > Craig
> >
> >
> > On Wed, May 30, 2018 at 1:40 AM Driesprong, Fokko <fokko@xxxxxxxxxxxxxx>
> > wrote:
> >
> > > Hi Craig,
> > >
> > > This is something that needs to be fixed. I agree with you this is very
> > > dirty. In your installation you're not installing the kubernetes stuff,
> > so
> > > the KubernetesPodOperator is ignored. We need to figure out how to have
> > > example dags that are not compatible with the vanilla installation, or
> we
> > > need to remove the kubernetes example for now, and move it to the
> > > documentation.
> > >
> > > Cheers, Fokko
> > >
> > >
> >
>


-- 
Craig Rodrigues
rodrigc@xxxxxxxxxxxxx


( ! ) Warning: include(msgfooter.php): failed to open stream: No such file or directory in /var/www/git/apache-airflow-development/msg03521.html on line 148
Call Stack
#TimeMemoryFunctionLocation
10.0006368792{main}( ).../msg03521.html:0

( ! ) Warning: include(): Failed opening 'msgfooter.php' for inclusion (include_path='.:/var/www/git') in /var/www/git/apache-airflow-development/msg03521.html on line 148
Call Stack
#TimeMemoryFunctionLocation
10.0006368792{main}( ).../msg03521.html:0