git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: celery problem, AttributeError: async


Thanks. Please do that.

On 21/05/2018, 14:59, "Craig Rodrigues" <crodr001@xxxxxxxxx> wrote:

    celery 4.1.1 was just released last night which has all the async problems fixed:
    
    https://github.com/celery/celery/commits/v4.1.1
    
    I'll test this out, and then submit a PR to bump airflow's celery version to 4.1.1
    
    --
    Craig
    
    On 2018/05/21 07:20:50, Craig Rodrigues <crodr001@xxxxxxxxx> wrote: 
    > Hi,
    > 
    > I used a requiremens.txt file with these three lines:
    > 
    > git+https://github.com/apache/incubator-airflow@v1-10-test#egg=apache-airflow[celery,crypto,emr,hive,hdfs,ldap,mysql,postgres,redis,slack,s3]
    > celery>=4.2.0rc3
    > kombu>=4.2.0
    > 
    > 
    > I did
    > 
    > pip install -r requirements.txt
    > 
    > When I started my worker, I got:
    > 
    > [2018-05-21 06:39:23,980] {__init__.py:48} INFO - Using executor CeleryExecutor
    > Traceback (most recent call last):
    >   File "/bin/airflow", line 32, in <module>
    >     args.func(args)
    >   File "/usr/lib/python2.7/site-packages/airflow/utils/cli.py", line 74, in wrapper
    >     return f(*args, **kwargs)
    >   File "/usr/lib/python2.7/site-packages/airflow/bin/cli.py", line 959, in worker
    >     worker.run(**options)
    >   File "/usr/lib/python2.7/site-packages/celery/bin/worker.py", line 257, in run
    >     **kwargs)
    >   File "/usr/lib/python2.7/site-packages/celery/worker/worker.py", line 101, in __init__
    >     self.setup_instance(**self.prepare_args(**kwargs))
    >   File "/usr/lib/python2.7/site-packages/celery/worker/worker.py", line 124, in setup_instance
    >     self.should_use_eventloop() if use_eventloop is None
    >   File "/usr/lib/python2.7/site-packages/celery/worker/worker.py", line 243, in should_use_eve
    >     self._conninfo.transport.implements.async and
    >   File "/usr/lib/python2.7/site-packages/kombu/transport/base.py", line 125, in __getattr__
    >     raise AttributeError(key)
    > AttributeError: async
    > 
    > 
    > 
    > I reported this as a bug in celery here:
    > 
    > https://github.com/celery/celery/issues/4747
    > 
    > It turns out that this is a known problem.  It looks like in kombu, they
    > renamed a bunch of uses of "async" to "asynchronous", to not clash with the new
    > 'async' keyword in Python 3.7.
    > 
    > In celery, they are catching up and doing the same rename.  
    > Almost at the same time that I reported this bug, the issue was fixed in the master branch of celery:
    > 
    > https://github.com/celery/celery/commit/c8ef7ad60b72a194654c58beb04a1d65cd0435ad
    > 
    > Hopefully the versions of kombu and celery will stabilize so that it all works properly with airflow.
    > 
    > --
    > Craig
    > 
    > 
    > 
    



Kaxil Naik 

Data Reply
2nd Floor, Nova South
160 Victoria Street, Westminster
London SW1E 5LB - UK 
phone: +44 (0)20 7730 6000
k.naik@xxxxxxxxx
www.reply.com


( ! ) Warning: include(msgfooter.php): failed to open stream: No such file or directory in /var/www/git/apache-airflow-development/msg03408.html on line 152
Call Stack
#TimeMemoryFunctionLocation
10.0009368632{main}( ).../msg03408.html:0

( ! ) Warning: include(): Failed opening 'msgfooter.php' for inclusion (include_path='.:/var/www/git') in /var/www/git/apache-airflow-development/msg03408.html on line 152
Call Stack
#TimeMemoryFunctionLocation
10.0009368632{main}( ).../msg03408.html:0