git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [IE] Re: [External] RE: [IE] airflow ui not showing logs


Ok, I finally had time to try and track down the issue. Looks like the cfg
option 'task_log_reader' was changed at some point prior to 1.10.0.
Previously, my airflow.cfg had:

task_log_reader = file.task

But now it needs to be set to

task_log_reader = task


On Mon, Oct 15, 2018 at 1:57 PM Frank Maritato <fmaritato@xxxxxxxxxxxxx>
wrote:

> James - No, my configuration doesn't have a setting for rbac, so I'm
> assuming the default is False.
>
> Sunil - We are currently running the LocalExecutor. Is the worker process
> just for Celery? According to the quickstart, I thought the only processes
> we need to start are the webserver and scheduler.
>
>
> On Mon, Oct 15, 2018 at 12:32 PM Sunil Varma Chiluvuri <
> SunilVarma.Chiluvuri@xxxxxxxxxxx> wrote:
>
>> Frank,
>>
>> The serve_logs process usually starts automatically with the 'airflow
>> worker' command. Try restarting your worker process and see if it also
>> starts the serve_logs process.
>>
>> I've had a situation where the serve_logs process went down for some
>> reason and restarting the worker fixed it.  It doesn't explain the cause
>> but could possibly resolve your issue.
>>
>> Sunil
>>
>> -----Original Message-----
>> From: Frank Maritato [mailto:fmaritato@xxxxxxxxxxxxx.INVALID]
>> Sent: Monday, October 15, 2018 2:14 PM
>> To: dev@xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>> Subject: [IE] Re: [External] RE: [IE] airflow ui not showing logs
>>
>> Hi Sunil,
>>
>> I don't see this process running. I have never had to run this command
>> previously. Should it have started as part of the 'airflow webserver'
>> command?
>> I ran it manually from the command line as 'airflow serve_logs &' (does
>> not
>> have a daemon option I guess) but I am still not seeing logs in the ui. I
>> did verify with netstat that it is running on port 8793.
>>
>>
>> On Mon, Oct 15, 2018 at 12:02 PM Sunil Varma Chiluvuri <
>> SunilVarma.Chiluvuri@xxxxxxxxxxx> wrote:
>>
>> > Check if the serve_logs process is running alongside your worker
>> > process(es). This is the process that takes the log files written to
>> disk
>> > and serves them to the web UI. It should be running on port 8793 by
>> default.
>> >
>> > Sunil
>> >
>> > -----Original Message-----
>> > From: Frank Maritato [mailto:fmaritato@xxxxxxxxxxxxx.INVALID]
>> > Sent: Monday, October 15, 2018 1:42 PM
>> > To: dev@xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>> > Subject: [IE] airflow ui not showing logs
>> >
>> > Hi All,
>> >
>> > I'm running airflow 1.10.0 and the ui isn't showing the task logs
>> anymore.
>> > This has worked in the past so I'm not sure what changed. I was able to
>> > verify that the logs are definitely being written to the same local
>> > directory as what is specified in the airflow.cfg 'base_log_folder'. I
>> > don't see any errors or debug in the airflow-webserver.{out|err} logs. I
>> > tried restarting the process and it still doesn't work.
>> >
>> > Anyone know how I can track this down?
>> > --
>> > Frank Maritato
>> > This message contains proprietary information from Equifax which may be
>> > confidential. If you are not an intended recipient, please refrain from
>> any
>> > disclosure, copying, distribution or use of this information and note
>> that
>> > such actions are prohibited. If you have received this transmission in
>> > error, please notify by e-mail postmaster@xxxxxxxxxxx. Equifax® is a
>> > registered trademark of Equifax Inc. All rights reserved.
>> >
>>
>>
>> --
>>
>> Frank Maritato
>> This message contains proprietary information from Equifax which may be
>> confidential. If you are not an intended recipient, please refrain from any
>> disclosure, copying, distribution or use of this information and note that
>> such actions are prohibited. If you have received this transmission in
>> error, please notify by e-mail postmaster@xxxxxxxxxxx. Equifax® is a
>> registered trademark of Equifax Inc. All rights reserved.
>>
>
>
> --
>
> Frank Maritato
>


-- 
Frank Maritato