git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Tail log instead of serving all logs for long running tasks/huge log volume


I wanted to work on it. But confirming did someone tried it already and had
some practical difficulties/limitations on not going ahead with and all.

On Tue 25 Sep, 2018, 16:13 Sumit Maheshwari, <sumeet.manit@xxxxxxxxx> wrote:

> Yeah, that would be the next step. Maybe you can build that on top of that
> PR and raise a new PR?
>
> On Tue, Sep 25, 2018 at 3:41 PM Sai Phanindhra <phani8996@xxxxxxxxx>
> wrote:
>
> > Hi sumit,
> > Along with a refresh log button if we can tail last N lines it will be
> more
> > useful. We can add filter on view to show N lines but lets implement it
> on
> > worker end to fetch last N lines.
> >
> > On Tue 25 Sep, 2018, 15:37 Sumit Maheshwari, <sumeet.manit@xxxxxxxxx>
> > wrote:
> >
> > > Tried to solve that problem (partially) with below PR
> > >
> > > https://github.com/apache/incubator-airflow/pull/1875
> > >
> > > On Tue, Sep 25, 2018 at 3:18 PM Sai Phanindhra <phani8996@xxxxxxxxx>
> > > wrote:
> > >
> > > > Hi guys,
> > > >    Now we serve whole log file irrespective of its size. Sometimes
> > when i
> > > > log size if few Mb(more than some 5mb or so) we end up rendering
> whole
> > > 5mb
> > > > file. This sometimes lead to huge payloads moving across network and
> > > after
> > > > rendering user have to scroll down thousands and thousands of lines.
> > > > Instead if we just tail last few hundreds of lines we can address
> both
> > of
> > > > the above mentioned issues.
> > > >   Can we have real time logging instead of reloading page again and
> > again
> > > > for logs.
> > > >
> > > > --
> > > > Sai Phanindhra,
> > > > Ph: +91 9043258999
> > > >
> > >
> >
>