[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [DISCUSS] Re-think CI strategy?

Also, at this point we're sometimes hitting the 50 minutes time limit on
our slowest Travis-CI matrix job, which means we have to restart it...
making the build even slower.

There's something perhaps suboptimal in the way we build Arrow C++ on
- first we build it for no particular Python version
- second we build it for Python 2.7
- third we build it for Python 3.6

Even those C++ files that don't depend on Python get re-compiled thrice
(and ccache doesn't save us, probably because the compile flags are



Le 06/08/2018 à 19:57, Antoine Pitrou a écrit :
> Not wanting to answer for Wes, but those are two sides of the same coin:
> reducing CI overhead and complexity helps increase developer
> productivity.  Reducing CI overhead is not a goal *in itself* (unless
> there are money issues I don't know about) ;-)
> The productivity cost of being Python 2-compatible is not very high
> *currently* (since much of the cost is a sunk cost by now), but these
> things all add up.  So at some point we should really drop Python 2.
> Whether it's 2019 or 2020, I don't know and I don't get to decide.
> However, anything later than 2020 is excessively conservative IMHO.
> Regards
> Antoine.
> Le 06/08/2018 à 19:46, Robert Nishihara a écrit :
>> Wes, do you primarily want to drop Python 2 to speed up Travis or to reduce
>> the development overhead? In my experience the development overhead is
>> minimal and well worth it. For Travis, we could consider looking into other
>> options like paying for more concurrency.
>> January 2019 is very soon and Python 2 is still massively popular.
>> On Mon, Aug 6, 2018 at 5:11 AM Wes McKinney <wesmckinn@xxxxxxxxx> wrote:
>>>> The 40+ minutes Travis-CI job already uses the toolchain packages AFAIK.
>>>>  Don't they include thrift?
>>> I was referring to your comment about "parquet-cpp AppVeyor builds are
>>> abysmally slow". I think the slowness is in significant part due to
>>> the ExternalProject builds, where Thrift is the worst offender.