git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [DISCUSS] Re-think CI strategy?


Also, at this point we're sometimes hitting the 50 minutes time limit on
our slowest Travis-CI matrix job, which means we have to restart it...
making the build even slower.

There's something perhaps suboptimal in the way we build Arrow C++ on
Travis:
- first we build it for no particular Python version
- second we build it for Python 2.7
- third we build it for Python 3.6

Even those C++ files that don't depend on Python get re-compiled thrice
(and ccache doesn't save us, probably because the compile flags are
different).

Regards

Antoine.


Le 06/08/2018 à 19:57, Antoine Pitrou a écrit :
> 
> Not wanting to answer for Wes, but those are two sides of the same coin:
> reducing CI overhead and complexity helps increase developer
> productivity.  Reducing CI overhead is not a goal *in itself* (unless
> there are money issues I don't know about) ;-)
> 
> The productivity cost of being Python 2-compatible is not very high
> *currently* (since much of the cost is a sunk cost by now), but these
> things all add up.  So at some point we should really drop Python 2.
> Whether it's 2019 or 2020, I don't know and I don't get to decide.
> 
> However, anything later than 2020 is excessively conservative IMHO.
> 
> Regards
> 
> Antoine.
> 
> 
> Le 06/08/2018 à 19:46, Robert Nishihara a écrit :
>> Wes, do you primarily want to drop Python 2 to speed up Travis or to reduce
>> the development overhead? In my experience the development overhead is
>> minimal and well worth it. For Travis, we could consider looking into other
>> options like paying for more concurrency.
>>
>> January 2019 is very soon and Python 2 is still massively popular.
>>
>> On Mon, Aug 6, 2018 at 5:11 AM Wes McKinney <wesmckinn@xxxxxxxxx> wrote:
>>
>>>> The 40+ minutes Travis-CI job already uses the toolchain packages AFAIK.
>>>>  Don't they include thrift?
>>>
>>> I was referring to your comment about "parquet-cpp AppVeyor builds are
>>> abysmally slow". I think the slowness is in significant part due to
>>> the ExternalProject builds, where Thrift is the worst offender.
>>>
>>



( ! ) Warning: include(msgfooter.php): failed to open stream: No such file or directory in /var/www/git/apache-arrow-development/msg05348.html on line 126
Call Stack
#TimeMemoryFunctionLocation
10.0013364568{main}( ).../msg05348.html:0

( ! ) Warning: include(): Failed opening 'msgfooter.php' for inclusion (include_path='.:/var/www/git') in /var/www/git/apache-arrow-development/msg05348.html on line 126
Call Stack
#TimeMemoryFunctionLocation
10.0013364568{main}( ).../msg05348.html:0