[DISCUSS] Re-think CI strategy?
Our CI jobs are taking longer and longer. The main reason seem not to
be that our test suites become more thorough (running tests actually
seems to account for a very minor fraction of CI times) but the combined
fact that 1) fetching dependencies and building is slow 2) we have many
configurations tested on our CI jobs.
The slowest job on our Travis-CI configuration routinely takes more 40
minutes (this is for a single job, where everything is essentially
serialized). As for AppVeyor, the different jobs in a build there are
mostly serialized, which balloons the total build time.
If we ever move parquet-cpp into the Arrow repo, this will probably
become worse again (though it might be better for parquet-cpp itself,
which doesn't seem to have received a lot of care on the CI side: for
instance, parquet-cpp AppVeyor builds are abysmally slow).
I think we will have to cut down on the number of things we exercise on
the per-build CI builds. This can be done along two axes:
1) remove some build steps that we deem non-critical or even unimportant
2) remove some build configurations entirely (for instance I don't
understand why we need a "static CRT" build on Windows, or worse, why we
need to have NMake-based builds at all)