Nightly tests for Arrow
After the packaging automation work for 0.10 was completed, we have
stalled out a bit on one of the objectives of this framework, which is
to allow contributors to define and add new tasks that can be run on
demand or as part of a nightly job.
So we have some problems to solve:
* How to define a task we wish to validate (like building the API
documentation, or building Arrow with some particular build
parameters) as a new Crossbow task -- document this well so that
people have some instructions to follow
* How to add a task to some kind of a nightly build manifest
* Where to schedule and run the nightly jobs
* Reporting nightly build failures to the mailing list
In terms of scalability requirements, this needs to accommodate 50-100 tasks.
This won't be the last time we need to do some infrastructure work to
scale our testing process, but this will help with testing things that
we want to make sure work but without having to increase the size of
our CI matrix.
Thoughts about how to proceed?