Re: A proposal...
On Mon, Apr 23, 2018 at 9:47 AM, Rainer Jung <rainer.jung@xxxxxxxxxxx> wrote:
> Am 23.04.2018 um 16:00 schrieb Jim Jagielski:
>> It seems that, IMO, if there was not so much concern about "regressions"
>> in releases, this whole revisit-versioning debate would not have come up.
Additional concerns that amplify the regressions; last minute code dumps
with minimal review upon each point release. A three day review window
for the success of the combined result. Insufficient community review of
new features (w/wo new directives) with no alpha or beta releases in over
half a decade (h2/md excepted.)
>> It does seem to me that each time we patch something, there should be a
>> test added or extended which covers that bug. We have gotten lax in that.
>> Same for features. And the more substantial the change (ie, the more core
>> code it touches, or the more it refactors something), the more we should
>> envision what tests can be in place which ensure nothing breaks.
>> In other words: nothing backported unless it also involves some changes to
>> the Perl test framework or some pretty convincing reasons why it's not
Or horse-before-the-cart, put in the test for a spec violation/problem behavior
in the code, and add it to TODO. The suite doesn't fail, but serves as a flag
for a defect to be corrected.
Even better (and we have been good about this)... make corresponding docs
changes a prereq, in addition to test.
> I agree with the importance of the test framework, but would also like to
> mention that getting additional test feedback from the community seems also
> important. That's why IMHO the RC style of releasing could be helpful by
> attracting more test effort before a release.
> And for the more complex modules like mod_proxy, mod_ssl and the event MPM,
> some of the hickups might have been hard to detect with the test framework.
> That's why I think having a more stable branch 2.4 with less feature
> backports and another branch that evolves faster would give downstreams a
+1; I see any "patch" releases (semver definition) as adopting well-tested bug
fixes. In some cases, complex patches could arrive first on a new minor branch
for longer alpha/beta scrutiny, before being accepted as-a-patch. This
helped our php-fpm users with that crazy 2.4.2# cycle of tweak-and-break.
I'd hope we would reintroduce alpha/beta review of new features coinciding
with release m.n.0 with a much longer tail for feature review. Maybe it requires
two or three patch releases before GA, maybe it is accepted as GA on the
very first candidate.
A patch release can be reviewed in a week, but needs to be reviewed in days
to move a security defect fix into users' hands after it is revealed
to our svn/git.
On very rare occasions (once a decade or so), we accelerate this to 24 hours.
A feature release/significant behavior change needs a community, and this is
not a review that happens in a week. I'd expect better adoption of new features
by drawing in our users@ and extended communities to help review additions.