git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Avatica CI test failures


I think the original proposed approach of using `-DskipDockerCheck` is not possible to implement in maven.

My original implementation is to implement profile activation using a property for the docker-check profile:

      <activation>
        <property>
          <name>!skipDockerCheck</name>
        </property>
      </activation>

Unfortunately, properties work on a global basis, so, as long as the skipDockerCheck property is not present, the profile will be activated, hence all executions of maven will need `-DskipDockerCheck` to avoid the check.

Options I considered, but do not work:
- Can we peg the docker check profile to a goal? Unsupported in Maven: https://stackoverflow.com/questions/15964170/how-to-activate-profile-based-on-goal-being-executed - Can we set a property in the root pom.xml under the "apache-release" profile and read the property in the docker-check profile to activate it? No: https://stackoverflow.com/questions/46927776/maven-child-profile-activated-on-parent-profile-property

Potential options for fixing this:
- The profile was originally named "apache-release", so that it would be activated with -Papache-release (which is also the name of the release profile in the pom.xml in the root of the project). We can rename the profile for the docker check to "docker-check". Then, when making a real release, we use `-Papache-release,docker-check`. When making a dry-run release, we just use `-Papache-release`.

- We use a property, but make it positive. So, when building a real release, we pass in `-DdockerCheck` to run the check.

The downside of these 2 approaches are:
- Disabling the docker check is not an exception, you will need to specifically request it to run the check. - It's possible to build releases with broken dockerfiles if the docker check is neglected.

Please let me know your thoughts.

Francis

On 21/11/2018 9:12 am, Josh Elser wrote:
Thanks. The first red build I saw was after your change had landed. I may be wrong, but I think that's how we got here :)

On 11/20/18 4:37 PM, Francis Chuang wrote:
Oh dear! I was under the impression that the builds were breaking before the fix for CALCITE-2385 was committed, but I was wrong! I'll try to get this fixed today.

On 21/11/2018 3:57 am, Josh Elser wrote:
Wait, back up a second? Francis -- did you break every other build just to try to fix CALCITE-2385?

We should not have had to be making changes to CI just to make this work...

On 11/19/18 4:04 PM, Francis Chuang wrote:
Thanks for organizing that, Michael!

Kevin: Currently, all builds will fail without -DskipDockerCheck in place. I was hoping there is some way to configure Maven to run the docker check when the goal is release:prepare and the -DdryRun argument is not present, however there doesn't seem to be a way to do this with the maven release plugin.

The disadvantage of reversing the logic (so that the docker check is triggered on the specification of the argument) is that it's possible to create a broken release with the wrong version numbers in the Dockerfiles if -DcheckDocker is accidentally omitted. My original thought was to make the ability to skip the check an exception (similar to -DskipTests).

Please let me know your thoughts

Francis

On 20/11/2018 4:27 am, Kevin Risden wrote:
So is it possible to change the logic so this only gets triggered during a release? Seems weird to have to specify -DskipDockerCheck on a regular build. My local jenkins has been failing too and I'm hesitant to make the same change. I would guess builds from the command line will fail too?

Kevin Risden


On Mon, Nov 19, 2018 at 12:07 PM Michael Mior <mmior@xxxxxxxxxx> wrote:

No problem. I also gave you access so you should be able to do this
yourself in the future. Let me know if that doesn't seem to have worked.

--
Michael Mior
mmior@xxxxxxxxxx


Le dim. 18 nov. 2018 à 21:10, Francis Chuang <francischuang@xxxxxxxxxx> a
écrit :

Thanks, Michael!

On 19/11/2018 11:37 am, Michael Mior wrote:
Thanks Francis! I updated the configuration on Jenkins and triggered a
rebuild and looks like things pass now.

--
Michael Mior
mmior@xxxxxxxxxx


Le dim. 18 nov. 2018 à 17:20, Francis Chuang <francischuang@xxxxxxxxxx
a
écrit :

This is the latest test failure for Avatica on Jenkins:


https://builds.apache.org/job/Calcite-Avatica-Master/129/jdk=JDK%201.8%20(latest),label_exp=ubuntu&&!cloud-slave/console
It looks like the failure is due to
check-dockerhub-dockerfile-version.
This check always fails during development or release:prepare
-DdryRun,
because it relies on tags and version numbers that are only available
when we are making a release for real. I committed a fix for
CALCITE-2385 [1] to fix this. I am not able to update the CI
configuration for Jenkins, but can someone update it to include the
-DskipDockerCheck argument to fix it?

For Travis, the cause of the test failures is also the same and I
updated the .travis.yml file to fix it.

[1] https://issues.apache.org/jira/browse/CALCITE-2385