git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [VOTE] Release 1.6.0, release candidate #4


I hereby close the vote. The result will be announced in a separate thread.

On Thu, Aug 9, 2018 at 11:47 AM Till Rohrmann <trohrmann@xxxxxxxxxx> wrote:

> +1
>
> - Checked checksums and signatures
> - Verified that no unwanted binaries are contained in source release
> - Checked LICENSE and NOTICE file
> - Checked that all newly added dependencies have a compatible license
> - Checked that a local cluster can be started and stopped without
> exceptions in the log
> - Verified that SBT quickstarts are up to date
> - Verified that Java quickstarts work with IntelliJ
> - Verified that all Jepsen tests pass
> - Verified that e2e tests modulo test_sql_client.sh (see
> https://issues.apache.org/jira/browse/FLINK-10107) pass
>
> Cheers,
> Till
>
> On Thu, Aug 9, 2018 at 8:18 AM vino yang <yanghua1127@xxxxxxxxx> wrote:
>
>> +1
>>
>> - checkout 1.6 source code and successfully run `mvn clean package
>> -DskipTests`
>> - searched '1.5' and '1.5.2' in all modules pom file and successfully
>> verified flink version was changed
>> - successfully run table and sql test locally
>>
>> Thanks, vino.
>>
>>
>> Timo Walther <twalthr@xxxxxxxxxx> 于2018年8月9日周四 上午3:24写道:
>>
>> > +1
>> >
>> > - successfully run `mvn clean verify` locally
>> > - successfully run end-to-end tests locally (except for SQL Client
>> > end-to-end test)
>> >
>> > Found a bug in the class loading of SQL JAR files. This is not a blocker
>> > but a bug that we should fix soon. As an easy workaround user should not
>> > use different Kafka versions as SQL Client dependencies.
>> >
>> > Regards,
>> > Timo
>> >
>> > Am 08.08.18 um 18:10 schrieb Dawid Wysakowicz:
>> > > +1
>> > >
>> > > - verified compilation, tests
>> > > - verified checksum and gpg files
>> > > - verified sbt templates (g8, quickstart) - run assemblies on local
>> > cluster
>> > >
>> > > - I could not execute the nightly-tests.sh though. The tests that were
>> > > failing most often are:
>> > >      - test_streaming_file_sink.sh
>> > >      - test_streaming_elasticsearch.sh
>> > >
>> > > Those are connectors though and it might be only tests flakiness so I
>> > > think it should not block the release.
>> > >
>> > > On 08/08/18 16:36, Chesnay Schepler wrote:
>> > >> I did not use the tools/list_deps.py script as I wasn't aware that it
>> > >> existed.
>> > >>
>> > >> Even if I were I wouldn't have used it and in fact would advocate for
>> > >> removing it.
>> > >> It manually parses and constructs dependency information which is
>> > >> utterly unnecessary as maven already provides this functionality,
>> with
>> > >> the added bonus of also accounting for dependencyManagement and
>> > >> transitive dependencies which we obviously have to take into account.
>> > >>
>> > >> I used this one-liner instead:
>> > >> |mvn dependency:list | ||grep| |":.*:.*:.*"| || ||grep| |-||v| |-e
>> > >> ||"Finished at"| |-e ||"Some problems"| || ||cut| |-d] -f2- | ||sed|
>> > >> |'s/:[a-z]*$//g'| || ||sort| |-u
>> > >>
>> > >> |which I have documented here:
>> > >> https://cwiki.apache.org/confluence/display/FLINK/Dependencies
>> > >>
>> > >> On 08.08.2018 15:06, Aljoscha Krettek wrote:
>> > >>> +1
>> > >>>
>> > >>> - verified checksum and gpg files
>> > >>> - verified LICENSE and NOTICE: NOTICE didn't change from 1.5,
>> LICENSE
>> > >>> had one unnecessary part removed
>> > >>>
>> > >>> Side comment: I'm not sure whether the "Verify that the LICENSE and
>> > >>> NOTICE file is correct for the binary and source releases" part is
>> > >>> valid anymore because we only have one LICENSE and NOTICE file. also
>> > >>> "The LICENSE and NOTICE files in flink-dist/src/main/flink-bin refer
>> > >>> to the binary distribution and mention all of Flink's Maven
>> > >>> dependencies as well" can be dropped because we don't have them
>> > anymore.
>> > >>>
>> > >>> I came to the same conclusion on dependencies. I used
>> > >>> tools/list_deps.py and diff'ed the output for 1.5 and 1.6, that's
>> > >>> probably what Chesnay also did ... :-)
>> > >>>
>> > >>>> On 8. Aug 2018, at 14:43, Chesnay Schepler <chesnay@xxxxxxxxxx>
>> > wrote:
>> > >>>>
>> > >>>> +1
>> > >>>>
>> > >>>> - verified source release contains no binaries
>> > >>>> - verified correct versions in source release
>> > >>>> - verified compilation, tests and E2E-tests pass (on travis)
>> > >>>> - verified checksum and gpg files
>> > >>>>
>> > >>>> New dependencies (excluding dependencies where we simply depend on
>> a
>> > >>>> different version now):
>> > >>>>      Apache licensed:
>> > >>>>          io.confluent:common-utils:jar:3.3.1
>> > >>>>          io.confluent:kafka-schema-registry-client:jar:3.3.1
>> > >>>>          io.prometheus:simpleclient_pushgateway:jar:0.3.0
>> > >>>>          various Apache Nifi dependencies
>> > >>>>          various Apache Parquet dependencies
>> > >>>>          various ElasticSearch dependencies
>> > >>>>      CDDL:
>> > >>>>          javax.ws.rs:javax.ws.rs-api:jar:2.1
>> > >>>>      Bouncycastle (MIT-like):
>> > >>>>          org.bouncycastle:bcpkix-jdk15on:jar:1.59
>> > >>>>          org.bouncycastle:bcprov-jdk15on:jar:1.59
>> > >>>>      MIT:
>> > >>>>          org.projectlombok:lombok:jar:1.16.20
>> > >>>>
>> > >>>> On 08.08.2018 13:28, Till Rohrmann wrote:
>> > >>>>> Thanks for reporting these problems Chesnay. The usage string in
>> > >>>>> `standalone-job.sh` is out dated and should be updated. The same
>> > >>>>> applies to
>> > >>>>> the typo.
>> > >>>>>
>> > >>>>> When calling `standalone-job.sh start --job-classname foobar.Job`
>> > >>>>> please
>> > >>>>> make sure that the user code jar is contained in the classpath
>> (e.g.
>> > >>>>> putting the jar in the lib directory). Documenting this behaviour
>> > >>>>> is part
>> > >>>>> of the pending issue FLINK-10001.
>> > >>>>>
>> > >>>>> We should fix all of these issues. They are, however, no release
>> > >>>>> blockers.
>> > >>>>>
>> > >>>>> Cheers,
>> > >>>>> Till
>> > >>>>>
>> > >>>>> On Wed, Aug 8, 2018 at 11:31 AM Chesnay Schepler
>> > >>>>> <chesnay@xxxxxxxxxx> wrote:
>> > >>>>>
>> > >>>>>> I found some issues with the standalone-job.sh script.
>> > >>>>>>
>> > >>>>>> I ran "./bin/standalone-job.sh start" as described by the usage
>> > >>>>>> string.
>> > >>>>>>
>> > >>>>>>       2018-08-08 09:22:34,385 ERROR
>> > >>>>>>       org.apache.flink.runtime.entrypoint.ClusterEntrypoint
>> > -
>> > >>>>>>       Could not parse command line arguments [--configDir,
>> > >>>>>>       /home/zento/svn/flink-1.6.0/flink-1.6.0/conf].
>> > >>>>>>       org.apache.flink.runtime.entrypoint.FlinkParseException:
>> > >>>>>> Failed to
>> > >>>>>>       parse the command line arguments.
>> > >>>>>>                at
>> > >>>>>>
>> > >>>>>>
>> >
>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
>> > >>>>>>
>> > >>>>>>                at
>> > >>>>>>
>> > >>>>>>
>> >
>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
>> > >>>>>>
>> > >>>>>>       Caused by: org.apache.commons.cli.MissingOptionException:
>> > >>>>>> Missing
>> > >>>>>>       required option: j
>> > >>>>>>                at
>> > >>>>>>
>> > >>>>>>
>> >
>> org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199)
>> > >>>>>>
>> > >>>>>>                at
>> > >>>>>>
>> > >>>>>>
>> org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130)
>> > >>>>>>                at
>> > >>>>>>
>> > >>>>>> org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
>> > >>>>>>                at
>> > >>>>>>
>> > >>>>>>
>> >
>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
>> > >>>>>>
>> > >>>>>>               ... 1 more
>> > >>>>>>
>> > >>>>>> The script should fail earlier if no jar is provided, with a
>> better
>> > >>>>>> error message.
>> > >>>>>> It is also undocumented, and the usage instructions don't appear
>> > >>>>>> correct.
>> > >>>>>>
>> > >>>>>> Passing a jar with the -j option leads to a
>> ClassNotFoundException:
>> > >>>>>> "./bin/standalone-job.sh start -j
>> examples/streaming/WordCount.jar"
>> > >>>>>>
>> > >>>>>>       2018-08-08 09:26:30,562 ERROR
>> > >>>>>>       org.apache.flink.runtime.entrypoint.ClusterEntrypoint
>> > -
>> > >>>>>>       Cluster initialization failed.
>> > >>>>>>       java.lang.reflect.UndeclaredThrowableException
>> > >>>>>>                at
>> > >>>>>>
>> > >>>>>>
>> >
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1854)
>> > >>>>>>
>> > >>>>>>                at
>> > >>>>>>
>> > >>>>>>
>> >
>> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
>> > >>>>>>
>> > >>>>>>                at
>> > >>>>>>
>> > >>>>>>
>> >
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:189)
>> > >>>>>>
>> > >>>>>>                at
>> > >>>>>>
>> > >>>>>>
>> >
>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:158)
>> > >>>>>>
>> > >>>>>>       Caused by: org.apache.flink.util.FlinkException: Could not
>> > >>>>>> load the
>> > >>>>>>       provied entrypoint class.
>> > >>>>>>                at
>> > >>>>>>
>> > >>>>>>
>> >
>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.createPackagedProgram(StandaloneJobClusterEntryPoint.java:92)
>> > >>>>>>
>> > >>>>>>                at
>> > >>>>>>
>> > >>>>>>
>> >
>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.retrieveJobGraph(StandaloneJobClusterEntryPoint.java:75)
>> > >>>>>>
>> > >>>>>>                at
>> > >>>>>>
>> > >>>>>>
>> >
>> org.apache.flink.runtime.entrypoint.JobClusterEntrypoint.createDispatcher(JobClusterEntrypoint.java:107)
>> > >>>>>>
>> > >>>>>>                at
>> > >>>>>>
>> > >>>>>>
>> >
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startClusterComponents(ClusterEntrypoint.java:353)
>> > >>>>>>
>> > >>>>>>                at
>> > >>>>>>
>> > >>>>>>
>> >
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:232)
>> > >>>>>>
>> > >>>>>>                at
>> > >>>>>>
>> > >>>>>>
>> >
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:190)
>> > >>>>>>
>> > >>>>>>                at
>> > >>>>>> java.security.AccessController.doPrivileged(Native Method)
>> > >>>>>>                at
>> javax.security.auth.Subject.doAs(Subject.java:422)
>> > >>>>>>                at
>> > >>>>>>
>> > >>>>>>
>> >
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
>> > >>>>>>
>> > >>>>>>                ... 3 more
>> > >>>>>>       Caused by: java.lang.ClassNotFoundException:
>> > >>>>>>       examples/streaming/WordCount.jar
>> > >>>>>>                at
>> > >>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>> > >>>>>>                at
>> > >>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>> > >>>>>>                at
>> > >>>>>>
>>  sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
>> > >>>>>>                at
>> > >>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>> > >>>>>>                at
>> > >>>>>>
>> > >>>>>>
>> >
>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.createPackagedProgram(StandaloneJobClusterEntryPoint.java:89)
>> > >>>>>>
>> > >>>>>>                ... 11 more
>> > >>>>>>
>> > >>>>>> So this seems to not work at all, but maybe I'm using it wrong?
>> > >>>>>>
>> > >>>>>> (There's also typo in "Could not load the provied entrypoint
>> class")
>> > >>>>>>
>> > >>>>>> On 08.08.2018 10:33, Piotr Nowojski wrote:
>> > >>>>>>> +1 from my side
>> > >>>>>>>
>> > >>>>>>> I’ve spent some time playing around with various examples
>> > (batching,
>> > >>>>>> streaming and SQL) on EMR 6 nodes cluster with yarn deployment,
>> with
>> > >>>>>> different configuration options (number of task
>> > >>>>>> managers/memory/Flip6/credit base flow control/metrics) and
>> > >>>>>> everything
>> > >>>>>> looks now fine (after fixing
>> > >>>>>> https://issues.apache.org/jira/browse/FLINK-9969 <
>> > >>>>>> https://issues.apache.org/jira/browse/FLINK-9969> ).
>> > >>>>>>> Piotrek
>> > >>>>>>>
>> > >>>>>>>> On 7 Aug 2018, at 17:17, Till Rohrmann <trohrmann@xxxxxxxxxx>
>> > >>>>>>>> wrote:
>> > >>>>>>>>
>> > >>>>>>>> Hi everyone,
>> > >>>>>>>> Please review and vote on the release candidate #4 for the
>> version
>> > >>>>>> 1.6.0,
>> > >>>>>>>> as follows:
>> > >>>>>>>> [ ] +1, Approve the release
>> > >>>>>>>> [ ] -1, Do not approve the release (please provide specific
>> > >>>>>>>> comments)
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>> The complete staging area is available for your review, which
>> > >>>>>>>> includes:
>> > >>>>>>>> * JIRA release notes [1],
>> > >>>>>>>> * the official Apache source release and binary convenience
>> > >>>>>>>> releases to
>> > >>>>>> be
>> > >>>>>>>> deployed to dist.apache.org [2], which are signed with the key
>> > with
>> > >>>>>>>> fingerprint 1F302569A96CFFD5 [3],
>> > >>>>>>>> * all artifacts to be deployed to the Maven Central Repository
>> > [4],
>> > >>>>>>>> * source code tag "release-1.6.0-rc4" [5],
>> > >>>>>>>> * website pull request listing the new release and adding
>> > >>>>>>>> announcement
>> > >>>>>> blog
>> > >>>>>>>> post [6].
>> > >>>>>>>>
>> > >>>>>>>> Please use this document for coordinating testing efforts: [7]
>> > >>>>>>>>
>> > >>>>>>>> The vote will be shortened since we only adde a minor fix on
>> top
>> > >>>>>>>> of the
>> > >>>>>> RC
>> > >>>>>>>> 3. It will close on Wednesday 6:30pm CET. It is adopted by
>> > majority
>> > >>>>>>>> approval, with at least 3 PMC affirmative votes.
>> > >>>>>>>>
>> > >>>>>>>> Thanks,
>> > >>>>>>>> Your friendly Release Manager
>> > >>>>>>>>
>> > >>>>>>>> [1]
>> > >>>>>>>>
>> > >>>>>>
>> >
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12342760
>> > >>>>>>
>> > >>>>>>>> [2] https://dist.apache.org/repos/dist/dev/flink/flink-1.6.0/
>> > >>>>>>>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
>> > >>>>>>>> [4]
>> > >>>>>>
>> > https://repository.apache.org/content/repositories/orgapacheflink-1178
>> > >>>>>>
>> > >>>>>>>> [5] https://github.com/apache/flink/tree/release-1.6.0-rc4
>> > >>>>>>>> [6] https://github.com/apache/flink-web/pull/117
>> > >>>>>>>> [7]
>> > >>>>>>>>
>> > >>>>>>
>> >
>> https://docs.google.com/document/d/1upBFZQ7tbaSkYvDiLqfUFXKg8Xxs-lVheEfb66e4jpo/edit?usp=sharing
>> > >>>>>>
>> > >>>>>>>> Pro-tip: you can create a settings.xml file with these
>> contents:
>> > >>>>>>>>
>> > >>>>>>>> <settings>
>> > >>>>>>>> <activeProfiles>
>> > >>>>>>>>     <activeProfile>flink-1.6.0</activeProfile>
>> > >>>>>>>> </activeProfiles>
>> > >>>>>>>> <profiles>
>> > >>>>>>>>     <profile>
>> > >>>>>>>>       <id>flink-1.6.0</id>
>> > >>>>>>>>       <repositories>
>> > >>>>>>>>         <repository>
>> > >>>>>>>>           <id>flink-1.6.0</id>
>> > >>>>>>>>           <url>
>> > >>>>>>>>
>> > >>>>>>>>
>> > https://repository.apache.org/content/repositories/orgapacheflink-1178/
>> > >>>>>>>>
>> > >>>>>>>>           </url>
>> > >>>>>>>>         </repository>
>> > >>>>>>>>         <repository>
>> > >>>>>>>>           <id>archetype</id>
>> > >>>>>>>>           <url>
>> > >>>>>>>>
>> > >>>>>>>>
>> > https://repository.apache.org/content/repositories/orgapacheflink-1178/
>> > >>>>>>>>
>> > >>>>>>>>           </url>
>> > >>>>>>>>         </repository>
>> > >>>>>>>>       </repositories>
>> > >>>>>>>>     </profile>
>> > >>>>>>>> </profiles>
>> > >>>>>>>> </settings>
>> > >>>>>>>>
>> > >>>>>>>> And reference that in you maven commands via --settings
>> > >>>>>>>> path/to/settings.xml. This is useful for creating a quickstart
>> > >>>>>>>> based on
>> > >>>>>> the
>> > >>>>>>>> staged release and for building against the staged jars.
>> > >>
>> >
>> >
>>
>