git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [VOTE] Release 1.6.0, release candidate #4


+1

- checkout 1.6 source code and successfully run `mvn clean package
-DskipTests`
- searched '1.5' and '1.5.2' in all modules pom file and successfully
verified flink version was changed
- successfully run table and sql test locally

Thanks, vino.


Timo Walther <twalthr@xxxxxxxxxx> 于2018年8月9日周四 上午3:24写道:

> +1
>
> - successfully run `mvn clean verify` locally
> - successfully run end-to-end tests locally (except for SQL Client
> end-to-end test)
>
> Found a bug in the class loading of SQL JAR files. This is not a blocker
> but a bug that we should fix soon. As an easy workaround user should not
> use different Kafka versions as SQL Client dependencies.
>
> Regards,
> Timo
>
> Am 08.08.18 um 18:10 schrieb Dawid Wysakowicz:
> > +1
> >
> > - verified compilation, tests
> > - verified checksum and gpg files
> > - verified sbt templates (g8, quickstart) - run assemblies on local
> cluster
> >
> > - I could not execute the nightly-tests.sh though. The tests that were
> > failing most often are:
> >      - test_streaming_file_sink.sh
> >      - test_streaming_elasticsearch.sh
> >
> > Those are connectors though and it might be only tests flakiness so I
> > think it should not block the release.
> >
> > On 08/08/18 16:36, Chesnay Schepler wrote:
> >> I did not use the tools/list_deps.py script as I wasn't aware that it
> >> existed.
> >>
> >> Even if I were I wouldn't have used it and in fact would advocate for
> >> removing it.
> >> It manually parses and constructs dependency information which is
> >> utterly unnecessary as maven already provides this functionality, with
> >> the added bonus of also accounting for dependencyManagement and
> >> transitive dependencies which we obviously have to take into account.
> >>
> >> I used this one-liner instead:
> >> |mvn dependency:list | ||grep| |":.*:.*:.*"| || ||grep| |-||v| |-e
> >> ||"Finished at"| |-e ||"Some problems"| || ||cut| |-d] -f2- | ||sed|
> >> |'s/:[a-z]*$//g'| || ||sort| |-u
> >>
> >> |which I have documented here:
> >> https://cwiki.apache.org/confluence/display/FLINK/Dependencies
> >>
> >> On 08.08.2018 15:06, Aljoscha Krettek wrote:
> >>> +1
> >>>
> >>> - verified checksum and gpg files
> >>> - verified LICENSE and NOTICE: NOTICE didn't change from 1.5, LICENSE
> >>> had one unnecessary part removed
> >>>
> >>> Side comment: I'm not sure whether the "Verify that the LICENSE and
> >>> NOTICE file is correct for the binary and source releases" part is
> >>> valid anymore because we only have one LICENSE and NOTICE file. also
> >>> "The LICENSE and NOTICE files in flink-dist/src/main/flink-bin refer
> >>> to the binary distribution and mention all of Flink's Maven
> >>> dependencies as well" can be dropped because we don't have them
> anymore.
> >>>
> >>> I came to the same conclusion on dependencies. I used
> >>> tools/list_deps.py and diff'ed the output for 1.5 and 1.6, that's
> >>> probably what Chesnay also did ... :-)
> >>>
> >>>> On 8. Aug 2018, at 14:43, Chesnay Schepler <chesnay@xxxxxxxxxx>
> wrote:
> >>>>
> >>>> +1
> >>>>
> >>>> - verified source release contains no binaries
> >>>> - verified correct versions in source release
> >>>> - verified compilation, tests and E2E-tests pass (on travis)
> >>>> - verified checksum and gpg files
> >>>>
> >>>> New dependencies (excluding dependencies where we simply depend on a
> >>>> different version now):
> >>>>      Apache licensed:
> >>>>          io.confluent:common-utils:jar:3.3.1
> >>>>          io.confluent:kafka-schema-registry-client:jar:3.3.1
> >>>>          io.prometheus:simpleclient_pushgateway:jar:0.3.0
> >>>>          various Apache Nifi dependencies
> >>>>          various Apache Parquet dependencies
> >>>>          various ElasticSearch dependencies
> >>>>      CDDL:
> >>>>          javax.ws.rs:javax.ws.rs-api:jar:2.1
> >>>>      Bouncycastle (MIT-like):
> >>>>          org.bouncycastle:bcpkix-jdk15on:jar:1.59
> >>>>          org.bouncycastle:bcprov-jdk15on:jar:1.59
> >>>>      MIT:
> >>>>          org.projectlombok:lombok:jar:1.16.20
> >>>>
> >>>> On 08.08.2018 13:28, Till Rohrmann wrote:
> >>>>> Thanks for reporting these problems Chesnay. The usage string in
> >>>>> `standalone-job.sh` is out dated and should be updated. The same
> >>>>> applies to
> >>>>> the typo.
> >>>>>
> >>>>> When calling `standalone-job.sh start --job-classname foobar.Job`
> >>>>> please
> >>>>> make sure that the user code jar is contained in the classpath (e.g.
> >>>>> putting the jar in the lib directory). Documenting this behaviour
> >>>>> is part
> >>>>> of the pending issue FLINK-10001.
> >>>>>
> >>>>> We should fix all of these issues. They are, however, no release
> >>>>> blockers.
> >>>>>
> >>>>> Cheers,
> >>>>> Till
> >>>>>
> >>>>> On Wed, Aug 8, 2018 at 11:31 AM Chesnay Schepler
> >>>>> <chesnay@xxxxxxxxxx> wrote:
> >>>>>
> >>>>>> I found some issues with the standalone-job.sh script.
> >>>>>>
> >>>>>> I ran "./bin/standalone-job.sh start" as described by the usage
> >>>>>> string.
> >>>>>>
> >>>>>>       2018-08-08 09:22:34,385 ERROR
> >>>>>>       org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> -
> >>>>>>       Could not parse command line arguments [--configDir,
> >>>>>>       /home/zento/svn/flink-1.6.0/flink-1.6.0/conf].
> >>>>>>       org.apache.flink.runtime.entrypoint.FlinkParseException:
> >>>>>> Failed to
> >>>>>>       parse the command line arguments.
> >>>>>>                at
> >>>>>>
> >>>>>>
> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
> >>>>>>
> >>>>>>                at
> >>>>>>
> >>>>>>
> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
> >>>>>>
> >>>>>>       Caused by: org.apache.commons.cli.MissingOptionException:
> >>>>>> Missing
> >>>>>>       required option: j
> >>>>>>                at
> >>>>>>
> >>>>>>
> org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199)
> >>>>>>
> >>>>>>                at
> >>>>>>
> >>>>>> org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130)
> >>>>>>                at
> >>>>>>
> >>>>>> org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
> >>>>>>                at
> >>>>>>
> >>>>>>
> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
> >>>>>>
> >>>>>>               ... 1 more
> >>>>>>
> >>>>>> The script should fail earlier if no jar is provided, with a better
> >>>>>> error message.
> >>>>>> It is also undocumented, and the usage instructions don't appear
> >>>>>> correct.
> >>>>>>
> >>>>>> Passing a jar with the -j option leads to a ClassNotFoundException:
> >>>>>> "./bin/standalone-job.sh start -j examples/streaming/WordCount.jar"
> >>>>>>
> >>>>>>       2018-08-08 09:26:30,562 ERROR
> >>>>>>       org.apache.flink.runtime.entrypoint.ClusterEntrypoint
> -
> >>>>>>       Cluster initialization failed.
> >>>>>>       java.lang.reflect.UndeclaredThrowableException
> >>>>>>                at
> >>>>>>
> >>>>>>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1854)
> >>>>>>
> >>>>>>                at
> >>>>>>
> >>>>>>
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> >>>>>>
> >>>>>>                at
> >>>>>>
> >>>>>>
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:189)
> >>>>>>
> >>>>>>                at
> >>>>>>
> >>>>>>
> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:158)
> >>>>>>
> >>>>>>       Caused by: org.apache.flink.util.FlinkException: Could not
> >>>>>> load the
> >>>>>>       provied entrypoint class.
> >>>>>>                at
> >>>>>>
> >>>>>>
> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.createPackagedProgram(StandaloneJobClusterEntryPoint.java:92)
> >>>>>>
> >>>>>>                at
> >>>>>>
> >>>>>>
> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.retrieveJobGraph(StandaloneJobClusterEntryPoint.java:75)
> >>>>>>
> >>>>>>                at
> >>>>>>
> >>>>>>
> org.apache.flink.runtime.entrypoint.JobClusterEntrypoint.createDispatcher(JobClusterEntrypoint.java:107)
> >>>>>>
> >>>>>>                at
> >>>>>>
> >>>>>>
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startClusterComponents(ClusterEntrypoint.java:353)
> >>>>>>
> >>>>>>                at
> >>>>>>
> >>>>>>
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:232)
> >>>>>>
> >>>>>>                at
> >>>>>>
> >>>>>>
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:190)
> >>>>>>
> >>>>>>                at
> >>>>>> java.security.AccessController.doPrivileged(Native Method)
> >>>>>>                at javax.security.auth.Subject.doAs(Subject.java:422)
> >>>>>>                at
> >>>>>>
> >>>>>>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
> >>>>>>
> >>>>>>                ... 3 more
> >>>>>>       Caused by: java.lang.ClassNotFoundException:
> >>>>>>       examples/streaming/WordCount.jar
> >>>>>>                at
> >>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> >>>>>>                at
> >>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> >>>>>>                at
> >>>>>>       sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
> >>>>>>                at
> >>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> >>>>>>                at
> >>>>>>
> >>>>>>
> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.createPackagedProgram(StandaloneJobClusterEntryPoint.java:89)
> >>>>>>
> >>>>>>                ... 11 more
> >>>>>>
> >>>>>> So this seems to not work at all, but maybe I'm using it wrong?
> >>>>>>
> >>>>>> (There's also typo in "Could not load the provied entrypoint class")
> >>>>>>
> >>>>>> On 08.08.2018 10:33, Piotr Nowojski wrote:
> >>>>>>> +1 from my side
> >>>>>>>
> >>>>>>> I’ve spent some time playing around with various examples
> (batching,
> >>>>>> streaming and SQL) on EMR 6 nodes cluster with yarn deployment, with
> >>>>>> different configuration options (number of task
> >>>>>> managers/memory/Flip6/credit base flow control/metrics) and
> >>>>>> everything
> >>>>>> looks now fine (after fixing
> >>>>>> https://issues.apache.org/jira/browse/FLINK-9969 <
> >>>>>> https://issues.apache.org/jira/browse/FLINK-9969> ).
> >>>>>>> Piotrek
> >>>>>>>
> >>>>>>>> On 7 Aug 2018, at 17:17, Till Rohrmann <trohrmann@xxxxxxxxxx>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Hi everyone,
> >>>>>>>> Please review and vote on the release candidate #4 for the version
> >>>>>> 1.6.0,
> >>>>>>>> as follows:
> >>>>>>>> [ ] +1, Approve the release
> >>>>>>>> [ ] -1, Do not approve the release (please provide specific
> >>>>>>>> comments)
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> The complete staging area is available for your review, which
> >>>>>>>> includes:
> >>>>>>>> * JIRA release notes [1],
> >>>>>>>> * the official Apache source release and binary convenience
> >>>>>>>> releases to
> >>>>>> be
> >>>>>>>> deployed to dist.apache.org [2], which are signed with the key
> with
> >>>>>>>> fingerprint 1F302569A96CFFD5 [3],
> >>>>>>>> * all artifacts to be deployed to the Maven Central Repository
> [4],
> >>>>>>>> * source code tag "release-1.6.0-rc4" [5],
> >>>>>>>> * website pull request listing the new release and adding
> >>>>>>>> announcement
> >>>>>> blog
> >>>>>>>> post [6].
> >>>>>>>>
> >>>>>>>> Please use this document for coordinating testing efforts: [7]
> >>>>>>>>
> >>>>>>>> The vote will be shortened since we only adde a minor fix on top
> >>>>>>>> of the
> >>>>>> RC
> >>>>>>>> 3. It will close on Wednesday 6:30pm CET. It is adopted by
> majority
> >>>>>>>> approval, with at least 3 PMC affirmative votes.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Your friendly Release Manager
> >>>>>>>>
> >>>>>>>> [1]
> >>>>>>>>
> >>>>>>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12342760
> >>>>>>
> >>>>>>>> [2] https://dist.apache.org/repos/dist/dev/flink/flink-1.6.0/
> >>>>>>>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> >>>>>>>> [4]
> >>>>>>
> https://repository.apache.org/content/repositories/orgapacheflink-1178
> >>>>>>
> >>>>>>>> [5] https://github.com/apache/flink/tree/release-1.6.0-rc4
> >>>>>>>> [6] https://github.com/apache/flink-web/pull/117
> >>>>>>>> [7]
> >>>>>>>>
> >>>>>>
> https://docs.google.com/document/d/1upBFZQ7tbaSkYvDiLqfUFXKg8Xxs-lVheEfb66e4jpo/edit?usp=sharing
> >>>>>>
> >>>>>>>> Pro-tip: you can create a settings.xml file with these contents:
> >>>>>>>>
> >>>>>>>> <settings>
> >>>>>>>> <activeProfiles>
> >>>>>>>>     <activeProfile>flink-1.6.0</activeProfile>
> >>>>>>>> </activeProfiles>
> >>>>>>>> <profiles>
> >>>>>>>>     <profile>
> >>>>>>>>       <id>flink-1.6.0</id>
> >>>>>>>>       <repositories>
> >>>>>>>>         <repository>
> >>>>>>>>           <id>flink-1.6.0</id>
> >>>>>>>>           <url>
> >>>>>>>>
> >>>>>>>>
> https://repository.apache.org/content/repositories/orgapacheflink-1178/
> >>>>>>>>
> >>>>>>>>           </url>
> >>>>>>>>         </repository>
> >>>>>>>>         <repository>
> >>>>>>>>           <id>archetype</id>
> >>>>>>>>           <url>
> >>>>>>>>
> >>>>>>>>
> https://repository.apache.org/content/repositories/orgapacheflink-1178/
> >>>>>>>>
> >>>>>>>>           </url>
> >>>>>>>>         </repository>
> >>>>>>>>       </repositories>
> >>>>>>>>     </profile>
> >>>>>>>> </profiles>
> >>>>>>>> </settings>
> >>>>>>>>
> >>>>>>>> And reference that in you maven commands via --settings
> >>>>>>>> path/to/settings.xml. This is useful for creating a quickstart
> >>>>>>>> based on
> >>>>>> the
> >>>>>>>> staged release and for building against the staged jars.
> >>
>
>