git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [VOTE] Release 1.6.0, release candidate #4


I found some issues with the standalone-job.sh script.

I ran "./bin/standalone-job.sh start" as described by the usage string.

   2018-08-08 09:22:34,385 ERROR
   org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
   Could not parse command line arguments [--configDir,
   /home/zento/svn/flink-1.6.0/flink-1.6.0/conf].
   org.apache.flink.runtime.entrypoint.FlinkParseException: Failed to
   parse the command line arguments.
            at
   org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52)
            at
   org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:143)
   Caused by: org.apache.commons.cli.MissingOptionException: Missing
   required option: j
            at
   org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.java:199)
            at
   org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130)
            at
   org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81)
            at
   org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:50)
           ... 1 more

The script should fail earlier if no jar is provided, with a better error message.
It is also undocumented, and the usage instructions don't appear correct.

Passing a jar with the -j option leads to a ClassNotFoundException:
"./bin/standalone-job.sh start -j examples/streaming/WordCount.jar"

   2018-08-08 09:26:30,562 ERROR
   org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
   Cluster initialization failed.
   java.lang.reflect.UndeclaredThrowableException
            at
   org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1854)
            at
   org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
            at
   org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:189)
            at
   org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:158)
   Caused by: org.apache.flink.util.FlinkException: Could not load the
   provied entrypoint class.
            at
   org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.createPackagedProgram(StandaloneJobClusterEntryPoint.java:92)
            at
   org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.retrieveJobGraph(StandaloneJobClusterEntryPoint.java:75)
            at
   org.apache.flink.runtime.entrypoint.JobClusterEntrypoint.createDispatcher(JobClusterEntrypoint.java:107)
            at
   org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startClusterComponents(ClusterEntrypoint.java:353)
            at
   org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:232)
            at
   org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:190)
            at java.security.AccessController.doPrivileged(Native Method)
            at javax.security.auth.Subject.doAs(Subject.java:422)
            at
   org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
            ... 3 more
   Caused by: java.lang.ClassNotFoundException:
   examples/streaming/WordCount.jar
            at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
            at
   sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
            at
   org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.createPackagedProgram(StandaloneJobClusterEntryPoint.java:89)
            ... 11 more

So this seems to not work at all, but maybe I'm using it wrong?

(There's also typo in "Could not load the provied entrypoint class")

On 08.08.2018 10:33, Piotr Nowojski wrote:
+1 from my side

I’ve spent some time playing around with various examples (batching, streaming and SQL) on EMR 6 nodes cluster with yarn deployment, with different configuration options (number of task managers/memory/Flip6/credit base flow control/metrics) and everything looks now fine (after fixing https://issues.apache.org/jira/browse/FLINK-9969 <https://issues.apache.org/jira/browse/FLINK-9969> ).

Piotrek

On 7 Aug 2018, at 17:17, Till Rohrmann <trohrmann@xxxxxxxxxx> wrote:

Hi everyone,
Please review and vote on the release candidate #4 for the version 1.6.0,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)


The complete staging area is available for your review, which includes:
* JIRA release notes [1],
* the official Apache source release and binary convenience releases to be
deployed to dist.apache.org [2], which are signed with the key with
fingerprint 1F302569A96CFFD5 [3],
* all artifacts to be deployed to the Maven Central Repository [4],
* source code tag "release-1.6.0-rc4" [5],
* website pull request listing the new release and adding announcement blog
post [6].

Please use this document for coordinating testing efforts: [7]

The vote will be shortened since we only adde a minor fix on top of the RC
3. It will close on Wednesday 6:30pm CET. It is adopted by majority
approval, with at least 3 PMC affirmative votes.

Thanks,
Your friendly Release Manager

[1]
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12342760
[2] https://dist.apache.org/repos/dist/dev/flink/flink-1.6.0/
[3] https://dist.apache.org/repos/dist/release/flink/KEYS
[4] https://repository.apache.org/content/repositories/orgapacheflink-1178
[5] https://github.com/apache/flink/tree/release-1.6.0-rc4
[6] https://github.com/apache/flink-web/pull/117
[7]
https://docs.google.com/document/d/1upBFZQ7tbaSkYvDiLqfUFXKg8Xxs-lVheEfb66e4jpo/edit?usp=sharing

Pro-tip: you can create a settings.xml file with these contents:

<settings>
<activeProfiles>
  <activeProfile>flink-1.6.0</activeProfile>
</activeProfiles>
<profiles>
  <profile>
    <id>flink-1.6.0</id>
    <repositories>
      <repository>
        <id>flink-1.6.0</id>
        <url>

https://repository.apache.org/content/repositories/orgapacheflink-1178/
        </url>
      </repository>
      <repository>
        <id>archetype</id>
        <url>

https://repository.apache.org/content/repositories/orgapacheflink-1178/
        </url>
      </repository>
    </repositories>
  </profile>
</profiles>
</settings>

And reference that in you maven commands via --settings
path/to/settings.xml. This is useful for creating a quickstart based on the
staged release and for building against the staged jars.