git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Exception when run flink-storm-example


Hi vino

       I tried change "localhost" to the real IP. But still throw exception as below. JobManager configuration is as below.

 

Thanks.

Hanjing

------------------------------------

flink-conf.yaml:

jobmanager.rpc.address: 170.0.0.46

 

# The RPC port where the JobManager is reachable.

 

jobmanager.rpc.port: 6123

 

 

# The heap size for the JobManager JVM

 

jobmanager.heap.size: 1024m

 

 

# The heap size for the TaskManager JVM

 

taskmanager.heap.size: 10240m

 

 

# The number of task slots that each TaskManager offers. Each slot runs one parallel pipeline.

 

taskmanager.numberOfTaskSlots: 16

 

# The parallelism used for programs that did not specify and other parallelism.

 

parallelism.default: 2

 

 

Excepiton log:

 

Starting execution of program

 

------------------------------------------------------------

The program finished with the following exception:

 

org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.

        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:546)

        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)

        at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:426)

        at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:804)

        at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:280)

        at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:215)

        at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1044)

        at org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1120)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:422)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)

        at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)

        at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1120)

Caused by: java.lang.RuntimeException: Could not connect to Flink JobManager with address 170.0.0.46:6123

        at org.apache.flink.storm.api.FlinkClient.getTopologyJobId(FlinkClient.java:304)

        at org.apache.flink.storm.api.FlinkSubmitter.submitTopology(FlinkSubmitter.java:107)

        at org.apache.flink.storm.wordcount.WordCountRemoteBySubmitter.main(WordCountRemoteBySubmitter.java:75)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)

        ... 12 more

Caused by: java.io.IOException: Actor at akka.tcp://flink@170.0.0.46:6123/user/jobmanager not reachable. Please make sure that the actor is running and its port is reachable.

        at org.apache.flink.runtime.akka.AkkaUtils$.getActorRef(AkkaUtils.scala:547)

        at org.apache.flink.runtime.akka.AkkaUtils.getActorRef(AkkaUtils.scala)

        at org.apache.flink.storm.api.FlinkClient.getJobManager(FlinkClient.java:339)

        at org.apache.flink.storm.api.FlinkClient.getTopologyJobId(FlinkClient.java:278)

        ... 19 more

Caused by: akka.actor.ActorNotFound: Actor not found for: ActorSelection[Anchor(akka.tcp://flink@170.0.0.46:6123/), Path(/user/jobmanager)]

        at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:68)

        at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:66)

        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)

        at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)

        at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:73)

        at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:76)

        at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:120)

        at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:75)

        at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)

        at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)

        at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:534)

        at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:97)

        at akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:982)

        at akka.actor.Actor$class.aroundReceive(Actor.scala:502)

        at akka.remote.EndpointActor.aroundReceive(Endpoint.scala:446)

        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)

        at akka.actor.ActorCell.invoke(ActorCell.scala:495)

        at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)

        at akka.dispatch.Mailbox.run(Mailbox.scala:224)

        at akka.dispatch.Mailbox.exec(Mailbox.scala:234)

        at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

        at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

        at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

        at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

 

On 9/10/2018 20:17vino yang<yanghua1127@xxxxxxxxx> wrote:
Hi Hanjing,

OK, I mean you change the "localhost" to the real IP.

Try it.

Thanks, vino. 

jing <hanjingzuzu@xxxxxxx> 于2018年9月10日周一 下午8:07写道:
Hi vino,
jonmanager rpc address value is setted by localhost.
hadoop3@p-a36-72 is the node host the jobmanager jvm.

Thanks.
Hanjing



jing
邮箱hanjingzuzu@xxxxxxx

签名由 网易邮箱大师 定制

On 09/10/2018 19:25, vino yang wrote:
Hi Hanjing,

I mean this configuration key.[1]

What's more, Is the "hadoop3@p-a36-72" also the node which host JobManager's jvm process?

Thanks, vino.


jing <hanjingzuzu@xxxxxxx> 于2018年9月10日周一 下午6:57写道:
Hi vino,
  I commit the job on the jvm code with the command below.
hadoop3@p-a36-72 flink-1.6.0]$ ./bin/flink run WordCount-StormTopology.jar input output

And I'm a new user, which configuation name should be set. All the configuations are the default setting now.

Thanks.
Hanjing

jing
邮箱hanjingzuzu@xxxxxxx

签名由 网易邮箱大师 定制

On 09/10/2018 15:49, vino yang wrote:
Hi Hanjing,

Did you perform a CLI commit on the JM node? Is the address bound to "localhost" in the Flink JM configuration?

Thanks, vino.

jing <hanjingzuzu@xxxxxxx> 于2018年9月10日周一 上午11:00写道:

Hello

       Im trying to run flink-storm-example on standalone clusters. But theres some exception I cant sovle. Could anyone please help me with trouble.

       flink-storm-example version: 1.60

       flink version: 1.60

       The log below is the Exception. My job manager status is as the picture.

       Iv tried to changed the IP address and port, but it doesntwork.

      

       Thanks a lot.

-------------------------------------------

[hadoop3@p-a36-72 flink-1.6.0]$ ./bin/flink run WordCount-StormTopology.jar input output

Starting execution of program


------------------------------------------------------------

 The program finished with the following exception:


org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.

        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:546)

        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)

        at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:426)

        at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:804)

        at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:280)

        at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:215)

        at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1044)

        at org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1120)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:422)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)

        at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)

        at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1120)

Caused by: java.lang.RuntimeException: Could not connect to Flink JobManager with address localhost:6123

        at org.apache.flink.storm.api.FlinkClient.getTopologyJobId(FlinkClient.java:304)

        at org.apache.flink.storm.api.FlinkSubmitter.submitTopology(FlinkSubmitter.java:107)

        at org.apache.flink.storm.wordcount.WordCountRemoteBySubmitter.main(WordCountRemoteBySubmitter.java:75)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)

        ... 12 more

Caused by: java.io.IOException: Actor at akka.tcp://flink@localhost:6123/user/jobmanager not reachable. Please make sure that the actor is running and its port is reachable.

        at org.apache.flink.runtime.akka.AkkaUtils$.getActorRef(AkkaUtils.scala:547)

        at org.apache.flink.runtime.akka.AkkaUtils.getActorRef(AkkaUtils.scala)

        at org.apache.flink.storm.api.FlinkClient.getJobManager(FlinkClient.java:339)

        at org.apache.flink.storm.api.FlinkClient.getTopologyJobId(FlinkClient.java:278)

        ... 19 more

Caused by: akka.actor.ActorNotFound: Actor not found for: ActorSelection[Anchor(akka.tcp://flink@localhost:6123/), Path(/user/jobmanager)]

        at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:68)

        at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:66)

        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)

        at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)

        at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:73)

        at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:76)

        at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:120)

        at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:75)

        at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)

        at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)

        at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:534)

        at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:97)

        at akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:982)

        at akka.actor.Actor$class.aroundReceive(Actor.scala:502)

        at akka.remote.EndpointActor.aroundReceive(Endpoint.scala:446)

        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)

        at akka.actor.ActorCell.invoke(ActorCell.scala:495)

        at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)

        at akka.dispatch.Mailbox.run(Mailbox.scala:224)

        at akka.dispatch.Mailbox.exec(Mailbox.scala:234)

        at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

        at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

        at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

        at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

  


 

Hanjing