git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

connection failed when running flink in a cluster


Hello everyone,

I am trying to run Flink on Raspberry Pis. My first test for word count in a single node worked. I just have to decrease the Heap memory of the jobmanager.heap.mb and taskmanager.heap.mb to 512.
My second test is to add 2 slave nodes I got the error: "Java HotSpot(TM) Client VM warning: G1 GC is disabled in this release." at the file log/flink-root-taskexecutor-0-*.out.

This link (https://blog.sflow.com/2016/06/raspberry-pi-real-time-network-analytics.html) says that in order to Raspberry Pi ARM architecture works with JVM it is necessary to configure the JVM as:
-Xms600M
-Xmx600M
-XX:+UseParNewGC
-XX:+UseConcMarkSweepGC
-XX:+CMSIncrementalMode

then I set this variables on the path inside the file flink-conf.yaml
env.java.opts: "-XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode"
env.java.opts.jobmanager: "-XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode"
env.java.opts.taskmanager: "-XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode"

and the error "Java HotSpot(TM) Client VM warning: G1 GC is disabled in this release." is not showing anymore. However, the connection from the master node to the slave node is still not possible. Does anybody know how I must configure flink to deal with that?

This is the error stack trace:

2017-05-25 12:40:26,421 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source: Socket Stream -> Flat Map (1/1) (b81b6492fc0860367be422d0b0bf4358) switched from DEPLOYING to RUNNING.
2017-05-25 12:40:26,891 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source: Socket Stream -> Flat Map (1/1) (b81b6492fc0860367be422d0b0bf4358) switched from RUNNING to FAILED.
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.flink.streaming.api.functions.source.SocketTextStreamFunction.run(SocketTextStreamFunction.java:96)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:87)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:56)
at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:99)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:306)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:745)
2017-05-25 12:40:26,898 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Job Socket Window WordCount (71c6d7796eccf6587d9d1deda0490e09) switched from state RUNNING to FAILING.
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.flink.streaming.api.functions.source.SocketTextStreamFunction.run(SocketTextStreamFunction.java:96)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:87)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:56)
at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:99)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:306)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
at java.lang.Thread.run(Thread.java:745)
2017-05-25 12:40:26,921 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Window(TumblingProcessingTimeWindows(5000), ProcessingTimeTrigger, ReduceFunction$1, PassThroughWindowFunction) -> Sink: Print to Std. Out (1/1) (aa1a0e7ee3a1d3ad8f99b2608bd64c5b) switched from RUNNING to CANCELING.
2017-05-25 12:40:26,975 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Window(TumblingProcessingTimeWindows(5000), ProcessingTimeTrigger, ReduceFunction$1, PassThroughWindowFunction) -> Sink: Print to Std. Out (1/1) (aa1a0e7ee3a1d3ad8f99b2608bd64c5b) switched from CANCELING to CANCELED.



Thanks, Felipe
--
-- Felipe Gutierrez
-- skype: felipe.o.gutierrez


( ! ) Warning: include(msgfooter.php): failed to open stream: No such file or directory in /var/www/git/apache-flink-users/msg11140.html on line 69
Call Stack
#TimeMemoryFunctionLocation
10.0022372792{main}( ).../msg11140.html:0

( ! ) Warning: include(): Failed opening 'msgfooter.php' for inclusion (include_path='.:/var/www/git') in /var/www/git/apache-flink-users/msg11140.html on line 69
Call Stack
#TimeMemoryFunctionLocation
10.0022372792{main}( ).../msg11140.html:0