git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Looking for a working POM file example for EMR cluster


Hi,

I'm looking for a sample POM file that works when running on EMR cluster. I'm new to Flink and EMR, so I'm simply following AWS EMR documentation on Flink and I am creating a Step and submitting my program JAR file. My program is just a slight modification of the Wikipedia example. 

I was trying to follow an example from AWS reference architecture for their Taxi events example: https://github.com/aws-samples/flink-stream-processing-refarch/blob/master/flink-taxi-stream-processor/pom.xml 

However, I've been seeing various errors having to do with dependencies and ClassNotFoundExceptions for basic common Flink dependencies. I tried removing excludes from the maven-shade-plugin section of the POM file from the reference architecture, and now I'm seeing the following exception:

Exception in thread "main" com.typesafe.config.ConfigException$UnresolvedSubstitution: Could not resolve substitution to a value: ${akka.stream.materializer}

If I run a local Flink cluster and submit my JAR, I'm not seeing any issues with pretty much any way I modify the POM file. I would greatly appreciate if someone can point me to a working POM example.

Thanks!
Turar

 



( ! ) Warning: include(msgfooter.php): failed to open stream: No such file or directory in /var/www/git/apache-flink-users/msg09562.html on line 85
Call Stack
#TimeMemoryFunctionLocation
10.0008363576{main}( ).../msg09562.html:0

( ! ) Warning: include(): Failed opening 'msgfooter.php' for inclusion (include_path='.:/var/www/git') in /var/www/git/apache-flink-users/msg09562.html on line 85
Call Stack
#TimeMemoryFunctionLocation
10.0008363576{main}( ).../msg09562.html:0