git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: VerifyError when running Python streaming job


Hi Joe,

Did you try the word_count example from the flink codebase?[1]

Recently, I tried this example, it works fine to me.

An example of an official document may not guarantee your success due to maintenance issues.

cc @Chesnay

[1]: https://github.com/apache/flink/blob/master/flink-libraries/flink-streaming-python/src/test/python/org/apache/flink/streaming/python/api/examples/word_count.py

Thanks, vino.

Joe Malt <jmalt@xxxxxxxx> 于2018年8月8日周三 上午5:29写道:
Hi,

I'm running into errors when trying to run a Flink streaming program. 

Running the WordCount example from the docs fails with this error:

java.lang.VerifyError: (class: site$py, method: _Helper$26 signature: (Lorg/python/core/PyFrame;Lorg/python/core/ThreadState;)Lorg/python/core/PyObject;) Incompatible argument to function
	at java.lang.Class.getDeclaredConstructors0(Native Method)
	at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
	at java.lang.Class.getConstructor0(Class.java:3075)
	at java.lang.Class.getConstructor(Class.java:1825)
	at org.python.core.BytecodeLoader.makeCode(BytecodeLoader.java:70)
	at org.python.core.util.importer.getModuleCode(importer.java:217)
	at org.python.core.util.importer.importer_load_module(importer.java:95)
	at org.python.modules.zipimport.zipimporter.zipimporter_load_module(zipimporter.java:163)
	at org.python.modules.zipimport.zipimporter$zipimporter_load_module_exposer.__call__(Unknown Source)
	at org.python.core.PyBuiltinMethodNarrow.__call__(PyBuiltinMethodNarrow.java:46)
	at org.python.core.imp.loadFromLoader(imp.java:587)
	at org.python.core.imp.find_module(imp.java:537)
	at org.python.core.imp.import_next(imp.java:840)
	at org.python.core.imp.import_first(imp.java:861)
	at org.python.core.imp.load(imp.java:716)
	at org.python.core.Py.importSiteIfSelected(Py.java:1558)
	at org.python.util.PythonInterpreter.<init>(PythonInterpreter.java:116)
	at org.python.util.PythonInterpreter.<init>(PythonInterpreter.java:94)
	at org.python.util.PythonInterpreter.<init>(PythonInterpreter.java:71)
	at org.apache.flink.streaming.python.util.InterpreterUtils.initPythonInterpreter(InterpreterUtils.java:122)
	at org.apache.flink.streaming.python.util.InterpreterUtils.deserializeFunction(InterpreterUtils.java:73)
	at org.apache.flink.streaming.python.api.functions.AbstractPythonUDF.open(AbstractPythonUDF.java:51)
	at org.apache.flink.streaming.python.api.functions.PythonFlatMapFunction.open(PythonFlatMapFunction.java:51)
	at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
	at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
	at org.apache.flink.streaming.api.operators.StreamFlatMap.open(StreamFlatMap.java:43)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:424)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:290)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
	at java.lang.Thread.run(Thread.java:748)

The only job I can get to run is one that reads from a KafkaConsumer010 and outputs the result without applying any operations (if there is a Filter, Map, etc. it will crash with the same error). However, removing all the operations from the WordCount example doesn't fix the issue there.

I'm running Flink 1.6-RC, I've also tried 1.6-SNAPSHOT which made no difference.

Any help would be much appreciated.

Thanks,

Joe Malt

Software Engineer Intern
Yelp