git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Cannot set classpath which can be used before job is submitted to yarn.


Hi Hequn,


I can read it by using the full path, but I want it to be in program's classpath. For example, I use “bin/flink run -m yarn-cluster” to run the program on Server1, and I have a conf file “config.conf" located in “/server/conf” on Server1, I can’t read this file by using ConfigFactory.load(“config.conf”) unless I make changes to the “bin/flink” shell like:


FLOW_CLASSPATH="/server/conf"

exec $JAVA_RUN $JVM_ARGS "${log_setting[@]}" -classpath "`manglePathList "$FLOW_CLASSPATH:$CC_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`" org.apache.flink.client.cli.CliFrontend "$@"


 Original Message 
Sender: Hequn Cheng<chenghequn@xxxxxxxxx>
Recipient: bupt_ljy<bupt_ljy@xxxxxxx>
Cc: user<user@xxxxxxxxxxxxxxxx>
Date: Friday, Sep 7, 2018 09:29
Subject: Re: Cannot set classpath which can be used before job is submitted to yarn.

Hi bupt,

No sure about the answer. 
Do you mean that you can't read the file from local FS? Have you ever tried load the file through a full path? or you choose a wrong classloader.

Best, Hequn

On Thu, Sep 6, 2018 at 11:01 PM bupt_ljy <bupt_ljy@xxxxxxx> wrote:

Hi,all

   I’m using “bin/flink run -m yarn-cluster” to run my program on yarn. However, it seems that I can’t add my own files into classpath before the the job is submitted to yarn. For example, I have a conf file, which located in my own conf directory, and I need to load file from the conf directory to initialize some classes instances before it’s submitted to yarn.

  I’m grateful if anyone can give me some help.

  Thanks.