[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: writing to maprfs?

Hello Akanksha,

Thank you for reporting this! It looks that website misses the details how to use “” with different FSs, so I created a Jira to improve this:


On 24 May 2018, at 10:36, Akanksha Sharma B <akanksha.b.sharma@xxxxxxxxxxxx> wrote:


Answering my own question 😊

writing to maprfs worked after I added following property to core-site.xml:-



From: Akanksha Sharma B
Sent: Thursday, May 24, 2018 9:24:25 AM
To: user@xxxxxxxxxxxxxxx
Subject: writing to maprfs?

Hi All,

I have a pipeline; it writes to hdfs, using "" package.
I was hoping that it will work with maprfs as well. However I have been debugging for some days, with no success.

I do not provide hdfsConfiguration from command line, and instead use the configuration file /opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/core-site.xml (please find it attached with the mail)
However I get the stack trace ("No FileSystem for scheme: maprfs"). Please find the detailed trace attached. I made sure that I have/opt/mapr/lib/maprfs-5.2.2-mapr.jar on classpath. I use spark submit to run the pipeline.

If you have already tried writing to maprfs from beam, ccould you please help?