[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Java based AWS IO connector

Beam works with S3 out of the box, you can provide s3://... paths to anything that works with files. Don't remember if it's already available in 2.4 or only starting 2.5 - does it currently not work for you?

On Tue, May 29, 2018, 2:26 PM S. Sahayaraj <ssahayaraj@xxxxxxxxx> wrote:

Hello All,

                The data source for our Beam pipleline is in S3 bucket, Is there any built-in I/O Connector available with Java samples? If so, can you please guide me how to integrate with them?.


                I am using Bean SDK for Java version 2.4.0 and Spark runner in clustered deployment.









S. Sahayaraj