[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Can JDBCSinkFunction support exectly once?


JDBCSinkFunction is a simple wrapper around the JDBCOutputFormat (the DataSet / Batch API output interface).
Dominik is right, that JDBCSinkFunction does not support exactly-once output.

It is not strictly required that an exactly-once sink implements TwoPhaseCommitFunction.
TPCF is a convenience interface that internally use the CheckpointListener interface. It makes it easier to implement exactly-once sinks but is not the only way to do it.

Implementing an exactly-once JDBC sink by extending TwoPhaseCommitFunction would require to be able to recover (and commit) an open transaction after a task was restarted.
Not sure if the JDBC interface supports this.

Best, Fabian

Am Mi., 21. Nov. 2018 um 11:27 Uhr schrieb Dominik Wosiński <wossyn@xxxxxxxxx>:

As far as I know, the function needs to implement the TwoPhaseCommitFunction and not the CheckpointListener. JDBCSinkFunction does not implement the two-phase commit, so currently it does not support exactly once.

Best Regards,

śr., 21 lis 2018 o 11:07 Jocean shi <jocean.shi@xxxxxxxxx> napisał(a):
Can JDBCSinkFunction support exectly once? Is it that The JDBCSinkFunction dont't implement CheckpointListener meaning JDBCSinkFunction dont't support exectly once?