git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Weird behaviour after change sources in a job.


Hello!

 

We have found a weird issue while replacing the source in one of our Flink SQL Jobs.

 

We have a job which was reading from a Kafka topic (with externalize checkpoints) and we needed to change the topic while keeping the same logic for the job/SQL.

After we restarted the job, instead of consuming from the new Kafka topic, it consumed from both! Duplicating the input of our job.

We were able to reproduce the issue but we don’t understand if this is a bug or expected behavior and in this case we should have restarted from a clean state.

We are using Flink 1.4 at the moment and Kafka 0.10.2.1

 

Thank you,

Juan