git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Encountered the following Expired Iterator exception in getRecords using FlinkKinesisConsumer


Hi,
Using FlinkKinesisConsumer in a long running Flink Streaming app consuming from a Kinesis Stream. 
Encountered the following Expired Iterator exception in getRecords():
 org.apache.flink.streaming.connectors.kinesis.internals.ShardConsumer [] - Encountered an unexpected expired iterator 
 
The error on the console ends up being a misleading one: "Caused by: org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.model.AmazonKinesisException: 1 validation error detected: Value 'EARLIEST_SEQUENCE_NUM' at 'startingSequenceNumber' failed to satisfy constraint: Member must satisfy regular _expression_ pattern: 0|([1-9]\d{0,128}) (Service: AmazonKinesis; Status Code: 400; Error Code: ValidationException; Request ID: ..)
"
 
How do I increase the ClientConfiguration.clientExecutiontimeout to avoid this issue or is this the right way to handle this issue ? I don't want the FlinkKinesisConsumer streaming app to fail just because there might be no records in the Kinesis Stream. I am using TRIM_HORIZON to read from the start of the Kinesis Stream.
 
 TIA,