git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: ConcurrentModificationException while accessing managed keyed state



Hi Garvit,

this is unexpected, could you please provide more information about this?

- which flink version are you using?
- what state backend are you using?
- are you using the incremental checkpoint?(in case you used the rocksdb backend)
- did you create the customer thread to operate the state

and the exception log would also definitely help a lot if you could share with us.

Best, Sihua


On 06/02/2018 12:08, Garvit Sharma wrote:
Hi,

I have a use case where I am keeping the keyed state in ProcessFunction. 

Key: Integer personId;/** * The data type stored in the state */ public class PersonDetails { public long count; public long lastModified; }

I have encountered a lot of ConcurrentModificationException. 

I thought Flink processes all the operators on a keyed stream in a single thread. It seems like operators being accessed through multiple threads.

If I get such exception then the data coming from Kafka would be consumed without making an update to the internal state. Making me lose the data.

Please help me in handling the case according to my use case.

Thanks,

--

Garvit Sharma
github.com/garvitlnmiit/

No Body is a Scholar by birth, its only hard work and strong determination that makes him master.