git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Lost counter updates during Cassandra upgrade 2.2.11 to 3.11.2


Hi,

During rolling upgrade of our cluster we noticed that some updates on table with counters were not being applied. It looked as if it depended on whether coordinator handling request was already upgraded or not. I observed similar behavior while using cqlsh and executing queries manually. Sometimes it took several retries to see counter updated. There were no errors/warns in neither application nor Cassandra logs. The updates started working reliably once again when all nodes in dc have been upgraded. However, the lost updates did not reappear. 

Our setup:
2 dc cluster, 5 + 5 nodes. However, only one is used for queries as client application is co-located in one region. I believe 1 dc is enough to reproduce it. 
Replication factor 3+2
Consistency level LOCAL_QUORUM
Upgrading 2.2.11 to 3.11.2

I haven't found any report of similar issue on the internet. Has anyone heard about such behavior? 

Thanks, 
Konrad

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@xxxxxxxxxxxxxxxxxxxx
For additional commands, e-mail: user-help@xxxxxxxxxxxxxxxxxxxx