I am perftesting cassandra over a longrun in a cluster of 8 nodes and i noticed the rate of service drops.
Most of the nodes have the CPU between 40-65% however one of the nodes has a higher CPU and also started performing a lot of read IOPS as seen in the image. (green is read IOPS)
My test has a mixed rw scenario.
1. insert row
2. after 60 seconds read row
3. delete row.
The rate of inserts is bigger than the rate of deletes so some delete will not happen.
I have checked the client it it does not accumulate RAM, GC is a straight line so o don't understand whats going on.