I have a problem with the hints in my cluster.
In fact, my cluster is composed of 5 nodes : 3 nodes in DC1 and 2 nodes in DC2.
The size of the hints folder increases rapidly and it reaches nearly the 90% of the hard drive (50 Gb).
The test scenario is the following : I applied 1000 UPDATE requests per second on the first DC. I wanted to make this test lasts 3 hours. The first node of the second DC stopped, and after few minutes, the second node stopped as well. The hints files started to increase rapidly and consumed a big part of my hard drive.
This test fails after few minutes.
The replication factor is 2 for both DCs and the consistency level is LOCAL_ONE.
Could you please tell me what is the best configuration of the hints properties : hinted_handoff_throttle_in_kb, max_hint_window_in_ms and max_hints_delivery_threads ?