Re: C* as fluent data storage, 10MB/sec/node?
Agree with JEFF in twcs. Also look
Chief Executive Officer
1010 Wisconsin Ave NW, Suite 250
Washington, D.C. 20007
We build and manage digital business technology platforms.
On Nov 28, 2018, 5:33 PM -0500, Adam Smith <adamsmith8745@xxxxxxxxx>, wrote:
Thanks for the excellent advice, this was extremely helpful! Did not know about TWCS... curing a lot of headache.
Probably fine as long as there’s some concept of time in the partition key to keep them from growing unbounded.
Use TWCS, TTLs and something like 5-10 minute buckets. Don’t use RF=1, but you can write at CL ONE. TWCS will largely just drop whole sstables as they expire (especially with 3.11 and the more aggressive expiration logic there)
> On Nov 28, 2018, at 11:24 AM, Adam Smith <adamsmith8745@xxxxxxxxx> wrote:
> Hi All,
> I need to use C* somehow as fluent data storage - maybe this is different to the queue antipattern? Lots of data come in (10MB/sec/node), remains for e.g. 1 hour and should then be evicted. It is somehow not critical when data would occasionally disappear/get lost.
> Thankful for any advice!
> Is this nowadays possible without suffering too much from compactation? I would not have ranged tombstones, and depending on a possible solution only using point deletes (PK+CK). There is only one CK, could also be empty.
> 1) The data is usually 1 MB. Can I just update with empty data? PK + CK would remain, but I would not carry about that. Would this create tombstones or is equivalent to a DELETE?
> 2) Like 1) and later then set a TTL == small amount of data to be deleted then? And hopefully small compactation?
> 3) Simply setting TTL 1h and hoping the best, because I am wrong with my worries?
> 4) Any optimization strategies like setting the RF to 1? Which compactation strategy is advised?
> 5) Are there any recent performance benchmarks for one of the scenarios?
> What else could I do?
> Thanks a lot!
To unsubscribe, e-mail: user-unsubscribe@xxxxxxxxxxxxxxxxxxxx
For additional commands, e-mail: user-help@xxxxxxxxxxxxxxxxxxxx