git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Cassandra crashes after loading data with sstableloader


What’s the cardinality of hash? 

Do they have the same schema? If so you may be able to take a snapshot and hardlink it in / refresh instead of sstableloader. Alternatively you could drop the index from the destination keyspace and add it back in after the load finishes.

How big are the sstables? How big is your heap? Are you already serving traffic? 

-- 
Jeff Jirsa


On Jul 29, 2018, at 3:43 PM, Rahul Singh <rahul.xavier.singh@xxxxxxxxx> wrote:

What does “hash” Data look like?

Rahul
On Jul 24, 2018, 11:30 AM -0400, Arpan Khandelwal <arpan.ps@xxxxxxxxx>, wrote:
I need to clone data from one keyspace to another keyspace.
We do it by taking snapshot of keyspace1 and restoring in keyspace2 using sstableloader.

Suppose we have following table with index on hash column. Table has around 10M rows.
---------------------
CREATE TABLE message (
 id     uuid,
 messageid     uuid,
 parentid     uuid,
 label     text,
 properties     map<text,text>,
 text1     text,
 text2     text,
 text3     text,
 category     text,
 hash     text,
 info     map<text,text>,
 creationtimestamp     bigint,
 lastupdatedtimestamp     bigint,
 PRIMARY KEY ( (id) )
 );

CREATE  INDEX  ON message ( hash );
---------------------
Cassandra crashes when i load data using sstableloader. Load is happening correctly but seems that cassandra crashes when its trying to build index on table with huge data.

I have two questions.
1. Is there any better way to clone keyspace?
2. How can i optimize sstableloader to load data and not crash cassandra while building index.

Thanks
Arpan