Cassandra Driver

Authors

Jingxin Feng

Xiaowei Wang

Implementation

As data in TPC-C is saved using relational schema, we need to transform all the relational tables into column-based style. We finally decided to use primary keys in original tables as row keys in each Cassandra’s Column Family. Say, for the table NEW-ORDER. The primary key of the table is NO_W_ID, NO_D_ID, NO_O_ID. So the row key for NEW-ORDER CF should be NO_W_ID+NO_D_ID+NO_O_ID. Row keys should be unique. To avoid repeating, we use zeros to fill each field in the row key. For each filed, there should be at least 5 digits. All the data saved in Column Families are bytes. We explicitly convert each field to the corresponding type in our driver.

In addition, we have to build secondary index on certain columns for queries such as

SELECT XXX FROM table WHERE XXX=xxx

Finally, we decided to give up batch_insert(), but only use single insert. Because after many tests, we found there’s no difference between them. And some one in Cassandra community has a good explanation for the reasons: batch_mutate() and insert() follow the a similar execution path to a single insert in the server.

It's not like putting multiple statements in a Transaction in the RDBMS. Where they do differ is that you can provide multiple columns for a row in a column family, and these will be applied as one operation including only one write to the commit log. However row you send requires a write to the commit log.

It seems the batch operation only batch multiple columns in a single row not multiple rows in column family.

Driver Dependencies

• Client: pycassa 1.0.8

Note: Make sure the the keyspace does not exist before load process. The program will create the keyspace specified in the configuration file.

Known Issues

• Memory consumption: Cassandra is very memory-consuming in our implementation of TPC-C. JVM performs GC a lot of times during running the benchmark. There may be communication errors between different nodes when GC is being performed on one or more nodes. We increased the RPC timeout and the problem was mitigated but can not be eliminated. For this reason, there are some node unavailable exceptions during the long load process, and some data is not loaded into Cassandra successfully. When the nodes recover from GC, the loading will proceed normally. The memory issue may be due to the strategy that data will first be written to memtable and then flushed to the disk. We did not limit the memory used be Cassandra because the load process is already slow and easily interrupted.

• Multiple clients: Another issue is that we use multiple processes to load data into database. As we mentioned, there will be some exceptions caused by GC. Then the process will die because of the exception. For example, we started 16 clients for 80 warehouses but it ended up with only 5 clients. This will greatly influence the speed of load.

• Exceptions: Due to the exceptions in load process, some data is missing and some transactions may encounter notFound exeptions.

• Bottleneck: STOCK_LEVEL is a join operation in RDBMS and it is the bottleneck of our implementation. We first use some constraints to filter data in two tables and then do a O(n2) comparison.

Future Work

• Memory Tuning
• STOCK_LEVEL Transaction Tuning
• notFoundException