RocksDB Performance
RocksDB is an open-source embeddable key-value store. RocksDB is based on an LSM (log-structured merge-tree) architecture, and it stores data in SST (Static Sorted Table) files. SST files support compression of data in blocks of configurable size. Several compression algorithms are supported, including zlib and zstd. RocksDB also provides a benchmarking tool called db_bench with many configurable options.
With zlib-accel, it is possible to enable hardware compression acceleration without any code changes in RocksDB. It is sufficient to preload the zlib-accel library and select zlib as compression type. For more information on Intel IAA and QAT accelerators, please refer to the README page.
We measured RocksDB performance with db_bench using compression hardware acceleration provided by zlib-accel and compared it to different software compression algorithms (zlib, zstd). For zlib, we also tested zlib-ng, a zlib-compatible library with several optimizations to leverage modern hardware features.
System/Software
- AWS EC2 instance: m7i.metal-24xl
- EBS: gp3, 1GB/s, 6000 IOPS
- OS: Ubuntu 24.04.2, kernel 6.8.0
- RocksDB v9.11.0
Compression libraries/compression levels (default levels used for all libraries)
- zlib v1.2.11 (level: 6)
- zlib-ng v2.2.4 (level: 6)
- zstd v1.5.5 (level: 3)
- QATzip v1.3.0 (for QAT acceleration, level: 1)
- QPL v1.7.0 (for IAA acceleration, level: default)
zlib-accel v1.0.0 built with logging disabled and Release build type
cmake -DUSE_IAA=ON -DQPL_PATH=<path> -DUSE_QAT=ON -DDEBUG_LOG=OFF -DCMAKE_BUILD_TYPE=Release ..
zlib-accel configuration for QAT runs
use_qat_compress = 1
use_qat_uncompress = 1
use_iaa_compress = 0
use_iaa_uncompress = 0
use_zlib_compress = 1
use_zlib_uncompress = 1
iaa_prepend_empty_block = 1
qat_periodical_polling = 0
qat_compression_level = 1
zlib-accel configuration for IAA runs
use_qat_compress = 0
use_qat_uncompress = 0
use_iaa_compress = 1
use_iaa_uncompress = 1
use_zlib_compress = 1
use_zlib_uncompress = 1
iaa_prepend_empty_block = 1
qat_periodical_polling = 0
qat_compression_level = 1
db_bench configuration
- Workload: readrandomwriterandom (80% reads, 20% writes)
- Block size: 4kB/16kB blocks
- Key size: 16B keys
- Value size: 32B/256B values
- Block cache disabled
- Data populated from Calgary corpus (refer to RocksDB PR 10395)
To select zlib/zstd compression
db_bench --compression_type=zlib <other_options>
db_bench --compression_type=zstd <other_options>
To run with zlib-ng, preload the zlib-ng library and select zlib compression
LD_PRELOAD=<zlib-ng path>
db_bench --compression_type=zlib <other_options>
unset LD_PRELOAD
To run with zlib-accel, preload the zlib-accel library and select zlib compression
LD_PRELOAD=<zlib-accel path>
db_bench --compression_type=zlib <other_options>
unset LD_PRELOAD
The plot below shows RocksDB throughput (ops/s) with different deflate compression techniques: zlib and zlib-ng for software compression, IAA and QAT for hardware-accelerated compression. For each block/value size condition, the data is normalized to the throughput achieved by zlib for that condition.
Compared to zlib-ng (faster option for software deflate)
- QAT provides 23%-47% throughput gain, 1%-24% p99 get latency reduction, with a compressed data size tradeoff of 1%-2%
- IAA provides 92%-205% throughput gain, 53%-74% p99 get latency reduction, with a compressed data size tradeoff of 5%-9%
The ranges are over the different block size/value size conditions tested.
If an application can use compression other than deflate, zstd is a compelling option. Compared to zstd, IAA provides 34%-95% throughput gain, 23%-51% p99 get latency reduction, with a compressed data size tradeoff of 2%-9%
Testing performed by Intel in May 2025
Performance varies by use, configuration and other factors. Learn more on the Performance Index site.
Performance results are based on testing as of dates shown in configurations and may not reflect all publicly available updates. See backup for configuration details. No product or component can be absolutely secure.
Your costs and results may vary.
Intel technologies may require enabled hardware, software or service activation.
© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.