Skip to content

Commit 81c2ac0

Browse files
committed
[df] Change default Snapshot compression settings
The default Snapshot compression setting has always been 101. Historically, this was done for simplicity reasons and following the principle of least surprise. TTree was the only output format available with Snapshot, so the operation was defaulting to the same value used by TTree. Now that Snapshot supports more than one output format, this reason is less strong than before. It has been established that 505 is a better default compression setting overall, so RDataFrame should follow that. The main disadvantage is that this change is hard to communicate. This commit proposes to introduce an information message being shown once per program execution, and only if the program is calling Snapshot. This message is supposed to help the users detect changes in their output file size with the next development cycle (6.38.x) and should be removed afterwards.
1 parent 4157d78 commit 81c2ac0

File tree

3 files changed

+12
-2
lines changed

3 files changed

+12
-2
lines changed

README/ReleaseNotes/v638/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,7 @@ If you want to keep using `TList*` return values, you can write a small adapter
116116
to numbers such as 8 would share one 3-d histogram among 8 threads, greatly reducing the memory consumption. This might slow down execution if the histograms
117117
are filled at very high rates. Use lower number in this case.
118118
- The Snapshot method has been refactored so that it does not need anymore compile-time information (i.e. either template arguments or JIT-ting) to know the input column types. This means that any Snapshot call that specifies the template arguments, e.g. `Snapshot<int, float>(..., {"intCol", "floatCol"})` is now redundant and the template arguments can safely be removed from the call. At the same time, Snapshot does not need to JIT compile the column types, practically giving huge speedups depending on the number of columns that need to be written to disk. In certain cases (e.g. when writing O(10000) columns) the speedup can be larger than an order of magnitude. The Snapshot template is now deprecated and it will issue a compile-time warning when called. The function overload is scheduled for removal in ROOT 6.40.
119+
- The default compression setting for the output dataset used by Snapshot has been changed from 101 (ZLIB level 1) to 505 (ZSTD level 5). This is a better setting on average, and makes more sense for RDataFrame since now the Snapshot operation supports more than just the TTree output data format. This change may result in smaller output file sizes for your analyses that use Snapshot with default settings.
119120

120121
## Python Interface
121122

tree/dataframe/inc/ROOT/RDF/RInterface.hxx

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@
3232
#include "ROOT/RResultPtr.hxx"
3333
#include "ROOT/RSnapshotOptions.hxx"
3434
#include <string_view>
35+
#include "ROOT/RLogger.hxx"
3536
#include "ROOT/RVec.hxx"
3637
#include "ROOT/TypeTraits.hxx"
3738
#include "RtypesCore.h" // for ULong64_t
@@ -60,6 +61,7 @@
6061
#include <utility> // std::index_sequence
6162
#include <vector>
6263
#include <any>
64+
#include <mutex>
6365

6466
class TGraph;
6567

@@ -1331,6 +1333,13 @@ public:
13311333
const ColumnNames_t &columnList,
13321334
const RSnapshotOptions &options = RSnapshotOptions())
13331335
{
1336+
// TODO: Remove before releasing 6.40.00
1337+
static std::once_flag once;
1338+
std::call_once(once, []() {
1339+
R__LOG_INFO(ROOT::Detail::RDF::RDFLogChannel())
1340+
<< "The default compression settings of Snapshot has been changed from 101 (ZLIB with compression level 1) "
1341+
"to 505 (ZSTD with compression level 5).";
1342+
});
13341343
// like columnList but with `#var` columns removed
13351344
auto colListNoPoundSizes = RDFInternal::FilterArraySizeColNames(columnList, "Snapshot");
13361345
// like columnListWithoutSizeColumns but with aliases resolved

tree/dataframe/inc/ROOT/RSnapshotOptions.hxx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -46,8 +46,8 @@ struct RSnapshotOptions {
4646
}
4747
std::string fMode = "RECREATE"; ///< Mode of creation of output file
4848
ECAlgo fCompressionAlgorithm =
49-
ROOT::RCompressionSetting::EAlgorithm::kZLIB; ///< Compression algorithm of output file
50-
int fCompressionLevel = 1; ///< Compression level of output file
49+
ROOT::RCompressionSetting::EAlgorithm::kZSTD; ///< Compression algorithm of output file
50+
int fCompressionLevel = 5; ///< Compression level of output file
5151
int fAutoFlush = 0; ///< AutoFlush value for output tree
5252
int fSplitLevel = 99; ///< Split level of output tree
5353
bool fLazy = false; ///< Do not start the event loop when Snapshot is called

0 commit comments

Comments
 (0)