BtrBlocks Benchmarking Repository

Overview

This repository provides benchmarking tools and results for evaluating the performance of BtrBlocks, an efficient columnar compression library designed for data lakes. The goal of this project is to analyze compression ratios and decompression speed.

Benchmarking Setup

To ensure accurate benchmarking results, this repository follows a structured approach:

1. Dataset Preparation

   • Uses public-bi datasets to test compression efficiency.

2. Benchmarking Metrics

   • Compression ratio (original size vs. compressed size).
   • Decompression speed (MB/s).

Repository Structure

• benchmarks/ - Scripts and configurations for running benchmarks.
• results/ - Stored benchmark results and reports.
• scripts/ - Helper scripts for data generation and automation.
• doc/ - Documentation and findings.

Building & Running Benchmarks

Prerequisites

Ensure you have the following dependencies installed:

• CMake (>=3.16)
• GCC or Clang with C++20 support
• Python 3.x (for result analysis)
• BtrBlocks (ensure it's installed or built from source)

Build the Benchmarking Suite

mkdir build
cd build
cmake ..
make

Running Benchmarks

Execute the benchmarking tool with:

./benchmarks/run_benchmarks

To compare results with Parquet and ORC:

./benchmarks/compare_with_parquet_orc

Results & Findings

Detailed results and analyses are available in the results/ directory. We provide breakdowns of compression ratios, speed, and efficiency across different data types and workloads.

Contributors

• Your Name
• Other Contributors

License

MIT - See License File.

---

** MODIFICATION **

• "-DCMAKE_CXX_FLAGS="-Wno-error=deprecated-declarations" is needed
• sudo apt install libtbb-dev