Skip to content

crate/crate-benchmarks

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

CrateDB Benchmarks

A collection of CrateDB benchmarks based on spec and track files for use with cr8.

Benchmark results are used to compare performance between two different CrateDB versions.

Note

Some benchmarks are not covering every single case but are more intended to be compared against others to see differences when certain features are enabled/disabled.

Usage

To install, run:

$ python3.6 -m venv venv
$ venv/bin/python -m pip install -r requirements.txt

To run all benchmarks, do:

$ venv/bin/cr8 run-track tracks/latest.toml [ -r result-host ]

To run a single benchmark (spec):

$ venv/bin/cr8 run-spec specs/specfile.toml localhost:4200

For more information on the parameters run:

$ venv/bin/cr8 run-spec -h

Note

venv is a Python module that provides support for creating lightweight virtual environments.

Note

Running an individual spec requires that you have a running CrateDB instance and that you pass its host:port as the last parameter.

Scripts

Scripts to simplify common tasks:

Writing Benchmarks

Since executing the benchmarks can take up quite some time we value quality over quantity here.

Benchmarks should be written so that they represent common execution paths or common use cases that differ a lot from other cases.

When writing new benchmarks it's also advisable to run compare_run.py once, where --v1 == --v2 to get a feeling of the stability of the benchmark. If there is a large difference, the benchmark should be tuned as it would be too unreliable to spot real differences.

Help

Looking for more help?