Linux/MacOS: 
cBench is a cross-platform client connected with the open Collective Knowledge portal to help researchers and practitioners reproduce SOTA ML and systems research and share results on live SOTA scoreobards: demo.
cBench is a part of the Collective Knowledge project (CK) and uses portable CK solutions describing how to download, build, benchmark and optimize applications across different hardware, software, models and data sets.
| As a host platform | As a target platform | |
|---|---|---|
| Generic Linux | ✓ | ✓ |
| Linux (Arm) | ✓ | ✓ |
| Raspberry Pi | ✓ | ✓ |
| MacOS | ✓ | ± |
| Windows | ✓ | ✓ |
| Android | ± | ✓ |
| iOS | TBD | TBD |
Install prerequisites:
sudo apt update
sudo apt install git wget libz-dev curl cmake
sudo apt install gcc g++ autoconf autogen libtool
sudo apt install libfreetype6-dev
sudo apt install python3.7-dev
sudo apt install -y libsm6 libxext6 libxrender-dev
Install cbrain:
python3 -m pip install cbrain
Initialize the CK solution for MLPerf:
cb init demo-obj-detection-coco-tf-cpu-benchmark-linux-portable-workflows
Participate in crowd-benchmarking:
cb benchmark demo-obj-detection-coco-tf-cpu-benchmark-linux-portable-workflows
See results on a public SOTA dashboard.
You can also use the stable Docker image to participate in crowd-benchmarking:
sudo docker run ctuning/cbench-obj-detection-coco-tf-cpu-benchmark-linux-portable-workflows /bin/bash -c "cb benchmark demo-obj-detection-coco-tf-cpu-benchmark-linux-portable-workflows"
You can also check all dependencies for this solution.
- This is an ongoing project - don't hesitate to contact us if you have any feedback and suggestions!
We would like to thank all CK partners for fruitful discussions and feedback!
Copyright 2020 cTuning foundation