Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate fake data for further analysis #2160

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

mamcx
Copy link
Contributor

@mamcx mamcx commented Jan 22, 2025

Description of Changes

Create a reducer in modules/benchmarks/src/synthetic.rs that executes loading of rows for several variations:

NOTE TO REVIEWER: Wanna confirm if the ones in this pr are enough or wanna more variations.

Examples:

❯ ./run_standalone_temp.sh

# In other terminal...

❯ cargo run --bin spacetimedb-cli publish quickstart -c -p modules/benchmarks

❯ cargo run --bin spacetimedb-cli  call quickstart load 'Large, Many, 100'

The input is a string with the following format:

`load_type`: [`Load`], `index_type`: [`Index`], `row_count`: `u32`

pub enum Load {
    Tiny,
    Small,
    Medium,
    Large,
}

pub enum Index {
    One,
    Many,
}

There is now a simple load tool in SpacetimeDB/crates/bench/load.py:

❯ python3 load.py -d quickstart -f 2 -s 10 -i "insert_bulk_small_rows 100" -l "queries 'small, inserts:10,query:10,deletes:10';"
Running load testing for database: quickstart
    Frequency: 2.0 calls/second, Duration: 10.0 seconds

Init reducers for database: quickstart ████████████████████ 1/1  insert_bulk_small_rows 100
Load reducers for database: quickstart █████████........... 9/20  queries 'small, inserts:10,query:10,deletes:10'

The -i 'initialize' / -l 'load' allows to call arbitrary reducers separated by ;

API and ABI breaking changes

None

Expected complexity level and risk

1

Testing

  • Run manually the new bin and inspect the generated .spacetime database

@mamcx mamcx added the release-any To be landed in any release window label Jan 22, 2025
@mamcx mamcx requested a review from jsdt January 22, 2025 21:02
@mamcx mamcx self-assigned this Jan 22, 2025
@mamcx mamcx force-pushed the mamcx/data-generate-fake branch 2 times, most recently from ca44456 to 880a918 Compare January 24, 2025 17:57
@mamcx mamcx force-pushed the mamcx/data-generate-fake branch from 880a918 to 334d1db Compare January 29, 2025 17:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-any To be landed in any release window
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant