Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

db: investigate optimizations for special keys (locks, intents, txn records) #3958

Open
RaduBerinde opened this issue Sep 24, 2024 · 0 comments

Comments

@RaduBerinde
Copy link
Member

RaduBerinde commented Sep 24, 2024

In addition to keys containing SQL table data, CRDB transactions read and write other internal keys like locks, intents, txn records. These keys have very special access patterns and can likely benefit from targeted optimizations. They also take up a small fraction of the storage space so we can trade off space for efficiency.

Some ideas of what tweaks we could explore for these keys:

  • larger bloom filters, use of bloom filters on L6
  • fewer LSM levels
  • smaller sstables
  • reduced compression

We would need #3957 to be able to break down the work we do for these keys in various workloads.

Long term, it would be ideal if Pebble detected special patterns automatically. But in the short-to-medium term, we can explicitly configure Pebble with specific hints for parts of the keyspace.

Jira issue: PEBBLE-261

@github-project-automation github-project-automation bot moved this to Incoming in Storage Sep 24, 2024
@nicktrav nicktrav moved this from Incoming to Backlog in Storage Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Backlog
Development

No branches or pull requests

1 participant