Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

db: fine-grained profiling #3957

Open
RaduBerinde opened this issue Sep 24, 2024 · 0 comments
Open

db: fine-grained profiling #3957

RaduBerinde opened this issue Sep 24, 2024 · 0 comments
Labels

Comments

@RaduBerinde
Copy link
Member

RaduBerinde commented Sep 24, 2024

This issue tracks implementing Pebble infrastructure for obtaining data on resource utilization (like CPU, disk bandwidth) broken down on various axes:

  • CRDB key type (lock, txn, intent, table)
  • read path / write path
  • compression / decompression
  • flushes / compaction (by target level)

Possibilities for tracked resources:

  • CPU usage
  • disk write bandwidth, iops
  • disk read bandwidth, iops
  • bloom filter negatives, false positives

The data should allow easily answering questions like "in this workload, how much CPU usage was caused by compactions of intent keys?"

We can implement this using pprof labels. For resources other than CPU we can maintain the statistics ourselves, as a map from label subsets. We could output them as pprof files or some other format.

This infrastructure can be behind a build flag to avoid adding overhead.

Jira issue: PEBBLE-260

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Backlog
Development

No branches or pull requests

1 participant