Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rocksdb performance improvements #9080

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

gfukushima
Copy link
Contributor

@gfukushima gfukushima commented Feb 3, 2025

PR Description

This PR adds a couple params that are bringing the db to much healthier state. Iterators are performing much better now looking at the timestamps of logs (order of ms against minutes using the current config).
Example:

"@timestamp":"2025-02-03T22:52:44,526","level":"DEBUG","thread":"CombinedStorageChannel-0","class":"CombinedKvStoreDao","message":"Fetching earliest beacon block from db","throwable":""}
{"@timestamp":"2025-02-03T22:52:44,585","level":"DEBUG","thread":"CombinedStorageChannel-0","class":"CombinedKvStoreDao","message":"Earliest block slot in column: Optional[SignedBeaconBlockDeneb{message=BeaconBlockDeneb{slot=2503200, proposer_index=254597, parent_root=0x09071872d788a160ce5c4f7f01214157a719c2c1ccb80a0114300bc62c691568, state_root=0x44cb56c1b485427921866e3907665bca04ff5c1579af5c48bc742ef925ec4752, body=BeaconBlockBodyDeneb{randao_reveal=SszByteVector{0x97c9b4986e

Increasing the cache allowed to rocksdb hasn't shown any negative impacts on the memory consumption.
Log rotation also helps to keep the amount of log files we keep to a minimum reusing already created ones.

Fixed Issue(s)

Fixes #8939

Documentation

  • I thought about documentation and added the doc-change-required label to this PR if updates are required.

Changelog

  • I thought about adding a changelog entry, and added one if I deemed necessary.

@gfukushima gfukushima closed this Feb 3, 2025
@gfukushima gfukushima reopened this Feb 4, 2025
@gfukushima gfukushima marked this pull request as draft February 4, 2025 10:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

JVM crash when restarting teku (24.10.3) running rocksDB
1 participant