Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache revision tree bytes for history call #1097

Merged
merged 4 commits into from
Feb 13, 2025

Conversation

minwoox
Copy link
Contributor

@minwoox minwoox commented Feb 11, 2025

Motivation:
The history call is resource-intensive, consuming significant CPU and memory when it cannot locate the requested content in a single call.
By caching revision trees, we can optimize resource usage.

Modifications:

  • Introduced CachingTreeObjectReader to cache revision trees and reduce redundant computations.

Result:

  • Improved CPU and memory efficiency for history calls.

@minwoox minwoox added this to the 0.74.0 milestone Feb 11, 2025
@minwoox minwoox changed the title [WIP]Cache revision tree bytes for history call Cache revision tree bytes for history call Feb 11, 2025
@minwoox
Copy link
Contributor Author

minwoox commented Feb 11, 2025

Head
Benchmark                                    (noCommits)  (noFiles)   Mode  Cnt   Score   Error  Units
GitRepositoryHistoryBenchmark.history                100          1  thrpt    5  41.574 ± 1.061  ops/s
GitRepositoryHistoryBenchmark.history                100          3  thrpt    5  14.277 ± 0.949  ops/s
GitRepositoryHistoryBenchmark.history                100          5  thrpt    5   8.725 ± 0.264  ops/s
GitRepositoryHistoryBenchmark.history                100         10  thrpt    5   4.311 ± 0.409  ops/s
GitRepositoryHistoryBenchmark.history                100         30  thrpt    5   1.623 ± 0.028  ops/s

GitRepositoryHistoryBenchmark.history               1000          1  thrpt    5   2.149 ± 0.218  ops/s
GitRepositoryHistoryBenchmark.history               1000          3  thrpt    5   0.713 ± 0.039  ops/s
GitRepositoryHistoryBenchmark.history               1000          5  thrpt    5   0.440 ± 0.013  ops/s
GitRepositoryHistoryBenchmark.history               1000         10  thrpt    5   0.220 ± 0.003  ops/s
GitRepositoryHistoryBenchmark.history               1000         30  thrpt    5   0.065 ± 0.009  ops/s

Improved
Benchmark                                    (noCommits)  (noFiles)   Mode  Cnt   Score   Error  Units
GitRepositoryHistoryBenchmark.history                100          1  thrpt    5  65.059 ± 2.150  ops/s
GitRepositoryHistoryBenchmark.history                100          3  thrpt    5  39.230 ± 1.945  ops/s
GitRepositoryHistoryBenchmark.history                100          5  thrpt    5  28.108 ± 1.357  ops/s
GitRepositoryHistoryBenchmark.history                100         10  thrpt    5  17.076 ± 0.236  ops/s
GitRepositoryHistoryBenchmark.history                100         30  thrpt    5   7.062 ± 0.365  ops/s

GitRepositoryHistoryBenchmark.history               1000          1  thrpt    5   3.108 ± 0.147  ops/s
GitRepositoryHistoryBenchmark.history               1000          3  thrpt    5   1.893 ± 0.027  ops/s
GitRepositoryHistoryBenchmark.history               1000          5  thrpt    5   1.350 ± 0.056  ops/s
GitRepositoryHistoryBenchmark.history               1000         10  thrpt    5   0.793 ± 0.013  ops/s
GitRepositoryHistoryBenchmark.history               1000         30  thrpt    5   0.374 ± 0.004  ops/s

try {
return delegate.open(objectId0, typeHint);
} catch (IOException e) {
throw new RuntimeException("failed to open an object: " + objectId0, e);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Suggested change
throw new RuntimeException("failed to open an object: " + objectId0, e);
throw new UncheckedIOException("failed to open an object: " + objectId0, e);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. 😉

Comment on lines 37 to 38
private static final Logger logger = LoggerFactory.getLogger(CachingTreeObjectReader.class);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: unused.

Suggested change
private static final Logger logger = LoggerFactory.getLogger(CachingTreeObjectReader.class);

Copy link
Contributor

@ikhoon ikhoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍👍

Copy link

codecov bot commented Feb 11, 2025

Codecov Report

Attention: Patch coverage is 74.00000% with 13 lines in your changes missing coverage. Please review.

Project coverage is 70.07%. Comparing base (f4dda40) to head (10cf0f4).
Report is 7 commits behind head on main.

Files with missing lines Patch % Lines
...r/internal/storage/repository/RepositoryCache.java 72.22% 3 Missing and 2 partials ⚠️
...rage/repository/git/CacheableObjectLoaderCall.java 61.53% 4 Missing and 1 partial ⚠️
...torage/repository/git/CachingTreeObjectReader.java 85.71% 2 Missing ⚠️
...internal/storage/repository/git/GitRepository.java 80.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1097      +/-   ##
============================================
- Coverage     70.29%   70.07%   -0.22%     
- Complexity     4284     4388     +104     
============================================
  Files           427      443      +16     
  Lines         17258    17700     +442     
  Branches       1915     1946      +31     
============================================
+ Hits          12131    12404     +273     
- Misses         4077     4245     +168     
- Partials       1050     1051       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@minwoox
Copy link
Contributor Author

minwoox commented Feb 13, 2025

Thanks for reviewing. 😉

@minwoox minwoox merged commit 47723bc into line:main Feb 13, 2025
11 of 12 checks passed
@minwoox minwoox deleted the blockingHistoryCache branch February 13, 2025 03:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants