Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

diff-index: integrate with the sparse index #1777

Closed

Conversation

derrickstolee
Copy link

This integration is motivated by an internal GUI tool using the 'git diff-index' builtin. It is hitting the index expansion during every execution, contributing significant performance issues. This was missed in previous estimates of Git builtin usages because the numbers changed as users started using this tool.

Thanks, - Stolee

cc: [email protected]
cc: [email protected]

The sparse index allows focusing the index data structure on the files
present in the sparse-checkout, leaving only tree entries for
directories not within the sparse-checkout. Each builtin needs a
repository setting to indicate that it has been tested with the sparse
index before Git will allow the index to be loaded into memory in its
sparse form. This is a safety precaution.

There are still some builtins that haven't been integrated due to the
complexity of the integration and the lack of significant use. However,
'git diff-index' was neglected only because of initial data showing low
usage. The diff machinery was already integrated and there is no more
work to be done there but add some tests to be sure 'git diff-index'
behaves as expected.

For this purpose, we can follow the testing pattern used in 51ba65b
(diff: enable and test the sparse index, 2021-12-06). One difference
here is that we only verify that the sparse index case agrees with the
full index case, but do not generate the expected output. The 'git diff'
tests use the '--name-status' option to ease the creation of the
expected output, but that's not an option for 'diff-index'. Since the
underlying diff machinery is the same, a simple comparison is sufficient
to give some coverage.

Signed-off-by: Derrick Stolee <[email protected]>
@derrickstolee
Copy link
Author

/submit

Copy link

gitgitgadget bot commented Aug 22, 2024

Submitted as [email protected]

To fetch this version into FETCH_HEAD:

git fetch https://github.com/gitgitgadget/git/ pr-1777/derrickstolee/diff-index-sparse-v1

To fetch this version to local tag pr-1777/derrickstolee/diff-index-sparse-v1:

git fetch --no-tags https://github.com/gitgitgadget/git/ tag pr-1777/derrickstolee/diff-index-sparse-v1

Copy link

gitgitgadget bot commented Aug 22, 2024

This patch series was integrated into seen via git@d0a0702.

@gitgitgadget gitgitgadget bot added the seen label Aug 22, 2024
Copy link

gitgitgadget bot commented Aug 23, 2024

This branch is now known as ds/sparse-diff-index.

Copy link

gitgitgadget bot commented Aug 23, 2024

This patch series was integrated into seen via git@1ea2c8d.

Copy link

gitgitgadget bot commented Aug 23, 2024

This patch series was integrated into next via git@d33f670.

@gitgitgadget gitgitgadget bot added the next label Aug 23, 2024
Copy link

gitgitgadget bot commented Aug 23, 2024

There was a status update in the "New Topics" section about the branch ds/sparse-diff-index on the Git mailing list:

The underlying machinery for "git diff-index" has long been made to
expand the sparse index as needed, but the command fully expanded
the sparse index upfront, which now has been taught not to do.

Will merge to 'master'.
source: <[email protected]>

Copy link

gitgitgadget bot commented Aug 25, 2024

This patch series was integrated into seen via git@3a8187c.

Copy link

gitgitgadget bot commented Aug 26, 2024

This patch series was integrated into seen via git@ee7db35.

Copy link

gitgitgadget bot commented Aug 27, 2024

This patch series was integrated into seen via git@0a59cd7.

Copy link

gitgitgadget bot commented Aug 27, 2024

There was a status update in the "Cooking" section about the branch ds/sparse-diff-index on the Git mailing list:

The underlying machinery for "git diff-index" has long been made to
expand the sparse index as needed, but the command fully expanded
the sparse index upfront, which now has been taught not to do.

Will merge to 'master'.
source: <[email protected]>

Copy link

gitgitgadget bot commented Aug 28, 2024

This patch series was integrated into seen via git@a37a4dc.

Copy link

gitgitgadget bot commented Aug 29, 2024

This patch series was integrated into seen via git@11fd53a.

Copy link

gitgitgadget bot commented Aug 29, 2024

This patch series was integrated into master via git@11fd53a.

Copy link

gitgitgadget bot commented Aug 29, 2024

This patch series was integrated into next via git@11fd53a.

@gitgitgadget gitgitgadget bot added the master label Aug 29, 2024
@gitgitgadget gitgitgadget bot closed this Aug 29, 2024
Copy link

gitgitgadget bot commented Aug 29, 2024

Closed via 11fd53a.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant