You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The module snakeoil.tar was created to basically clone python's tarfile logic while monkeypatching __slots__ into the clone; that slots usage knocked >33% of memory usage off for any tar interaction, which was an issue for dealing in binpkgs and merging. These days, upstream caught up to our work and now has the same sort of memory optimizations.
Within the pkgcore stack, the only user of this module is the binpkg repository in pkgcore, and the usages of that are primarily folks doing pquery --contents rather than pmerge -kK binpkg merging (pmerge isn't fully functional now afaik).
Our version of tar has a couple of extras wired in- support for our 'data source' interface that was used to abstract around local/remote/in memory files. That needs to be resolved before this can be deprecated and then removed.
My suggestion for the pathway here is thus:
rewrite pkgcore's usage of snakeoil.tar to use tarfile. Potentially split out any shims (or wrapper of tarfile) to handle the data source thing, assuming those components still live in snakeoil.
Shove a deprecation notice into the module scope of snakeoil.tar. As an aside, that deprecation notice should be wired so it throws an exception if this code isn't removed in the next major semver bump.
Remove the module at the next semver bump.
The text was updated successfully, but these errors were encountered:
The module snakeoil.tar was created to basically clone python's tarfile logic while monkeypatching
__slots__
into the clone; that slots usage knocked >33% of memory usage off for any tar interaction, which was an issue for dealing in binpkgs and merging. These days, upstream caught up to our work and now has the same sort of memory optimizations.Within the pkgcore stack, the only user of this module is the binpkg repository in pkgcore, and the usages of that are primarily folks doing
pquery --contents
rather thanpmerge -kK
binpkg merging (pmerge isn't fully functional now afaik).Our version of tar has a couple of extras wired in- support for our 'data source' interface that was used to abstract around local/remote/in memory files. That needs to be resolved before this can be deprecated and then removed.
My suggestion for the pathway here is thus:
The text was updated successfully, but these errors were encountered: