Merge functionality from github.com/moby/moby/pkg/idtools into user
#182
+708
−0
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The `pkg/idtools` package supports the creation of user(s) for
retrieving /etc/sub{u,g}id ranges and creation of the UID/GID mappings
provided to clone() to add support for user namespaces in Docker.
Docker-DCO-1.1-Signed-off-by: Phil Estes <[email protected]> (github: estesp)
Signed-off-by: John Howard <[email protected]>
This fixes errors in ownership on directory creation during build that can cause inaccessible files depending on the paths in the Dockerfile and non-existing directories in the starting image. Add tests for the mkdir variants in pkg/idtools Docker-DCO-1.1-Signed-off-by: Phil Estes <[email protected]> (github: estesp)
Since Docker is already skipping newlines in /etc/sub{uid,gid},
this patch skips commented out lines - otherwise Docker fails to start.
Add unit test also.
Signed-off-by: Antonio Murdaca <[email protected]>
Change user/group creation to use flags to adduser/useradd to enforce it being a system user. Use system user defaults that auto-create a matching group. These changes allow us to remove all group creation code, and in doing so we also removed the code that finds available uid, gid integers and use post-creation query to gather the system-generated uid and gid. The only added complexity is that today distros don't auto-create subordinate ID ranges for a new ID if it is a system ID, so we now need to handle finding a free range and then calling the `usermod` tool to add the ranges for that ID. Note that this requires the distro supports the `-v` and `-w` flags on `usermod` for subordinate ID range additions. Docker-DCO-1.1-Signed-off-by: Phil Estes <[email protected]> (github: estesp)
This should not have been in init() as it causes these lookups to happen in all reexecs of the Docker binary. The only time it needs to be resolved is when a user is added, which is extremely rare. Docker-DCO-1.1-Signed-off-by: Phil Estes <[email protected]>
Warn the user and fail daemon start if the graphdir path has any elements which will deny access to the remapped root uid/gid. Docker-DCO-1.1-Signed-off-by: Phil Estes <[email protected]>
When processing the --userns-remap flag, add the capability to call out to `getent` if the user and group information is not found via local file parsing code already in libcontainer/user. Signed-off-by: Phil Estes <[email protected]>
This remove a dependency on `go-check` (and more) when using `pkg/idtools`. `pkg/integration` should never be called from any other package then `integration`. Signed-off-by: Vincent Demeester <[email protected]>
Signed-off-by: Cristian Staretu <[email protected]>
Signed-off-by: Daniel Nephin <[email protected]>
Signed-off-by: Daniel Nephin <[email protected]>
Signed-off-by: Daniel Nephin <[email protected]>
Signed-off-by: Daniel Nephin <[email protected]>
Signed-off-by: Daniel Nephin <[email protected]>
There is no case which would resolve in this error. The root user always exists, and if the id maps are empty, the default value of 0 is correct. Signed-off-by: Daniel Nephin <[email protected]>
Signed-off-by: John Howard <[email protected]>
Switch some more usage of the Stat function and the Stat_t type from the syscall package to golang.org/x/sys. Those were missing in PR #33399. Signed-off-by: Tobias Klauser <[email protected]>
Signed-off-by: Danyal Khaliq <[email protected]>
In some cases (e.g. NFS), a chown may technically be a no-op but still return `EPERM`, so only call `chown` when neccessary. This is particularly problematic for docker users bind-mounting an NFS share into a container. Signed-off-by: Brian Goff <[email protected]>
idtools don't chown if not needed
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Standard golang's `os.MkdirAll()` function returns "not a directory" error in case a directory to be created already exists but is not a directory (e.g. a file). Our own `idtools.MkdirAs*()` functions do not replicate the behavior. This is a bug since all `Mkdir()`-like functions are expected to ensure the required directory exists and is indeed a directory, and return an error otherwise. As the code is using our in-house `system.Stat()` call returning a type which is incompatible with that of golang's `os.Stat()`, I had to amend the `system` package with `IsDir()`. A test case is also provided. Signed-off-by: Kir Kolyshkin <[email protected]>
This subtle bug keeps lurking in because error checking for `Mkdir()`
and `MkdirAll()` is slightly different wrt to `EEXIST`/`IsExist`:
- for `Mkdir()`, `IsExist` error should (usually) be ignored
(unless you want to make sure directory was not there before)
as it means "the destination directory was already there"
- for `MkdirAll()`, `IsExist` error should NEVER be ignored.
Mostly, this commit just removes ignoring the IsExist error, as it
should not be ignored.
Also, there are a couple of cases then IsExist is handled as
"directory already exist" which is wrong. As a result, some code
that never worked as intended is now removed.
NOTE that `idtools.MkdirAndChown()` behaves like `os.MkdirAll()`
rather than `os.Mkdir()` -- so its description is amended accordingly,
and its usage is handled as such (i.e. IsExist error is not ignored).
For more details, a quote from my runc commit 6f82d4b (July 2015):
TL;DR: check for IsExist(err) after a failed MkdirAll() is both
redundant and wrong -- so two reasons to remove it.
Quoting MkdirAll documentation:
> MkdirAll creates a directory named path, along with any necessary
> parents, and returns nil, or else returns an error. If path
> is already a directory, MkdirAll does nothing and returns nil.
This means two things:
1. If a directory to be created already exists, no error is
returned.
2. If the error returned is IsExist (EEXIST), it means there exists
a non-directory with the same name as MkdirAll need to use for
directory. Example: we want to MkdirAll("a/b"), but file "a"
(or "a/b") already exists, so MkdirAll fails.
The above is a theory, based on quoted documentation and my UNIX
knowledge.
3. In practice, though, current MkdirAll implementation [1] returns
ENOTDIR in most of cases described in moby#2, with the exception when
there is a race between MkdirAll and someone else creating the
last component of MkdirAll argument as a file. In this very case
MkdirAll() will indeed return EEXIST.
Because of moby#1, IsExist check after MkdirAll is not needed.
Because of moby#2 and moby#3, ignoring IsExist error is just plain wrong,
as directory we require is not created. It's cleaner to report
the error now.
Note this error is all over the tree, I guess due to copy-paste,
or trying to follow the same usage pattern as for Mkdir(),
or some not quite correct examples on the Internet.
[1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go
Signed-off-by: Kir Kolyshkin <[email protected]>
Files that are suffixed with `_linux.go` or `_windows.go` are already only built on Linux / Windows, so these build-tags were redundant. Signed-off-by: Sebastiaan van Stijn <[email protected]>
Remove dead code and redundant build tags
Signed-off-by: Daniel Nephin <[email protected]>
gty-migrate-from-testify --ignore-build-tags Signed-off-by: Daniel Nephin <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Merge the accessible() function into CanAccess, and check world- readable permissions first, before checking owner and group. Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
On unix, it's an alias for os.MkdirAll, so remove its use to be more transparent what's being used. Signed-off-by: Sebastiaan van Stijn <[email protected]>
Most of the package was using stdlib's errors package, so replacing two calls to pkg/errors with stdlib. Also fixing capitalization of error strings. Signed-off-by: Sebastiaan van Stijn <[email protected]>
Looks like we don't need the abstraction, so we can reduce the dependency on pkg/system. Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
The implementation of CanAccess() is very rudimentary, and should not be used for anything other than a basic check (and maybe not even for that). It's only used in a single location in the daemon, so move it there, and un-export it to not encourage others to use it out of context. Signed-off-by: Sebastiaan van Stijn <[email protected]>
The `execCmd()` utility was a basic wrapper around `exec.Command()`. Inlining it makes the code more transparent. Signed-off-by: Sebastiaan van Stijn <[email protected]>
Removed pre-go1.17 build-tags with go fix;
go mod init
go fix -mod=readonly ./...
rm go.mod
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Nicolas De Loof <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
When running a `docker cp` to copy files to/from a container, the lookup of the `getent` executable happens within the container's filesystem, so we cannot re-use the results. Unfortunately, that also means we can't preserve the results for any other uses of these functions, but probably the lookup should not be "too" costly. Signed-off-by: Sebastiaan van Stijn <[email protected]>
pkg/idtools: remove sync.Once, and include lookup error
The github.com/opencontainers/runc/libcontainer/user package was moved to a separate module. While there's still uses of the old module in our code-base, runc itself is migrating to the new module, and deprecated the old package (for runc 1.2). Signed-off-by: Sebastiaan van Stijn <[email protected]>
migrate to github.com/moby/sys/user
pkg/idtools/usergroupadd_linux.go:94:2: shadow: declaration of "ranges" shadows declaration at line 25 (govet)
ranges, err := parseSubuid(name)
^
pkg/idtools/usergroupadd_linux.go:131:2: shadow: declaration of "ranges" shadows declaration at line 25 (govet)
ranges, err := parseSubuid("ALL")
^
pkg/idtools/usergroupadd_linux.go:140:2: shadow: declaration of "ranges" shadows declaration at line 25 (govet)
ranges, err := parseSubgid("ALL")
^
Signed-off-by: Sebastiaan van Stijn <[email protected]>
pkg/idtools/idtools_unix_test.go:188:3: The copy of the 'for' variable "tc" can be deleted (Go 1.22+) (copyloopvar)
tc := tc
^
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
pkg/system: deprecate MkdirAll and remove custom volume GUID handling
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Separare idtools functionality that is used internally from the functionlality used by importers. The `pkg/idtools` package is now much smaller and more generic. Signed-off-by: Derek McGowan <[email protected]> Signed-off-by: Sebastiaan van Stijn <[email protected]>
Split idtools to an internal package and package to be moved
Signed-off-by: Derek McGowan <[email protected]>
dmcgowan
force-pushed
the
merge-idtools
branch
from
f875d71 to
56dd888
Compare
Removed duplicated structs Removed deprecated functionality Simplified calls which previously relied on unused functionality Signed-off-by: Derek McGowan <[email protected]>
dmcgowan
force-pushed
the
merge-idtools
branch
from
56dd888 to
d482c8a
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Merges the idtools package functionality from github.com/moby/moby/pkg/idtools into the user package. The package was previously trimmed down and is simplified further to better fit
moby/sys.Now the package interface will just expand with:
MkdirAllAndChownwithWithOnlyNewas aMkdirOptMkdirAndChownalso supporting the optsIdentityMappingits functionsRootPair,ToHost,ToContainer,EmptyNote the
Identitytype is gone sinceSIDwas not used in the package or by any of the functions. It is more common to just pass along the uid and gid.