Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase maxkeys and maxbytes in Known Issues #3806

Open
antoinetran opened this issue Dec 4, 2024 · 1 comment
Open

Increase maxkeys and maxbytes in Known Issues #3806

antoinetran opened this issue Dec 4, 2024 · 1 comment
Labels
kind/documentation Categorizes issue or PR as related to documentation.

Comments

@antoinetran
Copy link

antoinetran commented Dec 4, 2024

What would you like to be documented:
In https://kind.sigs.k8s.io/docs/user/known-issues/, to add this frequent error:

could not create session key: disk quota exceeded

And the fix:

sudo sysctl -w kernel.keys.maxkeys=20000
sudo sysctl -w kernel.keys.maxbytes=500000
cat <<EOF | sudo tee /etc/sysctl.d/01-keys.conf
# See https://github.com/moby/moby/issues/22865
# Default maxkeys: 200
kernel.keys.maxkeys = 20000
# Default maxbytes: 20000
kernel.keys.maxbytes = 500000
EOF

Why is this needed:
The default settings are not enough in the context of Kind. If the container runtime is podman, then the rootless user that runs kind cannot have more than 200 keys by default (in Debian/redhat family). Because:

We reach the limit very fast, with lots of empty keyring, that can be listed with:

cat /proc/keys

See moby/moby#22865 that is docker side of the fix, they increase root max keys and bytes to 1m and 25m. I suggest a bit less in the above code.

@antoinetran antoinetran added the kind/documentation Categorizes issue or PR as related to documentation. label Dec 4, 2024
@antoinetran
Copy link
Author

Non working workaround:
I tested disabling creating keyring in podman:

cat <<EOF >~/.config/containers/containers.conf
[containers]
keyring=false
EOF

This indeed does not increase keys when creating kind-worker and control-plane, however the runc inside creates lots of keyring.

On the runc side, I tried disabling new keyring:

cat /etc/containerd/config.toml
  [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
    NoNewKeyring = true
systemctl restart containerd.service

This does not seem to have any effect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/documentation Categorizes issue or PR as related to documentation.
Projects
None yet
Development

No branches or pull requests

1 participant