-
Notifications
You must be signed in to change notification settings - Fork 152
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unable to Start Docker Daemon in Sysbox Container with NFS Mount #811
Comments
I've also attempted to use separate NFS shares like so:
Next, I tried to mount them separately:
However, this resulted in a container start failure with an error message:
Then I decided to use a different directory for docker, like
The container started successfully and the shares seemed fine. Yet, when I tried to pull an image inside a containre with docker pull ubuntu, I encountered this:
So, as you can see, I'm pretty much stuck right now :-)
|
I've tried once more to make things work, but unfortunately, I haven't been successful. I realized that I had incorrectly configured NFS sharing; the idmapping for NFS wasn't functioning as it should, and I was using an older kernel that didn't support "Overlayfs on ID-mapped mounts". I've corrected all of these issues, but I still can't run the command Here are the configurations on my NFS Server:
In addition to the above, I ran this command:
(Here, I've added cs-user and cs-root using the specified commands for easier reading. On the NFS client (Sysbox host) side, I have this setup:
The command
At this point, I'm close to giving up... |
I'm not sure if this will be helpful but try taking a look at this: |
Thanks @nhoefer2 That looks like it could really help. I’ll give it a try next week and let you know how it goes. By the way, I’m using the XFS file system, but I’m not sure if ACL was enabled or not. |
Unfortunately, it does not help me. I decided to give it another shot at attaching an NFS volume to a Docker container. Below, I’ve listed all the steps I took to reproduce the issue (there weren’t that many). Description:When attempting to mount NFS volumes on a Docker container, the Docker daemon fails to start inside the container. This issue occurs on a setup involving two hosts running Ubuntu 24.04, with one acting as an NFS server and the other as an NFS client. Environment:
Steps to Reproduce:
Error Logs:
Additional Behavior:When restarting the container multiple times,
Expected Behavior:Docker should start successfully inside the container, and NFS shares should be correctly mounted without permission issues or overlapping mounts. Actual Behavior:The Docker daemon fails to start due to permission issues on the NFS-mounted directory. Additionally, |
Hi @bushev, thanks for trying Sysbox, hope you find it useful. I suspect the problem you are having is that Sysbox uses shiftfs or ID-mapped-mounts on host directories mounted into the container, and I don't believe either of these mechanisms work on top of NFS mounts (unfortunately). For example, when you do
how does |
Hey Cesar, thanks for looking into that! I just tried what you suggested, and strangely enough, the previous error seems to have disappeared. I can now confirm that Docker is starting within the container. This might be related to the fact that I rebooted the servers several times and enabled ACL with different parameters afterward. I can’t fully explain why, but it started working, and it seems to be functional for now. However, when I attempted to pull an image, for example, for MySQL, I encountered an error at the end stating that it couldn’t create a symbolic link. I believe this might be due to a limitation related to NFS and how it’s mounted inside the Sysbox container, but this is clearly a separate issue. Hopefully, this will be the last problem preventing full NFS compatibility. |
Hi @bushev, that's progress, thanks. I don't know however what could be causing the latest error you see when the image gets pulled by Docker inside the Sysbox contaienr. Does it occur with other images? Say for example, does |
Hmm, no this doesn’t work either, but the error is somehow related to a symlink as before.
|
I suspect the issue you are facing is not so much related to Sysbox, as it is related to placing Now, as to why it fails, I don't know. But it's probably due to limitations on NFS. Figuring that out would require a deeper investigation. If I am incorrect and you believe the problem is specific to running Docker engine on Sysbox, then we can dig further to see why that is. But I don't see any indication of this, the problem appears to be related to NFS than anything else. |
Hi, the Docker daemon fails to start within a Sysbox container when the Docker directory is mounted from an NFS server. The issue appears to be related to permissions on the NFS-mounted directory.
On NFS Server
Where, 166537 is 165536 + 1001 (1001 is GUI of the
user
in a Docker container)On Container's Host
Inside a container
Container created with mount
Just to give you a little context, we're using NFS share to store Docker data from the container. This way, we can quickly start up our containers and have a shared storage system.
The text was updated successfully, but these errors were encountered: