Can not use conda command as non root user.

[cpaulik]

Running conda as another user than root does not seem to work anymore with the latest images.
This used to work fine with older images

The dockerfile:

FROM continuumio/miniconda:latest
RUN apt-get update && \
    apt-get -y install sudo curl
RUN groupadd -g 1002 ubuntu && \
    useradd -r -m -u 1001 -g ubuntu -G sudo ubuntu
RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
USER ubuntu
RUN mkdir /home/ubuntu/code
WORKDIR /home/ubuntu/code
RUN conda install pip

give the following error when running docker build -f Dockerfile .

Step 1/8 : FROM continuumio/miniconda:latest
 ---> f3ca74752199
Step 2/8 : RUN apt-get update &&     apt-get -y install sudo curl
 ---> Using cache
 ---> 64d5f3d00441
Step 3/8 : RUN groupadd -g 1002 ubuntu &&     useradd -r -m -u 1001 -g ubuntu -G sudo ubuntu
 ---> Using cache
 ---> 6ef6acdf3bd2
Step 4/8 : RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
 ---> Using cache
 ---> cd7e2bc0a95e
Step 5/8 : USER ubuntu
 ---> Using cache
 ---> 331927bb4936
Step 6/8 : RUN mkdir /home/ubuntu/code
 ---> Using cache
 ---> f242c23d26dd
Step 7/8 : WORKDIR /home/ubuntu/code
 ---> Using cache
 ---> 7813f2656416
Step 8/8 : RUN conda install pip
 ---> Running in 9a8d14e3ef09
Collecting package metadata (current_repodata.json): ...working... failed

NotWritableError: The current user does not have write permissions to a required path.
  path: /opt/conda/pkgs/urls.txt
  uid: 1001
  gid: 1002

If you feel that permissions on this path are set incorrectly, you can manually
change them by executing

  $ sudo chown 1001:1002 /opt/conda/pkgs/urls.txt

In general, it's not advisable to use 'sudo conda'.


The command '/bin/sh -c conda install pip' returned a non-zero code: 1

[cpaulik]

Also environments are now stored in /opt/conda/envs which is not writeable by another user by default.

[msarahan]

I think the problem is that the package cache writability check succeeds for one file, but not for all files. See https://github.com/conda/conda/blob/master/conda/core/package_cache_data.py#L147-L165

The default pkgs_dirs setting is here: https://github.com/conda/conda/blob/master/conda/base/context.py#L461-L473

It appears to me that the real problem is that the fallback to ~/.conda is failing. I don't know why. Maybe you need to check and set permissions on /home/ubuntu?

[msarahan]

Scratch that. I'm pretty sure it's this: conda/conda#9215

It'll be fixed in conda 4.7.12.

[anupam128]

What are the impacted versions? @cpaulik was using latest tag.

[jsolbrig]

This might be a dumb question, but what is the purpose of forcing a non-root user? I'm sure there's a good reason that I'm missing, but it seems odd to me.

I don't expect my base images to be hardened and I use miniconda3 as a base image where I build other stuff. Being forced to use a non-root user makes that more difficult since I need to use a multi-stage build and copy from the miniconda3 container rather than just using the miniconda3 container directly.

[sirviot]

Best practices, safety first, beter safe than sorry, etc.

If a service can run without privileges, use USER to change to a non-root user
From: Best practices for writing Dockerfiles

EDIT: But yeah. It's a bit of a pain with multistage builds.

[cpaulik]

Our reason is that the containers mount a NFS. And since they all read and write there it is handy to have the same UID, GID.

There are probably better ways to do this but this is what we are using at the moment.

[jsolbrig]

Maybe an intermediate target could be added and built as a separate image before the user gets switched to root? I this this may force me to maintain my own conda image in the future otherwise, which is something I'd prefer not to do since it is generally more secure and faster to use official images.

[guidorice]

I have been trying older docker images, to work around this, and this issue seems to have cropped up sometime after tag 4.6.14. Here is a working Dockerfile in case it's helpful for anyone:

FROM continuumio/miniconda3:4.6.14

ARG username=dev
ARG condaenv=env

RUN useradd --create-home --shell /bin/bash ${username}
WORKDIR /home/${username}
USER ${username}

RUN conda create -y --name ${condaenv} python=3.7
RUN echo "source activate ${condaenv}" > ~/.bashrc

[westurner]

This seems to work with 4.7.12; though I'm not sure that it's all necessary?

FROM continuumio/miniconda3:4.7.12
# ...

RUN adduser --disabled-password --gecos "Default user" --uid 1000 cq

RUN mkdir /opt/conda/envs/cq /opt/conda/pkgs && \
    chgrp cq /opt/conda/pkgs && \
    chmod g+w /opt/conda/pkgs && \
    touch /opt/conda/pkgs/urls.txt && \
    chown cq /opt/conda/envs/cq /opt/conda/pkgs/urls.txt

COPY --chown=cq:cq environment.yml labextensions.txt /opt/conda/envs/cq/
WORKDIR /home/cq
USER cq
RUN conda env create -f /opt/conda/envs/cq/environment.yml -n cq && \
    conda run -n cq conda install -y nodejs

(This is adapted from https://github.com/bernhard-42/jupyter-cadquery/blob/master/docker/Dockerfile )

[westurner]

... Allowing the 'cq' user / group to modify /opt/conda/pkgs/, /opt/conda/pkgs/urls.txt, and /opt/conda/envs/cq/ is probably acceptable for this single user docker image (where, in most cases, conda will not be run beyond during the docker image build)

[nok]

JFYI: For me @guidorice solution with the older version continuumio/miniconda3:4.6.14 solves my issues with user permissions. In my case I use Docker images in Travis-CI and Binder.

[Peque]

@guidorice I did the same: stick to 4.6.14. 👍

4.7.12 does not work out-of-the-box. You need to chown /opt/conda, as @westurner suggests.

[embray]

I had this same issue and it smells like a bit of a bug (or maybe this is intentional for some reason I'm not aware of). I'm running

$ conda --version
conda 4.8.2

If I create a Dockerfile containing

RUN adduser --uid 1001 --disabled-password --gecos '' myuser
USER myuser

then inside the container I run:

$ conda config --show

it shows:

envs_dirs:
  - /home/dnadna/.conda/envs
  - /opt/conda/envs
...
pkgs_dirs:
  - /opt/conda/pkgs
  - /home/dnadna/.conda/pkgs
...

so although the user's local envs_dir takes precedence over /opt/conda/envs, the converse is true for the user's local pkgs_dir, hence the problem discussed in this issue. Some suggested running chown -R <user>:<group> /opt/conda but I don't think this is necessary. I was able to work around it simply by adding the following to my Dockerfile (after creating and switching to the user)

RUN conda config --prepend pkgs_dirs /home/myuser/.conda/pkgs

[westurner]

How is this similar or different to "CWE-427: Uncontrolled Search Path Element"
https://cwe.mitre.org/data/definitions/427.html and/or "CWE-426: Untrusted Search Path" https://cwe.mitre.org/data/definitions/426.html ; given that that's how e.g. Python site.ENABLE_USER_SITE works https://docs.python.org/3/library/site.html#site.ENABLE_USER_SITE : the user-writable packages preempt the root-written packages?

[lightmare]

This is still an issue as of miniconda 4.9.2 (currently latest tag on docker hub):

docker run -it --rm continuumio/miniconda3 sh 

# useradd -s /bin/bash -m ana
# su ana
ana@def4a70e5820:/$ conda info

     active environment : None
       user config file : /home/ana/.condarc
 populated config files : 
          conda version : 4.9.2
    conda-build version : not installed
         python version : 3.8.5.final.0
       virtual packages : __glibc=2.28=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /opt/conda  (read only)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /opt/conda/pkgs
                          /home/ana/.conda/pkgs
       envs directories : /home/ana/.conda/envs
                          /opt/conda/envs
               platform : linux-64
             user-agent : conda/4.9.2 requests/2.24.0 CPython/3.8.5 Linux/5.4.0-72-generic debian/10 glibc/2.28
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

ana@def4a70e5820:/$ conda create -n foo
Collecting package metadata (current_repodata.json): done
Solving environment: done

NotWritableError: The current user does not have write permissions to a required path.
  path: /opt/conda/pkgs/urls.txt
  uid: 1000
  gid: 1000

(manually creating the directory /home/ana/.conda/pkgs does not help, it still wants to write to /opt/conda/pkgs)

Upgrading to 4.10.1 appears to fix this issue:

docker run -it --rm continuumio/miniconda3 sh 

# conda update conda
...
# useradd -s /bin/bash -m ana
# su ana
ana@60e08abf6ac0:/$ conda info

     active environment : None
       user config file : /home/ana/.condarc
 populated config files : 
          conda version : 4.10.1
    conda-build version : not installed
         python version : 3.8.5.final.0
       virtual packages : __linux=5.4.0=0
                          __glibc=2.28=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /opt/conda  (read only)
      conda av data dir : /opt/conda/etc/conda
  conda av metadata url : https://repo.anaconda.com/pkgs/main
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /opt/conda/pkgs
                          /home/ana/.conda/pkgs
       envs directories : /home/ana/.conda/envs
                          /opt/conda/envs
               platform : linux-64
             user-agent : conda/4.10.1 requests/2.25.1 CPython/3.8.5 Linux/5.4.0-72-generic debian/10 glibc/2.28
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

ana@60e08abf6ac0:/$ conda create -n foo
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/ana/.conda/envs/foo



Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate foo
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Will 4.10.1 image be published on docker hub any time soon?

[rooby]

This is still a problem with continuumio/anaconda3:2021.05

[rooby]

It would be good if this Docker image could do something like create a conda group with the correct permissions to the relevant conda directories, and then when using the image you just have to create a user and assign the conda group and it automatically gets the required permissions. (also adding documentation to tells users this)

[westurner]

Could that be in the entrypoint script with a file-based lock so that the initial configuration (maybe from documented env vars that there may already be conventions for) is idempotent?

…

On Tue, Jul 27, 2021, 19:39 Reuben Turk ***@***.***> wrote: It would be good if this Docker image could do something like create a conda group with the correct permissions to the relevant conda directories, and then when using the image you just have to create a user and assign the conda group and it automatically gets the required permissions. (also adding documentation to tells users this) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#151 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAMNSZBSASRJXTML4K53VTTZ47RPANCNFSM4IWGMQQQ> .

[lightmare]

Could that be in the entrypoint script

No, setting initial file permissions really belongs to the Dockerfile. The entry point script does not necessarily run as root. It can be the last USER set in the image (custom built from the base), or --user arg passed to docker container create, so it might not even have permissions to set permissions.

Today I figured out the problem is not permissions, but simply missing /opt/conda/pkgs/urls.txt and confusing error message.

docker run -it --rm continuumio/miniconda3:4.10.3 bash

In container:

useradd -s /bin/bash -m ana
su -c 'conda info' ana

     active environment : None
       user config file : /home/ana/.condarc
 populated config files : 
          conda version : 4.10.3
    conda-build version : not installed
         python version : 3.9.5.final.0
       virtual packages : __linux=5.4.0=0
                          __glibc=2.28=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /opt/conda  (read only)
      conda av data dir : /opt/conda/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /opt/conda/pkgs
                          /home/ana/.conda/pkgs
       envs directories : /home/ana/.conda/envs
                          /opt/conda/envs
               platform : linux-64
             user-agent : conda/4.10.3 requests/2.25.1 CPython/3.9.5 Linux/5.4.0-80-generic debian/10 glibc/2.28
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

This fails:

su -c 'conda create -n foo' ana

Collecting package metadata (current_repodata.json): done
Solving environment: done

NotWritableError: The current user does not have write permissions to a required path.
  path: /opt/conda/pkgs/urls.txt
  uid: 1000
  gid: 1000

But the problem is not that user ana does not have write permissions.
The problem is that the path does not exist.

mkdir /opt/conda/pkgs
touch /opt/conda/pkgs/urls.txt
su -c 'conda create -n foo' ana

Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/ana/.conda/envs/foo



Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate foo
#
# To deactivate an active environment, use
#
#     $ conda deactivate

[westurner]

No, setting initial file permissions really belongs to the Dockerfile

Yup, then end users would need to rebuild the container image in order to change the uid/username. If the entrypoint script is not running as uid 0, don't try to chmod/chcon?

[lightmare]

Yup, then end users would need to rebuild the container image in order to change the uid/username.

I don't understand. Change the uid of what?

I want conda to work out of the box for a regular user. There's no reason to give a regular user write access somewhere outside their home.

[westurner]

Which parts of conda can/should be owned by any of eg. these roles?

• root (0)
• sysadmin [can_conda_install_and_upgrade anything]
• app [can_execute_read_only_code_only]