Skip to content

Commit

Permalink
add docker bit
Browse files Browse the repository at this point in the history
  • Loading branch information
eeholmes committed Oct 4, 2023
1 parent 4084348 commit e4e43bb
Show file tree
Hide file tree
Showing 10 changed files with 252 additions and 102 deletions.
36 changes: 32 additions & 4 deletions Set-up-centos.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ ls -Z $JHUBENV/bin
```
It got all the binaries but not the simlinks. Nonetheless it seemed to run ok.

## Enable our new service
### Enable our new service

```{bash eval=FALSE}
sudo systemctl enable jupyterhub.service
Expand All @@ -294,7 +294,7 @@ netstat -tuln
```
At this point, you will need to address security if your hub is open to the web, as opposed to being on an internal network and only accessible to that network. Learn about that [here](https://jupyterhub.readthedocs.io/en/1.2.0/installation-guide-hard.html).

## Create the base user environment
## Set up Docker for user environment

When you log in the jupyter notebooks will be trying to use the Python environment that was created to install JupyterHub, this is not what we want. We will use a docker image to "spawn" the user environment. Read [here](https://jupyterhub.readthedocs.io/en/1.2.0/installation-guide-hard.html#part-2-conda-environments) for other approaches.

Expand Down Expand Up @@ -332,7 +332,7 @@ conda install -c conda-forge docker-py

### Jupyter images

The image that we use must have the jupyterhub module installed. Also there seems to be some tweaking needed for the image to work as the image I used in a JupyterHub on Kubernetes did not work for me.
The image that we use must have the jupyterhub and notebook module installed. The jupyterhub version needs to also match what you have on your hub.

Check the version on your server:
```{bash eval=FALSE}
Expand Down Expand Up @@ -368,7 +368,20 @@ docker pull jupyter/scipy-notebook:7e1a19a8427f

Now you can restart the service and the user can start a notebook with the specified images.

### Create your own Docker images

Docker images that work with JupyterHub with Kubernetes will work with this set-up with the addition of jupyterhub and notebook.

Add the following to your Docker image
```{bash eval=FALSE}
RUN pip3 install \
'jupyter-rsession-proxy' \
'jupyterhub==3.1.*' \
'notebook==6.*' \
'jupyterlab'
CMD ["jupyterhub-singleuser"]
```

<!--
Did not do the below. Did add 8081 as a port but not sure I needed to.
Expand Down Expand Up @@ -410,12 +423,13 @@ Create user
useradd jhub
```

Open the 8000 port
Open the 8000 (for jupyterhub) and 8787 (for RStudio) port
```{bash eval=FALSE}
#sudo systemctl enable firewalld
#sudo systemctl start firewalld
sudo firewall-cmd --permanent --add-port 8000/tcp
sudo firewall-cmd --permanent --add-port 8787/tcp
sudo firewall-cmd --reload
sudo firewall-cmd --list-ports
```
Expand Down Expand Up @@ -466,6 +480,7 @@ c.DockerSpawner.image_whitelist = {
'datascience-r': 'jupyter/datascience-notebook:r-4.3.1',
'scipy-notebook': 'jupyter/scipy-notebook:7e1a19a8427f',
}
c.Spawner.cmd = 'jupyterhub-singleuser'
```

Docker pull of the images
Expand Down Expand Up @@ -513,6 +528,19 @@ sudo systemctl start jupyterhub.service

Done! See the long instructions if anything is not working.

To use your own docker images
Add the following to your Docker image
```{bash eval=FALSE}
FROM <your image with python 3+>
RUN pip3 install \
'jupyter-rsession-proxy' \
'jupyterhub==3.1.*' \
'notebook==6.*' \
'jupyterlab'
CMD ["jupyterhub-singleuser"]
```

<!--
Did not do the below. Did add 8081 as a port but not sure I needed to.
Expand Down
5 changes: 5 additions & 0 deletions ci/iorocker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -28,5 +28,10 @@ RUN R -e 'install.packages(c("geodata", "ggspatial", "rnaturalearth"))'
# general
RUN R -e 'install.packages(c("corrplot", "here", "matrixcalc", "cmocean", "DT", "janitor"))'

# for JUPYTERHUB
RUN conda install -c conda-forge jupyterhub
RUN conda install -c conda-forge notebook


# ENV JUPYTERHUB_HTTP_REFERER=https://dhub.opensci.live/
ENV JUPYTERHUB_HTTP_REFERER=https://itcoocean.2i2c.cloud/
2 changes: 1 addition & 1 deletion ci/iorocker/instructions.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Indian Ocean Summer Docker Images

https://hub.docker.com/repository/docker/eeholmes/iopython/general
<https://hub.docker.com/repository/docker/eeholmes/iorocker/general>

The one to use is the dated one. The `main` tag doesn't seem to always be recognized as a new tag when it changes.

Expand Down
13 changes: 13 additions & 0 deletions ci/minimal-jupyter/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#FROM openscapes/rocker:a7596b5
FROM eeholmes/py-rocket:20231004

RUN pip3 install \
'jupyter-rsession-proxy' \
'jupyterhub==3.1.*' \
'notebook==6.*' \
'jupyterlab'

#RUN R --quiet -e 'remotes::install_github("IRkernel/IRkernel@*release")'
#RUN R --quiet -e 'IRkernel::installspec(user = FALSE)'

CMD ["jupyterhub-singleuser"]
75 changes: 75 additions & 0 deletions ci/py-rocket/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
FROM rocker/geospatial:4.2

ENV CONDA_ENV=notebook \
DEBIAN_FRONTEND=noninteractive \
NB_USER=rstudio \
SHELL=/bin/bash \
# Setup locale to be UTF-8, avoiding gnarly hard to debug encoding errors
LANG=C.UTF-8 \
LC_ALL=C.UTF-8 \
CONDA_DIR=/srv/conda

# All env vars that reference other env vars need to be in their own ENV block
# Path to the python environment where the jupyter notebook packages are installed
ENV NB_PYTHON_PREFIX=${CONDA_DIR}/envs/${CONDA_ENV} \
# Home directory of our non-root user
HOME=/home/${NB_USER}

# Add both our notebook env as well as default conda installation to $PATH
# Thus, when we start a `python` process (for kernels, or notebooks, etc),
# it loads the python in the notebook conda environment, as that comes
# first here.
ENV PATH=${NB_PYTHON_PREFIX}/bin:${CONDA_DIR}/bin:${PATH}

# Ask dask to read config from ${CONDA_DIR}/etc rather than
# the default of /etc, since the non-root jovyan user can write
# to ${CONDA_DIR}/etc but not to /etc
ENV DASK_ROOT_CONFIG=${CONDA_DIR}/etc

RUN usermod -aG sudo ${NB_USER} && chown -R ${NB_USER}:${NB_USER} /srv
RUN usermod -d /home/jovyan rstudio

# bin to PATH earlier ($NB_PYTHON_PREFIX/bin)
RUN echo ". ${CONDA_DIR}/etc/profile.d/conda.sh ; conda activate ${CONDA_ENV}" > /etc/profile.d/init_conda.sh

USER ${NB_USER}

COPY --chown=rstudio:rstudio . /tmp/build

WORKDIR /tmp/build

# Install latest mambaforge in ${CONDA_DIR}
RUN echo "Installing Mambaforge..." \
&& URL="https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh" \
&& wget --quiet ${URL} -O installer.sh \
&& /bin/bash installer.sh -u -b -p ${CONDA_DIR} \
&& rm installer.sh \
&& mamba clean -afy \
# After installing the packages, we cleanup some unnecessary files
# to try reduce image size - see https://jcristharif.com/conda-docker-tips.html
&& find ${CONDA_DIR} -follow -type f -name '*.a' -delete \
&& find ${CONDA_DIR} -follow -type f -name '*.pyc' -delete

RUN echo "Checking for 'conda-linux-64.lock' or 'environment.yml'..." \
; if test -f "conda-linux-64.lock" ; then \
mamba create --name ${CONDA_ENV} --file conda-linux-64.lock \
; elif test -f "environment.yml" ; then \
mamba env create --name ${CONDA_ENV} -f environment.yml \
; else echo "No conda-linux-64.lock or environment.yml! *creating default env*" ; \
mamba create --name ${CONDA_ENV} pangeo-notebook \
; fi \
&& mamba clean -yaf \
&& find ${CONDA_DIR} -follow -type f -name '*.a' -delete \
&& find ${CONDA_DIR} -follow -type f -name '*.pyc' -delete \
&& find ${CONDA_DIR} -follow -type f -name '*.js.map' -delete \
; if [ -d ${NB_PYTHON_PREFIX}/lib/python*/site-packages/bokeh/server/static ]; then \
find ${NB_PYTHON_PREFIX}/lib/python*/site-packages/bokeh/server/static -follow -type f -name '*.js' ! -name '*.min.js' -delete \
; fi

RUN pip install jupyter-rsession-proxy

WORKDIR ${HOME}

EXPOSE 8888

CMD ["jupyter", "lab", "--ip=0.0.0.0", "--port=8888", "--allow-root", "--no-browser"]
9 changes: 9 additions & 0 deletions ci/rocker-binder/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
FROM rocker/binder:4.3

RUN pip3 install \
'jupyter-rsession-proxy' \
'jupyterhub==3.1.*' \
'notebook==6.*' \
'jupyterlab'

CMD ["jupyterhub-singleuser"]
Loading

0 comments on commit e4e43bb

Please sign in to comment.