Skip to content

Latest commit

 

History

History
191 lines (122 loc) · 7.04 KB

README.adoc

File metadata and controls

191 lines (122 loc) · 7.04 KB
Raw

README

Opionated JupyterHub deployment for workshops relying on GitHub for Authentication and Digital Ocean as the infrastructure provider.

This is a drastic departure of our previous environment available here: https://github.com/GoSecure/jupyterhub-workshop-environment-legacy

Features:

  • Supports JupyterLab and Jupyter Notebook (Python and Terminals)

  • Read-only shared files to all users (in users' homes)

  • Writeable and shared scratch directory available to all users (in users' homes)

  • Users have their own copies of notebook, created on first login from a template directory

  • Server deployed and managed by Terraform

  • Out of the box LetsEncrypt HTTPS

  • Very close to upstream The Littlest JupyterHub (TLJH) should make upgrades seamless

Originally forked from jupyterhub-deploy-docker in 2017 but little left if anything today.

Setup Instructions

Prepare for Deployment

These steps should be done locally before provisioning the JupterHub server to facilitate deployment.

Setup GitHub Authentication

Create a GitHub application reflecting the purpose of your JupyterHub workshop. It will be used to allow users to login and gain access to the workshop servers. You should know the following information ahead of time:

  • Workshop Name (Application Name)

  • Workshop Server URL (Doesn’t need to resolve yet)

  • Workshop Description

  • Workshop Icon if desired.

NOTE: Applications for workshops should be created under your organization at https://github.com/organizations/<orgname>/settings/applications/new

Ask organisation admin to create the application with the required information.

Populate terraform.tfvars with the following content:

github_client_id  = "<github_client_id>"
github_client_secret = "<github_client_secret>"
# GitHub Oauth Callback will be set to https://<workshop_domain>/hub/oauth_callback by default
#github_oauth_callback = "https://<myhost.mydomain>/hub/oauth_callback"

By default our setup is open to anyone to create an account like for an open workshop with no pre-registrations (different options available below).

Setup Administrator Accounts

The first administrator account is configured in the terraform.tfvars file with jupyterhub_admin. It must be a GitHub username. Once authenticated this user will be an administrator. Other admin accounts are configured in the /opt/tljh/config/config.yaml file. The administration GUI (/hub/admin) also allows you to elevate accounts via "Edit". These accounts can use sudo without a password.

Workshop Files

There are three ways to expose files to the workshop attendees.

Files exposed to attendees all rely to using the Linux /etc/skel/ home directory template. Remember: once the user is created, things in skel will not be copied again. You must destroy a user for it to be recreated (see instructions below under "Operational Procedures").

Read-only shared with all participants

  • Locally: workshop-data/

  • Deployed to: /srv/workshop/

  • Available to participants as: workshop-readonly/ in their home directory

All users will have access to a directory in their own homes (/home/jupyter-<github_username>/) called workshop-readonly/ that will be mapped to the servers' /srv/workshop/.

Changes made in /srv/workshop/* will be reflected instantly to all participants.

This is accomplished via a symlink in /etc/skel/.

Files for participants (writable and executable)

  • Locally: workshop-data/assignments/

  • Deployed to: /srv/workshop/assignments and /etc/skel/your-personal-lab/

  • Available to participants as: your-personal-lab/ in their home directory

On container creation, a directory called your-personal-lab/ will be created and populated from the servers' /etc/skel/your-personal-lab/ content.

Scratch for participants (shared and writable)

  • Locally: None

  • On the server: /srv/scratch/

  • Available to participants as: scratch/ in their home directory

A scratch/ directory is available in users' home which is world-writable. Creator ownership is retained, you can’t alter someone else’s files but you can read them.

Enabled by the tljh-shared-directory plugin.

Deploying the Server

Ensure that you have a recent version of Terraform installed and get it ready:

terraform init

Configure your Digital Ocean credentials and other settings in terraform.tfvars:

# the following values are used as tags in digital ocean so only lowercase, dash and underscore allowed
workshop_name = "<workshop-name>"
tag_owner = "<username>"
tag_event = "<event-name>"
do_token = "<your DO token>"
# default is 1 CPU 2 GB, you can find droplet sizes here: https://slugs.do-api.dev/
#instance_size = "c-4"
# default region is Toronto (tor1)
instance_region = "nyc3"
jupyterhub_admin = "<github-username>"
# A URL to a Python requirements.txt file for dependencies to be installed on the system
workshop_requirements_url = "<URL>"

# GitHub Authentication Parameters
github_client_id  = "<github_client_id>"
github_client_secret = "<github_client_secret>"

Spawn and provision the droplet. It might take a while to setup and provision, so be patient.

terraform validate
terraform plan
terraform apply
Note
 During the apply, you should hurry up and update the DNS of workshop_domain to point to the newly deployed droplet public IP. Failure to do so will end-up in repeated Let’s Encrypt certificate creation failures and will result in a block of one hour before you can attempt to create the certificate again. If it happens. Shutdown jupyterhub, make sure DNS has propagated and start jupyterhub again after one hour.

Server-Side Configuration

To connect to the server via SSH, run:

./bin/ssh-connect.sh

More Customization

At this point you have a 100% standard TLJH installation. You can rely on their documentation to further customize the installation:

Operational Procedures

Upgrading JupyterHub

Re-running the TLJH installer should upgrade JupyterHub

Deleting a user

This is useful when testing the initial setup of a user’s home directory.

  • In the /hub/admin GUI: Edit User → Delete

  • In a root shell: userdel -r jupyter-<username>

Reveal Solutions

From an admin notebook:

!sudo chmod go+rx /srv/workshop/solutions/
!sudo chmod go+r /srv/workshop/solutions/*

From an admin shell:

sudo chmod go+rx /srv/workshop/solutions/
sudo chmod go+r /srv/workshop/solutions/*

Debugging

Logging

Using systemctl status on the jupyterhub or traefik services:

systemctl status jupyterhub

Checking the logs with journalctl:

journalctl -f -u traefik