Merge branch 'release/2.7.1-alpha.2'

carvel-packages/training-platform/bundle/config/10-secrets-manager/07-deployments.yaml

@@ -37,9 +37,21 @@ spec:
           allowPrivilegeEscalation: false
           capabilities:
             drop: ["ALL"]
+        startupProbe:
+          initialDelaySeconds: 15
+          periodSeconds: 5
+          successThreshold: 1
+          failureThreshold: 4
+          httpGet:
+            path: /healthz?probe=startup
+            port: 8080
         livenessProbe:
+          initialDelaySeconds: 5
+          periodSeconds: 5
+          successThreshold: 1
+          failureThreshold: 3
           httpGet:
-            path: /healthz
+            path: /healthz?probe=liveness
             port: 8080
         volumeMounts:
         - name: config

carvel-packages/training-platform/bundle/config/11-session-manager/07-deployments.yaml

@@ -37,9 +37,21 @@ spec:
           allowPrivilegeEscalation: false
           capabilities:
             drop: ["ALL"]
+        startupProbe:
+          initialDelaySeconds: 15
+          periodSeconds: 5
+          successThreshold: 1
+          failureThreshold: 4
+          httpGet:
+            path: /healthz?probe=startup
+            port: 8080
         livenessProbe:
+          initialDelaySeconds: 5
+          periodSeconds: 5
+          successThreshold: 1
+          failureThreshold: 3
           httpGet:
-            path: /healthz
+            path: /healthz?probe=liveness
             port: 8080
         volumeMounts:
         - name: config

developer-docs/build-instructions.md

@@ -257,3 +257,13 @@ make prune-all
 Note that this will run `docker system prune` rather than `docker image prune`, which will also result in unused docker networks and volumes being cleaned up.
 
 Also note that this doesn't reclaim space used by the image cache of `containerd` on the Kubernetes cluster nodes. If you are doing a lot of work on Educates, especially changes to the workshop base images and you deploy workshops using many successive versions of the images, eventually you can run out of storage space due to the `containerd` image cache. In this case there isn't really anything simple you do can except for deleting the Kubernetes cluster and starting over.
+
+Building docs.educates.dev locally
+----------------------------------
+
+If you're working on updates or additions to the project documentation served at [docs.educates.dev](https://docs.educates.dev), you might want to preview your changes locally before opening a PR. To build and preview the docs locally, you can run:
+
+```
+make build-project-docs
+make open-project-docs
+```

project-docs/getting-started/quick-start-guide.md

@@ -15,7 +15,7 @@ To deploy Educates on your local machine using the Educates command line tool th
 
 * You need to be running macOS or Linux. If using Windows you will need WSL (Windows subsystem for Linux). The Educates command line tool has primarily been tested on macOS.
 
-* You need to have a working `docker` environment. The Educates command line tool has primarily been tested with Docker Desktop.
+* You need to have a working `docker` environment. The Educates command line tool has primarily been tested with Docker Desktop on macOS.
 
 * You need to have sufficient memory and disk resources allocated to the `docker` environment to run Kubernetes, Educates etc.
 
@@ -27,6 +27,14 @@ To deploy Educates on your local machine using the Educates command line tool th
 
 * You need to have port 5001 available on the local machine as this will be used for a local image registry.
 
+If you are using Docker Desktop, you need to have the following enabled:
+
+* Use kernel networking for UDP (Settings->Resources->Network).
+
+* Allow the default Docker socket to be used (Settings->Advanced).
+
+* Allow privileged port mapping (Settings->Advanced).
+
 Downloading the CLI
 -----------------------
 

project-docs/release-notes/version-2.7.1.md

@@ -1,6 +1,11 @@
 Version 2.7.1
 =============
 
+Features Changed
+----------------
+
+* Updated VS Code to version 1.89.1.
+
 Bugs Fixed
 ----------
 
@@ -20,3 +25,44 @@ Bugs Fixed
   use `netcat` by installing `netcat` package instead of `nc`. The `ncat`
   package is also installed if want newer variant of `nc`, but you will need to
   use the `ncat` command explicitly.
+
+* If the cluster DNS server was slow to start resolving DNS names after a new
+  node was started, the session manager could fail on startup and enter crash
+  loop back off state. To remedy both session manager and secrets manager now
+  ensure DNS is able to resolve cluster control plane DNS name before starting
+  up. Startup probes have also been added to these two operators.
+
+* If the cluster DNS didn't return a FQDN for the `kubernetes.default.svc` when
+  queried by that name, the value of the `CLUSTER_DOMAIN` variable provided to
+  the workshop sessions would be incorrect. This was occuring when Educates was
+  installed into some versions of a virtual cluster. When the returned host name
+  is not a FQDN, then `cluster.local` will now be used.
+
+* Workshop session dashboard configuration could not in some cases be overridden
+  from inside of the workshop session by modifying the injected workshop
+  definition. This included not being able to change workshop/terminal layout
+  and whether the dashboard tabs for the editor and console were displayed.
+
+* The builtin Google Analytics integration was broken due to the `TrainingPortal`
+  Content Security Policy (CSP) directives declaring outdated sources. The CSPs
+  now allow for `*.google-analytics.com` and `*.googletagmanager.com` to be
+  referenced.
+
+* The `CSRF_ALLOWED_ORIGINS` setting for the `TrainingPortal` Django backend was
+  breaking CSRF verification for any `TrainingPortal` with a custom
+  `PORTAL_HOSTNAME` configured. We now use the `PORTAL_HOSTNAME` as allowed
+  CSRF origin and only fall back to the previous implementation if no custom
+  hostname was provided.
+
+* The workshop title in the dropdown TOC of the workshop instructions was not
+  being populated with the workshop title from the workshop definition when the
+  Hugo renderer was being used.
+
+* If a workshop session had not been registered by the session manager within 30
+  seconds of creation and a workshop allocation was pending, the workshop
+  allocation would not progress properly to the allocated state and any request
+  objects associated with the workshop session would not be created. From the
+  perspective of a workshop user the session would still appear to work as the
+  workshop dashboard would still be accessible, but request objects would be
+  missing. Timeout for workshop session registration has been increased to 90
+  seconds.

project-docs/workshop-content/workshop-instructions.md

@@ -1155,6 +1155,37 @@ The shortcode for selecting based on the pathway is for example implemented as:
 {{ end }}
 ```
 
+Adding admonitions with shortcodes
+----------------------------------
+
+Since Educates v2.6.0, a range of custom admonitions is supported when using the ``hugo`` renderer. Currently, three types of admonitions exist:
+
+- **note** - rendered as blue text box
+- **warning** - rendered as yellow text box
+- **danger** - rendered as red text box
+
+The shortcodes can be used like this, with the respective admonition name as shortcode:
+
+```
+{{< note >}}
+A friendly admonition.
+{{< /note >}}
+
+{{< warning >}}
+Consider this admonition.
+{{< /warning >}}
+
+{{< danger >}}
+You better consider this admonition!
+{{< /danger >}}
+```
+
+The rendered version looks like this:
+
+![Rendered admonitions supported by Educates](admonitions.png)
+
+More information on shortcodes can be found in the [Hugo documentation](https://gohugo.io/content-management/shortcodes/).
+
 Embedding custom HTML content
 -----------------------------
 
@@ -1188,7 +1219,7 @@ Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin justo.
 Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin justo.
 ```
 
-If using the ``classic`` render and AsciiDoc, HTML can be embedded by using a passthrough block.
+If using the ``classic`` renderer and AsciiDoc, HTML can be embedded by using a passthrough block.
 
 ```
 Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin justo.
@@ -1218,7 +1249,7 @@ Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin justo.
 Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin justo.
 ```
 
-If using the ``hugo`` renderer, it provides as standard various shortcodes for embedding different custom HTML snippets, such as embedding videos or images. If you have a custom requirement of your own, you will need to provide your own shortcode by placing it in the ``workshop/layouts/shortcodes`` directory.
+If using the ``hugo`` renderer, it provides as standard various shortcodes for embedding different custom HTML snippets, such as embedding videos or images. If you have a custom requirement of your own, you will need to provide your own shortcode by placing it in the ``workshop/layouts/shortcodes`` directory and referencing that in your instructions.
 
 In all cases it is recommended that the HTML consist of only a single HTML element. If you have more than one, include them all in a ``div`` element. The latter is necessary if any of the HTML elements are marked as hidden and the embedded HTML will be a part of a collapsible section. If you don't ensure the hidden HTML element is placed under the single top level ``div`` element, the hidden HTML element will end up being made visible when the collapsible section is expanded.
 
@@ -1232,6 +1263,8 @@ Triggering actions from Javascript
 
 Clickable actions can be embedded in workshop instructions and reduce the manual steps that workhop users need to perform. If further automation is required, a subset of the underlying tasks which can be triggered through clickable actions can be executed from Javascript code embedded within the workshop instructions page. This can be used for tasks such as ensuring that a dashboard tab is made visible immediately a page in the workshop instructions is viewed.
 
+If using the ``classic`` renderer and Markdown, the Javascript can be embedded directly within the Markdown document.
+
 ```
 <script>
 window.addEventListener("load", function() {
@@ -1240,6 +1273,20 @@ window.addEventListener("load", function() {
 </script>
 ```
 
+If using the ``classic`` renderer and AsciiDoc, HTML can be embedded by using a passthrough block.
+
+```
+++++
+<script>
+window.addEventListener("load", function() {
+    educates.expose_dashboard("Editor");
+});
+</script>
+++++
+```
+
+If using the ``hugo`` renderer you will need to provide your own shortcode for embedding custom Javascript into a page by placing it in the ``workshop/layouts/shortcodes`` directory and referencing that in your instructions.
+
 All accessible functions are defined within the scope of the `educates` object. The available API is described by:
 
 ```

project-docs/workshop-migration/learning-center.md

@@ -88,6 +88,8 @@ Note that whereas Learning Center only bundled a single workshop base image, Edu
 
 ``conda-environment:*`` - A tagged version of the ``conda-environment`` workshop image which has been matched with the current version of the Educates operator.
 
+Note that any custom workshop images you may have created for Learning Center will need to be rebuilt using the corresponding workshop base image from Educates, as existing Learning Center based images will not work in Educates.
+
 Downloading of workshop content
 -------------------------------
 

secrets-manager/main.py

@@ -2,12 +2,51 @@
 import contextlib
 import logging
 import signal
+import socket
+import time
 
 from threading import Thread, Event
 
 import kopf
 import pykube
 
+
+logger = logging.getLogger("educates")
+
+
+def check_dns_is_ready():
+    # Check that DNS is actually ready and able to resolve the DNS for the
+    # Kubernetes control plane. This is a workaround for the fact that the DNS
+    # service may not be ready when the pod starts. Check at intervals but bail
+    # out and raise the original exception if we can't resolve the DNS name
+    # after 60 seconds.
+
+    logger.info("Checking DNS resolution for Kubernetes control plane.")
+
+    start_time = time.time()
+
+    while True:
+        try:
+            socket.getaddrinfo("kubernetes.default.svc", 0, flags=socket.AI_CANONNAME)
+            break
+        except socket.gaierror:
+            if time.time() - start_time > 60:
+                raise
+
+            # Wait for 1 second before trying again.
+
+            logger.info("DNS resolution for Kubernetes control plane is not ready yet, sleeping...")
+
+            time.sleep(1)
+
+    logger.info("DNS resolution for Kubernetes control plane is ready.")
+
+
+# Check that DNS is actually ready before importing the handlers.
+
+check_dns_is_ready()
+
+
 from handlers import namespace
 from handlers import secret
 from handlers import secretcopier
@@ -18,8 +57,6 @@
 
 _event_loop = None  # pylint: disable=invalid-name
 
-logger = logging.getLogger("educates")
-
 _stop_flag = Event()
 
 
@@ -36,7 +73,7 @@ def login_fn(**kwargs):
     return kopf.login_via_pykube(**kwargs)
 
 
-@kopf.on.probe(id='api')
+@kopf.on.probe(id="api")
 def check_api_access(**kwargs):
     try:
         api = pykube.HTTPClient(pykube.KubeConfig.from_env())

session-manager/handlers/operator_config.py

@@ -42,7 +42,11 @@
 RUNTIME_CLASS = xget(config_values, "clusterRuntime.class", "")
 
 CLUSTER_DOMAIN = socket.getaddrinfo("kubernetes.default.svc", 0, flags=socket.AI_CANONNAME)[0][3]
-CLUSTER_DOMAIN = CLUSTER_DOMAIN.replace("kubernetes.default.svc.", "")
+
+if CLUSTER_DOMAIN.startswith("kubernetes.default.svc."):
+    CLUSTER_DOMAIN = CLUSTER_DOMAIN.replace("kubernetes.default.svc.", "")
+else:
+    CLUSTER_DOMAIN = "cluster.local"
 
 INGRESS_DOMAIN = xget(config_values, "clusterIngress.domain", "educates-local-dev.test")
 INGRESS_CLASS = xget(config_values, "clusterIngress.class", "")

session-manager/handlers/workshopallocation.py

@@ -91,7 +91,7 @@ def workshop_allocation_create(
     parameters_name = f"{session_name}-request"
 
     if not (None, environment_name) in workshop_environment_index:
-        if runtime.total_seconds() >= 30:
+        if runtime.total_seconds() >= 45:
             patch["status"] = {
                 OPERATOR_STATUS_KEY: {
                     "phase": "Failed",
@@ -122,7 +122,7 @@ def workshop_allocation_create(
     environment_instance, *_ = workshop_environment_index[(None, environment_name)]
 
     if not (None, session_name) in workshop_session_index:
-        if runtime.total_seconds() >= 30:
+        if runtime.total_seconds() >= 90:
             patch["status"] = {
                 OPERATOR_STATUS_KEY: {
                     "phase": "Failed",