diff --git a/source/authentication/tutorial-oidc-keycloak-rhel7/configure-cilogon.rst b/source/authentication/tutorial-oidc-keycloak-rhel7/configure-cilogon.rst index 48b47766..954353de 100644 --- a/source/authentication/tutorial-oidc-keycloak-rhel7/configure-cilogon.rst +++ b/source/authentication/tutorial-oidc-keycloak-rhel7/configure-cilogon.rst @@ -5,7 +5,7 @@ Configure Keycloak with CILogon We will now use Keycloak's admin Web UI to setup the ability to log existing users in with CILogon. -When a user logs in with CILogon for the first time they will be redirected back to Keycloak to log in with their local (ie LDAP) +When a user logs in with CILogon for the first time they will be redirected back to Keycloak to log in with their local (i.e., LDAP) credentials. This performs a mapping of their CILogon identity with their Keycloak identity. .. warning:: @@ -18,11 +18,11 @@ Register your Keycloak instance with CILogon #. Go to ``https://cilogon.org/oauth2/register`` and fill out the form - #. The Home URL will be the base URL of your Keycloak instance, eg: ``https://ondemand-idpdev.hpc.osc.edu``. + #. The Home URL will be the base URL of your Keycloak instance, e.g.: ``https://ondemand-idpdev.hpc.osc.edu``. #. The callback URL will be ``https://ondemand-idpdev.hpc.osc.edu/auth/realms//broker/cilogon/endpoint``. Replace ``https://ondemand-idpdev.hpc.osc.edu`` with your Keycloak instance #. The box for "Is this a public client?" should not be checked - #. For "Scopes" be sure to check "profile" and "org.cilogon.userinfo" + #. For "Scopes" be sure to check ``profile`` and ``org.cilogon.userinfo`` You will be provided a Client ID and a Client Secret, be sure to save these values. Your registered client will not be usable until you receive an email from CILogon stating your client has been approved. @@ -36,7 +36,7 @@ Add the CILogon Identity Provider #. Select the "Add provider..." drop down and choose "OpenID Connect v1.0" #. Fill in the fields as noted below - #. Alias: cilogon (This must be cilogon as this alias is used in the callback URL) + #. Alias: ``cilogon`` (This must be ``cilogon`` as this alias is used in the callback URL) #. Display Name: CILogon #. Enabled: ON #. First Login Flow: browser @@ -46,7 +46,7 @@ Add the CILogon Identity Provider #. Client Authentication: Client secret sent as post #. Client ID: #. Client Secret: - #. Default Scopes: "openid profile org.cilogon.userinfo" + #. Default Scopes: ``openid profile org.cilogon.userinfo`` #. Click "Save" @@ -56,7 +56,7 @@ Support users removing CILogon mappings In order for a user to remove an existing CILogon mapping in Keycloak they must navigate to ``https://ondemand-idpdev.hpc.osc.edu/auth/realms//account/identity``. Replace ``ondemand-idpdev.hpc.osc.edu`` with the web URL for your Keycloak instance. -The URL can be added to the OnDemand Help dropdown with custom text to make it easier for users to access their Keycloak identity page. +The URL can be added to the OnDemand Help drop-down with custom text to make it easier for users to access their Keycloak identity page. #. Add ``OOD_DASHBOARD_HELP_CUSTOM_URL`` to ``/etc/ood/config/apps/dashboard/env`` that points to the URL of the identity page for your Keycloak instance. Example: ``https://ondemand-idpdev.hpc.osc.edu/auth/realms/osc/account/identity`` diff --git a/source/authentication/tutorial-oidc-keycloak-rhel7/configure-keycloak-webui.rst b/source/authentication/tutorial-oidc-keycloak-rhel7/configure-keycloak-webui.rst index 6ac157a9..b5abca7c 100644 --- a/source/authentication/tutorial-oidc-keycloak-rhel7/configure-keycloak-webui.rst +++ b/source/authentication/tutorial-oidc-keycloak-rhel7/configure-keycloak-webui.rst @@ -23,16 +23,16 @@ Configure LDAP ------------------------------------------ #. Choose User Federation on the left (verify ondemand realm is current realm) -#. Select "ldap" for provider +#. Select ``ldap`` for provider #. Import Users set to OFF #. Edit Mode set to READ_ONLY #. Vendor set to other – for OpenLDAP - #. User Object Classes set to posixAccount – OSC specific and odd - #. Connection URL: ldaps://ldap1.infra.osc.edu:636 ldaps://ldap2.infra.osc.edu:636 – using multiple to demonstrate more than 1 - #. User DN: ou=People,dc=osc,dc=edu + #. User Object Classes set to ``posixAccount`` – OSC specific and odd + #. Connection URL: ``ldaps://ldap1.infra.osc.edu:636 ldaps://ldap2.infra.osc.edu:636`` – using multiple to demonstrate more than 1 + #. User DN: ``ou=People,dc=osc,dc=edu`` #. Auth Type: none – OSC specific as we allow anonymous binds - #. Use Truststore SPI: never – OSC specific since our LDAP certificates are already trusted since from InCommon, leaving default is probably acceptable if no truststoreSpi defined in XML configs + #. Use Truststore SPI: never – OSC specific since our LDAP certificates are already trusted since from InCommon, leaving default is probably acceptable if no ``truststoreSpi`` defined in XML configurations. #. Save @@ -48,8 +48,8 @@ Add OnDemand as a client #. Choose Clients, then click Create in top right corner - #. Client ID: ondemand-dev.hpc.osc.edu - #. Client Protocol: openid-connect + #. Client ID: ``ondemand-dev.hpc.osc.edu`` + #. Client Protocol: ``openid-connect`` #. Save (leave Root URL blank) #. Then edit Settings for the newly created client: @@ -65,6 +65,6 @@ Add OnDemand as a client #. Finally, get the client secret to use with OnDemand installation: - #. Select the "Credentials" tab of the "Client" you are viewing i.e. "Clients >> ondemand-dev.hpc.osc.edu" + #. Select the "Credentials" tab of the "Client" you are viewing i.e. ``Clients >> ondemand-dev.hpc.osc.edu`` #. Copy the value for "secret" for future use in this tutorial (and keep it secure). diff --git a/source/authentication/tutorial-oidc-keycloak-rhel7/install_mod_auth_openidc.rst b/source/authentication/tutorial-oidc-keycloak-rhel7/install_mod_auth_openidc.rst index 18b20162..6b571eab 100644 --- a/source/authentication/tutorial-oidc-keycloak-rhel7/install_mod_auth_openidc.rst +++ b/source/authentication/tutorial-oidc-keycloak-rhel7/install_mod_auth_openidc.rst @@ -3,21 +3,21 @@ Configure OnDemand to authenticate with Keycloak ================================================ -OnDemand's Apache needs to use mod_auth_openidc to be able to act as an OpenID -Connect client to Keycloak. We will install mod_auth_openidc and modify -OnDemand's Apache configs to enable authentication via Keycloak. +OnDemand's Apache needs to use ``mod_auth_openidc`` to be able to act as an OpenID +Connect client to Keycloak. We will install ``mod_auth_openidc`` and modify +OnDemand's Apache configurations to enable authentication via Keycloak. -Install mod_auth_openidc ------------------------- +Install ``mod_auth_openidc`` +---------------------------- -#. Install httpd24-mod_auth_openidc from ondemand-web repo +#. Install ``httpd24-mod_auth_openidc`` from ondemand-web repository. .. code-block:: sh sudo yum install httpd24-mod_auth_openidc -Re-generate main config using ood-portal-generator +Re-generate main configuration using ood-portal-generator ----------------------------------------------------------- #. Edit the YAML configuration file for the :ref:`ood-portal-generator` located @@ -49,12 +49,12 @@ Re-generate main config using ood-portal-generator Notice that we are - * changing the Authentication directives for openid-connect - * specifying /oidc to be the sub-uri used by mod_auth_openidc - * specifying that /logout should redirect to this /oidc sub-uri to handle logout + * changing the Authentication directives for ``openid-connect`` + * specifying ``/oidc`` to be the URI used by ``mod_auth_openidc`` + * specifying that /logout should redirect to this ``/oidc`` URI to handle logout and specifying after logout, the user should be redirected back to OnDemand (which in this tutorial's case is ``https%3A%2F%2Fondemand-dev.hpc.osc.edu``, - the query param escaped format of ``https://ondemand-dev.hpc.osc.edu``) + the query parameter escaped format of ``https://ondemand-dev.hpc.osc.edu``) #. Then build and install the new Apache configuration file with: @@ -62,7 +62,7 @@ Re-generate main config using ood-portal-generator sudo /opt/ood/ood-portal-generator/sbin/update_ood_portal - The effect of this change in the Apache config (in case you want to apply the changes manually) are: + The effect of this change in the Apache configuration (in case you want to apply the changes manually) are: #. Change the authentication directives for all of the Locations that require authentication i.e.: @@ -100,10 +100,10 @@ Re-generate main config using ood-portal-generator Require valid-user -Add Keycloak config to OnDemand Apache for mod_auth_openidc ------------------------------------------------------------ +Add Keycloak configuration to OnDemand Apache for ``mod_auth_openidc`` +---------------------------------------------------------------------- -#. Add the file /opt/rh/httpd24/root/etc/httpd/conf.d/auth_openidc.conf with the contents: +#. Add the file ``/opt/rh/httpd24/root/etc/httpd/conf.d/auth_openidc.conf`` with the contents: .. code-block:: none @@ -140,7 +140,7 @@ Add Keycloak config to OnDemand Apache for mod_auth_openidc #. Then restart OnDemand's Apache. OnDemand should now be authenticating using KeyCloak. - Stop both servives: + Stop both services: .. code-block:: sh diff --git a/source/customizations.rst b/source/customizations.rst index 1ebb49c9..553a2696 100644 --- a/source/customizations.rst +++ b/source/customizations.rst @@ -47,7 +47,7 @@ To add an announcement message that appears at the top of the dashboard you can On each request the dashboard will check for the existence of this file. If it exists, the contents will be converted using markdown converter to HTML and displayed inside a bootstrap alert. -For example, if I create an announcement.md file with the contents: +For example, if I create an ``announcement.md`` file with the contents: .. code-block:: md @@ -62,7 +62,7 @@ the user would see this message at the top of the dashboard: Example of the Dashboard announcement. -If the announcement file has the extension ``yml`` and is a yaml file it is first rendered using ERB and then the resulting file is parsed as YAML. The valid keys are: +If the announcement file has the extension ``yml`` and is a YAML file it is first rendered using ERB and then the resulting file is parsed as YAML. The valid keys are: .. list-table:: Announcement configuration keys. @@ -70,19 +70,19 @@ If the announcement file has the extension ``yml`` and is a yaml file it is firs - Description * - type - The type of announcement. Values can be ``warning``, ``info``, ``success``, or ``danger``. - * - msg + * - ``msg`` - The announcement's message. - * - id + * - ``id`` - Optional unique identifier for the announcement. This is useful for managing changes to announcements that are used as ToS or EULA that users need to agree to. When provided, it will be used to validate if the announcement has been dismissed regardless of the content. Changing the ``id`` will make the announcement appear again. - * - dismissible + * - ``dismissible`` - Specify if the announcement is dismissible or not with ``true`` or ``false``. Defaults to ``true``. - * - required + * - ``required`` - Specify if the announcement is required or not with ``true`` or ``false``. Defaults to ``false``. When this is set to ``true``, the OOD UI will only render the announcement and the user will not be able to submit jobs until the announcement has been accepted. - * - button_text + * - ``button_text`` - Optional parameter to customize the text content for the button to dismiss the announcement. Defaults to ``Accept`` for required and ``OK`` for dismissible. @@ -101,7 +101,7 @@ Because the announcement is rendered via ERB you can do some interesting things, will prevent SSH login to Ruby nodes and Ruby VDI sessions. <% end %> -.. note:: Warnings about the announcement file being missing may be present in users' nginx logs. Despite the warning the Dashboard will still function normally without those files being present. +.. note:: Warnings about the announcement file being missing may be present in users' Nginx logs. Despite the warning the Dashboard will still function normally without those files being present. The file path for the announcement message can be customize using configuration properties, see :ref:`OnDemand configuration documentation `. @@ -110,7 +110,7 @@ Because the announcement is rendered via ERB you can do some interesting things, Message of the Day (MOTD) ------------------------- -You can configure the Dashboard to display the /etc/motd file on the front page - the same file that is displayed when ssh-ing to a login node. +You can configure the Dashboard to display the ``/etc/motd`` file on the front page - the same file that is displayed when logging into to a login node. To display a MOTD file on the Dashboard ensure that the environment variables ``$MOTD_PATH`` and ``$MOTD_FORMAT`` are set, where @@ -158,28 +158,28 @@ Currently only the dashboard uses the colors in the navbar. - Property - Details * - Title - - dashboard_title + - ``dashboard_title`` - The title appears in the navbar and is controlled by the property ``dashboard_title``. The default value is "Open OnDemand". * - Logo - - dashboard_logo - - The default value for ``dashboard_logo`` is ``/public/logo.png`` and this should be the URL to the logo. By default if you place a logo.png at ``/var/www/ood/public/logo.png`` it will be accessible via the URL ``https://your.ondemand.institution.edu/public/logo.png``. SVG logo format is also supported. + - ``dashboard_logo`` + - The default value for ``dashboard_logo`` is ``/public/logo.png`` and this should be the URL to the logo. By default if you place a ``logo.png`` at ``/var/www/ood/public/logo.png`` it will be accessible via the URL ``https://your.ondemand.institution.edu/public/logo.png``. SVG logo format is also supported. * - Logo height - - dashboard_logo_height + - ``dashboard_logo_height`` - The CSS height of the dashboard logo. * - Favicon - - public_url + - ``public_url`` - The favicon is expected to exist at the path ``$public_url/favicon.ico``. For a default OOD installation the favicon will be located at ``/var/www/ood/public/favicon.ico``. * - Brand background color - - brand_bg_color + - ``brand_bg_color`` - Controls the background color of the navbar in the dashboard * - Brand foreground color - - brand_link_active_bg_color + - ``brand_link_active_bg_color`` - Controls the background color the active link in the navbar in the dashboard * - Replace header title with logo - - dashboard_header_img_logo - - Value should be url to logo i.e. ``/public/logo.png``. the background color the active link in the navbar in the dashboard + - ``dashboard_header_img_logo`` + - Value should be URL to logo i.e. ``/public/logo.png``. the background color the active link in the navbar in the dashboard * - Use white text on black background for navbar. - - navbar_type + - ``navbar_type`` - By default we use ``inverse`` for this value, which specifies to use `Bootstrap 3's inverted navbar `_ where text is white and background is black (or dark grey). You can set this to ``default`` to use black text on light grey background if it fits your branding better. .. note:: It is possible to configure these settings using environment variables, although this is deprecated. @@ -188,7 +188,7 @@ Currently only the dashboard uses the colors in the navbar. .. figure:: /images/dashboard_navbar_branding_bluered.png :align: center - Nav bar if I set ``brand_bg_color`` to ``#0000ff`` and ``brand_link_active_bg_color`` to ``#ff0000`` and ``dashboard_title`` to ``OSC OnDemand`` + Navigation bar when ``brand_bg_color`` is ``#0000ff`` and ``brand_link_active_bg_color`` is ``#ff0000`` and ``dashboard_title`` is ``OSC OnDemand``. Custom CSS files ................ @@ -231,7 +231,7 @@ These URLs can be specified, which will appear in the Help menu and on other loc - https://www.osc.edu/ondemand * - Developer Documentation - OOD_DASHBOARD_DEV_DOCS_URL - - https://osc.github.io/ood-documentation/master/app-development.html (link appears in Develop dropdown if developer mode enabled for user) + - https://osc.github.io/ood-documentation/master/app-development.html (link appears in Develop drop-down if developer mode enabled for user) * - Change Password URL - OOD_DASHBOARD_PASSWD_URL - https://my.osc.edu @@ -240,7 +240,7 @@ These URLs can be specified, which will appear in the Help menu and on other loc - https://idp.osc.edu/auth/realms/osc/account/identity Since OnDemand 2.1, custom links can be added to the Help menu using the configuration property ``help_menu``. -Links will be inserted at the end of the core links already included in the menu by the OnDemand codebase. +Links will be inserted at the end of the core links already included in the menu by the OnDemand code-base. ``help_menu`` supports all the link definitions developed for the custom navigation configuration. For more information on how to create custom links, see for example :ref:`adding urls to menus `. @@ -324,7 +324,7 @@ Start by creating the file - For possible scratch space directories, we look for either :file:`/fs/scratch/{project_name}` or :file:`/fs/scratch/{user_name}` - Additionally project scratch directories have a 'title' attribute and will - with in the dropdown with both the title and the path. + with in the drop-down with both the title and the path. The following example adds all directories within the specified base directories to the favorite paths. This approach is ideal when there is no specific directory naming logic to follow. It also appropriately handles Access Control Lists (ACLs). If a directory does not exist, no error is raised, making this configuration easily exportable to different clusters. @@ -469,8 +469,8 @@ can never read files the user cannot read). By setting a colon delimited `OOD_ALLOWLIST_PATH` environment variable, the Job Composer, File Editor, and Files app respect the allowlist in the following manner: -1. Users will be prevented from navigating to, uploading, downloading, viewing, or editing files that are not an eventual child of the allowlisted paths -2. Users will be prevented from copying a template directory from an arbitrary path in the Job Composer if the arbitrary path that is not an eventual child of the allowlisted paths +1. Users will be prevented from navigating to, uploading, downloading, viewing, or editing files that are not an eventual child of the allowed paths. +2. Users will be prevented from copying a template directory from an arbitrary path in the Job Composer if the arbitrary path that is not an eventual child of the allowed paths. 3. Users should not be able to get around this using symlinks We recommend setting this environment variable in ``/etc/ood/config/nginx_stage.yml`` as a YAML mapping (key value pairs) in the mapping (hash/dictionary) ``pun_custom_env`` i.e. below would a list that allows access to home directories, project space, and scratch space at OSC: @@ -495,7 +495,7 @@ Set Default SSH Host Because there are no hosts configured, no hosts are allowed. -In ``/etc/ood/config/apps/shell/env`` set the env var ``OOD_DEFAULT_SSHHOST`` to change the default ssh host. +In ``/etc/ood/config/apps/shell/env`` set the environment variable ``OOD_DEFAULT_SSHHOST`` to change the default ssh host. Since 1.8, there is no out of the box default (in previous versions it was 'localhost', but this has been removed). This will control what host the shell app ssh's to when the URL accessed is ``/pun/sys/shell/ssh/default`` which is the URL other apps will use (unless there is context to specify the cluster to ssh to). @@ -546,7 +546,7 @@ Enable and configure Shell Ping Pong Version 3.1 added the ability for the shell application to send and receive ping pong messages to keep the connection alive, and thus the terminal session alive. -The drawback to this is that these persistant connections can actually outlive your +The drawback to this is that these persistent connections can actually outlive your authentication timeout settings. Meaning users can have active shell sessions for much longer than your authentication systems would normally allow. This is because the connection was made while you were authenticated and it persists after your session @@ -620,13 +620,13 @@ Here's a simple example of what a wrapper script could look like. Fix Unauthorized WebSocket Connection in Shell App -------------------------------------------------- -If you see a 401 error when attempting to launch a Shell app session, where the request URL starts with wss:// and the response header includes ``X-OOD-Failure-Reason: invalid origin``, you may need to set the ``OOD_SHELL_ORIGIN_CHECK`` configuration option. +If you see a 401 error when attempting to launch a Shell app session, where the request URL starts with ``wss://`` and the response header includes ``X-OOD-Failure-Reason: invalid origin``, you may need to set the ``OOD_SHELL_ORIGIN_CHECK`` configuration option. There is a security feature that adds proper CSRF_ protection using both the Origin request header check and a CSRF_ token check. The Origin check uses X-Forwarded-Proto_ and X-Forwarded-Host_ that Apache mod_proxy_ sets to build the string that is used to compare with the Origin request header the browser sends in the WebSocket upgrade request. -In some edge cases this string may not be correct, and as a result valid WebSocket connections will be denied. In this case you can either set ``OOD_SHELL_ORIGIN_CHECK`` env var to the correct https string, or disable the origin check altogether by setting ``OOD_SHELL_ORIGIN_CHECK=off`` (or any other value that does not start with "http") in the ``/etc/ood/config/apps/shell/env`` file. +In some edge cases this string may not be correct, and as a result valid WebSocket connections will be denied. In this case you can either set ``OOD_SHELL_ORIGIN_CHECK`` environment variable to the correct https string, or disable the origin check altogether by setting ``OOD_SHELL_ORIGIN_CHECK=off`` (or any other value that does not start with "http") in the ``/etc/ood/config/apps/shell/env`` file. Either way the CSRF token will still provide protection from this vulnerability. @@ -697,7 +697,7 @@ The manifest contains additional metadata about a job, such as a name, the defau In the event that a job is created from a template that is missing from the `manifest.yml`, "Job Composer" will assign the following default values: - ``name`` The name of the template folder. -- ``host`` The cluster id of the first cluster with a valid resource_mgr listed in the OOD cluster config +- ``host`` The cluster id of the first cluster with a valid resource_mgr listed in the OOD cluster configuration. - ``script`` The first ``.sh`` file appearing in the template folder. - ``notes`` The path to the location where a template manifest should be located. @@ -707,7 +707,7 @@ Job Composer Script Size Limit Since 1.7 the Job composer shows users 'Suggested file(s)' and 'Other valid file(s)'. Other valid files are _any_ files less than ``OOD_MAX_SCRIPT_SIZE_KB`` which defaults to 65 (meaning 65kb). -To reconfigure this, simply set the environment variable in the job composers' env file +To reconfigure this, simply set the environment variable in the job composers' environment file ``/etc/ood/config/apps/myjobs/env`` like so: .. code:: sh @@ -721,7 +721,7 @@ Hiding Job Arrays When composing a new job, the job arrays field is shown on supported clusters. To Hide this field even on supported clusters, an option was added. -To reconfigure this, simply set the environment variable in the job composers' env file +To reconfigure this, simply set the environment variable in the job composers' environment file ``/etc/ood/config/apps/myjobs/env`` like so: .. code:: sh @@ -768,7 +768,7 @@ When configured a widget like the one below will appear on the dashboard's landi The configuration for what apps to pin allows for three variants. You can configure specific apps with a string of the type ``router/app_name``. -For example ``sys/jupyter`` is the system installed app named jupyter. +For example ``sys/jupyter`` is the system installed app named ``jupyter``. Secondly you can configure globs like ``sys/*`` to pin all system installed apps. Or Maybe ``sys/minimal_*`` to pin all system installed apps that begin with 'minimal'. @@ -829,13 +829,13 @@ from grouping by category): .. figure:: /images/grouped_pinned_apps.png One can also change the menu length in the 'App's menu item. If you've -pinned more than 6 apps and you want to them to show up in this dropdown +pinned more than 6 apps and you want to them to show up in this drop-down list, simply increase the length with the option below. .. code:: yaml # /etc/ood/config/ondemand.d/ondemand.yml - pinned_apps_menu_length: 6 # the default number of items in the dropdown menu list + pinned_apps_menu_length: 6 # the default number of items in the drop-down menu list pinned_apps_group_by: category # defaults to nil, no grouping Pinned Apps customizations @@ -857,7 +857,7 @@ All the values are optional and any set will override the default from the appli Text Line 2 Text Line 3 -The CSS for the pinned app tiles has been optimized to support upto three lines of text for the ``sub_caption`` property. +The CSS for the pinned app tiles has been optimized to support up to three lines of text for the ``sub_caption`` property. .. figure:: /images/custom_pinned_apps.png @@ -885,7 +885,7 @@ Simply drop new files into ``/etc/ood/config/apps/dashboard/views/widgets`` and in the configuration. These partial files can be any format Rails recognizes, notably ``.html`` or ``.html.erb`` extensions. -Also if you use subdirectories under widgets, they can be referenced by relative paths. For example +Also if you use sub-directories under widgets, they can be referenced by relative paths. For example ``views/widgets/cluster/_my_cluster_widget.html.erb`` would be referenced in the configuration as ``cluster/my_cluster_widget``. @@ -933,7 +933,7 @@ Initial translation dictionary files with defaults that work well for OSC and us #. Copy the translation dictionary file (or create a new file with the same structure of the keys you want to modify) to ``/etc/ood/config/locales/en.yml`` and modify that copy. #. If you want apps to look for these dictionary files in a different location than ``/etc/ood/config/locales/en.yml`` you can change the location by defining ``OOD_LOCALES_ROOT`` environment variable. -#. The default locale is "en". You can use a custom locale. For example, if you want the locale to be French, you can create a ``/etc/ood/config/locales/fr.yml`` and then configure the Dashboard to use this locale by setting the environment variable ``OOD_LOCALE=fr`` where the locale is just the name of the file without the extension. Do this in either the nginx_stage config or in the Dashboard and Job Composer env config file. +#. The default locale is "en". You can use a custom locale. For example, if you want the locale to be French, you can create a ``/etc/ood/config/locales/fr.yml`` and then configure the Dashboard to use this locale by setting the environment variable ``OOD_LOCALE=fr`` where the locale is just the name of the file without the extension. Do this in either the ``nginx_stage`` configuration or in the Dashboard and Job Composer environment configuration file. In each default translation dictionary file the values that are most site-specific (and thus relevant for change) appear at the top. @@ -951,7 +951,7 @@ In each default translation dictionary file the values that are most site-specif - `Job Composer`_ - ``jobcomposer`` * - ``/etc/ood/config/locales/en.yml`` - - All localizable apps will check this path, unless ``OOD_LOCALES_ROOT`` is set. + - All localize-able apps will check this path, unless ``OOD_LOCALES_ROOT`` is set. - Any .. warning:: @@ -1000,7 +1000,7 @@ Two messages related to file system usage that sites may want to change: Customize Text in the Job Composer's options form ................................................. -The OSC-default value for ``options_account_help`` says that the account field is optional unless a user is a member of multiple projects. +The OSC default value for ``options_account_help`` says that the account field is optional unless a user is a member of multiple projects. Items of note include what to call Accounts which might also be Charge Codes, or Projects. At OSC entering an account is optional unless a user is a member of multiple projects which is reflected in the default value for the string ``options_account_help``. @@ -1074,8 +1074,8 @@ is given in the following format: .. warning:: A block must be equal to 1 KiB for proper conversions. -Individual Fileset Quota -........................ +Individual File-set Quota +......................... If the quota is defined as a ``fileset`` quota, then it applies to all disk resources used underneath a given volume. This requires the object to be @@ -1099,7 +1099,7 @@ The format is given as: Where ``block_usage`` and ``file_usage`` are the disk resource usages attributed to the specified user only. -.. note:: For each user with resources under this fileset, the above object will be repeated with just ``user``, ``block_usage``, and ``file_usage`` changing. +.. note:: For each user with resources under this ``fileset``, the above object will be repeated with just ``user``, ``block_usage``, and ``file_usage`` changing. .. _balance-warnings-on-dashboard: @@ -1195,7 +1195,7 @@ You can do this by setting Open OnDemand in 'Maintenance Mode'. While Maintenance mode is active, Apache will not serve requests for paths outside the ``/public/maintenance/*`` wildcard. Instead, it will serve the ``/var/www/ood/public/maintenance/index.html`` file, which you can change or brand to be your own. Changes to this file will persist through upgrades. -Any assets (e.g., images, stylesheets, or javascript) needed by the HTML file should be placed +Any assets (e.g., images, stylesheets, or JavaScript) needed by the HTML file should be placed in the ``/var/www/ood/public/maintenance/`` directory. You can also put symbolic links into the ``/var/www/ood/public/maintenance/`` directory, if you want to reuse assets located elsewhere in your file system. @@ -1222,7 +1222,7 @@ These are the settings you'll need for this functionality. To start maintenance mode (and thus start serving this page) simply ``touch /etc/ood/maintenance.enable`` to create the necessary file. When your downtime is complete just remove the file and all the traffic will be served normally again. The existence of this file is what starts or stops maintenance -mode, not it's content, so you will not need to restart apache or modify it's config files for this to +mode, not it's content, so you will not need to restart apache or modify it's configuration files for this to take affect. @@ -1252,7 +1252,7 @@ Grafana must be configured to support embedded panels and at this time it is als The dashboard used by OSC is the `OnDemand Clusters `_ dashboard. -Settings used to access Grafana are configured in the cluster config. The following is an example from OSC: +Settings used to access Grafana are configured in the cluster configuration. The following is an example from OSC: .. code:: yaml @@ -1274,7 +1274,7 @@ Settings used to access Grafana are configured in the cluster config. The follo When viewing a dashboard in Grafana choose the panel you'd wish to display and select `Share`. Then choose the `Embed` tab which will provide you with the iframe URL that will need to be generated within OnDemand. -The time ranges and values for labels (eg: `var-cluster=`) will be autofilled by OnDemand. +The time ranges and values for labels (e.g.: `var-cluster=`) will be auto-filled by OnDemand. * ``orgId`` is the ``orgId`` query parameter * The dashboard ``name`` is the last segment of the URI before query parameters @@ -1296,7 +1296,7 @@ Batch connect session cards like this have links to the compute node on which th However, some sites may want to disable this feature because they do not allow ssh sessions on the compute nodes. -To disable this, simply set the environment variable in the dashboards' env file +To disable this, simply set the environment variable in the dashboards' environment file ``/etc/ood/config/apps/dashboard/env`` to a falsy value (0, false, off). .. code:: sh @@ -1404,26 +1404,26 @@ XDMoD Integration requires XDMoD 9+, OnDemand 1.8+, and the ability to facilitat Steps to enable the XDMoD reports in the OnDemand Dashboard: -#. Configure OnDemand with XDMoD host URL in PUN /etc/ood/config/nginx_stage.yml +#. Configure OnDemand with XDMoD host URL in PUN ``/etc/ood/config/nginx_stage.yml``. .. code-block:: yaml pun_custom_env: OOD_XDMOD_HOST: "https://xdmod.osc.edu" -#. Add OnDemand host as domain to XDMoD portal settings for CORS /etc/xdmod/portal_settings.ini +#. Add OnDemand host as domain to XDMoD portal settings for CORS ``/etc/xdmod/portal_settings.ini``. .. code-block:: none domains = "https://ondemand.osc.edu" -#. Configure identity provider to include OnDemand host in HTTP `Content-Security-Policy for frame-ancestors `_ since OnDemand uses iFrames to trigger SSO with XDMoD when a user logs in. Below is what we ensured Content-Security-Policy header for frame-ancestors was set to when configuring Keycloak: +#. Configure identity provider to include OnDemand host in HTTP `Content-Security-Policy for frame-ancestors `_ since OnDemand uses an iframe to trigger SSO with XDMoD when a user logs in. Below is what we ensured Content-Security-Policy header for frame-ancestors was set to when configuring Keycloak: .. code-block:: none frame-ancestors https://*.osc.edu 'self' -#. If you want the XDMoD links in the OnDemand Job Composer you also need to configure OnDemand with XDMoD resource id in each cluster config. For example, in the hpctoolset the resource_id for the hpc cluster is 1 in XDMoD, so we modify /etc/ood/config/clusters.d/hpc.yml to add a xdmod map to the custom map at the bottom of the file: +#. If you want the XDMoD links in the OnDemand Job Composer you also need to configure OnDemand with XDMoD resource id in each cluster configuration. For example, in the ``hpctoolset`` the resource_id for the HPC cluster is 1 in XDMoD, so we modify ``/etc/ood/config/clusters.d/hpc.yml`` to add a XDMoD map to the custom map at the bottom of the file: .. code-block:: yaml :emphasize-lines: 10- @@ -1485,7 +1485,7 @@ This feature is disabled behind a feature toggle. To enable it, set the configur For more information on how to configure properties, see :ref:`configuration documentation `. When enabled, the cancel button will appear for active sessions. -When the session is cancelled, the job will be cancelled in the scheduler, +When the session is canceled, the job will be canceled in the scheduler, the status will change to ``completed``, and the session card will be kept. For completed sessions, the system will only show the delete button. diff --git a/source/customizations/custom-pages.inc b/source/customizations/custom-pages.inc index abe031a1..00dd3b05 100644 --- a/source/customizations/custom-pages.inc +++ b/source/customizations/custom-pages.inc @@ -28,7 +28,7 @@ It will render a layout with the ``pinned_apps`` widget under the URL: ``/pun/sy .. figure:: /images/custom-pages-documentation.png :align: center -Links to the custom pages can be added the Help dropdown menu. +Links to the custom pages can be added the Help drop-down menu. To add links to the Help menu use the configuration property ``help_menu``. See the :ref:`documentation on adding urls to the Help menu ` for details and examples. diff --git a/source/customizations/main-navigation.inc b/source/customizations/main-navigation.inc index ac1b746d..3d26d1cd 100644 --- a/source/customizations/main-navigation.inc +++ b/source/customizations/main-navigation.inc @@ -18,7 +18,7 @@ Understanding the default navigation bar ........................................ Some navigation menus and links are built-in to the system. The system -will generate dropdown menus for ``Files``, ``Jobs``, ``Clusters`` and +will generate drop-down menus for ``Files``, ``Jobs``, ``Clusters`` and ``Interactive Apps`` automatically. It will also generate the link for ``My Interactive Sessions`` by default. @@ -26,7 +26,7 @@ Here's an image of you'd expect to see by default. .. figure:: /images/main-navigation-menu.png :align: center - :alt: An image of the default navigation bar showing dropdown menus for files, jobs, clusters, interactive apps, and my interactive apps. + :alt: An image of the default navigation bar showing drop-down menus for files, jobs, clusters, interactive apps, and my interactive apps. Open OnDemand is a platform for other applications, like Jupyter & RStudio. As such, the default navigation bar builds itself based on what applications @@ -34,11 +34,11 @@ it has discovered. The system will search ``/var/www/ood/apps/sys/`` for these applications. Every application should have :ref:`app-development-manifest`. The system reads -these manifests files and builds additional dropdown menus (or adds to existing menus) +these manifests files and builds additional drop-down menus (or adds to existing menus) based on the ``category`` field. -By default, the system will generate a new dropdown menu for every ``category`` -it encounters. Within each dropdown menu it will add separators for each +By default, the system will generate a new drop-down menu for every ``category`` +it encounters. Within each drop-down menu it will add separators for each ``subcategory`` it encounters. .. _limit-auto-generated-menu-bars: @@ -46,10 +46,10 @@ it encounters. Within each dropdown menu it will add separators for each Limit auto generated menu bars .............................. -As described in the previous section, the system will generate new dropdown +As described in the previous section, the system will generate new drop-down menus for every ``category`` it discovers. -If you wish to add or remove which categories create dropdown menus +If you wish to add or remove which categories create drop-down menus use the :ref:`nav_categories configuration property `. .. tip:: @@ -69,24 +69,24 @@ Using manifests to create menu items .................................... As described in :ref:`understanding-default-nav-bar`, the system automatically -creates dropdown menus based on the applications it encounters. +creates drop-down menus based on the applications it encounters. -So, with that knowledge, one can create or add to dropdown menus by creating +So, with that knowledge, one can create or add to drop-down menus by creating applications with the correct information in their `manifest.yml `_. It's fairly simple to make this application. Simply create the directory in the correct location and create a `manifest.yml `_. -* sudo mkdir /var/www/ood/apps/sys/ -* sudo vi /var/www/ood/apps/sys//manifest.yml +* ``sudo mkdir /var/www/ood/apps/sys/`` +* ``sudo vi /var/www/ood/apps/sys//manifest.yml`` .. code-block:: yaml --- - # This is the text that will show up in the dropdown menu. + # This is the text that will show up in the drop-down menu. name: Add Menu Items to the navbar - # This will create the 'Links' dropdown menu if it doesn't already exist. + # This will create the 'Links' drop-down menu if it doesn't already exist. category: Links description: |- A description of what the menu item does. @@ -111,7 +111,7 @@ through the ``nav_bar`` and ``help_bar`` :ref:`ondemand-d-ymls` configuration pr .. warning:: As described in :ref:`understanding-default-nav-bar`, the system auto-generates - dropdown menus based on the applications it discovers. If the ``nav_bar`` or + drop-down menus based on the applications it discovers. If the ``nav_bar`` or ``help_bar`` configurations are set, the system will **only** show what you've defined. Before we look at how to modify it in detail, let's look at what the default @@ -121,7 +121,7 @@ configuration would be (without showing any discovered apps). # 'nav_bar' is the left side of the navigation bar. nav_bar: - # 'apps' dropdown menu is shown if you've set 'pinned_apps'. + # 'apps' drop-down menu is shown if you've set 'pinned_apps'. - apps - files - jobs @@ -157,12 +157,12 @@ Menus by Application category One of the easiest ways to create menus is to create them based off of the applications categories. -The title of the dropdown menu will be the ``category`` itself, and menu items +The title of the drop-down menu will be the ``category`` itself, and menu items (applications) will be grouped by their ``subcategory``. It is not possible to set an icon. The ``category`` matching is case insensitive. Here's an example using the ``interactive apps`` ``category``. Just like the default -navigation bar, this will generate a dropdown menu of all the applications the system +navigation bar, this will generate a drop-down menu of all the applications the system has discovered in that ``category``. As you can see from the image, the system has discovered eight applications in two @@ -175,7 +175,7 @@ subcategories. .. figure:: /images/def-category-menu.png :align: center - :alt: An open dropdown menu titled "Interactive Apps". The first submenu is titled "Desktops" and has two list items. The second submenu is titled "Servers" and has six list items. + :alt: An open drop-down menu titled "Interactive Apps". The first sub-menu is titled "Desktops" and has two list items. The second sub-menu is titled "Servers" and has six list items. Predefined templates ********************* @@ -188,8 +188,8 @@ A complete list of predefined templates is: * ``all_apps`` - The link to the all applications page. * ``apps`` - A drop down menu of ``pinned_apps``. Note that you need to set ``pinned_apps`` for this to show up. * ``sessions`` - The link to "My Interactive Sessions" page. -* ``develop`` - The dropdown menu for developers, titled "Develop". -* ``help`` - The "Help" dropdown menu. +* ``develop`` - The drop-down menu for developers, titled "Develop". +* ``help`` - The "Help" drop-down menu. * ``log_out`` - The link to logout, titled "Logout". * ``user`` - The "Logged in as " text. @@ -208,14 +208,14 @@ Here's an example of defining all of them and what that would look like. .. figure:: /images/def-templates-menu.png :align: center - :alt: An image of the navigation bar. From left to right the items are "All Apps" link, "Apps" dropdown menu, "My Interactive Sessions" link, "Develop" drop down menu, "Help" drop down menu, "Logout" link and 'logged in as ood' text. + :alt: An image of the navigation bar. From left to right the items are "All Apps" link, "Apps" drop-down menu, "My Interactive Sessions" link, "Develop" drop down menu, "Help" drop down menu, "Logout" link and 'logged in as ood' text. .. _menus-based-on-applications: Menus based on applications *************************** -You can create dropdown menus based on applications the system can discover. +You can create drop-down menus based on applications the system can discover. Menu items are automatically grouped based on the applications ``subcategory``. Use the ``apps`` array to define the app tokens for this navigation menu. @@ -233,7 +233,7 @@ The order of the menu items is based on the order you've defined. .. figure:: /images/def-apps-menu.png :align: center - :alt: An open dropdown menu titled "My Apps". The menu shows two submenus. The first is titled "Servers" and has five list items for all the discovered through the "sys/Rstudio/*" configuration. The second submenu is titled "Desktops" and has one list item for the discovered through the "sys/bc_desktop" configuration. + :alt: An open drop-down menu titled "My Apps". The menu shows two sub-menus. The first is titled "Servers" and has five list items for all the discovered through the "sys/Rstudio/*" configuration. The second sub-menu is titled "Desktops" and has one list item for the discovered through the "sys/bc_desktop" configuration. .. _menus-based-on-links: @@ -241,17 +241,17 @@ The order of the menu items is based on the order you've defined. Menus based on links ******************** -Menus based on links provide even more granular control on how the dropdown menu is generated. +Menus based on links provide even more granular control on how the drop-down menu is generated. As seen in the previous sections, using an ``apps`` array will automatically generate -submenus based on the ``subcategory`` of the applications that have been discovered. +sub-menus based on the ``subcategory`` of the applications that have been discovered. -When using links, we can override this behaviour to provide different titles to -the submenus, or with no submenus and subtitles at all. +When using links, we can override this behavior to provide different titles to +the sub-menus, or with no sub-menus and subtitles at all. Let's look at this example using the same application tokens, but using ``links`` instead of ``apps``. The system will discover and generate list items for the same eight applications, -only without **any** submenus. It is simply a list of all the discovered applications with +only without **any** sub-menus. It is simply a list of all the discovered applications with no further grouping. .. code-block:: yaml @@ -265,15 +265,15 @@ no further grouping. .. figure:: /images/app-links-no-group.png :align: center - :alt: An open dropdown menu titled "My Apps". The menu has no submenus. It simply lists all eight the apps. + :alt: An open drop-down menu titled "My Apps". The menu has no sub-menus. It simply lists all eight the apps. -To add submenus, simply use the ``group`` attribute. +To add sub-menus, simply use the ``group`` attribute. When ``links`` are used with the ``group`` attribute, the discovered applications -will now be grouped and submenu titles created based on the configuration provided. +will now be grouped and sub-menu titles created based on the configuration provided. This will look and feel similar to `menus-based-on-applications`_ with the exception -being that submenus are defined by the ``group`` configuration instead of the ``subcategory`` +being that sub-menus are defined by the ``group`` configuration instead of the ``subcategory`` of the applications in the list. .. code-block:: yaml @@ -289,7 +289,7 @@ of the applications in the list. .. figure:: /images/def-full-menu.png :align: center - :alt: An image showing one open drop down menu titled "My Apps". The first submenu, titled "Rstudio" has five list items that have been discovered through the "sys/Rstudio/*" configuration. The second submenu, titled "Remote" has one list item that has been discovered through the "sys/bc_desktop" configuration. + :alt: An image showing one open drop down menu titled "My Apps". The first sub-menu, titled "Rstudio" has five list items that have been discovered through the "sys/Rstudio/*" configuration. The second sub-menu, titled "Remote" has one list item that has been discovered through the "sys/bc_desktop" configuration. .. tip:: You can also used predefined links (described above) in the ``links`` section here. @@ -372,4 +372,4 @@ through the ``url`` attribute. .. figure:: /images/def-navigation-link.png :align: center - :alt: An image of the navigation bar with a single link "Title" and an open dropdown menu titled "Menu Item". The dropdown menu has one list item titled "Title". + :alt: An image of the navigation bar with a single link "Title" and an open drop-down menu titled "Menu Item". The drop-down menu has one list item titled "Title". diff --git a/source/customizations/profiles.inc b/source/customizations/profiles.inc index d69ab9f4..f96b44ca 100644 --- a/source/customizations/profiles.inc +++ b/source/customizations/profiles.inc @@ -51,7 +51,7 @@ The selection of a profile can be done manually by updating the user settings pr Profile selection can be done using configurable navigation links. These links will update the profile, save the newly selected profile in the user settings file and reload the Dashboard. -To add profile links to the Help navigation dropdown menu use the configuration property ``profile_links``. +To add profile links to the Help navigation drop-down menu use the configuration property ``profile_links``. For more information on how to add profile links to the navigation bar in various places, see :ref:`menus for profile links `. @@ -100,7 +100,7 @@ Automatic Profile Selection .. tip:: If you have a need to create multiple profile definitions that have the same configuration, you could use aliases. - This is a feaure of YAML files. You can use YAML ``anchors`` and ``aliases`` to set the same profile configuration + This is a feature of YAML files. You can use YAML ``anchors`` and ``aliases`` to set the same profile configuration to more than one definition without duplication. This YAML feature only works when defined and used within the same file. The example below creates 4 profiles all with the same configuration: diff --git a/source/installation/add-cluster-config.rst b/source/installation/add-cluster-config.rst index ffdd6e98..5a32c070 100644 --- a/source/installation/add-cluster-config.rst +++ b/source/installation/add-cluster-config.rst @@ -68,7 +68,7 @@ client binaries. .. toctree:: :maxdepth: 1 - :caption: Cluster Config + :caption: Cluster Configuration cluster-config-schema resource-manager/torque diff --git a/source/installation/cluster-config-schema.rst b/source/installation/cluster-config-schema.rst index 1d2fa332..2f9b8e96 100644 --- a/source/installation/cluster-config-schema.rst +++ b/source/installation/cluster-config-schema.rst @@ -1,9 +1,9 @@ .. _cluster-config-schema: -Cluster Config Schema v2 -======================== +Cluster Configuration Schema v2 +=============================== -The cluster config controls many OnDemand features including job submission, shell access, names in menus. +The cluster configuration controls many OnDemand features including job submission, shell access, names in menus. First an example: ***************** @@ -134,8 +134,8 @@ bin_overrides: Adapter support for this feature is mixed. For example for Slurm `sbatch`, `scontrol`, `scancel` and `squeue` are all supported. For Torque only `qsub` is supported. Unsupported options are ignored. -acls: -##### +``acls``: +######### .. warning:: @@ -144,7 +144,7 @@ acls: are being handled in the Linux kernel, is much faster, simpler and frankly, safer. Access control lists provide a method to limit cluster access by group membership. -ACLs are implicitly allowlists but may be set explicitly to either `allowlist` or `blocklist`. +ACLs are implicitly allow-lists but may be set explicitly to either `allowlist` or `blocklist`. .. code-block :: yaml @@ -196,7 +196,7 @@ Suppose you need want to create a *login cluster that does not schedule or run j To accomplish this, you need to simply leave out the ``v2.job`` stanza that associates a scheduler with the cluster. -An example config file in ``ondemand.d/pitzer_01_login.yml``: +An example configuration file in ``clusters.d/pitzer_01_login.yml``: .. code-block:: yaml @@ -209,5 +209,5 @@ An example config file in ``ondemand.d/pitzer_01_login.yml``: login: host: "pitzer-login01.hpc.osu.edu" -Again, the thing to note here is we've left off the ``v2.job`` which renders the cluster useable only for logins, i.e. +Again, the thing to note here is we've left off the ``v2.job`` which renders the cluster use-able only for logins, i.e. *no jobs will be scheduleable on this cluster.* diff --git a/source/installation/install-software.rst b/source/installation/install-software.rst index bac97137..025ca7d6 100644 --- a/source/installation/install-software.rst +++ b/source/installation/install-software.rst @@ -8,7 +8,7 @@ Open OnDemand uses these packages, among many others. - Apache HTTP Server 2.4 - Ruby 3.3 with :command:`rake`, :command:`bundler`, and development libraries -- Node.js 20 +- NodeJs 20 .. note:: diff --git a/source/installation/modify-system-security.rst b/source/installation/modify-system-security.rst index eb7443fb..4c7ef38e 100644 --- a/source/installation/modify-system-security.rst +++ b/source/installation/modify-system-security.rst @@ -20,7 +20,7 @@ SELinux The OnDemand SELinux package makes several changes to allow OnDemand to run with SELinux enabled. -* Set contexts of several filesystem paths specific to OnDemand. +* Set contexts of several file-system paths specific to OnDemand. * Enable SELinux booleans. * Apply a custom policy to allow actions to performed by `ood_pun_t` context. @@ -40,12 +40,12 @@ The custom SELinux booleans provided by the OnDemand SELinux policy: The following SELinux booleans are enabled by the ``ondemand-selinux`` package: -* httpd_setrlimit -* httpd_mod_auth_pam -* httpd_run_stickshift -* httpd_can_network_connect -* daemons_use_tty -* use_nfs_home_dirs (can be disabled if the OnDemand web node is not using NFS for home directories) +* ``httpd_setrlimit`` +* ``httpd_mod_auth_pam`` +* ``httpd_run_stickshift`` +* ``httpd_can_network_connect`` +* ``daemons_use_tty`` +* ``use_nfs_home_dirs`` (can be disabled if the OnDemand web node is not using NFS for home directories) The following example disabled the OnDemand SSH SELinux boolean. diff --git a/source/installation/resource-manager/ccq.rst b/source/installation/resource-manager/ccq.rst index 68fe514c..5497084e 100644 --- a/source/installation/resource-manager/ccq.rst +++ b/source/installation/resource-manager/ccq.rst @@ -30,21 +30,21 @@ cluster looks like: with the following configuration options: -adapter +``adapter`` This is set to ``ccq``. -image +``image`` The default cloud image to use when launching jobs. There is no default. -cloud +``cloud`` The cloud provider being used. Valid options are ``gcp`` or ``aws``. Defaults to ``gcp``. -scheduler +``scheduler`` The name of the scheduler being used. There is no default. -bin +``bin`` The path to the CCQ client installation binaries. Defaults to ``/opt/CloudyCluster/srv/CCQ``. -jobid_regex - The regular expression to extract the job id from the ccqstat output. Defaults to ``job id is: (?\\d+) you``. - You should only need this if the ccqstat output changes format. If you are required to reconfigure, you'll need to +``jobid_regex`` + The regular expression to extract the job id from the ``ccqstat`` output. Defaults to ``job id is: (?\\d+) you``. + You should only need this if the ``ccqstat`` output changes format. If you are required to reconfigure, you'll need to extract the named group ``job_id`` as the default does. -bin_overrides +``bin_overrides`` Replacements/wrappers for CCQ's job submission and control clients. *Optional* Supports the following clients: @@ -64,7 +64,7 @@ You may see this error when you initially try to start a job. .. code-block:: text The /opt/CloudyCluster/srv/CCQ/ccqsub command was prompted. You need - to generate the certificate manually in a shell by running 'ccqstat' + to generate the certificate manually in a shell by running ``ccqstat`` and entering your username/password This is because CCQ libraries require a certificate to be generated to communicate with the diff --git a/source/installation/resource-manager/kubernetes.rst b/source/installation/resource-manager/kubernetes.rst index bba6afa0..abe62501 100644 --- a/source/installation/resource-manager/kubernetes.rst +++ b/source/installation/resource-manager/kubernetes.rst @@ -39,37 +39,37 @@ cluster looks like: ssh_allow: false -adapter +``adapter`` This is set to ``kubernetes``. -config_file - The KUBECONFIG file. *Optional*. Defaults to ~/.kube/config. Sites can also - set the KUBECONFIG environment varaible, but this configuration has precedence. -cluster +``config_file`` + The KUBECONFIG file. *Optional*. Defaults to ``~/.kube/config``. Sites can also + set the KUBECONFIG environment variable, but this configuration has precedence. +``cluster`` The cluster name. Saved to and referenced from your KUBECONFIG. -context +``context`` The context to use when issuing kubectl commands. *Optional*. Defaults to cluster when using OIDC authentication. Saved to and referenced from your KUBECONFIG. -username_prefix +``username_prefix`` The prefix to your users in your KUBECONFIG. Use this prefix to differentiate between different clusters (like test and production). -namespace_prefix +``namespace_prefix`` The prefix to your namespace. Use this prefix if you have assertions on what namespaces are available. I.e., a Kyverno policy that ensures all namespaces are ``user-\w+``. -all_namespaces +``all_namespaces`` A boolean to determine if the user will query for pods in other namespaces. When false users will only query in their namespace. If true they will query and display pods from all namespaces. -auto_supplemental_groups: +``auto_supplemental_groups`` Automatically populate a container's ``securityContext.supplementalGroups`` with the users supplemental groups. -server - The kubernetes server to communicate with. This field is a map with ``endpoint`` and +```server`` + The Kubernetes server to communicate with. This field is a map with ``endpoint`` and ``cert_authority_file`` keys. -auth: +``auth`` See the notes on `Authentication`_ below. -mounts: - Site wide mount points for all kubernetes jobs. See the - :ref:`documentation on kubernetes mounts ` for more details. +``mounts`` + Site wide mount points for all Kubernetes jobs. See the + :ref:`documentation on Kubernetes mounts ` for more details. .. note:: @@ -79,7 +79,7 @@ mounts: Per User Kubernetes ******************* -To get kubernetes to act like a Per User resource there are some conventions +To get Kubernetes to act like a Per User resource there are some conventions we put in place. Users only schedule pods in their own namespaces and they always run those pods as themselves. @@ -91,9 +91,9 @@ viewing active jobs and seeing pods from other namespaces in that view. Second is that we specify the `Kubernetes security context`_ such that pods have the same UID and GID as the actual user. -Open OnDemand will always use the users UID and GID as the runAsUser and runAsGroup. -fsGroup is always the same as runAsGroup. runAsNonRoot is always set to true. -supplementalGroups are empty by default. One can automatically populate them with a +Open OnDemand will always use the users UID and GID as the ``runAsUser`` and ``runAsGroup``. +``fsGroup`` is always the same as ``runAsGroup``. ``runAsNonRoot`` is always set to true. +``supplementalGroups`` are empty by default. One can automatically populate them with a cluster configuration above or specify them for each app individually. You should have policies in place to enforce these. @@ -102,7 +102,7 @@ Bootstrapping the Kuberenetes cluster ************************************* Before anyone can use your Kubernetes cluster from Open OnDemand, you'll need -to create the `open ondemand kubernetes resources`_ on your cluster. +to create the `open ondemand Kubernetes resources`_ on your cluster. Below is an example of adding the necessary resources: @@ -115,11 +115,11 @@ Bootstrapping OnDemand web node to communicate with Kubernetes ************************************************************** The OnDemand web node ``root`` user must be configured -to use the ``ondemand`` service account deployed by the `open ondemand kubernetes resources`_ and +to use the ``ondemand`` service account deployed by the `open ondemand Kubernetes resources`_ and be able to execute ``kubectl`` commands. First deploy ``kubectl`` to the OnDemand web node. -Replace ``$VERSION`` with the version of the Kubernetes controller, eg. ``1.21.5``. +Replace ``$VERSION`` with the version of the Kubernetes controller, e.g., ``1.21.5``. .. code-block:: sh @@ -167,7 +167,7 @@ to refresh your tokens before they expire. If you wish to create a non-expiring token, you will need to create the secret through a -``kubectl apply`` command on the yaml below. +``kubectl apply`` command on the YAML below. Next extract the ``ondemand`` ServiceAccount token. Here is an example command to extract the token using an account that has ClusterAdmin privileges: @@ -190,7 +190,7 @@ the token using an account that has ClusterAdmin privileges: TOKEN=$(kubectl describe serviceaccount ondemand -n ondemand | grep Tokens | awk '{ print $2 }') kubectl describe secret $TOKEN -n ondemand | grep "token:" -Below are example commands to bootstrap the kubeconfig for ``root`` user on the OnDemand web node +Below are example commands to bootstrap the Kubernetes configuration for ``root`` user on the OnDemand web node using the token from above. Run these commands as ``root`` on the OnDemand web node. .. code-block:: sh @@ -206,7 +206,7 @@ Replace the following values: - ``$CACERT`` the path to Kubernetes cluster CA cert - ``$TOKEN`` the token for ``ondemand`` ServiceAccount -Below is an example of verifying the kubeconfig is valid: +Below is an example of verifying the Kubernetes configuration is valid: .. code-block:: sh @@ -217,7 +217,7 @@ Deploy Hooks to bootstrap users Kubernetes configuration We ship with `open ondemand provided hooks`_ to bootstrap users when the login to Open OnDemand. These scripts will create their namespace, a networking policy, -and rolebindings for user and the service accounts in their namespace. +and role-bindings for user and the service accounts in their namespace. A user ``oakley`` would create the ``oakley`` namespace. If you've configured to use prefix ``user-``, then the namespace would be ``user-oakley``. @@ -225,7 +225,7 @@ to use prefix ``user-``, then the namespace would be ``user-oakley``. The networking policy ensures that pods cannot communicate inbetween namespaces. The RoleBindings give user, ``oakley`` in this case, sufficient privileges -to the ``oakley`` namespace. Refer to the `open ondemand kubernetes resources`_ +to the ``oakley`` namespace. Refer to the `open ondemand Kubernetes resources`_ for details on the roles and privileges created. You'll need to employ :ref:`PUN pre hooks ` @@ -264,7 +264,7 @@ You can refer to `osc's prehook`_ but we'll also provide this example. As you can see in this pre hook, the username is passed in to the script which then defines the ``HOOKENV`` and calls two `open ondemand provided hooks`_. -``k8s-bootstrap-ondemand.sh`` boostraps the user in the kubernetes cluster as described +``k8s-bootstrap-ondemand.sh`` boostraps the user in the Kubernetes cluster as described above. Since we use OIDC at OSC we use ``set-k8s-creds.sh`` to add or update the user in their @@ -321,7 +321,7 @@ is managed outside of Open OnDemand. We will not ``set-context`` or ``set-cluster``. We will pass ``--context`` to kubectl commands if you have it configured -in the cluster config (above). Otherwise, it's assumed that the current context +in the cluster configuration (above). Otherwise, it's assumed that the current context is set out of bounds. OIDC Authentication @@ -345,7 +345,7 @@ Another approach would be to use the same OIDC client configuration for OnDemand auth: type: 'odic' -This uses the OIDC credentails that you've logged in with. When +This uses the OIDC credentials that you've logged in with. When the dashboard starts up it will ``set-context`` and ``set-cluster`` to what you've configured. @@ -367,13 +367,13 @@ GKE Authentication svc_acct_file: '~/.gke/my-service-account-file' It's expected that you have a service account that can then manipulate -the cluster you're interacting with. Every user should have a cooresponding +the cluster you're interacting with. Every user should have a corresponding service account to interact with GKE. -When the dasbhoard starts up, we use ``gcloud`` to configure your KUBECONFIG. +When the dashboard starts up, we use ``gcloud`` to configure your KUBECONFIG. -Google Cloud's Goolge Kubernetes Engine (GKE) needs some more documentation -on what privileges this serivce account is setup with and how one may bootstrap +Google Cloud's Google Kubernetes Engine (GKE) needs some more documentation +on what privileges this service account is setup with and how one may bootstrap it. .. _oidc_k8_audience: @@ -412,19 +412,19 @@ Open OnDemand apps in a Kuberenetes cluster ******************************************* Kuberenetes is so different from other HPC clusters that the interface we have for -other schedulers didn't quite fit. So Open OnDemand apps developed for kubernetes +other schedulers didn't quite fit. So Open OnDemand apps developed for Kubernetes clusters look quite different from other schedulers. Essentially most things we'll need are packed into the ``native`` key of the ``submit.yml.erb`` files. -See the :ref:`tutorial for a kubernetes app that behaves like HPC compute ` as well as -the :ref:`tutorial for a kubernetes app ` +See the :ref:`tutorial for a Kubernetes app that behaves like HPC compute ` as well as +the :ref:`tutorial for a Kubernetes app ` for more details. Kyverno Policies **************** -Once Kubernetes is available to OnDemand, it's possible for users to use ``kubectl`` to submit arbirary pods to +Once Kubernetes is available to OnDemand, it's possible for users to use ``kubectl`` to submit arbitrary pods to Kubernetes. To ensure proper security with Kubernetes a policy engine such as `Kyverno`_ can be used to ensure certain security standards. @@ -529,7 +529,7 @@ a user's namespaces with the ability to pull from this registry: .. _kubernetes security context: https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#security-context .. _open ondemand provided hooks: https://github.com/OSC/ondemand/tree/master/hooks -.. _open ondemand kubernetes resources: https://github.com/OSC/ondemand/blob/master/hooks/k8s-bootstrap/ondemand.yaml +.. _open ondemand Kubernetes resources: https://github.com/OSC/ondemand/blob/master/hooks/k8s-bootstrap/ondemand.yaml .. _osc's prehook: https://github.com/OSC/osc-ood-config/blob/master/hooks/pre-hook.sh .. _kyverno: https://kyverno.io .. _kyverno baseline and restricted sescurity policies: https://github.com/kyverno/kyverno/tree/main/charts/kyverno-policies/templates diff --git a/source/installation/resource-manager/linuxhost.rst b/source/installation/resource-manager/linuxhost.rst index d5ca0d37..ca0894e1 100644 --- a/source/installation/resource-manager/linuxhost.rst +++ b/source/installation/resource-manager/linuxhost.rst @@ -12,15 +12,15 @@ The LinuxHost adapter facilitates launching jobs immediately without using a bat The adapter pieces together several tools to achieve behavior similar to what a scheduler offers: -ssh +``ssh`` connects from the web node to a configured host such as a login node. -tmux +``tmux`` Specially named ``tmux`` sessions offer the ability to rediscover running jobs -timeout +``timeout`` is used to set a 'walltime' after which the job is killed -pstree +``pstree`` is used to detect the job's parent ``sinit`` process so that it can be killed -singularity +``singularity`` containerization provides a PID namespace without requiring elevated privileges that ensures that all child processes are cleaned up when the job either times out or is killed .. note:: There are many recipes for managing processes on arbitrary Linux hosts, and we will be exploring others in future development of this adapter. @@ -97,25 +97,25 @@ A YAML cluster configuration file for a Linux host looks like: with the following configuration options: -adapter +``adapter`` This is set to ``linux_host``. -submit_host +``submit_host`` The target execution host for jobs. May be the head for a login round robin. May also be "localhost". -ssh_hosts +``ssh_hosts`` All nodes the submit_host can DNS resolve to. -site_timeout +``site_timeout`` The number of seconds that a user's job is allowed to run. Distinct from the length of time that a user selects. -debug +``debug`` When set to ``true`` job scripts are written to ``$HOME/tmp.UUID_tmux`` and ``$HOME/tmp.UUID_sing`` for debugging purposes. When ``false`` those files are written to ``/tmp`` and deleted as soon as they have been read. -singularity_bin +``singularity_bin`` The absolute path to the ``singularity`` executable on the execution host(s). -singularity_bindpath +``singularity_bindpath`` The comma delimited list of paths to bind mount into the host; cannot simply be ``/`` because Singularity expects certain dot files in its containers' root; defaults to: ``/etc,/media,/mnt,/opt,/run,/srv,/usr,/var,/users``. -singularity_image - The absolute path to the Singularity image used when simply PID namespacing jobs; expected to be a base distribution image with no customizations. -strict_host_checking +``singularity_image`` + The absolute path to the Singularity image used to create PID namespaces; expected to be a base distribution image with no customizations. +``strict_host_checking`` When ``false`` the SSH options include ``StrictHostKeyChecking=no`` and ``UserKnownHostsFile=/dev/null`` this prevents jobs from failing to launch. -tmux_bin +``tmux_bin`` The absolute path to the ``tmux`` executable on the execution host(s). @@ -125,14 +125,14 @@ tmux_bin software on the target host, not launching specific application containers. As a result, even if your use of this adapter is reserved to launching specific application containers, you currently must specify a value in the - cluster config for ``singularity_bindpath`` and ``singularity_image``, even + cluster configuration for ``singularity_bindpath`` and ``singularity_image``, even if these will be specified in each interactive app plugin. .. note:: In order to communicate with the execution hosts the adapter uses SSH in ``BatchMode``. The adapter does not take a position on whether authentication - is performed by user owned passwordless keys, or host-based authentication; + is performed by user owned password-less keys, or host-based authentication; however OSC has chosen to provide `host based authentication `_ to its users. @@ -155,7 +155,7 @@ First update the PAM stack to include the following line: session required pam_exec.so type=open_session /etc/security/limits.sh -This goes into a file used by the ``sshd`` PAM configs which on CentOS/RHEL default to ``/etc/pam.d/password-auth-ac`` and needs to be included in the proper position, after ``pam_systemd.so``. Also set ``pam_systemd.so`` to ``required``: +This goes into a file used by the ``sshd`` PAM configurations which on CentOS/RHEL default to ``/etc/pam.d/password-auth-ac`` and needs to be included in the proper position, after ``pam_systemd.so``. Also set ``pam_systemd.so`` to ``required``: .. code-block:: none :emphasize-lines: 3,4 @@ -185,10 +185,10 @@ The following example of ``/etc/security/limits.sh`` is used by OSC on interacti CPUQuota=700% fi -Approach #2: libcgroup cgroups -.............................. +Approach #2: ``libcgroup`` cgroups +................................... -The libcgroup cgroups rules and configurations are a per-group resource limit where the group is defined in the examples at ``/etc/cgconfig.d/limits.conf``. The following examples limit resources of all tmux processes launched for the LinuxHost Adapter so they all share 700 CPU shares and 64GB of RAM. This requires setting ``tmux_bin`` to a wrapper script that in this example will be ``/usr/local/bin/ondemand_tmux``. +The ``libcgroup`` cgroups rules and configurations are a per-group resource limit where the group is defined in the examples at ``/etc/cgconfig.d/limits.conf``. The following examples limit resources of all ``tmux`` processes launched for the LinuxHost Adapter so they all share 700 CPU shares and 64GB of RAM. This requires setting ``tmux_bin`` to a wrapper script that in this example will be ``/usr/local/bin/ondemand_tmux``. Example of ``/usr/local/bin/ondemand_tmux``: @@ -260,7 +260,7 @@ error while loading shared libraries .................................... The default mounts for singularity are ``'/etc,/media,/mnt,/opt,/srv,/usr,/var,/users'``. It's likely -either you've overwritten this with too few mounts (like /lib, /opt or /usr) or your container lacks +either you've overwritten this with too few mounts (like ``/lib``, ``/opt`` or ``/usr``) or your container lacks the library in question. If the library exists on the host, consider mounting it into the container. Otherwise install it in @@ -285,7 +285,7 @@ Enable this, and you'll see the two shell scripts that ran during this job. Open Export the SINGULARITY_BINDPATH so you're sure to have the same mounts, and run this ``/usr/bin/singularity exec ... tmp.73S0QFxC5e_sing`` command manually on one of the ssh hosts. This will -emulate what the linuxhost adapter is doing and you should be able to modify and rerun until you fix +emulate what the ``linuxhost`` adapter is doing and you should be able to modify and rerun until you fix the issue. @@ -310,7 +310,7 @@ Again, mounting ``var`` fixed this error too. .. note:: - Subsequent versions of the adapter are expected to use `unshare `_ for PID namespacing as the default method instead of Singularity. Singularity will continue to be supported. + Subsequent versions of the adapter are expected to use `unshare `_ for PID namespaces as the default method instead of Singularity. Singularity will continue to be supported. diff --git a/source/installation/resource-manager/lsf.rst b/source/installation/resource-manager/lsf.rst index e22ed099..2820bdde 100644 --- a/source/installation/resource-manager/lsf.rst +++ b/source/installation/resource-manager/lsf.rst @@ -32,21 +32,21 @@ looks like: with the following configuration options: -adapter +``adapter`` This is set to ``lsf``. -bindir +``bindir`` The path to the LSF client ``bin/`` directory. -libdir +``libdir`` The path to the LSF client ``lib/`` directory. -envdir +``envdir`` The path to the LSF client ``conf/`` directory. -serverdir +``serverdir`` The path to the LSF client ``etc/`` directory. -bin_overrides +``bin_overrides`` Replacements/wrappers for LSF's job submission and control clients. *Optional* -submit_host +``submit_host`` A different, optional host to ssh to and *then* issue commands. *Optional* -cluster +``cluster`` The cluster to interact with when running LSF's multi-cluster mode. *Required to enable multi-cluster mode* Supports the following clients: @@ -71,7 +71,7 @@ Given these settings in ``lsf.conf``: LSB_QUERY_PORT=688X LSB_QUERY_ENH=y -It's recommented that you set ``LSB_MBD_PORT`` to the same value +It's recommended that you set ``LSB_MBD_PORT`` to the same value as ``LSB_QUERY_PORT`` It has also been observed that setting these values in ``lsb.params`` diff --git a/source/installation/resource-manager/sge.rst b/source/installation/resource-manager/sge.rst index aa568754..57cf620d 100644 --- a/source/installation/resource-manager/sge.rst +++ b/source/installation/resource-manager/sge.rst @@ -32,21 +32,21 @@ cluster looks like: with the following configuration options: -adapter +``adapter`` This is set to ``sge``. -cluster +``cluster`` The Grid Engine cluster name. *Optional* -bin +``bin`` The path to the Grid Engine client installation binaries. -sge_root +``sge_root`` The path to the root directory of the Grid Engine installation. *Default:* ``/var/lib/gridengine`` -conf +``conf`` The path to the Grid Engine configuration file for this cluster. *Optional* -libdrmaa_path - The full path to libdrmaa.so. Provide this to enable use of libdrmaa for more precise job status reporting. *Optional* -submit_host +``libdrmaa_path`` + The full path to ``libdrmaa.so``. Provide this to enable use of ``libdrmaa`` for more precise job status reporting. *Optional* +``submit_host`` A different, optional host to ssh to and *then* issue commands. *Optional* -bin_overrides +``bin_overrides`` Replacements/wrappers for Grid Engine's job submission and control clients. *Optional* Supports the following clients: @@ -59,7 +59,7 @@ bin_overrides .. tip:: - DRMAA improves OnDemand's ability to report on the precise status of jobs. To use this feature ensure that libdrmaa.so for Grid Engine is installed or built and set the config value for ``libdrmaa_path`` and ``sge_root``. If DRMAA is not installed then OnDemand is unable to get a precise job status for single jobs and will only return either queued or complete. + DRMAA improves OnDemand's ability to report on the precise status of jobs. To use this feature ensure that ``libdrmaa.so`` for Grid Engine is installed or built and set the configuration value for ``libdrmaa_path`` and ``sge_root``. If DRMAA is not installed then OnDemand is unable to get a precise job status for single jobs and will only return either queued or complete. Common Issues ------------- @@ -76,7 +76,7 @@ substitution and there's a syntax error. line 156: done < <(tail -f --pid=${SCRIPT_PID} “vnc.log”) &’ What you'll need to do is add a ``script_wrapper`` element to your clusters' configuration like below. -This sets the sh shell to behave like bash and ensures you've sourced your users' bashrc file. +This sets the sh shell to behave like bash and ensures you've sourced your users' ``.bashrc`` file. .. code-block:: yaml diff --git a/source/installation/resource-manager/slurm.rst b/source/installation/resource-manager/slurm.rst index 0c4de157..f92e3a6a 100644 --- a/source/installation/resource-manager/slurm.rst +++ b/source/installation/resource-manager/slurm.rst @@ -41,22 +41,22 @@ cluster looks like: with the following configuration options: -adapter +``adapter`` This is set to ``slurm``. -cluster +``cluster`` The Slurm cluster name. *Optional*, passed to SLURM as ``-M `` .. warning:: Using the ``cluster`` option is discouraged. This is because maintenance outages on the Slurm database will propagate to Open OnDemand. Instead sites should use different ``conf`` files for each cluster to limit maintenance outages. -bin +``bin`` The path to the Slurm client installation binaries. -conf +``conf`` The path to the Slurm configuration file for this cluster. *Optional* -submit_host +``submit_host`` A different, optional host to ssh to and *then* issue commands. *Optional* -bin_overrides +``bin_overrides`` Replacements/wrappers for Slurm's job submission and control clients. *Optional* Supports the following clients: @@ -66,8 +66,8 @@ bin_overrides - `scontrol` - `scancel` -copy_environment - Copies the environment of the PUN when issuing CLI commands. Default behaviour +``copy_environment`` + Copies the environment of the PUN when issuing CLI commands. Default behavior for Open OnDemand is to use ``--export=NONE`` flag. Setting this to true will cause Open OnDemand to issue CLI commands with ``--export=ALL``. Though this may cause issues as the PUN's environment is very different than a regular shell session. diff --git a/source/spelling_wordlist.txt b/source/spelling_wordlist.txt index faac15bf..864695ab 100644 --- a/source/spelling_wordlist.txt +++ b/source/spelling_wordlist.txt @@ -71,6 +71,7 @@ localhost Gemfile noVNC walltime +Walltimes GiB prepopulate dismissible @@ -104,6 +105,7 @@ Multiphysics Tensorboard deprecations httpd +https hostname hostnames virtualhosts @@ -118,6 +120,26 @@ TurboVNC lookups sudo grey +Truststore +SystemD +kubectl +Kyverno +Kuberenetes +crontab +cronjob +inbetween +boostraps +cgroup +Virtualbox +Kerberos +ToS +Gib +KiB +kb +stylesheets +iframe +falsy +rclone Aday Wisniewski