From e97a0e7a1c0fbac53d479b690808a64205e38e47 Mon Sep 17 00:00:00 2001 From: Ria Gupta <74742605+rg663@users.noreply.github.com> Date: Wed, 19 Jul 2023 11:54:17 -0400 Subject: [PATCH 1/3] remove extra backtick that creates issues when running copied command --- coldfront/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/coldfront/README.md b/coldfront/README.md index 066feba..97f4f86 100644 --- a/coldfront/README.md +++ b/coldfront/README.md @@ -117,7 +117,7 @@ cat ~/slurm_dump/hpc.cfg - Load the slurm dump into slurm database: ``` -sacctmgr load file=~/slurm_dump/hpc.cfg` +sacctmgr load file=~/slurm_dump/hpc.cfg Type 'Y' to add the new account & associations for cgray ``` From e1e06c24159573b041c3fda780ad287df9cd686d Mon Sep 17 00:00:00 2001 From: Ria Gupta <74742605+rg663@users.noreply.github.com> Date: Wed, 19 Jul 2023 11:58:34 -0400 Subject: [PATCH 2/3] additional removal of extra backtick --- coldfront/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/coldfront/README.md b/coldfront/README.md index 97f4f86..6a8d9bd 100644 --- a/coldfront/README.md +++ b/coldfront/README.md @@ -145,7 +145,7 @@ Switch over to the PI user account and try to run a job again: ``` su - cgray password: test123 -sbatch --wrap "sleep 600"` +sbatch --wrap "sleep 600" squeue (the job should be running on a node) exit (logout from cgray account) ``` From 0f56b226865e3f5ae2eb30ff0a67f1e937f3f5c3 Mon Sep 17 00:00:00 2001 From: Ria Gupta <74742605+rg663@users.noreply.github.com> Date: Thu, 20 Jul 2023 11:26:58 -0400 Subject: [PATCH 3/3] Update README.md Proofreading changes made, along with substituting uses of "login," "logout,", and "setup" with "log in," "log out," and "set up" where appropriate to improve grammatical correctness. In the beginning of the tutorial, the link to the Half Day Tutorial, which was previously broken, has also been fixed. --- coldfront/README.md | 283 ++++++++++++++++++++++---------------------- 1 file changed, 142 insertions(+), 141 deletions(-) diff --git a/coldfront/README.md b/coldfront/README.md index 6a8d9bd..5e4b5e8 100644 --- a/coldfront/README.md +++ b/coldfront/README.md @@ -15,7 +15,7 @@ NOTE: The databases created for this were made for specific dates these tutorial ## Full Day Tutorial: Using ColdFront
-If interested, you can [view the steps taken](#seeding-the-database-for-the-full-day-tutorial) to pre-seed the database for the extended version of the tutorial +If interested, you can [view the steps taken](#seeding-the-database-for-the-full-day-tutorial) to pre-seed the database for the extended version of the tutorial. **By default, the containers start with the database being pre-seeded for the full day tutorial.** If you are starting from a different spot in these instructions, you may want to ensure you have the right database in place. To do so, stop (destroy) the containers, copy in the full day tutorial database, and then restart the containers, following these steps: @@ -25,59 +25,59 @@ cp coldfront/db_options/coldfront-full.dump database/coldfront.dump ./hpcts start ``` -Once the containers are started, navigate to the [ColdFront site - https://localhost:2443](https://localhost:2443) +Once the containers are started, navigate to the [ColdFront site - https://localhost:2443](https://localhost:2443).
## ColdFront Overview -This section should give you an idea of how ColdFront can be used in your center by PIs (principal investigators - also faculty, professors, project managers) to manage access to resources for their research groups or classes, by your system administrators to manage the resources & grant the access, and your center director or other staff to review who has access and what type of research they're doing. It's just one short example but it provides the full product lifecycle so you can see how the different roles might utilize the software. The next section provides more details specifically for system administrators managing resources at an HPC center. +This section should give you an idea of how ColdFront can be used in your center by PIs (principal investigators - also faculty, professors, project managers). PIs can manage access to resources for their research groups or classes, by your system administrators to manage the resources & grant the access, and your center director or other staff to review who has access and what type of research they're doing. It's just one short example but it provides the full product lifecycle so you can see how the different roles might utilize the software. The next section provides more details specifically for system administrators managing resources at an HPC center. ### PI View: Annual Project Review, Allocation Renewal & Allocation Change Requests -- Login as `cgray` password `test123` -- Click on the project and review the information we've added as part of the database pre-seeding. There is project information populated and several allocations listed. One allocation is coming up for renewal and will expire in less than a month. Notice the `Needs Review` label next to the project. -- Click on the `Add Users` button and in the search box enter `csimmons` then click the `Search` button. +- Log in as `cgray` password `test123` +- Click on the project and review the information we've added as part of the database pre-seeding. There is project information populated and several allocations listed. One allocation is coming up for renewal and will expire in less than a month. Notice the `Needs Review` label next to the project +- Click on the `Add Users` button and in the search box enter `csimmons` then click the `Search` button - You'll be presented the user information for the `csimmons` account as well as a list of allocations on the project. You can choose to add the account to either of the allocations or all of them. You may also change the role of the account from `User` to `Manager` -- Then click the `Add Selected Users to Project` button and you'll be returned to the Project Detail page. -- Click on the yellow `Expires in X days Click to renew` banner next to the allocation coming up on expiration and try to renew it. You should see a warning banner telling you it can't be done because the project review is due. +- Then click the `Add Selected Users to Project` button and you'll be returned to the Project Detail page +- Click on the yellow `Expires in X days Click to renew` banner next to the allocation coming up on expiration and try to renew it. You should see a warning banner telling you it can't be done because the project review is due NOTE: When a project review is required, a PI can't request new allocations or allocation change requests nor renew expiring allocations. They can, however, add/remove users, publications, grants, and research output. -- Click the `Review Project` link to start the project review process. Provide a reason for not providing grant or publication information, check the box to acknowledge the update and click the `Submit` button. -- Now try to renew the expiring allocation. Your request should be accepted and the allocation should now be in the `Renewal Requested` status. It's now possible to request new allocations or allocation changes as well. +- Click the `Review Project` link to start the project review process. Provide a reason for not providing grant or publication information, check the box to acknowledge the update and click the `Submit` button +- Now try to renew the expiring allocation. Your request should be accepted and the allocation should now be in the `Renewal Requested` status. It's now possible to request new allocations or allocation changes as well - Add a publication: Click the `Add Publication` button and in the search box enter the DOI: `10.1145/3437359.3465585` Click the `Search` button and it should display information about a ColdFront publication. Click on the checkbox next to the search result and then click the `Add Selected Publications to Project` button - Take a look at the other project sections such as Project Attributes, Grants, and Research Outputs -- Logout +- Log out ### Center Director View: Project Review Approval At part of the database seeding we did prior to the start of the tutorial, we configured the user `sfoster` with the proper account permissions to act as the Center Director. This allows `sfoster` to view all projects, allocations, publications, and grants. We've also given permission to view the pending project review list. -- Login as `sfoster` password `ilovelinux` to see what additional menus and functionality this account has access to. -- Navigate to the `Director` menu and click on `Project Reviews` -Click the `Email` button to see this functionality. Go back to the `Project Reviews` and click `Mark Complete`. +- Log in as `sfoster` password `ilovelinux` to see what additional menus and functionality this account has access to +- Navigate to the `Director` menu and click on `Project Reviews` +- Click the `Email` button to see this functionality. Go back to the `Project Reviews` and click `Mark Complete` - Click through the different options in the `Director` menu if desired -- Logout +- Log out ### Activate the allocation request -- Login as `hpcadmin` password `ilovelinux` +- Log in as `hpcadmin` password `ilovelinux` - Navigate to the `Admin` menu and click on `Allocation Requests` Note: the project review status is a green check mark, indicating our Center Director has already approved the submitted project review. -At part of the database seeding we did at the start of the tutorial, we activated and set attributes on the allocations requested on the `cgray` project. Let's look at that allocation and how it was setup. +At part of the database seeding we did at the start of the tutorial, we activated and set attributes on the allocations requested on the `cgray` project. Let's look at that allocation and how it was set up. -- Click the `Details` button to review the Allocation Detail page. -- Notice that allocation status is `Renewal Requested` and there is a start and end date associated with it. -- Scroll down to look at the allocation attributes set. There is a `slurm_account` attribute as well as `slurm_specs` and `slurm_user_specs` attributes. This is what is used by the Slurm plugin to sync with the Slurm database. -- Click the `Approve` button to re-activate the allocation. This updates the status to `Active` and changes the expiration date to one year from today. -- Logout as the `hpcadmin` user +- Click the `Details` button to review the Allocation Detail page +- Notice that allocation status is `Renewal Requested` and there is a start and end date associated with it +- Scroll down to look at the allocation attributes set. There is a `slurm_account` attribute as well as `slurm_specs` and `slurm_user_specs` attributes. This is what is used by the Slurm plugin to sync with the Slurm database +- Click the `Approve` button to re-activate the allocation. This updates the status to `Active` and changes the expiration date to one year from today +- Log out as the `hpcadmin` user ### Try to run a job as the PI user Now let's go outside of ColdFront to the command line and try to submit a batch job as the `cgray` user. -- Login to the frontend container: +- Log in to the frontend container: ``` ssh -p 6222 hpcadmin@localhost password: ilovelinux @@ -94,12 +94,12 @@ You will get an error message that you do not have permission to run on the clus ``` sacctmgr show user cgray -s list ``` -You should not see any Slurm account information for the `cgray` user +You should not see any Slurm account information for the `cgray` user. _**This is because we have not synced the allocation information in ColdFront with Slurm yet.**_ -- Type `exit` to log out of the cgray account and you should be on the frontend logged in as the hpcadmin account. +- Type `exit` to log out of the cgray account and you should be on the frontend logged in as the hpcadmin account ### Run Slurm plugin to sync active allocations from ColdFront to Slurm -- Login to the coldfront container & setup ColdFront environment +- Log in to the coldfront container & set up ColdFront environment ``` ssh coldfront source /srv/www/venv/bin/activate @@ -130,12 +130,12 @@ sacctmgr show user cgray -s list sacctmgr show user csimmons -s list ``` NOTE: The csimmons user is under the cgray slurm account -- Logout of ColdFront container +- Log out of ColdFront container ``` exit ``` -### Login (or go back) to frontend container +### Log in (or go back) to frontend container NOTE: you should already be on the frontend but just in case you're not: ``` ssh -p 6222 hpcadmin@localhost @@ -147,25 +147,25 @@ su - cgray password: test123 sbatch --wrap "sleep 600" squeue (the job should be running on a node) -exit (logout from cgray account) +exit (log out from cgray account) ```
**Tip:** - You can also view information about this job in OnDemand, even though you launched the job from the command line. [Login](https://localhost:3443) as the PI user and navigate to the `Jobs` menu, selecting `Active Jobs` from the drop down menu. + You can also view information about this job in OnDemand, even though you launched the job from the command line. [Log in](https://localhost:3443) as the PI user and navigate to the `Jobs` menu, selecting `Active Jobs` from the drop down menu.
## ColdFront Admin Tasks -Building on what you learned in the previous section, this part of the tutorial provides information on an array of administrative tasks and setup steps you'll need to know in order to setup ColdFront at your center. We'll step through a few of these tasks from the ColdFront administration page and then go back into the ColdFront front end to test these. +Building on what you learned in the previous section, this part of the tutorial provides information on an array of administrative tasks and setup steps you'll need to know in order to set up ColdFront at your center. We'll step through a few of these tasks from the ColdFront administration page and then go back into the ColdFront front end to test these. ### Elevating User to PI Status: In order for a user to create a new project, they need to have `PI` status. Let's give the `sfoster` account PI access: -- If necessary, login as `hpcadmin` password `ilovelinux` +- If necessary, log in as `hpcadmin` password `ilovelinux` - Go to Admin menu and click on `ColdFront Administration` - Scroll to the bottom under the `User` section and click on `User Profiles` - Click on `sfoster` check ``"Is pi"`` - click `SAVE` @@ -173,18 +173,18 @@ In order for a user to create a new project, they need to have `PI` status. Let ### Adding a Resource -The tutorial database comes with two resources already created. However, when running this at your center, you'll need to manually create a resource for each system or service you want your users to access. Let's add a cloud resource +The tutorial database comes with two resources already created. However, when running this at your center, you'll need to manually create a resource for each system or service you want your users to access. Let's add a cloud resource: Still on the `ColdFront Administration` page: - Scroll to the `Resource` section and click on `Resources` - Click on the `Add Resource` button -- Select `Cloud` for `Resource Type`, enter a name and a description. -- Make sure the checkboxes for `Is available`, `Is public`, and `Is allocatable` are checked. +- Select `Cloud` for `Resource Type`, enter a name and a description +- Make sure the checkboxes for `Is available`, `Is public`, and `Is allocatable` are checked - Under the `Resource Attributes` section, click `Add another resource attribute` -- Select the option `quantity_default_value` and enter a number here. +- Select the option `quantity_default_value` and enter a number here - Select the option `quantity_label` and enter `Enter number of CPU hours to purchase` -- Select the `eula` (End User License Agreement) option and enter the text you'd like your project managers to see when they're requesting allocations for this resource. -- Click the `SAVE` button. +- Select the `eula` (End User License Agreement) option and enter the text you'd like your project managers to see when they're requesting allocations for this resource +- Click the `SAVE` button ### Allocation Change Requests @@ -193,11 +193,11 @@ Allocation change requests allow a project manager to request a change on an act Still on the `ColdFront Administration` page: - Under the `Allocation` section click on `Allocation Attribute Types` - Let's allow project managers to request an increase in their storage quotas. Click on the number next to `Storage Quota (GB)` -- Check the box `Is Changeable` and then click the `Save` button. +- Check the box `Is Changeable` and then click the `Save` button - Now we'll switch over to our PI user. Under the `Authentication and Authorization` section, click on `Users` then click on `cgray` -- At the top right, click on `Login As` which redirects us to the ColdFront home page for the user `cgray`. Click on the project name to get to the project detail page. -- Click on the actions icon next to the `Project Storage` resource which takes you to the Allocation Detail page. Click on the `Request Change` button, select a date extension, enter a new amount of storage, and provide a justification. Then click the `SUBMIT` button. You should now see a pending allocation change request on the allocation detail page. -- Click on the `release cgray` button at the top in the yellow banner. +- At the top right, click on `Login As` which redirects us to the ColdFront home page for the user `cgray`. Click on the project name to get to the project detail page +- Click on the actions icon next to the `Project Storage` resource which takes you to the Allocation Detail page. Click on the `Request Change` button, select a date extension, enter a new amount of storage, and provide a justification. Then click the `SUBMIT` button. You should now see a pending allocation change request on the allocation detail page +- Click on the `release cgray` button at the top in the yellow banner We'll go back to activating that request in a bit.... @@ -210,35 +210,35 @@ Now that you've set up the new cloud resource, let's create a new project and re - Using the `ColdFront Administration` page, use the "Login As" option: Under the `Authentication and Authorization` section, click on `Users` then click on `sfoster` - At the top right, click on `Login As` which redirects us to the ColdFront home page for the user -- Click the `Add a project` button and fill out the `Title` and `Description` fields and select a field of science. Click the `Save` button. -- Once redirected to the project detail page, click `Request Resource Allocation` under the Allocations section. +- Click the `Add a project` button and fill out the `Title` and `Description` fields and select a field of science. Click the `Save` button +- Once redirected to the project detail page, click `Request Resource Allocation` under the Allocations section - Select the cloud resource from the drop down, provide a justification, enter the number of CPU hours, and click the `Submit` button. -Click on the `release sfoster` button at the top in the yellow banner. +Click on the `release sfoster` button at the top in the yellow banner As the admin, let's configure and activate that allocation: -- From the `ColdFront Administration` page, click on the `View Site` link at the top right. +- From the `ColdFront Administration` page, click on the `View Site` link at the top right - Navigate to the `Admin` menu and click on `Allocation Requests` - Click on the `Details` button next to the `Research Cloud` allocation request to configure and activate the allocation: click the `Add Allocation Attribute` button and select these allocation attributes from the drop down menu: `Core Usage (Hours)` Enter: `10000` `Cloud Storage Quota (TB)` Enter: `10` `Cloud Account Name` Enter: `cgray` -Notice as you add the core usage and cloud storage quota attributes you see usage graphs displayed. These can tie into a plugin such as XDMoD or the OpenStack community developed plugins for usage information. -- Click the `Approve` button to change the allocation status to `Active` and set the expiration date out one year from now. +Notice as you add the core usage and cloud storage quota attributes you see usage graphs displayed. These can tie into a plugin such as XDMoD or the OpenStack community developed plugins for usage information +- Click the `Approve` button to change the allocation status to `Active` and set the expiration date out one year from now Now let's go look at and activate the allocation change request submitted by `cgray` for the storage resource. - Navigate to the `Admin` menu and click on `Allocation Change Requests` -- Click on the `Details` button to review and approve the allocation changes requested. As the admin you have the ability to approve the date extension, change it to another setting or select `no extension` You can remove the `storage_quota` request or change it. You can add notes for the PI and users on the allocation to see. Then you can take action such as `Approve` or `Deny` the request. For this demo, let's click the `Approve` button. -- At the bottom of the page, click the `View Allocation` button and notice the `Storage Quota` attribute value has changed from the origianl 10000 to whatever you set it as. +- Click on the `Details` button to review and approve the allocation changes requested. As the admin you have the ability to approve the date extension, change it to another setting or select `no extension` You can remove the `storage_quota` request or change it. You can add notes for the PI and users on the allocation to see. Then you can take action such as `Approve` or `Deny` the request. For this demo, let's click the `Approve` button +- At the bottom of the page, click the `View Allocation` button and notice the `Storage Quota` attribute value has changed from the origianl 10000 to whatever you set it as ### Removing Access When an allocation expires or is revoked, the users on that allocation should lose access to the resource. If the allocation has the `freeipa_group` attribute set, all allocation users are removed from the group when the FreeIPA plugin is run. If the allocation is for a Slurm resource, all Slurm user associations and the Slurm account are removed when the Slurm plugin is run. Let's expire a Slurm allocation and then run the `slurm_check` tool. -- Navigate to the `Admin` menu, click on `All allocations` and click on the allocation for the hpc cluster resource. -- Change the status to `Expired` and the End Date to today. Click the `Update` button. +- Navigate to the `Admin` menu, click on `All allocations` and click on the allocation for the HPC cluster resource +- Change the status to `Expired` and the End Date to today. Click the `Update` button - If you're not still logged into the coldfront container, log back in via the terminal & activate the ColdFront virtual environment: @@ -266,13 +266,13 @@ There should be no account or association listed any longer ### Staff Member View -We have setup the `astewart` account as an example of what a center staff member might see in ColdFront. -- Login as `astewart` password `ilovelinux` to see what additional menus and functionality this account has access to. -- Navigate to the `Staff` menu and click through the menu options to get a sense of the access we recommend for staff members. -- Click on the `User Search` menu option and enter one of the other account names and click the `Search` button. Click on the username and then click on the `View User Projects and Managers` button. You'll see a list of projects the user is a member of and if they are a PI or manager on any of them. -- Logout +We have set up the `astewart` account as an example of what a center staff member might see in ColdFront. +- Log in as `astewart` password `ilovelinux` to see what additional menus and functionality this account has access to +- Navigate to the `Staff` menu and click through the menu options to get a sense of the access we recommend for staff members +- Click on the `User Search` menu option and enter one of the other account names and click the `Search` button. Click on the username and then click on the `View User Projects and Managers` button. You'll see a list of projects the user is a member of and if they are a PI or manager on any of them +- Log out -You can see how these different permissions were setup for the tutorial in the [pre-seeding section](#seeding-the-database-for-the-full-day-tutorial) below. +You can see how these different permissions were set up for the tutorial in the [pre-seeding section](#seeding-the-database-for-the-full-day-tutorial) below. For more options on allowing permissions for various types of staff access, see the ColdFront manual: https://coldfront.readthedocs.io/en/latest/manual/users/ @@ -284,9 +284,9 @@ That wraps up the full day tutorial but there is a lot more you can do with Cold -## Half Day Tutorial: Using ColdFront +## Half Day Tutorial: Using ColdFront
-If interested, you can [view the steps taken](#seeding-the-database-for-the-half-day-tutorial) to pre-seed the database for this condensed version of the tutorial +If interested, you can [view the steps taken](#seeding-the-database-for-the-half-day-tutorial) to pre-seed the database for this condensed version of the tutorial. Before beginning the tutorial, you must stop the containers, copy in the correct database, and then restart the containers. Follow these steps: @@ -296,34 +296,35 @@ cp coldfront/db_options/coldfront-half.dump database/coldfront.dump ./hpcts start ``` -Once the containers are started, navigate to the [ColdFront site - https://localhost:2443](https://localhost:2443) +Once the containers are started, navigate to the [ColdFront site - https://localhost:2443](https://localhost:2443). ### PI View: Annual Project Review, Allocation Renewal & Allocation Change Requests -- Login as `cgray` password `test123` -- Click on the project and review the information we've added as part of the database pre-seeding. There is project information populated and several allocations listed. One allocation is coming up for renewal and will expire in less than a month. Notice the `Needs Review` label next to the project. -- Click on the `Add Users` button and in the search box enter `csimmons` then click the `Search` button. +- Log in as `cgray` password `test123` +- Click on the project and review the information we've added as part of the database pre-seeding. There is project information populated and several allocations listed. One allocation is coming up for renewal and will expire in less than a month. Notice the `Needs Review` label next to the project +- Click on the `Add Users` button and in the search box enter `csimmons` then click the `Search` button - You'll be presented the user information for the `csimmons` account as well as a list of allocations on the project. You can choose to add the account to either of the allocations or all of them. You may also change the role of the account from `User` to `Manager` -- Then click the `Add Selected Users to Project` button and you'll be returned to the Project Detail page. -- Click on the yellow `Expires in X days Click to renew` banner next to the allocation coming up on expiration and try to renew it. You should see a warning banner telling you it can't be done because the project review is due. +- Then click the `Add Selected Users to Project` button and you'll be returned to the Project Detail page +- Click on the yellow `Expires in X days Click to renew` banner next to the allocation coming up on expiration and try to renew it. You should see a warning banner telling you it can't be done because the project review is due -NOTE: When a project review is required, a PI can't request new allocations or allocation change requests nor renew expiring allocations. They can, however, add/remove users, publications, grants, and research output. +NOTE: When a project review is required, a PI can't request new allocations or allocation change requests nor renew expiring allocations. They can, however, add/remove users, publications, grants, and research output -- Click the `Review Project` link to start the project review process. Provide a reason for not providing grant or publication information, check the box to acknowledge the update and click the `Submit` button. +- Click the `Review Project` link to start the project review process. Provide a reason for not providing grant or publication information, check the box to acknowledge the update and click the `Submit` button -- Now try to renew the expiring allocation. Your request should be accepted and the allocation should now be in the `Renewal Requested` status. It's now possible to request new allocations or allocation changes as well. +- Now try to renew the expiring allocation. Your request should be accepted and the allocation should now be in the `Renewal Requested` status. It's now possible to request new allocations or allocation changes as well -- Navigate back to the project detail page and click on the actions icon next to the `Project Storage` resource which takes you to the Allocation Detail page. Click on the `Request Change` button, select a date extension, enter a new amount of storage, and provide a justification. Then click the `SUBMIT` button. You should now see a pending allocation change request on the allocation detail page. Logout. +- Navigate back to the project detail page and click on the actions icon next to the `Project Storage` resource which takes you to the Allocation Detail page. Click on the `Request Change` button, select a date extension, enter a new amount of storage, and provide a justification. Then click the `SUBMIT` button. You should now see a pending allocation change request on the allocation detail page. +- Log out ### Center Director View: Project Review Approval At part of the database seeding we did at the start of the tutorial, we configured the user `sfoster` with the `Staff Status` role and gave the account permissions to act as the Center Director. This allows `sfoster` to view all projects, allocations, publications, and grants. We've also given permission to view the pending project review list. -- Login as `sfoster` password `ilovelinux` to see what additional menus and functionality this account has access to. +- Log in as `sfoster` password `ilovelinux` to see what additional menus and functionality this account has access to - Navigate to the `Staff` menu and click on `Project Reviews` -Click the `Email` button to see this functionality. Go back to the `Project Reviews` and click `Mark Complete`. -- Logout +Click the `Email` button to see this functionality. Go back to the `Project Reviews` and click `Mark Complete` +- Log out -For more options on allowing permissions for various types of staff access, see the ColdFront manual: https://coldfront.readthedocs.io/en/latest/manual/users/ +For more options on allowing permissions for various types of staff access, see the ColdFront manual: https://coldfront.readthedocs.io/en/latest/manual/users/. ### Activate the allocation request @@ -331,24 +332,24 @@ For more options on allowing permissions for various types of staff access, see - Navigate to the `Admin` menu and click on `Allocation Requests` Note: the project review status is a green check mark, indicating our Center Director has already approved the submitted project review. -At part of the database seeding we did at the start of the tutorial, we activated and set attributes on the allocations requested on the `cgray` project. Let's look at that allocation and how it was setup. +At part of the database seeding we did at the start of the tutorial, we activated and set attributes on the allocations requested on the `cgray` project. Let's look at that allocation and how it was set up. -- Click the `Details` button to review the Allocation Detail page. -- Notice that allocation status is `Renewal Requested` and there is a start and end date associated with it. -- Scroll down to look at the allocation attributes set. There is a slurm_account attribute as well as slurm_specs and slurm_user_specs attributes. This is what is used by the Slurm plugin to sync with the Slurm database. -- Click the `Approve` button to re-activate the allocation. This updates the status to `Active` and changes the expiration date to one year from today. +- Click the `Details` button to review the Allocation Detail page +- Notice that allocation status is `Renewal Requested` and there is a start and end date associated with it +- Scroll down to look at the allocation attributes set. There is a slurm_account attribute as well as slurm_specs and slurm_user_specs attributes. This is what is used by the Slurm plugin to sync with the Slurm database +- Click the `Approve` button to re-activate the allocation. This updates the status to `Active` and changes the expiration date to one year from today -Now let's go look at and activate the allocation change request submitted by `cgray` for the storage resource. As the HPC admin user, activate and setup the new allocation: +Now let's go look at and activate the allocation change request submitted by `cgray` for the storage resource. As the HPC admin user, activate and set up the new allocation: - Navigate to the `Admin` menu and click on `Allocation Change Requests` -- Click on the `Details` button to review and approve the allocation changes requested. As the admin you have the ability to approve the date extension, change it to another setting or select `no extension` You can remove the `storage_quota` request or change it. You can add notes for the PI and users on the allocation to see. Then you can take action such as `Approve` or `Deny` the request. For this demo, let's click the `Approve` button. +- Click on the `Details` button to review and approve the allocation changes requested. As the admin you have the ability to approve the date extension, change it to another setting or select `no extension` You can remove the `storage_quota` request or change it. You can add notes for the PI and users on the allocation to see. Then you can take action such as `Approve` or `Deny` the request. For this demo, let's click the `Approve` button For more information about configuring Allocation Change Requests [see here](#more-info-on-allocation-change-requests) -- Logout as the `hpcadmin` user +- Log out as the `hpcadmin` user ### Try to run a job as the PI user Now let's go outside of ColdFront to the command line and try to submit a batch job as the `cgray` user. -- Login to the frontend container: +- Log in to the frontend container: `ssh -p 6222 hpcadmin@localhost` password: `ilovelinux` Switch to the PI user account: @@ -356,12 +357,12 @@ Switch to the PI user account: password: `test123` `sbatch --wrap "sleep 600"` You will get an error message that you do not have permission to run on the cluster -`sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified` +`sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified.` _**This is because we have not synced the allocation information in ColdFront with Slurm yet.**_ -- Type `exit` to log out of the cgray account and you should be on the frontend logged in as the hpcadmin account. +- Type `exit` to log out of the cgray account and you should be on the frontend logged in as the hpcadmin account ### Run Slurm plugin to sync active allocations from ColdFront to Slurm -- Login to the coldfront container & setup ColdFront environment +- Log in to the coldfront container & set up ColdFront environment `ssh coldfront` `cd /srv/www` `source venv/bin/activate` @@ -382,10 +383,10 @@ _**This is because we have not synced the allocation information in ColdFront wi - Let's look at csimmons's slurm account again: `sacctmgr show user csimmons -s list` NOTE: The csimmons user is under the cgray slurm account -- Logout of ColdFront container +- Log out of ColdFront container `exit` -### Login (or go back) to frontend container +### Log in (or go back) to frontend container NOTE: you should already be on the frontend but just in case you're not: `ssh -p 6222 hpcadmin@localhost` password: `ilovelinux` @@ -395,18 +396,18 @@ Switch over to the PI user account and try to run a job again: password: `test123` `sbatch --wrap "sleep 600"` `squeue` (the job should be running on a node) -`exit` (logout from cgray account) +`exit` (log out from cgray account) -Tip: You can also view information about this job in OnDemand, even though you launched the job from the command line. [Login](https://localhost:3443) as the PI user and navigate to the `Jobs` menu, selecting `Active Jobs` from the drop down menu. +Tip: You can also view information about this job in OnDemand, even though you launched the job from the command line. [Log in](https://localhost:3443) as the PI user and navigate to the `Jobs` menu, selecting `Active Jobs` from the drop down menu.

# Reference Section -The information in this section can be used for reference or to go through the tutorial pre-seeding steps and see how all the setup was done to get the database in the state we needed it for the half day and full day tutorials or to start from scratch. +The information in this section can be used for reference or to go through the tutorial pre-seeding steps and see how all the set up was done to get the database in the state we needed it for the half day and full day tutorials or to start from scratch. ## Seeding the Database for the Full Day Tutorial
-These steps were done in advance to allow for the presentation of a more hands-on tutorial which spends time on learning the features rather than the minutia of the setup. If you would like to go through these steps yourself, destroy the containers, delete the ColdFront database, and start the containers. This will create a new, empty ColdFront database. Then you'll be able to login to ColdFront and follow these steps to populate the database. +These steps were done in advance to allow for the presentation of a more hands-on tutorial which spends time on learning the features rather than the minutia of the set up. If you would like to go through these steps yourself, destroy the containers, delete the ColdFront database, and start the containers. This will create a new, empty ColdFront database. Then you'll be able to log in to ColdFront and follow these steps to populate the database. ``` ./hpcts destroy @@ -414,21 +415,21 @@ rm database/coldfront.dump ./hpcts start ``` -### Login to ColdFront, setup account permissions & create resource +### Log in to ColdFront, set up account permissions & create resource URL https://localhost:2443/ -You'll need to login as some of the users for this tutorial to get things started. Do NOT use the OpenID Connect login option at this point. -- Login locally as username `hpcadmin` password: `ilovelinux` -- Logout -- Login locally as username `cgray` password: `test123` -- Logout -- Login locally as username `csimmons` password: `ilovelinux` -- Logout -- Login locally as username `sfoster` password: `ilovelinux` -- Logout -- Login locally as username `astewart` password: `ilovelinux` -- Logout -- Login locally as username `admin` password: `admin` -- Go to Admin menu and click on `ColdFront Administration` Once there, scroll halfway down to the `Authentication and Authorization` section. Then click on the `Users` link. +You'll need to log in as some of the users for this tutorial to get things started. Do NOT use the OpenID Connect log in option at this point. +- Log in locally as username `hpcadmin` password: `ilovelinux` +- Log out +- Log in locally as username `cgray` password: `test123` +- Log out +- Log in locally as username `csimmons` password: `ilovelinux` +- Log out +- Log in locally as username `sfoster` password: `ilovelinux` +- Log out +- Log in locally as username `astewart` password: `ilovelinux` +- Log out +- Log in locally as username `admin` password: `admin` +- Go to Admin menu and click on `ColdFront Administration` Once there, scroll halfway down to the `Authentication and Authorization` section. Then click on the `Users` link - Click on the `hpcadmin` user and scroll down to the `Permissions` section - Make this user a `superuser` by checking the boxes next to `Staff Status` and `Superuser Status` - scroll to the bottom and click `SAVE` - Click on the `sfoster` account. Under the `User Permissions` section add permissions to make this user the Center Director @@ -475,22 +476,22 @@ Click `Add another Resource attribute` and select `OnDemand` from the drop down ### Create a project & request an allocation As the PI user: Create a project and request an allocation for the new resource: -- Login as the PI using local account username: `cgray` password: `test123` +- Log in as the PI using local account username: `cgray` password: `test123` - Click the `Add a project` button to create a new project, filling in the name, description, and selecting any field of science - Once redirected to the project detail page, request an allocation by clicking on the `Request Resource Allocation` button. Select the `hpc` resource from the drop down menu, provide any justification, and click the `Submit` button - Request another allocation by clicking on the `Request Resource Allocation` button. Select the `Project Storage` resource from the drop down menu, enter a quantity in TB or leave the default 1, provide any justification, and click the `Submit` button -- Logout +- Log out ### Activate the allocation requests -As the HPC admin user, activate and setup the new allocation: -- Login using local account username: `hpcadmin` password: `ilovelinux` +As the HPC admin user, activate and set up the new allocation: +- Log in using local account username: `hpcadmin` password: `ilovelinux` - Navigate to the `Admin` menu and click on `Allocation Requests` - Click on the `Details` button next to the `HPC Cluster` allocation request to configure and activate the allocation: click the `Add Allocation Attribute` button and select these allocation attributes from the drop down menu: `slurm_account_name` Enter: `cgray` `slurm_specs` Enter: `Fairshare=100` `slurm_user_specs` Enter: `Fairshare=parent` -- Set the status to `Active`, set the start date to today, and set the expiration date to the end of this month. If you click the `Approve` button, this will set the status to `Active` and set the expiration date out to one year from today. For the purposes of this demo, we wanted to shorten the allocation length. [See here](https://coldfront.readthedocs.io/en/latest/config/#coldfront-core-settings) for more on changing the allocation length default. +- Set the status to `Active`, set the start date to today, and set the expiration date to the end of this month. If you click the `Approve` button, this will set the status to `Active` and set the expiration date out to one year from today. For the purposes of this demo, we wanted to shorten the allocation length. [See here](https://coldfront.readthedocs.io/en/latest/config/#coldfront-core-settings) for more on changing the allocation length default - Click the `Update` button - Return back to the `Admin` menu and click on the `Allocation Requests` - Click on the `Details` button next to the `Project Storage` allocation request to configure and activate the allocation: @@ -504,16 +505,16 @@ add a description to let the user know the directory name: `/projects/cgray` ### Annual Project Review When the project review functionality is enabled (it is by default) a PI will be forced to review their project once every 365 days. We can force a project to be under review in less than a year which is what we'll do for the cgray project. [See here](https://coldfront.readthedocs.io/en/latest/config/#coldfront-core-settings) for more on disabling the annual project review process. -- If necessary, login as `hpcadmin` password `ilovelinux` +- If necessary, log in as `hpcadmin` password `ilovelinux` - Navigate to the `Admin` menu and click on the `ColdFront Administration` link. Scroll to the `Project` section and click on `Projects` then click on the project that we created earlier. Check the box next to `Force Review` -- Scroll to the bottom and click the `Save` button. +- Scroll to the bottom and click the `Save` button NOTE: If there is a project you never want project reviews on, uncheck 'Requires review'

## Starting from Scratch (Half Day Tutorial)
-These steps were done in advance to allow for the presentation of a condensed half day version of the tutorial. If you would like to go through these steps yourself, destroy the containers, and delete the ColdFront database. Start the containers which will create a new, empty coldfront database. Then login to ColdFront and follow the steps below: +These steps were done in advance to allow for the presentation of a condensed half day version of the tutorial. If you would like to go through these steps yourself, destroy the containers, and delete the ColdFront database. Start the containers which will create a new, empty coldfront database. Then log in to ColdFront and follow the steps below: ``` ./hpcts destroy @@ -521,21 +522,21 @@ rm database/coldfront.dump ./hpcts start ``` -### Login to ColdFront, setup account permissions & create resource +### Log in to ColdFront, set up account permissions & create resource URL https://localhost:2443/ -You'll need to login as some of the users for this tutorial to get things started. Do NOT use the OpenID Connect login option at this point. -- Login locally as username `hpcadmin` password: `ilovelinux` -- Logout -- Login locally as username `cgray` password: `test123` -- Logout -- Login locally as username `csimmons` password: `ilovelinux` -- Logout -- Login locally as username `sfoster` password: `ilovelinux` -- Logout -- Login locally as username `astewart` password: `ilovelinux` -- Logout -- Login locally as username `admin` password: `admin` -- Go to Admin menu and click on `ColdFront Administration` Once there, scroll halfway down to the `Authentication and Authorization` section. Then click on the `Users` link. +You'll need to log in as some of the users for this tutorial to get things started. Do NOT use the OpenID Connect login option at this point. +- Log in locally as username `hpcadmin` password: `ilovelinux` +- Log out +- Log in locally as username `cgray` password: `test123` +- Log out +- Log in locally as username `csimmons` password: `ilovelinux` +- Log out +- Log in locally as username `sfoster` password: `ilovelinux` +- Log out +- Log in locally as username `astewart` password: `ilovelinux` +- Log out +- Log in locally as username `admin` password: `admin` +- Go to Admin menu and click on `ColdFront Administration` Once there, scroll halfway down to the `Authentication and Authorization` section. Then click on the `Users` link - Click on the `hpcadmin` user and scroll down to the `Permissions` section - Make this user a `superuser` by checking the boxes next to `Staff Status` and `Superuser Status` - scroll to the bottom and click `SAVE` - Click on the `sfoster` account. Under the `User Permissions` section add permissions to make this user the Center Director @@ -584,35 +585,35 @@ Resource type: select `cloud` Name: type `on-prem cloud` Description: enter anything you want Ensure that the following are checked: `Is available`, `Is public`, `Is allocatable` -- We will not set any resource attributes on this resource. Scroll to the bottom and click `SAVE`. +- We will not set any resource attributes on this resource. Scroll to the bottom and click `SAVE` Add an allocation attribute type: - Click on the Home link to go to back to the Admin interface. Under the `Allocation` section click on `Allocation attribute types` -- Click `Add Allocation Attribute Type` button, select `Text` from the `Attribute Type` drop down menu and name it `Storage Directory` Make sure all checkboxes are unchecked and click the `SAVE` button. +- Click `Add Allocation Attribute Type` button, select `Text` from the `Attribute Type` drop down menu and name it `Storage Directory` Make sure all checkboxes are unchecked and click the `SAVE` button Make an allocation attribute changeable: - Under the `Allocation` section, click on `Allocation Attribute Types` -- Click on `Storage Quota` check the box next to `Is changeable` and then click the `SAVE` button. -- Logout +- Click on `Storage Quota` check the box next to `Is changeable` and then click the `SAVE` button +- Log out ### Create a project & request an allocation As the PI user: Create a project and request an allocation for the new resource: -- Login as the PI using local account username: `cgray` password: `test123` +- Log in as the PI using local account username: `cgray` password: `test123` - Click the `Add a project` button to create a new project, filling in the name, description, and selecting any field of science - Once redirected to the project detail page, request an allocation by clicking on the `Request Resource Allocation` button. Select the `hpc` resource from the drop down menu, provide any justification, and click the `Submit` button - Request another allocation by clicking on the `Request Resource Allocation` button. Select the `Project Storage` resource from the drop down menu, enter a quantity in TB or leave the default 1, provide any justification, and click the `Submit` button -- Logout +- Log out ### Activate the allocation requests -As the HPC admin user, activate and setup the new allocation: -- Login using local account username: `hpcadmin` password: `ilovelinux` +As the HPC admin user, activate and set up the new allocation: +- Log in using local account username: `hpcadmin` password: `ilovelinux` - Navigate to the `Admin` menu and click on `Allocation Requests` - Click on the `Details` button next to the `HPC Cluster` allocation request to configure and activate the allocation: click the `Add Allocation Attribute` button and select these allocation attributes from the drop down menu: `slurm_account_name` Enter: `cgray` `slurm_specs` Enter: `Fairshare=100:DefaultQOS=normal` `slurm_user_specs` Enter: `Fairshare=parent:DefaultQOS=normal` -- Set the status to `Active`, set the start date to today, and set the expiration date to the end of this month. If you click the `Approve` button, this will set the status to `Active` and set the expiration date out to one year from today. For the purposes of this demo, we wanted to shorten the allocation length. [See here](https://coldfront.readthedocs.io/en/latest/config/#coldfront-core-settings) for more on changing the allocation length default. +- Set the status to `Active`, set the start date to today, and set the expiration date to the end of this month. If you click the `Approve` button, this will set the status to `Active` and set the expiration date out to one year from today. For the purposes of this demo, we wanted to shorten the allocation length. [See here](https://coldfront.readthedocs.io/en/latest/config/#coldfront-core-settings) for more on changing the allocation length default - Click the `Update` button - Return back to the `Admin` menu and click on the `Allocation Requests` - Click on the `Details` button next to the `Project Storage` allocation request to configure and activate the allocation: @@ -624,9 +625,9 @@ click the `Add Allocation Attribute` button and select these allocation attribut ### Annual Project Review When the project review functionality is enabled (it is by default) a PI will be forced to review their project once every 365 days. We can force a project to be under review in less than a year which is what we'll do for the cgray project. [See here](https://coldfront.readthedocs.io/en/latest/config/#coldfront-core-settings) for more on disabling the annual project review process. -- If necessary, login as `hpcadmin` password `ilovelinux` +- If necessary, log in as `hpcadmin` password `ilovelinux` - Navigate to the `Admin` menu and click on the `ColdFront Administration` link. Scroll to the `Project` section and click on `Projects` then click on the project that we created earlier. Check the box next to `Force Review` -- Scroll to the bottom and click the `Save` button. +- Scroll to the bottom and click the `Save` button NOTE: If there is a project you never want project reviews on, uncheck 'Requires review' This wraps up the setup done to the ColdFront database to prepare for the condensed half-day tutorial format. @@ -641,9 +642,9 @@ If you want PIs to be able to request changes to allocation attributes (i.e. sto ## More info on the OnDemand Plugin -This is a very simple example of modifying the ColdFront configuration to use a plugin. This plugin allows us to provide a link to our OnDemand instance for any allocations for resources that have "OnDemand enabled" +This is a very simple example of modifying the ColdFront configuration to use a plugin. This plugin allows us to provide a link to our OnDemand instance for any allocations for resources that have "OnDemand enabled". -We have already added the OnDemand instance URL to the ColdFront config. You can see this outside the containers in your git directory: See `hpc-toolset-tutorial/coldfront/coldfront.env` +We have already added the OnDemand instance URL to the ColdFront config. You can see this outside the containers in your git directory: See `hpc-toolset-tutorial/coldfront/coldfront.env`. When creating the resource at the start of the tutorial, we added the `OnDemand` attribute to the `hpc` resource which tells it to display the OnDemand logo and link to the OnDemand URL for any allocations for this resource. Notice on the ColdFront home page next to the allocation for the HPC cluster resource you see the OnDemand logo. Click on the Project name and see this logo also shows up next to the allocation. When we click on that logo, it directs us to the OnDemand instance.