Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revise tutorial content after working through all of them #335

Merged
merged 13 commits into from
Dec 8, 2023
7 changes: 5 additions & 2 deletions examples/basic_gromacs_config.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
ramble:
env_vars:
set:
OMP_NUM_THREADS: '{n_threads}'
variables:
processes_per_node: 16
mpi_command: 'mpirun -n {n_ranks} -ppn {processes_per_node}'
Expand Down Expand Up @@ -38,8 +41,8 @@ ramble:
concretized: true
packages:
gcc9:
spack_spec: gcc@9.3.0 target=x86_64
compiler_spec: gcc@9.3.0
spack_spec: gcc@9.4.0 target=x86_64
compiler_spec: gcc@9.4.0
impi2018:
spack_spec: [email protected] target=x86_64
compiler: gcc9
Expand Down
9 changes: 6 additions & 3 deletions examples/vector_gromacs_software_config.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
ramble:
env_vars:
set:
OMP_NUM_THREADS: '{n_threads}'
variables:
processes_per_node: 16
mpi_command: 'mpirun -n {n_ranks} -ppn {processes_per_node}'
Expand All @@ -12,7 +15,7 @@ ramble:
'{type}_{n_ranks}ranks-{gromacs_version}': # Arbitrary experiment name
variables:
app_workloads: ['water_gmx50', 'water_bare']
n_ranks: [1, 2, 4]
n_ranks: [1, 2]
n_threads: '1'
size: '0003'
type: ['pme', 'rf']
Expand All @@ -26,8 +29,8 @@ ramble:
concretized: true
packages:
gcc9:
spack_spec: gcc@9.3.0 target=x86_64
compiler_spec: gcc@9.3.0
spack_spec: gcc@9.4.0 target=x86_64
compiler_spec: gcc@9.4.0
impi2018:
spack_spec: [email protected] target=x86_64
compiler: gcc9
Expand Down
9 changes: 6 additions & 3 deletions examples/vector_matrix_gromacs_config.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
ramble:
env_vars:
set:
OMP_NUM_THREADS: '{n_threads}'
variables:
processes_per_node: 16
mpi_command: 'mpirun -n {n_ranks} -ppn {processes_per_node}'
Expand All @@ -11,7 +14,7 @@ ramble:
'{type}_{n_ranks}ranks': # Arbitrary experiment name
variables:
app_workloads: ['water_gmx50', 'water_bare']
n_ranks: [1, 2, 4]
n_ranks: [1, 2]
n_threads: '1'
size: '0003'
type: ['pme', 'rf']
Expand All @@ -23,8 +26,8 @@ ramble:
concretized: true
packages:
gcc9:
spack_spec: gcc@9.3.0 target=x86_64
compiler_spec: gcc@9.3.0
spack_spec: gcc@9.4.0 target=x86_64
compiler_spec: gcc@9.4.0
impi2018:
spack_spec: [email protected] target=x86_64
compiler: gcc9
Expand Down
2 changes: 2 additions & 0 deletions lib/ramble/docs/getting_started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,8 @@ that experiments can be generated from. The ``ramble info`` command can also be
used to see what variables each workload has, and potentially some suggested
values for variables with a limited set of allowed values.

.. _ramble-workspaces:

------------------
Ramble Workspaces
------------------
Expand Down
27 changes: 18 additions & 9 deletions lib/ramble/docs/tutorials/10_using_modifiers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -121,8 +121,11 @@ The resulting configuration file should look like the following.
.. code-block:: YAML

ramble:
env_vars:
set:
OMP_NUM_THREADS: '{n_threads}'
variables:
processes_per_node: 4
processes_per_node: 16
n_ranks: '{processes_per_node}*{n_nodes}'
batch_submit: '{execute_experiment}'
mpi_command: mpirun -n {n_ranks}
Expand All @@ -135,12 +138,12 @@ The resulting configuration file should look like the following.
experiments:
scaling_{n_nodes}:
variables:
n_nodes: [1, 2, 4]
n_nodes: [1, 2]
spack:
concretized: true
packages:
gcc9:
spack_spec: gcc@9.3.0
spack_spec: gcc@9.4.0
intel-mpi:
spack_spec: [email protected]
compiler: gcc9
Expand Down Expand Up @@ -212,8 +215,11 @@ look like the following:
.. code-block:: YAML

ramble:
env_vars:
set:
OMP_NUM_THREADS: '{n_threads}'
variables:
processes_per_node: 4
processes_per_node: 16
n_ranks: '{processes_per_node}*{n_nodes}'
batch_submit: '{execute_experiment}'
mpi_command: mpirun -n {n_ranks}
Expand All @@ -227,12 +233,12 @@ look like the following:
experiments:
scaling_{n_nodes}:
variables:
n_nodes: [1, 2, 4]
n_nodes: [1, 2]
spack:
concretized: true
packages:
gcc9:
spack_spec: gcc@9.3.0
spack_spec: gcc@9.4.0
intel-mpi:
spack_spec: [email protected]
compiler: gcc9
Expand Down Expand Up @@ -279,8 +285,11 @@ configuration file should look like the following:
.. code-block:: YAML

ramble:
env_vars:
set:
OMP_NUM_THREADS: '{n_threads}'
variables:
processes_per_node: 4
processes_per_node: 16
n_ranks: '{processes_per_node}*{n_nodes}'
batch_submit: '{execute_experiment}'
mpi_command: mpirun -n {n_ranks}
Expand All @@ -294,12 +303,12 @@ configuration file should look like the following:
experiments:
scaling_{n_nodes}:
variables:
n_nodes: [1, 2, 4]
n_nodes: [1, 2]
spack:
concretized: true
packages:
gcc9:
spack_spec: gcc@9.3.0
spack_spec: gcc@9.4.0
intel-mpi:
spack_spec: [email protected]
compiler: gcc9
Expand Down
18 changes: 12 additions & 6 deletions lib/ramble/docs/tutorials/11_using_internals.rst
Original file line number Diff line number Diff line change
Expand Up @@ -109,8 +109,11 @@ the following:
.. code-block:: YAML

ramble:
env_vars:
set:
OMP_NUM_THREADS: '{n_threads}'
variables:
processes_per_node: 4
processes_per_node: 16
n_ranks: '{processes_per_node}*{n_nodes}'
batch_submit: '{execute_experiment}'
mpi_command: mpirun -n {n_ranks}
Expand All @@ -133,12 +136,12 @@ the following:
redirect: '{experiment_run_dir}/end_time'
use_mpi: false
variables:
n_nodes: [1, 2, 4]
n_nodes: [1, 2]
spack:
concretized: true
packages:
gcc9:
spack_spec: gcc@9.3.0
spack_spec: gcc@9.4.0
intel-mpi:
spack_spec: [email protected]
compiler: gcc9
Expand Down Expand Up @@ -203,8 +206,11 @@ configuration file should look like the following:
.. code-block:: YAML

ramble:
env_vars:
set:
OMP_NUM_THREADS: '{n_threads}'
variables:
processes_per_node: 4
processes_per_node: 16
n_ranks: '{processes_per_node}*{n_nodes}'
batch_submit: '{execute_experiment}'
mpi_command: mpirun -n {n_ranks}
Expand Down Expand Up @@ -237,12 +243,12 @@ configuration file should look like the following:
- execute
- end_time
variables:
n_nodes: [1, 2, 4]
n_nodes: [1, 2]
spack:
concretized: true
packages:
gcc9:
spack_spec: gcc@9.3.0
spack_spec: gcc@9.4.0
intel-mpi:
spack_spec: [email protected]
compiler: gcc9
Expand Down
47 changes: 45 additions & 2 deletions lib/ramble/docs/tutorials/1_hello_world.rst
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,19 @@ filter available application definitions. For example:

might output the following:

.. code-block:: console

==> 10 applications
hmmer hostname hpcc hpcg hpl intel-hpl intel-mpi-benchmarks lulesh osu-micro-benchmarks ufs-weather-model

The ``ramble list`` command also accepts regular expressions. For example:

.. code-block:: console

$ ramble list h*

might output the following:

.. code-block:: console

==> 5 applications
Expand Down Expand Up @@ -95,7 +108,7 @@ workspace. You can call this workspace ``hello_world``.

$ ramble workspace create hello_world

This will create a workspace for you in:
This will create a named workspace for you in:

.. code-block:: console

Expand All @@ -111,6 +124,16 @@ Now you can activate the workspace and view its default configuration.
You can use the ``ramble workspace info`` command after editing configuration
files to see how ramble would use the changes you made.

As an aside, if you had used:

.. code-block:: console

$ ramble workspace create -d hello_world

Ramble would create an anonymous workspace for you in ``${PWD}/hello_world``
for more information on named and anonymous workspaces, see
:ref:`Ramble workspace documentation<ramble-workspaces>`.

Configure the Workspace
^^^^^^^^^^^^^^^^^^^^^^^^

Expand All @@ -125,7 +148,11 @@ includes the software stack, the experiments, and all variables.
* ``execute_experiment.tpl`` This file is a template shell script that will be
rendered to execute each of the experiments that ramble generates.

You can edit these files directly or with the command ``ramble workspace edit``.
You can edit these files directly or with the command:

.. code-block:: console

$ ramble workspace edit

To begin, you should edit the ``ramble.yaml`` file to set up the configuration
for your experiments. For this tutorial, replace the default yaml text with the
Expand Down Expand Up @@ -217,3 +244,19 @@ extracted figures of merit. If the experiments were successful, this file will
show the following results:

* possible hostname: hostname of machine the experiment was executed on

Cleanup the Workspace
^^^^^^^^^^^^^^^^^^^^^

After you are finished exploring the workspace and tutorial content, make sure
you deactivate the workspace using:

.. code-block:: console

$ ramble workspace deactivate

Additionally, you can remove the workspace you used with:

.. code-block:: console

$ ramble workspace remove hello_world
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,12 @@ Application Information
-----------------------

As mentioned above, this tutorial uses the `GROMACS <https://www.gromacs.org/>`_
application definition. We will begin with the `water_bare` and `water_gmx50`
application definition. We will begin with the ``water_bare`` and ``water_gmx50``
workloads, as they are able to execute in a short amount of time.

Using a previously installed ``ramble``, the following command can be used to
get information about these workloads:

.. code-block:: console

$ ramble info gromacs
Expand Down Expand Up @@ -125,8 +126,13 @@ For each setup run, a set of logs will be created at:

$ $ramble_root/var/ramble/workspaces/$workspace_root/logs

Each run will have its own primary log, along with a folder containing a log for each
experiment that is being configured.
Each run will have its own primary log, along with a folder containing a log
for each experiment that is being configured. While setup is running, you can
monitor the process by looking at the contents of:

.. code-block:: console

$ $ramble_root/var/ramble/workspaces/basic_gromacs/logs/setup.latest/gromacs.water_gmx50.pme_single_rank.out

Executing Experiments
---------------------
Expand Down Expand Up @@ -160,3 +166,37 @@ show the following results:
* Nanosecs per day: Nanoseconds of simulation per day at the speed achieved
* Hours per nanosec: Hours required to calculate 1 nanosecond of simulation at
the speed achieved

Workspace Directory Structure
-----------------------------

After analyzing the workspace, you can exmine the structure of the workspace at:

.. code-block:: console

$ $ramble_root/var/ramble/workspaces/basic_gromacs

Within this directory, you should see the following directories:

* ``configs`` - Contains all of the configuration files for the workspace
* ``experiments`` - Contains all of the experiment execution directories
* ``inputs`` - Contains all of the input files needed for the experiments
* ``logs`` - Contains all log files from any Ramble command that acts on the workspace
* ``shared`` - Contains auxiliary files the experiment might need, such as environment variable information for licenses
* ``software`` - Contains software environments for the experiments

Cleanup the Workspace
---------------------

Once you are finished with the tutorial content, ensure you have deactivated
the workspace using:

.. code-block:: console

$ ramble workspace deactivate

Additionally, you can remove the workspace using:

.. code-block:: console

$ ramble workspace remove basic_gromacs
Loading