From 506f6fa092ba86cc536ac4b8a7f3c47df87d82c5 Mon Sep 17 00:00:00 2001 From: fiona-naughton Date: Sun, 29 Sep 2024 07:38:32 +1000 Subject: [PATCH 01/11] split making tutorial --- docs/source/making-a-kit/documentation.rst | 23 + .../source/making-a-kit/from-cookiecutter.rst | 43 ++ .../making-a-kit/guided-example/add-code.rst | 124 ++++ .../making-a-kit/guided-example/add-docs.rst | 224 ++++++ .../making-a-kit/guided-example/add-tests.rst | 174 +++++ .../guided-example/make-release.rst | 71 ++ .../making-a-kit/guided-example/to-github.rst | 43 ++ .../guided-example/use-cookiecutter.rst | 97 +++ docs/source/making-a-kit/hosting.rst | 22 + docs/source/making-a-kit/license.rst | 24 + docs/source/making-a-kit/testing.rst | 51 ++ docs/source/makingakit.rst | 694 +----------------- 12 files changed, 919 insertions(+), 671 deletions(-) create mode 100644 docs/source/making-a-kit/documentation.rst create mode 100644 docs/source/making-a-kit/from-cookiecutter.rst create mode 100644 docs/source/making-a-kit/guided-example/add-code.rst create mode 100644 docs/source/making-a-kit/guided-example/add-docs.rst create mode 100644 docs/source/making-a-kit/guided-example/add-tests.rst create mode 100644 docs/source/making-a-kit/guided-example/make-release.rst create mode 100644 docs/source/making-a-kit/guided-example/to-github.rst create mode 100644 docs/source/making-a-kit/guided-example/use-cookiecutter.rst create mode 100644 docs/source/making-a-kit/hosting.rst create mode 100644 docs/source/making-a-kit/license.rst create mode 100644 docs/source/making-a-kit/testing.rst diff --git a/docs/source/making-a-kit/documentation.rst b/docs/source/making-a-kit/documentation.rst new file mode 100644 index 00000000..79e8a6b4 --- /dev/null +++ b/docs/source/making-a-kit/documentation.rst @@ -0,0 +1,23 @@ +************* +Documentation +************* + +Basic documentation is required for MDAKit registration. +The detail and depth of the documentation is ultimately up to you, but +we require at a minimum that you provide README-style documentation +explaining what the code is supposed to do, how to install it, and the +basics of its use. + +Although this is the minimum, we highly recommend that you consider +generating your documentation with dedicated tools such as +`Sphinx `_, which allows you to +generate static documentation using +`reStructuredText `_ +formatted plain-text directly from your code. This makes it easier for +your documentation to change alongside code changes. + +Using a documentation hosting service such as +`Read the Docs `_ or +`GitHub Pages `_ makes public access to your +automatically generated documentation. + diff --git a/docs/source/making-a-kit/from-cookiecutter.rst b/docs/source/making-a-kit/from-cookiecutter.rst new file mode 100644 index 00000000..88e8dd34 --- /dev/null +++ b/docs/source/making-a-kit/from-cookiecutter.rst @@ -0,0 +1,43 @@ +************************************************ +Building from the cookiecutter: a guided example +************************************************ + +In this section, we provide a guided example for creating an MDAKit +using the `MDAKit cookiecutter `_. +A `video walk-through of this tutorial `_ +is available on YouTube. + +We will create a kit for RMSF analysis, replicating the functionality +of the `RMSF analysis class `_ +in the core library. + + +**MDAKit requirements** + +As a reminder, here are (in brief) the requirements (and recommendations) +that we are aiming to fulfil to make a 'registerable' kit. See the +requirements in more detail :ref:`here ` +We'll check back in at the end of each part to see how we're going! + +#. Uses MDAnalysis +#. Open source + OSI license +#. Versioned + on a version-controlled repository +#. Designated authors and maintainers +#. (At least) minimal documentation +#. (At least) minimal regression tests +#. Installable as a standard package +#. (Recommended) community information available +#. (Recommended) on a package distribution platform + + +.. toctree:: + :maxdepth: 1 + :caption: Steps + + guided-example/use-cookiecutter + guided-example/add-code + guided-example/add-tests + guided-example/to-github + guided-example/add-docs + guided-example/make-release + diff --git a/docs/source/making-a-kit/guided-example/add-code.rst b/docs/source/making-a-kit/guided-example/add-code.rst new file mode 100644 index 00000000..ae1fec4d --- /dev/null +++ b/docs/source/making-a-kit/guided-example/add-code.rst @@ -0,0 +1,124 @@ +*************************************************** +Part 2: Adding code to the newly created repository +*************************************************** + +*For a video demonstration of this section,* +`click here `_. + +Now that we have a shell for our MDAKit, we can begin to add our code +to the ``rmsfkit/`` directory. + +In this example, we are recreating the exist MDAnalysis RMSD analysis, +so we will simply copy and paste the analysis class into +``rmsfkit/rmsfkit.py``.The in-code documentation (which has been trimmed +here for brevity) is written in +`reStructuredText syntax `_ +syntax for building with Sphinx later. + +The contents of the file should now resemble the following code block: + +.. code-block:: python + + """ + rmsfkit.py + A package to perform RMSF analysis on molecular dynamics data. + + All code and documentation, both of which are trimmed for the sake of brevity, + are taken from the MDAnalysis Python package. Used under GPLv2+. + """ + + from MDAnalysis.analysis.base import AnalysisBase + import numpy as np + + class RMSF(AnalysisBase): + r"""Calculate RMSF of given atoms across a trajectory. + + Note + ---- + No RMSD-superposition is performed; it is assumed that the user is + providing a trajectory where the protein of interest has been structurally + aligned to a reference structure. The protein also has to be whole because + periodic boundaries are not taken into account. + + Run the analysis with :meth:`RMSF.run`, which stores the results in the + array :attr:`RMSF.results.rmsf`. + + """ + def __init__(self, atomgroup, **kwargs): + r"""Parameters + ---------- + atomgroup : AtomGroup + Atoms for which RMSF is calculated + verbose : bool (optional) + Show detailed progress of the calculation if set to ``True``; the + default is ``False``. + + Raises + ------ + ValueError + raised if negative values are calculated, which indicates that a + numerical overflow or underflow occurred + + Notes + ----- + The root mean square fluctuation of an atom :math:`i` is computed as the + time average + + .. math:: + + \rho_i = \sqrt{\left\langle (\mathbf{x}_i - \langle\mathbf{x}_i\rangle)^2 \right\rangle} + + No mass weighting is performed. + + This method implements an algorithm for computing sums of squares while + avoiding overflows and underflows :cite:p:`Welford1962`. + + References + ---------- + .. bibliography:: + :filter: False + + Welford1962 + + """ + super(RMSF, self).__init__(atomgroup.universe.trajectory, **kwargs) + self.atomgroup = atomgroup + + def _prepare(self): + self.sumsquares = np.zeros((self.atomgroup.n_atoms, 3)) + self.mean = self.sumsquares.copy() + + def _single_frame(self): + k = self._frame_index + self.sumsquares += (k / (k+1.0)) * (self.atomgroup.positions - self.mean) ** 2 + self.mean = (k * self.mean + self.atomgroup.positions) / (k + 1) + + def _conclude(self): + k = self._frame_index + self.results.rmsf = np.sqrt(self.sumsquares.sum(axis=1) / (k + 1)) + + if not (self.results.rmsf >= 0).all(): + raise ValueError("Some RMSF values negative; overflow " + + "or underflow occurred") + + +Finally, to make our ``RMSF`` analysis class easier to access, we +import it in ``__init__.py`` by adding: + +.. code-block:: python + + from .rmsfkit import RMSF + + +**Progress: MDAKit requirements** + +#. **✓ Uses MDAnalysis** +#. **✓ Open source + OSI license** +#. *Versioned + on a version-controlled repository* +#. **✓ Designated authors and maintainers** +#. *(At least) minimal documentation* +#. *(At least) minimal regression tests* +#. **✓ Installable as a standard package** +#. **✓ (Recommended) community information available** +#. *(Recommended) on a package distribution platform* + diff --git a/docs/source/making-a-kit/guided-example/add-docs.rst b/docs/source/making-a-kit/guided-example/add-docs.rst new file mode 100644 index 00000000..710929f7 --- /dev/null +++ b/docs/source/making-a-kit/guided-example/add-docs.rst @@ -0,0 +1,224 @@ +******************************* +Part 5: Providing documentation +******************************* + +*For a video demonstration of this section,* +`click here `_. + +The auto-generated ``README.md`` file constituted a form of basic +documentation. However, it is ideal to provide more thorough +documentation for your code. + +Additional documentation can consist of *API documentation* or +*User Guide* documentation. The former provides details of the +functions, inputs/outputs, datatypes, etc of the code and can +be autogenerated from docstrings in the code; while the +later may show usage and provide more details, explanations and +examples for users. + +`Read the Docs `_ can be used to build and host +documentation online. The cookiecutter includes a +Read the Docs configuration as well a pre-made documentation +environment file that is used by Read the Docs and for building +locally. Frameworks for autogenerated API and basic User Guide +documentation are also provided. + + +Building documentation locally +############################## + +First, we need to install the correct environment for building +the documentation. In the ``docs/`` directory, run: + +.. code-block:: bash + + mamba env update --name rmsfkit -f requirements.yaml + +We can now build the documentation HTML files using the included +``Makefile``. Without looking at any of the documentation source +files, run: + +.. code-block:: bash + + make html + +This will convert the reStructuredText files into HTML in the +``_build`` directory. + +Open ``index.html`` and look around. + +Notice that the "API Documentation" does not contain all of the +information found in the docstrings of our code. +This is because within ``source/api.rst``, the only contents are: + +.. code-block:: rst + + API Documentation + ================= + + .. autosummary:: + :toctree: autosummary + + rmsfkit + +Instead, it should contain: + +.. code-block:: rst + + API Documentation + ================= + + .. autosummary:: + :toctree: autosummary + + rmsfkit.RMSF + +After making the change, rerun ``make html`` and refresh the page. There will +now be a table entry for the RMSF class. Clicking this entry will open the +documentation that was present in the docstrings! + + +Adding to the documentation +########################### + +Other than the newly visible docstring contents, the rest of the documentation +is completely empty. Let's update the ``index.rst`` file to include a short +description of the package: + +.. code-block:: rst + + Welcome to rmsfkit's documentation! + ========================================================= + + ``rmsfkit`` is an example MDAKit that implements the functionality of the `analysis.rms.RMSF` class within the MDAnalysis package. + This MDAKit does not serve as a replacement for this functionality and using this MDAKit for real work is discouraged. + + .. toctree:: + :maxdepth: 2 + :caption: Contents: + + getting_started + api + + + Indices and tables + ================== + + * :ref:`genindex` + * :ref:`modindex` + * :ref:`search` + +Additionally, we can update ``getting_started.rst`` to let potential +users know how to install the package. + +.. code-block:: rst + + Getting Started + =============== + + The ``rmsfkit`` package is installable from source. + + .. code-block:: bash + + git clone git@github.com:ianmkenney/rmsfkit.git + cd rmsfkit/ + pip install . + +Run the ``make html`` command again and refresh the browser window to +view the resulting changes. + +Citations in documentation +-------------------------- +You'll notice the RMSF class documentation has an unformatted citation +(``:cite:p:`Welford1962```). If your documentation need citations, you +can easily include them using the bibtex format. + +We first create the bibtex file, ``references.bib`` in the ``doc/source/`` +directory: + +.. code-block:: + + @article{Welford1962, + author = { B. P. Welford}, + title = {Note on a Method for Calculating Corrected Sums of Squares and Products}, + journal = {Technometrics}, + volume = {4}, + number = {3}, + pages = {419-420}, + year = {1962}, + publisher = {Taylor & Francis}, + doi = {10.1080/00401706.1962.10490022} + } + +In ``conf.py``, we need to add a new extension (``sphinxcontrib.bibtex``) +as well as the name of the bibtex file. + +.. code-block:: + + extensions = [ + 'sphinx.ext.autosummary', + 'sphinx.ext.autodoc', + 'sphinx.ext.mathjax', + 'sphinx.ext.viewcode', + 'sphinx.ext.napoleon', + 'sphinx.ext.intersphinx', + 'sphinx.ext.extlinks', + 'sphinxcontrib.bibtex', # add this line + ] + + bibtex_bibfiles = ['references.bib'] + +In addition, we have to add this extension the install requirements +in ``docs/requirements.yaml``. Add ``- sphinxcontrib-bibtex`` as an +additional dependency here. + +Update your environment with + +.. code-block:: bash + + mamba env update --name rmsfkit -f requirements.yaml + +before once again running `make html`. + +Refreshing the RMSF documentation will now show a properly formatted citation +using the information in the bibtex file. + + +Deploying the documentation +=========================== + +Once you've pushed any documentation changes to GitHub, you can the deploy +their own shiny new website. + +Log into `Read the Docs `_ (using your GitHub account) and +navigate to the dashboard. Click the "Import a Project" button and find the +repository in the list. Click the "+" and confirm that the name, URL, and default branch are +correct. + +.. image:: img/rmsftutorial/import.png + :alt: Import a project into RTD + +.. image:: img/rmsftutorial/adding.png + :alt: Adding your repository to RTD + +Clicking next will begin the deployment. This immediately starts the build +process, which can be confirmed by clicking the "Builds" tab. Once the build is +completed, you can view the deployed documentation, which is now public. This +completes the documentation requirement for an MDAKit. Notice that the "docs" +badge in the GitHub rendered ``README.md`` is now green. + + +**Progress: MDAKit requirements** + +#. **✓ Uses MDAnalysis** +#. **✓ Open source + OSI license** +#. **✓ Versioned + on a version-controlled repository** +#. **✓ Designated authors and maintainers** +#. **✓ (At least) minimal documentation** +#. **✓(At least) minimal regression tests** +#. **✓ Installable as a standard package** +#. **✓ (Recommended) community information available** +#. *(Recommended) on a package distribution platform* + + +.. _read the docs: https://readthedocs.org diff --git a/docs/source/making-a-kit/guided-example/add-tests.rst b/docs/source/making-a-kit/guided-example/add-tests.rst new file mode 100644 index 00000000..a8264569 --- /dev/null +++ b/docs/source/making-a-kit/guided-example/add-tests.rst @@ -0,0 +1,174 @@ +******************** +Part 3: Adding tests +******************** + +*For a video demonstration of this section,* +`click here `_. + +The cookiecutter set up the framework for tests in ``rmsfkit/tests``, +but we need to add the tests themselves to ``test_rmsfkit.py``. +Once again, we take directly from the MDAnalysis package. +Here we take the `RMSF testing class `_ +and update the contents of ``rmsfkit/tests/test_rmsfkit.py`` as follows: + +.. code-block:: python + + """ + Unit and regression test for the rmsfkit package. + + The TestRMSF class was taken from the MDAnalysis rms tests file and + the relevant modules were switched. + """ + + # Import package, test suite, and other packages as needed + from MDAnalysisTests.datafiles import GRO, XTC, rmsfArray + import MDAnalysis as mda + + from numpy.testing import assert_equal, assert_almost_equal + import numpy as np + import os + import pytest + + import rmsfkit + import sys + + def test_rmsfkit_imported(): + """Sample test, will always pass so long as import statement worked""" + assert "rmsfkit" in sys.modules + + + class TestRMSF(object): + @pytest.fixture() + def universe(self): + return mda.Universe(GRO, XTC) + + def test_rmsf(self, universe): + rmsfs = rmsfkit.RMSF(universe.select_atoms('name CA')) + rmsfs.run() + test_rmsfs = np.load(rmsfArray) + + assert_almost_equal(rmsfs.results.rmsf, test_rmsfs, 5, + err_msg="error: rmsf profile should match test " + "values") + + def test_rmsf_single_frame(self, universe): + rmsfs = rmsfkit.RMSF(universe.select_atoms('name CA')).run(start=5, stop=6) + + assert_almost_equal(rmsfs.results.rmsf, 0, 5, + err_msg="error: rmsfs should all be zero") + + def test_rmsf_identical_frames(self, universe, tmpdir): + + outfile = os.path.join(str(tmpdir), 'rmsf.xtc') + + # write a dummy trajectory of all the same frame + with mda.Writer(outfile, universe.atoms.n_atoms) as W: + for _ in range(universe.trajectory.n_frames): + W.write(universe) + + universe = mda.Universe(GRO, outfile) + rmsfs = rmsfkit.RMSF(universe.select_atoms('name CA')) + rmsfs.run() + assert_almost_equal(rmsfs.results.rmsf, 0, 5, + err_msg="error: rmsfs should all be 0") + + +Adding a test dependency +------------------------ + +Since these tests use files included with the ``MDAnalysisTests`` package, +we need to add this as a dependency. We do this in two configuration +files: +- In ``devtools/conda-envs/test_env.yaml``. +- In ``pyproject.toml``: Under the ``[project.optional-dependencies]`` + table, ensure that ``MDAnalysisTests>=2.0.0`` is listed in ``test``. + + .. code-block:: toml + + [project.optional-dependencies] + test = [ + "pytest>=6.0", + "pytest-xdist>=2.5", + "pytest-cov>=3.0", + "MDAnalysisTests>=2.0.0", + ] + + +Running the tests +################# + +Following the instructions from the generated ``README.md``, we can +create a testing environment using ``mamba`` (preferred) or ``conda``: + +.. code-block:: bash + + $ mamba create -n rmsfkit + $ mamba env update --name rmsfkit --file devtools/conda-envs/test_env.yaml + $ mamba activate rmsfkit + +And then install the package: + +.. code-block:: bash + $ pip install -e . + +We can now run tests locally using: + +.. code-block:: bash + + $ pytest rmsfkit/tests + +This should pass without errors, though with some potential warnings. + + +Preparing for CI +################ + +Local tests passing is only half of testing. Ideally, tests shouls also +pass through continuous integration services. The cookiecutter generates +the necessary GitHub workflow files ``.github/workslows/gh-ci.yaml`` +to do this on GitHub. + +Since our tests use the ``MDAnalysisTests`` package, we again need to make +a change to this file. Change the block: + +.. code-block:: yaml + + - name: Install MDAnalysis version + uses: MDAnalysis/install-mdanalysis@main + with: + version: ${{ matrix.mdanalysis-version }} + install-tests: false + installer: mamba + shell: bash -l {0} + +to + +.. code-block:: yaml + + - name: Install MDAnalysis version + uses: MDAnalysis/install-mdanalysis@main + with: + version: ${{ matrix.mdanalysis-version }} + install-tests: true # this needs to be true! + installer: mamba + shell: bash -l {0} + +Currently, our code only exists locally. We'll next need to upload it to a +GitHub repository to run the pre-built continuous integration provided +by the cookiecutter. + + +**Progress: MDAKit requirements** + +#. **✓ Uses MDAnalysis** +#. **✓ Open source + OSI license** +#. *Versioned + on a version-controlled repository* +#. **✓ Designated authors and maintainers** +#. *(At least) minimal documentation* +#. **✓(At least) minimal regression tests** - these should be ready + to go! CI should be finalised in the next step, when we push to + GitHub. +#. **✓ Installable as a standard package** +#. **✓ (Recommended) community information available** +#. *(Recommended) on a package distribution platform* + diff --git a/docs/source/making-a-kit/guided-example/make-release.rst b/docs/source/making-a-kit/guided-example/make-release.rst new file mode 100644 index 00000000..edb6020c --- /dev/null +++ b/docs/source/making-a-kit/guided-example/make-release.rst @@ -0,0 +1,71 @@ +********************************* +Part 6: Making an initial release +********************************* + +*For a video demonstration of this section,* +`click here `_. + +While using git does satisfy that our MDAKit code be versioned, +it is a good idea to make numbered releases with (human-understndable) +version numbers as improvements to your code are made. Let's make an +initial release for rmsfkit. + +The MDAKit cookiecutter uses the +`versioneer utility `_ +for version management. In brief, software versions are set through +`git tags `_. + +Since the kit is initiated without tags, the current version is set to ``0.0.0``. + +.. code-block:: + + >>> import rmsfkit + >>> rmsfkit.__version__ + '0+untagged.3.g3eed836' + +We can bump this to a ``0.1.0`` in preparation for an initial release on GitHub. + +.. code-block:: bash + + $ git tag 0.1.0 + +This should now be reflected in the interpreter (may require a package reinstall): + +.. code-block:: bash + + >>> import rmsfkit + >>> rmsfkit.__version__ + '0.1.0' + +Tags are not pushed to remote repositories by default. To push the +``0.1.0`` tag, use: + +.. code-block:: bash + + $ git push --tags + +Viewing the repository tags page on GitHub, you should now see a +``0.1.0`` tag, which can then be used to create a release by expanding +its menu options. Enter a release name, such as `v0.1.0` and publish! + +.. image:: img/rmsftutorial/creating_a_release.gif + :alt: Process for creating a release by making a tag + + +You could now use this release to publish to package distribution platforms +such as PyPi or Conda-Forge, though this is not currently covered in this +example. + + +**Progress: MDAKit requirements** + +#. **✓ Uses MDAnalysis** +#. **✓ Open source + OSI license** +#. **✓ Versioned + on a version-controlled repository** +#. **✓ Designated authors and maintainers** +#. **✓ (At least) minimal documentation** +#. **✓(At least) minimal regression tests** +#. **✓ Installable as a standard package** +#. **✓ (Recommended) community information available** +#. *(Recommended) on a package distribution platform* + diff --git a/docs/source/making-a-kit/guided-example/to-github.rst b/docs/source/making-a-kit/guided-example/to-github.rst new file mode 100644 index 00000000..2a126310 --- /dev/null +++ b/docs/source/making-a-kit/guided-example/to-github.rst @@ -0,0 +1,43 @@ +************************* +Part 4: Pushing to GitHub +************************* + +*For a video demonstration of this section,* +`click here `_. + +We need to upload our code to GitHub in order to finalise the continuous +integration set up in the previous step, and to fullfil the requirement +that the Kit be hosted on a version-controlled repository. + +During the cookie generation process, the target GitHub repository was +identified and inserted into the workflows configuration, as well as +into the ``README.md`` file. + +Add this repository as a remote to your local git repository: + +.. code-block:: bash + + $ git remote add origin git@github.com:myusername/rmsfkit + +(substituting ``myusername`` for your GitHub username). Make sure that this repository exists on GitHub and is *empty*. Then, push the local code to GitHub: + +.. code-block:: bash + + $ git push origin main + +Open the repository in GitHub and navigate to actions. Here you can +see the status of the tests. If all was done correctly in the previous +sections, these tests will pass! + +**Progress: MDAKit requirements** + +#. **✓ Uses MDAnalysis** +#. **✓ Open source + OSI license** +#. **✓ Versioned + on a version-controlled repository** +#. **✓ Designated authors and maintainers** +#. *(At least) minimal documentation* +#. **✓(At least) minimal regression tests** +#. **✓ Installable as a standard package** +#. **✓ (Recommended) community information available** +#. *(Recommended) on a package distribution platform* + diff --git a/docs/source/making-a-kit/guided-example/use-cookiecutter.rst b/docs/source/making-a-kit/guided-example/use-cookiecutter.rst new file mode 100644 index 00000000..69674e2a --- /dev/null +++ b/docs/source/making-a-kit/guided-example/use-cookiecutter.rst @@ -0,0 +1,97 @@ +************************************ +Part 1: Using the MDAKit cookicutter +************************************ + +*For a video demonstration of this section,* +`click here `_. + +The MDAKits cookiecutter (a +`Cookiecutter tool `_) +can be used to rapidly develop a FAIR-compliant MDAKit by generating +placeholder code for documentation, testing and installation. + +More infromation on its usage is can be found in the +`MDAKit cookiecutter documentation `_, +but in short, using the cookiecutter involves running a command line +prompt and then responding to a set of subsequent prompts. You will +need Python 3.10+ and the +`Cookiecutter tool `_ installed. + +In this example, we use the cookiecutter as follows to generate +a new MDAKit template ``rmsfkit``. Some prompts are left +blank, accepting the the default option. + +.. code-block:: bash + + $ cookiecutter gh:MDAnalysis/cookiecutter-mdakit + + project_name (ProjectName): rmsfkit + repo_name (rmsfkit): + package_name (rmsfkit): + description (A short description of the project.): A package to perform RMSF analysis on molecular dynamics data. + github_username (Your personal GitHub username): myusername + github_host_account (GitHub account for hosting rmsfkit (e.g. myusername or an organization account). Press Enter to use myusername): + author_name (Your name (or your organization/company/team)): Firstname Lastname + author_email (Your email (or your organization/company/team)): myemail@email-provider.com + Select dependency_source: + 1 - Prefer conda-forge over the default anaconda channel with pip fallback + 2 - Prefer default anaconda channel with pip fallback + 3 - Dependencies from pip only (no conda) + Choose from [1/2/3] (1): + Select include_ReadTheDocs: + 1 - y + 2 - n + Choose from [1/2] (1): + template_analysis_class (Class name to template (e.g. Rmsfkit ). Press Enter to skip including analysis templates): + +This generates a new git repository named ``rmsfkit`` (note, the +author of the initial commit will match that of the user's global git +configurations). + +Navigating into this directory, we find a number of files and +subdirectories. The +`cookiecutter documentation `_ +provides more information on these, but those to take note of now +include: + +- ``LICENSE`` -- a text file containing the license information (GLPv2+ by default) +- ``AUTHORS.md`` -- a template text file for listing code authors as each + new contribution is made. Auto-populated with your name as the + initial project creator. +- ``README.md`` -- a text file containing the project description and other + useful information; typically landing page content rendered on GitHub. +- ``pyproject.toml`` - a configuration file with information for + packaging (and other tools). +- ``rmsfkit/`` -- template for the python package source code, i.e. + where your code (and tests) will live. +- ``CODE_OF_CONDUCT.md`` and ``CONTRIBUTING.md`` -- text files containing + community guidelines and instructions on reporting issues and contributing + (once the code is hosted on GitHub). +- ``docs/`` -- folder containing documentation template, instructions + and build environment documented within. + +We'll address other files as they come up in subsequent parts. + + +** Progress: MDAKit requirementsi** + +#. *Uses MDAnalysis* +#. **✓ Open source + OSI license** -- the default GPLv2 license falls + under OSI. If you wish to choose another license, you can replace + the default LICENSE file with another. See :ref:`` for more. +#. *Versioned + on a version-controlled repository* -- the cookiecutter has + set up versioning with git; in a later step, we'll add it to the the + GitHub repository. +#. **✓ Designated authors and maintainers** -- in the generated AUTHORS.md + file, assuming that the maintained and author are the same person. + More authors will be added as the project evolves. +#. *(At least) minimal documentation* -- the generated README.md file + provides a form of minimal documentation; however, we will be adding + more documentation in a later step. +#. *(At least) minimal regression tests* +#. **✓ Installable as a standard package** -- through `rmsfkit/` and + `pyproject.toml`. +#. **✓ (Recommended) community information available** -- provided with + CODE_OF_CONDUCT.md and CONTRIBUTING.md. Adjust these files to suit your + needs. +#. *(Recommended) on a package distribution platform* diff --git a/docs/source/making-a-kit/hosting.rst b/docs/source/making-a-kit/hosting.rst new file mode 100644 index 00000000..a3b5b151 --- /dev/null +++ b/docs/source/making-a-kit/hosting.rst @@ -0,0 +1,22 @@ +*********************************************** +Hosting code in a version controlled repository +*********************************************** + +Since the MDAKits registry makes heavy use of the GitHub actions +infrastructure, registration of a kit requires that all code maintainers +also have a GitHub account for communication purposes. +For this reason, if your code is not already hosted in an accessible +version controlled repository, hosting on `GitHub `_ +is recommended, although other services such as +`Bitbucket `_, `GitLab `_, +or self hosting is possible. + +The registry does not require that your code be available through +packaging repositories such as the Python Package Index or conda-forge, +although having your code available through these services is highly +encouraged. + +After registration, users can find the installation instructions for the +source code on your MDAKit page, which is specified in the +``src_install`` field in the ``metadata.yaml`` file (see :ref:`specification`). + diff --git a/docs/source/making-a-kit/license.rst b/docs/source/making-a-kit/license.rst new file mode 100644 index 00000000..d85cc84a --- /dev/null +++ b/docs/source/making-a-kit/license.rst @@ -0,0 +1,24 @@ +.. _licensing: + +********* +Licensing +********* + +One of the more pressing requirements for kit registration is clearly +identifying the license that is applied to your code. This is typically +included in a LICENSE file at the top level of your repository. +`Without a license `_, the +only assumption a user can make about your code is that they are not in +a position to use your code. + +Your license needs to be compatible with the GPLv2+ license currently +used by MDAnalysis, in addition to the licenses of any other packages +your mdakit depends on. Take time to consider how you would like to +`license your project `_. + +Further information on open source licensing can be found from sources +such as: `choose a license `_, +`tl;dr Legal `_, the +`Open Source Initiative `_, and the +`Software Sustainability Insitute `_. + diff --git a/docs/source/making-a-kit/testing.rst b/docs/source/making-a-kit/testing.rst new file mode 100644 index 00000000..19f1c954 --- /dev/null +++ b/docs/source/making-a-kit/testing.rst @@ -0,0 +1,51 @@ +******* +Testing +******* + +We also require that minimal regression tests are present. +These tests are not just useful for when you make changes to your code, +but also when any package dependencies (e.g. MDAnalysis, NumPy, and +Python) change. Additionally, tests inform the users of your packages +that the code performs at least the way you say it should and give +them confidence that it can be used. + +Basic tests can be written with a variety of packages, such as the +`pytest package `_ (the default +choice for MDAnalysis organization projects) or the +`unittest package `_. +Further improvements to your testing procedure may include automatically +running the tests on pushing to your remote repositories, often referred +to as continuous integration (CI). CI can be set up using repository +pipeline tools, such as `GitHub Actions `_. + +When submitting an MDAKit to the registry, include the instructions for +running the tests in the required ``metadata.yaml`` file (see a full example +in the `registration `_ section below). +Assuming that your tests are in a ``test/`` directory at the top level of +your repository, you could define your test commands as: + +.. code-block:: yaml + + run_tests: + - git clone latest + - pytest -v tests/ + +This makes a clone of your repository based on your latest +`release tag on GitHub `_ +and navigates into the repository root. Note that this is not a true +:program:`git` command, but is instead specific to the MDAKits registry +workflow and depends on the ``project_home`` field in the ``metadata.yaml`` +file (see :ref:`specification`). +The :program:`pytest` command then runs the tests found inside the ``tests/`` directory. +If your tests are elsewhere, change this path appropriately. + +Dependencies that are only required for testing are indicated in the +``test_dependencies`` object. Suppose your package uses :program:`pytest` +and used the `MDAnalysisTests `_ +for sample data. This is reflected in your MDAKit metadata with: + +.. code-block:: yaml + + test_dependencies: + - mamba install pytest MDAnalysis + diff --git a/docs/source/makingakit.rst b/docs/source/makingakit.rst index 9bdced09..8334e498 100644 --- a/docs/source/makingakit.rst +++ b/docs/source/makingakit.rst @@ -2,684 +2,36 @@ Making an MDAKit **************** -Here, we outline the process of creating an MDAKit that fulfills all of the requirements for acceptance into the MDAKit registry. -For a video walk-through of this tutorial, watch `our recorded tutorial `_ on YouTube. +The MDAKit registry has a number of :ref:`requirements ` +to ensure that registered packages are FAIR-compliant and hold up to +an ideal scientific standard. -Unlike the code in the core MDAnalysis library, the structure of an MDAKit is much less restrictive. -In order to be accepted, there are several :ref:`requirements ` that must be addressed: -#. Code in the package uses the MDAnalysis library -#. The code is open source and published under an `OSI approved license `_ -#. Code is versioned and provided in an accessible version-controlled repository (GitHub, GitLab, Bitbucket, etc.) -#. Code authors and maintainers are clearly designated -#. Minimal documentation is provided (what your code does, how to install it, - and how to use it) -#. At least minimal regression tests are present; continuous integration is encouraged -#. The source code is installable as a standard package +**Building a Kit from the cookiecutter** -It is also highly encouraged that the MDAKit also satisfies: +Without prior experience, some of the MDAKit registry requirements +can be daunting. The `MDAKit cookiecutter `_ +can be used to aid rapid development of a FAIR-compliant MDAKit +by generating placeholder code for documentation, testing and +installation. In this section, we provide a walk-through for the +creation of an MDAKit using the cookiecutter. -#. Information on bug reporting, user discussions, and community guidelines is made available -#. The code is made available on a package distribution platform (e.g. PyPi or conda-forge). +.. toctree:: + :maxdepth: 2 -These requirements ensure that registered packages are FAIR-compliant and hold up to an ideal scientific standard. -Without prior experience, some of the requirements listed above can be daunting. -To aid in this process, we make use of the MDAKit cookiecutter in this example. + making-a-kit/from-cookiecutter -Building from the cookiecutter -############################## -The MDAKits cookiecutter template (using the `Cookiecutter tool `_) can be used to rapidly develop a FAIR-compliant MDAKit by generating placeholder code for documentation, testing, and installation. -While its usage is outlined in detail in the `MDAKit cookiecutter `_ documentation, here we provide a full walk-through for creating an RMSF analysis kit, recreating the functionality of the `RMSF analysis class `_ in the core library. +**Building from an existing project** -Starting from an environment with Python 3.10+ and the `Cookiecutter tool `_, the MDAKit template is generated using +If you have an existing package you wish to register as an MDAKit, +(or if you wish to develop one yourself without using the cookiecutter), +chances are very little restructuring is needed for registration. +The primary concern is ensuring that the core MDAKit requirements +are met - check the links below for things to consider. -.. code-block:: +.. toctree:: + :maxdepth: 1 - $ cookiecutter gh:MDAnalysis/cookiecutter-mdakit - - project_name [ProjectName]: rmsfkit - repo_name [rmsfkit]: - package_name [rmsfkit]: - description [A short description of the project.]: A package to perform RMSF analysis on molecular dynamics data. - github_username [Your personal GitHub username]: myusername - github_host_account [GitHub account for hosting rmsfkit (e.g. myusername or an organization account). Press Enter to use myusername]: - author_name [Your name (or your organization/company/team)]: Firstname Lastname - author_email [Your email (or your organization/company/team)]: myemail@email-provider.com - Select dependency_source: - 1 - Prefer conda-forge over the default anaconda channel with pip fallback - 2 - Prefer default anaconda channel with pip fallback - 3 - Dependencies from pip only (no conda) - Choose from 1, 2, 3 [1]: - Select include_ReadTheDocs: - 1 - y - 2 - n - Choose from 1, 2 [1]: - template_analysis_class [Class name to template (e.g. Rmsfkit ). Press Enter to skip including analysis templates]: - -This generates a new git repository named ``rmsfkit`` (note that the author of the initial commit will match that of the user's global git configurations). -Navigating into this directory, we find the following notable files and directories: - -#. LICENSE -- defaults to GLPv2+ -#. pyproject.toml -- pip dependencies -#. MANIFEST.in -- Packaging information -#. README.md -- Project description and typically landing page content rendered on GitHub -#. docs/ -- Sphinx documentation template, instructions and build environment documented within! -#. rmsfkit/ -- Python package source code template - -See `the cookiecutter documentation `_ for information on additional files. -Using the cookiecutter checks off a couple of the requirements for a valid MDAKit right from the start: - -* Open source code is published under an OSI approved license \[defaults to GPLv2\] -* Code authors and maintainers are clearly designated (assuming that the maintainer and the author are the same person) -* Code is installable as a standard package (optional feature) -* Information on bug reporting, user discussions, and community guidelines is made available (optional feature) - -Adding our code to the newly created repository -*********************************************** - -Since we are recreating the RMSF analysis, we are simply copy and pasting the -analysis class into the ``rmsfkit/rmsfkit.py`` file, where the documentation has been -trimmed for the sake of brevity. Documentation is written in `reStructuredText syntax `_ -syntax for building with Sphinx. The contents of the file should now resemble -the following code block: - -.. code-block:: python - - """ - rmsfkit.py - A package to perform RMSF analysis on molecular dynamics data. - - All code and documentation, both of which are trimmed for the sake of brevity, - are taken from the MDAnalysis Python package. Used under GPLv2+. - """ - - from MDAnalysis.analysis.base import AnalysisBase - import numpy as np - - class RMSF(AnalysisBase): - r"""Calculate RMSF of given atoms across a trajectory. - - Note - ---- - No RMSD-superposition is performed; it is assumed that the user is - providing a trajectory where the protein of interest has been structurally - aligned to a reference structure. The protein also has to be whole because - periodic boundaries are not taken into account. - - Run the analysis with :meth:`RMSF.run`, which stores the results in the - array :attr:`RMSF.results.rmsf`. - - """ - def __init__(self, atomgroup, **kwargs): - r"""Parameters - ---------- - atomgroup : AtomGroup - Atoms for which RMSF is calculated - verbose : bool (optional) - Show detailed progress of the calculation if set to ``True``; the - default is ``False``. - - Raises - ------ - ValueError - raised if negative values are calculated, which indicates that a - numerical overflow or underflow occurred - - Notes - ----- - The root mean square fluctuation of an atom :math:`i` is computed as the - time average - - .. math:: - - \rho_i = \sqrt{\left\langle (\mathbf{x}_i - \langle\mathbf{x}_i\rangle)^2 \right\rangle} - - No mass weighting is performed. - - This method implements an algorithm for computing sums of squares while - avoiding overflows and underflows :cite:p:`Welford1962`. - - References - ---------- - .. bibliography:: - :filter: False - - Welford1962 - - """ - super(RMSF, self).__init__(atomgroup.universe.trajectory, **kwargs) - self.atomgroup = atomgroup - - def _prepare(self): - self.sumsquares = np.zeros((self.atomgroup.n_atoms, 3)) - self.mean = self.sumsquares.copy() - - def _single_frame(self): - k = self._frame_index - self.sumsquares += (k / (k+1.0)) * (self.atomgroup.positions - self.mean) ** 2 - self.mean = (k * self.mean + self.atomgroup.positions) / (k + 1) - - def _conclude(self): - k = self._frame_index - self.results.rmsf = np.sqrt(self.sumsquares.sum(axis=1) / (k + 1)) - - if not (self.results.rmsf >= 0).all(): - raise ValueError("Some RMSF values negative; overflow " + - "or underflow occurred") - - -To make our ``RMSF`` analysis class easier to access, we import it in ``__init__.py``. Add - -.. code-block:: python - - from .rmsfkit import RMSF - -This satisfies the first MDAKit requirement ("code in the package uses MDAnalysis"). - -Filling in tests -**************** - -Once again, we take directly from the MDAnalysis package. -Here we take the `RMSF testing class `_ and update the contents of ``rmsfkit/tests/test_rmsfkit.py`` to reflect the following code block: - -.. code-block:: python - - """ - Unit and regression test for the rmsfkit package. - - The TestRMSF class was taken from the MDAnalysis rms tests file and - the relevant modules were switched. - """ - - # Import package, test suite, and other packages as needed - from MDAnalysisTests.datafiles import GRO, XTC, rmsfArray - import MDAnalysis as mda - - from numpy.testing import assert_equal, assert_almost_equal - import numpy as np - import os - import pytest - - import rmsfkit - import sys - - def test_rmsfkit_imported(): - """Sample test, will always pass so long as import statement worked""" - assert "rmsfkit" in sys.modules - - - class TestRMSF(object): - @pytest.fixture() - def universe(self): - return mda.Universe(GRO, XTC) - - def test_rmsf(self, universe): - rmsfs = rmsfkit.RMSF(universe.select_atoms('name CA')) - rmsfs.run() - test_rmsfs = np.load(rmsfArray) - - assert_almost_equal(rmsfs.results.rmsf, test_rmsfs, 5, - err_msg="error: rmsf profile should match test " - "values") - - def test_rmsf_single_frame(self, universe): - rmsfs = rmsfkit.RMSF(universe.select_atoms('name CA')).run(start=5, stop=6) - - assert_almost_equal(rmsfs.results.rmsf, 0, 5, - err_msg="error: rmsfs should all be zero") - - def test_rmsf_identical_frames(self, universe, tmpdir): - - outfile = os.path.join(str(tmpdir), 'rmsf.xtc') - - # write a dummy trajectory of all the same frame - with mda.Writer(outfile, universe.atoms.n_atoms) as W: - for _ in range(universe.trajectory.n_frames): - W.write(universe) - - universe = mda.Universe(GRO, outfile) - rmsfs = rmsfkit.RMSF(universe.select_atoms('name CA')) - rmsfs.run() - assert_almost_equal(rmsfs.results.rmsf, 0, 5, - err_msg="error: rmsfs should all be 0") - -Since these tests use files included with the ``MDAnalysisTests`` package, -we need to add it as a dependency in ``devtools/conda-envs/test_env.yaml``. -Additionally, we need to add this dependency to the ``pyproject.toml`` -file. Under the ``[project.optional-dependencies]`` table, ensure that -``MDAnalysisTests>=2.0.0`` is listed in ``test``. - -.. code-block:: toml - - [project.optional-dependencies] - test = [ - "pytest>=6.0", - "pytest-xdist>=2.5", - "pytest-cov>=3.0", - "MDAnalysisTests>=2.0.0", - ] - - - -Confirm that the code and tests work -************************************ - -Up until this point we haven't tested any of the code. Following the the -instructions from the generated ``README.md``, we can create a testing -environment using ``conda`` or ``mamba`` (recommended). - -.. code-block:: bash - - $ mamba create -n rmsfkit - $ mamba env update --name rmsfkit --file devtools/conda-envs/test_env.yaml - $ mamba activate rmsfkit - $ pip install -e . - -This installs the package as well as the testing environment. We can run tests locally using: - -.. code-block:: bash - - $ pytest rmsfkit/tests - -This should pass without errors, but with some potential warnings. Local -tests passing is only half of the testing requirement for an MDAKit. To fully -satisfy the MDAKit testing requirement, tests must also pass through continuous -integration services. The cookiecutter generates the necessary GitHub workflow -files and can be found in ``.github/workflows/gh-ci.yaml``. Since our tests use -the ``MDAnalysisTests`` package, we need to make one change to this file. - -Change the block - -.. code-block:: yaml - - - name: Install MDAnalysis version - uses: MDAnalysis/install-mdanalysis@main - with: - version: ${{ matrix.mdanalysis-version }} - install-tests: false - installer: mamba - shell: bash -l {0} - -to - -.. code-block:: yaml - - - name: Install MDAnalysis version - uses: MDAnalysis/install-mdanalysis@main - with: - version: ${{ matrix.mdanalysis-version }} - install-tests: true # this needs to be true! - installer: mamba - shell: bash -l {0} - -Finishing up the tests -********************** - -Our code only exists locally, but will need to be uploaded to a GitHub to run -the prebuilt continuous integration provided by the cookiecutter. During the -cookie generation process, the target GitHub repository was identified and -inserted into the workflows configuration as well as the ``README.md`` file. Add -this repository as a remote to your local git repository, - -.. code-block:: bash - - $ git remote add origin git@github.com:myusername/rmsfkit - -substituting ``myusername`` for your GitHub username. Make sure that this repository exists on GitHub and is *empty*. - -.. code-block:: bash - - $ git push origin main - -Open the repository in GitHub and navigate to actions. Here you can see the status of the tests. If all was done correctly in the previous sections, these tests will pass! - -After this point, two more requirements are satisfied: - -* Code is versioned and provides in an accessible version-controlled repository -* Tests and continuous integration are present - -Providing documentation -*********************** - -The cookiecutter includes a `Read the Docs `_ -configuration as well a premade documentation environment file that is used by -Read the Docs and for building locally. First, we need to install the correct -environment for building the documentation. In the ``docs/`` directory, run: - -.. code-block:: bash - - mamba env update --name rmsfkit -f requirements.yaml - -We can now build the documentation HTML files using the included ``Makefile``. Without looking at any of the documentation source files, run: - -.. code-block:: bash - - make html - -This will convert the reStructuredText files into HTML in the ``_build`` directory. -Open ``index.html`` and look around. - -Notice that the "API Documentation" does not contain all of the information found in the docstrings of our code. -This is because within ``source/api.rst``, the only contents are: - -.. code-block:: rst - - API Documentation - ================= - - .. autosummary:: - :toctree: autosummary - - rmsfkit - -Instead, it should contain: - -.. code-block:: rst - - API Documentation - ================= - - .. autosummary:: - :toctree: autosummary - - rmsfkit.RMSF - -After making the change, rerun ``make html`` and refresh the page. There will -now be a table entry for the RMSF class. Clicking this entry will open the -documentation that was present in the docstrings! Other than the newly visible -docstring contents, the rest of the documentation is completely empty. Let's -update the ``index.rst`` file to include a short description of the package: - -.. code-block:: rst - - Welcome to rmsfkit's documentation! - ========================================================= - - ``rmsfkit`` is an example MDAKit that implements the functionality of the `analysis.rms.RMSF` class within the MDAnalysis package. - This MDAKit does not serve as a replacement for this functionality and using this MDAKit for real work is discouraged. - - .. toctree:: - :maxdepth: 2 - :caption: Contents: - - getting_started - api - - - Indices and tables - ================== - - * :ref:`genindex` - * :ref:`modindex` - * :ref:`search` - -Additionally, we can update ``getting_started.rst`` to let potential users know how to install the package. - -.. code-block:: rst - - Getting Started - =============== - - The ``rmsfkit`` package is installable from source. - - .. code-block:: bash - - git clone git@github.com:ianmkenney/rmsfkit.git - cd rmsfkit/ - pip install . - -Run the ``make html`` command again and refresh the browser window to view the resulting changes. -You'll notice the RMSF class documentation has an unformatted citation (``:cite:p:`Welford1962```). -If your documentation need citations, you can easily include them using the bibtex format. -We first create the bibtex file, ``references.bib`` in the ``doc/source/`` directory: - -.. code-block:: - - @article{Welford1962, - author = { B. P. Welford}, - title = {Note on a Method for Calculating Corrected Sums of Squares and Products}, - journal = {Technometrics}, - volume = {4}, - number = {3}, - pages = {419-420}, - year = {1962}, - publisher = {Taylor & Francis}, - doi = {10.1080/00401706.1962.10490022} - } - -In ``conf.py``, we need to add a new extension (``sphinxcontrib.bibtex``) as well as the name of the bibtex file. - -.. code-block:: - - extensions = [ - 'sphinx.ext.autosummary', - 'sphinx.ext.autodoc', - 'sphinx.ext.mathjax', - 'sphinx.ext.viewcode', - 'sphinx.ext.napoleon', - 'sphinx.ext.intersphinx', - 'sphinx.ext.extlinks', - 'sphinxcontrib.bibtex', # add this line - ] - - bibtex_bibfiles = ['references.bib'] - -In addition, we have to add this extension the install requirements in ``docs/requirements.yaml``. -Add ``- sphinxcontrib-bibtex`` as an additional dependency here. - -Update your environment with - -.. code-block:: bash - - mamba env update --name rmsfkit -f requirements.yaml - -before once again running `make html`. -Refreshing the RMSF documentation will now show a properly formatted citation using the information in the bibtex file. - -Deploying the documentation -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Log into `Read the Docs `_ and navigate to the -dashboard. Click the "Import a Project" button and find the repository in the -list. Click the "+" and confirm that the name, URL, and default branch are -correct. - -.. image:: img/rmsftutorial/import.png - :alt: Import a project into RTD - -.. image:: img/rmsftutorial/adding.png - :alt: Adding your repository to RTD - -Clicking next will begin the deployment. This immediately starts the build -process, which can be confirmed by clicking the "Builds" tab. Once the build is -completed, you can view the deployed documentation, which is now public. This -completes the documentation requirement for an MDAKit. Notice that the "docs" -badge in the GitHub rendered ``README.md`` is now green. - -Making an initial release -************************* - -The MDAKit cookiecutter uses the `versioneer utility `_ for version management. -In brief, software versions are set through `git tags `_. -Since the kit is initiated without tags, the current version is set to ``0.0.0``. - -.. code-block:: - - >>> import rmsfkit - >>> rmsfkit.__version__ - '0+untagged.3.g3eed836' - -We can bump this to a ``0.1.0`` in preparation for an initial release on GitHub. - -.. code-block:: bash - - $ git tag 0.1.0 - -This should now be reflected in the interpreter (may require a package reinstall): - -.. code-block:: bash - - >>> import rmsfkit - >>> rmsfkit.__version__ - '0.1.0' - -Tags are not pushed to remote repositories by default. To push the `0.1.0` tag, use: - -.. code-block:: bash - - $ git push --tags - -Viewing the repository tags page on GitHub, you should now see a ``0.1.0`` tag, which can then be used to create a release by expanding its menu options. -Enter a release name, such as `v0.1.0` and publish! - -.. image:: img/rmsftutorial/creating_a_release.gif - :alt: Process for creating a release by making a tag - -Building from an existing project -################################# - -Registering an existing package as an MDAKit is a straightforward process. -Since the structure of an MDAKit is not as strict as the code found in the MDAnalysis core library, chances are very little restructuring is needed for registration. -The primary concern is ensuring that the core MDAKit :ref:`requirements` are met, as listed at the top of this document. - -Licensing -********* - -One of the more pressing requirements for kit registration is clearly identifying the license that is applied to your code. -This is typically included in a LICENSE file at the top level of your repository. -`Without a license `_, the only assumption a user can make about your code is that they are not in a position to use your code. -Your license needs to be compatible with the GPLv2+ license currently used by MDAnalysis, in addition to the licenses of any other packages your mdakit depends on. -Take time to consider how you would like to `license your project `_. -Take time to consider how you would like to license your project. Further information on open source licensing can be found from sources such as: `choose a license `_, `tl;dr Legal `_, the `Open Source Initiative `_, and the `Software Sustainability Insitute `_. - -Hosting code in a version controlled repository -*********************************************** - -Since the MDAKits registry makes heavy use of the GitHub actions infrastructure, registration of a kit requires that all code maintainers also have a GitHub account for communication purposes. -For this reason, if your code is not already hosted in an accessible version controlled repository, hosting on `GitHub `_ is recommended, although other services such as `Bitbucket `_, `GitLab `_, or self hosting is possible. -The registry does not require that your code be available through packaging repositories such as the Python Package Index or conda-forge, although having your code available through these services is highly encouraged. -After registration, users can find the installation instructions for the source code on your MDAKit page, which is specified in the ``src_install`` field in the ``metadata.yaml`` file (see :ref:`specification`). - -Documentation -************* - -Basic documentation is required for MDAKit registration. -The detail and depth of the documentation is ultimately up to you, but we require at a minimum that you provide README-style documentation explaining what the code is supposed to do, how to install it, and the basics of its use. -Although this is the minimum, we highly recommend that you consider generating your documentation with dedicated tools such as `Sphinx `_, which allows you to generate static documentation using `reStructuredText `_ formatted plain-text directly from your code. -This makes it easier for your documentation to change alongside code changes. - -Using a documentation hosting service such as `Read the Docs `_ or `GitHub Pages `_ makes public access to your generated documentation automatic. - -Testing -******* - -We also require that minimal regression tests are present. -These tests are not just useful for when you make changes to your code, but also when any package dependencies (e.g. MDAnalysis, NumPy, and Python) change. -Additionally, tests inform the users of your packages that the code performs at least the way you say it should and give them confidence that it can be used. -Basic tests can be written with a variety of packages, such as the `pytest package `_ (the default choice for MDAnalysis organization projects) or the `unittest package `_. -Further improvements to your testing procedure may include automatically running the tests on pushing to your remote repositories, often referred to as continuous integration (CI). -CI can be set up using repository pipeline tools, such as `GitHub Actions `_. - -When submitting an MDAKit to the registry, include the instructions for running the tests in the required ``metadata.yaml`` file (see a full example in the `registration `_ section below). -Assuming that your tests are in a ``test/`` directory at the top level of your repository, you could define your test commands as: - -.. code-block:: yaml - - run_tests: - - git clone latest - - pytest -v tests/ - -This makes a clone of your repository based on your latest `release tag on GitHub `_ and navigates into the repository root. Note that this is not a true :program:`git` command, but is instead specific to the MDAKits registry workflow and depends on the ``project_home`` field in the ``metadata.yaml`` file (see :ref:`specification`). -The :program:`pytest` command then runs the tests found inside the ``tests/`` directory. -If your tests are elsewhere, change this path appropriately. - -Dependencies that are only required for testing are indicated in the ``test_dependencies`` object. -Suppose your package uses :program:`pytest` and used the `MDAnalysisTests `_ for sample data. -This is reflected in your MDAKit metadata with - -.. code-block:: yaml - - test_dependencies: - - mamba install pytest MDAnalysis - -.. _registration: - -Registering an MDAKit -##################### - -The MDAKit registration is the same regardless of the creation process for the kit. -For simplicity, the follow examples will reference the ``rmsfkit`` MDAKit created in the cookiecutter section. -In order to submit your MDAKit to the registry, you will need to create a pull request on GitHub against the MDAnalysis/MDAKits repository. -Do this by creating a fork of the MDAnalysis/MDAKits repository. -Clone the fork to your machine, navigate into ``MDAKits/mdakits/``, and make an empty directory with your MDAKit name: - -.. code-block:: bash - - git clone git@github.com:yourusername/MDAKits - cd MDAKits/mdakits - mkdir rmsfkit/ - cd rmsfkit - -Add the ``metadata.yaml`` for your MDAKit in this directory (see -:ref:`specification` for details). -The contents of ``metadata.yaml`` for ``rmsfkit`` are: - -.. code-block:: yaml - - project_name: rmsfkit - authors: - - https://github.com/yourusername/rmsfkit/blob/main/AUTHORS.md - maintainers: - - yourusername - description: - An analysis module for calculating the root-mean-square fluctuation of atoms in molecular dynamics simulations. - keywords: - - rms - - rmsf - license: GPL-2.0-or-later - project_home: https://github.com/yourusername/rmsfkit - documentation_home: https://rmsfkit.readthedocs.io/en/latest/ - documentation_type: API - src_install: - - git clone https://github.com/yourusername/rmsfkit.git - - cd rmsfkit/ - - pip install . - - ## Optional entries - python_requires: ">=3.9" - mdanalysis_requires: ">=2.0.0" - run_tests: - - pytest --pyargs rmsfkit.tests - development_status: Beta - -Commit and push this to your fork: - -.. code-block:: bash - - git add metadata.yaml - git commit -m "Adding rmsfkit" - git push origin main - -Refresh the forked repository page in your browser. -Under "Contribute", open a pull request. -Add a title with the name of the kit and add a quick description. -Click "Create pull request" and wait for the tests to pass. -Once this is done, you can add a comment along the lines of "@MDAnalysis/mdakits-reviewers, ready for review". -The reviewers will get back to you with any change requests before merging it in as a kit. -At this point there are no additional steps for registering your kit! - -.. image:: img/rmsftutorial/submitting.gif - :alt: Process for submitting a kit to the registry - -Maintaining a kit -################# - -There are a variety of reasons a kit may behave unexpectedly after being submitted to the registry. -Apart from actively developing the kit, changes in kit dependencies, or even Python itself, can introduce (deprecate) new (old) functionality. -For this reason, the kits' continuous integration is rerun weekly to confirm the kits expected behavior. -In the event that a kit no longer passes its tests, an issue in MDAnalysis/MDAKits is automatically raised while notifying the maintainers indicated in the `metadata.yaml` file. -While the registry developers will be happy to help where possible, ultimately, the maintainers of the MDAKit are responsible for resolving such issues and ensuring that the tests pass. -The issue will automatically close after the next CI run if the tests pass again. - -.. _read the docs: https://readthedocs.org + making-a-kit/licensing + making-a-kit/hosting From 9e52328ed67c34abab2858597a6237b9042eff35 Mon Sep 17 00:00:00 2001 From: fiona-naughton Date: Mon, 30 Sep 2024 00:18:48 +1000 Subject: [PATCH 02/11] from-existing updates --- docs/source/making-a-kit/documentation.rst | 22 ++++---- docs/source/making-a-kit/hosting.rst | 30 ++++++----- docs/source/making-a-kit/license.rst | 4 +- docs/source/making-a-kit/testing.rst | 58 +++++++--------------- docs/source/makingakit.rst | 27 ++++++---- 5 files changed, 65 insertions(+), 76 deletions(-) diff --git a/docs/source/making-a-kit/documentation.rst b/docs/source/making-a-kit/documentation.rst index 79e8a6b4..0a9d7191 100644 --- a/docs/source/making-a-kit/documentation.rst +++ b/docs/source/making-a-kit/documentation.rst @@ -1,23 +1,25 @@ +.. _documentation: + ************* Documentation ************* -Basic documentation is required for MDAKit registration. -The detail and depth of the documentation is ultimately up to you, but -we require at a minimum that you provide README-style documentation -explaining what the code is supposed to do, how to install it, and the -basics of its use. +Basic documentation is **required** for MDAKit registration. The detail +and depth of the documentation is ultimately up to you, but we require +**at a minimum** that you provide README-style documentation explaining +what the code is supposed to do, how to install it, and the basics of +its use. Although this is the minimum, we highly recommend that you consider generating your documentation with dedicated tools such as `Sphinx `_, which allows you to generate static documentation using -`reStructuredText `_ -formatted plain-text directly from your code. This makes it easier for +`reStructuredText `_ +-formatted plain text directly from your code. This makes it easier for your documentation to change alongside code changes. -Using a documentation hosting service such as +Using a **documentation hosting service** such as `Read the Docs `_ or -`GitHub Pages `_ makes public access to your -automatically generated documentation. +`GitHub Pages `_ allows easy public access +to your automatically generated documentation. diff --git a/docs/source/making-a-kit/hosting.rst b/docs/source/making-a-kit/hosting.rst index a3b5b151..0108a7c3 100644 --- a/docs/source/making-a-kit/hosting.rst +++ b/docs/source/making-a-kit/hosting.rst @@ -1,22 +1,26 @@ +.. _hosting: + *********************************************** Hosting code in a version controlled repository *********************************************** -Since the MDAKits registry makes heavy use of the GitHub actions -infrastructure, registration of a kit requires that all code maintainers -also have a GitHub account for communication purposes. +The MDAKits registry makes heavy use of the GitHub actions +infrastructure. Thus, registration of a kit **requires** that all code +maintainers also have a GitHub account for communication purposes. + For this reason, if your code is not already hosted in an accessible version controlled repository, hosting on `GitHub `_ -is recommended, although other services such as +is *recommended*, although other services such as `Bitbucket `_, `GitLab `_, -or self hosting is possible. - -The registry does not require that your code be available through -packaging repositories such as the Python Package Index or conda-forge, -although having your code available through these services is highly -encouraged. +or self hosting are possible. -After registration, users can find the installation instructions for the -source code on your MDAKit page, which is specified in the -``src_install`` field in the ``metadata.yaml`` file (see :ref:`specification`). +The registry **does not require** that your code be available through +packaging repositories such as the Python Package Index (`PyPi `_) +or `conda-forge `_, although having your code +available through these services is highly encouraged. +However, your code **does** need to be installable from source. You'll +specify the commands for this in the ``scr_install`` field of your +``metadata.yaml`` file (see the :ref:`metadata.yaml documentation `). +This will then be made available to users on your MDAKit's page on the +:ref:`Registry `. diff --git a/docs/source/making-a-kit/license.rst b/docs/source/making-a-kit/license.rst index d85cc84a..41f95ac7 100644 --- a/docs/source/making-a-kit/license.rst +++ b/docs/source/making-a-kit/license.rst @@ -6,14 +6,14 @@ Licensing One of the more pressing requirements for kit registration is clearly identifying the license that is applied to your code. This is typically -included in a LICENSE file at the top level of your repository. +included in a ``LICENSE`` file at the top level of your repository. `Without a license `_, the only assumption a user can make about your code is that they are not in a position to use your code. Your license needs to be compatible with the GPLv2+ license currently used by MDAnalysis, in addition to the licenses of any other packages -your mdakit depends on. Take time to consider how you would like to +your MDAKit depends on. Take time to consider how you would like to `license your project `_. Further information on open source licensing can be found from sources diff --git a/docs/source/making-a-kit/testing.rst b/docs/source/making-a-kit/testing.rst index 19f1c954..2c968046 100644 --- a/docs/source/making-a-kit/testing.rst +++ b/docs/source/making-a-kit/testing.rst @@ -1,51 +1,27 @@ +.. _testing: + ******* Testing ******* -We also require that minimal regression tests are present. -These tests are not just useful for when you make changes to your code, -but also when any package dependencies (e.g. MDAnalysis, NumPy, and -Python) change. Additionally, tests inform the users of your packages -that the code performs at least the way you say it should and give -them confidence that it can be used. +The MDAKit Registry requires that **minimal regression tests are +present**. These tests are not just useful for when you make changes +to your code, but also when any package dependencies (e.g. MDAnalysis, +NumPy, and Python) change. Additionally, tests inform the users of +your packages that the code performs at least the way you say it +should and give them confidence that it can be used. -Basic tests can be written with a variety of packages, such as the -`pytest package `_ (the default +Basic tests can be written with a variety of packages, such as +`pytest `_ (the default choice for MDAnalysis organization projects) or the -`unittest package `_. +`unittest `_. + Further improvements to your testing procedure may include automatically running the tests on pushing to your remote repositories, often referred -to as continuous integration (CI). CI can be set up using repository +to as **continuous integration (CI)**. CI can be set up using repository pipeline tools, such as `GitHub Actions `_. -When submitting an MDAKit to the registry, include the instructions for -running the tests in the required ``metadata.yaml`` file (see a full example -in the `registration `_ section below). -Assuming that your tests are in a ``test/`` directory at the top level of -your repository, you could define your test commands as: - -.. code-block:: yaml - - run_tests: - - git clone latest - - pytest -v tests/ - -This makes a clone of your repository based on your latest -`release tag on GitHub `_ -and navigates into the repository root. Note that this is not a true -:program:`git` command, but is instead specific to the MDAKits registry -workflow and depends on the ``project_home`` field in the ``metadata.yaml`` -file (see :ref:`specification`). -The :program:`pytest` command then runs the tests found inside the ``tests/`` directory. -If your tests are elsewhere, change this path appropriately. - -Dependencies that are only required for testing are indicated in the -``test_dependencies`` object. Suppose your package uses :program:`pytest` -and used the `MDAnalysisTests `_ -for sample data. This is reflected in your MDAKit metadata with: - -.. code-block:: yaml - - test_dependencies: - - mamba install pytest MDAnalysis - +When submitting an MDAKit to the registry, you'll include the +instructions for running the tests in the required ``metadata.yaml`` +file under ``run_tests`` - see the +:ref:`metadata.yaml documentation ` for more. diff --git a/docs/source/makingakit.rst b/docs/source/makingakit.rst index 8334e498..84101ce0 100644 --- a/docs/source/makingakit.rst +++ b/docs/source/makingakit.rst @@ -7,14 +7,15 @@ to ensure that registered packages are FAIR-compliant and hold up to an ideal scientific standard. -**Building a Kit from the cookiecutter** +Building a Kit using the cookiecutter +------------------------------------- -Without prior experience, some of the MDAKit registry requirements +Without prior experience, some of the MDAKit Registry requirements can be daunting. The `MDAKit cookiecutter `_ can be used to aid rapid development of a FAIR-compliant MDAKit by generating placeholder code for documentation, testing and -installation. In this section, we provide a walk-through for the -creation of an MDAKit using the cookiecutter. +installation. A guided example of creating an MDAKit, from cookiecutter +to registration, is provided below. .. toctree:: :maxdepth: 2 @@ -22,16 +23,22 @@ creation of an MDAKit using the cookiecutter. making-a-kit/from-cookiecutter -**Building from an existing project** +Building without the cookiecutter/from an existing project +---------------------------------------------------------- If you have an existing package you wish to register as an MDAKit, -(or if you wish to develop one yourself without using the cookiecutter), -chances are very little restructuring is needed for registration. -The primary concern is ensuring that the core MDAKit requirements -are met - check the links below for things to consider. +or if you wish to develop a Kit without use of the cookiecutter, that's +also OK! For existing code, chances are very little restructuring is +required before registration - just make sure you meet the core +:ref:`requirements ` for registration. + +You may still find the above links helpful, along with the additional +information below. .. toctree:: :maxdepth: 1 - making-a-kit/licensing + making-a-kit/license making-a-kit/hosting + making-a-kit/testing + making-a-kit/documentation From 08fa1e7b8fb9fbf212d5bf9666955ae707a36014 Mon Sep 17 00:00:00 2001 From: fiona-naughton Date: Mon, 30 Sep 2024 00:20:09 +1000 Subject: [PATCH 03/11] update requirements text --- docs/source/about.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/about.rst b/docs/source/about.rst index db5dc7c2..4c34bbd1 100644 --- a/docs/source/about.rst +++ b/docs/source/about.rst @@ -27,7 +27,7 @@ requirements**: #. Minimal documentation is provided (what your code does, how to install it, and how to use it) #. At least minimal regression tests are present; continuous integration is encouraged -#. The code is installable as a standard package +#. The source code is installable as a standard package It is also highly encouraged that the MDAKit also satisfies: From 1f13d3bd5b65acdfd58b494fdc4e4c7d97b9e0b2 Mon Sep 17 00:00:00 2001 From: fiona-naughton Date: Wed, 2 Oct 2024 00:32:19 +1000 Subject: [PATCH 04/11] making-a-kit overhaul --- docs/source/addingakit.rst | 27 ++++ docs/source/index.rst | 13 +- docs/source/maintainingakit.rst | 16 ++ .../source/making-a-kit/from-cookiecutter.rst | 30 ++-- .../making-a-kit/guided-example/add-code.rst | 37 ++--- .../making-a-kit/guided-example/add-docs.rst | 149 +++++++++--------- .../making-a-kit/guided-example/add-tests.rst | 111 +++++++------ .../guided-example/registering.rst | 86 ++++++++++ .../making-a-kit/guided-example/to-github.rst | 34 ++-- .../guided-example/use-cookiecutter.rst | 109 +++++++------ .../registering-a-kit/metadata-yaml.rst | 125 +++++++++++++++ .../source/registering-a-kit/step-by-step.rst | 59 +++++++ 12 files changed, 581 insertions(+), 215 deletions(-) create mode 100644 docs/source/addingakit.rst create mode 100644 docs/source/maintainingakit.rst create mode 100644 docs/source/making-a-kit/guided-example/registering.rst create mode 100644 docs/source/registering-a-kit/metadata-yaml.rst create mode 100644 docs/source/registering-a-kit/step-by-step.rst diff --git a/docs/source/addingakit.rst b/docs/source/addingakit.rst new file mode 100644 index 00000000..87761783 --- /dev/null +++ b/docs/source/addingakit.rst @@ -0,0 +1,27 @@ +.. _add-mdakit: + +******************************** +Adding an MDAKit to the Registry +******************************** + +Do you have an MDAKit? Consider adding it to the `MDAKit registry`_! + + +.. note:: + The `MDAKit registry`_ is still in its initial stages. We expect that the way in + which MDAKits are added, and the type of information required, may change + over time. Please reach out via the `issue tracker`_ if you have any + questions. + +**The registration process** + +Registering an MDAKit requires you to create a single file with metadata +(called ``metadata.yaml``) describing the kit, and . You add this file to the `MDAnalysis/mdakits +repository`_ on GitHub. Links below. + +.. toctree:: + :maxdepth: 1 + + registering-a-kit/step-by-step + registering-a-kit/metadata-yaml + diff --git a/docs/source/index.rst b/docs/source/index.rst index 93a7065f..356cc092 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -24,12 +24,19 @@ additional resources about MDAKits and how to write and/or add your own. .. toctree:: :maxdepth: 1 - :caption: Resources + :caption: Resources for MDAKit Authors - about makingakit - add + addingakit + maintainingakit troubleshooting + + +.. toctree:: + :maxdepth: 1 + :caption: Other Resources + + about reviewersguide diff --git a/docs/source/maintainingakit.rst b/docs/source/maintainingakit.rst new file mode 100644 index 00000000..45d7eed9 --- /dev/null +++ b/docs/source/maintainingakit.rst @@ -0,0 +1,16 @@ +********************* +Maintaining an MDAKit +********************* + +* More TBA* + +There are a variety of reasons a kit may behave unexpectedly after +being submitted to the registry. +Apart from actively developing the kit, changes in kit dependencies, +or even Python itself, can introduce (deprecate) new (old) functionality. +For this reason, the kits' continuous integration is rerun weekly to +confirm the kits expected behavior. + +In the event that a kit no longer passes its tests, an issue in MDAnalysis/MDAKits is automatically raised while notifying the maintainers indicated in the `metadata.yaml` file. +While the registry developers will be happy to help where possible, ultimately, the maintainers of the MDAKit are responsible for resolving such issues and ensuring that the tests pass. +The issue will automatically close after the next CI run if the tests pass again. diff --git a/docs/source/making-a-kit/from-cookiecutter.rst b/docs/source/making-a-kit/from-cookiecutter.rst index 88e8dd34..1b99cebb 100644 --- a/docs/source/making-a-kit/from-cookiecutter.rst +++ b/docs/source/making-a-kit/from-cookiecutter.rst @@ -1,38 +1,41 @@ -************************************************ -Building from the cookiecutter: a guided example -************************************************ +.. _guided-example: + +********************************************* +Guided Example: Building an RMSF analysis Kit +********************************************* In this section, we provide a guided example for creating an MDAKit using the `MDAKit cookiecutter `_. -A `video walk-through of this tutorial `_ -is available on YouTube. +A `video walk-through `_ +of this tutorial is available on YouTube. We will create a kit for RMSF analysis, replicating the functionality of the `RMSF analysis class `_ in the core library. +First, let's refresh ourselves on the :ref:`requirements ` +for a 'registerable' kit. In brief: **MDAKit requirements** -As a reminder, here are (in brief) the requirements (and recommendations) -that we are aiming to fulfil to make a 'registerable' kit. See the -requirements in more detail :ref:`here ` -We'll check back in at the end of each part to see how we're going! - #. Uses MDAnalysis #. Open source + OSI license -#. Versioned + on a version-controlled repository +#. Versioned + on version-controlled repository #. Designated authors and maintainers #. (At least) minimal documentation #. (At least) minimal regression tests #. Installable as a standard package #. (Recommended) community information available -#. (Recommended) on a package distribution platform +#. (Recommended) available on a package distribution platform + +Below are the steps we'll go through to create our MDAKit. We'll check +back in after each part to see how we're proressing through these +requirements! .. toctree:: :maxdepth: 1 - :caption: Steps + :caption: Steps for creating an MDAKit from the cookiecutter guided-example/use-cookiecutter guided-example/add-code @@ -40,4 +43,5 @@ We'll check back in at the end of each part to see how we're going! guided-example/to-github guided-example/add-docs guided-example/make-release + guided-example/registering diff --git a/docs/source/making-a-kit/guided-example/add-code.rst b/docs/source/making-a-kit/guided-example/add-code.rst index ae1fec4d..e94fe76d 100644 --- a/docs/source/making-a-kit/guided-example/add-code.rst +++ b/docs/source/making-a-kit/guided-example/add-code.rst @@ -1,23 +1,23 @@ -*************************************************** -Part 2: Adding code to the newly created repository -*************************************************** +******************* +Part 2: Adding code +******************* *For a video demonstration of this section,* -`click here `_. +`click here `_. Now that we have a shell for our MDAKit, we can begin to add our code to the ``rmsfkit/`` directory. -In this example, we are recreating the exist MDAnalysis RMSD analysis, -so we will simply copy and paste the analysis class into -``rmsfkit/rmsfkit.py``.The in-code documentation (which has been trimmed -here for brevity) is written in -`reStructuredText syntax `_ -syntax for building with Sphinx later. +#. In this example, we are recreating the existing MDAnalysis RMSF analysis, + so we will simply copy and paste this analysis class into a new file + ``rmsfkit/rmsfkit.py``.The in-code documentation (which has been trimmed + here for brevity) is written in + `reStructuredText syntax `_ + for building with Sphinx later. -The contents of the file should now resemble the following code block: + The contents of the file should now resemble the following code block: -.. code-block:: python + .. code-block:: python """ rmsfkit.py @@ -101,18 +101,19 @@ The contents of the file should now resemble the following code block: raise ValueError("Some RMSF values negative; overflow " + "or underflow occurred") +#. Finally, to make our ``RMSF`` analysis class easier to access, we + import it in ``__init__.py`` by adding: -Finally, to make our ``RMSF`` analysis class easier to access, we -import it in ``__init__.py`` by adding: - -.. code-block:: python + .. code-block:: python from .rmsfkit import RMSF -**Progress: MDAKit requirements** +Progress: MDAKit requirements +----------------------------- -#. **✓ Uses MDAnalysis** +#. **✓ Uses MDAnalysis** - the added code uses the MDAnalysis + ``AnalysisBase`` class and runs on MDAnalysis ``AtomGroup`` objects! #. **✓ Open source + OSI license** #. *Versioned + on a version-controlled repository* #. **✓ Designated authors and maintainers** diff --git a/docs/source/making-a-kit/guided-example/add-docs.rst b/docs/source/making-a-kit/guided-example/add-docs.rst index 710929f7..b6409a8e 100644 --- a/docs/source/making-a-kit/guided-example/add-docs.rst +++ b/docs/source/making-a-kit/guided-example/add-docs.rst @@ -5,65 +5,65 @@ Part 5: Providing documentation *For a video demonstration of this section,* `click here `_. -The auto-generated ``README.md`` file constituted a form of basic -documentation. However, it is ideal to provide more thorough -documentation for your code. - -Additional documentation can consist of *API documentation* or -*User Guide* documentation. The former provides details of the +The auto-generated ``README.md`` file contains a description of our +Kit and installation instructions. To meet the "minimal documentaiton" +requirement, we could just add basic usage to this file, giving a +simple *"README"-style* documentation. However, it is ideal to provide +more thorough documentation. + +Additional documentation can consist of **API documentation** or +**User Guide** documentation. The former provides details of the functions, inputs/outputs, datatypes, etc of the code and can -be autogenerated from docstrings in the code; while the -later may show usage and provide more details, explanations and -examples for users. +be autogenerated from docstrings; while the later shows usage +and provides more details, explanation and even examples for users. `Read the Docs `_ can be used to build and host documentation online. The cookiecutter includes a -Read the Docs configuration as well a pre-made documentation +ReadtheDocs configuration as well a pre-made documentation environment file that is used by Read the Docs and for building locally. Frameworks for autogenerated API and basic User Guide documentation are also provided. Building documentation locally -############################## +------------------------------ -First, we need to install the correct environment for building -the documentation. In the ``docs/`` directory, run: +First, we can build a copy of the documentation to see what the +cookiecutter template has started us with. -.. code-block:: bash +#. We install the correct environment for building the documentation + by running the following in the ``docs/`` directory: + + .. code-block:: bash mamba env update --name rmsfkit -f requirements.yaml -We can now build the documentation HTML files using the included -``Makefile``. Without looking at any of the documentation source -files, run: +#. We can now build the documentation HTML files using the included + ``Makefile``. -.. code-block:: bash + .. code-block:: bash make html -This will convert the reStructuredText files into HTML in the -``_build`` directory. - -Open ``index.html`` and look around. + This will convert the reStructuredText files into HTML in the + ``_build`` directory. -Notice that the "API Documentation" does not contain all of the -information found in the docstrings of our code. -This is because within ``source/api.rst``, the only contents are: +We can now view the documentation by opening ``_build/index.html``! +If we make changes to any documentation files, we can reruin +``make html`` and refresh the browser to view the changes. -.. code-block:: rst - API Documentation - ================= - - .. autosummary:: - :toctree: autosummary - - rmsfkit +Adding to the documentation +--------------------------- -Instead, it should contain: +#. **Expanding the API documentation**: the autogenerated "API + Documentation" does not contain all of the information found in the + docstrings of our code. This is because the 'automummary' in + ``source/api.rst`` specified ``rmsfkit``, rather than specifically + our analysis class, ``rmsfkit.RMSF``. We update it, so ``api.rst`` + now reads: -.. code-block:: rst + .. code-block:: rst API Documentation ================= @@ -73,19 +73,16 @@ Instead, it should contain: rmsfkit.RMSF -After making the change, rerun ``make html`` and refresh the page. There will -now be a table entry for the RMSF class. Clicking this entry will open the -documentation that was present in the docstrings! - - -Adding to the documentation -########################### + After making the change, we rerun ``make html`` and refresh the + page - there will now be a table entry for the RMSF class! Clicking + this entry will open the documentation that was present in the + docstrings -Other than the newly visible docstring contents, the rest of the documentation -is completely empty. Let's update the ``index.rst`` file to include a short -description of the package: -.. code-block:: rst +#. The rest of our documentation is currently empty. Let's update ``index.rst``, + which serves as our "home" page. We add a short description of the package: + + .. code-block:: rst Welcome to rmsfkit's documentation! ========================================================= @@ -108,8 +105,9 @@ description of the package: * :ref:`modindex` * :ref:`search` -Additionally, we can update ``getting_started.rst`` to let potential -users know how to install the package. +#. Finally, we add some installation instructions to the + ``getting_started.rst`` pare to let potential users know how to + install the package. .. code-block:: rst @@ -124,19 +122,25 @@ users know how to install the package. cd rmsfkit/ pip install . -Run the ``make html`` command again and refresh the browser window to -view the resulting changes. +Run the ``make html`` command again and refresh the browser window will +let us view the resulting changes. + +Ideally, we could also add more information explaining the usage of +``rmsfkit``; however, the API documentation generated from the docstrings +is already reasonably thorough, so we'll leave our documentation here +for now. + + +**Citations in documentation** -Citations in documentation --------------------------- You'll notice the RMSF class documentation has an unformatted citation -(``:cite:p:`Welford1962```). If your documentation need citations, you -can easily include them using the bibtex format. +(``:cite:p:`Welford1962```). We can easily include citations using the +bibtex format. -We first create the bibtex file, ``references.bib`` in the ``doc/source/`` -directory: +#. We first create the bibtex file, ``references.bib`` in the ``doc/source/`` + directory: -.. code-block:: + .. code-block:: bib @article{Welford1962, author = { B. P. Welford}, @@ -150,10 +154,10 @@ directory: doi = {10.1080/00401706.1962.10490022} } -In ``conf.py``, we need to add a new extension (``sphinxcontrib.bibtex``) -as well as the name of the bibtex file. +#. In ``conf.py``, we need to add a new extension + (``sphinxcontrib.bibtex``) as well as the name of the bibtex file. -.. code-block:: + .. code-block:: Python extensions = [ 'sphinx.ext.autosummary', @@ -168,37 +172,37 @@ as well as the name of the bibtex file. bibtex_bibfiles = ['references.bib'] -In addition, we have to add this extension the install requirements -in ``docs/requirements.yaml``. Add ``- sphinxcontrib-bibtex`` as an -additional dependency here. +#. We also have to add this extension the install requirements in + ``docs/requirements.yaml``. Add ``- sphinxcontrib-bibtex`` as an + additional dependency here. -Update your environment with +#. We update the environment with .. code-block:: bash mamba env update --name rmsfkit -f requirements.yaml -before once again running `make html`. + before once again running `make html`. Refreshing the RMSF documentation will now show a properly formatted citation using the information in the bibtex file. Deploying the documentation -=========================== +--------------------------- -Once you've pushed any documentation changes to GitHub, you can the deploy -their own shiny new website. +After pushing documentation changes to GitHub, we can then display them +on their own shiny new website. Log into `Read the Docs `_ (using your GitHub account) and navigate to the dashboard. Click the "Import a Project" button and find the repository in the list. Click the "+" and confirm that the name, URL, and default branch are correct. -.. image:: img/rmsftutorial/import.png +.. image:: ../../img/rmsftutorial/import.png :alt: Import a project into RTD -.. image:: img/rmsftutorial/adding.png +.. image:: ../../img/rmsftutorial/adding.png :alt: Adding your repository to RTD Clicking next will begin the deployment. This immediately starts the build @@ -208,14 +212,15 @@ completes the documentation requirement for an MDAKit. Notice that the "docs" badge in the GitHub rendered ``README.md`` is now green. -**Progress: MDAKit requirements** +Progress: MDAKit requirements +----------------------------- #. **✓ Uses MDAnalysis** #. **✓ Open source + OSI license** #. **✓ Versioned + on a version-controlled repository** #. **✓ Designated authors and maintainers** #. **✓ (At least) minimal documentation** -#. **✓(At least) minimal regression tests** +#. **✓ (At least) minimal regression tests** #. **✓ Installable as a standard package** #. **✓ (Recommended) community information available** #. *(Recommended) on a package distribution platform* diff --git a/docs/source/making-a-kit/guided-example/add-tests.rst b/docs/source/making-a-kit/guided-example/add-tests.rst index a8264569..7d6ccc29 100644 --- a/docs/source/making-a-kit/guided-example/add-tests.rst +++ b/docs/source/making-a-kit/guided-example/add-tests.rst @@ -6,12 +6,14 @@ Part 3: Adding tests `click here `_. The cookiecutter set up the framework for tests in ``rmsfkit/tests``, -but we need to add the tests themselves to ``test_rmsfkit.py``. -Once again, we take directly from the MDAnalysis package. -Here we take the `RMSF testing class `_ -and update the contents of ``rmsfkit/tests/test_rmsfkit.py`` as follows: +but we now need to add the tests themselves, and make sure that they run. -.. code-block:: python +#. Once again, we take directly from the MDAnalysis package. + We take the + `RMSF testing class `_ + and update the contents of ``rmsfkit/tests/test_rmsfkit.py`` as follows: + + .. code-block:: python """ Unit and regression test for the rmsfkit package. @@ -73,83 +75,93 @@ and update the contents of ``rmsfkit/tests/test_rmsfkit.py`` as follows: err_msg="error: rmsfs should all be 0") -Adding a test dependency ------------------------- +#. **Adding a test dependency:** Since these tests use files included + with the ``MDAnalysisTests`` package, we need to add this as a + dependency. We do this in two configuration files: + + - In ``devtools/conda-envs/test_env.yaml``, in the ``dependencies:`` section: + + .. code-block:: yaml + + dependencies: + # Base depends + - python + - pip + + # MDAnalysis + - MDAnalysis + + # Testing + - pytest + - pytest-cov + - pytest-xdist + - codecov + - MDAnalysisTests # <-- add this! -Since these tests use files included with the ``MDAnalysisTests`` package, -we need to add this as a dependency. We do this in two configuration -files: -- In ``devtools/conda-envs/test_env.yaml``. -- In ``pyproject.toml``: Under the ``[project.optional-dependencies]`` - table, ensure that ``MDAnalysisTests>=2.0.0`` is listed in ``test``. + - In ``pyproject.toml``, under ``[project.optional-dependencies]``: - .. code-block:: toml + .. code-block:: toml [project.optional-dependencies] test = [ "pytest>=6.0", "pytest-xdist>=2.5", "pytest-cov>=3.0", - "MDAnalysisTests>=2.0.0", + "MDAnalysisTests>=2.0.0", # <-- add this! ] -Running the tests -################# +**Running the tests** -Following the instructions from the generated ``README.md``, we can -create a testing environment using ``mamba`` (preferred) or ``conda``: +#. Following the instructions from the generated ``README.md``, we can + create a testing environment using ``mamba`` (preferred) or ``conda``: -.. code-block:: bash + .. code-block:: bash - $ mamba create -n rmsfkit - $ mamba env update --name rmsfkit --file devtools/conda-envs/test_env.yaml + $ mamba create -n rmsfkit + $ mamba env update --name rmsfkit --file devtools/conda-envs/test_env.yaml $ mamba activate rmsfkit -And then install the package: -.. code-block:: bash - $ pip install -e . +#. Then install the package: -We can now run tests locally using: + .. code-block:: bash -.. code-block:: bash + $ pip install -e . - $ pytest rmsfkit/tests -This should pass without errors, though with some potential warnings. +#. We can now run tests locally using: + .. code-block:: bash -Preparing for CI -################ + $ pytest rmsfkit/tests -Local tests passing is only half of testing. Ideally, tests shouls also -pass through continuous integration services. The cookiecutter generates -the necessary GitHub workflow files ``.github/workslows/gh-ci.yaml`` -to do this on GitHub. + This should pass without errors, though with some potential warnings. -Since our tests use the ``MDAnalysisTests`` package, we again need to make -a change to this file. Change the block: +We only need to perform Step 1-2 once. Thereafter, tests can be run at +any point with ``pytest rmsfkit/tests`` (though you may need to reactivate +the environment, ``mamba activate rmsfkit``). -.. code-block:: yaml - - name: Install MDAnalysis version - uses: MDAnalysis/install-mdanalysis@main - with: - version: ${{ matrix.mdanalysis-version }} - install-tests: false - installer: mamba - shell: bash -l {0} +**Preparing for CI** + +Local tests passing is only half of testing. Ideally, tests shouls also +pass through **continuous integration services**. The cookiecutter generates +the necessary GitHub workflow files ``.github/workflows/gh-ci.yaml`` +to do this on GitHub. -to +#. Since our tests use the ``MDAnalysisTests`` package, we again need to make + a change to this file. In the ``Install MDAnalysis version`` job + of ``.github/workflows/gh-ci.yaml`` we change the ``install-tests`` + flag to ``true`` - it should now look as follows: -.. code-block:: yaml + .. code-block:: yaml - name: Install MDAnalysis version uses: MDAnalysis/install-mdanalysis@main with: version: ${{ matrix.mdanalysis-version }} - install-tests: true # this needs to be true! + install-tests: true # <-- this needs to be true! installer: mamba shell: bash -l {0} @@ -158,7 +170,8 @@ GitHub repository to run the pre-built continuous integration provided by the cookiecutter. -**Progress: MDAKit requirements** +Progress: MDAKit requirements +############################# #. **✓ Uses MDAnalysis** #. **✓ Open source + OSI license** diff --git a/docs/source/making-a-kit/guided-example/registering.rst b/docs/source/making-a-kit/guided-example/registering.rst new file mode 100644 index 00000000..cc27c94b --- /dev/null +++ b/docs/source/making-a-kit/guided-example/registering.rst @@ -0,0 +1,86 @@ +.. _registration: + +***************************** +Part 7: Registering an MDAKit +***************************** + +*For a video demonstration of this section,* +`click here `_. + +The MDAKit registration is the same regardless of the creation process +for the kit. Here, we continue our guided example by showing the +registration process for our example ``rmsfkit``. When registering +your own kit, be sure to read the general information on the registration +process is found in the :ref: section. + +In order to submit your MDAKit to the registry, you will need to create +a pull request on GitHub against the MDAnalysis/MDAKits repository. + +First create a fork of the MDAnalysis/MDAKits repository on GitHub, then +clone the fork to your machine, navigate into ``MDAKits/mdakits/``, +and make an empty directory with your MDAKit name: + +.. code-block:: bash + + git clone git@github.com:yourusername/MDAKits + cd MDAKits/mdakits + mkdir rmsfkit/ + cd rmsfkit + +Now add a ``metadata.yaml`` for your MDAKit in this directory. See +:ref:`specification` for more details on this file. +The contents of ``metadata.yaml`` for ``rmsfkit`` are: + +.. code-block:: yaml + + project_name: rmsfkit + authors: + - https://github.com/yourusername/rmsfkit/blob/main/AUTHORS.md + maintainers: + - yourusername + description: + An analysis module for calculating the root-mean-square fluctuation of atoms in molecular dynamics simulations. + keywords: + - rms + - rmsf + license: GPL-2.0-or-later + project_home: https://github.com/yourusername/rmsfkit + documentation_home: https://rmsfkit.readthedocs.io/en/latest/ + documentation_type: API + src_install: + - git clone https://github.com/yourusername/rmsfkit.git + - cd rmsfkit/ + - pip install . + + ## Optional entries + python_requires: ">=3.9" + mdanalysis_requires: ">=2.0.0" + run_tests: + - pytest --pyargs rmsfkit.tests + development_status: Beta + +Commit and push this to your fork: + +.. code-block:: bash + + git add metadata.yaml + git commit -m "Adding rmsfkit" + git put origin main + +Refresh the forked repository page in your browser. Under "Contribute", +open a Pull Request. Add a title with the name of the kit and add a quick +description. Click "Create pull request" and wait for the tests to pass. + +Once this is done, you can add a comment along the lines of +"@MDAnalysis/mdakits-reviewers, ready for review". The reviewers will get +back you you with any change requests before merging it in as a kit. + +At this point there are no additional steps for registering your kit! +However, your responsibility for your new MDAKit does not end here. +Once your kit is on the {weekly CI run, also keep an eye on other things, continue to improve/develop}. +Read the information on the :ref:`maintaining` page. + +.. image:: img/rmsftutorial/submitting.gif + :alt: Process for submitting a kit to the registy + + diff --git a/docs/source/making-a-kit/guided-example/to-github.rst b/docs/source/making-a-kit/guided-example/to-github.rst index 2a126310..bebfc3d4 100644 --- a/docs/source/making-a-kit/guided-example/to-github.rst +++ b/docs/source/making-a-kit/guided-example/to-github.rst @@ -9,34 +9,42 @@ We need to upload our code to GitHub in order to finalise the continuous integration set up in the previous step, and to fullfil the requirement that the Kit be hosted on a version-controlled repository. -During the cookie generation process, the target GitHub repository was -identified and inserted into the workflows configuration, as well as -into the ``README.md`` file. +During the cookie generation process, the target GitHub repository, +``rmsfkit``, was identified and inserted into the workflows +configuration, as well as into the ``README.md`` file. -Add this repository as a remote to your local git repository: +#. We add this repository as a remote to the local git repository: -.. code-block:: bash + .. code-block:: bash $ git remote add origin git@github.com:myusername/rmsfkit -(substituting ``myusername`` for your GitHub username). Make sure that this repository exists on GitHub and is *empty*. Then, push the local code to GitHub: + (substituting ``myusername`` for your GitHub username). Make sure + that this repository exists on GitHub and is *empty*. + +#. Then, we push the local code to GitHub: -.. code-block:: bash + .. code-block:: bash $ git push origin main -Open the repository in GitHub and navigate to actions. Here you can -see the status of the tests. If all was done correctly in the previous -sections, these tests will pass! +The ``rmsfkit`` repository on GitHub will now be up-to-date the local +repository we have been working on. We can navigate to *Actions* +on GitHub to see the status of our tests. If all was done +correctly in the previous sections, these will all pass! -**Progress: MDAKit requirements** + +Progress: MDAKit requirements +----------------------------- #. **✓ Uses MDAnalysis** #. **✓ Open source + OSI license** -#. **✓ Versioned + on a version-controlled repository** +#. **✓ Versioned + on a version-controlled repository** -- our code is + now on GitHub! #. **✓ Designated authors and maintainers** #. *(At least) minimal documentation* -#. **✓(At least) minimal regression tests** +#. **✓ (At least) minimal regression tests** -- CI should be up and + running, and the tests added in the last step all passing! #. **✓ Installable as a standard package** #. **✓ (Recommended) community information available** #. *(Recommended) on a package distribution platform* diff --git a/docs/source/making-a-kit/guided-example/use-cookiecutter.rst b/docs/source/making-a-kit/guided-example/use-cookiecutter.rst index 69674e2a..2c1b0184 100644 --- a/docs/source/making-a-kit/guided-example/use-cookiecutter.rst +++ b/docs/source/making-a-kit/guided-example/use-cookiecutter.rst @@ -3,25 +3,25 @@ Part 1: Using the MDAKit cookicutter ************************************ *For a video demonstration of this section,* -`click here `_. +`click here `_. -The MDAKits cookiecutter (a -`Cookiecutter tool `_) +The **MDAKits cookiecutter** (a `Cookiecutter tool`_) can be used to rapidly develop a FAIR-compliant MDAKit by generating placeholder code for documentation, testing and installation. More infromation on its usage is can be found in the -`MDAKit cookiecutter documentation `_, -but in short, using the cookiecutter involves running a command line -prompt and then responding to a set of subsequent prompts. You will -need Python 3.10+ and the -`Cookiecutter tool `_ installed. +`MDAKit cookiecutter documentation`_, but in short, using the +cookiecutter involves running a command line prompt and then responding +to a set of subsequent prompts. -In this example, we use the cookiecutter as follows to generate -a new MDAKit template ``rmsfkit``. Some prompts are left -blank, accepting the the default option. +#. To start, we need a GitHub account, and an environment with Python + 3.10+ and the `Cookiecutter tool`_ installed. -.. code-block:: bash +#. We then use the cookiecutter to generate a template for an + ``rmsfkit`` project with the following command and responses. Note + that some prompts are left blank, accepting the default option. + + .. code-block:: bash $ cookiecutter gh:MDAnalysis/cookiecutter-mdakit @@ -44,54 +44,69 @@ blank, accepting the the default option. Choose from [1/2] (1): template_analysis_class (Class name to template (e.g. Rmsfkit ). Press Enter to skip including analysis templates): -This generates a new git repository named ``rmsfkit`` (note, the -author of the initial commit will match that of the user's global git -configurations). + Details on these options can be found on the cookiecutter + documentation `here `_. We have opted to + *include ReadTheDocs* (so we can use it to host our documentation + later), and to *not create a template Analysis class* for our code + (since we will be copying the existing MDAnalysis RMSF code). -Navigating into this directory, we find a number of files and -subdirectories. The -`cookiecutter documentation `_ -provides more information on these, but those to take note of now -include: +A new git repository named ``rmsfkit`` has been generated. Navigating +into this directory, we find a number of files and subdirectories. The +`cookiecutter documentation `_ provides more +information on these, but some to take note of now: -- ``LICENSE`` -- a text file containing the license information (GLPv2+ by default) -- ``AUTHORS.md`` -- a template text file for listing code authors as each - new contribution is made. Auto-populated with your name as the +- ``LICENSE`` -- a text file containing the license information (GLPv2+ + by default). +- ``AUTHORS.md`` -- a template text file for listing code authors as + each new contribution is made. Auto-populated with your name as the initial project creator. -- ``README.md`` -- a text file containing the project description and other - useful information; typically landing page content rendered on GitHub. -- ``pyproject.toml`` - a configuration file with information for +- ``README.md`` -- a text file containing the project description and + other useful information. This will be the landing page content + rendered on GitHub. +- ``pyproject.toml`` -- a configuration file with information for packaging (and other tools). -- ``rmsfkit/`` -- template for the python package source code, i.e. - where your code (and tests) will live. -- ``CODE_OF_CONDUCT.md`` and ``CONTRIBUTING.md`` -- text files containing - community guidelines and instructions on reporting issues and contributing - (once the code is hosted on GitHub). -- ``docs/`` -- folder containing documentation template, instructions - and build environment documented within. +- ``rmsfkit/`` -- folder containing template files for a python + package. This is where the source code (and tests) will live. +- ``CODE_OF_CONDUCT.md`` and ``CONTRIBUTING.md`` -- text files + containing sample community guidelines and autogenerated instructions + for reporting issues, contributing, etc (once the code is on GitHub). +- ``docs/`` -- folder containing documentation templates and + configuration files for building the documentation. -We'll address other files as they come up in subsequent parts. +We'll address some further files as they become relevant. -** Progress: MDAKit requirementsi** +Progress: MDAKit requirements +----------------------------- #. *Uses MDAnalysis* #. **✓ Open source + OSI license** -- the default GPLv2 license falls under OSI. If you wish to choose another license, you can replace - the default LICENSE file with another. See :ref:`` for more. + the default ``LICENSE`` file with another; see :ref:`licensing` for more + information. #. *Versioned + on a version-controlled repository* -- the cookiecutter has set up versioning with git; in a later step, we'll add it to the the - GitHub repository. -#. **✓ Designated authors and maintainers** -- in the generated AUTHORS.md - file, assuming that the maintained and author are the same person. - More authors will be added as the project evolves. -#. *(At least) minimal documentation* -- the generated README.md file - provides a form of minimal documentation; however, we will be adding - more documentation in a later step. + GitHub repository to complete this requirement. +#. **✓ Designated authors and maintainers** -- in the autogenerated + ``AUTHORS.md`` file, assuming that the maintained and author are the + same person. Future authors will also be added here. +#. *(At least) minimal documentation* -- the autogenerated ``README.md`` + does contain the Kit description, which is a start. We will be adding + more formal documentation in a later step. #. *(At least) minimal regression tests* -#. **✓ Installable as a standard package** -- through `rmsfkit/` and - `pyproject.toml`. +#. **✓ Installable as a standard package** -- the configuration file + ``pyproject.toml`` and the file structure set up in ``rmsfkit/`` will + allow this. #. **✓ (Recommended) community information available** -- provided with - CODE_OF_CONDUCT.md and CONTRIBUTING.md. Adjust these files to suit your - needs. + ``CODE_OF_CONDUCT.md`` and ``CONTRIBUTING.md``. Adjust these files to + suit your needs. #. *(Recommended) on a package distribution platform* + + +.. _`Cookiecutter tool`: https://cookiecutter.readthedocs.io/en/stable/ + +.. _`MDAKit cookiecutter documentation`: https://cookiecutter-mdakit.readthedocs.io/en/latest/ + +.. _`cookiecutter options`: https://cookiecutter-mdakit.readthedocs.io/en/latest/options + +.. _`cookiecutter usage`: https://cookiecutter-mdakit.readthedocs.io/en/latest/usage diff --git a/docs/source/registering-a-kit/metadata-yaml.rst b/docs/source/registering-a-kit/metadata-yaml.rst new file mode 100644 index 00000000..15982907 --- /dev/null +++ b/docs/source/registering-a-kit/metadata-yaml.rst @@ -0,0 +1,125 @@ +.. _specification: + +Specification of the ``metadata.yaml`` file +=========================================== + +The ``metadata.yaml`` file is in `YAML format`_. + +The file contains required and optional entries. + +**Required entries** + Any variable in the *Required entries* section **must** be + provided. If missing or empty, the MDAKit cannot be registered. + +**Optional entries** + Any variable in the *Optional entries* section **may** be left + empty or be left out. However, it is recommended to provide as much + information as possible. Some aspects of the registry checks may not work if these details are missing. + + +.. Note:: + + For now, the :ref:`template file