Evaluate the example code in the documentation at build time

[kinow]

Hi,

I am testing PyFlow, and managed to get a suite configured and installed in ecFlow 🎉 It was super easy, and the current documentation is great (really, kudos to all involved in writing it).

However, the getting started example is broken, as there is no pyflow.get_host. I checked on both develop and master, and looks like this function might have been removed (or hosted in another module). I guess code and documentation are evolving separately.

This could be prevented by evaluating the example code in the documentation at build time. So a pull request that changes the code, and makes the documentation invalid, would fail the build and require developers to update the docs at the same time (e.g. common-workflow-language/user_guide#248, where a patched version of https://github.com/invenia/sphinxcontrib-runcmd/ was used).

Or this issue can be closed once the getting started docs are fixed, in case there's no bandwidth to implement something like what was described here.

Thanks!
Bruno

[kinow]

Here's the error I get running make html in the docs folder, after installing the requirements.txt for docs.

updating environment: [new config] 22 added, 0 changed, 0 removed
reading sources... [100%] index                                                                                                                                                           
WARNING: autodoc: failed to import class 'TrimurtiHost' from module 'pyflow'; the following exception was raised:
Traceback (most recent call last):
  File "/home/bdepaula/mambaforge/envs/pyflow/lib/python3.10/site-packages/sphinx/util/inspect.py", line 440, in safe_getattr
    return getattr(obj, name, *defargs)
AttributeError: module 'pyflow' has no attribute 'TrimurtiHost'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bdepaula/mambaforge/envs/pyflow/lib/python3.10/site-packages/sphinx/ext/autodoc/importer.py", line 102, in import_object
    obj = attrgetter(obj, mangled_name)
  File "/home/bdepaula/mambaforge/envs/pyflow/lib/python3.10/site-packages/sphinx/ext/autodoc/__init__.py", line 327, in get_attr
    return autodoc_attrgetter(self.env.app, obj, name, *defargs)
  File "/home/bdepaula/mambaforge/envs/pyflow/lib/python3.10/site-packages/sphinx/ext/autodoc/__init__.py", line 2827, in autodoc_attrgetter
    return safe_getattr(obj, name, *defargs)
  File "/home/bdepaula/mambaforge/envs/pyflow/lib/python3.10/site-packages/sphinx/util/inspect.py", line 456, in safe_getattr
    raise AttributeError(name) from exc
AttributeError: TrimurtiHost

WARNING: autodoc: failed to import function 'get_host' from module 'pyflow'; the following exception was raised:
Traceback (most recent call last):
  File "/home/bdepaula/mambaforge/envs/pyflow/lib/python3.10/site-packages/sphinx/util/inspect.py", line 440, in safe_getattr
    return getattr(obj, name, *defargs)
AttributeError: module 'pyflow' has no attribute 'get_host'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bdepaula/mambaforge/envs/pyflow/lib/python3.10/site-packages/sphinx/ext/autodoc/importer.py", line 102, in import_object
    obj = attrgetter(obj, mangled_name)
  File "/home/bdepaula/mambaforge/envs/pyflow/lib/python3.10/site-packages/sphinx/ext/autodoc/__init__.py", line 327, in get_attr
    return autodoc_attrgetter(self.env.app, obj, name, *defargs)
  File "/home/bdepaula/mambaforge/envs/pyflow/lib/python3.10/site-packages/sphinx/ext/autodoc/__init__.py", line 2827, in autodoc_attrgetter
    return safe_getattr(obj, name, *defargs)
  File "/home/bdepaula/mambaforge/envs/pyflow/lib/python3.10/site-packages/sphinx/util/inspect.py", line 456, in safe_getattr
    raise AttributeError(name) from exc
AttributeError: get_host

looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index                                                                                                                                                            
generating indices... genindex py-modindex done
copying notebooks ... [100%] content/introductory-course/variables.ipynb                                                                                                                  
writing additional pages... search done
copying images... [100%] _static/images/ecflow-ui-complete-suite.png                                                                                                                      
copying static files... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build finished with problems, 2 warnings.
make: *** [Makefile:22: html] Error 1

The same error I had when I copied the getting started code.

[corentincarton]

Hi @kinow,

The get_host() function is indeed deprecated as it was pointing to old ECMWF machines. We removed this part of the code to use more generic hosts. For your tests, you can use the LocalHost() class instead or more advanced hosts, such as the PBSHost or the SlurmHost if you are working on an HPC.

The documentation still points to a previous version of pyflow unfortunately. I'm working on moving it to readthedocs but it could take a few days. I'll have a look at the error you obtained.

In the mean time, I suggest you stick to the notebooks in the tutorials folder (they should be up-to-date) or search directly the code (the hosts should be pretty straightforward).

Don't hesitate to contact me if you need more help!

Cheers,
Corentin

[kinow]

In the mean time, I suggest you stick to the notebooks in the tutorials folder (they should be up-to-date) or search directly the code (the hosts should be pretty straightforward).

I used them yesterday. Extremely helpful (I'm taking notes of a few things in pyflow to suggest to be added in autosubmit too 😬 )!

Thank you @corentincarton !

[corentincarton]

@kinow, can you try to build the documentation using the develop branch? I just pushed a fix.

[kinow]

@kinow, can you try to build the documentation using the develop branch? I just pushed a fix.

Sure, just a sec.

[kinow]

Documentation building fine now! 🎉

(pyflow):~/Development/python/workspace/pyflow/docs$ make html
Running Sphinx v4.5.0
loading pickled environment... done
[autosummary] generating autosummary for: content/api-reference.rst, content/installation.rst, content/introduction.rst, content/introductory-course.rst, content/introductory-course/additional-examples.ipynb, content/introductory-course/advanced-attributes.ipynb, content/introductory-course/anchor-families.ipynb, content/introductory-course/configuring-suites.ipynb, content/introductory-course/deployable-resources.ipynb, content/introductory-course/flow-control.ipynb, ..., content/introductory-course/host-management.ipynb, content/introductory-course/monitoring-tasks.ipynb, content/introductory-course/object-oriented-suites.ipynb, content/introductory-course/repository-layout.rst, content/introductory-course/script-handling.ipynb, content/introductory-course/suite-basics.ipynb, content/introductory-course/time-dependencies.ipynb, content/introductory-course/variables.ipynb, genindex.rst, index.rst
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 22 source files that are out of date
updating environment: [config changed ('version')] 22 added, 0 changed, 0 removed
reading sources... [100%] index                                                                                                                                                               
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index                                                                                                                                                                
generating indices... genindex py-modindex done
copying notebooks ... [100%] content/introductory-course/variables.ipynb                                                                                                                      
writing additional pages... search done
copying images... [100%] _static/images/ecflow-ui-complete-suite.png                                                                                                                          
copying static files... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded.

The HTML pages are in _build/html.
(pyflow):~/Development/python/workspace/pyflow/docs$ git log -n 1
commit e860a576339842d3365a851a3ed83d601713ef14 (HEAD -> develop, upstream/develop)
Author: Corentin Carton de Wiart <[email protected]>
Date:   Wed Nov 16 11:50:42 2022 +0000

    update documentation

[corentincarton]

@kinow, the documentation is now available there: https://ecmwf-pyflow.readthedocs.io
We'll add later a check to make sure the documentation builds after each PR.

[kinow]

I can confirm https://ecmwf-pyflow.readthedocs.io/en/latest/content/introductory-course/getting-started.html has been updated. Thank you @corentincarton !