-
Notifications
You must be signed in to change notification settings - Fork 15
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #330 from bird-house/setup-cruft-for-cookiecutter-…
…refresh Setup cruft for cookiecutter refresh. This PR is part of issue bird-house/cookiecutter-birdhouse#61, rolling out cruft as a mechanism to easily update cookiecutter template changes to existing project. Require cookiecutter-birdhouse PR bird-house/cookiecutter-birdhouse#89 In `.cruft.json`, I set `"project_short_description": "A Web Processing Service Testbed."` from the `README.rst`, let me know if that is still correct. In `README.rst`, I took the Flyingpigeon bird description from `docs/source/index.rst`. Noticeable changes: * Notebooks testing enabled in Travis-CI but allowed to fail, waiting for this issue #328 according to @nilshempelmann * Fix pymetalink install to be able to run notebook tests * Docs testing also enabled in Travis-CI Travis-CI build working: https://travis-ci.org/github/bird-house/flyingpigeon/builds/694013491 RtD build working: https://readthedocs.org/projects/flyingpigeon/builds/11184473/
- Loading branch information
Showing
14 changed files
with
189 additions
and
79 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| { | ||
| "template": "https://github.com/bird-house/cookiecutter-birdhouse.git", | ||
| "commit": "fce99f3071d65bb494a44f7527b5dc5d92fbd390", | ||
| "skip": [ | ||
| "flyingpigeon/processes/wps_say_hello.py", | ||
| "tests/test_wps_hello.py", | ||
| "docs/source/notebooks/example.ipynb", | ||
| "environment.yml", | ||
| "requirements.txt", | ||
| "AUTHORS.rst", | ||
| "CHANGES.rst", | ||
| "docs/source/notebooks/index.rst", | ||
| "docs/source/processes.rst", | ||
| "flyingpigeon/processes/__init__.py", | ||
| "tests/test_wps_caps.py" | ||
| ], | ||
| "context": { | ||
| "cookiecutter": { | ||
| "full_name": "Nils Hempelmann", | ||
| "email": "[email protected]", | ||
| "github_username": "bird-house", | ||
| "project_name": "Flyingpigeon", | ||
| "project_slug": "flyingpigeon", | ||
| "project_repo_name": "flyingpigeon", | ||
| "project_readthedocs_name": "flyingpigeon", | ||
| "project_short_description": "A Web Processing Service Testbed.", | ||
| "version": "1.5.1", | ||
| "open_source_license": "Apache Software License 2.0", | ||
| "http_port": "8093", | ||
| "_copy_without_render": [ | ||
| "{{cookiecutter.project_slug}}/templates/*.cfg" | ||
| ], | ||
| "_template": "https://github.com/bird-house/cookiecutter-birdhouse.git" | ||
| } | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,38 +1,44 @@ | ||
| dist: xenial # required for Python >= 3.7 | ||
| language: python | ||
| os: | ||
| - linux | ||
| python: | ||
| - "3.6" | ||
| - "3.7" | ||
| branches: | ||
| only: | ||
| - master | ||
| os: | ||
| - linux | ||
| # - osx | ||
| sudo: false | ||
| before_install: | ||
| # Useful for debugging Travis CI environment | ||
| - printenv | ||
| install: | ||
| # Python 3.x is default | ||
| - wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh | ||
| - bash miniconda.sh -b -p $HOME/miniconda | ||
| - export PATH="$HOME/miniconda/bin:$PATH" | ||
| - hash -r | ||
| - conda config --set always_yes yes --set changeps1 no | ||
| - conda install setuptools | ||
| - conda update -q conda | ||
| # Useful for debugging any issues with conda | ||
| - conda info -a | ||
| # Prepare env with Python version | ||
| - conda create -n flyingpigeon python=$TRAVIS_PYTHON_VERSION | ||
| - conda create -n flyingpigeon -c conda-forge python=$TRAVIS_PYTHON_VERSION | ||
| # Update now the env with our environment | ||
| - conda env update -f environment.yml | ||
| - source activate flyingpigeon | ||
| # Packages for testing | ||
| - conda install pytest flake8 | ||
| # Packages for testing and generating docs | ||
| - make develop | ||
| # Install WPS | ||
| - python setup.py install | ||
| before_script: | ||
| # Start WPS service on port 5000 on 0.0.0.0 | ||
| - flyingpigeon start --daemon --bind-host 0.0.0.0 --port 5000 | ||
| # Start WPS service on port 8093 on 0.0.0.0 | ||
| - flyingpigeon start --daemon --bind-host 0.0.0.0 --port 8093 | ||
| - sleep 2 | ||
| script: | ||
| - pytest | ||
| - make test-notebooks || echo "allow fail" | ||
| - flake8 | ||
| - make docs |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5,6 +5,9 @@ APP_NAME := flyingpigeon | |
| # TODO read in from server configuration | ||
| WPS_URL = http://localhost:8093 | ||
| OUTPUT_URL = https://pavics.ouranos.ca/wpsoutputs | ||
|
|
||
| SANITIZE_FILE := https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/raw/master/notebooks/output-sanitize.cfg | ||
|
|
||
| # end of configuration | ||
|
|
||
| .DEFAULT_GOAL := help | ||
|
|
@@ -26,7 +29,9 @@ help: | |
| @echo "\nTesting targets:" | ||
| @echo " test to run tests (but skip long running tests)." | ||
| @echo " test-all to run all tests (including long running tests)." | ||
| @echo " test-notebooks to verify Jupyter Notebook test outputs are valid." | ||
| @echo " lint to run code style checks with flake8." | ||
| @echo " refresh-notebooks to verify Jupyter Notebook test outputs are valid." | ||
| @echo "\nSphinx targets:" | ||
| @echo " docs to generate HTML documentation with Sphinx." | ||
| @echo "\nDeployment targets:" | ||
|
|
@@ -44,6 +49,7 @@ install: | |
| develop: | ||
| @echo "Installing development requirements for tests and docs ..." | ||
| @-bash -c 'pip install -e ".[dev]"' | ||
| @-bash -c 'pip install git+https://github.com/metalink-dev/[email protected]#egg=pymetalink --upgrade' | ||
|
|
||
| .PHONY: start | ||
| start: | ||
|
|
@@ -61,15 +67,15 @@ restart: stop start | |
|
|
||
| .PHONY: status | ||
| status: | ||
| @echo "Show status ..." | ||
| @echo "Showing status ..." | ||
| @-bash -c "$(APP_NAME) status" | ||
|
|
||
| .PHONY: clean | ||
| clean: clean-build clean-pyc clean-test ## remove all build, test, coverage and Python artifacts | ||
|
|
||
| .PHONY: clean-build | ||
| clean-build: | ||
| @echo "Remove build artifacts ..." | ||
| @echo "Removing build artifacts ..." | ||
| @-rm -fr build/ | ||
| @-rm -fr dist/ | ||
| @-rm -fr .eggs/ | ||
|
|
@@ -80,21 +86,22 @@ clean-build: | |
|
|
||
| .PHONY: clean-pyc | ||
| clean-pyc: | ||
| @echo "Remove Python file artifacts ..." | ||
| @echo "Removing Python file artifacts ..." | ||
| @-find . -name '*.pyc' -exec rm -f {} + | ||
| @-find . -name '*.pyo' -exec rm -f {} + | ||
| @-find . -name '*~' -exec rm -f {} + | ||
| @-find . -name '__pycache__' -exec rm -fr {} + | ||
|
|
||
| .PHONY: clean-test | ||
| clean-test: | ||
| @echo "Remove test artifacts ..." | ||
| @echo "Removing test artifacts ..." | ||
| @-rm -fr .pytest_cache | ||
|
|
||
| .PHONY: clean-dist | ||
| clean-dist: clean | ||
| @echo "Run 'git clean' ..." | ||
| @git diff --quiet HEAD || echo "There are uncommited changes! Not doing 'git clean' ..." | ||
| @echo "Running 'git clean' ..." | ||
| @git diff --quiet HEAD || echo "There are uncommitted changes! Aborting 'git clean' ..." | ||
| ## do not use git clean -e/--exclude here, add them to .gitignore instead | ||
| @-git clean -dfx | ||
|
|
||
| ## Test targets | ||
|
|
@@ -109,37 +116,41 @@ test-all: | |
| @echo "Running all tests (including slow and online tests) ..." | ||
| @bash -c 'pytest -v tests/' | ||
|
|
||
| .PHONY: notebook-sanitizer | ||
| notebook-sanitizer: | ||
| @echo "Copying notebook output sanitizer ..." | ||
| @-bash -c "curl -L $(SANITIZE_FILE) -o $(CURDIR)/docs/source/output-sanitize.cfg --silent" | ||
|
|
||
| .PHONY: test-notebooks | ||
| test-notebooks: | ||
| test-notebooks: notebook-sanitizer | ||
| @echo "Running notebook-based tests" | ||
| @bash -c "curl -L https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/raw/master/notebooks/output-sanitize.cfg --output $(CURDIR)/docs/source/output_sanitize.cfg --silent" | ||
| @bash -c "env WPS_URL=$(WPS_URL) pytest --nbval --verbose $(CURDIR)/docs/source/notebooks/ --sanitize-with $(CURDIR)/docs/source/output_sanitize.cfg --ignore $(CURDIR)/docs/source/notebooks/.ipynb_checkpoints" | ||
|
|
||
| .PHONY: refresh-notebooks | ||
| refresh-notebooks: | ||
| @echo "Refresh all notebook outputs under docs/source/notebooks" | ||
| cd docs/source/notebooks; for nb in *.ipynb; do WPS_URL="$(WPS_URL)" jupyter nbconvert --to notebook --execute --ExecutePreprocessor.timeout=60 --output "$$nb" "$$nb"; sed -i "s@$(WPS_URL)/outputs/@$(OUTPUT_URL)/@g" "$$nb"; done; cd $(APP_ROOT) | ||
|
|
||
|
|
||
| @bash -c "env WPS_URL=$(WPS_URL) pytest --nbval --verbose $(CURDIR)/docs/source/notebooks/ --sanitize-with $(CURDIR)/docs/source/output-sanitize.cfg --ignore $(CURDIR)/docs/source/notebooks/.ipynb_checkpoints" | ||
|
|
||
| .PHONY: lint | ||
| lint: | ||
| @echo "Running flake8 code style checks ..." | ||
| @bash -c 'flake8' | ||
|
|
||
| .PHONY: refresh-notebooks | ||
| refresh-notebooks: | ||
| @echo "Refresh all notebook outputs under docs/source/notebooks" | ||
| @bash -c 'for nb in $(CURDIR)/docs/source/notebooks/*.ipynb; do WPS_URL="$(WPS_URL)" jupyter nbconvert --to notebook --execute --ExecutePreprocessor.timeout=60 --output "$$nb" "$$nb"; sed -i "s@$(WPS_URL)/outputs/@$(OUTPUT_URL)/@g" "$$nb"; done; cd $(APP_ROOT)' | ||
|
|
||
| ## Sphinx targets | ||
|
|
||
| .PHONY: docs | ||
| docs: | ||
| @echo "Generating docs with Sphinx ..." | ||
| @-bash -c '$(MAKE) -C $@ clean html' | ||
| @echo "open your browser: open file://$(APP_ROOT)/docs/build/html/index.html" | ||
| @echo "Open your browser to: file:/$(APP_ROOT)/docs/build/html/index.html" | ||
| ## do not execute xdg-open automatically since it hangs travis and job does not complete | ||
| @echo "xdg-open $(APP_ROOT)/docs/build/html/index.html" | ||
|
|
||
| ## Deployment targets | ||
|
|
||
| .PHONY: dist | ||
| dist: clean | ||
| @echo "Builds source and wheel package ..." | ||
| @echo "Building source and wheel package ..." | ||
| @-python setup.py sdist | ||
| @-python setup.py bdist_wheel | ||
| ls -l dist | ||
| @-bash -c 'ls -l dist/' | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.