Skip to content

Commit

Permalink
Add CONTRIBUTING, document release process and improve README
Browse files Browse the repository at this point in the history
  • Loading branch information
axelboc committed Dec 22, 2020
1 parent 5caba81 commit a8d212c
Show file tree
Hide file tree
Showing 6 changed files with 238 additions and 140 deletions.
1 change: 0 additions & 1 deletion .prettierrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
{
"files": "*.md",
"options": {
"printWidth": 120,
"proseWrap": "always"
}
}
Expand Down
169 changes: 169 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
# Contributing

- [Quick start](#quick-start-)
- [Build](#build)
- [Code quality](#code-quality)
- [Automatic fixing and formatting](#automatic-fixing-and-formatting)
- [Editor integration](#editor-integration)
- [Testing](#testing)
- [Visual regression](#visual-regression)
- [Continuous deployment](#continuous-deployment)
- [NPM packages](#npm-packages-)
- [Release process](#release-process)
- [Local testing](#local-testing)
- [Icon set](#icon-set)

## Quick start 🚀

```bash
npm install
npm start
```

## Build

- `npm run build` - build H5Web for production
- `npm run storybook:build` - build the component library's Storybook
documentation site

## Code quality

- `npm run lint` - run all linting and code formatting commands
- `npm run lint:eslint` - lint all TS and JS files with ESLint
- `npm run lint:tsc` - type-check the whole project, test files included
- `npm run lint:prettier` - check that all files have been formatted with
Prettier

### Automatic fixing and formatting

- `npm run lint:eslint -- --fix` - auto-fix linting issues
- `npm run lint:prettier -- --write` - format all files with Prettier

### Editor integration

Most editors support fixing and formatting files automatically on save. The
configuration for VSCode is provided out of the box, so all you need to do is
install the recommended extensions.

## Testing

- `npm test` - run unit and feature tests with Jest
- `npm run cypress` - open the
[Cypress](https://docs.cypress.io/guides/overview/why-cypress.html) end-to-end
test runner (local dev server must be running in separate terminal)
- `npm run cypress:run` - run end-to-end tests once (local dev server must be
running in separate terminal)
- `npm run storybook` - manually test components in isolation in
[Storybook](https://storybook.js.org/docs/react/get-started/introduction), at
http://localhost:6006

### Visual regression

Cypress is used for end-to-end testing but also for visual regression testing.
The idea is to take a screenshot (or "snapshot") of the app in a known state and
compare it with a previously approved "reference snapshot". If any pixel has
changed, the test fails and a diff image highlighting the differences is
created.

Taking consistent screenshots across platforms is impossible because the exact
rendering of the app depends on the GPU. For this reason, visual regression
tests are run only on the CI. This is done through an environment variable
called `CYPRESS_TAKE_SNAPSHOTS`.

Visual regression tests may fail in the CI, either expectedly (e.g. when
implementing a new feature) or unexpectedly (when detecting a regression). When
this happens, the diff images and debug screenshots that Cypress generates are
uploaded as artifacts of the workflow, which can be downloaded and reviewed.

If the visual regressions are expected, the version-controlled reference
snapshots can be updated by posting a comment in the Pull Request with this
exact text: `/approve`. This triggers the _Approve snapshots_ workflow, which
runs Cypress again but this time telling it to update the reference snapshots
when it finds differences and to pass the tests. Once Cypress has updated the
reference snapshots, the workflow automatically opens a PR to merge the new
and/or updated snapshots into the working branch. After this PR is merged, the
visual regression tests in the working branch succeed and the branch can be
merged into `master`.

Here is the summarised workflow (also described with screenshots in
[PR #306](https://github.com/silx-kit/h5web/pull/306)):

1. Push your working branch and open a PR.
2. If the `e2e` job of the _Lint & Test_ CI workflow fails, check out the logs.
3. If the fail is caused by a visual regression (i.e. if a test fails on a
`cy.matchImageSnapshot()` call), download the workflow's artifacts.
4. Review the snapshot diffs. If the visual regression is unexpected: fix the
bug, push it and start from step 2 again. If the visual regression is
expected: continue to step 5.
5. In the PR, post a comment with `/approve`.
6. Go to the _Actions_ page and wait for the _Approve snapshots_ workflow to
complete.
7. Go to the newly opened PR titled _Update Cypress reference snapshots_.
8. Review the new reference snapshots once more and merge the PR.
9. Go back to your main PR and wait for the jobs of the _Lint & Test_ workflow
to succeed.

## Deployment

- The project's `master` branch is continuously deployed to
https://h5web.panosc.eu/ with [Netlify](https://www.netlify.com/).
- The component library's Storybook documentation site is deployed to GitHub
Pages on every release: https://h5web-docs.panosc.eu

## NPM packages 📚

The `src/packages` folder contains entry points for the packages published to
NPM: [@h5web/lib](https://www.npmjs.com/package/@h5web/lib) and
[@h5web/app](https://www.npmjs.com/package/@h5web/app).

### Release process

To release a new version:

1. Check out `master` and pull the latest changes.
1. Make sure your working tree doesn't have uncommitted changes and that the
latest commit on `master` has passed the CI.
1. Run `npm version [ patch | minor | major | <new-version> ]`

This command bumps the version number in `package.json`, commits the change and
then tags the commit with the same version number. The `postversion` script then
runs automatically and pushes the new commit and the new tag. This, in turn,
triggers the _Release_ workflow on the CI, which builds and publishes the
packages to NPM and deploys the Storybook site. The workflow is detailed in
[issue #358](https://github.com/silx-kit/h5web/issues/358).

Once the _Release_ worklow has completed:

- Make sure the new package versions are available on NPM and that the live
Storybook site still works as expected.
- Upgrade and test the packages in apps and code sandboxes, as required.
- Write and publish [release notes](https://github.com/silx-kit/h5web/releases)
on GitHub.

### Build and test locally

To build the packages, run the following commands:

```bash
cd packages
npm install
npm run build
```

To test a package locally, build the packages then run the following commands:

```bash
cd lib/dist
npm link
cd /your/test/app
npm link @h5web/lib
```

> If you see an "invalid hook call" error, you may need to
> [alias the `react` and `react-dom` imports](https://github.com/facebook/react/issues/13991#issuecomment-435587809)
> to point to your test app's `node_modules` folder.
## Icon set

H5Web uses the [Feather icon set](https://react-icons.netlify.com/#/icons/fi).
Icons can be imported as React components from `react-icons/fi`.
24 changes: 14 additions & 10 deletions LICENSE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,19 @@

**Copyright (c) 2020 European Synchrotron Radiation Facility**

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the Software without restriction, including without limitation the
rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit
persons to whom the Software is furnished to do so, subject to the following conditions:
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the
Software.
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
164 changes: 42 additions & 122 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,147 +2,67 @@

[![Demo](https://img.shields.io/website?down_message=offline&label=demo&up_message=online&url=https%3A%2F%2Fh5web.panosc.eu%2F)](https://h5web.panosc.eu/)

H5Web is a web-based viewer to explore HDF5 files. It is built with React and uses
[react-three-fiber](https://github.com/react-spring/react-three-fiber) for visualizations.
H5Web is a web-based viewer to explore HDF5 files. It is built with React and
uses [react-three-fiber](https://github.com/react-spring/react-three-fiber) for
visualizations.

## Quick start 🚀
![H5Web GIF demo](https://user-images.githubusercontent.com/2936402/102904182-ab3fa400-4471-11eb-9c7a-606deffebc43.gif)

```bash
npm install
npm start
```

## Development 🛠️

### Testing

- `npm test` - run unit and feature tests with Jest
- `npm run cypress` - open the [Cypress](https://docs.cypress.io/guides/overview/why-cypress.html) end-to-end test
runner (local dev server must be running in separate terminal)
- `npm run cypress:run` - run end-to-end tests once (local dev server must be running in separate terminal)
- `npm run storybook` - manually test components in isolation in
[Storybook](https://storybook.js.org/docs/react/get-started/introduction), at http://localhost:6006

#### Visual regression

Cypress is used for end-to-end testing, but also for visual regression testing. The idea is to take screenshots
(referred to as "snapshots") of the app in various situations and compare them with previously approved "reference
snapshots". If any pixel has changed, the test fails and a diff image that highlights the differences is created.

Cypress is unable to take consistent screenshots across platforms because the exact rendering depends on the GPU (even
when running in Docker). For this reason, visual regression tests are only run on the CI. Environment variable
`CYPRESS_TAKE_SNAPSHOTS` is used to enable visual regression testing in the CI. This variable is set automatically in
the _Cypress_ GitHub workflow, which runs on push to any branch.

Visual regression tests may fail in the CI, either expectedly (e.g. when implementing a new feature) or unexpectedly
(when detecting a regression). When this happens, the diff images and debug screenshots that Cypress generates are
uploaded as artifacts of the workflow, which can be downloaded and reviewed.

If the visual regressions are expected, the version-controlled reference snapshots can be updated by posting a comment
in the Pull Request with the following text: `/approve`. This action triggers the _Approve snapshots_ workflow, which
runs Cypress again but this time telling it to update the reference snapshots when it finds differences (instead of
failing the tests).

Once Cypress has updated the reference snapshots, the _Approve snapshots_ workflow automatically opens a Pull Request to
merge the new and/or updated snapshots into the working branch. After this PR is merged, the visual regression tests in
the working branch should succeed.

Here is the summarised workflow (also described with screenshots in
[PR #306](https://github.com/silx-kit/h5web/pull/306)):

1. Push your working branch and open a PR.
2. If the _Cypress_ CI workflow fails, check out the logs.
3. If the fail is caused by a visual regression (i.e. if a test fails on a `cy.matchImageSnapshot()` call), download the
workflow's artifacts.
4. Review the snapshot diffs:

- If the visual regressions are unexpected: fix the bug, push it and start from step 2 again.
- If the visual regressions are expected: continue to step 5.

5. In the PR, post a comment with the following text: `/approve`.
6. Go to the _Actions_ page and wait for the _Approve snapshots_ workflow to complete.
7. Go to the newly opened PR titled _Update Cypress reference snapshots_.
8. Review the new reference snapshots and merge the PR.
9. Go back to your main PR and wait for the _Cypress_ workflow to succeed.

### Build

- `npm run build` - build H5Web for production
- `npm run storybook:build` - build the Component Library's Storybook documentation site

### Code quality

- `npm run lint` - run all linting and code formatting commands
- `npm run lint:eslint` - lint all TS and JS files with ESLint
- `npm run lint:tsc` - type-check the whole project, test files included
- `npm run lint:prettier` - check that all files have been formatted with Prettier

#### Automatic fixing and formatting

- `npm run lint:eslint -- --fix` - auto-fix linting issues
- `npm run lint:prettier -- --write` - format all files with Prettier

#### Editor integration
## NPM packages 📚

Most editors support fixing and formatting files automatically on save. The configuration for VSCode is provided out of
the box, so all you need to do is install the recommended extensions.
### [@h5web/lib](https://www.npmjs.com/package/@h5web/lib)

### Continuous integration and deployment
H5Web's component library, which includes the main visualization components
(`LineVis`, `HeatmapVis`, etc.) as well as some of their lower-level building
blocks (`VisCanvas`, `ColorBar`, etc.)

- The project's `master` branch is continuously deployed to https://h5web.panosc.eu/ with
[Netlify](https://www.netlify.com/).
- Netlify also deploys each pull request individually to URLs of the form:
https:\//deploy-preview-<pr-number\>--h5web.netlify.app/
- On push to any branch, a [GitHub Action](https://github.com/silx-kit/h5web/actions):
- runs the `lint` and `test` scripts;
- runs end-to-end tests with Cypress.
- On push to `master`, another GitHub Action continuously deploys the Component Library's Storybook documentation site
to GitHub Pages: https://h5web-docs.panosc.eu
The library is documented in a Storybook site accessible at
https://h5web-docs.panosc.eu.

### Icon set
### [@h5web/app](https://www.npmjs.com/package/@h5web/app)

H5Web uses the [Feather icon set](https://react-icons.netlify.com/#/icons/fi). Icons can be imported as React components
from `react-icons/fi`.
H5Web's top-level `App` component and built-in data providers.

## NPM packages 📚
Data providers are components that fetch data from HDF5 back-end solutions and
provide this data to the app through React Context. H5Web currently includes two
providers out of the box, which are both under active development:

The `src/packages` folder contains entry points for packages published to NPM.
- `JupyterProvider` for the
[Jupyter Lab HDF5 extension](https://github.com/jupyterlab/jupyterlab-hdf5)
- `HsdsProvider` for [HSDS](https://github.com/HDFGroup/hsds)

To publish the packages, bump their version numbers in their respective `packages/<name>/package.json` files, log in to
NPM locally with `npm login`, then follow the steps below:
## Demos

```bash
cd packages
npm install
npm run build
cd lib/dist && npm publish
cd ../../app/dist && npm publish
```
The demo at https://h5web.panosc.eu/ shows the content of an HDF5 file converted
to JSON with [hdf5-json](https://github.com/HDFGroup/hdf5-json) and hosted on a
remote static server.

> To test the packages locally, run `npm link` instead of `npm publish`, and then `npm link @h5web/<package>` in your
> test app. If you see an "invalid hook call" error, you may need to
> [alias the `react` and `react-dom` imports](https://github.com/facebook/react/issues/13991#issuecomment-435587809) to
> point to your test app's `node_modules` folder.
At https://h5web.panosc.eu/mock, you can view a set of mock data generated
entirely on the front-end. This demo and its provider, `MockProvider` are used
for end-to-end testing purposes.

### @h5web/lib
### [Jupyter Lab HDF5 extension](https://github.com/jupyterlab/jupyterlab-hdf5)

H5Web's Component Library, which includes the main visualization components (`LineVis`, `HeatmapVis`, etc.) as well as
some of their lower-level building blocks (`AxisSystem`, `ColorBar`, etc.)
This demo is available at https://h5web.panosc.eu/jupyter.

The library is documented in a Storybook site accessible at https://h5web-docs.panosc.eu.
The following HDF5 files can be reached with a URL of the form
`https://h5web.panosc.eu/jupyter?domain=<name>`:

### @h5web/app
- `/home/reader/water` (**default**): A file with data similar to the main demo
https://h5web.panosc.eu/. Some datasets cannot be displayed as bitshuffle
compression is not supported by HSDS yet.
- `/home/reader/tall`: The demo file of HSDS.
- `/home/reader/chunked`: A file containing a simple 2D chunked dataset.

H5Web's top-level `App` component and data providers (`SilxProvider`, etc.)
### [HSDS](https://github.com/HDFGroup/hsds)

## HSDS demo
This demo is available at https://h5web.panosc.eu/hsds.

H5web can display data requested to an [HSDS](https://github.com/HDFGroup/hsds) server instance. A demo is available on
https://h5web.panosc.eu/hsds with several files (or domains) available:
The following HDF5 files can be reached with a URL of the form
`https://h5web.panosc.eu/hsds?domain=<name>`:

- `/home/reader/water` (**default**): A file with data similar to the demo https://h5web.panosc.eu/. Some datasets
cannot be displayed as bitshuffle compression is not supported by HSDS yet.
- `/home/reader/water` (**default**): A file with data similar to the main demo
https://h5web.panosc.eu/. Some datasets cannot be displayed as bitshuffle
compression is not supported by HSDS yet.
- `/home/reader/tall`: The demo file of HSDS.
- `/home/reader/chunked`: A file containing a simple 2D chunked dataset.

The different domains can be reached through `https://h5web.panosc.eu/hsds?domain=<domain_name>`.
Loading

0 comments on commit a8d212c

Please sign in to comment.