Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docs section #37

Merged
merged 27 commits into from
Feb 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
5dc1130
Add docs section and start writing
tverbeiren Feb 16, 2024
e50534c
Add content to docs/
tverbeiren Feb 16, 2024
ebd1082
Minor updates
tverbeiren Feb 19, 2024
8b68d55
Add Github screenshots
tverbeiren Feb 19, 2024
ce1444a
Use Github - wip
tverbeiren Feb 19, 2024
60541b6
Rename one file
tverbeiren Feb 19, 2024
e94cdff
Rename images
tverbeiren Feb 19, 2024
41bd877
Add info on how to use github (wip)
tverbeiren Feb 19, 2024
070cc78
Add info on Github use
tverbeiren Feb 20, 2024
085a541
Add extra flow chart
tverbeiren Feb 20, 2024
6c746cb
Add extra flow chart - 2
tverbeiren Feb 20, 2024
14dcdbf
Only render site to Netlify for PRs
tverbeiren Feb 20, 2024
734b60d
Add screenshot for PR with comflicts
tverbeiren Feb 20, 2024
6044749
Update CI to generate both families and toolbox json files
tverbeiren Feb 20, 2024
c6824ea
Update scenarios
tverbeiren Feb 20, 2024
3dea135
Fix CI
tverbeiren Feb 21, 2024
8a94ee1
Add scripts for installing stuff on Codespaces
tverbeiren Feb 21, 2024
6b77cec
Update README
tverbeiren Feb 21, 2024
aa5f226
Add .gitignore
tverbeiren Feb 21, 2024
9d71d84
Various updates
tverbeiren Feb 21, 2024
bc91116
Add info on rendering failure handling
tverbeiren Feb 21, 2024
40f2ce7
Minor update
tverbeiren Feb 21, 2024
a8a3bf3
Update version of Quarto to latest stable
tverbeiren Feb 21, 2024
5f84766
Add info on reverting changes
tverbeiren Feb 22, 2024
ce2f108
Add more info on what virus information looks like
tverbeiren Feb 22, 2024
8c43a96
Minor update
tverbeiren Feb 22, 2024
5de149a
Add pointer to docs
tverbeiren Feb 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 12 additions & 7 deletions .github/workflows/release-to-virusbank.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
on:
push:
branches: [ main ]
branches: [main]

name: Render and deploy to Netlify and VirusBank

Expand All @@ -15,11 +15,16 @@ jobs:
steps:
- uses: actions/checkout@v3

- name: Assemble alpha/toolbox.json
- name: Assemble alpha/toolbox.json
uses: mikefarah/yq@master
with:
cmd: yq -M -o=json src/toolbox.yaml > toolbox/toolbox.json

- name: Create families.json based src/families.yaml
uses: mikefarah/yq@master
with:
cmd: yq -M -o=json src/families.yaml > families.json

- uses: r-lib/actions/setup-r@v2
with:
r-version: "4.2"
Expand All @@ -32,22 +37,22 @@ jobs:
- name: Install Quarto
uses: quarto-dev/quarto-actions/setup@v2
with:
version: 1.4.449
version: 1.4.549
tinytex: true

- name: Render Quarto Project
uses: quarto-dev/quarto-actions/render@v2
with:
to: html

- name: Deploy to Netlify 🚀
uses: data-intuitive/netlify-deploy-site@v1
with:
auth: ${{ secrets.NETLIFY_AUTH_TOKEN }}
site: ${{ env.NETLIFY_SITE_ID }}
dir: '_site'
dir: "_site"
prod: true
message: 'Deploy production ${{ github.ref }}'
message: "Deploy production ${{ github.ref }}"

- name: Deploy preview to VirusBank
id: deploy_virusbank
Expand All @@ -56,6 +61,6 @@ jobs:
server: ftp.virusbankplatformorg.webhosting.be
username: floor@virusbankplatformorg
password: ${{ secrets.FTP_AUTH_SECRET }}
local-dir: './_site/'
local-dir: "./_site/"
log-level: standard
# dangerous-clean-slate: true
19 changes: 12 additions & 7 deletions .github/workflows/release_to_netlify.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
on:
push:
branches-ignore: [ main ]
# push:
# branches-ignore: [ main ]
pull_request:
types:
- opened
Expand All @@ -20,11 +20,16 @@ jobs:
steps:
- uses: actions/checkout@v3

- name: Assemble alpha/toolbox.json
- name: Create toolbox.json based on src/toolbox.yaml
uses: mikefarah/yq@master
with:
cmd: yq -M -o=json src/toolbox.yaml > toolbox/toolbox.json

- name: Create families.json based src/families.yaml
uses: mikefarah/yq@master
with:
cmd: yq -M -o=json src/families.yaml > families.json

- uses: r-lib/actions/setup-r@v2
with:
r-version: "4.2"
Expand All @@ -37,24 +42,24 @@ jobs:
- name: Install Quarto
uses: quarto-dev/quarto-actions/setup@v2
with:
version: 1.4.449
version: 1.4.549
tinytex: true

- name: Render Quarto Project
uses: quarto-dev/quarto-actions/render@v2
with:
to: html

- name: Deploy preview 🚀
id: deploy_preview
if: github.event_name == 'pull_request'
uses: data-intuitive/netlify-deploy-site@v1
with:
auth: ${{ secrets.NETLIFY_AUTH_TOKEN }}
site: ${{ env.NETLIFY_SITE_ID }}
dir: '_site'
dir: "_site"
alias: "${{ env.BRANCH_NAME }}"
message: 'Deploy production ${{ github.ref }}'
message: "Deploy production ${{ github.ref }}"

- uses: thollander/actions-comment-pull-request@v2
if: github.event_name == 'pull_request'
Expand Down
13 changes: 11 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,13 @@
alpha/index.html
alpha/index_files/
.Rhistory
_material

# node
node_modules
package.json
package-lock.json

# quarto
/.quarto/
_site
index_files
_extensions
215 changes: 25 additions & 190 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,211 +1,46 @@
# VirusBank Phylotree project

# Introduction

Update

## Introduction

This repository contains the code *and* content for the PhyloViz part of
the VirusBank platform. The idea is that as much as possible of the site
is generated, either during pre-rendering or during a site visit. Next
to that, all functionality that is generic is handled as such and can be
found under `_js/`.
the \[VirusBank platform\](https://virusbankplatform.be. The idea is
that as much as possible of the site is generated, either during
pre-rendering or page load. Next to that, all functionality that is
generic is handled as such and can be found under `_js/`.

We use [Quarto]() for rendering the site from source code and content to
what you see on the internet. Quarto files have the `.qmd` extension and
typically contain a YAML header (the part between the `---` lines). By
using Quarto, it’s possible for instance to include files from `_js/`
into another file using a `{{< include … >}}` statement.
We use [Quarto](https://quarto.org) for rendering the site from source
code and content to what you see on the internet. Quarto files have the
`.qmd` extension and typically contain a YAML header (the part between
the `---` lines). By using Quarto, it’s possible for instance to include
files from `_js/` into another file using a `` statement.

This can be illustrated using three situations that the site/app is
created for:

## A new tool is available for a virus
### A new tool is available for a virus

Adding a new tool for a virus should be extrememly easy, especially if
we keep in mind that developing tools is the core business of the
VirusBank.
Adding a new tool for a virus should be easy, especially if we keep in
mind that developing tools is the core business of the VirusBank.

## Adding a virus to an existing family
### Adding a virus to an existing family

If tools are developed or made available for a virus that was not yet
listed, we should be able quickly add it this information and let this
listed, we should be able to quickly add this information and let this
be reflected where relevant.

## Adding a new virus family
### Updating information about a virus

Also this should be easy. The hard task should be collecting tools and
curated information about the viruses at hand, not updating the site and
visualization.
Next to structure information about a virus, we also store and show
semi-structured data: transmission and symptoms, for instance. This
information should be easy to add or update.

These scenarios (and others) have been taken into account for the design
of the site/app.

# Details

## Structure

Each virus family is stored in a *directory* or *folder*, for instance:
`flaviviridae/`. Inside this folder, there are a couple of files that
are required to bootstrap the family page:

- `tree.newick`: The newick-format tree file downloaded from the ICTV
website
- `family.xlsx`: The Excel file containing annotations for the viruses
of interest. The IDs should match those in the `tree.newick` file.

Based on these two files, the following files are generated using
`_js/update_family.js`:

- `index.qmd`: This will become the main family overview page. This page
in turn *includes* all the virus page content
- `_details.qmd`: The part of the page where individual virus
information is shown in tabs. This file is included in `index.qmd`
- For every virus in the family (say `VIR`), multiple files are
generated:
- `VIR.qmd`: the main virus page, basically a number of *includes* of
the files below
- `_VIR.qmd`: The content for the Virus page, also included in the
virus family `_details.qmd` and displayed as tabs
- `_VIR-symptoms.qmd`: The text description of the symptoms for `VIR`
- `_VIR-symptoms.qmd`: A pointer to the symptoms figure, usually
stored under `img/`
- `_VIR-transmission.qmd`: A text description of the transmission of
`VIR`. This is kept empty for now because the figure is
self-explanatory.
- `_VIR-transmission-fig.qmd`: A figure explaining the transmission of
`VIR`. The image itself is usually stored under `img/`
- `_VIR-relevance.qmd`: A text description of the medical relevance of
`VIR`
- `_VIR-relevance-fig.qmd`: A pointer to a figure for the medical
relevance (and optionally a caption)

Graphically, the includes can be depicted as follows:

``` mermaid
graph LR

subgraph _js/
_js/_ojs_data.qmd
_js/virus-breadcrumb.qmd
_js/family-breadcrumb.qmd
end

_details.qmd --&gt; index.qmd
_js/_ojs_data.qmd --&gt; index.qmd
_js/_ojs_data.qmd --&gt; VIR.qmd
_js/virus-breadcrumb.qmd --&gt; VIR.qmd
_js/family-breadcrumb.qmd --&gt; index.qmd

_VIR.qmd --&gt; _details.qmd
_VIR.qmd --&gt; VIR.qmd

_VIR-symptoms.qmd --&gt; _VIR.qmd
_VIR-symptoms.qmd --&gt; _VIR.qmd
_VIR-transmission.qmd --&gt; _VIR.qmd
_VIR-transmission-fig.qmd --&gt; _VIR.qmd
_VIR-relevance.qmd --&gt; _VIR.qmd
_VIR-relevance-fig.qmd --&gt; _VIR.qmd
```

The includes themselves are not parametrized, but the YAML header
`VIR.qmd` is, for example `alphaviruses/CHIKV.qmd` contains the
following header:

``` yaml
---
params:
family: alphaviruses
virus: Chikungunya virus (CHIKV)
---
```

Based on this the correct toolbox entries are shown, for instance.

## Build and rendering

As mentioned already, different build stages are performed and each
generates different aspects of the site.

### Rendering step 1: Bootstrap

Remember that we start off with `family.xlsx` and `tree.newick` for ever
family. That’s all we need. The bootstrap step generates all the other
files mentioned above based on the information in `family.xlsx`:

- The `index.qmd` file as well as the `_details.qmd` for the family
- Per virus, it generates the different `_VIR-...` files.

Please note that this step also generates the content files itself, even
though we do not have content yet. Instead, placeholder text and figures
are inserted to be replaced later.

In order to build this step, one needs NodeJS installed and the `npm`
package manager. The package manager can be used to install two
packages:

``` sh
npm install xlsx liquid
```

This has to be done just once.

Next, we create a directory for the virus family, say
`virusbankviridae`:

``` sh
mkdir virusbankviridae
```

and store the correct `family.xlsx` and `tree.newick` in this directory.

The following steps are required to bootstrap the site for this
*family*:

``` sh
node _js/update_family.js -i virusbankviridae
```

**TODO**: Add a script to build *all* families that are available and
let the CI run these steps.

## How the site is built

# Development

## Requirements and setup

In order to *render* this site/app locally, the following are required:

0. `wget`

1. Quarto

2. `yq`:

``` sh
sudo wget -qO /usr/local/bin/yq https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64
sudo chmod a+x /usr/local/bin/yq
```

## Rendering

## Setup

### Toolbox page

The toolbox page is based on a YAML file.

### Virus family pages

Every virus family is contained in a *folder* of its own. Inside this
folder, everything related to this specific family can be found.

### Virus pages

## Various remarks

- `d3` is loaded by Quarto already, there is not need to load it again

## Things to add
## Documentation

- What is *including* what and why
- How to update *existing* info
- How to add new info
Documentation has been integrated in the app itself and [is stored onder
`docs/`](https://steady-beijinho-13360d.netlify.app//docs).
Loading
Loading