Generating documentation HTML after Markdown is added or edited

Generating the documentation HTML (FAQ, News, docs) from the command line

Requirements

• A check-out of the website repo
  • See: Get a copy of the website code
• Pandoc
  • available as a Ubuntu package: "pandoc"
  • v2.9.2.1 (latest from Ubuntu) definitely works and versions back to v2.2 should be OK too
• Perl modules: YAML and JSON
  • apt install libyaml-perl libjson-perl
  • versions shouldn't matter
• curl
• jq (Ubuntu package: "jq")

To install the dependencies in mac:

brew install pandoc
brew install jq

# perl modules
# update perl package manager
sudo perl -MCPAN -e 'install Bundle::CPAN' # might take ages
sudo perl -MCPAN -e 'install Text::CSV'
sudo perl -MCPAN -e 'install YAML::LibYAML'
sudo perl -MCPAN -e 'install JSON::XS'

If that does not work, see: https://formyfriendswithmacs.com/cpan.html

Generating the HTML using update_generated_files.sh

In the top-level directory of the website checkout, run:

./etc/update_generated_files.sh <POMBASE-CONFIG>/website/pombase_v2_config.json <CURATION-REPOSITORY>/data_files/

Where <POMBASE-CONFIG> is a pombase-config and <CURATION-REPOSITORY> is pombase-curation.

The main output is written to src/app/documentation/docs/docs.component.html. When viewed in a browser, it's one big page with no style and internal links don't work. But it's useful for checking basic formatting like tables and lists.

This command is run automatically during the release process but it is still useful for checking the HTML output when editing the Markdown files.

Checking links

The internal links in the Markdown file can be checked with:

./etc/check_docs_broken_links.pl <WEBSITE-CONFIG-PATH>/pombase_v2_config.json src/docs src/assets 

This check is also performed using a GitHub Action on every "git push".

Checking that the site can be built

Converting markdown that includes html tags into html can lead to problems (see https://github.com/pombase/website/pull/1924). You can check in advance if the site can be built. This is also checked in github actions on push.

This requires Node.js to be installed. On recent versions of Ubuntu, you can do apt install nodejs

# if you haven't, install the node dependencies
# you may need to switch to the right node version
npm install

# Get some files
curl -L https://raw.githubusercontent.com/pombase/pombase-config/master/website/pombase_v2_config.json > main_config.json
curl -OL http://current.geneontology.org/metadata/GO.xrf_abbs
curl -OL https://raw.githubusercontent.com/pombase/pombase-config/master/website/trackListTemplate.json
curl -OL https://raw.githubusercontent.com/pombase/pombase-config/master/website/pombase_jbrowse_track_metadata.csv
curl -OL https://raw.githubusercontent.com/pombase/pombase-chado/master/etc/create_jbrowse_track_list.pl && chmod a+x create_jbrowse_track_list.pl
cp src/pombase/index.html src/index.html 

# Run some scripts
./create_jbrowse_track_list.pl trackListTemplate.json pombase_jbrowse_track_metadata.csv /dev/null /dev/null ./minimal_jbrowse_track_list.json

# Build the site (this will fail if there are errors in the html files generated from markdown)
node --max_old_space_size=8000 ./node_modules/@angular/cli/bin/ng build

Converting from word documents to html

You may want to write documentation using word, but then include it in the website as markdown. You can convert word documents to markdown using pandoc:

# -t commonmark to use regular markdown as the output
# --wrap to prevent markdown from wrapping lines with `<` characters
pandoc -t commonmark -f docx --wrap=none doc.docx>test.md

Current status: