Updating to PyData Sphinx Theme

[choldgraf]

This is a PR to start updating the theme to the PyData Sphinx Theme. It is a basic "topbar-based theme".

For now, the PR is just a very basic theme switch to see how the new one looks "out of the box".

ToDo

• I noticed one bug in the PyData theme when I switched over the theme which should be resolved in FIX: Updating the right sidebar width breakpoint pydata/pydata-sphinx-theme#341
• Version selector - right now this doesn't exist in the pydata theme (see ENH: Add version switcher dropdown menu pydata/pydata-sphinx-theme#276 for one proposal)
• Language selector - right now this doesn't exist in the pydata theme at all
  • ReadTheDocs selector support - another option is that we could try to use the RTD language/version switcher javascript, rather than hand-coding our own. This would entail figuring out the HTML that RTD needs to display this information and coding it into the theme.
• Custom footer - So that we can display the privacy notice
• Logo size - Logo should be bigger, how to do this given current logo ratio? @jpmckinney do you have thoughts?
• Sidebar splash image - "Developed by Open Contracting" splash image needs to be in there somewhere

[jpmckinney]

We can change the logo to PNG, and I think we have a new OCDS logo we can use.

Some things to keep:

• Language switcher
• Version switcher
• Clear indication of version being viewed
• Additional links (like the Data Review Tool)

Ideally we can keep a footer for the license and privacy policy.

I’ll double check if we need the OCP logo on every page or if we can just put it on the homepage.

I can have a look at our layout template to see what else we had overridden.

[choldgraf]

@jpmckinney I added a few more todo items to the top comment, LMK if I'm missing some stuff

[choldgraf]

@jpmckinney could you explain how the language and version switchers work on the current theme? I see that you've got a language_options block in there (https://github.com/open-contracting/standard/blob/1.1-dev/docs/_templates/layout.html#L29) but is that over-riding a RTD-theme-specific area?

[jpmckinney]

Here's the relevant part of the theme: https://github.com/open-contracting/standard_theme/blob/open_contracting/standard_theme/layout.html#L100-L122

The switchers are implemented in Apache (you don't need to know this, but FYI): https://github.com/open-contracting/deploy/blob/main/salt/apache/files/sites/docs.conf.include#L118-L168

[choldgraf]

ah nice - so that should be somewhat drop-inable into a template or something? Is that what you had in mind?

As an aside, this conversation sparked some renewed energy into figuring this out in the pydata theme itself, feel free to provide thoughts there if you like! pydata/pydata-sphinx-theme#360

[jpmckinney]

Yes, that should work. I think adding the language and version as navbar items like in this pydata-sphinx-theme PR should be fine: https://github.com/pydata/pydata-sphinx-theme/pull/276/files

For the linked issue, I think the approach of a versions.json file makes sense for the general case. In our case we have a fairly bespoke setup already, so we're likely to only inherit HTML/CSS changes.

[ChaiByte]

Can we add custom HTML blocks for specific pages (like html_sidebars).

• Collect user feedback at the bottom of some pages and jump to somewhere such as a forum or open a mail to window
• Not only do I use Google Analytics to collect website data, but I also need to insert more customized scripts

[choldgraf]

@megchai is this question specifically for this repository's documentation? or about the pydata theme in general?

[ChaiByte]

@MegChai is this question specifically for this repository's documentation? or about the pydata theme in general?

pydata theme :) Sorry for not clarifying

[choldgraf]

ah gotcha - think you could open an issue in the pydata theme repository if you have questions about it? This PR is for a totally different project, it just happens to be using the pydata theme :-)

[choldgraf]

hey @jpmckinney - I just wanted to give a heartbeat on this. I'm trying to get pydata/pydata-sphinx-theme#355 finished and then make a release, which lays some groundwork for templating that we can re-use here. We've also merged a few other PRs in that theme that I think you'll like (e.g., collapsible side-bars) 👍

[jpmckinney]

Nice! I appreciate the update.

[choldgraf]

I just released another patch to pydata sphinx theme to fix some bugs that were making this theme update look weird, and added the templates that I think should get this to work. Here's how the theme looks now:

A few questions for @jpmckinney

• What do you think about the look and feel? Doesn't need to be perfect but it should be good enough for an MVP
• I think that I got the version/language switching correct but let me know if that seems off
• The error seems to be related to translations...could that be related to the theme change somehow?
• I think the logo might be a bit tiny, but the ratio constraints us a bit. What do you think?

By the way, any chance you could activate ReadTheDocs previews for PRs? I find that it's really helpful in reviewing design and website builds.

[jpmckinney]

By the way, any chance you could activate ReadTheDocs previews for PRs? I find that it's really helpful in reviewing design and website builds.

We don't use ReadTheDocs. We have our own system to deploy to https://standard.open-contracting.org/staging/BRANCH-NAME, but only if CI passes :)

I'm not sure why util/add_translation_notes.py is failing to find notes.mo (the last step of the build). I'll try locally. Update: I fixed the initial issue (notes.po not compiled). Now I need to update the script to work with the new theme.

Need to:

• Knock-on effects
  • Update tests for new theme (uncomment old line in test_docs.py?)
  • Update ocdsindex for new theme
• Internationalization
  • Translate the theme i18n: Add full support pydata/pydata-sphinx-theme#399
  • Check that translations work
  • Update add_translation_notes.py for new theme (note: removed from 1.1-dev)
• Features
  • Fix language and version switchers extend version switcher to include language switching pydata/pydata-sphinx-theme#507
  • Move, update and use search.js from standard_theme
  • Copy relevant changes to search.html and searchbox.html from standard_theme
  • Restore Fathom Analytics with code HTWZHRIZ
    • Can use {% block extrahead %} and call {{ block.super() }}
• Look and feel
  • Find a place for the profile's name (OCDS for European Union, etc.)
  • CSS customization (inspiration: https://pandas.pydata.org/docs/ https://framework.frictionlessdata.io)
  • Use color and shadow to create "layers" like in Google's API docs, to make the right-hand menu more noticeable
  • Use previous fonts (was using Typekit)
• Navigation
  • Review toctree (recommonmark previously nested sub-pages under in-page headings)
  • Interlink documentation websites via top navigation, e.g. Profiles (and OC4IDS), Tools (registry, flattening tool)
• Miscellaneous
  • Fix TODOs in conf.py
  • Add <!--#include virtual="$BANNER" --> somewhere
  • Go through all templates
• Look into allowing the theme to be customized via html_theme_options as before.
  • Use html_context, which has the same effect as html_theme_options.
  • analytics_id: See Fathom Analytics task.
  • display_version: Pending theme configuration.
  • root_url: Relevant to the version switcher. Pending getting that working.
  • short_project: Only relevant to mobile UI. Pending theme configuration.
  • copyright, etc.: These relate to footer.html

    'analytics_id': 'HTWZHRIZ',
    'display_version': True,
    'root_url': '',
    'short_project': project.replace('Open Contracting Data Standard', 'OCDS'),
    'copyright': copyright,
    'license_name': 'Apache License 2.0',
    'license_url': '{}/blob/HEAD/LICENSE'.format(repository_url),
    'repository_url': repository_url,

[jpmckinney]

If my commits build, I'll test the switchers at https://standard.open-contracting.org/staging/theme Then, I'll discuss with our Head of Communications what CSS customizations I should prioritize.

Previously, we had forked a theme to customize it. Is it straightforward to customize the theme without forking? Is it all described in the theme's docs, or are there some generic Sphinx ways of customizing themes?

[jpmckinney]

The very last step is failing (ocdsindex), but it still deploys the rest! https://standard.open-contracting.org/staging/theme/en/

[choldgraf]

@jpmckinney in general the PyData theme is designed to be pretty customizable and flexible, so I'd say we try that first and then only fork if really needed, since that'll significantly increase the maintenance burden in my opinion

[jpmckinney]

@jpmckinney in general the PyData theme is designed to be pretty customizable and flexible, so I'd say we try that first and then only fork if really needed, since that'll significantly increase the maintenance burden in my opinion

We'll be using the theme on 4 other documentation websites, which means if we edit the theme, we have to repeat it in 3 other repositories, which might exceed the maintenance burden of having a fork? What do you think?

That said, we already have script/update and script/diff scripts to align each website with https://github.com/open-contracting/standard_profile_template, so it might not be so bad to repeat changes 3-4 times.

[choldgraf]

@jpmckinney ah that's good to know. I think that the approach is still the same - try to make the edits on one site, and see how nasty the customizations are. E.g., if it's just a CSS file that needs to be shared across sites, that could be centralized in one place and the other sites could pull from it, without creating a whole new theme.