[Compiler] Add sitemap to doc generator #8348
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…al-lang#5990)" This reverts commit a6d0571.
straight-shoota
force-pushed
the
feature/docs-sitemap
branch
from
dfb4071
to
236e120
Compare
RX14
approved these changes
Oct 29, 2019
bcardiff
reviewed
Oct 29, 2019
straight-shoota
force-pushed
the
feature/docs-sitemap
branch
from
236e120
to
c809501
Compare
This is useful for specs where all the configuration options are irrelevant.
straight-shoota
force-pushed
the
feature/docs-sitemap
branch
from
e781fc4
to
6536e81
Compare
bcardiff
approved these changes
Oct 29, 2019
straight-shoota
force-pushed
the
feature/docs-sitemap
branch
from
6536e81
to
3e07569
Compare
straight-shoota
added a commit
to straight-shoota/crystal
that referenced
this pull request
Nov 14, 2020
The previous attempt to use sitemaps to indicate preference of latest version over older ones to search engines doesn't seem to work. Instead, the solution applied for similar situations is canonical links. This mostly reinstates the canonical link behaviour prior to crystal-lang#8348
bcardiff
pushed a commit
that referenced
this pull request
Dec 1, 2020
The previous attempt to use sitemaps to indicate preference of latest version over older ones to search engines doesn't seem to work. Instead, the solution applied for similar situations is canonical links. This mostly reinstates the canonical link behaviour prior to #8348
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The doc generator can produce a
sitemap.xmlwhich lists all HTML pages accessible for search engines. The goal is to use this sitemap to assign lower priorities to outdated doc pages on https://crystal-lang.org/api/ to make the most recent release (latest) show up first in search results. See crystal-lang/crystal-website#79 for details. Having such a sitemap is generally useful for shards as well, so it makes sense to directly integrate it into the docs generator.Sitemaps are only generated when
--sitemap-base-url=URLoption is set. Sitemaps require absolute paths, so it can't be produced without a base url. Additional options are--sitemap-priorityand--sitemap-changefreqwhich correspond do the respective sitemap features.This also reverts #5990 which tried an alternative approach to solving the search priority issue using canonical URLs. But this completely removes older versions from search results.
Ideally,
sitemap.xmlshould be compressed, but the compiler is supposed to be built without libz, so this needs to be done separately.