Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for maintaining old/multiple versions of site/documents by enabling ghpagesKeepVersions flag. #39

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
*~
target/
boot/
.idea
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,16 @@ sbt-ghpages will only delete files which are matched by the FileFilter specified
For example, to prevent sbt-ghpages from deleting the "CNAME" file located at the root of your site, and any file
named "versions.html", add the following to your build.sbt:

## Maintaining Old Versions
By setting just `ghpagesKeepVersions` flag, you can maintain history of all your old and current versions of site/documents.
Behind the scenes, this flag plays two important roles:
1. Set's the `includeFilter in ghpagesCleanSite` to include all the files from current versions.
(That means, if you publish same version again, old content from that version will be replaced by new content)
2. While publishing site to gh-pages, new directory with current version's name is created and all the site contents will be
Copy link

@julienrf julienrf Mar 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the fact that all the mappings will all be prefixed with the version number should be more emphasized.

Also, how would you recommend to do to have a page that automatically points to the latest version?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For automatic pointing, we can copy latest version at top level as well as inside respective version directory. This can be filtered for snapshot versions which can reside only inside respective snapshot dirs.
We are doing exactly same here:
https://github.com/tmtsoftware/sbt-docs/blob/master/src/main/scala/org/tmt/sbt/docs/Settings.scala

Copy link

@julienrf julienrf Mar 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can copy

You mean sbt-doc or the users of sbt-doc? (Edit: sorry I meant sbt-ghpages)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the GhpagesPlugin, when creating ghpagesPrivateMapping check can be added to detect if its a SNAPSHOT version or not and based on that generate mappings. Something like below:

  private def ghpagesPrivateMappingsTask = Def.task {
    val isSnapshot = version.value.contains("SNAPSHOT")
    val makeSiteMappings = (mappings in SitePlugin.autoImport.makeSite).value
    
    val updatedMappings =
      if (ghpagesKeepVersions.value) {
        val updated = makeSiteMappings map { case (file, target) => (file, 
        version.value + "/" + target) }
        if(isSnapshot) upadated
        else makeSiteMappings ++ updated
      }
      else makeSiteMappings

    updatedMappings
  }

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sbt-doc uses ghpages plugin, I have added workaround I mentioned above in sbt-doc plugin. As currently ghpages plugin does not support versioning.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. Do we want to add a setting key letting users control whether they want to have their website at the root in addition to have it prefixed by the version number?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, make sense!

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have incorporated all these points in my last commit, let me know if those make sense!

copied to that directory

Avoid setting `includeFilter in ghpagesCleanSite` explicitly to protect old files as this is set by plugin to protect files when `ghpagesKeepVersions := true`

```scala
excludeFilter in ghpagesCleanSite :=
new FileFilter{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import sbt._
import Keys._

trait GhpagesKeys {
lazy val ghpagesKeepVersions = settingKey[Boolean]("If this flag is set, ghpages will not clean existing versions of site and make sure that new site is pushed inside current version directory")
lazy val ghpagesCommitOptions = settingKey[Seq[String]]("commit options")
lazy val ghpagesRepository = settingKey[File]("sandbox environment where git project ghpages branch is checked out.")
lazy val ghpagesBranch = settingKey[String]("Name of the git branch in which to store ghpages content. Defaults to gh-pages.")
Expand Down
61 changes: 41 additions & 20 deletions src/main/scala/com/typesafe/sbt/sbtghpages/GhpagesPlugin.scala
Original file line number Diff line number Diff line change
@@ -1,28 +1,31 @@
package com.typesafe.sbt
package sbtghpages

import sbt._
import Keys._
import sbt.{FileFilter, _}
import Keys.{mappings, _}
import com.typesafe.sbt.SbtGit.GitKeys
import com.typesafe.sbt.git.GitRunner
import GitKeys.{gitBranch, gitRemoteRepo}
import com.typesafe.sbt.site.SitePlugin

import scala.util.control.NonFatal

// Plugin to make use of github pages.
object GhpagesPlugin extends AutoPlugin {
override val trigger: PluginTrigger = noTrigger
override val requires: Plugins = SitePlugin && GitPlugin
override val requires: Plugins = SitePlugin && GitPlugin
override lazy val globalSettings: Seq[Setting[_]] = ghpagesGlobalSettings
override lazy val projectSettings: Seq[Setting[_]] = ghpagesProjectSettings
override lazy val projectSettings: Seq[Setting[_]] = ghpagesProjectSettings

object autoImport extends GhpagesKeys

import autoImport._

// TODO - Add some sort of locking to the repository so only one thread accesses it at a time...

def ghpagesGlobalSettings: Seq[Setting[_]] = Seq(
ghpagesBranch := "gh-pages",
ghpagesKeepVersions := false,
ghpagesNoJekyll := true
)

Expand All @@ -32,18 +35,34 @@ object GhpagesPlugin extends AutoPlugin {
ghpagesRepository := {
val buildHash: String =
Hash.toHex(Hash.apply(sbt.Keys.thisProjectRef.value.build.toASCIIString))
file(System.getProperty("user.home")) / ".sbt" / "ghpages" / buildHash / organization.value / name.value
file(System.getProperty("user.home")) / ".sbt" / "ghpages" / buildHash / organization.value / name.value
},
gitBranch in ghpagesUpdatedRepository := gitBranch.?.value getOrElse Some(ghpagesBranch.value),
ghpagesUpdatedRepository := updatedRepo(ghpagesRepository, gitRemoteRepo, gitBranch in ghpagesUpdatedRepository).value,
ghpagesPushSite := pushSiteTask.value,
ghpagesPrivateMappings := (mappings in SitePlugin.autoImport.makeSite).value,
ghpagesPrivateMappings := ghpagesPrivateMappingsTask.value,
ghpagesSynchLocal := synchLocalTask.value,
ghpagesCleanSite := cleanSiteTask.value,
includeFilter in ghpagesCleanSite := AllPassFilter,
includeFilter in ghpagesCleanSite := includeFilterInCleanSiteTask.value,
excludeFilter in ghpagesCleanSite := NothingFilter
)

private def ghpagesPrivateMappingsTask = Def.task {
val makeSiteMappings = (mappings in SitePlugin.autoImport.makeSite).value
val updatedMappings =
if (ghpagesKeepVersions.value) makeSiteMappings map { case (file, target) => (file, version.value + "/" + target) }
else makeSiteMappings
updatedMappings
}

private def includeFilterInCleanSiteTask =
Def.setting {
if (ghpagesKeepVersions.value) new FileFilter {
override def accept(pathname: File): Boolean = pathname.getAbsolutePath.contains(s"/${version.value}")
}
else AllPassFilter
}

private def updatedRepo(repo: SettingKey[File], remote: SettingKey[String], branch: SettingKey[Option[String]]) =
Def.task {
val local = repo.value
Expand All @@ -66,23 +85,25 @@ object GhpagesPlugin extends AutoPlugin {
cleanSiteForRealz(repo, GitKeys.gitRunner.value, s, incl, excl)
// Now copy files.
IO.copy(betterMappings)
if(ghpagesNoJekyll.value) IO.touch(repo / ".nojekyll")
if (ghpagesNoJekyll.value) IO.touch(repo / ".nojekyll")
repo
}

private def cleanSiteTask =
Def.task {
cleanSiteForRealz(ghpagesUpdatedRepository.value, GitKeys.gitRunner.value, streams.value, (includeFilter in ghpagesCleanSite).value, (excludeFilter in ghpagesCleanSite).value)
}

private def cleanSiteForRealz(dir: File, git: GitRunner, s: TaskStreams, incl: FileFilter, excl: FileFilter): Unit = {
val toClean = IO.listFiles(dir)
.filter(f ⇒ f.getName != ".git" && incl.accept(f) && !excl.accept(f)).map(_.getAbsolutePath).toList
if(!toClean.isEmpty)
git(("rm" :: "-r" :: "-f" :: "--ignore-unmatch" :: toClean) :_*)(dir, s.log)
.filter(f ⇒ f.getName != ".git" && incl.accept(f) && !excl.accept(f)).map(_.getAbsolutePath).toList
if (!toClean.isEmpty)
git("rm" :: "-r" :: "-f" :: "--ignore-unmatch" :: toClean: _*)(dir, s.log)
()
}

lazy val commitMessage = sys.env.getOrElse("SBT_GHPAGES_COMMIT_MESSAGE", "updated site")

private def pushSiteTask =
Def.task {
val git = GitKeys.gitRunner.value
Expand All @@ -100,14 +121,14 @@ object GhpagesPlugin extends AutoPlugin {
}

/** TODO - Create ghpages in the first place if it doesn't exist.
*$ cd /path/to/fancypants
*$ git symbolic-ref HEAD refs/heads/gh-pages
*$ rm .git/index
*$ git clean -fdx
*<copy api and documentation>
*$ echo "My GitHub Page" > index.html
*$ git add .
*$ git commit -a -m "First pages commit"
*$ git push origin gh-pages
* $ cd /path/to/fancypants
* $ git symbolic-ref HEAD refs/heads/gh-pages
* $ rm .git/index
* $ git clean -fdx
* <copy api and documentation>
* $ echo "My GitHub Page" > index.html
* $ git add .
* $ git commit -a -m "First pages commit"
* $ git push origin gh-pages
*/
}
10 changes: 10 additions & 0 deletions src/sbt-test/ghpages/site/build.sbt
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
enablePlugins(GhpagesPlugin)
enablePlugins(ParadoxSitePlugin)

ghpagesKeepVersions := true
git.remoteRepo := "git://github.com/sbt/sbt-ghpages.git"
ghpagesBranch := "scripted-test"
sourceDirectory in Paradox := baseDirectory.value / "src" / "main"
sourceDirectory in(Paradox, paradoxTheme) := (sourceDirectory in Paradox).value / "_template"
ghpagesRepository := target.value / "ghpages"
paradoxTheme := None
10 changes: 10 additions & 0 deletions src/sbt-test/ghpages/site/expected/hello.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
<!DOCTYPE html>
<html lang="en">
<head>
<title>Hello · site</title>
</head>
<body>
<h1><a href="#hello" name="hello" class="anchor"><span class="anchor-link"></span></a>Hello</h1>
<p>Hello World!</p>
</body>
</html>
1 change: 1 addition & 0 deletions src/sbt-test/ghpages/site/project/plugins.sbt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
addSbtPlugin("com.typesafe.sbt" % "sbt-ghpages" % sys.props("plugin.version"))
9 changes: 9 additions & 0 deletions src/sbt-test/ghpages/site/src/main/_template/page.st
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
<!DOCTYPE html>
<html lang="en">
<head>
<title>$page.title$ · $page.properties.("project.name")$</title>
</head>
<body>
$page.content$
</body>
</html>
3 changes: 3 additions & 0 deletions src/sbt-test/ghpages/site/src/main/hello.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Hello

Hello World!
32 changes: 32 additions & 0 deletions src/sbt-test/ghpages/site/test
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#######################################################################################
# Test that site is generated in version number directory (ghpagesKeepVersions = true)
#######################################################################################
> 'set version := "1.0.0"'
> ghpagesSynchLocal

$ must-mirror target/ghpages/1.0.0/hello.html expected/hello.html
# Verify files does not get generated at top level
-$ must-mirror target/ghpages/hello.html expected/hello.html

> 'set version := "2.0.0"'
> ghpagesSynchLocal

# Verify files get generated in 2.0.0 folder
$ must-mirror target/ghpages/2.0.0/hello.html expected/hello.html

# Verify existing versions are preserved
$ must-mirror target/ghpages/1.0.0/hello.html expected/hello.html

#######################################################################################
# Test that site is generated in top level directory (ghpagesKeepVersions = false)
#######################################################################################
> 'set version := "3.0.0"'
> 'set ghpagesKeepVersions := false'
> ghpagesSynchLocal

# Verify files get generated in top level folder when ghpagesKeepVersions flag is false
$ must-mirror target/ghpages/hello.html expected/hello.html

# Verify existing versions are preserved
$ must-mirror target/ghpages/1.0.0/hello.html expected/hello.html
$ must-mirror target/ghpages/2.0.0/hello.html expected/hello.html