Skip to content

Notes on migrating to a Go mono repo

Jump to bottom
Paul Jolly edited this page Jul 2, 2018 · 31 revisions

Ultimately this wiki will become a blog post, but maintaining here for now until things settle.

Goals

  1. Ultimately to migrate all myitcv.io/... packages to a mono-repo, housed at https://github.com/myitcv/x
  2. This mono-repo should support vgo and, critically, be backwards compatible with "old" go get
  3. https://godoc.org/ should work as expected
  4. Have multiple modules within the mono-repo, myitcv.io/react being one of them. This will mean that we have submodules within the outer myitcv.io module (although it's not strictly required to have a root module, I will use it as a catch-all for any packages below myitcv.io/ that are not submodules)
  5. Drop any existing _vendor directories that were previously used to "cache" (dev) dependencies, e.g. myitcv.io/react/_vendor
  6. categorise issues/PRs in much the same way as the Go project does, e.g. myitcv.io/react issues would be prefixed by react, react/cmd/reactGen etc

Pros

  • Less overhead in terms of maintaining multiple repos, issue lists, CI setups etc
  • ...

Cons

  • It's a change from where we are today; loss of continuity, history etc (although the history of the old repos will remain intact in the original repo)
  • ...

Notes

  • The custom import paths for all myitcv.io/... packages will continue to be driven by https://myitcv.io, which is an extremely basic (and hacky) Google App Engine Go App (source code https://github.com/myitcv/myitcv.io). Once everything is migrated to the mono-repo we can remove the explicit package listing and tidy things up considerably
  • The go-import git meta tag for all myitcv.io/... packages needs to be the root of all packages, myitcv.io in this case:
$ curl https://myitcv.io/gogenerate?go-get=1

<!DOCTYPE html>
<html>
<head>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
        <meta name="go-import" content="myitcv.io git https://github.com/myitcv/x">
        <meta name="go-source" content="myitcv.io https://github.com/myitcv/x/wiki https://github.com/myitcv/x/tree/master{/dir} https://github.com/myitcv/x/blob/master{/dir}/{file}#L{line}">
        <meta http-equiv="refresh" content="0; url=https://godoc.org/myitcv.io/gogenerate">
</head>
<body>
Redirecting to docs at <a href="https://godoc.org/myitcv.io/gogenerate">godoc.org/myitcv.io/gogenerate</a>...
</body>
</html>
$ curl https://myitcv.io/blah2?go-get=1

<!DOCTYPE html>
<html>
<head>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
        <meta name="go-import" content="myitcv.io git https://github.com/myitcv/x">
        <meta name="go-import" content="myitcv.io/blah2 mod https://raw.githubusercontent.com/myitcv/pubx/master">
        <meta name="go-source" content="myitcv.io https://github.com/myitcv/x/wiki https://github.com/myitcv/x/tree/master{/dir} https://github.com/myitcv/x/blob/master{/dir}/{file}#L{line}">
        <meta http-equiv="refresh" content="0; url=https://godoc.org/myitcv.io/blah2">
</head>
<body>
Redirecting to docs at <a href="https://godoc.org/myitcv.io/blah2">godoc.org/myitcv.io/blah2</a>...
</body>
</html>
  • It appears however that the old go get does not like there being two go-import meta tags which satisfy a myitcv.io/... package. Raised as https://github.com/golang/go/issues/24751
  • The issue with the dual-meta tags also affects https://godoc.org/, because it (https://godoc.org/) uses go get in order to access code. Otherwise, however, things appear to work well: https://godoc.org/myitcv.io/blah. This has been raised as https://github.com/golang/gddo/issues/558
  • It seems my approach to migrating package-by-package is somewhat flawed. The "issue" is me creating a mono-repo that has as its effective import path myitcv.io. Because unless I move everything over in one go, then vgo or go both have to handle a situation where there is one repo that is rooted at myitcv.io and others that are rooted one level deeper. And vgo|go understandably can't handle that
    • This actually has the consequence that a package in the mono-repo that depends on another package in the mono-repo has to do so via a replace directive until the "flip" is complete. e.g. immutable
    • Also, once the "flip" is complete I will need to drop the exceptions from this .gitignore, thereby ignoring everything myitcv.io/...
  • vgo test ./... at the root modifies the root go.mod; related to https://github.com/golang/go/issues/26048
  • vgo install pkg installs to GOBIN using the module version suffix; https://github.com/golang/go/issues/24667

CI

Issues/vgo enhancements/proposals

All issues listed above have been raised in the Go repo

In addition, I link here to a number of interesting issues/proposals:

Open questions

TODO

Clone this wiki locally