Update the spec and roadmap and link to them from the README

tarides · May 30, 2022 · b1169a7 · b1169a7
1 parent 4be4a74
commit b1169a7
Show file tree

Hide file tree

Showing 3 changed files with 268 additions and 143 deletions.
diff --git a/README.md b/README.md
@@ -6,20 +6,20 @@ The OCaml Platform represents the best way for developers, both new and old, to
 
 This repository contains the `ocaml-platform` tool. This tool allows to easily install all the projects of the Platform in a switch and aims at offering a unified workflow to work with the different Platform tools.
 
-## Trying the platform
+## Trying the Platform
 
 Just download and execute the [install script](https://github.com/tarides/ocaml-platform/releases/download/0.0.1-alpha/installer.sh) of the latest release.
 
 The install script simply downloads and installs the suitable version of `ocaml-platform` for your system, as well as `opam` if needed.
 
-You are then able to run `ocaml-platform`. It will install the platform tools in the current switch. Note that the first time the command is run for a given version of ocaml, installing the tools for the switch takes a few minutes.
+You are then able to run `ocaml-platform`. It will install the Platform tools in the current switch. Note that the first time the command is run for a given version of OCaml, installing the tools for the switch takes a few minutes.
 
 ### The advantages of using `ocaml-platform`
 
 The advantages are the following:
 
-- The dependencies of the platform tools, which are only needed for the development, are not mixed with the dependencies of your project.
-- Installing the platform tools is very fast if you have already installed them for the same version of OCaml in another switch.
+- The dependencies of the Platform tools, which are only needed for the development, are not mixed with the dependencies of your project.
+- Installing the Platform tools is very fast if you have already installed them for the same version of OCaml in another switch.
 
 ## Status
 
@@ -33,7 +33,7 @@ The following Platform tools are currently distributed in the OCaml Platform:
 - LSP server: [`ocaml-lsp`](https://github.com/ocaml/ocaml-lsp)
 - Editor helper: [`merlin`](https://github.com/ocaml/merlin)
 
-Note that the following is not yet distributed but is still in the platform:
+Note that the following is not yet distributed but is still in the Platform:
 
 - REPL: [`utop`](https://github.com/ocaml/utop)
 
@@ -46,3 +46,10 @@ git clone [email protected]:tarides/ocaml-platform.git
 ```
 
 To run the test, see the [README](https://github.com/tarides/ocaml-platform/blob/main/test/README.md).
+
+## Roadmap
+
+You can read a high-level specification for the OCaml Platform [here](./doc/spec.md).
+
+Our current plan is to release an installer for the Platform tools in sync with OCaml 5.
+You can read more about this [here](./doc/roadmap-q2.md).
diff --git a/doc/roadmap-q2.md b/doc/roadmap-q2.md
@@ -0,0 +1,176 @@
+# OCaml Platform Q2 Roadmap
+
+## 1. Define the Platform governance and contents
+
+The first step of building the platform is to define what it is. We will start by documenting the OCaml 5.0 Platform and will involve the community very early: 
+
+1. What are the recommended workflows to get productive in OCaml? This guide will be created and (automatically) maintained on OCaml.org, and will be the recommended way to work with OCaml;
+2. What tools are part of the platform, and what is the governance of the project? Especially, the platform will define how the existing tools maintainers collaborate and communicate together to converge to a more consistent user experience, how the community can propose a new project, etc.
+
+## 2. Release the Platform Installer
+
+Properly integrating all the Platform tools and designing a new CLI is a long-term goal. In particular, this requires solving technical challenges as the tools haven't been built to be integrated. An alternative CLI for the Platform is also a risky endeavor and will require thoughtful design to ensure that we have a user-friendly upgrade story.
+
+Hence, we want to build and distribute the platform, incrementally, and slowly evolve the existing tools to work more consistently together.
+
+We have been exploring various ways to integrate the Platform tools, and the design of such a CLI and have decided to roll out the Platform into three phases that each solve a different aspect of the issues listed previously:
+
+- **Release of an installer**: an installer is distributed as a pre-built binary on all supported systems. It installs the Platform tools in the current switch by either compiling them from sources in a separate switch, or by embedding pre-built versions of the tools.
+- **Integrate the Platform in the installer**: instead of installing the tools, we gradually move to a fat-binary that exposes the tools CLIs using the projects libraries.
+- **Improved CLI and user experience**: once the tools are well integrated, we can solve the user experience issues by building an alternative, improved CLI that will drive the different tools. We will focus on exposing the right metadata in the user's project instead of hiding it in each of the tools' states.
+
+The immediate focus is the first phase of this roadmap: releasing an installer for the Platform. We will build the required infrastructure to support the release of the installer (i.e. a CI system that tests and produces a pre-built installer on all supported platforms).
+
+The following phases are yet to be fully spec-ed out and the overall roadmap is subject to change, especially as we learn from our work on the installer and receive user feedback along the way.
+
+The next part of the document goes into the detail of the deliverables for the installer, coined "OCaml 5.0 Platform".
+
+### Installer
+
+The installer starts with checking if opam is available on the user's system.
+
+
+```flow
+st=>start: Start
+e=>end: End
+
+opam_detected=>condition: Opam binary
+detected?
+
+install_opam=>operation: Install opam
+init_opam=>operation: Init opam
+
+st->opam_detected
+opam_detected(yes)->e
+opam_detected(no)->install_opam->init_opam
+
+opam_detected->opam_installed
+```
+
+When we checked that opam is installed and initialized, we go on to create an opam switch to install the other Platform tools.
+
+### Tool Selection
+
+The OCaml Platform tools are defined by the "Active" and "Incubate" projects listed [here](https://ocaml.org/platform/). Each element of the platform lives at a different point in the lifecycle of being a Platform tool. Some "Incubate" projects might duplicate features that are provided by "Active" projects. The aim of the platform is to limit these duplications as much as possible and to document the one blessed way to be productive in OCaml.
+
+To be more specific, for the first iteration of the OCaml Platform we are considering the following tools (to be revisited later, when the policies for governing how projects can go in and out of the Platform are ready):
+
+- Opam - Package manager
+- Dune - Build system
+- Utop - OCaml toplevel
+- Merlin - Syntax completion
+- Odoc - Documentation generator
+- Mdx - Markdown code block execution
+- LSP - OCaml's Language server protocol
+- OCamlformat - OCaml formatter
+- Dune-release - CLI to release packages on Opam
+
+We are intentionally omitting Platform tools such as `ppxlib` as they are to be installed as a library with Opam, and are less impacted by the problems we are discussing here.
+
+From this list, we classify the tools into different categories
+
+|              | Can be outside of switch | Doesn't depend on project | Doesn't depend on compiler | Can install from binaries |
+| ------------ | ------------------------ | ------------------------- | -------------------------- | ------------------------- |
+| Opam         | ✅                        | ✅                         | ✅                          | ✅                         |
+| Dune         | ✅                        | ❌ [^1]                    | ✅                          | ❌                         |
+| Utop         | ✅                        | ✅                         | ✅                          | ❌                         |
+| Merlin       | ✅                        | ✅                         | ❌                          | ❌                         |
+| Odoc         | ✅                        | ✅                         | ❌                          | ❌                         |
+| Mdx          | ✅                        | ✅                         | ✅                          | ❌                         |
+| LSP          | ✅                        | ✅                         | ❌                          | ❌                         |
+| OCamlformat  | ✅                        | ❌                         | ❌ [^2]                     | ❌                         |
+| Dune-release | ✅                        | ✅                         | ✅                          | ❌                         |
+
+[^1]: Dune depends on the user's project but is backward compatible, so it can be considered project-independent if using the latest version
+
+[^2]: OCamlformat can be compiled with the latest version of the compiler
+
+### Possible Directory Layout
+
+```
+.opam/tools/
+         / dune
+            / 2.8 
+              / dune
+         / opam
+            / 2.0.7
+              / opam
+            / 2.1.0
+              / opam
+.opam/4.10/
+         / odoc
+            / 2.0.0
+              / odoc
+         / lsp-server
+            / 1.8.0
+              / ocaml-lsp-server
+         / ocamlformat
+           /0.15.0
+              / ocamlformat
+           /0.14.0
+              / ocamlformat
+_opam/bin
+   symlinks to above ^^^
+
+```
+
+With this classification, we can decide how to install each specific tool:
+
+- **Opam**: Download the latest binary, and cache them.
+- **Dune**: Build the latest version in a switch with the latest compiler and cache them.
+- **Utop**: Install from sources in the switch
+- **Merlin**: Detect the switch compiler, build in a sandbox with the same compiler and cache them.
+- **Odoc**: Detect the switch compiler, build in a sandbox with the same compiler and cache them.
+- **Mdx**: Build in a switch with the latest compiler and cache them.
+- **LSP**: Detect the switch compiler, build in a sandbox with the same compiler and cache them.
+- **OCamlformat**: Detect the project's version, detect the compiler version, install the correct version in a switch with the same compiler and cache them.
+- **Dune-release**: Build in a switch with the latest compiler and cache them.
+
+The Platform tools are intended to be used as binaries, and users should not need their sources (with the exception of `utop` possibly[^3]). So we can cache the compiled binaries and re-use them. If the correct version of the binary can be found, it is copied to the user's opam switch. If not, they are compiled or downloaded and copied thereafter.
+
+[^3]: The user requires the sources of Utop because dune compiles it when running `dune utop`. Not sure if that's something we want to support? The REPL integration in VSCode could load the libraries themselves.
+
+Note that the installation should be idempotent: if the current switch is already set up, we don't need to do anything.
+This property is important for the execution of the tool (see below).
+
+### Execution
+
+If we keep the binaries outside of the switch:
+
+- How do we allow the user to select which version to use?
+
+If we copy the binaries in the switch:
+
+- How do we follow up with the project update? (e.g. update dune version, update ocamlformat)
+
+Solution: we make sure the tools are installed every time we execute them.
+
+This workflow will be used by VSCode, but let's illustrate it with some command lines:
+
+```sh
+ocaml-platform opam # Uses the latest opam outside of the user's switch
+ocaml-platform utop # Uses the utop installed in the switch
+ocaml-platform merlin # Uses the merlin binary copied in the switch
+ocaml-platform ocamlformat # Checks the project's config and install the correct version
+ocaml-platform dune # Uses the merlin binary copied in the switch
+```
+
+An execution of a tool will trigger an installation confirmation if it is not installed, so this fits perfectly with VSCode's workflow.
+
+## VSCode integration
+
+We will send patches to `vscode-ocaml-platform` to remove the need to manually install the OCaml Platform.
+Once the extension is installed, VSCode will take care of downloading the installer and driving it.
+
+## CI
+
+The CI is responsible to run end-to-end tests for the supported systems and creating the pre-built binaries for the installer itself.
+
+## Documentation
+
+As part of the release, we will be documenting:
+
+- An updated [Up and Running](https://ocaml.org/docs/up-and-running) guide to using the OCaml Platform installer to get started.
+- An updated [State of the Platform](https://ocaml.org/docs/platform)
+- The developer workflows are supported by the Platform. An initial draft is available at https://ocaml.org/docs/best-practices
+- The governance of the Platform and the processes to be incubated, promoted and demoted.