Skip to content

Commit

Permalink
docs: Add proto v0.14 blog post. (#994)
Browse files Browse the repository at this point in the history
* First pass.

* Update examples.

* Add blog post.

* Add new section.
  • Loading branch information
milesj committed Aug 14, 2023
1 parent 9d4c5b1 commit 994da85
Show file tree
Hide file tree
Showing 6 changed files with 127 additions and 38 deletions.
2 changes: 1 addition & 1 deletion proto-plugin.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# A TOML plugin for proto:
# https://moonrepo.dev/docs/proto/plugins#toml-plugin
# https://moonrepo.dev/docs/proto/toml-plugin

name = "moon"
type = "cli"
Expand Down
73 changes: 73 additions & 0 deletions website/blog/2023-08-11_proto-v0.14.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
---
slug: proto-v0.14
title: proto v0.14 - Node.js and Rust now powered by WASM plugins
authors: [milesj]
tags: [proto, wasm, plugin]
---

With this release, we've finished our migration to a WASM based plugin architecture.

<!--truncate-->

## Full migration to WASM

In [our last release](./proto-v0.13), we migrated the Bun, Deno, and Go plugins to WASM. In this
release, we're excited to announce that we have completed our WASM migration by also converting
Node.js (and npm, etc) and Rust to WASM plugins. We've also updated our TOML plugin layer to use
WASM under the hood.

With this, 100% of our tools are now powered by WASM, and the APIs are stable enough for you, the
consumers, to build your own plugins! We have plans for additional APIs in the next release,
primarily installing and uninstall globals, and a hooks system, so look forward to that! To learn
more about our official plugins, check out the their repositories!

- [Bun](https://github.com/moonrepo/bun-plugin)
- [Deno](https://github.com/moonrepo/deno-plugin)
- [Go](https://github.com/moonrepo/go-plugin)
- [Node](https://github.com/moonrepo/node-plugin)
- [Rust](https://github.com/moonrepo/rust-plugin)
- [Schema](https://github.com/moonrepo/schema-plugin)

This release may seem light, but we
[rewrote the entire core](https://github.com/moonrepo/proto/pull/161) from the ground up to support
a pure plugin based architecture. This was a lot of work, but our implementation is far easier to
work with, and more robust. As part of this process, we improved version and range handling, archive
unpacking, and much more!

> Because of the rewrite, we may have introduced a few bugs. Please report any issues!

## Breaking changes for TOML plugins

In preparation for the next release, and as clean up during the rewrite, we've made a few breaking
changes to the TOML schema, primarily around global handling. To start, this old code:

```toml
[install]
globals-dir = ["$PROTOSTAR_HOME/bin", "$HOME/.protostar/bin"]
global-args = ["install", "--force", "{dependency}"]
```

Can now be rewritten as:

```toml
[globals]
lookup-dirs = ["$PROTOSTAR_HOME/bin", "$HOME/.protostar/bin"]
install-args = ["install", "--force", "{dependency}"]
```

And lastly, the `install.unpack` setting has been removed. We believe our installation layer is
smart enough to unpack or move files automatically.

## Other changes

View the [official release](https://github.com/moonrepo/proto/releases/tag/v0.14.0) for a full list
of changes.

- Versions defined in `.prototools` must be fully-qualified semantic versions. Partial versions
(missing patch or minor) are no longer supported. This may change in the future based on feedback.
- Tool and plugin names must now be in kebab-case.
- Node
- Yarn 2+ is now installed using `@yarnpkg/cli-dist`. We no longer downgrade to the latest v1.
- Will no longer detect a version from `engines` in `package.json`.
- Rust
- Will now detect a version/channel from `rust-toolchain.toml` and `rust-toolchain`.
9 changes: 5 additions & 4 deletions website/docs/proto/config.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,14 @@ proto supports pinning versions of tools on a per-project or per-directory basis
`.prototools` configuration file. This file takes precedence during [version detection](./detection)
and can be created/updated with [`proto local`](./commands/local).

This configuration simply maps tools to semantic versions for the current directory.
This configuration simply maps tools to fully-qualified semantic versions for the current directory.
Partial versions are not supported.

```toml title=".prototools"
node = "16.16.0"
npm = "9.0.0"
go = "1.20"
rust = "1.68"
go = "1.20.0"
rust = "1.68.0"
```

> The names of tools are the same used on the command line!
Expand Down Expand Up @@ -82,6 +83,6 @@ jobs:
runs-on: 'ubuntu-latest'
steps:
- uses: 'actions/checkout@v3'
- uses: 'moonrepo/setup-proto@v0'
- uses: 'moonrepo/setup-proto@v1'
- run: 'proto use'
```
24 changes: 14 additions & 10 deletions website/docs/proto/toml-plugin.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ are available:
- `{arch}` - The architecture of the host machine, like `x86_64`. These values map to Rust's
[`ARCH` constant](https://doc.rust-lang.org/std/env/consts/constant.ARCH.html), but can be
customized with [`install.arch`](#downloading-and-installing).
- `{os}` - The operating system of the host machine, like `windows`. These values map to Rust's
[`OS` constant](https://doc.rust-lang.org/std/env/consts/constant.OS.html).
- `{libc}` - For Linux machines, this is the current libc implementation, either `gnu` or `musl`.

### Downloading and installing
Expand All @@ -85,8 +87,6 @@ be downloaded and installed into the toolchain. The following settings are avail
- `checksum-url` - A secure URL to download the checksum file for verification. If the tool does not
support checksum verification, this setting can be omitted.
- `download-url` (required) - A secure URL to download the tool/archive.
- `unpack` - Whether the downloaded file should be unpacked or not. For tools that download the
binary directly, this should be disabled.

The URL settings support `{checksum_file}` and `{download_file}` tokens, which will be replaced with
the values from the `[platform]` section.
Expand All @@ -103,21 +103,25 @@ aarch64 = "arm64"
x86_64 = "x64"
```

The `[install]` sections also supports the following settings that dictate the installation of
global binaries for the tool. If the tool does not support globals (is not a programming language),
these settings can be omitted.
#### Global packages

- `globals-dir` - A list of directories to locate and install the global binaries into. This setting
The `[globals]` sections can be configured that dictates the installation of global binaries for the
tool. If the tool does not support globals (is not a programming language), these settings can be
omitted.

- `lookup-dirs` - A list of directories to locate and install the global binaries into. This setting
supports interpolating environment variables via the syntax `$ENV_VAR`.
- `global-args` - A list of CLI arguments to pass to the tool to install a global binary. The name
- `install-args` - A list of CLI arguments to pass to the tool to install a global binary. The name
of the binary can be inserted using the `{dependency}` token.
- `package-prefix` - A string that all package names are prefixed with. The prefix will be removed
when filtering/listing packages. For example, Cargo/Rust binaries are prefixed wih `cargo-`.

```toml title="protostar.toml"
# ...

[install]
globals-dir = ["$PROTOSTAR_HOME/bin", "$HOME/.protostar/bin"]
global-args = ["install", "--force", "{dependency}"]
[globals]
lookup-dirs = ["$PROTOSTAR_HOME/bin", "$HOME/.protostar/bin"]
install-args = ["install", "--force", "{dependency}"]
```

### Resolving versions
Expand Down
17 changes: 6 additions & 11 deletions website/docs/proto/tools.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ $ proto install go -- --no-gobin

## Node.js

<WasmLink to="https://github.com/moonrepo/node-plugin" />

[Node.js](https://nodejs.org/) (`node`) is a JavaScript runtime built on Chrome's V8 engine.

#### Post-install hook
Expand All @@ -80,18 +82,11 @@ $ proto install node -- --no-bundled-npm

- An [`.nvmrc`](https://github.com/nvm-sh/nvm) or
[`.node-version`](https://github.com/nodenv/nodenv) file.
- The [`engines`](https://docs.npmjs.com/cli/v9/configuring-npm/package-json#engines) field in
`package.json`.

:::info

When parsing engines, explicit versions (`1.2.3`, `=1.2.3`) will be used as-is, but
requirements/ranges (`^1.2.3`, `>=1.2.3`) will scan `~/.proto/tools/node` for an applicable match.

:::

### npm, pnpm, yarn

<WasmLink to="https://github.com/moonrepo/node-plugin" />

proto supports all popular Node.js package managers: `npm`, `pnpm`, `yarn`.

#### Global binaries
Expand All @@ -103,8 +98,6 @@ proto supports all popular Node.js package managers: `npm`, `pnpm`, `yarn`.
#### Detection sources

- The `packageManager` field in `package.json`.
- The [`engines`](https://docs.npmjs.com/cli/v9/configuring-npm/package-json#engines) field in
`package.json` (same notice above applies).

### node-gyp

Expand All @@ -117,6 +110,8 @@ which is a far more poweful alternative.

## Rust

<WasmLink to="https://github.com/moonrepo/rust-plugin" />

[Rust](https://www.rust-lang.org/) (`rust`) is blazingly fast and memory-efficient systems language.

:::caution
Expand Down
40 changes: 28 additions & 12 deletions website/docs/proto/wasm-plugin.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,7 @@ we'll use [Deno](https://deno.land/[email protected]/tools/script_installer).
#[plugin_fn]
pub fn locate_bins(Json(_): Json<LocateBinsInput>) -> FnResult<Json<LocateBinsOutput>> {
Ok(Json(LocateBinsOutput {
fallback_last_globals_dir: true,
globals_lookup_dirs: vec!["$DENO_INSTALL_ROOT/bin".into(), "$HOME/.deno/bin".into()],
..LocateBinsOutput::default()
}))
Expand All @@ -224,7 +225,7 @@ versions to actually be installed. To provide a list of versions and language sp
#[plugin_fn]
pub fn load_versions(Json(_): Json<LoadVersionsInput>) -> FnResult<Json<LoadVersionsOutput>> {
let mut output = LoadVersionsOutput::default();
let response: Vec<NodeDistVersion> = fetch_url("https://nodejs.org/dist/index.json")?;
let response: Vec<NodeDistVersion> = fetch_url_with_cache("https://nodejs.org/dist/index.json")?;

for (index, item) in response.iter().enumerate() {
let version = Version::parse(&item.version[1..])?; // Starts with v
Expand Down Expand Up @@ -323,19 +324,21 @@ shims to be created. By default, proto _will always_ create a global shim of the

Global shims are optional scripts that are available on `PATH` as they are generated in the
`~/.proto/bin` directory, and can be defined with the `global_shims` parameter. This param is a hash
map, where the key is the shim file name, and the value is a relative path (from the tool's
directory) to a binary to execute.
map, where the key is the shim file name, and the value is a configuration object.

```rust
#[plugin_fn]
pub fn create_shims(Json(input): Json<CreateShimsInput>) -> FnResult<Json<CreateShimsOutput>> {
let mut global_shims = HashMap::new();

global_shims.insert("npx".into(), if input.env.os == HostOS::Windows {
"npx.cmd".into()
} else {
"bin/npx".into()
});
global_shims.insert(
"npx".into(),
ShimConfig::global_with_alt_bin(if input.env.os == HostOS::Windows {
"npx.cmd".into()
} else {
"bin/npx".into()
}),
);

Ok(Json(CreateShimsOutput {
global_shims,
Expand All @@ -344,6 +347,19 @@ pub fn create_shims(Json(input): Json<CreateShimsInput>) -> FnResult<Json<Create
}
```

Furthermore, if you'd prefer to _not_ create a primary global shim at all, enable the
`no_primary_global` field.

```rust
#[plugin_fn]
pub fn create_shims(Json(input): Json<CreateShimsInput>) -> FnResult<Json<CreateShimsOutput>> {
Ok(Json(CreateShimsOutput {
no_primary_global: true,
..CreateShimsOutput::default()
}))
}
```

## Testing

The best way to test the plugin is to execute it through `proto` directly. To do this, you'll need
Expand Down Expand Up @@ -394,8 +410,7 @@ output is correct. This can be achieved by creating a plugin instance with `crea
calling the appropriate method.

```rust
use proto_pdk::*;
use proto_pdk_test_utils::create_plugin;
use proto_pdk_test_utils::*;
use starbase_sandbox::create_empty_sandbox;

#[test]
Expand Down Expand Up @@ -430,8 +445,7 @@ To test downloading and installing, use `generate_download_install_tests!`. This
plugin ID and a real version to test with.

```rust
use proto_pdk::*;
use proto_pdk_test_utils::{create_plugin, generate_download_install_tests};
use proto_pdk_test_utils::*;
use starbase_sandbox::create_empty_sandbox;

generate_download_install_tests!("id", "1.2.3");
Expand Down Expand Up @@ -523,6 +537,8 @@ Some helpful resources for learning about and building plugins.
- [Go](https://github.com/moonrepo/go-plugin)
- [Node](https://github.com/moonrepo/node-plugin)
- [npm, pnpm, yarn](https://github.com/moonrepo/node-plugin)
- [Rust](https://github.com/moonrepo/rust-plugin)
- [Schema](https://github.com/moonrepo/schema-plugin)
- Plugin development kit
- [`proto_pdk` docs](https://docs.rs/proto_pdk/)
- [`proto_pdk_test_utils` docs](https://docs.rs/proto_pdk_test_utils/)

0 comments on commit 994da85

Please sign in to comment.