wazero v1.0.0-rc.1

wazero v1.0.0-rc.1 starts our journey towards 1.0, with no public API changes. We will have at least one more release candidate between now and 1.0, in a week or two. The next candidate will include CLI binaries, which will allow our non-Go community to use wazero without installing Go, first.

wazero 1.0 will happen March 24th, during the wazero release party at wasmio hosted by Tetrate. Follow Edoardo for updates on on the party, who's doing great job organizing including lightning talks by end users.

Many of you have requested a community page on our website. We added this including a list of users who explicitly opted in. The user list is opt-in, and generally higher signal than the "Used By" list on the GitHub site, as the latter includes transient dependencies.

The biggest internal code change in v1.0.0-rc.1 let to wazero passing the entire wasi-testsuite on linux, darwin (macOS) and windows, with zero exceptions. To do so took tens of days of effort from both @codefromthecrypt and @mathetake, and described at the bottom for the curious. By passing every test defined by WebAssembly, as well stdlib tests in TinyGo and Zig, it is easier for end users to feel confident wazero is a great choice for stability. This is worth the couple weeks of pain.

We intentionally didn't do any other large changes, but there were several people to thank for minor changes. Some of the below took many days of effort each!

• @achille-roussel for refactoring an internal type with generics so we can use it for open directories.
• @ckaznocha for fixing a concurrency issue on context cancellation.
• @mathetake for a mountain of optimizations to reduce compilation overhead
• @dmvolod for removing a redundant wasm decoding validation.
• @evacchi for adding -timeout duration to the CLI which stops runaway processes.
• @evacchi for fixing a corner case around max memory limit
• @mathetake for allowing the CLI to be built with an external version (for packaging)

In closing, thanks very much for sticking with us this last year and a half almost, leading to 1.0. Our fantastic ecosystem is the reason you have a zero dependency runtime, something you can embed without thinking about work or version clashes with other tools. We've done great work together to bring Go forward in the WebAssembly ecosystem, and people are noticing!

Notes on wasi-testsuite compliance

Most people don't need to read this part, so only do if you are interested in low level details, or bored!

wasi-testsuite compliance means passing tests that verify the expected behaviors of wasi_snapshot_preview1, as decided by the WASI W3C subgroup. It does so by running tests compiled in different languages, currently AssemblyScript, C and Rust.

As described at the beginning of the release notes, wazero v1.0.0-rc.1 passes all tests as doing so is least confusing to the ecosystem. This means passing things we don't advise or agree with. Our former release left out a couple tests due to performance overhead, but we pass them now despite it. If you care about this, read on!

The last pre-release skipped a test on dot and dot-dot entries in directory listings. Go throws away these entries before we can read them. To resurrect them costs tens of microseconds. The cost is fixed overhead around inodes, but this same topic has a large performance footprint when multiplied in directory listings.

Recently, a test was added to require inode data (file serial numbers) inside all directory listings. For example, if your directory includes "wazero.txt", a non-zero inode must be returned

inode data isn't typically used except comparing file identity, and even then requires the device to do so properly. For example, Go has a SameFile function which lazily gets this data, as it is well.. expensive. wasi-libc made a change recently to fetch this data regardless of it it is used or not. Not all compilers use wasi-libc, for example Zig has their own directory logic. However, at least C and Rust do, so mitigating this problem became an issue for us, not just for the spectest, but the underlying logic in wasi-libc. Specifically, compilers that use wasi-libc 17+ will perform a guest side stat fan-out when the ino returned is zero. Doing a guest-side fan out is much more expensive than host side.

This problem is roughly analogous to ReadDir vs Readdir problem in Go. The lower-d version says "Most clients are better served by the more efficient ReadDir method", ultimately due to an internal stat fanout in worst case. Unfortunately, the performance impact is worse than upper vs lower d readdir in go. In windows, the inode information isn't in the FileInfo.Sys data, so there's an additional N syscalls to get it from somewhere else.

System performance isn't consistent, but compliance with WASI on this is at least in tens of microseconds additional overhead on directory listings, and it is also linear wrt directory size. Not all users will list directories at after configuration time, neither will all find this overhead intolerable even if they did.

In case you are curious, we did discuss these topics at length with the WASI maintainers. The end of it is that this performance overhead is something they feel as a least bad option of choices before them. That said, progress was made, as they changed the next specification, wasi-filesystem to be performant by default, both not requiring dot directories or eager inodes. We're grateful to have been listened to.

If your performance after initialization time is dominated by this, you may of course file an issue to request a non-strict wasi setting. That said, we'd prefer to not make that setting. We have near term plans, likely this summer, to make the whole filesystem behaviour pluggable. In other words, avoiding this cost will be something you can choose to do on your own later, even for the current WASI specs.

If you'd like to discuss more on this or anything else, jump on gophers slack #wazero channel. Note: You may need an invite to join gophers. If you like what we are doing, please star our repo as folks appreciate it.