Memory Grow Callback

[samyfodil]

When used in a Serverless environment wazero needs to provide a mean to limit memory used by a module and its dependencies. A Call back before grow is executed would help implement this and many more use cases like debugging.

[codefromthecrypt]

Thanks for raising this! TL;DR; we have an incomplete "memory" log scope which needs this anyway (in the experimental interceptor).

cc @ckaznocha @achille-roussel @inkeliz @danielpacak who might be interested, but I'm sure also interested in zero overhead unless active.

There have been many issues around this #417 #483 #500. Today, some recompile with flags (not every compiler has one), use wabin to rewrite the wasm with a max, or assign a different max with RuntimeConfig. However, none of these approaches are hooks. That said, emscripten and GOOS=js already insert memory hook host functions.

I think the best way forward is to finish the interceptor stuff, and augment the "memory" log scope to include this. This will prove it is interceptable. However, the devil is in the details as "memory.grow" is actually a host function, but it isn't invoked by "call" or "call_indirect", rather well.. "memory.grow". There's some interesting work that would need to complete to make built-in host functions interceptable. That way, you can keep zero overhead, and also change the input or result like any other function.

Unless someone contributes, I doubt this will get into 1.0, however, if someone with an appetite for internal refactoring plus feedback cycles can start on this, it could. We won't ever consider "experimental" packages, which include the interceptors at the moment, a part of any compatibility promise. If no one volunteers until after 1.0 I think someone in the project team, possibly me, can take this on.

some notes in general.

The memory.grow op returns a bool, which when false the guest interprets in a compiler-specific way to produce an error like "out of memory", so for example, I wouldn't suggest anyone panic in a hook as it prevents the guest from knowing a better message.

It is true that if there was a callback, you can do host logic, and while not sure precisely what debugging might mean (maybe you can elaborate) logging could be one form of that.

In fact we are overdue adding a logging hook for memory.grow, (using the existing interceptor) and the impl would allow the same as you mention. I think that could be a lightweight way to start this.

For the interceptor, I think the easiest way out is to make "memory.grow" a pseudo host function. In fact it is a host function, just it isn't in the function table. I've thought about that in the past, but it takes some thinking to do it right. The jist of it would be to make a pseudo-functions for "memory.grow" "table.grow" (there are a couple more I think). This way the interceptors can keep zero overhead when nothing is being intercepted.

[mathetake]

#2177