vmx

2021-01-29 15:00

The goal was to run some WebAssembly within different host languages. I needed a WASM file that is independent of the host language, hence I decided to code the FFI manually, without using any tooling like wasm-bindgen, which is JavaScript specific. It needed a bit of custom tooling, but in the end I succeeded in having a WASM binary that has a multi-value return, generated with today's Rust compiler, without using wasm-bindgen annotations.

Introduction

In my case I wanted to pass some bytes into the WASM module, do some processing and returning some other bytes. I found all information I needed in this excellent A practical guide to WebAssembly memory from radu. There he mentions the WebAssembly multi-value proposal and links to a blog post from 2019 called Multi-Value All The Wasm! which explains its implementation for the Rust ecosystem.

As it's from 2019 I just went ahead and thought I can use multi-value returns in Rust.

The journey

My function signature for the FFI looks like this:

pub extern "C" fn decode(data_ptr: *const u8, data_len: usize) -> (*const u8, usize) { … }

When I compiled it, I got this warning:

warning: `extern` fn uses type `(*const u8, usize)`, which is not FFI-safe
 --> src/lib.rs:2:67
  |
2 | pub extern "C" fn decode(data_ptr: *const u8, data_len: usize) -> (*const u8, usize) {
  |                                                                   ^^^^^^^^^^^^^^^^^^ not FFI-safe
  |
  = note: `#[warn(improper_ctypes_definitions)]` on by default
  = help: consider using a struct instead
  = note: tuples have unspecified layout

Multi-value returns are certainly not meant for C APIs, but for WASM it might still work, I thought. Running wasm2wat shows:

(module
  (type (;0;) (func (param i32 i32 i32)))
  (func $decode (type 0) (param i32 i32 i32)
…

This clearly isn't a multi-value return. It doesn't even have a return at all, it takes 3 parameters, instead of the 2 the function definition has. I found an issue called Multi value Wasm compilation #73755 and was puzzled why it doesn't work. Is this a regression? Why did it work in that blog post from 2019? I gave the Multi-Value All The Wasm! blog post another read, and it turns out it explains all this in detail (look at the wasm-bindgen section). Back then it wasn't supported by the Rust compiler directly, but by wasm-bindgen.

So perhaps I can just use the wasm-bindgen command line tool and transform my compiled WASM binary into a multi-value return one. There is a command-line flag called WASM_BINDGEN_MULTI_VALUE=1 to enable that transformation. Sadly that doesn't really work as it needs some interface-types present in the WASM binary (which I don't have).

Thanks to open source, the blog post about the implementation of the transformation feature and some trial an error, I was able to extract the pieces I needed and created a tool called wasm-multi-value-reverse-polyfill. I didn't need to do any of the hard parts, just some wiring up. I was now able to transform my WASM binary into a multi-value return one simply by running:

$ multi-value-reverse-polyfill ./target/wasm32-unknown-unknown/release/wasm_multi_value_retun_in_rust.wasm 'decode i32 i32'
Make `decode` function return `[I32, I32]`.

The WAT disassembly now looks like that:

  (type (;0;) (func (param i32 i32) (result i32 i32)))
  (type (;1;) (func (param i32 i32 i32)))
  (func $decode_multivalue_shim (type 0) (param i32 i32) (result i32 i32)
…

There you go. There is now a shim function that has the multi-value return, which calls the original method. I can now use my newly created WASM binary with WebAssembly runtimes that support multi-value returns (like Wasmer or Node.js).

Conclusion

With wasm-multi-value-reverse-polyfill I'm now able to create multi-value return functions with the current Rust compiler without depending on all the magic wasm-bindgen is doing.

Categories: en, WASM, Rust