Injecting platform specific instructions or blocks as inlined function imports at compile-time (capabilities)

[ttraenkler]

The discussion during the CG presentation seemed to centered mostly around the question of how to in general handle the tradeoff between portability and platform specific capabilities as this could break compatibility with certain platforms, unexpected performance cliffs, manually curating alternative code paths, or unconsciously introducing non determinism or runtime traps by importing a module as a sub-dependency using a non portable feature.

I would like to come back to the suggestion on slide 11 of the presentation given at the in person CG meeting, to handle "Relaxed Instructions as Imports" since this to me seems to be the most desirable of those options.

As stated in the slide the upside would be:

1. "Sidesteps introducing a new kind of non-determinism in WAsm": In other words, this moves the problem outside of the module by allowing the module consumers to take the decision best for their use case by providing the instruction as a normal function import that resolves how this instruction is handled: It could be fast but non deterministic between architectures, forced to emulate one version in software on architectures that do not support the instruction or could implement it completely differently with an arbitrary function that applies custom logic or alternative code paths (if/else) matching the needs of the consumer in their specific case, which gives control to the consumer instead of forcing a one size fits all hurting either determinism or performance constraints. In contrast to a simple macro like if/else block, this decision could be taken locally just for one module, and also would put the module consumer in control of the implementation, who would otherwise be bound to the solution deemed most suitable by the module implementer. If the instruction would not be provided as an import explicitly, the host could either have a globally set default depending on the normal solution in this environment, or it could be provided as "undefined" so the module could detect this capability does not exist and if it cannot handle it would "trap" predictably at compile or link time instead of an undesirable runtime trap or provide a default implementation itself. This solutions seems to encompass both the ups of the other suggestions "drop consistency" and "explicit tests for arch-specifics" and similar to WASI capabilities limits the blast radius of the taken tradeoff for the module. I believe this could also be a nice feature detection mechanism, by simple allowing an optional import for the non core Wasm features (that the host could provide by default if the consumer does not object or override).

Regarding the two downsides mentioned on slide 11:

1. "The difference is mostly theoretical, moving non-determinism from Wasm into environment doesn’t change anything in practice": I assume this is in comparison to the explicit tests for arch specifics (slide 12). Imho it is not mostly theoretical, as it is much more flexible and puts control into the hands of the consumer that can pick a default or customize the behavior based on their platform and task requirements. "Explicit tests for arch-specifics" in contrast would take the control out of the hands of the consumer how the instruction is implemented and forcing them to answer a simple yes/no for all modules to enable this feature. So lost would be the consumer choice per module and the freedom to taking application requirements into consideration by adapting the implementation to it, which the module implementer without knowledge of the use case naturally cannot do.

2. "Mozilla suggested this does not fit well into SpiderMonkey": What in specific is the problem with this suggestion in relation with SpiderMonkey? Would this be resolved with "pre-imports" as proposed in @rossberg Staged compilation and module linking presentation by allowing to provide the implementation at the time of module compilation as compare to instantiation?

The potential "downside" of platform specific instructions as function imports could be that modules cannot rely on these instructions to behave identical across modules anymore, so they should be seen as (potentially hardware accelerated) functions instead - but this stems from the nature of "relaxed" platform specific instructions itself, so I believe is a better tradeoff to make than introducing a one size fits all solution globally and stop out certain use cases completely or splitting the ecosystem.

[eqrion]

2. **"Mozilla suggested this does not fit well into SpiderMonkey"**: What in specific is the problem with this suggestion in relation with SpiderMonkey? Would this  be resolved with "pre-imports" as proposed in @rossberg Staged compilation and module linking presentation by allowing to provide the implementation at the time of module compilation as compare to instantiation?

It's not so much a problem with SpiderMonkey as it is with the general compilation model of WebAssembly so-far. When we compile a module, we don't know anything about the import other than the function type. The import name is only used for linking, and cannot be used to signify that the import should have an alternate behavior.

What this means is that if you were to import one of these instructions as a function, we would need to compile it to essentially an indirect function call, as we don't know whether the special 'intrinsic function' (or whatever it would be called) is going to be provided as the import or something else (like a JS function). This would likely hurt the performance gains of the relaxed instructions pretty significantly.

Something like "pre-imports" could solve this, but there would need to be an actual proposal to evaluate this to say this for sure. Additionally, the presentation Rossberg gave at the meeting relied on a version of the module linking proposal, which is not a quick feature to add to SpiderMonkey (I would guess other JS engines as well, but don't know for sure). So it could be some time to add it if this proposal required it.

[ttraenkler]

Thanks for clarifying, so it's indeed the dynamic nature of the imports not known at compile time.

Something like "pre-imports" could solve this, but there would need to be an actual proposal to evaluate this to say this for sure. Additionally, the presentation Rossberg gave at the meeting relied on a version of the module linking proposal, which is not a quick feature to add to SpiderMonkey (I would guess other JS engines as well, but don't know for sure). So it could be some time to add it if this proposal required it.

Waiting for the module linking proposal to be revived and landed is probably on a timeframe blocking the progress of this proposal for too long. I imagine to make this work what would be needed is a way to import a function at compile time - could be called a "compile time" or "static" import.

The simplest option that comes to mind for browsers until the day the module linking proposal arrives would be to extend compileStreaming with a second parameter like in instantiateStreaming and provide the import there? Other runtimes could do a similar thing. This would maybe also help clarifying the semantic difference between compileStreaming and instantiateStreaming as from what I have heard instantiateStreaming is sometimes the function where the code is actually compiled in some engines.

[ttraenkler]

I imagine it could also be desirable for performance optimization reasons in general other than SIMD (direct function calls, inlining) to provide some imports at compile time.

[Maratyszcza]

I feel there're some misunderstandings as to what this suggestion entailed. The suggestion was to define Relaxed SIMD instructions as "imported" functions. However, these functions can't really be imported: as @eqrion explained above, this would require WAsm engine to emit a call instruction for each use of the Relaxed SIMD operations, as it doesn't know the definition of these operations at the time it generates machine code. So instead, the WAsm engines would have to treat these "imports" as built-in operations, i.e. detect a call to function named e.g. "wams_relaxed_madd_f32x4" and emit Relaxed Multiply-Add operation without a function call. Thus, it sidesteps defining Relaxed SIMD operations in WAsm bytecode format, but instead makes them a part of the environment provided by the WAsm engine. This is where the major drawback comes from: "The difference is mostly theoretical", as we've removed these non-deterministic operations from WAsm bytecode only to make them a required part of the environment this bytecode depends on.

[sunfishcode]

@Maratyszcza That's the issue that "pre-imports", or "staged compilation" is proposed to solve, by having a compilation stage where some imports are resolved in a separate stage, before the main compilation stage begins.

[Maratyszcza]

IIUC, pre-imports doesn't imply that these functions are inlined, and in this use-case inlining is critical.

[sunfishcode]

Pre-imports would imply that they can be inlined, without changing the rest of the compilation model of Wasm.

[ttraenkler]

Could compiler engineers working on the runtimes comment on this idea if a "static function import" compilation stage would unlock inlining of imported functions and if this would be feasible to be added to existing runtimes?

Could this compilation stage be polyfilled by a preprocessing tool in the style of wasm-ld taking multiple modules as imports, producing a single module as output so this can be done without runtime support in the meanwhile?

[penzn]

Pre-imports would imply that they can be inlined, without changing the rest of the compilation model of Wasm.

For pre-imports to have acceptable performance they have to be inlined, and it will have immense cost in runtimes that don't inline. Implementing relaxed SIMD via pre-imports would require runtimes that want to support it to implement function inlining first. This is in addition to "non-determinism moving around" problem which @Maratyszcza explained so well above.

[ttraenkler]

For pre-imports to have acceptable performance they have to be inlined, and it will have immense cost in runtimes that don't inline. Implementing relaxed SIMD via pre-imports would require runtimes that want to support it to implement function inlining first.

Thanks for these insights. While not being a compiler engineer I have a limited understanding of how difficult it would be to implement function inlining of macro expansion for compile time static function imports, I do understand this would require sth like inlining or macros.

What would a MVP of a suitable mechanism like function inlining or macro expansion of definitions provided at compile time look like in terms of complexity and is there interest in this from the side of compiler engineers? Is there a prior discussion or proposal already?

[eqrion]

Trying to touch on a couple of the different discussion points here.

1. Normal function imports are not useful here because names are not significant and the imported function is not known at compile time, resulting in indirect function calls being emitted. This is why something 'new' would be required, like @Maratyszcza mentioned.
2. Pre-imports or some form of staged compilation would make it so the imported function is known at compile time. This would allow (but not require) an imported function to be inlined or handled in some special way.
3. Engines could make a (non-specified) guarantee that whenever a relaxed SIMD intrinsic function (e.g. WebAssembly.relaxedMulAdd) is pre-imported, that we will inline it and it will be as fast as an instruction.
4. The most general form of pre-imports or staged compilation based on module linking is likely a lot of work for engines. It's not clear what a MVP proposal in this space would look like, I don't think anyone has proposed one. My guess would be finding a way to limit the pre-imports to just functions with certain simple params and result types.
5. A form of inlining certain intrinsic functions that are known at compile-time 'somehow' likely wouldn't be too hard to implement in SpiderMonkey.
6. The meta point of 'this just moves the non-determinism around' from @Maratyszcza is valid. It sounds like any solution here is going to have non-determinism somewhere, and we probably just need to find place that makes the most people comfortable. I'm not sure if host provided intrinsic imports really is a large improvement for the downsides it introduces.