winch(x64): Tail call

[saulecabrera]

The Wasm Tail Call proposal is considered Tier 1 according to Wasmtime's Tiers of support.

Winch currently doesn't support this proposal.

• return_call @MarinPostma
• return_call_indirect @MarinPostma

[MarinPostma]

@saulecabrera you can assign me to that 👍

[MarinPostma]

@saulecabrera

Did you have a sketch of a plan on how to implement that already?

I did some homework this weekend, trying to find a path toward tail calls. I think we could do something like described in this post about v8 baseline compiler, or as initially implemented in cranelift, basically:

• set up the frame as if we were about to perform a normal call
• memcpy the frame down to override the caller frame
• jump to the callee entry-point

However this is over simplistic. There are many things getting in our way. Here's my understanding of the situation, thus far:

1. The callee and the caller may not require the same amount of stack space for their arguments. In the Winch calling convention, it is the caller that cleans up the stack space for the arguments. This is an issue for tail calls, because the caller doesn't know ahead of time how much stack space will need to be cleaned up when the callee returns. cranelift TailCall convention solves that by requiring that the callee cleans-up it's stack space. I suppose there are alternatives to let the caller know dynamically the amount of stack space to pop on cleanup, but in any case, it seems to me that we need a calling convention that supports that.
2. The ret_area is inherited from the caller, rather and allocated in the caller stack. This is easy to do: we can get it from the caller's initial control frame. (see there). This works, because the callee must have the same return as the caller.
3. We need to emit a jump to the callee, rather than a call, because we must not push a return address, since the state is already set up during the return_call preparation. Winch relies of cranelift to emit relocs for the callee address. The story around converting a function ref to a MachLabel that can be used for a jump is not entirely clear to me yet. Alternatively, we could pop the FP back into it's register and let call push it back (this is what v8 baseline does).
4. Cranelift has Inst::CallReturn* macro-instruction, but it's not clear to me if we can use it, since it emits code specific to cranelift Tail calling convention.

I am glancing over a lot of things here, obviously. My principal concern is with 1) right now; I don't see a way forward without addressing it.