Tail call VM [2] #18720

arnaud-lb · 2025-05-31T12:13:00Z

This is the second part of Tail call VM #17849
First part was Pass opline as argument to opcode handlers in CALL VM #17952

This part takes tail-calling and preserve_none from #17849:

Opcode handlers dispatch directly to the next handler by tail-calling, which reduces function call overhead and avoids returning to the executor loop
preserve_none reduces register saving overhead in opcode handlers

This also implements JIT support.

Non-dispatching opcode handlers

JIT needs non-dispatching opcode handlers (opcode handlers that return instead of calling the next one). I've tried two approaches for this:

Generate a second, non-dispatching, variant of each handler. Use the variant as call_handler
Use indirect dispatch similarly to the hybrid VM: zend_op->handler is a function that calls the real handler and dispatches.

I've tried both approach (the first one in this branch, and the second one in master...arnaud-lb:php-src:hybrid-tailcall.

The second approach resulted in a slightly slower VM due to indirect dispatching, and JIT generated more spilling when calling handlers as they clobber all registers.

Therefore I've taken the first approach in this PR.

A 3rd approach would be to control dispatching via an additional handler parameter, or to pass a dispatch function to handlers, but I suspect this would have been slower.

Fixed regs and preserved regs

Thanks to the preserve_none convention, JIT'ed code only has to preserve rbp, which reduces the size of prologue/epilogue. Instead of preserving it, I add it to the set of fixed registers, so it's not used. This results in faster code.

Also, quite conveniently, preserve_none receives its first arguments via registers that are callee-saved in sysv. Therefore we can use the arg1 and arg2 regs as our fixed registers. This avoids moving arg1 and arg2 to SP/IP in prologue, or setting arg1/arg2 when tail-calling other handlers.

Benchmarks:

Benchmark	Mode	vs base	vs gcc	vs valgrind
bench.php	JIT	-4%	-0%	-5%
bench.php	Non-JIT	-44%	+3%
symfony demo	JIT	-0%	-0%
symfony demo	Non-JIT	-2.8%	-0%

base: Clang build of master, wall time
gcc: GCC build of master, wall time
valgrind: Clang build of master, valgrind instructions

Conclusion: Clang builds are now as fast GCC builds on the Symfony Demo benchmark in both JIT and non-JIT modes.

Issues

The preserve_none calling convention is documented as unstable. JIT would break if it changed. I suggest checking this at build time, and disabling this optimization (tailcalling + preserve_none) if preserve_none changed.
Clang currently fails to build PHP when using preserve_none and ASAN, therefore we disable this if ASAN is enabled. Edit: This seems to be fixed in recent Clang versions.

TODO

x86
aarch64
IR PRs
preserve_none configure check

arnaud-lb · 2025-05-31T12:16:38Z

ext/opcache/jit/ir/ir.h

-#define IR_GEN_CODE            (1<<22) /* C or LLVM */
-
-#define IR_GEN_CACHE_DEMOTE    (1<<23) /* Demote the generated code from closest CPU caches */
+#define IR_PRESERVE_NONE_FUNC  (1<<2) /* Generate a function with preserve_none calling convention */


Added basic preserve_none support to IR.

What's missing:

Tail-calling is supported, but not normal calls

Var args

I will submit this as a proper PR separately

arnaud-lb · 2025-05-31T12:25:49Z

ext/opcache/jit/ir/ir_x86.dasc

+	/* Move op2 to a scratch register before epilogue if it's in
+	 * used_preserved_regs, because it will be overridden. */


This fixes a bug where TAILCALL() op2 would be overridden by epilogue before the call.

I will submit this as a proper PR separately.

arnaud-lb · 2025-05-31T12:27:22Z

Zend/zend_compile.h

@@ -135,7 +136,7 @@ void zend_const_expr_to_zval(zval *result, zend_ast **ast_ptr, bool allow_dynami
 typedef int (*user_opcode_handler_t) (zend_execute_data *execute_data);

 struct _zend_op {
-	const void *handler;
+	zend_vm_opcode_handler_t handler;


Typed handler pointers in a few places to prevent confusion between orig handler and call handlers (zend_vm_opcode_handler_t / zend_vm_opcode_handler_func_t).

arnaud-lb · 2025-05-31T12:31:16Z

ext/opcache/jit/zend_jit_vm_helpers.c

-ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_func_trace_helper(ZEND_OPCODE_HANDLER_ARGS)
+ZEND_OPCODE_HANDLER_RET ZEND_OPCODE_HANDLER_CCONV zend_jit_func_trace_helper(ZEND_OPCODE_HANDLER_ARGS)
 {
-	ZEND_OPCODE_TAIL_CALL_EX(zend_jit_trace_counter_helper,
-		((ZEND_JIT_COUNTER_INIT + JIT_G(hot_func) - 1) / JIT_G(hot_func)));
+	zend_jit_op_array_trace_extension *jit_extension =
+		(zend_jit_op_array_trace_extension*)ZEND_FUNC_INFO(&EX(func)->op_array);
+	size_t offset = jit_extension->offset;
+	uint32_t cost = ((ZEND_JIT_COUNTER_INIT + JIT_G(hot_func) - 1) / JIT_G(hot_func));
+
+	*(ZEND_OP_TRACE_INFO(opline, offset)->counter) -= cost;
+
+	ZEND_OPCODE_TAIL_CALL(zend_jit_trace_counter_helper);


Tail-calling was not possible due to the extra arg in zend_jit_trace_counter_helper, so I removed it.

Alternative would be to define these handlers in IR, as we do for the hybrid JIT.

I didn't understand why extra_arg became a problem

Clang ˋmusttailˋ disallows calling functions with a different signature. Presumably this is to garantee portability. This is discussed here: https://blog.reverberate.org/2025/02/10/tail-call-updates.html

arnaud-lb · 2025-06-02T08:41:51Z

@dstogov this is still a WIP, but I would like to have your opinion on this. This is the same ideas as #17849, but with JIT support which implies a few things described above. I would like your opinion before handling the items in the todo list.

dstogov

The idea makes sense and the benchmark results look promising.
It's also interesting how it affects the VM code size.

Unfortunately at this moment I can't review a huge path like this in all details.
Anyway, I think it makes sense to finalize and land it.

dstogov · 2025-06-09T07:38:05Z

Zend/zend_vm.h

@@ -20,6 +20,7 @@
 #define ZEND_VM_H

 #include "zend_portability.h"
+#include "zend_vm_opcodes.h"


Originally zend_vm.h was an abstraction layer API over the actual VM implementation.
Including zend_vm_opcodes.h breaks this decision.
This is not a huge problem, just a note.

dstogov · 2025-06-09T09:49:12Z

Zend/zend_vm_gen.php

+                            out($f,"typedef struct _zend_vm_trampoline {\n");
+                            out($f,"    const zend_op            *opline;\n");
+                            out($f,"    zend_vm_opcode_handler_t  handler;\n");
+                            out($f,"} zend_vm_trampoline;\n");


C standard doesn't require returning content of struct in a pair of registers.
x86_64 ABI does this, but all other ABIs have to be checked (x86, Windows64, ...).

dstogov · 2025-06-09T09:57:10Z

ext/opcache/jit/ir/ir_emit.c

-# include "ir_emit_x86.h"
+# include <ir_emit_x86.h>
 #elif defined(IR_TARGET_AARCH64)
-# include "ir_emit_aarch64.h"
+# include <ir_emit_aarch64.h>


It makes sense to merge this separately.

dstogov · 2025-06-09T10:01:30Z

ext/opcache/jit/ir/ir_x86.dasc

+				ir_reg tmp_reg;
+				if (ir_is_preserve_none(ctx, insn)) {
+					tmp_reg = IR_REG_R10;
+				} else {
+					tmp_reg = IR_REG_INT_RET1;
+				}


It would be better to assign a temporary register during register allocation/assignment phase.
Why can't we select a good op2_reg in the first place?

I think adding a constraint will sometimes result in less efficient code. E.g. if op2 is RLOAD(x), with x not in the used preserved regs, we might emit a mov when it's not necessary.

We can use a hint, but then we still need to also handle it in emit.

One difficulty either way is we don't know the set of used preserved registers when assigning hints / constraints. (Except when it's fixed.)

That being said I'm not familiar with the RA so there may be a solution I'm not seeing. Do you have hints / pointers?

In general, if you request for a temporary register (like here https://github.com/dstogov/ir/blob/master/ir_x86.dasc#L1186), RA should select one that doesn't interfere with other used registers. This may don't take into account the epilogue code assumed by TAILCALL.

dstogov · 2025-06-09T10:12:58Z

ext/opcache/jit/zend_jit_ir.c

+				 * https://github.com/llvm/llvm-project/blob/a414877a7a5f000d01370acb1162eb1dea87f48c/llvm/lib/Target/X86/X86RegisterInfo.cpp#L319
+				 * https://github.com/llvm/llvm-project/blob/68bfe91b5a34f80dbcc4f0a7fa5d7aa1cdf959c2/llvm/lib/Target/X86/X86CallingConv.td#L1183
+				 */
+				jit->ctx.fixed_regset |= (1<<5);


fixed_regs are the registers reserved for the register variables.
I remember there were some problems with RBP usage, but I don't remember the exact problem.

In this case I add the register only to prevent the RA from using it. This is less expensive than adding it to the set of preserved registers. The same thing is done for the HYBRID VM a few lines below.

dstogov · 2025-06-09T10:24:10Z

ext/opcache/jit/zend_jit_vm_helpers.c

-ZEND_OPCODE_HANDLER_RET ZEND_FASTCALL zend_jit_func_trace_helper(ZEND_OPCODE_HANDLER_ARGS)
+ZEND_OPCODE_HANDLER_RET ZEND_OPCODE_HANDLER_CCONV zend_jit_func_trace_helper(ZEND_OPCODE_HANDLER_ARGS)
 {
-	ZEND_OPCODE_TAIL_CALL_EX(zend_jit_trace_counter_helper,
-		((ZEND_JIT_COUNTER_INIT + JIT_G(hot_func) - 1) / JIT_G(hot_func)));
+	zend_jit_op_array_trace_extension *jit_extension =
+		(zend_jit_op_array_trace_extension*)ZEND_FUNC_INFO(&EX(func)->op_array);
+	size_t offset = jit_extension->offset;
+	uint32_t cost = ((ZEND_JIT_COUNTER_INIT + JIT_G(hot_func) - 1) / JIT_G(hot_func));
+
+	*(ZEND_OP_TRACE_INFO(opline, offset)->counter) -= cost;
+
+	ZEND_OPCODE_TAIL_CALL(zend_jit_trace_counter_helper);


I didn't understand why extra_arg became a problem

github-actions bot added Category: Engine Extension: opcache ABI break labels May 31, 2025

arnaud-lb commented May 31, 2025

View reviewed changes

Tail call VM

9118090

arnaud-lb force-pushed the tailcall branch from 07624db to 9118090 Compare May 31, 2025 20:01

dstogov reviewed Jun 9, 2025

View reviewed changes

		/* Move op2 to a scratch register before epilogue if it's in
		* used_preserved_regs, because it will be overridden. */

Tail call VM [2] #18720

Are you sure you want to change the base?

Tail call VM [2] #18720

Uh oh!

Conversation

arnaud-lb commented May 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Non-dispatching opcode handlers

Fixed regs and preserved regs

Benchmarks:

Issues

TODO

Uh oh!

arnaud-lb May 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arnaud-lb commented Jun 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dstogov left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

arnaud-lb commented May 31, 2025 •

edited

Loading

arnaud-lb May 31, 2025 •

edited

Loading

arnaud-lb commented Jun 2, 2025 •

edited

Loading