`reinterpret(Type, v)` performance degrades with `-O1`

[NHDaly]

$ julia -O2 -e 'using BenchmarkTools; @btime reinterpret.(NTuple{48,UInt8}, $([(UInt128(1), UInt128(2), UInt128(3)) for _ in 1:1000]));'
  2.719 μs (2 allocations: 46.92 KiB)

$ julia -O1 -e 'using BenchmarkTools; @btime reinterpret.(NTuple{48,UInt8}, $([(UInt128(1), UInt128(2), UInt128(3)) for _ in 1:1000]));'
  15.292 μs (2002 allocations: 171.92 KiB)

We are discussing to extend bitcast to support non-padded (packed) isbits types, and switching to that instead.
Draft PR for that here: #57227.

[jakobnissen]

Out of curiosity, how does this apply to the user-facing reinterpret? Specifically:

• Is the plan also to switch the isprimitivetype check in reinterpret to a datatype_haspadding check?
• Since reinterpret already works using memcpy for two structs that has identical padding, can bitcast be used in that case also, simplifying the implementation of Base._reinterpret?

[NHDaly]

I think the answer could be yes to both of those bullets!

However, the JuliaHub folks I was talking to (@topolarity , @JeffBezanson , @gbaraldi , @oscardssmith) are hesitant to open up bitcast too much, so the second bullet would need to be discussed.

So far we only discussed doing the first bullet.

The perf issue in this ticket comes from using LLVM to compile away the Refs used for the memcpy, so I'm hoping to eliminate that as much as possible, by having julia's compiler directly generate the "identity" function that changes the types. That's my main intention with this thread.

[NHDaly]

CC: @Seelengrab as well

[Seelengrab]

I haven't tried, but how does this look without broadcasting? Or with using bitcast directly in a custom reduction/map? I'm pretty sceptical that tweaking how we perform reinterpret is going to be fruitful in the grand scheme of things, since this kind of targeted specialization is likely going to lead to a game of whack-a-mole.

IMO, it shouldn't be surprising that -O1 degrades performance relative to -O2 in some way. As far as I know, that switch more or less affects which LLVM passes are run, so which functions are affected can change in very hard-to-predict ways. You can't have it both ways, with both small & fast code 🤷

I guess my main question is: what's the underlying actual problem you're trying to solve?

[KristofferC]

I think the goal is to have a reinterpret that doesn't rely so heavily on optimization passes for reasonable performance so that code that typically does not need -O2 (for latency reasons) can still use it without hitting this quite sever performance regression.

[NHDaly]

Yeah, exactly. Since this is ultimately quite a simple operation (literally a noop - as Jeff pointed out, reinterpret on packed data is the identity function, it just changes a value's type in the compiler domain), I believed it could be implemented in julia's compiler without needing to rely on an LLVM optimization to make it work.

However, it turns out that it's surprisingly tricky to do, as the restriction that T is a primitive type comes from LLVM's reinterpret instruction, and working around that is a bit annoying.

So the new proposal is that @gbaraldi is going to bring this one specific LLVM pass into -O1, so that reinterpret works correctly with reasonable performance in both versions, as Kristoffer says.

[KristofferC]

@gbaraldi, did you check yet how putting the relevant pass into O1 fares when it comes to this?