Test 1.12-nightly

[christiangnrd]

Suggested here #575 (comment).

I went with replacing nightly for the moment as llvmdowngrader is not yet supported for llvm 18+ so we know CI will fail for both.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.67%. Comparing base (caf2996) to head (6801da3).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #576   +/-   ##
=======================================
  Coverage   80.67%   80.67%           
=======================================
  Files          61       61           
  Lines        2681     2681           
=======================================
  Hits         2163     2163           
  Misses        518      518           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

Metal Benchmarks

Benchmark suite	Current: 6801da3	Previous: caf2996	Ratio
private array/construct	24263.833333333332 ns	25048.666666666668 ns	0.97
private array/broadcast	462209 ns	457958 ns	1.01
private array/random/randn/Float32	817312.5 ns	800854.5 ns	1.02
private array/random/randn!/Float32	624542 ns	641125 ns	0.97
private array/random/rand!/Int64	552625 ns	568834 ns	0.97
private array/random/rand!/Float32	590000 ns	597333 ns	0.99
private array/random/rand/Int64	787292 ns	774854.5 ns	1.02
private array/random/rand/Float32	609833.5 ns	631333 ns	0.97
private array/copyto!/gpu_to_gpu	658167 ns	660958 ns	1.00
private array/copyto!/cpu_to_gpu	677271 ns	832125 ns	0.81
private array/copyto!/gpu_to_cpu	802749.5 ns	590416 ns	1.36
private array/accumulate/1d	1352500 ns	1352208 ns	1.00
private array/accumulate/2d	1386834 ns	1397271 ns	0.99
private array/iteration/findall/int	1792917 ns	1811500 ns	0.99
private array/iteration/findall/bool	1596250 ns	1579583 ns	1.01
private array/iteration/findfirst/int	1725333.5 ns	1738479.5 ns	0.99
private array/iteration/findfirst/bool	1665666 ns	1671583.5 ns	1.00
private array/iteration/scalar	3900562.5 ns	3925542 ns	0.99
private array/iteration/logical	2992708.5 ns	3003542 ns	1.00
private array/iteration/findmin/1d	1777563 ns	1771542 ns	1.00
private array/iteration/findmin/2d	1361125 ns	1358104 ns	1.00
private array/reductions/reduce/1d	1048042 ns	1034333 ns	1.01
private array/reductions/reduce/2d	671417 ns	661771 ns	1.01
private array/reductions/mapreduce/1d	1046854 ns	1042688 ns	1.00
private array/reductions/mapreduce/2d	672875 ns	656417 ns	1.03
private array/permutedims/4d	2538313 ns	2535708 ns	1.00
private array/permutedims/2d	1031458 ns	1024729.5 ns	1.01
private array/permutedims/3d	1588812.5 ns	1604959 ns	0.99
private array/copy	603875 ns	573375 ns	1.05
latency/precompile	9827099250 ns	9802163250 ns	1.00
latency/ttfp	4100019584 ns	4108894000 ns	1.00
latency/import	1284734645.5 ns	1286845416.5 ns	1.00
integration/metaldevrt	743666 ns	708167 ns	1.05
integration/byval/slices=1	1585417 ns	1642916 ns	0.97
integration/byval/slices=3	10826417 ns	9533624.5 ns	1.14
integration/byval/reference	1525021 ns	1619479 ns	0.94
integration/byval/slices=2	2691604.5 ns	2720083.5 ns	0.99
kernel/indexing	452895.5 ns	461167 ns	0.98
kernel/indexing_checked	451500 ns	459625 ns	0.98
kernel/launch	37250 ns	7958 ns	4.68
metal/synchronization/stream	15000 ns	14750 ns	1.02
metal/synchronization/context	15250 ns	15250 ns	1
shared array/construct	23972.166666666668 ns	23722.25 ns	1.01
shared array/broadcast	463417 ns	462042 ns	1.00
shared array/random/randn/Float32	786000 ns	809042 ns	0.97
shared array/random/randn!/Float32	627000 ns	641042 ns	0.98
shared array/random/rand!/Int64	562083 ns	572917 ns	0.98
shared array/random/rand!/Float32	590584 ns	603417 ns	0.98
shared array/random/rand/Int64	767333 ns	792625 ns	0.97
shared array/random/rand/Float32	663083 ns	625166 ns	1.06
shared array/copyto!/gpu_to_gpu	80458 ns	89375 ns	0.90
shared array/copyto!/cpu_to_gpu	83208 ns	82834 ns	1.00
shared array/copyto!/gpu_to_cpu	83000 ns	78708 ns	1.05
shared array/accumulate/1d	1352792 ns	1358229 ns	1.00
shared array/accumulate/2d	1401750 ns	1393083.5 ns	1.01
shared array/iteration/findall/int	1795291.5 ns	1846500.5 ns	0.97
shared array/iteration/findall/bool	1601416.5 ns	1593458 ns	1.00
shared array/iteration/findfirst/int	1405083.5 ns	1406500 ns	1.00
shared array/iteration/findfirst/bool	1381500 ns	1369834 ns	1.01
shared array/iteration/scalar	160458 ns	160875 ns	1.00
shared array/iteration/logical	2871166.5 ns	2872291.5 ns	1.00
shared array/iteration/findmin/1d	1469084 ns	1514166 ns	0.97
shared array/iteration/findmin/2d	1377750 ns	1377125 ns	1.00
shared array/reductions/reduce/1d	729208 ns	725229 ns	1.01
shared array/reductions/reduce/2d	671959 ns	664500 ns	1.01
shared array/reductions/mapreduce/1d	738334 ns	739395.5 ns	1.00
shared array/reductions/mapreduce/2d	671979 ns	666562.5 ns	1.01
shared array/permutedims/4d	2551063 ns	2514583 ns	1.01
shared array/permutedims/2d	1035584 ns	1027667 ns	1.01
shared array/permutedims/3d	1598958 ns	1579792 ns	1.01
shared array/copy	248541 ns	250375 ns	0.99

This comment was automatically generated by workflow using github-action-benchmark.

[maleadt]

Oh right, I forgot we don't actually support 1.12 yet. I guess we need to update the downgrader...

[maleadt]

Not sure this is valuable without the bumped downgrader; now you're just removing opaque pointer testing from the CI roster without a replacement that will ever pass.

So I wouldn't merge this until we have that JLL at the least.