Add PR performance gate #8970

bric3 · 2025-06-12T09:42:59Z

What Does This Do

Adds a PR Gate

Motivation

Additional Notes

Depends on

Fix benchmark failures and improve results quality #8962

Gate skippable via performance/ignore-performance-regression label, bp-runner PR still in progress.

=> https://datadoghq.atlassian.net/wiki/x/8YFzMwE

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

.gitlab/benchmarks.yml

pr-commenter · 2025-06-12T11:07:47Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	bdu/pr-gate
git_commit_date	1750259064	1750259346
git_commit_sha	`712d5c3`	`5bf274c`
release_version	1.50.0-SNAPSHOT~712d5c320e	1.50.0-SNAPSHOT~5bf274c473

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1750261104	1750261104
ci_job_id	988351977	988351977
ci_pipeline_id	68125690	68125690
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-clrnfvyp-project-304-concurrent-1-pxg1o7kq 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-clrnfvyp-project-304-concurrent-1-pxg1o7kq 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 7 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~5bf274c473, baseline=1.50.0-SNAPSHOT~712d5c320e

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.035 s) : 0, 1034885
Total [baseline] (10.66 s) : 0, 10659818
Agent [candidate] (1.029 s) : 0, 1028966
Total [candidate] (10.558 s) : 0, 10558415
section appsec
Agent [baseline] (1.185 s) : 0, 1185494
Total [baseline] (10.656 s) : 0, 10656420
Agent [candidate] (1.182 s) : 0, 1181973
Total [candidate] (10.832 s) : 0, 10831634
section iast
Agent [baseline] (1.158 s) : 0, 1157755
Total [baseline] (10.841 s) : 0, 10840980
Agent [candidate] (1.157 s) : 0, 1156889
Total [candidate] (10.863 s) : 0, 10863346
section profiling
Agent [baseline] (1.27 s) : 0, 1269712
Total [baseline] (10.83 s) : 0, 10830419
Agent [candidate] (1.273 s) : 0, 1273273
Total [candidate] (10.872 s) : 0, 10872021

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.035 s	-
Agent	appsec	1.185 s	150.609 ms (14.6%)
Agent	iast	1.158 s	122.87 ms (11.9%)
Agent	profiling	1.27 s	234.827 ms (22.7%)
Total	tracing	10.66 s	-
Total	appsec	10.656 s	-3.398 ms (-0.0%)
Total	iast	10.841 s	181.162 ms (1.7%)
Total	profiling	10.83 s	170.601 ms (1.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.029 s	-
Agent	appsec	1.182 s	153.007 ms (14.9%)
Agent	iast	1.157 s	127.923 ms (12.4%)
Agent	profiling	1.273 s	244.307 ms (23.7%)
Total	tracing	10.558 s	-
Total	appsec	10.832 s	273.219 ms (2.6%)
Total	iast	10.863 s	304.931 ms (2.9%)
Total	profiling	10.872 s	313.606 ms (3.0%)

gantt
    title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~5bf274c473, baseline=1.50.0-SNAPSHOT~712d5c320e

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (688.935 ms) : 0, 688935
BytebuddyAgent [candidate] (684.347 ms) : 0, 684347
GlobalTracer [baseline] (242.342 ms) : 0, 242342
GlobalTracer [candidate] (241.752 ms) : 0, 241752
AppSec [baseline] (60.17 ms) : 0, 60170
AppSec [candidate] (59.72 ms) : 0, 59720
Debugger [baseline] (6.169 ms) : 0, 6169
Debugger [candidate] (6.121 ms) : 0, 6121
Remote Config [baseline] (748.419 µs) : 0, 748
Remote Config [candidate] (730.515 µs) : 0, 731
Telemetry [baseline] (12.798 ms) : 0, 12798
Telemetry [candidate] (12.888 ms) : 0, 12888
section appsec
BytebuddyAgent [baseline] (711.015 ms) : 0, 711015
BytebuddyAgent [candidate] (707.821 ms) : 0, 707821
GlobalTracer [baseline] (236.007 ms) : 0, 236007
GlobalTracer [candidate] (235.617 ms) : 0, 235617
AppSec [baseline] (179.131 ms) : 0, 179131
AppSec [candidate] (179.477 ms) : 0, 179477
Debugger [baseline] (5.873 ms) : 0, 5873
Debugger [candidate] (5.837 ms) : 0, 5837
Remote Config [baseline] (609.303 µs) : 0, 609
Remote Config [candidate] (604.341 µs) : 0, 604
Telemetry [baseline] (7.304 ms) : 0, 7304
Telemetry [candidate] (7.245 ms) : 0, 7245
IAST [baseline] (21.963 ms) : 0, 21963
IAST [candidate] (21.796 ms) : 0, 21796
section iast
BytebuddyAgent [baseline] (805.787 ms) : 0, 805787
BytebuddyAgent [candidate] (805.455 ms) : 0, 805455
GlobalTracer [baseline] (231.61 ms) : 0, 231610
GlobalTracer [candidate] (231.556 ms) : 0, 231556
AppSec [baseline] (56.091 ms) : 0, 56091
AppSec [candidate] (54.125 ms) : 0, 54125
Debugger [baseline] (6.036 ms) : 0, 6036
Debugger [candidate] (6.035 ms) : 0, 6035
Remote Config [baseline] (606.37 µs) : 0, 606
Remote Config [candidate] (597.789 µs) : 0, 598
Telemetry [baseline] (7.889 ms) : 0, 7889
Telemetry [candidate] (7.949 ms) : 0, 7949
IAST [baseline] (26.188 ms) : 0, 26188
IAST [candidate] (27.702 ms) : 0, 27702
section profiling
ProfilingAgent [baseline] (105.621 ms) : 0, 105621
ProfilingAgent [candidate] (106.827 ms) : 0, 106827
BytebuddyAgent [baseline] (674.5 ms) : 0, 674500
BytebuddyAgent [candidate] (676.246 ms) : 0, 676246
GlobalTracer [baseline] (359.148 ms) : 0, 359148
GlobalTracer [candidate] (359.827 ms) : 0, 359827
AppSec [baseline] (64.744 ms) : 0, 64744
AppSec [candidate] (64.568 ms) : 0, 64568
Debugger [baseline] (6.18 ms) : 0, 6180
Debugger [candidate] (6.123 ms) : 0, 6123
Remote Config [baseline] (656.431 µs) : 0, 656
Remote Config [candidate] (647.845 µs) : 0, 648
Telemetry [baseline] (8.214 ms) : 0, 8214
Telemetry [candidate] (8.14 ms) : 0, 8140
Profiling [baseline] (105.645 ms) : 0, 105645
Profiling [candidate] (106.851 ms) : 0, 106851

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~5bf274c473, baseline=1.50.0-SNAPSHOT~712d5c320e

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.03 s) : 0, 1029508
Total [baseline] (8.528 s) : 0, 8527831
Agent [candidate] (1.027 s) : 0, 1027289
Total [candidate] (8.528 s) : 0, 8527590
section iast
Agent [baseline] (1.153 s) : 0, 1153012
Total [baseline] (9.224 s) : 0, 9223582
Agent [candidate] (1.154 s) : 0, 1153703
Total [candidate] (9.212 s) : 0, 9212242

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.03 s	-
Agent	iast	1.153 s	123.504 ms (12.0%)
Total	tracing	8.528 s	-
Total	iast	9.224 s	695.751 ms (8.2%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.027 s	-
Agent	iast	1.154 s	126.414 ms (12.3%)
Total	tracing	8.528 s	-
Total	iast	9.212 s	684.653 ms (8.0%)

gantt
    title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~5bf274c473, baseline=1.50.0-SNAPSHOT~712d5c320e

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (685.66 ms) : 0, 685660
BytebuddyAgent [candidate] (684.117 ms) : 0, 684117
GlobalTracer [baseline] (241.887 ms) : 0, 241887
GlobalTracer [candidate] (241.206 ms) : 0, 241206
AppSec [baseline] (58.663 ms) : 0, 58663
AppSec [candidate] (58.478 ms) : 0, 58478
Debugger [baseline] (6.173 ms) : 0, 6173
Debugger [candidate] (6.171 ms) : 0, 6171
Remote Config [baseline] (748.96 µs) : 0, 749
Remote Config [candidate] (743.136 µs) : 0, 743
Telemetry [baseline] (12.841 ms) : 0, 12841
Telemetry [candidate] (13.091 ms) : 0, 13091
section iast
BytebuddyAgent [baseline] (803.145 ms) : 0, 803145
BytebuddyAgent [candidate] (802.845 ms) : 0, 802845
GlobalTracer [baseline] (230.393 ms) : 0, 230393
GlobalTracer [candidate] (230.813 ms) : 0, 230813
AppSec [baseline] (53.182 ms) : 0, 53182
AppSec [candidate] (55.154 ms) : 0, 55154
Debugger [baseline] (5.954 ms) : 0, 5954
Debugger [candidate] (6.033 ms) : 0, 6033
Remote Config [baseline] (591.311 µs) : 0, 591
Remote Config [candidate] (605.848 µs) : 0, 606
Telemetry [baseline] (7.936 ms) : 0, 7936
Telemetry [candidate] (7.981 ms) : 0, 7981
IAST [baseline] (28.463 ms) : 0, 28463
IAST [candidate] (26.843 ms) : 0, 26843

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	bdu/pr-gate
git_commit_date	1750259064	1750259346
git_commit_sha	`712d5c3`	`5bf274c`
release_version	1.50.0-SNAPSHOT~712d5c320e	1.50.0-SNAPSHOT~5bf274c473

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1750261782	1750261782
ci_job_id	988440857	988440857
ci_pipeline_id	68125690	68125690
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-8bgay5fx-project-304-concurrent-0-1oo0qykj 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-8bgay5fx-project-304-concurrent-0-1oo0qykj 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 2 performance regressions! Performance is the same for 10 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:tracing:high_load	worse [+273.296µs; +520.079µs] or [+3.626%; +6.900%]	unstable [-106.430op/s; +46.055op/s] or [-17.324%; +7.497%]	7.934ms	584.156op/s	7.537ms	614.344op/s
scenario:load:insecure-bank:profiling:high_load	worse [+628.904µs; +939.504µs] or [+7.280%; +10.875%]	unstable [-110.089op/s; +21.964op/s] or [-20.508%; +4.092%]	9.423ms	492.750op/s	8.639ms	536.812op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~5bf274c473, baseline=1.50.0-SNAPSHOT~712d5c320e
    dateFormat X
    axisFormat %s
section baseline
no_agent (36.905 ms) : 36610, 37201
.   : milestone, 36905,
appsec (46.569 ms) : 46153, 46985
.   : milestone, 46569,
code_origins (45.631 ms) : 45250, 46012
.   : milestone, 45631,
iast (45.317 ms) : 44924, 45711
.   : milestone, 45317,
profiling (49.833 ms) : 49366, 50300
.   : milestone, 49833,
tracing (43.955 ms) : 43600, 44310
.   : milestone, 43955,
section candidate
no_agent (37.756 ms) : 37460, 38052
.   : milestone, 37756,
appsec (46.338 ms) : 45926, 46749
.   : milestone, 46338,
code_origins (45.471 ms) : 45118, 45824
.   : milestone, 45471,
iast (44.95 ms) : 44544, 45356
.   : milestone, 44950,
profiling (48.45 ms) : 47992, 48907
.   : milestone, 48450,
tracing (45.039 ms) : 44658, 45419
.   : milestone, 45039,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	36.905 ms [36.61 ms, 37.201 ms]	-
appsec	46.569 ms [46.153 ms, 46.985 ms]	9.664 ms (26.2%)
code_origins	45.631 ms [45.25 ms, 46.012 ms]	8.726 ms (23.6%)
iast	45.317 ms [44.924 ms, 45.711 ms]	8.412 ms (22.8%)
profiling	49.833 ms [49.366 ms, 50.3 ms]	12.928 ms (35.0%)
tracing	43.955 ms [43.6 ms, 44.31 ms]	7.05 ms (19.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.756 ms [37.46 ms, 38.052 ms]	-
appsec	46.338 ms [45.926 ms, 46.749 ms]	8.581 ms (22.7%)
code_origins	45.471 ms [45.118 ms, 45.824 ms]	7.715 ms (20.4%)
iast	44.95 ms [44.544 ms, 45.356 ms]	7.194 ms (19.1%)
profiling	48.45 ms [47.992 ms, 48.907 ms]	10.694 ms (28.3%)
tracing	45.039 ms [44.658 ms, 45.419 ms]	7.282 ms (19.3%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~5bf274c473, baseline=1.50.0-SNAPSHOT~712d5c320e
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.327 ms) : 4278, 4377
.   : milestone, 4327,
iast (9.364 ms) : 9210, 9518
.   : milestone, 9364,
iast_FULL (13.914 ms) : 13634, 14194
.   : milestone, 13914,
iast_GLOBAL (10.054 ms) : 9878, 10229
.   : milestone, 10054,
profiling (8.639 ms) : 8503, 8774
.   : milestone, 8639,
tracing (7.537 ms) : 7428, 7646
.   : milestone, 7537,
section candidate
no_agent (4.282 ms) : 4228, 4336
.   : milestone, 4282,
iast (9.713 ms) : 9554, 9872
.   : milestone, 9713,
iast_FULL (13.986 ms) : 13709, 14263
.   : milestone, 13986,
iast_GLOBAL (10.177 ms) : 9988, 10365
.   : milestone, 10177,
profiling (9.423 ms) : 9270, 9576
.   : milestone, 9423,
tracing (7.934 ms) : 7814, 8054
.   : milestone, 7934,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.327 ms [4.278 ms, 4.377 ms]	-
iast	9.364 ms [9.21 ms, 9.518 ms]	5.037 ms (116.4%)
iast_FULL	13.914 ms [13.634 ms, 14.194 ms]	9.586 ms (221.5%)
iast_GLOBAL	10.054 ms [9.878 ms, 10.229 ms]	5.726 ms (132.3%)
profiling	8.639 ms [8.503 ms, 8.774 ms]	4.311 ms (99.6%)
tracing	7.537 ms [7.428 ms, 7.646 ms]	3.21 ms (74.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.282 ms [4.228 ms, 4.336 ms]	-
iast	9.713 ms [9.554 ms, 9.872 ms]	5.431 ms (126.8%)
iast_FULL	13.986 ms [13.709 ms, 14.263 ms]	9.704 ms (226.6%)
iast_GLOBAL	10.177 ms [9.988 ms, 10.365 ms]	5.895 ms (137.7%)
profiling	9.423 ms [9.27 ms, 9.576 ms]	5.141 ms (120.1%)
tracing	7.934 ms [7.814 ms, 8.054 ms]	3.652 ms (85.3%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	bdu/pr-gate
git_commit_date	1750259064	1750259346
git_commit_sha	`712d5c3`	`5bf274c`
release_version	1.50.0-SNAPSHOT~712d5c320e	1.50.0-SNAPSHOT~5bf274c473

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1750261364	1750261364
ci_job_id	988351982	988351982
ci_pipeline_id	68125690	68125690
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-8bgay5fx-project-304-concurrent-1-qumobxn9 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-8bgay5fx-project-304-concurrent-1-qumobxn9 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~5bf274c473, baseline=1.50.0-SNAPSHOT~712d5c320e
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.478 ms) : 1467, 1490
.   : milestone, 1478,
appsec (2.415 ms) : 2366, 2464
.   : milestone, 2415,
iast (2.199 ms) : 2138, 2260
.   : milestone, 2199,
iast_GLOBAL (2.235 ms) : 2174, 2296
.   : milestone, 2235,
profiling (2.035 ms) : 1986, 2084
.   : milestone, 2035,
tracing (2.017 ms) : 1970, 2064
.   : milestone, 2017,
section candidate
no_agent (1.475 ms) : 1464, 1487
.   : milestone, 1475,
appsec (2.408 ms) : 2359, 2456
.   : milestone, 2408,
iast (2.199 ms) : 2138, 2260
.   : milestone, 2199,
iast_GLOBAL (2.238 ms) : 2176, 2299
.   : milestone, 2238,
profiling (2.061 ms) : 2010, 2111
.   : milestone, 2061,
tracing (2.005 ms) : 1958, 2052
.   : milestone, 2005,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.478 ms [1.467 ms, 1.49 ms]	-
appsec	2.415 ms [2.366 ms, 2.464 ms]	936.748 µs (63.4%)
iast	2.199 ms [2.138 ms, 2.26 ms]	720.993 µs (48.8%)
iast_GLOBAL	2.235 ms [2.174 ms, 2.296 ms]	756.74 µs (51.2%)
profiling	2.035 ms [1.986 ms, 2.084 ms]	556.6 µs (37.7%)
tracing	2.017 ms [1.97 ms, 2.064 ms]	538.528 µs (36.4%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.475 ms [1.464 ms, 1.487 ms]	-
appsec	2.408 ms [2.359 ms, 2.456 ms]	932.572 µs (63.2%)
iast	2.199 ms [2.138 ms, 2.26 ms]	724.011 µs (49.1%)
iast_GLOBAL	2.238 ms [2.176 ms, 2.299 ms]	762.593 µs (51.7%)
profiling	2.061 ms [2.01 ms, 2.111 ms]	585.636 µs (39.7%)
tracing	2.005 ms [1.958 ms, 2.052 ms]	530.205 µs (35.9%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~5bf274c473, baseline=1.50.0-SNAPSHOT~712d5c320e
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.344 s) : 15344000, 15344000
.   : milestone, 15344000,
appsec (15.007 s) : 15007000, 15007000
.   : milestone, 15007000,
iast (18.65 s) : 18650000, 18650000
.   : milestone, 18650000,
iast_GLOBAL (17.993 s) : 17993000, 17993000
.   : milestone, 17993000,
profiling (15.099 s) : 15099000, 15099000
.   : milestone, 15099000,
tracing (15.076 s) : 15076000, 15076000
.   : milestone, 15076000,
section candidate
no_agent (14.963 s) : 14963000, 14963000
.   : milestone, 14963000,
appsec (14.71 s) : 14710000, 14710000
.   : milestone, 14710000,
iast (19.066 s) : 19066000, 19066000
.   : milestone, 19066000,
iast_GLOBAL (17.94 s) : 17940000, 17940000
.   : milestone, 17940000,
profiling (15.809 s) : 15809000, 15809000
.   : milestone, 15809000,
tracing (14.838 s) : 14838000, 14838000
.   : milestone, 14838000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.344 s [15.344 s, 15.344 s]	-
appsec	15.007 s [15.007 s, 15.007 s]	-337.0 ms (-2.2%)
iast	18.65 s [18.65 s, 18.65 s]	3.306 s (21.5%)
iast_GLOBAL	17.993 s [17.993 s, 17.993 s]	2.649 s (17.3%)
profiling	15.099 s [15.099 s, 15.099 s]	-245.0 ms (-1.6%)
tracing	15.076 s [15.076 s, 15.076 s]	-268.0 ms (-1.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.963 s [14.963 s, 14.963 s]	-
appsec	14.71 s [14.71 s, 14.71 s]	-253.0 ms (-1.7%)
iast	19.066 s [19.066 s, 19.066 s]	4.103 s (27.4%)
iast_GLOBAL	17.94 s [17.94 s, 17.94 s]	2.977 s (19.9%)
profiling	15.809 s [15.809 s, 15.809 s]	846.0 ms (5.7%)
tracing	14.838 s [14.838 s, 14.838 s]	-125.0 ms (-0.8%)

…ion.yml path

…on-regression.yml path

bric3 · 2025-06-18T15:07:06Z

.gitlab/benchmarks.yml

+      artifacts: true
+  when: on_success
+  tags: ["arch:amd64"]
+  allow_failure: true # Exercise the job before making it mandatory


todo: From @ddyurchenko

Could you please rebase your PR on the latest master and remove allow_failure: true from gating job?
In worst case scenario if we block something that should not be blocked, there is a label mechanism to bypass the gate

Ignore mechanism in bp-runner

Thanks for flagging it! We need to remove allow_failure: true, the bypass mechanism inside bp-runner is specified in https://datadoghq.atlassian.net/wiki/spaces/APMINT/pages/5158175217/Performance+quality+gates+-+User+Guide#How-to-bypass%3F.

…efined)

ddyurchenko

Looks good! Both cases where gates should block and where they should let pass work. ✔️

PerfectSlayer

Can you link doc about it?

Gate skippable via performance/ignore-performance-regression label

Is that a GitHub issue label? If so, is it configurable as it does not fit our usual pattern?

bric3 · 2025-06-19T10:25:51Z

@PerfectSlayer The bypass link is mentioned in @ddyurchenko's comment, I added it in the PR description.

Is that a GitHub issue label?

Yes.

If so, is it configurable as it does not fit our usual pattern?

I already brought this to @ddyurchenko, this is something to be addressed in bp-runner PR (which is not merged yet). (Hence the higher threshold at this time).

bric3 added tag: performance Performance related changes tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Jun 12, 2025

bric3 requested a review from a team June 12, 2025 09:46

ddyurchenko reviewed Jun 12, 2025

View reviewed changes

.gitlab/benchmarks.yml Outdated Show resolved Hide resolved

ddyurchenko reviewed Jun 12, 2025

View reviewed changes

.gitlab/benchmarks.yml Outdated Show resolved Hide resolved

bric3 force-pushed the bdu/pr-gate branch from 89c9ea7 to 301e9f8 Compare June 12, 2025 11:38

bric3 marked this pull request as ready for review June 13, 2025 16:17

bric3 requested a review from a team as a code owner June 13, 2025 16:17

bric3 requested review from colin-higgins and removed request for a team June 13, 2025 16:17

bric3 force-pushed the bdu/pr-gate branch from 1deb0cd to f76655f Compare June 13, 2025 19:54

PerfectSlayer changed the title ~~feat(ci): PR performance gate~~ Add PR performance gate Jun 14, 2025

PerfectSlayer added the type: enhancement label Jun 14, 2025

bric3 force-pushed the bdu/pr-gate branch 3 times, most recently from f806916 to 4aeb1c8 Compare June 18, 2025 14:50

bric3 added 7 commits June 18, 2025 17:05

feat(ci): PR performance gate

51cdcd5

chore(ci): Review comments

1e75a62

feat(ci): Explicitly rely on job artifacts, drop dacapo

109573f

fix(ci): revert removal of ARTIFACTSDIR for bp-runner.fail-on-regress…

3f56fdf

…ion.yml path

chore(ci): replace ARTIFACTSDIR by CI_PROJECT_DIR for bp-runner.fail-…

52b5b58

…on-regression.yml path

chore(ci): Move threshold to 10%

4018257

chore(ci): Allow failure on PR gate to exercise it

e6af269

bric3 commented Jun 18, 2025

View reviewed changes

chore(ci): Disable failure but increase regression threshold (to be r…

5bf274c

…efined)

bric3 force-pushed the bdu/pr-gate branch from 4aeb1c8 to 5bf274c Compare June 18, 2025 15:09

ddyurchenko approved these changes Jun 18, 2025

View reviewed changes

PerfectSlayer reviewed Jun 19, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add PR performance gate #8970

Add PR performance gate #8970

Uh oh!

bric3 commented Jun 12, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

pr-commenter bot commented Jun 12, 2025 •

edited

Loading

Uh oh!

bric3 Jun 18, 2025

Uh oh!

ddyurchenko Jun 18, 2025

Uh oh!

ddyurchenko left a comment

Uh oh!

PerfectSlayer left a comment

Uh oh!

bric3 commented Jun 19, 2025

Uh oh!

Uh oh!

Add PR performance gate #8970

Are you sure you want to change the base?

Add PR performance gate #8970

Uh oh!

Conversation

bric3 commented Jun 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

Uh oh!

Uh oh!

pr-commenter bot commented Jun 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

bric3 Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

ddyurchenko Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

ddyurchenko left a comment

Choose a reason for hiding this comment

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

bric3 commented Jun 19, 2025

Uh oh!

Uh oh!

bric3 commented Jun 12, 2025 •

edited

Loading

pr-commenter bot commented Jun 12, 2025 •

edited

Loading