Update AgentTestRunner to use JUnit5 - Continuation pt 2 #9431
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sarahchen6
force-pushed
the
sarahchen6/experiment-2
branch
from
774f994 to
2a02021
Compare
|
🎯 Code Coverage 🔗 Commit SHA: 9db45c5 | Docs | Was this helpful? Give us feedback! |
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 43 metrics, 16 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.53.0-SNAPSHOT~9db45c5209, baseline=1.53.0-SNAPSHOT~d07fee98fb
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.047 s) : 0, 1046972
Total [baseline] (8.634 s) : 0, 8633849
Agent [candidate] (1.044 s) : 0, 1044453
Total [candidate] (8.59 s) : 0, 8590440
section iast
Agent [baseline] (1.181 s) : 0, 1180553
Total [baseline] (9.334 s) : 0, 9334190
Agent [candidate] (1.193 s) : 0, 1193463
Total [candidate] (9.326 s) : 0, 9325702
gantt
title insecure-bank - break down per module: candidate=1.53.0-SNAPSHOT~9db45c5209, baseline=1.53.0-SNAPSHOT~d07fee98fb
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.446 ms) : 0, 1446
crashtracking [candidate] (1.448 ms) : 0, 1448
BytebuddyAgent [baseline] (732.173 ms) : 0, 732173
BytebuddyAgent [candidate] (731.03 ms) : 0, 731030
GlobalTracer [baseline] (241.941 ms) : 0, 241941
GlobalTracer [candidate] (242.043 ms) : 0, 242043
AppSec [baseline] (29.992 ms) : 0, 29992
AppSec [candidate] (30.108 ms) : 0, 30108
Debugger [baseline] (6.047 ms) : 0, 6047
Debugger [candidate] (6.038 ms) : 0, 6038
Remote Config [baseline] (659.042 µs) : 0, 659
Remote Config [candidate] (659.703 µs) : 0, 660
Telemetry [baseline] (13.768 ms) : 0, 13768
Telemetry [candidate] (12.23 ms) : 0, 12230
section iast
crashtracking [baseline] (1.468 ms) : 0, 1468
crashtracking [candidate] (1.47 ms) : 0, 1470
BytebuddyAgent [baseline] (851.744 ms) : 0, 851744
BytebuddyAgent [candidate] (863.436 ms) : 0, 863436
GlobalTracer [baseline] (234.331 ms) : 0, 234331
GlobalTracer [candidate] (235.849 ms) : 0, 235849
IAST [baseline] (27.735 ms) : 0, 27735
IAST [candidate] (28.573 ms) : 0, 28573
AppSec [baseline] (27.824 ms) : 0, 27824
AppSec [candidate] (28.371 ms) : 0, 28371
Debugger [baseline] (6.664 ms) : 0, 6664
Debugger [candidate] (5.72 ms) : 0, 5720
Remote Config [baseline] (603.104 µs) : 0, 603
Remote Config [candidate] (611.102 µs) : 0, 611
Telemetry [baseline] (9.138 ms) : 0, 9138
Telemetry [candidate] (8.285 ms) : 0, 8285
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.53.0-SNAPSHOT~9db45c5209, baseline=1.53.0-SNAPSHOT~d07fee98fb
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.054 s) : 0, 1053711
Total [baseline] (10.75 s) : 0, 10749586
Agent [candidate] (1.053 s) : 0, 1052538
Total [candidate] (10.774 s) : 0, 10773997
section appsec
Agent [baseline] (1.222 s) : 0, 1222111
Total [baseline] (10.816 s) : 0, 10815938
Agent [candidate] (1.224 s) : 0, 1224283
Total [candidate] (10.854 s) : 0, 10854194
section iast
Agent [baseline] (1.193 s) : 0, 1192547
Total [baseline] (11.06 s) : 0, 11059761
Agent [candidate] (1.198 s) : 0, 1197589
Total [candidate] (10.886 s) : 0, 10886078
section profiling
Agent [baseline] (1.2 s) : 0, 1200136
Total [baseline] (10.942 s) : 0, 10941809
Agent [candidate] (1.193 s) : 0, 1193100
Total [candidate] (10.882 s) : 0, 10882314
gantt
title petclinic - break down per module: candidate=1.53.0-SNAPSHOT~9db45c5209, baseline=1.53.0-SNAPSHOT~d07fee98fb
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.47 ms) : 0, 1470
crashtracking [candidate] (1.461 ms) : 0, 1461
BytebuddyAgent [baseline] (736.63 ms) : 0, 736630
BytebuddyAgent [candidate] (736.436 ms) : 0, 736436
GlobalTracer [baseline] (243.597 ms) : 0, 243597
GlobalTracer [candidate] (243.896 ms) : 0, 243896
AppSec [baseline] (30.268 ms) : 0, 30268
AppSec [candidate] (30.47 ms) : 0, 30470
Debugger [baseline] (6.074 ms) : 0, 6074
Debugger [candidate] (6.064 ms) : 0, 6064
Remote Config [baseline] (675.065 µs) : 0, 675
Remote Config [candidate] (676.237 µs) : 0, 676
Telemetry [baseline] (13.837 ms) : 0, 13837
Telemetry [candidate] (12.407 ms) : 0, 12407
section appsec
crashtracking [baseline] (1.456 ms) : 0, 1456
crashtracking [candidate] (1.453 ms) : 0, 1453
BytebuddyAgent [baseline] (754.715 ms) : 0, 754715
BytebuddyAgent [candidate] (756.209 ms) : 0, 756209
GlobalTracer [baseline] (234.691 ms) : 0, 234691
GlobalTracer [candidate] (235.611 ms) : 0, 235611
IAST [baseline] (23.565 ms) : 0, 23565
IAST [candidate] (23.615 ms) : 0, 23615
AppSec [baseline] (170.017 ms) : 0, 170017
AppSec [candidate] (168.313 ms) : 0, 168313
Debugger [baseline] (7.407 ms) : 0, 7407
Debugger [candidate] (8.801 ms) : 0, 8801
Remote Config [baseline] (655.281 µs) : 0, 655
Remote Config [candidate] (653.024 µs) : 0, 653
Telemetry [baseline] (8.455 ms) : 0, 8455
Telemetry [candidate] (8.432 ms) : 0, 8432
section iast
crashtracking [baseline] (1.482 ms) : 0, 1482
crashtracking [candidate] (1.465 ms) : 0, 1465
BytebuddyAgent [baseline] (860.774 ms) : 0, 860774
BytebuddyAgent [candidate] (865.257 ms) : 0, 865257
GlobalTracer [baseline] (234.515 ms) : 0, 234515
GlobalTracer [candidate] (235.779 ms) : 0, 235779
IAST [baseline] (30.145 ms) : 0, 30145
IAST [candidate] (28.91 ms) : 0, 28910
AppSec [baseline] (26.547 ms) : 0, 26547
AppSec [candidate] (27.495 ms) : 0, 27495
Debugger [baseline] (8.58 ms) : 0, 8580
Debugger [candidate] (8.378 ms) : 0, 8378
Remote Config [baseline] (623.02 µs) : 0, 623
Remote Config [candidate] (611.688 µs) : 0, 612
Telemetry [baseline] (8.527 ms) : 0, 8527
Telemetry [candidate] (8.407 ms) : 0, 8407
section profiling
crashtracking [baseline] (1.445 ms) : 0, 1445
crashtracking [candidate] (1.441 ms) : 0, 1441
BytebuddyAgent [baseline] (763.275 ms) : 0, 763275
BytebuddyAgent [candidate] (759.382 ms) : 0, 759382
GlobalTracer [baseline] (223.0 ms) : 0, 223000
GlobalTracer [candidate] (221.677 ms) : 0, 221677
AppSec [baseline] (30.116 ms) : 0, 30116
AppSec [candidate] (29.947 ms) : 0, 29947
Debugger [baseline] (6.294 ms) : 0, 6294
Debugger [candidate] (7.027 ms) : 0, 7027
Remote Config [baseline] (710.909 µs) : 0, 711
Remote Config [candidate] (708.843 µs) : 0, 709
Telemetry [baseline] (16.454 ms) : 0, 16454
Telemetry [candidate] (15.65 ms) : 0, 15650
ProfilingAgent [baseline] (109.086 ms) : 0, 109086
ProfilingAgent [candidate] (107.68 ms) : 0, 107680
Profiling [baseline] (109.731 ms) : 0, 109731
Profiling [candidate] (108.345 ms) : 0, 108345
LoadParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 12 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~9db45c5209, baseline=1.53.0-SNAPSHOT~d07fee98fb
dateFormat X
axisFormat %s
section baseline
no_agent (38.05 ms) : 37735, 38364
. : milestone, 38050,
appsec (46.947 ms) : 46546, 47349
. : milestone, 46947,
code_origins (45.453 ms) : 45070, 45835
. : milestone, 45453,
iast (46.238 ms) : 45829, 46647
. : milestone, 46238,
profiling (48.318 ms) : 47905, 48731
. : milestone, 48318,
tracing (43.424 ms) : 43047, 43801
. : milestone, 43424,
section candidate
no_agent (35.971 ms) : 35685, 36258
. : milestone, 35971,
appsec (47.708 ms) : 47281, 48135
. : milestone, 47708,
code_origins (45.75 ms) : 45352, 46147
. : milestone, 45750,
iast (45.131 ms) : 44746, 45515
. : milestone, 45131,
profiling (49.515 ms) : 49047, 49983
. : milestone, 49515,
tracing (43.095 ms) : 42734, 43455
. : milestone, 43095,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~9db45c5209, baseline=1.53.0-SNAPSHOT~d07fee98fb
dateFormat X
axisFormat %s
section baseline
no_agent (4.431 ms) : 4381, 4480
. : milestone, 4431,
iast (9.466 ms) : 9300, 9633
. : milestone, 9466,
iast_FULL (13.633 ms) : 13361, 13904
. : milestone, 13633,
iast_GLOBAL (10.369 ms) : 10185, 10553
. : milestone, 10369,
profiling (9.003 ms) : 8863, 9143
. : milestone, 9003,
tracing (7.454 ms) : 7351, 7558
. : milestone, 7454,
section candidate
no_agent (4.387 ms) : 4336, 4438
. : milestone, 4387,
iast (9.507 ms) : 9342, 9672
. : milestone, 9507,
iast_FULL (14.003 ms) : 13723, 14282
. : milestone, 14003,
iast_GLOBAL (10.166 ms) : 9988, 10344
. : milestone, 10166,
profiling (8.789 ms) : 8637, 8941
. : milestone, 8789,
tracing (7.635 ms) : 7522, 7749
. : milestone, 7635,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~9db45c5209, baseline=1.53.0-SNAPSHOT~d07fee98fb
dateFormat X
axisFormat %s
section baseline
no_agent (1.476 ms) : 1464, 1487
. : milestone, 1476,
appsec (3.672 ms) : 3456, 3889
. : milestone, 3672,
iast (2.209 ms) : 2146, 2271
. : milestone, 2209,
iast_GLOBAL (2.244 ms) : 2181, 2307
. : milestone, 2244,
profiling (2.041 ms) : 1990, 2091
. : milestone, 2041,
tracing (2.018 ms) : 1970, 2067
. : milestone, 2018,
section candidate
no_agent (1.477 ms) : 1466, 1489
. : milestone, 1477,
appsec (3.657 ms) : 3440, 3874
. : milestone, 3657,
iast (2.205 ms) : 2142, 2268
. : milestone, 2205,
iast_GLOBAL (2.24 ms) : 2177, 2303
. : milestone, 2240,
profiling (2.027 ms) : 1977, 2077
. : milestone, 2027,
tracing (2.018 ms) : 1970, 2067
. : milestone, 2018,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~9db45c5209, baseline=1.53.0-SNAPSHOT~d07fee98fb
dateFormat X
axisFormat %s
section baseline
no_agent (14.71 s) : 14710000, 14710000
. : milestone, 14710000,
appsec (14.935 s) : 14935000, 14935000
. : milestone, 14935000,
iast (18.147 s) : 18147000, 18147000
. : milestone, 18147000,
iast_GLOBAL (18.047 s) : 18047000, 18047000
. : milestone, 18047000,
profiling (15.991 s) : 15991000, 15991000
. : milestone, 15991000,
tracing (14.705 s) : 14705000, 14705000
. : milestone, 14705000,
section candidate
no_agent (15.242 s) : 15242000, 15242000
. : milestone, 15242000,
appsec (14.568 s) : 14568000, 14568000
. : milestone, 14568000,
iast (18.778 s) : 18778000, 18778000
. : milestone, 18778000,
iast_GLOBAL (17.828 s) : 17828000, 17828000
. : milestone, 17828000,
profiling (15.173 s) : 15173000, 15173000
. : milestone, 15173000,
tracing (14.845 s) : 14845000, 14845000
. : milestone, 14845000,
|
Kafka / producer-benchmarkParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics. See unchanged results
|
Kafka / consumer-benchmarkParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics. See unchanged results
|
sarahchen6
commented
Aug 28, 2025
| @@ -93,7 +94,7 @@ abstract class IastSourcesTest<SERVER> extends IastHttpServerTest<SERVER> { | |||
| void 'test form source'() { | |||
| when: | |||
| final url = "${address}/iast/sources/form" | |||
| final body = new Request.Builder().add('name', 'value').build() | |||
| final body = new FormBody.Builder().add('name', 'value').build() | |||
| final request = new Request.Builder().url(url).post(body).build() | |||
There was a problem hiding this comment.
add method needs to be FormBody. See FormBody.Builder vs Request.Builder.
sarahchen6
commented
Aug 28, 2025
| @@ -77,7 +77,7 @@ class WrapperForkedTest extends Specification { | |||
| then: | |||
| 2 * mockReader.readLine() >>> ['!', null] | |||
| then: | |||
| 1 * endCb.apply(_, _) | |||
| 1 * endCb.apply(_, _) >> Flow.ResultFlow.empty() | |||
|
|
|||
There was a problem hiding this comment.
Without >> Flow.ResultFlow.empty(), this was resulting in a NPE where flow.getAction() was called on null objects.
sarahchen6
commented
Aug 29, 2025
| @@ -866,7 +866,22 @@ abstract class HttpClientTest extends VersionedNamingTestBase { | |||
| "$DDTags.PATHWAY_HASH" { String } | |||
| } | |||
| if (exception) { | |||
| errorTags(exception.class, exception.message) | |||
| // PlayWS classes throw different exception types for the same connection failures | |||
There was a problem hiding this comment.
I resolved these failures by loosening the expected exception types specifically for PlayWS classes. Is there a better way to address this?
|
Relevant changes included in #9445 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
This PR builds on #9264 to address the failing instrumentation tests.
Motivation
Update
AgentTestRunnerto use JUnit 5, necessary to support Java 25 and test JUnit 6.Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any usefull labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]