Update AgentTestRunner to use JUnit5 - Continuation #9413

sarahchen6 · 2025-08-22T19:38:46Z

What Does This Do

Built on #9264 to address failing instrumentation tests

Motivation

Update AgentTestRunner to use JUnit5, necessary to support Java 25 and test JUnit6

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

* Support async servlet for RUM injection * Move isRumEnabled to instrumenter config

* chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * Rollback some dependencies to fix compilation. * Added dependency of `compilePlayRoutes` on `fixPlayRoutesDirectory`. * Rollback some dependencies to fix compilation. --------- Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com> Co-authored-by: Alexey Kuznetsov <[email protected]>

… directory. (#9328)

Bumps the gh-actions-packages group with 3 updates: [actions/checkout](https://github.com/actions/checkout), [actions/cache](https://github.com/actions/cache) and [github/codeql-action](https://github.com/github/codeql-action). Updates `actions/checkout` from 4.2.2 to 5.0.0 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@11bd719...08c6903) Updates `actions/cache` from 4.2.3 to 4.2.4 - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@5a3ec84...0400d5f) Updates `github/codeql-action` from 3.29.7 to 3.29.8 - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@51f7732...76621b6) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: 5.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: gh-actions-packages - dependency-name: actions/cache dependency-version: 4.2.4 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: gh-actions-packages - dependency-name: github/codeql-action dependency-version: 3.29.8 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: gh-actions-packages ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Avoid NPE on featureDiscovery creation * Review

What Does This Do Modifies WafModule#buildEvents to safely handle actionWithData.data being null. Motivation Fix #9346 Additional Notes The current version of libddwaf may return null in the data field of actionWithData. This was previously not handled and could cause unexpected logged exceptions. The method now explicitly checks for null to prevent this.

#9351) * use magic mirror depot * remove separate cache name used for testing

by eliminating the chance identity hash might collide even after 3 rehashes the easiest way to do this is to bump the size of the cache used in testing note this is only necessary because we want to keep both keys in the cache to test the 'visit' method - otherwise we'd need to relax that assertion

* Add dependency to OCI publish jobs * Override new publishing-gate job * Change stage order because publishing-gate depends on deploy_to_maven_central * Add verify_maven_central_deployment job * Test verify_maven_central_deployment job by temporarily removing dependency * Change to registry.ddbuild image * Test failing version * Clean * Change image for verify_maven_central_deployment job * Fix verify_maven_central_deployment script * Remove publishing-gate rules * Test verify job * Add manual trigger * Fix verify script * Add test-job to see what happens when verify job fails * Clean test code * Change retry rate * Try test-job again * Clean again * Create a separate job to skip the verify maven job --------- Co-authored-by: Laplie Anderson <[email protected]>

…Client5NamingV0Test`. (#9361)

* feat(env): Migrate internal apis to environment component * feat(env): Add new API to EnvironmentVariables and SystemProperties

* Refactor matchOperator to use method reference comparator for efficiency * Implement better exception handling and debug logging for StableConfig parsing errors * Add a debug log about no rules getting applied * Improve test file logic and cleanup in StableConfigSourceTest class * Github suggestions: Use static factory methods as constructors for Rule and Selector classes * Improve exception message: include value of object that caused the failure * Add more test cases to StableConfigSourceTest to improve codecov * Add test coverage for StableConfigMappingException * Update internal-api/src/main/java/datadog/trace/bootstrap/config/provider/stableconfig/Rule.java Co-authored-by: Alexey Kuznetsov <[email protected]> * Update internal-api/src/main/java/datadog/trace/bootstrap/config/provider/stableconfig/Rule.java Co-authored-by: Alexey Kuznetsov <[email protected]> * Use safeToString when printing stableconfig map * rearrange logic in selectors creation for better readability * delete superfluous else * Use constant to represent max length in safeToString * use throwStableConfigMappingException helper function * refactor exception catching in StableConfigSource * restore semicolon in calls to throwStableConfigMappingException * Use static import for throwStableConfigMappingException * test other kinds of exceptions in StableConfigSource * remove duplicate cfg assignments * fix missing operator test case * Update internal-api/src/main/java/datadog/trace/bootstrap/config/provider/stableconfig/Rule.java Co-authored-by: Alexey Kuznetsov <[email protected]> * Improve readability of StableConfigMappingException and its function invocations * modify safeToString logic to print first 50 and last 50 chars * remove extra placeholder in string message when printing exception * Update internal-api/src/main/java/datadog/trace/bootstrap/config/provider/StableConfigSource.java Co-authored-by: Alexey Kuznetsov <[email protected]> * refactor Rules.from to perform type assertions before heavyweight list manipulation * run spotless --------- Co-authored-by: Alexey Kuznetsov <[email protected]>

* Add stage for publishing-gate * Allow publishing-gate to run when populating cache

* chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * chore: Update Gradle dependencies * Rollback some dependencies to fix compilation. --------- Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com> Co-authored-by: Alexey Kuznetsov <[email protected]>

* feat(bootstrap): Migrate Spray instrumentation to Context API * feat(bootstrap): Migrate Tomcat instrumentation to Context API * feat(bootstrap): Migrate Liberty instrumentations to Context API * feat(bootstrap): Migrate instrumentations to Context API * fix(bootstrap): Fix unexpected call to IG callback * feat(play): Simplify testing * fix(bootstrap): Fix import

Do not stop on git log check Fix next patch release version compute

@see

…generated keys (#9321) * Prevent crash in SQL Server JDBC when tracing execute methods with generated keys When tracing the JDBC statement execute methods, we prepend a comment to the SQL query used for DBM trace propagation. However, there is a bug in the SQL Server JDBC driver that prevents the generated keys from being returned when the SQL comment is prepended to the SQL query. I decided to only append the comment in this specific case to avoid the comment from being truncated. @see microsoft/mssql-jdbc#2729 * remove dynamic typing when packing all the JDBC arguments * move JDBC test on injection with generated keys to SQL Server file * remove useless constant in StatementAdvice

…#9382) Bumps the gh-actions-packages group with 1 update: [github/codeql-action](https://github.com/github/codeql-action). Updates `github/codeql-action` from 3.29.8 to 3.29.10 - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@76621b6...96f518a) --- updated-dependencies: - dependency-name: github/codeql-action dependency-version: 3.29.10 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: gh-actions-packages ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* feat(dsm): Improve code style * feat(aws): Refactor AWS instrumentation around Context

* initial introduction of config_id * fix tests, fix NPE in ConfigSetting.equals, include configId in telemetryrequestbody when nonnull * report config id for stableconfigsources in get * Fix call to outdated ConfigSetting constructor in EventSourceTest * fix config_id report * fix where null config id was getting treated as string 'null' * report config id for all get methods * remove superfluous file * simplify configId resolution logic in configProvider * report sourceValue instead of value to ConfigCollector in get method * add tests for configid and no configid

datadog-official · 2025-08-22T19:56:45Z

🎯 Code Coverage
• Patch Coverage: 82.59%
• Total Coverage: 59.87% (+0.27%)

View detailed report

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 4955894 | Docs | Was this helpful? Give us feedback!}

pr-commenter · 2025-08-22T20:11:00Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/experiment
git_commit_date	1756307636	1756345431
git_commit_sha	`02a3250`	`4955894`
release_version	1.53.0-SNAPSHOT~02a3250561	1.53.0-SNAPSHOT~49558946ff

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1756347352	1756347352
ci_job_id	1102640566	1102640566
ci_pipeline_id	74926707	74926707
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-m6f66n5g 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-m6f66n5g 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 47 metrics, 12 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.53.0-SNAPSHOT~49558946ff, baseline=1.53.0-SNAPSHOT~02a3250561

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.065 s) : 0, 1064501
Total [baseline] (8.725 s) : 0, 8724685
Agent [candidate] (1.049 s) : 0, 1048920
Total [candidate] (8.647 s) : 0, 8647421
section iast
Agent [baseline] (1.187 s) : 0, 1186835
Total [baseline] (9.404 s) : 0, 9403689
Agent [candidate] (1.192 s) : 0, 1191865
Total [candidate] (9.358 s) : 0, 9357597

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.065 s	-
Agent	iast	1.187 s	122.335 ms (11.5%)
Total	tracing	8.725 s	-
Total	iast	9.404 s	679.004 ms (7.8%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.049 s	-
Agent	iast	1.192 s	142.945 ms (13.6%)
Total	tracing	8.647 s	-
Total	iast	9.358 s	710.176 ms (8.2%)

gantt
    title insecure-bank - break down per module: candidate=1.53.0-SNAPSHOT~49558946ff, baseline=1.53.0-SNAPSHOT~02a3250561

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.474 ms) : 0, 1474
crashtracking [candidate] (1.46 ms) : 0, 1460
BytebuddyAgent [baseline] (743.618 ms) : 0, 743618
BytebuddyAgent [candidate] (733.676 ms) : 0, 733676
GlobalTracer [baseline] (246.135 ms) : 0, 246135
GlobalTracer [candidate] (242.95 ms) : 0, 242950
AppSec [baseline] (30.761 ms) : 0, 30761
AppSec [candidate] (30.096 ms) : 0, 30096
Debugger [baseline] (6.132 ms) : 0, 6132
Debugger [candidate] (6.079 ms) : 0, 6079
Remote Config [baseline] (681.064 µs) : 0, 681
Remote Config [candidate] (665.059 µs) : 0, 665
Telemetry [baseline] (14.452 ms) : 0, 14452
Telemetry [candidate] (12.952 ms) : 0, 12952
section iast
crashtracking [baseline] (1.457 ms) : 0, 1457
crashtracking [candidate] (1.471 ms) : 0, 1471
BytebuddyAgent [baseline] (856.219 ms) : 0, 856219
BytebuddyAgent [candidate] (860.346 ms) : 0, 860346
GlobalTracer [baseline] (234.459 ms) : 0, 234459
GlobalTracer [candidate] (235.228 ms) : 0, 235228
IAST [baseline] (28.163 ms) : 0, 28163
IAST [candidate] (28.176 ms) : 0, 28176
AppSec [baseline] (30.508 ms) : 0, 30508
AppSec [candidate] (28.894 ms) : 0, 28894
Debugger [baseline] (5.806 ms) : 0, 5806
Debugger [candidate] (7.435 ms) : 0, 7435
Remote Config [baseline] (626.314 µs) : 0, 626
Remote Config [candidate] (604.932 µs) : 0, 605
Telemetry [baseline] (8.493 ms) : 0, 8493
Telemetry [candidate] (8.479 ms) : 0, 8479

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.53.0-SNAPSHOT~49558946ff, baseline=1.53.0-SNAPSHOT~02a3250561

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.059 s) : 0, 1058596
Total [baseline] (10.78 s) : 0, 10780464
Agent [candidate] (1.048 s) : 0, 1048357
Total [candidate] (10.769 s) : 0, 10768967
section appsec
Agent [baseline] (1.223 s) : 0, 1223167
Total [baseline] (10.789 s) : 0, 10788501
Agent [candidate] (1.224 s) : 0, 1224017
Total [candidate] (10.734 s) : 0, 10733976
section iast
Agent [baseline] (1.188 s) : 0, 1188482
Total [baseline] (10.952 s) : 0, 10951868
Agent [candidate] (1.18 s) : 0, 1180004
Total [candidate] (10.884 s) : 0, 10883755
section profiling
Agent [baseline] (1.207 s) : 0, 1206582
Total [baseline] (10.999 s) : 0, 10999255
Agent [candidate] (1.196 s) : 0, 1196431
Total [candidate] (10.912 s) : 0, 10912497

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.059 s	-
Agent	appsec	1.223 s	164.571 ms (15.5%)
Agent	iast	1.188 s	129.886 ms (12.3%)
Agent	profiling	1.207 s	147.986 ms (14.0%)
Total	tracing	10.78 s	-
Total	appsec	10.789 s	8.038 ms (0.1%)
Total	iast	10.952 s	171.405 ms (1.6%)
Total	profiling	10.999 s	218.792 ms (2.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.048 s	-
Agent	appsec	1.224 s	175.66 ms (16.8%)
Agent	iast	1.18 s	131.647 ms (12.6%)
Agent	profiling	1.196 s	148.074 ms (14.1%)
Total	tracing	10.769 s	-
Total	appsec	10.734 s	-34.992 ms (-0.3%)
Total	iast	10.884 s	114.788 ms (1.1%)
Total	profiling	10.912 s	143.53 ms (1.3%)

gantt
    title petclinic - break down per module: candidate=1.53.0-SNAPSHOT~49558946ff, baseline=1.53.0-SNAPSHOT~02a3250561

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.474 ms) : 0, 1474
crashtracking [candidate] (1.467 ms) : 0, 1467
BytebuddyAgent [baseline] (742.649 ms) : 0, 742649
BytebuddyAgent [candidate] (733.384 ms) : 0, 733384
GlobalTracer [baseline] (244.329 ms) : 0, 244329
GlobalTracer [candidate] (242.642 ms) : 0, 242642
AppSec [baseline] (30.431 ms) : 0, 30431
AppSec [candidate] (30.167 ms) : 0, 30167
Debugger [baseline] (6.081 ms) : 0, 6081
Debugger [candidate] (6.048 ms) : 0, 6048
Remote Config [baseline] (669.759 µs) : 0, 670
Remote Config [candidate] (665.454 µs) : 0, 665
Telemetry [baseline] (11.705 ms) : 0, 11705
Telemetry [candidate] (12.901 ms) : 0, 12901
section appsec
crashtracking [baseline] (1.45 ms) : 0, 1450
crashtracking [candidate] (1.458 ms) : 0, 1458
BytebuddyAgent [baseline] (755.269 ms) : 0, 755269
BytebuddyAgent [candidate] (755.736 ms) : 0, 755736
GlobalTracer [baseline] (235.633 ms) : 0, 235633
GlobalTracer [candidate] (235.757 ms) : 0, 235757
IAST [baseline] (23.468 ms) : 0, 23468
IAST [candidate] (23.74 ms) : 0, 23740
AppSec [baseline] (171.323 ms) : 0, 171323
AppSec [candidate] (171.388 ms) : 0, 171388
Debugger [baseline] (5.794 ms) : 0, 5794
Debugger [candidate] (5.732 ms) : 0, 5732
Remote Config [baseline] (638.294 µs) : 0, 638
Remote Config [candidate] (633.569 µs) : 0, 634
Telemetry [baseline] (8.512 ms) : 0, 8512
Telemetry [candidate] (8.407 ms) : 0, 8407
section iast
crashtracking [baseline] (1.466 ms) : 0, 1466
crashtracking [candidate] (1.457 ms) : 0, 1457
BytebuddyAgent [baseline] (857.402 ms) : 0, 857402
BytebuddyAgent [candidate] (851.655 ms) : 0, 851655
GlobalTracer [baseline] (235.434 ms) : 0, 235434
GlobalTracer [candidate] (233.262 ms) : 0, 233262
IAST [baseline] (28.734 ms) : 0, 28734
IAST [candidate] (29.137 ms) : 0, 29137
AppSec [baseline] (28.569 ms) : 0, 28569
AppSec [candidate] (26.97 ms) : 0, 26970
Debugger [baseline] (6.613 ms) : 0, 6613
Debugger [candidate] (7.558 ms) : 0, 7558
Remote Config [baseline] (631.62 µs) : 0, 632
Remote Config [candidate] (602.67 µs) : 0, 603
Telemetry [baseline] (8.428 ms) : 0, 8428
Telemetry [candidate] (8.322 ms) : 0, 8322
section profiling
crashtracking [baseline] (1.457 ms) : 0, 1457
crashtracking [candidate] (1.449 ms) : 0, 1449
BytebuddyAgent [baseline] (768.695 ms) : 0, 768695
BytebuddyAgent [candidate] (761.941 ms) : 0, 761941
GlobalTracer [baseline] (224.453 ms) : 0, 224453
GlobalTracer [candidate] (222.794 ms) : 0, 222794
AppSec [baseline] (30.325 ms) : 0, 30325
AppSec [candidate] (29.932 ms) : 0, 29932
Debugger [baseline] (7.083 ms) : 0, 7083
Debugger [candidate] (7.751 ms) : 0, 7751
Remote Config [baseline] (724.72 µs) : 0, 725
Remote Config [candidate] (692.39 µs) : 0, 692
Telemetry [baseline] (15.721 ms) : 0, 15721
Telemetry [candidate] (14.709 ms) : 0, 14709
ProfilingAgent [baseline] (108.009 ms) : 0, 108009
ProfilingAgent [candidate] (107.458 ms) : 0, 107458
Profiling [baseline] (108.643 ms) : 0, 108643
Profiling [candidate] (108.103 ms) : 0, 108103

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/experiment
git_commit_date	1756307636	1756345431
git_commit_sha	`02a3250`	`4955894`
release_version	1.53.0-SNAPSHOT~02a3250561	1.53.0-SNAPSHOT~49558946ff

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1756347010	1756347010
ci_job_id	1102640567	1102640567
ci_pipeline_id	74926707	74926707
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-39gfds9b 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-39gfds9b 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:profiling:high_load	better [-903.401µs; -599.932µs] or [-9.738%; -6.467%]	unstable [-26.490op/s; +113.490op/s] or [-5.295%; +22.687%]	8.526ms	543.750op/s	9.277ms	500.250op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~49558946ff, baseline=1.53.0-SNAPSHOT~02a3250561
    dateFormat X
    axisFormat %s
section baseline
no_agent (37.193 ms) : 36890, 37495
.   : milestone, 37193,
appsec (48.082 ms) : 47656, 48507
.   : milestone, 48082,
code_origins (45.237 ms) : 44842, 45632
.   : milestone, 45237,
iast (45.598 ms) : 45202, 45994
.   : milestone, 45598,
profiling (48.094 ms) : 47595, 48593
.   : milestone, 48094,
tracing (45.598 ms) : 45198, 45997
.   : milestone, 45598,
section candidate
no_agent (37.225 ms) : 36916, 37533
.   : milestone, 37225,
appsec (47.143 ms) : 46739, 47547
.   : milestone, 47143,
code_origins (46.206 ms) : 45812, 46600
.   : milestone, 46206,
iast (45.384 ms) : 44978, 45790
.   : milestone, 45384,
profiling (49.156 ms) : 48737, 49574
.   : milestone, 49156,
tracing (46.808 ms) : 46396, 47220
.   : milestone, 46808,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.193 ms [36.89 ms, 37.495 ms]	-
appsec	48.082 ms [47.656 ms, 48.507 ms]	10.889 ms (29.3%)
code_origins	45.237 ms [44.842 ms, 45.632 ms]	8.044 ms (21.6%)
iast	45.598 ms [45.202 ms, 45.994 ms]	8.405 ms (22.6%)
profiling	48.094 ms [47.595 ms, 48.593 ms]	10.901 ms (29.3%)
tracing	45.598 ms [45.198 ms, 45.997 ms]	8.405 ms (22.6%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.225 ms [36.916 ms, 37.533 ms]	-
appsec	47.143 ms [46.739 ms, 47.547 ms]	9.918 ms (26.6%)
code_origins	46.206 ms [45.812 ms, 46.6 ms]	8.981 ms (24.1%)
iast	45.384 ms [44.978 ms, 45.79 ms]	8.159 ms (21.9%)
profiling	49.156 ms [48.737 ms, 49.574 ms]	11.931 ms (32.1%)
tracing	46.808 ms [46.396 ms, 47.22 ms]	9.583 ms (25.7%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~49558946ff, baseline=1.53.0-SNAPSHOT~02a3250561
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.446 ms) : 4396, 4496
.   : milestone, 4446,
iast (9.327 ms) : 9173, 9481
.   : milestone, 9327,
iast_FULL (14.175 ms) : 13893, 14456
.   : milestone, 14175,
iast_GLOBAL (10.065 ms) : 9890, 10240
.   : milestone, 10065,
profiling (9.277 ms) : 9126, 9429
.   : milestone, 9277,
tracing (7.977 ms) : 7852, 8103
.   : milestone, 7977,
section candidate
no_agent (4.428 ms) : 4378, 4478
.   : milestone, 4428,
iast (9.531 ms) : 9378, 9685
.   : milestone, 9531,
iast_FULL (14.261 ms) : 13973, 14550
.   : milestone, 14261,
iast_GLOBAL (10.32 ms) : 10141, 10499
.   : milestone, 10320,
profiling (8.526 ms) : 8396, 8655
.   : milestone, 8526,
tracing (7.758 ms) : 7646, 7870
.   : milestone, 7758,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.446 ms [4.396 ms, 4.496 ms]	-
iast	9.327 ms [9.173 ms, 9.481 ms]	4.881 ms (109.8%)
iast_FULL	14.175 ms [13.893 ms, 14.456 ms]	9.728 ms (218.8%)
iast_GLOBAL	10.065 ms [9.89 ms, 10.24 ms]	5.619 ms (126.4%)
profiling	9.277 ms [9.126 ms, 9.429 ms]	4.831 ms (108.7%)
tracing	7.977 ms [7.852 ms, 8.103 ms]	3.531 ms (79.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.428 ms [4.378 ms, 4.478 ms]	-
iast	9.531 ms [9.378 ms, 9.685 ms]	5.103 ms (115.2%)
iast_FULL	14.261 ms [13.973 ms, 14.55 ms]	9.833 ms (222.0%)
iast_GLOBAL	10.32 ms [10.141 ms, 10.499 ms]	5.892 ms (133.1%)
profiling	8.526 ms [8.396 ms, 8.655 ms]	4.097 ms (92.5%)
tracing	7.758 ms [7.646 ms, 7.87 ms]	3.329 ms (75.2%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/experiment
git_commit_date	1756307636	1756345431
git_commit_sha	`02a3250`	`4955894`
release_version	1.53.0-SNAPSHOT~02a3250561	1.53.0-SNAPSHOT~49558946ff

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1756347551	1756347551
ci_job_id	1102640568	1102640568
ci_pipeline_id	74926707	74926707
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-hrdc40bz 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-hrdc40bz 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~49558946ff, baseline=1.53.0-SNAPSHOT~02a3250561
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.478 ms) : 1466, 1489
.   : milestone, 1478,
appsec (3.628 ms) : 3413, 3842
.   : milestone, 3628,
iast (2.212 ms) : 2149, 2276
.   : milestone, 2212,
iast_GLOBAL (2.252 ms) : 2189, 2315
.   : milestone, 2252,
profiling (2.072 ms) : 2020, 2124
.   : milestone, 2072,
tracing (2.033 ms) : 1984, 2082
.   : milestone, 2033,
section candidate
no_agent (1.477 ms) : 1465, 1488
.   : milestone, 1477,
appsec (3.622 ms) : 3409, 3835
.   : milestone, 3622,
iast (2.203 ms) : 2140, 2266
.   : milestone, 2203,
iast_GLOBAL (2.253 ms) : 2190, 2316
.   : milestone, 2253,
profiling (2.062 ms) : 2011, 2114
.   : milestone, 2062,
tracing (2.031 ms) : 1981, 2080
.   : milestone, 2031,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.478 ms [1.466 ms, 1.489 ms]	-
appsec	3.628 ms [3.413 ms, 3.842 ms]	2.15 ms (145.5%)
iast	2.212 ms [2.149 ms, 2.276 ms]	734.353 µs (49.7%)
iast_GLOBAL	2.252 ms [2.189 ms, 2.315 ms]	774.388 µs (52.4%)
profiling	2.072 ms [2.02 ms, 2.124 ms]	593.979 µs (40.2%)
tracing	2.033 ms [1.984 ms, 2.082 ms]	555.585 µs (37.6%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.477 ms [1.465 ms, 1.488 ms]	-
appsec	3.622 ms [3.409 ms, 3.835 ms]	2.145 ms (145.3%)
iast	2.203 ms [2.14 ms, 2.266 ms]	726.413 µs (49.2%)
iast_GLOBAL	2.253 ms [2.19 ms, 2.316 ms]	776.412 µs (52.6%)
profiling	2.062 ms [2.011 ms, 2.114 ms]	585.683 µs (39.7%)
tracing	2.031 ms [1.981 ms, 2.08 ms]	554.147 µs (37.5%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~49558946ff, baseline=1.53.0-SNAPSHOT~02a3250561
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.05 s) : 15050000, 15050000
.   : milestone, 15050000,
appsec (15.058 s) : 15058000, 15058000
.   : milestone, 15058000,
iast (18.479 s) : 18479000, 18479000
.   : milestone, 18479000,
iast_GLOBAL (18.211 s) : 18211000, 18211000
.   : milestone, 18211000,
profiling (15.616 s) : 15616000, 15616000
.   : milestone, 15616000,
tracing (14.965 s) : 14965000, 14965000
.   : milestone, 14965000,
section candidate
no_agent (14.968 s) : 14968000, 14968000
.   : milestone, 14968000,
appsec (14.95 s) : 14950000, 14950000
.   : milestone, 14950000,
iast (18.307 s) : 18307000, 18307000
.   : milestone, 18307000,
iast_GLOBAL (18.022 s) : 18022000, 18022000
.   : milestone, 18022000,
profiling (15.416 s) : 15416000, 15416000
.   : milestone, 15416000,
tracing (15.392 s) : 15392000, 15392000
.   : milestone, 15392000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.05 s [15.05 s, 15.05 s]	-
appsec	15.058 s [15.058 s, 15.058 s]	8.0 ms (0.1%)
iast	18.479 s [18.479 s, 18.479 s]	3.429 s (22.8%)
iast_GLOBAL	18.211 s [18.211 s, 18.211 s]	3.161 s (21.0%)
profiling	15.616 s [15.616 s, 15.616 s]	566.0 ms (3.8%)
tracing	14.965 s [14.965 s, 14.965 s]	-85.0 ms (-0.6%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.968 s [14.968 s, 14.968 s]	-
appsec	14.95 s [14.95 s, 14.95 s]	-18.0 ms (-0.1%)
iast	18.307 s [18.307 s, 18.307 s]	3.339 s (22.3%)
iast_GLOBAL	18.022 s [18.022 s, 18.022 s]	3.054 s (20.4%)
profiling	15.416 s [15.416 s, 15.416 s]	448.0 ms (3.0%)
tracing	15.392 s [15.392 s, 15.392 s]	424.0 ms (2.8%)

pr-commenter · 2025-08-22T20:12:09Z

Kafka / producer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	nikita-tkachenko/junit-6	sarahchen6/experiment
git_commit_date	1754670324	1756345431
git_commit_sha	`7c7286a`	`4955894`

See matching parameters

	Baseline	Candidate
ci_job_date	1756346578	1756346578
ci_job_id	1102640571	1102640571
ci_pipeline_id	74926707	74926707
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion	11.0.25	11.0.25
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.25+9-post-Ubuntu-1ubuntu122.04	11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce	same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce	same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce	same

pr-commenter · 2025-08-22T20:25:09Z

Kafka / consumer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	nikita-tkachenko/junit-6	sarahchen6/experiment
git_commit_date	1754670324	1756345431
git_commit_sha	`7c7286a`	`4955894`

See matching parameters

	Baseline	Candidate
ci_job_date	1756346806	1756346806
ci_job_id	1102640572	1102640572
ci_pipeline_id	74926707	74926707
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion	11.0.25	11.0.25
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.25+9-post-Ubuntu-1ubuntu122.04	11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume	same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume	same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume	same

…sting, and servlet tests

sarahchen6 · 2025-08-29T17:11:57Z

Closing in lieu of #9431

amarziali and others added 30 commits August 8, 2025 13:59

Support async servlet for RUM injection (#9333)

5ed2f59

* Support async servlet for RUM injection * Move isRumEnabled to instrumenter config

chore: Detect dirtiness only if gradle property is enabled (#9344)

0eeb9e5

Fixed flaky test for play-2.8: Source directory '.../routes' is not a…

893780c

… directory. (#9328)

Update Spock to 2.4-M6. (#9326)

d28a3ef

Circle CI leftovers cleanup. (#9348)

4ff8f05

Avoid triggering skipped workflows on PR (#9349)

a3836e6

Avoid NPE on featureDiscovery creation (#9353)

8d4316a

* Avoid NPE on featureDiscovery creation * Review

Improve Instrumenter API to use Context instead of Span (#9211)

f9278b6

Use magic mirror depot rather than artifactual for maven central proxy (

ee43e5f

#9351) * use magic mirror depot * remove separate cache name used for testing

Update ddapm-test-agent to latest 1.31.1. (#9359)

0f3ab90

Mark client error request with parent as flaky for `ApacheHttpAsync…

836075f

…Client5NamingV0Test`. (#9361)

Migrate internal apis to environment component (#9291)

12e10fb

* feat(env): Migrate internal apis to environment component * feat(env): Add new API to EnvironmentVariables and SystemProperties

Fix for play-2.8 'Source directory xxx is not a directory.` (#9357)

dda3d7a

Add policy for add-release-to-cloudfoundry (#9364)

fa0bdff

Updated flaky tests. (#9366)

65b2349

Fix nightly pipeline (#9368)

f15598a

* Add stage for publishing-gate * Allow publishing-gate to run when populating cache

fix(tooling): Fix release script (#9374)

6d0e730

Do not stop on git log check Fix next patch release version compute

Use mockito mocks in Runtime CSI tests (#9376)

af486f2

test: increase timeout for :dd-java-agent:agent-ci-visibility: (#9375)

db0df01

Migrate instrumentations to Context API (#9378)

5380221

* feat(dsm): Improve code style * feat(aws): Refactor AWS instrumentation around Context

mtoffl01 and others added 7 commits August 21, 2025 14:42

Address merge conflicts

47eee78

Fix Iast tests

3a8d6de

Fix karate tests

ba0a7e4

Fix play-ws tests

8d4d04b

Fix servlet tests

0f1bb18

Fix instrumentation test

9871ee0

Minor clean up

3d944f2

sarahchen6 added 15 commits August 26, 2025 11:51

Propagate karate and play-ws/karate fixes

d5f7524

Fix jaxrs tests

236519a

More http fixes

70408f6

Fix jetty test

63cd3ab

Fix vertx tests

fa5ebf0

Fix karate skip

4ed4a1a

Migrate IgnoreIf to Requires for instrumentations and core files

51f80c7

Codenarc

980d95f

Try to fix vertx tests again

135a319

Give more time for aws sqs, kafka, and vertx tests

f5177cd

Comment out karate setup test

18f1335

Comment out remaining failing play-ws test

a97ce08

Comment out remaining failing java-http-client test

2e48831

Run latestDepJava11Test on Java 11 only

066c729

Comment out remaining failing exception-profiling, instrumentation-te…

4955894

…sting, and servlet tests

sarahchen6 mentioned this pull request Aug 28, 2025

Update AgentTestRunner to use JUnit5 - Continuation pt 2 #9431

Draft

sarahchen6 closed this Aug 29, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update AgentTestRunner to use JUnit5 - Continuation #9413

Update AgentTestRunner to use JUnit5 - Continuation #9413

sarahchen6 commented Aug 22, 2025

Uh oh!

datadog-official bot commented Aug 22, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Aug 22, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Aug 22, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Aug 22, 2025 •

edited

Loading

Uh oh!

sarahchen6 commented Aug 29, 2025

Uh oh!

Uh oh!

Update AgentTestRunner to use JUnit5 - Continuation #9413

Update AgentTestRunner to use JUnit5 - Continuation #9413

Conversation

sarahchen6 commented Aug 22, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

datadog-official bot commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter bot commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

pr-commenter bot commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Kafka / producer-benchmark

Parameters

Summary

Uh oh!

pr-commenter bot commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Kafka / consumer-benchmark

Parameters

Summary

Uh oh!

sarahchen6 commented Aug 29, 2025

Uh oh!

Uh oh!

datadog-official bot commented Aug 22, 2025 •

edited

Loading

pr-commenter bot commented Aug 22, 2025 •

edited

Loading

pr-commenter bot commented Aug 22, 2025 •

edited

Loading

pr-commenter bot commented Aug 22, 2025 •

edited

Loading