Skip to content

fix(di): interaction with external wrapping contexts #13586

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Jun 10, 2025

Conversation

P403n1x87
Copy link
Contributor

@P403n1x87 P403n1x87 commented Jun 4, 2025

We fix an incompatibility within the interaction between Dynamic Instrumentation and external wrapping context (e.g. Code Origin). This was due to a loss of link between a function and its original code object due to a wrapping context being wrapped on a function not retrieved via the function discovery machinery. We make sure that any wrapping via wrapping context preserves the link between the function and its original code object. With the universal wrapping context mechanism, we don't have to worry about potential multiple wrapping, as bytecode is modified only once.

Refs: DEBUG-3970

Checklist

  • PR author has checked that all the criteria below are met
  • The PR description includes an overview of the change
  • The PR description articulates the motivation for the change
  • The change includes tests OR the PR description describes a testing strategy
  • The PR description notes risks associated with the change, if any
  • Newly-added code is easy to change
  • The change follows the library release note guidelines
  • The change includes or references documentation updates if necessary
  • Backport labels are set (if applicable)

Reviewer Checklist

  • Reviewer has checked that all the criteria below are met
  • Title is accurate
  • All changes are related to the pull request's stated goal
  • Avoids breaking API changes
  • Testing strategy adequately addresses listed risks
  • Newly-added code is easy to change
  • Release note makes sense to a user of the library
  • If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • Backport labels are set in a manner that is consistent with the release branch maintenance policy

We fix an incompatibility within the interaction between Dynamic
Instrumentation and external wrapping context (e.g. Code Origin). This
was due to a loss of link between a function and its original code
object due to a wrapping context being wrapped on a function not
retrieved via the function discovery machinery. We make sure that any
wrapping via wrapping context preserves the link between the function
and its original code object. With the universal wrapping context
mechanism, we don't have to worry about potential multiple wrapping, as
bytecode is modified only once.
@P403n1x87 P403n1x87 added the Dynamic Instrumentation Dynamic Instrumentation/Live Debugger label Jun 4, 2025
Copy link
Contributor

github-actions bot commented Jun 4, 2025

CODEOWNERS have been resolved as:

releasenotes/notes/fix-di-with-external-wrapping-context-e26b46a6fc7f6a47.yaml  @DataDog/apm-python
ddtrace/debugging/_function/discovery.py                                @DataDog/debugger-python
ddtrace/internal/utils/inspection.py                                    @DataDog/apm-core-python
ddtrace/internal/wrapping/context.py                                    @DataDog/apm-core-python
tests/debugging/conftest.py                                             @DataDog/debugger-python
tests/debugging/function/test_discovery.py                              @DataDog/debugger-python
tests/debugging/test_debugger_span_decoration.py                        @DataDog/debugger-python

Copy link
Contributor

github-actions bot commented Jun 4, 2025

Bootstrap import analysis

Comparison of import times between this PR and base.

Summary

The average import time from this PR is: 271 ± 2 ms.

The average import time from base is: 279 ± 5 ms.

The import time difference between this PR and base is: -8.2 ± 0.2 ms.

Import time breakdown

The following import paths have appeared:

ddtrace.auto 2.722 ms (1.01%)
ddtrace 2.722 ms (1.01%)
ddtrace._logger 2.722 ms (1.01%)
ddtrace.internal.telemetry 2.722 ms (1.01%)
ddtrace.settings._agent 2.722 ms (1.01%)
ddtrace.settings 2.722 ms (1.01%)
ddtrace.settings.integration 2.722 ms (1.01%)
ddtrace.vendor.debtcollector 2.722 ms (1.01%)
ddtrace.vendor 2.722 ms (1.01%)
ddtrace.internal.module 2.722 ms (1.01%)
ddtrace.internal.wrapping.context 2.722 ms (1.01%)
ddtrace.internal.utils.inspection 2.722 ms (1.01%)
ddtrace.internal.safety 2.422 ms (0.89%)
wrapt 2.107 ms (0.78%)
wrapt.__wrapt__ 0.953 ms (0.35%)
wrapt.wrappers 0.495 ms (0.18%)
wrapt._wrappers 0.269 ms (0.10%)
wrapt.decorators 0.418 ms (0.15%)
wrapt.arguments 0.115 ms (0.04%)
wrapt.importer 0.206 ms (0.08%)
wrapt.patches 0.182 ms (0.07%)
wrapt.weakrefs 0.153 ms (0.06%)
ddtrace.internal.utils.attrdict 0.131 ms (0.05%)

The following import paths have disappeared:

ddtrace.auto 4.312 ms (1.59%)
ddtrace 3.598 ms (1.33%)
ddtrace._logger 3.598 ms (1.33%)
ddtrace.internal.telemetry 3.598 ms (1.33%)
ddtrace.settings._agent 3.598 ms (1.33%)
ddtrace.settings 3.598 ms (1.33%)
ddtrace.settings.integration 3.598 ms (1.33%)
ddtrace.vendor.debtcollector 3.342 ms (1.23%)
ddtrace.vendor.debtcollector.moves 3.342 ms (1.23%)
wrapt 3.342 ms (1.23%)
wrapt.__wrapt__ 1.384 ms (0.51%)
wrapt.wrappers 0.581 ms (0.21%)
wrapt._wrappers 0.487 ms (0.18%)
wrapt.decorators 0.698 ms (0.26%)
wrapt.arguments 0.256 ms (0.09%)
wrapt.importer 0.341 ms (0.13%)
wrapt.patches 0.314 ms (0.12%)
wrapt.weakrefs 0.283 ms (0.10%)
ddtrace.internal.utils.attrdict 0.255 ms (0.09%)
ddtrace.bootstrap.sitecustomize 0.714 ms (0.26%)
ddtrace.bootstrap.preload 0.714 ms (0.26%)
ddtrace.internal.symbol_db.remoteconfig 0.714 ms (0.26%)
ddtrace.internal.symbol_db.symbols 0.714 ms (0.26%)
ddtrace.internal.utils.inspection 0.388 ms (0.14%)
ddtrace.internal.safety 0.326 ms (0.12%)

The following import paths have grown:

ddtrace.auto 1.570 ms (0.58%)
ddtrace 1.414 ms (0.52%)
ddtrace._logger 1.414 ms (0.52%)
ddtrace.internal.telemetry 1.414 ms (0.52%)
ddtrace.settings._agent 1.050 ms (0.39%)
ddtrace.settings 0.983 ms (0.36%)
ddtrace.settings.integration 0.983 ms (0.36%)
ddtrace.vendor.debtcollector 0.983 ms (0.36%)
ddtrace.vendor.debtcollector._utils 0.815 ms (0.30%)
ddtrace.vendor 0.090 ms (0.03%)
ddtrace.internal.module 0.090 ms (0.03%)
ddtrace.internal.wrapping.context 0.090 ms (0.03%)
ddtrace.vendor.debtcollector.removals 0.079 ms (0.03%)
socket 0.067 ms (0.02%)
array 0.067 ms (0.02%)
ddtrace.internal.telemetry.writer 0.279 ms (0.10%)
http.client 0.279 ms (0.10%)
ssl 0.230 ms (0.08%)
_ssl 0.230 ms (0.08%)
ddtrace.settings._otel_remapper 0.084 ms (0.03%)
ddtrace.bootstrap.sitecustomize 0.156 ms (0.06%)
ddtrace.appsec._common_module_patches 0.060 ms (0.02%)
ddtrace.appsec._asm_request_context 0.060 ms (0.02%)
ddtrace.appsec._utils 0.060 ms (0.02%)
ddtrace._trace.trace_handlers 0.040 ms (0.01%)
ddtrace.contrib.trace_utils 0.040 ms (0.01%)
ddtrace.contrib.internal.trace_utils 0.040 ms (0.01%)
shlex 0.035 ms (0.01%)
ddtrace.contrib.internal.subprocess.constants 0.021 ms (0.01%)

The following import paths have shrunk:

ddtrace.auto 4.747 ms (1.75%)
ddtrace.bootstrap.sitecustomize 2.938 ms (1.09%)
ddtrace.bootstrap.preload 2.267 ms (0.84%)
ddtrace.internal.remoteconfig.client 0.771 ms (0.28%)
ddtrace.internal.flare.flare 0.174 ms (0.06%)
logging.handlers 0.128 ms (0.05%)
ddtrace.internal.symbol_db.remoteconfig 0.169 ms (0.06%)
ddtrace.internal.symbol_db.symbols 0.169 ms (0.06%)
ddtrace.debugging._import 0.103 ms (0.04%)
ddtrace.debugging._function.discovery 0.103 ms (0.04%)
ddtrace.settings.crashtracker 0.091 ms (0.03%)
ddtrace.internal.datadog.profiling.crashtracker 0.065 ms (0.02%)
ddtrace.internal.datadog.profiling.crashtracker._crashtracker 0.065 ms (0.02%)
ddtrace.settings.profiling 0.054 ms (0.02%)
ddtrace.vendor.psutil 0.054 ms (0.02%)
ddtrace._trace.trace_handlers 0.276 ms (0.10%)
ddtrace._trace._inferred_proxy 0.077 ms (0.03%)
ddtrace.propagation.http 0.077 ms (0.03%)
ddtrace.contrib.trace_utils 0.020 ms (0.01%)
ddtrace.contrib.internal.redis_utils 0.020 ms (0.01%)
ddtrace.appsec._common_module_patches 0.207 ms (0.08%)
ddtrace.appsec._asm_request_context 0.111 ms (0.04%)
ddtrace.appsec._metrics 0.096 ms (0.04%)
ddtrace 1.809 ms (0.67%)
ddtrace._logger 1.021 ms (0.38%)
ddtrace.internal.telemetry 1.021 ms (0.38%)
ddtrace.internal.telemetry.writer 0.673 ms (0.25%)
http.client 0.565 ms (0.21%)
ssl 0.363 ms (0.13%)
http 0.118 ms (0.04%)
email.parser 0.084 ms (0.03%)
email.feedparser 0.019 ms (0.01%)
email._policybase 0.019 ms (0.01%)
email.utils 0.019 ms (0.01%)
email._parseaddr 0.019 ms (0.01%)
ddtrace.internal.telemetry.data 0.057 ms (0.02%)
ddtrace.internal.packages 0.057 ms (0.02%)
ddtrace.internal.periodic 0.036 ms (0.01%)
ddtrace.internal._threads 0.036 ms (0.01%)
ddtrace.internal.telemetry.logging 0.016 ms (0.01%)
ddtrace.settings._agent 0.348 ms (0.13%)
socket 0.147 ms (0.05%)
_socket 0.037 ms (0.01%)
ddtrace.settings 0.105 ms (0.04%)
ddtrace.settings.integration 0.105 ms (0.04%)
ddtrace.vendor.debtcollector 0.088 ms (0.03%)
ddtrace.settings._core 0.096 ms (0.04%)
envier 0.096 ms (0.04%)
envier.env 0.096 ms (0.04%)
ddtrace.internal._unpatched 0.111 ms (0.04%)
subprocess 0.086 ms (0.03%)
selectors 0.086 ms (0.03%)
ddtrace.trace 0.016 ms (0.01%)
ddtrace._trace.filters 0.016 ms (0.01%)
ddtrace._trace.processor 0.016 ms (0.01%)
ddtrace.internal.dogstatsd 0.016 ms (0.01%)
ddtrace.vendor.dogstatsd 0.016 ms (0.01%)
ddtrace.vendor.dogstatsd.base 0.016 ms (0.01%)
ddtrace.vendor.dogstatsd.format 0.016 ms (0.01%)

@pr-commenter
Copy link

pr-commenter bot commented Jun 4, 2025

Benchmarks

Benchmark execution time: 2025-06-10 10:48:21

Comparing candidate commit 2ad7a5d in PR branch fix/di-with-external-wrapping-context with baseline commit 12791bc in branch main.

Found 4 performance improvements and 3 performance regressions! Performance is the same for 502 metrics, 7 unstable metrics.

scenario:iastaspects-ljust_aspect

  • 🟩 execution_time [-2.639µs; -2.565µs] or [-20.392%; -19.823%]

scenario:iastaspects-lstrip_aspect

  • 🟩 execution_time [-2.556µs; -2.439µs] or [-19.676%; -18.778%]

scenario:iastaspects-rstrip_aspect

  • 🟩 execution_time [-1.795µs; -1.732µs] or [-14.027%; -13.533%]

scenario:iastaspects-strip_aspect

  • 🟩 execution_time [-2.523µs; -2.369µs] or [-19.365%; -18.180%]

scenario:startup-import_ddtrace_auto

  • 🟥 execution_time [+43.133ms; +45.168ms] or [+10.906%; +11.421%]

scenario:startup-import_ddtrace_auto_django

  • 🟥 execution_time [+73.146ms; +76.917ms] or [+14.550%; +15.301%]

scenario:startup-import_ddtrace_auto_flask

  • 🟥 execution_time [+61.568ms; +65.465ms] or [+11.056%; +11.756%]

@P403n1x87 P403n1x87 marked this pull request as ready for review June 4, 2025 12:41
@P403n1x87 P403n1x87 requested review from a team as code owners June 4, 2025 12:41
@P403n1x87 P403n1x87 enabled auto-merge (squash) June 4, 2025 19:01
@P403n1x87 P403n1x87 force-pushed the fix/di-with-external-wrapping-context branch from 9aeb716 to 99cc9dc Compare June 5, 2025 09:52
@P403n1x87 P403n1x87 force-pushed the fix/di-with-external-wrapping-context branch from cd0e270 to ff6bf37 Compare June 6, 2025 11:43
@P403n1x87 P403n1x87 force-pushed the fix/di-with-external-wrapping-context branch from ff6bf37 to 6e7e3d2 Compare June 6, 2025 11:57
@DataDog DataDog deleted a comment from datadog-datadog-prod-us1 bot Jun 10, 2025
@DataDog DataDog deleted a comment from datadog-datadog-prod-us1 bot Jun 10, 2025
@P403n1x87 P403n1x87 merged commit c02a068 into main Jun 10, 2025
806 of 808 checks passed
@P403n1x87 P403n1x87 deleted the fix/di-with-external-wrapping-context branch June 10, 2025 12:10
Copy link
Contributor

The backport to 2.21 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.21 2.21
# Navigate to the new working tree
cd .worktrees/backport-2.21
# Create a new branch
git switch --create backport-13586-to-2.21
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c02a0686ace8f06bdfe856f11bf3d6e3aa1755c3
# Push it to GitHub
git push --set-upstream origin backport-13586-to-2.21
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.21

Then, create a pull request where the base branch is 2.21 and the compare/head branch is backport-13586-to-2.21.

P403n1x87 added a commit that referenced this pull request Jun 10, 2025
We fix an incompatibility within the interaction between Dynamic
Instrumentation and external wrapping context (e.g. Code Origin). This
was due to a loss of link between a function and its original code
object due to a wrapping context being wrapped on a function not
retrieved via the function discovery machinery. We make sure that any
wrapping via wrapping context preserves the link between the function
and its original code object. With the universal wrapping context
mechanism, we don't have to worry about potential multiple wrapping, as
bytecode is modified only once.

- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

- [x] Reviewer has checked that all the criteria below are met
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

(cherry picked from commit c02a068)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.21 Dynamic Instrumentation Dynamic Instrumentation/Live Debugger
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants