Skip to content

Conversation

@wantsui
Copy link
Contributor

@wantsui wantsui commented Oct 16, 2025

Motivation

For AIDM-147, the goal is to have a way to manually send the data from the test to a non proxy backend.

Instructions:

export DD_API_KEY=<API KEY>
export DD_APP_KEY=<APP KEY>
export DD_SITE="<relevant site>"

For a specific test:

./run.sh OTEL_COLLECTOR_E2E tests/otel_postgres_metrics_e2e/ -k "test_postgresql_metrics_received_by_backend" -v

For all tests

./run.sh OTEL_COLLECTOR_E2E tests/otel_postgres_metrics_e2e/ -k -v

Changes

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • If PR title starts with [<language>], double-check that only <language> is impacted by the change
  • No system-tests internal is modified. Otherwise, I have the approval from R&P team
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added (or removed)?

@github-actions
Copy link
Contributor

github-actions bot commented Oct 16, 2025

CODEOWNERS have been resolved as:

tests/otel_postgres_metrics_e2e/test_postgres_metrics.py                @DataDog/system-tests-core
utils/_context/_scenarios/__init__.py                                   @DataDog/system-tests-core
utils/_context/_scenarios/otel_collector.py                             @DataDog/system-tests-core
utils/build/docker/otelcol-config-with-postgres.yaml                    @DataDog/system-tests-core
utils/build/docker/postgres-init-db.sh                                  @DataDog/system-tests-core
utils/interfaces/_backend.py                                            @DataDog/system-tests-core


class OtelCollectorScenario(DockerScenario):
def __init__(self, name: str):
use_proxy = os.environ.get("OTELCOLLECTOR_PROXY", "true").lower() == "true"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A given scenario can't use both mode, it would make test inconsistent.

Rather, you must declare two scenario, one that mocks the backend (and so do not perform any backend based assertions), and another one that do not mock it, and can perform backend-based assertions

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback! I'll take a look and see how splitting affects what I'm trying to do.

scenario_groups=[scenario_groups.end_to_end],
include_postgres_db=True,
use_proxy=True,
use_proxy=use_proxy,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just keep the proxy, having the logs is very helpful, and does not prevent using the real backend.

)
self.library = ComponentVersion("otel_collector", "0.0.0")

collector_env: dict[str, str | None] = {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part must be moved inside OpenTelemetryCollectorContainer constructor.

you can also move the _require_api_key from EndToEndScenario to DockerScenario to add a safe guard

}
)

if use_proxy:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not needed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use case is that the Otel Collector Postgres yaml file would have access to the environment variable to publish as a metric tag. Is there another way to grab this information already?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point is you'll use the proxy in both mode (real backend, or mocked backend). So you will never have use_proxy==False.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants