Skip to content

Add to OTel reporting for feature parity #5549

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
Parent: [OpenTelemetry] Postgres metrics updates
wants to merge 5 commits into
base: otel-pg-poc-metrics-test
Choose a base branch
from
Open

Add to OTel reporting for feature parity #5549

wants to merge 5 commits into from

Conversation

@quinna-h
Copy link
Contributor

@quinna-h quinna-h commented Oct 17, 2025 •
edited
Loading

Motivation

Changes

How to test:
Run the scenario: ./run.sh OTEL_COLLECTOR

See feature parity output: cat logs_otel_collector/feature_parity.json

Expected:

{
  "runUrl": "https://github.com/DataDog/system-tests",
  "runDate": 1760981931.02175,
  "environment": "local",
  "testSource": "systemtests",
  "language": "otel_collector",
  "variant": "",
  "testedDependencies": [
    {
      "name": "docker.Cgroup",
      "version": "2"
    },
    {
      "name": "postgresql",
      "version": "alpine"
    }
  ],
  "configuration": {
    "collector_version": "0.138.0-nightly.4ab263f",
    "collector_image": "otel/opentelemetry-collector-contrib:nightly",
    "collector_image_commit": "4ab263fa462facd3f3a05cce69cecb37a9967e61",
    "config_file": "otelcol-config-with-postgres.yaml",
    "receivers": [
      "otlp",
      "postgresql"
    ],
    "postgresql_receiver": {
      "endpoint": "postgres:5433",
      "databases": [
        "system_tests_dbname"
      ]
    },
    "exporters": [
      "datadog",
      "debug",
      "file/traces",
      "file/metrics",
      "file/logs"
    ],
    "datadog_exporter_config": {
      "metrics": {
        "instrumentation_scope_metadata_as_tags": true
      }
    },
    "pipelines": [
      "traces",
      "metrics",
      "metrics/postgresql",
      "logs"
    ]
  },
  "scenario": "OTEL_COLLECTOR",
  "tests": [
    {
      "path": "tests/otel_postgres_metrics_e2e/test_postgres_metrics.py::Test_PostgreSQLMetricsCollection::test_postgresql_metrics_received_by_collector",
      "lineNumber": 142,
      "outcome": "passed",
      "testDeclaration": null,
      "details": null,
      "features": [
        218
      ]
    },
    {
      "path": "tests/otel_postgres_metrics_e2e/test_postgres_metrics.py::Test_BackendValidity::test_postgresql_metrics_received_by_backend",
      "lineNumber": 180,
      "outcome": "skipped",
      "testDeclaration": "irrelevant",
      "details": "irrelevant",
      "features": [
        218
      ]
    },
    {
      "path": "tests/otel_postgres_metrics_e2e/test_postgres_metrics.py::Test_Smoke::test_main",
      "lineNumber": 228,
      "outcome": "passed",
      "testDeclaration": null,
      "details": null,
      "features": [
        218
      ]
    }
  ]
}%

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • If PR title starts with [<language>], double-check that only <language> is impacted by the change
  • No system-tests internal is modified. Otherwise, I have the approval from R&P team
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added (or removed)?

@github-actions
Copy link
Contributor

github-actions bot commented Oct 17, 2025 •
edited
Loading

CODEOWNERS have been resolved as:

utils/_context/_scenarios/otel_collector.py                             @DataDog/system-tests-core
utils/interfaces/_backend.py                                            @DataDog/system-tests-core

@quinna-h quinna-h requested review from bojbrook and wantsui October 21, 2025 18:56
@quinna-h quinna-h marked this pull request as ready for review October 22, 2025 18:02
@quinna-h quinna-h requested a review from a team as a code owner October 22, 2025 18:02
@quinna-h quinna-h changed the title [wip] Add to OTel reporting for feature parity Add to OTel reporting for feature parity Oct 23, 2025
cbeauchesne
cbeauchesne requested changes Oct 24, 2025
View reviewed changes
utils/_context/_scenarios/otel_collector.py
self.collector_config_file = "./utils/build/docker/otelcol-config-with-postgres.yaml"
self.collector_container = OpenTelemetryCollectorContainer(
config_file="./utils/build/docker/otelcol-config-with-postgres.yaml",
config_file=self.collector_config_file,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll dispatch this info in two places, can you rather rename OpenTelemetryCollectorContainer._otel_config_host_path to OpenTelemetryCollectorContainer.config_file, and use in this class self.collector_container,config_file ?

utils/_context/_scenarios/otel_collector.py
result["configuration"]["pipelines"] = list(otel_config["service"]["pipelines"].keys())

except Exception as e:
logger.warning(f"Failed to parse OTel collector config: {e}")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be an error :

Suggested change
logger.warning(f"Failed to parse OTel collector config: {e}")
pytest.exit(f"Failed to parse OTel collector config: {e}", 1)

utils/_context/_scenarios/otel_collector.py
logger.warning(f"Failed to parse OTel collector config: {e}")

# Add PostgreSQL database version dynamically from container
if hasattr(self, "postgres_container") and self.postgres_container:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OtelCollectorScenario always has a postgres_container.

utils/_context/_scenarios/otel_collector.py
# Add PostgreSQL database version dynamically from container
if hasattr(self, "postgres_container") and self.postgres_container:
postgres_version = "unknown"
if hasattr(self.postgres_container, "image") and self.postgres_container.image:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't get this one, TestedContainer has always an image

utils/_context/_scenarios/otel_collector.py
if hasattr(self.postgres_container, "image") and self.postgres_container.image:
# Extract version from image name
image_name = self.postgres_container.image.name
postgres_version = image_name.split(":")[-1].split("-")[0] if ":" in image_name else "alpine"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels very fragile, there must be a way to get this version in a cleaner way

utils/_context/_scenarios/otel_collector.py
def customize_feature_parity_dashboard(self, result: dict) -> None:
result["configuration"]["collector_version"] = str(self.library.version)

if hasattr(self.collector_container, "image") and self.collector_container.image:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The is always an image here, no ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cbeauchesne cbeauchesne cbeauchesne requested changes

@wantsui wantsui Awaiting requested review from wantsui

@bojbrook bojbrook Awaiting requested review from bojbrook

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@quinna-h @cbeauchesne