MON-4344: chore: port the test suite over from openshift-tests-private #2635

machine424 · 2025-08-01T09:04:35Z

To run using make test-ginkgo.

openshift/release#69119 will be adding the CI job that will be running that.

This is the first step, the goal is to merge Ginkgo tests into the E2E suite to get a single, unified testing framework.

I added CHANGELOG entry for this change.
No user facing changes, so no entry in CHANGELOG was needed.

machine424 · 2025-08-01T18:06:11Z

/retest-required

machine424 · 2025-08-04T07:06:26Z

/testwith openshift/cluster-monitoring-operator/main/e2e-agnostic-operator openshift/release#67778

juzhao · 2025-08-05T06:45:49Z

/retest-required

machine424 · 2025-08-05T09:27:12Z

tests are quite long, some of them fail.
Had to run them locally:

Summarizing 6 Failures:
  [FAIL] [sig-monitoring] Cluster_Observability parallel monitoring [It] Author:juzhao-Medium-43748-Ensure label namespace exists on all alerts
  /Users/machine424/personal-projects/github/cluster-monitoring-operator/test/monitoring/util/assert.go:34
  [FAIL] [sig-monitoring] Cluster_Observability parallel monitoring [It] Author:tagao-Medium-48942-validation for scrapeTimeout and relabel configs
  /Users/machine424/personal-projects/github/cluster-monitoring-operator/test/monitoring/util/assert.go:34
  [FAIL] [sig-monitoring] Cluster_Observability parallel monitoring user workload monitoring [It] Author:tagao-Low-43037-Should not have error for oc adm inspect clusteroperator monitoring command
  /Users/machine424/personal-projects/github/cluster-monitoring-operator/test/monitoring/monitoring.go:1476
  [FAIL] [sig-monitoring] Cluster_Observability parallel monitoring [It] Author:tagao-NonPreRelease-Longduration-Medium-49191-Enforce body_size_limit [Serial]
  /Users/machine424/personal-projects/github/cluster-monitoring-operator/test/monitoring/monitoring_utils.go:237
  [FAIL] [sig-monitoring] Cluster_Observability parallel monitoring [It] Author:juzhao-Medium-66736-add option to specify resource requests and limits for components [Serial]
  /Users/machine424/personal-projects/github/cluster-monitoring-operator/test/monitoring/util/assert.go:34
  [FAIL] [sig-monitoring] Cluster_Observability parallel monitoring [It] Author:tagao-High-67503-check On/Off switch of processes Collector in Node Exporter [Serial]
  /Users/machine424/personal-projects/github/cluster-monitoring-operator/test/monitoring/util/assert.go:34

Ran 89 of 90 Specs in 13934.579 seconds
FAIL! -- 83 Passed | 6 Failed | 0 Pending | 1 Skipped
--- FAIL: TestMonitoring (13935.01s)
FAIL
FAIL	github.com/openshift/cluster-monitoring-operator/test/monitoring	13935.944s
FAIL

juzhao · 2025-08-06T14:58:18Z

[FAIL] [sig-monitoring] Cluster_Observability parallel monitoring [It] Author:tagao-Medium-48942-validation for scrapeTimeout and relabel configs
/Users/machine424/personal-projects/github/cluster-monitoring-operator/test/monitoring/util/assert.go:34

the auto case code should be updated for 4.20, tracked in https://issues.redhat.com/browse/MON-4314

juzhao · 2025-08-07T11:14:31Z

[FAIL] [sig-monitoring] Cluster_Observability parallel monitoring [It] Author:juzhao-Medium-43748-Ensure label namespace exists on all alerts
/Users/machine424/personal-projects/github/cluster-monitoring-operator/test/monitoring/util/assert.go:34

checked locally, no issue, and we're going to remove this case for e2e, please ignore it if it failed

[FAIL] [sig-monitoring] Cluster_Observability parallel monitoring [It] Author:juzhao-Medium-66736-add option to specify resource requests and limits for components [Serial]
/Users/machine424/personal-projects/github/cluster-monitoring-operator/test/monitoring/util/assert.go:34

checked locally, case passed

Tai-RedHat · 2025-08-14T06:40:21Z

[FAIL] [sig-monitoring] Cluster_Observability parallel monitoring [It] Author:tagao-NonPreRelease-Longduration-Medium-49191-Enforce body_size_limit [Serial]

case updated for 4.20, tracked in https://issues.redhat.com/browse/MON-4315

…perator ginkgo tests the tests are to be added via openshift/cluster-monitoring-operator#2635

machine424 · 2025-09-10T20:17:44Z

/testwith openshift/cluster-monitoring-operator/main/ginkgo-tests openshift/release#69119

openshift-ci · 2025-09-10T20:17:49Z

@machine424, testwith: could not generate prow job. ERROR:

BUG: test 'ginkgo-tests' not found in injected config

machine424 · 2025-09-11T10:54:17Z

Makefile

@@ -29,8 +29,8 @@ BIN_DIR ?= $(shell pwd)/tmp/bin

 # Docgen related variables
 TYPES_TARGET=pkg/manifests/types.go
-K8S_VERSION=$(shell echo -n v1. &&  cat go.mod | grep -w "k8s.io/api" | awk '{ print $$2 }' | cut -d "." -f 2)
-PO_VERSION=$(shell cat go.mod | grep "github.com/prometheus-operator/prometheus-operator[^=>]\+$$" | awk '{ print $$2 }' | sort -u)
+K8S_VERSION=$(shell go list -m -json k8s.io/api | jq -r '.Version | split(".")[1] | "v1." + .')


the old one didn't handle "replace"s well

Makefile

machine424 · 2025-09-12T14:40:03Z

/retitle MON-4344: chore: port the test suite over from openshift-tests-private
/verified by 2e-agnostic-operator
which I made run the new ginkgo tests instead.

openshift-ci-robot · 2025-09-12T14:40:12Z

@machine424: This pull request references MON-4344 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.21.0" version, but no target version was set.

In response to this:

I added CHANGELOG entry for this change.

No user facing changes, so no entry in CHANGELOG was needed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2025-09-12T14:40:16Z

@machine424: This PR has been marked as verified by 2e-agnostic-operator.

In response to this:

/retitle MON-4344: chore: port the test suite over from openshift-tests-private
/verified by 2e-agnostic-operator
which I made run the new ginkgo tests instead.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2025-09-12T14:57:31Z

@machine424: This pull request references MON-4344 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.21.0" version, but no target version was set.

In response to this:

To run using make test-ginkgo.

openshift/release#69119 will be adding the CI job that will be running that.

This is the first step, the goal is to merge Ginkgo tests into the E2E suite to get a single, unified testing framework.

I added CHANGELOG entry for this change.

No user facing changes, so no entry in CHANGELOG was needed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

machine424 · 2025-09-12T15:15:52Z

/verified by e2e-agnostic-operator
which I made, in a previous version of the PR, run the new ginkgo tests instead.

machine424 · 2025-09-12T15:16:03Z

/jira refresh

openshift-ci-robot · 2025-09-12T15:16:04Z

@machine424: This PR has been marked as verified by e2e-agnostic-operator.

In response to this:

/verified by e2e-agnostic-operator
which I made, in a previous version of the PR, run the new ginkgo tests instead.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2025-09-12T15:16:07Z

@machine424: This pull request references MON-4344 which is a valid jira issue.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

machine424 · 2025-09-12T19:07:34Z

/retest-required

openshift-ci · 2025-09-12T22:09:28Z

@machine424: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/okd-scos-e2e-aws-ovn	`5cdc92c`	link	false	`/test okd-scos-e2e-aws-ovn`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Tai-RedHat · 2025-09-15T01:38:35Z

/retest-required

machine424 · 2025-09-15T09:38:43Z

/skip

jan--f · 2025-09-15T14:26:30Z

Nice! I won't pretend to have reviewed the 380k lines :) But the integration part
/lgtm

openshift-ci · 2025-09-15T14:29:08Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jan--f, machine424

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [jan--f,machine424]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci bot requested review from jan--f and slashpai August 1, 2025 09:06

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 1, 2025

machine424 changed the title ~~chore: port the test suite over from openshift-tests-private~~ WIP: chore: port the test suite over from openshift-tests-private Aug 5, 2025

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 5, 2025

machine424 mentioned this pull request Sep 10, 2025

MON-4344: chore(openshift/monitoring): add jobs to run new cluster-monitoring-operator ginkgo tests openshift/release#69119

Open

machine424 added a commit to machine424/release that referenced this pull request Sep 10, 2025

chore(openshift/monitoring): add jobs to run new cluster-monitoring-o…

a831309

…perator ginkgo tests the tests are to be added via openshift/cluster-monitoring-operator#2635

machine424 force-pushed the test-ext branch from b1dd16d to cc142c9 Compare September 10, 2025 20:12

machine424 commented Sep 11, 2025

View reviewed changes

machine424 commented Sep 12, 2025

View reviewed changes

Makefile Show resolved Hide resolved

machine424 commented Sep 12, 2025

View reviewed changes

Makefile Outdated Show resolved Hide resolved

machine424 force-pushed the test-ext branch from f7cdf63 to e3c268b Compare September 12, 2025 10:27

openshift-ci bot changed the title ~~WIP: chore: port the test suite over from openshift-tests-private~~ MON-4344: chore: port the test suite over from openshift-tests-private Sep 12, 2025

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Sep 12, 2025

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 12, 2025

openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Sep 12, 2025

machine424 force-pushed the test-ext branch from e3c268b to aa19182 Compare September 12, 2025 14:42

openshift-ci-robot removed the verified Signifies that the PR passed pre-merge verification criteria label Sep 12, 2025

chore: port the test suite over from openshift-tests-private

5cdc92c

machine424 force-pushed the test-ext branch from aa19182 to 5cdc92c Compare September 12, 2025 14:55

openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Sep 12, 2025

openshift-ci bot assigned jan--f Sep 15, 2025

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Sep 15, 2025

MON-4344: chore: port the test suite over from openshift-tests-private #2635

Are you sure you want to change the base?

MON-4344: chore: port the test suite over from openshift-tests-private #2635

Uh oh!

Conversation

machine424 commented Aug 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

machine424 commented Aug 1, 2025

Uh oh!

machine424 commented Aug 4, 2025

Uh oh!

juzhao commented Aug 5, 2025

Uh oh!

machine424 commented Aug 5, 2025

Uh oh!

juzhao commented Aug 6, 2025

Uh oh!

juzhao commented Aug 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Tai-RedHat commented Aug 14, 2025

Uh oh!

machine424 commented Sep 10, 2025

Uh oh!

openshift-ci bot commented Sep 10, 2025

Uh oh!

machine424 Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

machine424 commented Sep 12, 2025 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci-robot commented Sep 12, 2025 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci-robot commented Sep 12, 2025 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci-robot commented Sep 12, 2025 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

machine424 commented Sep 12, 2025

Uh oh!

machine424 commented Sep 12, 2025

Uh oh!

openshift-ci-robot commented Sep 12, 2025

Uh oh!

openshift-ci-robot commented Sep 12, 2025 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

machine424 commented Sep 12, 2025

Uh oh!

openshift-ci bot commented Sep 12, 2025

Uh oh!

Tai-RedHat commented Sep 15, 2025

Uh oh!

machine424 commented Sep 15, 2025

Uh oh!

jan--f commented Sep 15, 2025

Uh oh!

openshift-ci bot commented Sep 15, 2025

Uh oh!

Uh oh!

machine424 commented Aug 1, 2025 •

edited

Loading

juzhao commented Aug 7, 2025 •

edited

Loading

machine424 commented Sep 12, 2025 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Sep 12, 2025 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Sep 12, 2025 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Sep 12, 2025 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Sep 12, 2025 •

edited by openshift-ci bot

Loading