Skip to content

Conversation

@stbenjam
Copy link
Member

This allows us to add additional important information into the JUnit BigQuery table for use in Component Readness, or AI tools for analysis (e.g. correlating test failures with intervals).

@openshift-ci-robot
Copy link

openshift-ci-robot commented Oct 28, 2025

@stbenjam: This pull request references TRT-2350 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.21.0" version, but no target version was set.

In response to this:

This allows us to add additional important information into the JUnit BigQuery table for use in Component Readness, or AI tools for analysis (e.g. correlating test failures with intervals).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Oct 28, 2025
@stbenjam stbenjam marked this pull request as draft October 28, 2025 13:07
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 28, 2025
@openshift-ci openshift-ci bot requested review from p0lyn0mial and sjenning October 28, 2025 13:08
@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 28, 2025
@stbenjam
Copy link
Member Author

/test e2e-aws-ovn

@stbenjam stbenjam marked this pull request as ready for review October 28, 2025 15:22
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 28, 2025
@openshift-ci openshift-ci bot requested a review from deads2k October 28, 2025 15:23
@neisw
Copy link
Contributor

neisw commented Nov 3, 2025

We updated TRT-2350 to also include test_repo and test_path attributes instead of doing it separately via autodl as originally requested in TRT-2326, is that something that can be added in this PR or needs to be done later?

@stbenjam stbenjam marked this pull request as draft November 12, 2025 17:12
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 12, 2025
@stbenjam stbenjam force-pushed the ote-metadata-junit branch 2 times, most recently from 247aa01 to fd4fb09 Compare November 12, 2025 17:16
@stbenjam
Copy link
Member Author

/test e2e-aws-ovn

@stbenjam stbenjam marked this pull request as ready for review November 13, 2025 00:31
@stbenjam
Copy link
Member Author

@neisw Updated to include additional info:

<testcase
  name="[Jira:config-operator][sig-api-machinery] sanity test should always pass [Suite:openshift/cluster-config-operator/conformance/parallel]"
  time="0"
  start-time="2025-11-12T20:06:48Z"
  end-time="2025-11-12T20:06:48Z"
  lifecycle="blocking"
  source-image="cluster-config-operator"
  source-binary="cluster-config-operator-tests-ext"
  source-commit="347bebcc1"
/>

@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 13, 2025
Extend JUnit test case schema with OpenShift Test Extensions (OTE)
metadata attributes.

Metadata fields added:
- Lifecycle classification (blocking/informing)
- Test execution timestamps (start-time, end-time)
- Source tracking (source-image, source-binary, source-url, source-commit)

The source fields allow tracking which container image, binary,
repository, and commit produced each test result, enabling better
traceability and debugging of test failures across different builds.
@neisw
Copy link
Contributor

neisw commented Nov 14, 2025

Ok so the commit isn't specific to the the test so it won't help with knowing where the code for the test lives.

You had mentioned that the code path in ginkgo was somewhat nondeterministic I believe. Are we saying that for TRT-2326 we won't be able to do better than get them to the repo and the commit of that repo we are testing against?

I'm also wondering if we have a good way to translate source-image="hyperkube" to openshift/kubernetes?

I'm guessing not here but we would most likely do that in sippy for something like TRT-2326

<testcase 
name="[sig-api-machinery] FieldValidation should detect unknown metadata fields in both the root and embedded object of a CR [Conformance]" 
time="4.4" 
start-time="2025-11-13T17:00:11Z" 
end-time="2025-11-13T17:00:16Z" 
lifecycle="blocking" 
source-image="hyperkube" 
source-binary="k8s-tests-ext" 
source-commit="891f5bb0306166d5625b89fc8dc86bbc8c85f549"/>

@stbenjam
Copy link
Member Author

  • A commit SHA just tells tells you the level of the binary. There is no single commit attached to a test. You would never have any idea if some deep dependency of a test was touched in any particular commit. We just report the most recent sha of the repo when the binary was built

  • I get the value of knowing a test’s source but Ginkgo’s codeLocations isn’t something I want to depend on yet. Our Ginkgo fork complicates things (we hack this codeLocations together). The list of locations is huge (some tests report 50 locations) and ambiguous without parsing the golang AST, and the format even changes depending on the build.

  • source-repo is present in the JUnit XML, but not configured in most of the binaries. We need to do something like: UPSTREAM: <carry>: OTE extension should store repo source kubernetes#2516

But knowing the image name is enough to find it from the release payload:

$ oc adm release info registry.ci.openshift.org/ocp/release:4.21.0-0.nightly-2025-11-14-050926 -o json | jq '.references.spec.tags[] | select(.name == "hyperkube")'
{
  "name": "hyperkube",
  "annotations": {
    "io.openshift.build.commit.id": "891f5bb0306166d5625b89fc8dc86bbc8c85f549",
    "io.openshift.build.commit.ref": "",
    "io.openshift.build.source-location": "https://github.com/openshift/kubernetes",
    "io.openshift.build.version-display-names": "",
    "io.openshift.build.versions": "kubernetes=1.34.1"
  },
  "from": {
    "kind": "DockerImage",
    "name": "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e31da3a2f57b7e72cfd0d089faa56a1cef3c11a227bf1ffac8ab078fea0bac41"
  },
  "generation": null,
  "importPolicy": {},
  "referencePolicy": {
    "type": ""
  }
}

@neisw
Copy link
Contributor

neisw commented Nov 14, 2025

/lgtm

It is getting us more data than we have now. Any gaps like the test location we can continue to evaluate.

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Nov 14, 2025
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Nov 14, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: neisw, stbenjam

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@stbenjam
Copy link
Member Author

/verified by presubmits

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Nov 17, 2025
@openshift-ci-robot
Copy link

@stbenjam: This PR has been marked as verified by presubmits.

In response to this:

/verified by presubmits

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@openshift-merge-bot openshift-merge-bot bot merged commit 12a5219 into openshift:main Nov 17, 2025
8 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants