Skip to content

LOG-7009: remove parse and merge data from message to the log event #3033

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: release-5.9
Choose a base branch
from

Conversation

vparfonov
Copy link
Contributor

Description

This PR addresses several issues caused by problematic behavior introduced during the migration from Fluentd to Vector. These issues are related to parsing and merging data from the message into the log event, which can result in system information being overwritten, data being duplicated, or log events becoming corrupted. This fix also helps align the Syslog output with other output types because it was introduced only for Syslog output.

/cc @Clee2691 @cahartma
/assign @jcantrill

Links

@openshift-ci-robot
Copy link

openshift-ci-robot commented Apr 30, 2025

@vparfonov: This pull request references LOG-7009 which is a valid jira issue.

In response to this:

Description

This PR addresses several issues caused by problematic behavior introduced during the migration from Fluentd to Vector. These issues are related to parsing and merging data from the message into the log event, which can result in system information being overwritten, data being duplicated, or log events becoming corrupted. This fix also helps align the Syslog output with other output types because it was introduced only for Syslog output.

/cc @Clee2691 @cahartma
/assign @jcantrill

Links

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Apr 30, 2025
@openshift-ci openshift-ci bot requested review from alanconway and Clee2691 April 30, 2025 11:44
Copy link
Contributor

openshift-ci bot commented Apr 30, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: vparfonov
Once this PR has been reviewed and has the lgtm label, please assign cahartma for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@@ -131,30 +131,6 @@ var _ = Describe("[Functional][Outputs][Syslog] Functional tests", func() {
Expect(getProcID(fields)).To(Equal("myproc"))
Expect(getMsgID(fields)).To(Equal("mymsg"))
})
It("should take values of appname, procid, messageid from record", func() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try to enable these tests by introducing the JSON parse filter to demonstrate how this functionality should be work

Copy link
Contributor Author

@vparfonov vparfonov Apr 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5.9 don't have JSON parse filter, i will do it in 6.x

@@ -55,37 +55,9 @@ var _ = Describe("[Functional][Outputs][Syslog] RFC3164 tests", func() {
Expect(outputlogs[0]).To(MatchRegexp(expMatch), "Exp to find tag in received message")
Expect(outputlogs[0]).To(MatchRegexp(`{"index":.*1,.*"timestamp":.*1,.*"tag_key":.*"rec_tag"}`), "Exp to find the original message in received message")
},

Entry("should use the value from the record and include the message", "$.message.tag_key", "rec_tag", false),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment here. Also, let's add a test which demonstrates the bug no longer exists and proves we fixed it

@jcantrill
Copy link
Contributor

/hold

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 30, 2025
Copy link
Contributor

openshift-ci bot commented Apr 30, 2025

@vparfonov: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@vparfonov vparfonov marked this pull request as draft May 7, 2025 06:23
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. release/5.9
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants