Skip to content

metrics: enable input metrics by default #4966

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 21, 2025

Conversation

Watson1978
Copy link
Contributor

@Watson1978 Watson1978 commented May 19, 2025

Which issue(s) this PR fixes:
Fixes #

What this PR does / why we need it:
I ran fluend with --enable-input-metrics option and read 10GB file with the in_tail plugin
There was no significant performance degradation.

So, I think it is safe to always enable --enable-input-metrics.

environment

  • ruby 3.4.3 (2025-04-14 revision d0b7e5b6a0) +PRISM [x86_64-linux]

results

  • fluentd master branch (efbb51d)
    • ruby bin/fluentd -c in_tail.conf
      • 65.939232375 seconds
    • ruby bin/fluentd --enable-input-metrics -c in_tail.conf
      • 67.776236261 seconds
  • fluentd 1.16.9 (daccbc6)
    • ruby bin/fluentd -c in_tail.conf
      • 106.312500419 seconds

config

<source>
  @type tail
  tag test
  path "#{File.expand_path '~/tmp/fluentd/access-*.log'}"
  read_from_head true
  <parse>
    @type json
  </parse>
</source>

<match **>
  @type file
  path "#{File.expand_path '~/tmp/fluentd/log'}"
</match>

script to generate 10GB

# frozen_string_literal: true
require "json"
require "fileutils"

def data_generater(str)
  {
    "message": str * 1000,
  }.to_json
end

FILE_MAX_SIZE = 10 * 1024 * 1024 * 1024
FILE_PATH = "/home/watson/tmp/fluentd/access-1.log"

dir = File.dirname(FILE_PATH)
FileUtils.mkdir_p(dir)

File.open(FILE_PATH, "w") do |f|
  data = data_generater('a')
  loop do
    f.puts data
    break if File.size(FILE_PATH) > FILE_MAX_SIZE
  end
end

Docs Changes:
fluent/fluentd-docs-gitbook#578

Release Note:
The same as the title.

@Watson1978 Watson1978 force-pushed the enable-input-metrics branch 2 times, most recently from d231091 to fc546e9 Compare May 20, 2025 05:04
@Watson1978 Watson1978 marked this pull request as ready for review May 20, 2025 05:42
@Watson1978 Watson1978 requested review from kenhys and daipom May 20, 2025 05:42
@Watson1978 Watson1978 force-pushed the enable-input-metrics branch from fc546e9 to ff5c4b8 Compare May 20, 2025 09:26
@Watson1978 Watson1978 force-pushed the enable-input-metrics branch from ff5c4b8 to 0c76db6 Compare May 21, 2025 01:43
cosmo0920
cosmo0920 previously approved these changes May 21, 2025
Copy link
Contributor

@cosmo0920 cosmo0920 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pleased this feature to be enabled by default. Thank you for your work! 👍

Copy link
Contributor

@daipom daipom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to add some tests to check if the option for disabling the feature works intendedly.

@daipom
Copy link
Contributor

daipom commented May 21, 2025

Thanks!
Could you update this test as well?

def test_system_config

@daipom daipom added this to the v1.19.0 milestone May 21, 2025
@Watson1978 Watson1978 force-pushed the enable-input-metrics branch from 8f2f8ae to b53d79f Compare May 21, 2025 06:24
Co-authored-by: Daijiro Fukuda <[email protected]>
Signed-off-by: Shizuo Fujita <[email protected]>
@Watson1978 Watson1978 force-pushed the enable-input-metrics branch from 301c625 to 9d06878 Compare May 21, 2025 06:59
Copy link
Contributor

@daipom daipom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks!

@daipom daipom changed the title metrics: enable input metrics metrics: enable input metrics by default May 21, 2025
@daipom daipom merged commit eeb384e into fluent:master May 21, 2025
12 of 13 checks passed
@Watson1978 Watson1978 deleted the enable-input-metrics branch May 22, 2025 00:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants