Skip to content

Decision log plugin uses the upload_size_limit_bytes to represent both the compressed and uncompressed limit #7526

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sspaink opened this issue Apr 21, 2025 · 0 comments · May be fixed by #7521
Assignees
Labels
bug monitoring Issues related to decision log and status plugins

Comments

@sspaink
Copy link
Contributor

sspaink commented Apr 21, 2025

Short description

From the documentation the upload_size_limit_bytes configuration says Decision log upload size limit in bytes. OPA will chunk uploads to cap message body to this limit.. This limit is there for the user to configure the maximum size the message body will be when sent over the network. The decision log plugin sends multiple events as gzipped chunks. In practice the upload_size_limit_bytes also is being used to limit the events uncompressed size. I think this is unexpected behavior, and controlling the individual event size should be a separate configuration. A configuration like this hasn't been requested by users, so I think the best thing would be to fix upload_size_limit_bytes to only restrict the compressed size.

Steps To Reproduce

Using the latest OPA version (v1.4.2) with the following policy:

package example

allow if {
    true
}

and the following config:

services:
  fakeservice:
    url: http://localhost:8080

decision_logs:
  service: fakeservice
  reporting:
    upload_size_limit_bytes: 400

The uncompressed size of the event will be 435 bytes, which exceed the upload_size_limit_bytes. Although when the event is compressed it should fit just fine but when running OPA as a server and sending a decision:

> opa run -c opa-conf.yaml --server ./example.rego
> curl -X POST http://localhost:8181/v1/data/example/allow

the following error is logged:

{"level":"error","msg":"Log encoding failed: upload chunk size (435) exceeds upload_size_limit_bytes (400).","plugin":"decision_logs","time":"2025-05-09T16:00:55-05:00"}
@sspaink sspaink added the bug label Apr 21, 2025
@sspaink sspaink moved this from Backlog to In Progress in Open Policy Agent Apr 29, 2025
@sspaink sspaink self-assigned this Apr 29, 2025
@sspaink sspaink added monitoring Issues related to decision log and status plugins runtime and removed usability labels Apr 29, 2025
@sspaink sspaink changed the title Chunk encoder has multiple issues Decision log plugin uses the upload_size_limit_bytes to represent both the compressed and uncompressed limit May 9, 2025
@sspaink sspaink removed the runtime label May 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug monitoring Issues related to decision log and status plugins
Projects
Status: In Progress
Development

Successfully merging a pull request may close this issue.

1 participant