Refactor: Externalize Scheduler's saturation logic and criticality-based service differentiation

[LukeAVanDrie]

This commit refactors the request processing pipeline, externalizing saturation detection and criticality-based service differentiation from the Scheduler. These responsibilities are now primarily managed by the RequestControl.Director.

This change is a preparatory step for the introduction of a new Flow Controller component, which will eventually absorb these admission control duties.

Diff base is: #808 (split out for easier reviewing)
Related to: #674

Key changes include:

• Introduced PreDispatch method to RequestControl.Director. It utilizes the SaturationDetector for admission control of non-critical requests and handles request criticality to determine if saturation checks are bypassed.
• The saturation detection logic for dropping non-critical requests is intentionally preserved within the Director at this stage. This allows the option to bypass the future Flow Controller component during its maturation, ensuring the existing saturation and sheddable request behavior can be maintained as a fallback.
• Simplified the Scheduler to focus solely on preference-based filtering and pod selection for requests that have already been admitted by the Director.
• Removed the SheddableRequestFilter and the distinct critical/sheddable filter paths from the Scheduler's internal logic. The Scheduler now applies a single, unified preference filter chain to all incoming requests.
• Updated main.go to instantiate the SaturationDetector, wiring it into the request handling flow.
• Updated tests across scheduler_test.go, director_test.go, and filter_test.go to align with the new component responsibilities, adding additional coverage where necessary.

This refactoring leads to a cleaner architecture, making the Scheduler a more focused component and centralizing initial admission control logic, while paving the way for the future Flow Controller.

This is aligned with the direction in 0683-epp-architecture-proposal and should be nearly no-op in terms of EPP behavior.

[netlify]

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	4a7de3f
🔍 Latest deploy log	https://app.netlify.com/sites/gateway-api-inference-extension/deploys/6822aa6ed948fd0008227048
😎 Deploy Preview	https://deploy-preview-805--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: LukeAVanDrie
Once this PR has been reviewed and has the lgtm label, please assign kfswain for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

Hi @LukeAVanDrie. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[ahg-g]

/ok-to-test

[LukeAVanDrie]

This change necessitates regression testing. It should be no-op, but I will get quantitative data to confirm that.

[LukeAVanDrie]

I split out the addition of the saturation detector subdir into a separate PR to be submitted before this one (#808 ). It is just unused until this PR gets submitted, wiring it up.

[k8s-ci-robot]

@LukeAVanDrie: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-gateway-api-inference-extension-test-unit-main	4a7de3f	link	true	/test pull-gateway-api-inference-extension-test-unit-main

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[k8s-ci-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.