refactor: move envoy subset filter to a screener plugin - #2334
Open
elevran wants to merge 2 commits into
Open
Conversation
The envoy.lb.subset_hint / x-gateway-destination-endpoint-subset handling was hardcoded in DatastoreEndpointCandidates.Locate. Promote it to a request-control Screener (pkg/epp/framework/plugins/requestcontrol/screener/envoysubset) registered as a system built-in by the runner, gated only by the existing --disable-endpoint-subset-filter CLI flag. No config-file knob is added, so the user cannot enable it in two places at once. DatastoreEndpointCandidates and CachedEndpointCandidates now always return the full pod set; the cached path no longer varies its key by request metadata. InferenceRequest gains a Metadata field that carries the Envoy dynamic-metadata map from extractMetadataValues to the screener, matching how Headers already flows. Signed-off-by: Etai Lev Ran <elevran@gmail.com>
github-actions
Bot
added
size/XL
The hermetic integration tests assert the exact ServiceUnavailable message text. With the envoysubset screener now responsible for the empty-result case (when the subset filter matches nothing), the director must emit the same "failed to find endpoint candidates for serving the request" message the old Locate path produced, so the existing test expectations keep holding. Signed-off-by: Etai Lev Ran <elevran@gmail.com>
elevran
commented
Aug 9, 2026
Collaborator
Author
There was a problem hiding this comment.
unsure if this is an envoy specific feature or any GAIE compliant gateway supports it.
Can rename path from envoysubset to gwsubset (or similar)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/kind cleanup
What this PR does / why we need it:
The envoy subset filter (envoy.lb.subset_hint / x-gateway-destination-endpoint-subset) was hardcoded in
DatastoreEndpointCandidates.Locate. This refactor moves it into a request-control Screener plugin (pkg/epp/framework/plugins/requestcontrol/screener/envoysubset) and registers it as a system built-in from the runner, gated only by the existing--disable-endpoint-subset-filterCLI flag. No config-file knob is added, so the user cannot enable it in two places at once.DatastoreEndpointCandidates and CachedEndpointCandidates now always return the full pod set; the cache key is no longer metadata-dependent.
InferenceRequestgains aMetadatafield that carries the Envoy dynamic-metadata map fromenvoy.ExtractMetadataValuesto the screener, mirroring howHeadersalready flows.The
--disable-endpoint-subset-filterCLI flag is the single switch. Behavior is unchanged.Which issue(s) this PR fixes:
NA. Cleanup following the introduction of endpoint screening plugins.
Release note (write
NONEif no user-facing change):New tests in
pkg/epp/framework/plugins/requestcontrol/screener/envoysubset/plugin_test.gocover the screenerUpdated tests in
pkg/epp/requestcontrol/candidates_test.gocover the simplified Locate and cache-key behavior