feat: EPP peer discovery via EndpointSlices - #2341
Conversation
There was a problem hiding this comment.
Pull request overview
Adds an opt-in, Kubernetes-native mechanism for discovering peer EPP replicas by watching the EPP Service’s EndpointSlice objects, storing the live peer set for future cross-replica state synchronization work.
Changes:
- Introduce
EPPPeerReconciler(controller-runtime) to aggregate ready endpoints across EndpointSlices, exclude self, and emit add/update/delete via aPeerNotifier. - Add
PeerMetadata/PeerStoreabstractions and an in-memoryMemoryPeerStore, plus unit + envtest-style integration coverage. - Wire feature gating via
--enable-peer-discovery+--epp-peer-service, and extend RBAC to allowget/list/watchondiscovery.k8s.io/endpointslices.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| test/integration/epp/epp_peer_discovery_integration_test.go | Integration test exercising the reconciler against envtest by manually creating/updating/deleting EndpointSlices. |
| pkg/epp/statesync/peerstore.go | Adds MemoryPeerStore implementation for storing live peer membership. |
| pkg/epp/statesync/peerstore_test.go | Unit tests for MemoryPeerStore upsert/delete semantics and deterministic ordering. |
| pkg/epp/server/runserver.go | Wires peer discovery reconciler into the server runner when enabled; instantiates the in-memory peer store. |
| pkg/epp/server/options.go | Adds CLI flags and validation for opt-in peer discovery configuration. |
| pkg/epp/server/options_test.go | Adds validation test ensuring peer discovery requires specifying the peer Service. |
| pkg/epp/server/controller_manager.go | Scopes the controller-runtime cache for EndpointSlices to the pool namespace + Service label when enabled. |
| pkg/epp/server/controller_config.go | Extends controller config to carry the peer Service name for cache scoping. |
| pkg/epp/framework/interface/datalayer/peer.go | Introduces peer discovery interfaces/types (PeerMetadata, PeerStore, PeerNotifier). |
| pkg/epp/controller/setup_test.go | Adds a test asserting peer discovery is never leader-election gated. |
| pkg/epp/controller/epp_peer_reconciler.go | Implements the EndpointSlice-based peer discovery reconciler and diffing against prior peer state. |
| pkg/epp/controller/epp_peer_reconciler_test.go | Unit tests for peer aggregation, self-exclusion, diffing behavior, and ID fallback logic. |
| deploy/components/inference-gateway/rbac.yaml | Adds EndpointSlice read permissions to the kustomize component Role. |
| config/charts/routerlib/templates/_rbac.yaml | Adds EndpointSlice read permissions to the Helm Role template. |
| cmd/epp/runner/runner.go | Plumbs enablement/config into controller config + runner, and captures POD_IP for self-exclusion. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| - apiGroups: ["discovery.k8s.io"] | ||
| resources: ["endpointslices"] | ||
| verbs: ["get", "watch", "list"] |
| // | ||
| // Peer discovery (active-active state synchronization). | ||
| // | ||
| EnablePeerDiscovery bool // Enables discovery of peer EPP replicas. Requires PeerServiceName. |
There was a problem hiding this comment.
do we need this flag, i mean isnt PeerServiceName enough to enable the feature
There was a problem hiding this comment.
yeah debatable, I added that because implicitly enabling when PeerServiceName is present seemed brittle, but I am not strongly opinionated
| // endpointReady reports whether an endpoint is serving. A nil Ready condition is | ||
| // treated as not ready. | ||
| func endpointReady(ep *discoveryv1.Endpoint) bool { | ||
| return ep.Conditions.Ready != nil && *ep.Conditions.Ready |
There was a problem hiding this comment.
should not check serving instead?
There was a problem hiding this comment.
incorrect doc string, Ready is when it is serving and not terminating, so I think Ready should be the appropriate one (you don't use a peer that's serving but about to terminate) -- updated
There was a problem hiding this comment.
right, the one in termination should be skipped
Signed-off-by: Edoardo Vacchi <evacchi@users.noreply.github.com>
Signed-off-by: Edoardo Vacchi <evacchi@users.noreply.github.com>
5a15125 to
b793749
Compare
Signed-off-by: Edoardo Vacchi <evacchi@users.noreply.github.com>
…verlay) Signed-off-by: Edoardo Vacchi <evacchi@users.noreply.github.com>
What type of PR is this?
/kind feature
What this PR does / why we need it:
Adds Kubernetes-based discovery of peer EPP replicas. It follows the approach outlined in #1892 (comment): a controller-runtime reconciler that watches the EPP Service's EndpointSlices to track the live set of replicas.
Follow-up: a file-based k8s-independent discovery mechanism.
PeerMetadata,PeerStore,PeerNotifier, and an in-memoryMemoryPeerStore. The reconciler populates the store; nothing consumes it in this PR (will be implemented in another follow-up).EPPPeerReconciler: aggregates the ready endpoints across the Service's EndpointSlices, excludes this replica, and drives peer add/update/delete through the notifier. It runs on every replica (leader election disabled) since each replica needs its own peer view.--enable-peer-discovery, which requires--epp-peer-servicenaming the EPP's own Service. Off by default: no behavior change and no extra RBAC unless enabled.discovery.k8s.io/endpointslicespermission to the kustomize component and the Helm chart.Out of scope (follow-ups): the distributed
CrossReplicaSyncerthat consumes the peer set, and file-based peer discovery for non-Kubernetes environments.Testing:
TargetRefID fallback;MemoryPeerStore; options validation (enable without service fails); leader-election guard (never leader-gated).Which issue(s) this PR fixes:
Part of #1892, related to #1946.
Release note (write
NONEif no user-facing change):