Add filter tracing decorator to scheduling pipeline#1707
Open
chethanuk wants to merge 3 commits into
Open
Conversation
github-actions
Bot
added
the
size/L
gyliu513
reviewed
Jun 22, 2026
| filteredEndpoints := endpoints | ||
| logger.V(logutil.DEBUG).Info("Before running filter plugins", "endpoints", filteredEndpoints) | ||
|
|
||
| ctx, span := tracing.Tracer(schedplugins.TracerScope).Start(ctx, "filter_endpoints", |
Member
There was a problem hiding this comment.
I'd prefer the TracerScope as llm-d-router/pkg/epp/scheduling
Author
There was a problem hiding this comment.
The span now uses TracerScope = "llm-d-router/pkg/epp/scheduling" from a local constants.go file (matching the pattern used in other packages) instead of the plugins scope. The leftover schedplugins import has also been removed.
Wrap the filter chain in runFilterPlugins in a single inline "filter_endpoints"
span via tracing.Tracer(schedplugins.TracerScope) (SpanKindInternal), recording
the input and output endpoint counts (llm_d.epp.filter.candidate_endpoints,
llm_d.epp.filter.filtered_endpoints) plus the conditional gen_ai.request.{model,id}
keys. Follows the single-step span convention from llm-d#1565 and llm-d#1693
(score_prefix_cache); filtering behavior and the per-plugin latency metric are
unchanged.
Refs: llm-d#1693, llm-d#1483
Signed-off-by: ChethanUK <chethanuk@outlook.com>
chethanuk
force-pushed
the
issue-1693-filter-span
branch
from
efd78ac to
a8095b8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Add a single OTel
filter_endpointsspan over the scheduling filter stage,carrying the input/output endpoint counts and request-correlation keys.
Implements #1693 (sub-task of #1483).
Design
Inline, single-step span, following the merged tracing convention from #1565
(the repo standard:
score_prefix_cache/pick_pd_profileare traced inline,not via a decorator) and the maintainer guidance on #1693:
runFilterPluginsstarts onefilter_endpointsspan (SpanKindInternal) viatracing.Tracer(schedplugins.TracerScope)around the whole filter chain, withdefer span.End()so it ends on the drainbreak.llm_d.epp.filter.candidate_endpoints(input count),llm_d.epp.filter.filtered_endpoints(final output count), plus the conditionalshared
gen_ai.request.model/gen_ai.request.idkeys — matchingscore_prefix_cache. Notype/nameattrs (the span name identifies theoperation).
filter.Filter(...), so inner spansnest. Filtering behavior and the per-plugin latency metric are unchanged.
Scope:
schedplugins.TracerScope=llm-d-router/pkg/epp/framework/plugins/scheduling.Review feedback addressed
TracedFilterdecoratoris removed entirely; there is no per-request wrapper.
otel.Tracer()vs the project helper (codeant): now usestracing.Tracer(...),so spans carry the BuildRef / commit-sha instrumentation metadata.
decision (single-step span,
llm_d.epp.filter.*keys, package scope).Tests
scheduler_profile_tracing_test.go(spans read as a slice via atracetestrecorder): single span with name/kind/parent and counts; one span for a
multi-filter chain (candidate = first input, filtered = final output);
drain-break still ends the span with
filtered_endpoints == 0;gen_ai.*omitted when request fields are empty; inner delegate span nests under
filter_endpoints.Gates:
go build ./...,go test ./pkg/epp/scheduling/... -race,go vet,and
make lint(new-only) all green.Refs: #1693, #1483