Add picker tracing decorator to scheduling pipeline#1708
Open
chethanuk wants to merge 3 commits into
Open
Conversation
github-actions
Bot
added
the
size/L
gyliu513
reviewed
Jun 22, 2026
| logger.V(logutil.VERBOSE).Info("Running picker plugin", "plugin", p.picker.TypedName()) | ||
| logger.V(logutil.DEBUG).Info("Candidate pods for picking", "endpoints-weighted-score", scoredEndpoints) | ||
|
|
||
| ctx, span := tracing.Tracer(schedplugins.TracerScope).Start(ctx, "pick_endpoints", |
Member
There was a problem hiding this comment.
TraceScope should be llm-d-router/pkg/epp/scheduling
Wrap the picker call in runPickerPlugin in a single inline "pick_endpoints" span
via tracing.Tracer(schedplugins.TracerScope) (SpanKindInternal), recording the
candidate and selected endpoint counts (llm_d.epp.picker.candidate_endpoints,
llm_d.epp.picker.selected_endpoints = len(result.TargetEndpoints), nil-guarded)
plus the conditional gen_ai.request.{model,id} keys. request is threaded into
runPickerPlugin (sole caller Run) so the span carries the request keys, matching
the filter and scorer spans. Follows the single-step span convention from llm-d#1565
and llm-d#1693; picking behavior and the per-plugin latency metric are unchanged.
Refs: llm-d#1694, llm-d#1483
Signed-off-by: ChethanUK <chethanuk@outlook.com>
chethanuk
force-pushed
the
issue-1694-picker-span
branch
from
de9d2fc to
59c4b6a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Add a single OTel
pick_endpointsspan over the scheduling picker stage,carrying the candidate/selected endpoint counts and request-correlation keys.
Implements #1694 (sub-task of #1483).
Design
Inline, single-step span, following the merged tracing convention from #1565
(the repo standard:
pick_pd_profile/score_prefix_cacheare traced inline,not via a decorator) and the maintainer guidance on #1693:
runPickerPluginstarts onepick_endpointsspan (SpanKindInternal) viatracing.Tracer(schedplugins.TracerScope), withdefer span.End()so it endson every path including a nil result.
llm_d.epp.picker.candidate_endpoints(scored-endpoint count),llm_d.epp.picker.selected_endpoints(len(result.TargetEndpoints),nil-guarded), plus the conditional shared
gen_ai.request.model/gen_ai.request.idkeys — matchingscore_prefix_cache. Notype/nameattrs (the span name identifies the operation).
requestis threaded intorunPickerPlugin(sole callerRun) so the spancarries the request keys, consistent with the filter and scorer spans. The span
context is threaded into
picker.Pick(...), so inner spans nest. Pickingbehavior and the per-plugin latency metric are unchanged.
Scope:
schedplugins.TracerScope=llm-d-router/pkg/epp/framework/plugins/scheduling.Review feedback addressed
TracedPickerdecorator is removed entirely; there is no per-request wrapper.
otel.Tracer()(gemini): now usestracing.Tracer(...), sospans carry the BuildRef / commit-sha instrumentation metadata.
new test comments capture only non-obvious rationale.
decision (single-step span,
llm_d.epp.picker.*keys, package scope).Tests
scheduler_profile_picker_tracing_test.go(spans read as a slice via atracetestrecorder): single selected (candidate 3 -> selected 1) withname/kind/parent and counts; multiple selected (== 2); nil result -> span ended,
selected_endpoints == 0, no panic;gen_ai.*omitted when request fields areempty; inner delegate span nests under
pick_endpoints.Gates:
go build ./...,go test ./pkg/epp/scheduling/... -race,go vet,and
make lint(new-only) all green.Refs: #1694, #1483