Skip to content

fix(sidecar): strip ec_transfer_params from decode request in EPD mode#1704

Draft
revit13 wants to merge 2 commits into
llm-d:mainfrom
revit13:fixconn
Draft

fix(sidecar): strip ec_transfer_params from decode request in EPD mode#1704
revit13 wants to merge 2 commits into
llm-d:mainfrom
revit13:fixconn

Conversation

@revit13

@revit13 revit13 commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

What type of PR is this?
/kind bug

What this PR does / why we need it:

In EPD (Encoder-Prefill-Decode) mode, every decode request arrived at the
decode pod carrying ec_transfer_params, causing vLLM to log a warning per
request:

(EngineCore) WARNING Got ec_transfer_params, but no ECConnector found.
Disabling ECTransfer for this request.

Requests succeeded (200 OK), but the warning is incorrect — the decode pod
has no EC connector and does not use these params.

Root cause

handleNIXLV2 receives a request body that runPDPipeline already
serialized with ec_transfer_params present (injected by the encoder
stage). The same completionRequest map is mutated in place for both the
prefill and decode stages. Encoder-only fields were never removed before
marshaling the decode body.

Fix

Delete ec_transfer_params from completionRequest when preparing the
decode stage, before setting kv_transfer_params and marshaling the decode
body (connector_nixlv2.go).

Test

Added a Ginkgo integration test in connector_nixlv2_test.go that drives
the full EPD path (encoder backend + prefill mock + decode mock) and
asserts:

  • prefill request carries ec_transfer_params (correct — prefiller needs it)
  • decode request does not carry ec_transfer_params (the fix)

The test fails before the fix and passes after.

Which issue(s) this PR fixes:
Fixes #1703

Release note (write NONE if no user-facing change):

NONE

@revit13 revit13 requested review from a team and shmuelk as code owners June 22, 2026 10:12
@revit13 revit13 requested review from ahg-g and vMaroon June 22, 2026 10:12
@revit13 revit13 marked this pull request as draft June 22, 2026 10:12
@github-actions github-actions Bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. kind/bug Categorizes issue or PR as related to a bug. labels Jun 22, 2026
@revit13 revit13 marked this pull request as ready for review June 23, 2026 03:32
@revit13 revit13 marked this pull request as draft June 23, 2026 03:41
@revit13 revit13 marked this pull request as ready for review June 23, 2026 05:26
@revit13 revit13 marked this pull request as draft June 23, 2026 06:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/bug Categorizes issue or PR as related to a bug. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Got ec_transfer_params warning in decode logs

1 participant