Skip to content

feat(coordinator) Add DP-rank pinning parity to the coordinator's kv-nixl connector - #2480

Draft
roytman wants to merge 2 commits into
llm-d:mainfrom
roytman:dp2
Draft

feat(coordinator) Add DP-rank pinning parity to the coordinator's kv-nixl connector#2480
roytman wants to merge 2 commits into
llm-d:mainfrom
roytman:dp2

Conversation

@roytman

@roytman roytman commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

What type of PR is this?
/kind feature

What this PR does / why we need it:
The sidecar's nixlv2 KV connector pins a disaggregated request's prefill and decode legs to the same vLLM data-parallel rank via the x-data-parallel-rank header, guarding against SO_REUSEPORT-sharing DP backends routing the two legs to different ranks. The coordinator's equivalent connector, kv-nixl, had no DP awareness at all.

This adds the same behavior to the coordinator:

New pipeline.dp_size config option (default 1, disabled), merged into step params the same way kv_connector/ec_connector already are.
kv-nixl hashes the request ID into a deterministic rank and sets it as the x-data-parallel-rank header on both the prefill and decode requests, preferring a remote_dp_rank the prefill response echoes back when present. kv-sglang and kv-shared-storage are unaffected — their sidecar equivalents never reference DP rank either.
No kv_transfer_params body field changes; this is header-only, matching the sidecar's plain (non-MoRI-IO) NIXLv2 behavior.

Which issue(s) this PR fixes:

Fixes #2478

Release note (write NONE if no user-facing change):

Add `pipeline.dp_size` coordinator config option for the `kv-nixl` connector: pins a request's prefill and decode calls to the same vLLM data-parallel rank via the `x-data-parallel-rank` header, matching the llm-d-router sidecar's `nixlv2` connector behavior.

Signed-off-by: roytman <roytman@il.ibm.com>
@github-actions github-actions Bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. kind/feature Categorizes issue or PR as related to a new feature. area/coordinator area/docs labels Aug 20, 2026
Signed-off-by: roytman <roytman@il.ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/coordinator area/docs kind/feature Categorizes issue or PR as related to a new feature. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(coordinator) Add DP-rank pinning parity to the coordinator's kv-nixl connector

1 participant