[RL] Add rank-local IPC weight updates

[aoshen02]

Motivation

MoE training ranks may own different expert parameters from rollout workers. A weight-update request therefore needs different IPC metadata for each rollout worker rank without broadcasting every expert to every worker.

Design

• Keep update_info: a dictionary is shared; a list is indexed by worker rank; None skips that worker.
• Select the local entry in GPUWorker, then call the existing transfer engine unchanged.
• Extend HTTP serialization to encode CUDA IPC handles inside list entries.

This matches SGLang's rank-indexed payload model. It adds no backend capability flag, request type, transport engine branch, or new endpoint.

Validation

• Changed-file pre-commit suite passes, including Ruff, formatting, Markdownlint, and Python 3.10 mypy.
• Focused trainer-client tests: 6 passed.
• Two-host GPU crossover A/B passed with Qwen3-30B-A3B on 8 H200 GPUs per run.

Performance

With the same image, model, BF16 rollout, DeepEP topology, and two training
steps, baseline IPC updates averaged 8.753 s and rank-local updates averaged
3.889 s: 2.25x faster and 55.6% lower latency. Swapping baseline and
candidate between h200-0 and h200-1 reproduced the result on both hosts.

AI assistance

Developed with OpenAI Codex assistance; human review is required before merge.

[mergify]

Documentation preview: https://vllm--52497.org.readthedocs.build/en/52497/

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[aoshen02]

/ci run

[github-actions]

✅ Triggered Buildkite CI #84070 for commit 0a1f4bd668fb.

[mergify]

Hi @aoshen02, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.