fix(x/inference): derive pricing capacity from per-model throughput per tokenomics-v2 spec

[len5ky]

fix(x/inference): derive pricing capacity from per-model throughput per tokenomics-v2 spec

Part 2 of 2 for #1450 — replace the raw TotalWeight pricing-capacity proxy
with per-model throughput derived from the model registry.

Dynamic pricing utilization = load / (capacity × 5s). CacheAllModelCapacities
(inference-chain/x/inference/keeper/dynamic_pricing.go) sets
capacity := modelEpochData.TotalWeight — raw PoC nonce weight treated as
tokens/second, calibrated once against QwQ-32B. Per-nonce throughput varies
strongly by model (~7× across the three currently served), so a single
weight × 1 constant puts the 0.40–0.60 stability band at a different (and
unknown) true utilization for every model. This PR derives capacity from the
registry's per-model throughput instead, per tokenomics-v2/dynamic-pricing.md
§2.2.3.

Why this is needed — analysis excerpt

The formula. Per model, per block:

utilization = averageLoadPerBlock / (capacity × 5s)
0.40 ≤ u ≤ 0.60 → no change; below → down (≤2%/block); above → up (≤2%/block)

The numerator is a rolling ~60s average of real prompt + completion tokens of
completed inferences of that model — fine. The denominator is a proxy the code
itself flags as temporary:

// TODO: The proposal mentions copying from a `total_throughput` field, but this field
// doesn't exist in the current EpochGroupData structure. For now, we use TotalWeight
// as a proxy for capacity (tokens per second), as 1000 nonce of PoC produce aproximetely
// 1000 tokens for of QwQ-32B model. ...
capacity := modelEpochData.TotalWeight

So the ratio assumes 1 PoC nonce ≈ 1 token/s, calibrated on QwQ-32B, applied
uniformly to every model. No per-model normalization enters this weight upstream:
a sub-group's member weight is the sum of raw PocWeight (the code comment says
"no coefficient"), and the per-model weight_scale_factor applies only to rewards
and consensus voting weight, never to the pricing denominator.

The on-chain registry says the constant can't be right. Each model carries a
per-model throughput_per_nonce (mainnet values via .../inference/models_all):

model	throughput_per_nonce
MiniMaxAI/MiniMax-M2.7	5000
moonshotai/Kimi-K2.6	1500
zai-org/GLM-5.2-FP8	700

A 7× spread across the three served models. The same nonce buys very different
token throughput per model, so the weight × 1 constant makes the stability band
sit at a different true hardware utilization for each.

A second on-chain conversion disagrees by ~12–16×. The devshard gateway's
admission control converts the same raw weight into serving capacity at 5
concurrent requests per 10,000 weight per devshard instance
(devshard/cmd/devshardctl/gateway_limiter.go); across uncoordinated shards the
workable network total is roughly 32 concurrent per 10,000 weight (a rough Gonka-
team figure, not on-chain). At ~20–26 tok/s decode per Kimi stream that is
~640–830 tok/s per 10,000 weight, while the pricing keeper assumes 10,000 tok/s
for the same weight — a ~12–16× disagreement, both conversions model-uniform.

Kimi K2.6 concretely (live, epoch 326, 2026-07-14): sub-group TotalWeight is
95,298, so the keeper assumes ~95k tok/s. The price stops falling only above
~38k tok/s (U = 0.4) and rises only above ~57k tok/s (U = 0.6). Two independent
estimates of the fleet's real sustainable throughput land far below: a hardware
estimate (~65 B200-class GPUs, ~12–36k tok/s decode, overstatement ~3–8×) and a
concurrency estimate (~300 admittable streams × ~20–26 tok/s ≈ 6–8k tok/s,
overstatement ~12–16×). Either way the decode ceiling sits below the 0.40 lower
bound, so under organic decode-bound demand Kimi's utilization can never reach the
stability zone — the mechanism can only lower its price. (Sub-group weight is also
volatile, 38k–95k across recent epochs, so the thresholds swing ~2.5× per epoch.)

Caveat (both directions). Load counts prompt + completion equally and prefill
is far cheaper than decode, so a prompt-heavy workload can post completed-token
rates above decode throughput — the thresholds are unreachable for real demand yet
reachable by deliberate prompt-stuffing, making the signal manipulable upward.
Separately, rejected demand (429s under load) never becomes a completed inference,
so genuine congestion is invisible to this utilization signal.

Units are unpinned. The spec does not pin the units of throughput_per_nonce
(tokens/s per nonce? per 1000 nonces?). Whatever they are, the same nonce clearly
buys different throughput per model, so the fix must carry the per-model factor
through; this PR's specific unit reading is flagged for confirmation below.

Spec alignment

proposals/tokenomics-v2/dynamic-pricing.md §2.2.3: at epoch activation the
system copies each model subgroup's total_throughput field into the
capacity/{model_id} KV store. This PR implements exactly that (and the two
in-tree TODOs at model_assignment.go and dynamic_pricing.go), nothing more.
The fix is half-built already: EpochGroupData.TotalThroughput exists and
epoch_group.go already sums TotalThroughput += node.Throughput — but
MLNodeInfo.Throughput is never assigned, so the sum is always 0 and the
capacity path falls back to TotalWeight.

Implementation

• Populate per-node throughput (epochgroup/throughput.go, new):
  populateNodeThroughputs is called in
  epochgroup.updateEpochGroupWithNewMember, right after node selection and
  before the existing per-model TotalThroughput accumulation. This site has
  ModelKeeper.GetGovernanceModel access, runs after final model assignment,
  and needs no new keeper plumbing (chainvalidation lacks Model access;
  model_assignment's TODO is earlier and would bake the wrong model on
  reassignment).
• Formula (computeMLNodeThroughput, pure integer, math/big):
  Throughput = PocWeight × ThroughputPerNonce / UnitsOfComputePerToken.
  Zero/missing params → 0; products over int64 saturate to math.MaxInt64
  instead of wrapping.
• Assignment is unconditional. Every node's Throughput is overwritten each
  formation, including back to 0 for zero-param models. A node preserved from a
  prior epoch carries last epoch's computed value on its pointer; without an
  unconditional overwrite that stale nonzero would leak into TotalThroughput
  and silently defeat the TotalWeight fallback for legacy/zero-param models.
  (This is guarded by a dedicated test.)
• Capacity fallback chain (CacheAllModelCapacities):
  TotalThroughput > 0 → else TotalWeight proxy → else default 1000. The
  QwQ context in the comment is compressed, not deleted.

Units assumption — please confirm

The spec does not pin the units of throughput_per_nonce. This PR reads it as
compute-units/second per nonce of PoC weight, with
units_of_compute_per_token converting to tokens/second. This uses only
existing fields, is dimensionally consistent, and yields sane magnitudes:
Kimi K2.6 at 95,298 subgroup weight → 14,294 tokens/s (vs 95,298 under the
weight proxy). If the intended units differ, this is a one-line change in
computeMLNodeThroughput; the rest of the wiring is unaffected. Flagging
prominently because the whole calibration depends on this reading.

Reader impact

Non-pricing readers of MLNodeInfo.Throughput are only the dedup tiebreaker
compareMLNodePreference (module/model_assignment.go) and log-only helpers.
The tiebreaker breaks ties on Throughput only after PocWeight is equal;
its dedupMLNodesById call operates on per-participant duplicates that differ
only in Throughput (overwritten at formation) and TimeslotAllocation (reset
downstream), so whichever copy it keeps is functionally identical — selection
and ordering are unchanged. EpochGroupData.TotalThroughput has no consumer
besides the epoch_group writer and the new capacity read. Populated Throughput
also persists in ValidationWeights and is copied into next-epoch rebuilds; the
same reasoning holds there.

Rollout safety

Capacities re-cache each epoch; node throughput populates at the next formation.
The fix self-activates one epoch after a coordinated upgrade. Until then,
and for any model without registry data, the TotalWeight fallback preserves
current behavior exactly.

Tests

throughput_test.go (normal case 95298×1500/10000=14294, each zero-param path,
negative weight, int64-bound saturation, large-in-range), the single-member
subgroup sum (TotalThroughput now nonzero after formation), the stale-value
overwrite guard, and dynamic_pricing_test.go capacity-selection
(TotalThroughput > TotalWeight > default 1000).

Consensus impact

TotalThroughput and MLNodeInfo.Throughput become non-zero for models with
registry params, changing cached capacity and thus utilization/price
trajectories. Consensus-breaking; requires a coordinated upgrade.

Out of scope (disclosed)

• Multi-member TotalThroughput accumulation across successive
  updateEpochGroupWithNewMember calls is not directly unit-tested (a
  single-member, two-node sum plus capacity selection are tested separately).
• The aggregate int64 TotalThroughput += sum is not overflow-saturated
  (only the per-node compute is); bounded far below int64 in practice.
• The fallback chain intentionally mixes units (tokens/s vs raw weight) when
  throughput data is absent — inherent to the spec's fallback design.
• Capacity is cached from the effective epoch group after
  moveUpcomingToEffectiveGroup — pre-existing timing semantics, unchanged.

🤖 Generated with Claude Code