Skip to content

Studio: fix the hub download rate estimator and bound long ETAs - #7679

Merged
danielhanchen merged 2 commits into
unslothai:mainfrom
ousamabenyounes:fix/issue-7667
Aug 3, 2026
Merged

Studio: fix the hub download rate estimator and bound long ETAs#7679
danielhanchen merged 2 commits into
unslothai:mainfrom
ousamabenyounes:fix/issue-7667

Conversation

@ousamabenyounes

@ousamabenyounes ousamabenyounes commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Summary

Addresses part of #7667.

  • suppress hub download ETA labels at or above 24 hours instead of rendering misleading multi-day estimates
  • retain the existing seconds, minutes, and hours formatting below the cutoff
  • cover the reported 753d 5h case and both sides of the 24-hour boundary

This is intentionally a display safeguard only. It does not change the download manager's rate estimator or claim to resolve its startup/stall stability.

Verification

  • Full local validation suite passes
  • Upstream RED: the reported case rendered 753d 5h left
  • Patched GREEN: the targeted regression test passes
  • Frontend tests: 73/73 pass
  • TypeScript typecheck passes
  • Production build passes
  • ESLint and Biome checks pass
  • Changed production lines: 100% covered (14/14)
  • Chromium UI check: 102 B/s remains visible while 753d 5h left is absent
  • Independent and OCR-assisted reviews found no blocking issues

Files changed

File Change
studio/frontend/src/features/hub/lib/format.ts Cap displayable ETA labels below 24 hours and replace time literals with named constants
studio/frontend/tests/hub-format.test.ts Add regression and cutoff-boundary coverage

AI-assisted with Codex; reviewed and tested by the contributor.


Maintainer update

Follow-up commit pushed on top of the above. The scope is now the root cause rather than the label alone:

File Change
studio/frontend/src/features/hub/download-manager/poll-loop.ts Derive the rate from the shared rolling-window estimator instead of the EMA, so ramp-up and stalls report no rate rather than a tiny positive one
studio/frontend/src/lib/transfer-stats.ts Moved out of features/chat/utils/; it is shared by the chat, training and hub surfaces
studio/frontend/src/features/hub/lib/format.ts Long ETAs render > 24h left instead of disappearing
studio/frontend/tests/hub-format.test.ts Split into cases, plus the rounding boundary and non-finite inputs
studio/frontend/tests/hub-download-rate.test.ts New: ramp-up, steady state, stall and restart gating

Measured effect and the cross-platform, cross-engine verification are in the comment below.

Cap hub ETA labels below 24 hours so unstable throughput cannot render implausible multi-day estimates.

Generated by Ora Studio
Vibe coded by ousama

Co-Authored-By: Codex <noreply@users.noreply.github.com>
@danielhanchen

Copy link
Copy Markdown
Member

Taking this over and pushed a follow-up commit.

The display cap was the right thing to flag, but on its own it left two problems: the misleading rate stayed on screen next to the hidden ETA, and a genuinely slow download lost its estimate entirely.

What changed

1. Fixed the rate at the estimator (poll-loop.ts)

The hub download manager now derives its rate from the shared rolling-window estimator in transfer-stats.ts instead of the EMA. The EMA published its very first sample verbatim, so connection ramp-up was rendered as if it were steady state, and a zero-growth poll contributed sample = 0, which decayed the rate toward zero without ever reaching it, so an idle tick read as a tiny positive rate. That is what produced 102 B/s and 753d 5h left together. The rate is now withheld until the window holds 3 samples spanning 3 seconds of forward progress, which hides both labels while the measurement is not trustworthy. This is the behaviour training-start-overlay.tsx already had.

2. Moved transfer-stats.ts to src/lib/

It is no longer chat specific, and the deep cross-feature import is blocked by the no-restricted-imports rule.

3. formatEta returns > 24h left rather than an empty string

Hiding the label meant a 6.8 GB download on a 60 KB/s line showed no estimate at all until it was roughly 7 hours in.

4. Tests

hub-format.test.ts is split into cases so one failure does not mask the rest, and now covers the rounding boundary (the cutoff effectively starts at 86399.5s because Math.round runs before the comparison), non-finite inputs, and a sweep asserting no ETA is ever reported in days. New hub-download-rate.test.ts pins the gating the progress bar relies on: ramp-up, steady state, stall, and restart.

Effect

15 download traces (521k polls) replayed through the estimator and the progress bar, with every rendered label scored against the true remaining time the simulation knows. "absurd" means a label claiming 24h or more while the transfer actually had under a day left.

configuration accurate absurd misleading hidden worst label seen
main today 517,200 3,211 855 59 23795d 15h left
this PR as submitted 256,669 0 855 263,801 23h 59m left
after this commit 517,508 3,175 527 115 > 24h left

The remaining 3,175 come almost entirely from a trace that decays exponentially to a 2 KB/s trickle, where a stable estimator genuinely does measure more than a day left. Those now render as > 24h left instead of 21d 20h left.

Verified

  • 82/82 frontend tests, typecheck, production build, eslint clean on every changed file
  • ubuntu-latest, macos-14 (arm64) and windows-latest on Node 22 and 24, with npm test run under bash, cmd.exe and PowerShell
  • Chromium, Firefox and WebKit (covering WebView2, WKWebView and WebKitGTK) over 34,705 input vectors in 7 locales, byte-identical in all of them
  • 902,333 inputs through the old and new formatter side by side: identical output for everything below the cutoff
  • Upgrade path: persisted download jobs from an older Studio rehydrate with bytesPerSec: 0, so no stale rate or ETA survives an update

@danielhanchen danielhanchen changed the title Studio: hide implausible multi-day download ETAs Studio: fix the hub download rate estimator and bound long ETAs Aug 3, 2026
danielhanchen added a commit to shimmyshimmer/unsloth-staging-4 that referenced this pull request Aug 3, 2026
@danielhanchen

Copy link
Copy Markdown
Member

Before and after in the real Studio UI.

Both panels are the actual hub download card, rendered by the real frontend against a real Studio backend. The only thing injected is the download telemetry itself, replayed from issue #7667, since a 102 B/s transfer cannot be produced on demand any other way. Pre-PR is main, post-PR is this branch, running side by side against the same backend.

The reported case: 6.8 GB repo, byte counter creeping at ~102 B/s

Reported case, before and after

795d 14h left becomes > 24h left. The rate is genuinely 102 B/s at that moment, so it stays on screen; what goes away is the multi-day figure.

The first seconds of a healthy 40 MB/s download

Ramp-up, before and after

This is the case the estimator change is really for. The download is healthy and about to run at 40 MB/s, but the first poll pair lands during connection setup. The old EMA seeds itself from that single sample and immediately renders 674 B/s and 116d 19h left. The new one reports nothing until the window holds three samples spanning three seconds of forward progress, so the card just shows the byte counters, then picks up the correct rate a second later.

@danielhanchen
danielhanchen merged commit 34e22ef into unslothai:main Aug 3, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants