Skip to content

Commit 94a8f09

Browse files
committed
LLMs: document uncuration
1 parent 620d39b commit 94a8f09

1 file changed

Lines changed: 19 additions & 0 deletions

File tree

kb/modules/LLM-editorial-pubdate.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,3 +105,22 @@ Three categories:
105105
- **Bedrock** finds Anthropic editorial via `llmBedrockFindAnthropicModel` and strips unsupported interfaces - `pubDate` inherits from Anthropic.
106106
- **Ollama** is deferred: 209 entries keyed by upstream model family (e.g. `qwen3.6`, `kimi-k2`, `glm-4.6`). Each entry's `pubDate` would need to be the upstream creator's release date (Meta, Alibaba, Moonshot, Z.AI, etc.). This is large-scale upstream research; better handled in a follow-up pass once cross-vendor `pubDate` data is consolidated and reusable.
107107
- **Dynamic-only** vendors get nothing automatic. To add `pubDate` for them we'd have to seed editorial entries (which is what `fromManualMapping`'s mapping mechanism was built for); this is a per-vendor decision and out of scope for the initial rollout.
108+
109+
110+
## Uncurated-model marker (`[?]`)
111+
112+
Canonical symbols: `llmsLabelUncurated()` / `llmsIsLabelUncurated()` in `src/modules/llms/server/models.mappings.ts` (the doc comment there is the decision record).
113+
114+
**Semantics**: the `[?] ` label prefix means "the list API does not establish what this model IS" - an ids-only catalog where a video/TTS/embedding id could masquerade as chat (the FLUX-on-Modular failure, 2026-08-14). It is NOT a "not yet editorialized" badge: API-characterized 0-day arrivals stay unmarked so they publish and badge as new.
115+
116+
**Downstream effects** (why the marker must not be applied loosely):
117+
- `tools/data/llms/llm-registry-sync.ts` drops `[?]`-labeled models with `contextWindow === null` before both its local DB and the PostHog `llms_model_spec` push - they never reach big-agi.com.
118+
- The website strips bracketed segments from `[?]` labels and sinks the pubDate (no NEW badge, bottom of Released sort). Its detection is an independent regex in the website repo (`posthog.server.ts` / `llm.vendors.rankings.ts`) - keep in sync on change.
119+
120+
**Marked** (type-blind list APIs): `fromManualMapping` 'super' resolution (unknown variant of a known family), plus the 0-day fallbacks in nvidianim, modular, sakanaai, moonshot, groq, deepseek, alibaba, and native OpenAI (Azure included via `isLikelyOpenAI: true`).
121+
122+
**Unmarked** (a type/modality filter proves chat): gemini, xai, together, novita, chutesai, cerebras - each carries an in-file "no '[?]' marker (evaluated 2026-08-14)" comment. Companion rule everywhere: never invent a context window - API value or `null`.
123+
124+
**Exempt** (unknown is the norm; marking would blank whole services): OpenAI-compatible custom hosts (lenient bare `?` fallback, 128K/8K assumption), fastapi, tlusapi, lmstudio, localai, ollama.
125+
126+
**Legacy, not unified**: bedrock's ` [?]` label suffix (real context windows, so no publication effect; unifying would churn pushed specs).

0 commit comments

Comments
 (0)