Skip to content

Commit c45d593

Browse files
committed
feat(core): usage-analytics layer in ratel-ai-core
Network-free token/cost estimation, per-source footprints, and Rollup/SourceTokens that both SDK bindings build on; plus observability trace events (ObservationKind/Status, TokensSaved). Lands ADR-0013 and the ratel-observability skill. Foundation for the per-package observability stack.
1 parent 4b53c4f commit c45d593

14 files changed

Lines changed: 768 additions & 8 deletions

File tree

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
---
2+
name: ratel-observability
3+
description: Wire Ratel's lean observability SDK (ratel-ai) into an existing Python agent so every model interaction ships a usage rollup to the dashboard. Attributes token counts to the five context sources (skills/tools/history/memory/user_input), optionally computes Ratel tool savings, and is best-effort — never breaks the host app, no-ops without a key. Invoke when instrumenting a customer's agent for Ratel cloud analytics.
4+
---
5+
6+
# /ratel-observability
7+
8+
Instruments an **existing** Python agent — one that already has its own LLM loop — with Ratel's lean cloud-analytics layer. One `track()` call per model interaction ships a usage rollup to `POST {RATEL_HOST}/api/v1/events` (default host `https://cloud.ratel.sh`), best-effort in a background thread, so the dashboard fills with real per-interaction token data. You are **not** asking the customer to adopt Ratel's tool catalog or restructure their loop; instrumentation works with whatever they already have.
9+
10+
Two invariants hold throughout, and you should reassure the customer of both:
11+
12+
- **It never breaks their app.** Export is background, batched, and best-effort — it never blocks or raises into their code.
13+
- **No key → no-op.** `get_client()` returns a singleton that silently does nothing until `RATEL_API_KEY` is set; you can add the calls now and turn ingestion on later.
14+
15+
## Procedure
16+
17+
### 1. Install and set the key
18+
19+
```bash
20+
pip install 'ratel-ai'
21+
export RATEL_API_KEY=rtl_... # the project ingest key from the dashboard
22+
export RATEL_HOST=https://cloud.ratel.sh # optional; this is the default
23+
```
24+
25+
`RATEL_API_KEY` is the only required env var. Absent it, every call below is a no-op — safe to land in the customer's codebase before they have a key.
26+
27+
### 2. Find the per-interaction boundary
28+
29+
Locate the one place the customer calls the model per agent turn — the function that builds the prompt and invokes the LLM (e.g. `client.chat.completions.create(...)`, `anthropic.messages.create(...)`, an `agent.run(...)` step). That call site is where one `track()` goes. **One `track()` per agent interaction** — not per token, not per retry.
30+
31+
If their loop calls the model several times for one logical interaction (tool-use round-trips), pick the granularity that matches a dashboard "run": usually one `track()` per user turn, summing the tokens across the inner calls.
32+
33+
### 3. Add one `track()` per interaction
34+
35+
After the model call, attribute the prompt's tokens to the five context sources Ratel reports — **`skills`, `tools`, `history`, `memory`, `user_input`** — from whatever the customer already has, and ship the rollup:
36+
37+
```python
38+
from ratel_ai import get_client
39+
40+
client = get_client() # env-configured singleton; no-op without RATEL_API_KEY
41+
42+
# ... the customer's existing model call ...
43+
response = chat.completions.create(model="claude-sonnet-4-6", messages=messages, ...)
44+
45+
client.track(
46+
tokens_by_category={
47+
"skills": skills_tokens, # system/skill instructions, playbooks
48+
"tools": tools_tokens, # tool / function definitions in the prompt
49+
"history": history_tokens, # prior turns carried forward
50+
"memory": memory_tokens, # retrieved memory / RAG context
51+
"user_input": user_input_tokens, # this turn's user message
52+
},
53+
model="claude-sonnet-4-6",
54+
output_tokens=response.usage.completion_tokens,
55+
latency_ms=elapsed_ms, # optional
56+
cost_usd=None, # optional; auto-estimated from model + tokens if omitted
57+
)
58+
```
59+
60+
Mapping the customer's reality to the five sources:
61+
62+
- Map whatever they have to the closest source; **omit or zero a source they don't use** (a bare prompt-completion agent might only have `history` + `user_input`).
63+
- Prefer **real token counts** when the provider already returns a per-segment breakdown or the customer tokenizes the prompt themselves.
64+
- **No token counts? Approximate with `len(text) // 4`** — a serviceable chars-per-token estimate. The dashboard cares about proportions and trends, so a consistent estimate is fine:
65+
66+
```python
67+
def toks(text: str) -> int:
68+
return len(text) // 4
69+
70+
tokens_by_category = {
71+
"skills": toks(system_prompt),
72+
"tools": toks(tools_json),
73+
"history": sum(toks(m["content"]) for m in prior_messages),
74+
"memory": toks(retrieved_context),
75+
"user_input": toks(user_message),
76+
}
77+
```
78+
79+
Optional fields: `latency_ms`, `cost_usd` (auto-estimated from `model` + tokens when omitted), and `occurred_at` (a `datetime`; defaults to now server-side — pass it only when backfilling).
80+
81+
### 4. (Optional) Compute tool savings if they have a tool list
82+
83+
If the customer passes a list of tool/function definitions to the model, Ratel can measure what selection would keep **out** of the prompt. Build a `ToolCatalog(observe=True)`, register their tools once, and after each `search` read `cat.last_savings`:
84+
85+
```python
86+
from ratel_ai import ToolCatalog, ExecutableTool
87+
88+
cat = ToolCatalog(observe=True)
89+
for t in customer_tools: # register each of their tools once, at startup
90+
cat.register(ExecutableTool(
91+
id=t["name"], name=t["name"], description=t["description"],
92+
input_schema=t["parameters"], execute=lambda args: {}, # metadata-only is fine for sizing
93+
))
94+
95+
# per interaction, before the model call:
96+
cat.search(user_message, top_k=5)
97+
# cat.last_savings → {"full_catalog_tokens", "selected_tokens", "tokens_saved", "top_k"}
98+
99+
client.track(
100+
tokens_by_category={...},
101+
saved_by_category={"tools": cat.last_savings["tokens_saved"]}, # if they act on selection
102+
model="claude-sonnet-4-6",
103+
output_tokens=response.usage.completion_tokens,
104+
)
105+
```
106+
107+
Two modes, pick the one that matches what the customer's loop actually does:
108+
109+
- **They send the selected top-K** to the model → feed the saving into `saved_by_category={"tools": cat.last_savings["tokens_saved"]}` (what Ratel kept out of the prompt this run).
110+
- **Observe-only** — they still send the full catalog but want to know the upside → use `saveable_by_category={"tools": cat.last_savings["tokens_saved"]}` instead (what it *could* save).
111+
112+
Skip this step entirely if the customer has no tool list — `track()` with just `tokens_by_category` is a complete rollup.
113+
114+
### 5. Flush on shutdown
115+
116+
Rollups ship from a background thread. Drain it before the process exits so nothing is lost:
117+
118+
```python
119+
client.flush() # also auto-flushed at exit
120+
```
121+
122+
`flush()` is auto-registered at interpreter exit, so an explicit call is belt-and-suspenders — add it to the customer's shutdown path (signal handler, `atexit`, web-server lifespan teardown, or the `finally` of a script) when the process may be killed before atexit runs.
123+
124+
## Conventions
125+
126+
- **Sources are exactly five**: `skills`, `tools`, `history`, `memory`, `user_input`. Don't invent new keys — unmapped context folds into the nearest of these.
127+
- **One `track()` per interaction.** A "usage rollup" is per agent turn, not per provider call or per token.
128+
- **Best-effort, never load-bearing.** Don't wrap `track()` in error handling that changes app behavior, and don't block the request path on it — the SDK already swallows failures and runs off-thread.
129+
- **No key is a feature.** Landing the instrumentation before the customer has provisioned a key is fine and expected; it stays a no-op until `RATEL_API_KEY` is set.
130+
131+
## Why this exists
132+
133+
The lean observability layer and its `POST /api/v1/events` wire contract are recorded in [ADR 0013](../../../docs/adr/0013-observability-and-analytics.md). A runnable end-to-end demo — live skill/tool suggestions plus an SDK-driven "Ratel off → on" adoption story — lives at [`src/sdk/python/examples/observability_demo.py`](../../../src/sdk/python/examples/README.md).

README.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -164,6 +164,8 @@ invoke = invoke_tool_tool(catalog)
164164
- End-to-end Pydantic AI: [examples/pydantic-ai/](examples/pydantic-ai/README.md)
165165
- Full SDK reference: [src/sdk/python/README.md](src/sdk/python/README.md)
166166

167+
The Python SDK also **measures** your agent, no catalog required: Langfuse-style observability via drop-in wrappers (`from ratel_ai.openai import OpenAI`, ships traces to your dashboard / Langfuse) and transparent token-saving tool selection (`OpenAI(select_tools=True)` BM25-prunes the `tools` you already pass). See [Observability & analytics](src/sdk/python/README.md#observability--analytics) and [Transparent tool selection](src/sdk/python/README.md#transparent-tool-selection-no-catalog).
168+
167169
**Showcase: drop Ratel between Claude Code and your existing MCP servers**
168170

169171
For the canonical "Ratel as a product" experience — managing scopes, importing from Claude Code, OAuth for HTTP upstreams, serving over stdio — use the MCP-server showcase repo:
@@ -206,7 +208,7 @@ Tool selection is the wedge, not the destination. Same catalog, same retrieval e
206208
- **v0.3.x — memories** — prior decisions, preferences, and artifacts ranked into the current turn.
207209
- **v0.4.x — context graph** — unified tools-skills-memories substrate.
208210

209-
The **Python SDK** (`pip install ratel-ai`) shipped early — a second host language on the same Rust core, at full parity with the TS SDK.
211+
The **Python SDK** (`pip install ratel-ai`) shipped early — a second host language on the same Rust core, at parity with the TS SDK. Both SDKs also carry a **lean usage-analytics client**: one `track()` per interaction ships a usage rollup to Ratel's cloud, with the token / savings / cost maths in the Rust core ([ADR 0013](docs/adr/0013-observability-and-analytics.md)).
210212

211213
Dated milestones: [`docs/roadmap.md`](docs/roadmap.md). Thesis: [`docs/overview.md`](docs/overview.md).
212214

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# 13. Observability and usage-analytics layer
2+
3+
Date: 2026-06-25
4+
5+
## Status
6+
7+
Accepted
8+
9+
Extends [ADR-0009](0009-trace-events-core-owned-schema.md).
10+
11+
## Context
12+
13+
Ratel is a context-engineering library. To show customers the value it delivers, and to power a cloud
14+
dashboard, the SDK needs to report, per agent interaction, how many tokens went into the prompt
15+
(broken down by context source), how many Ratel's selection kept out, and the model / latency / cost.
16+
This must work in both SDKs (Python and TS), must never slow down or break the host app, and must keep
17+
prompt/output text out of the core (no PII on the core's on-disk JSONL, per ADR-0009).
18+
19+
The branch first prototyped a heavier, Langfuse-shaped design: a rich trace/observation/generation
20+
tree shipped to a bespoke `POST /v1/ingest`, plus drop-in OpenAI/Anthropic wrappers, with the
21+
token/savings/cost logic living in Python. That was discarded before merge for three reasons: the
22+
analytics logic belongs in the shared Rust core (not duplicated per language, and not Python-only);
23+
the cloud dashboard already ingests a leaner per-interaction *rollup* at `POST /api/v1/events` that the
24+
prototype never spoke; and monkey-patching provider clients is a large surface to own before the
25+
pipeline is proven. This ADR records the design we kept.
26+
27+
## Decision
28+
29+
**1 — The analytics logic lives in `ratel-ai-core` (`usage` module), pure and network-free.**
30+
`estimate_tokens`, tool/skill footprints, `ToolRegistry`/`SkillRegistry::catalog_tokens` and
31+
`tokens_for`, `tokens_saved`, `estimate_cost_usd`, and the `SourceTokens` / `Rollup` types. It carries
32+
only counts and identity, never prompt/output text, so the core and its on-disk JSONL stay PII-free. It
33+
binds identically into Python (PyO3) and TS (napi); the SDKs are thin orchestration over one
34+
implementation.
35+
36+
The core trace schema (ADR-0009) also carries additive, PII-free identity/usage variants (`TraceRoot`,
37+
`ObservationStart`, `ObservationEnd`, `Generation`, `TokensSaved`) so trace consumers can correlate
38+
interactions; `ToolCatalog(observe=True)` emits `TokensSaved` on each search.
39+
40+
**2 — One SDK→cloud contract: `POST {host}/api/v1/events`.** `Authorization: Bearer <key>`; the body is
41+
a single rollup object or a JSON array of them. A rollup is one agent interaction:
42+
43+
```jsonc
44+
{
45+
"tokens_by_category": { "skills": 120, "tools": 2000, "history": 3400, "memory": 260, "user_input": 340 },
46+
"saved_by_category": { "tools": 7200 }, // kept OUT of the prompt this run (optional)
47+
"saveable_by_category": { "tools": 7000 }, // could save in observe-only mode (optional)
48+
"input_tokens": 6120, "output_tokens": 180,
49+
"model": "claude-sonnet-4-6", "latency_ms": 420,
50+
"cost_usd": 0.0231, // optional; estimated in-core from model + tokens if absent
51+
"occurred_at": "2026-06-25T09:12:00Z" // optional; server uses receipt time otherwise
52+
}
53+
```
54+
55+
The context sources are exactly `skills, tools, history, memory, user_input`.
56+
57+
**3 — The host SDK is a lean, best-effort shipper.** A background, batched client (`RatelClient.track(...)`
58+
plus `flush()`) ships the array; it retries 5xx, drops 4xx, samples by `sample_rate`, never blocks or
59+
raises into customer code, and absent an API key is a no-op. `ToolCatalog(observe=True)` records savings
60+
from the native registry onto the local trace stream and `last_savings`, ready to fold into a `track()`
61+
call.
62+
63+
**4 — No provider wrappers.** Integration is manual and documented, driven by a Claude Code skill
64+
(`/ratel-observability`), rather than monkey-patching provider clients.
65+
66+
## Consequences
67+
68+
- **The cloud renders SDK data with zero translation** — the SDK emits exactly the shape the dashboard
69+
reads, so observability is real end-to-end (the SDK seeds the dashboard's adoption story directly).
70+
- **Parity by construction** — Python and TS get the same numbers from one Rust implementation; a new
71+
language inherits the contract.
72+
- **Smaller, safer surface** — no provider-SDK dependencies or monkey-patching; the existing
73+
tool-catalog behavior is unchanged when `observe` is unset.
74+
- **Lost (for now):** a Langfuse-isomorphic per-call observation tree and automatic LLM-call capture.
75+
These can return later as an *additive* layer that emits alongside rollups, without changing this
76+
contract. Cloud-side Langfuse forwarding, if wanted, maps from rollups.
77+
- The in-core cost table is coarse and demo-grade; callers with real pricing pass `cost_usd` explicitly.

docs/lessons.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,3 +31,8 @@ Keep entries short. If a rule grows beyond ~5 lines, promote it to an ADR or a d
3131
- **Situation**: The pydantic-ai example mirrored TS's `await execute(args)` for catalog tools. In JS, `await` on a non-Promise is a harmless no-op; in Python `await {dict}` raises `TypeError`. So sync executors (the BM25 top-K stubs) crashed on every call — masked because the async gateway/MCP executors awaited fine and diagnostic mode never fired a tool.
3232
- **Rule**: `Executor` is `sync | async` by design (simple tools stay sync; MCP/HTTP tools are genuinely async — `await session.call_tool`). When porting to any language, dispatch every tool through that language's `ToolCatalog.invoke`, which must accept both kinds (Python: call then `inspect.isawaitable`; never bare-`await`). Examples must route through `invoke`, not re-derive it, and ship a model-free test that actually invokes a tool.
3333
- **Why**: `await`/promise semantics differ across languages, so the dual-mode handling belongs in one tested function, not copied to each call site. A "smoke run" that stops before any tool fires (e.g. diagnostic mode) is not coverage of the tool-call path.
34+
35+
### 2026-06-24: claim ADR numbers against the remote, not the local working copy
36+
- **Situation**: A feature branch authored ADRs 0012 and 0013 while `main` had concurrently merged a different ADR 0012 (`0012-first-class-skills.md`). The collision only surfaced at merge time, forcing a rename to 0013/0014 and a sweep of every cross-reference (code docstrings, READMEs, CHANGELOG, roadmap).
37+
- **Rule**: Before assigning an ADR number, check the highest number on the up-to-date default branch and any open PRs, not just `ls docs/adr/` locally — `git fetch origin main && git ls-tree origin/main docs/adr/`. On a long-lived branch, re-check before opening the PR. If two branches race, the later-merging one renumbers.
38+
- **Why**: ADR numbers are a shared sequential namespace allocated across concurrent branches; a local `ls` only sees your own allocations, so the number can be stale the moment someone else's branch lands.

docs/roadmap.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,10 @@ The end state: tools, skills, and memories live in one graph. One substrate, mul
6464

6565
`ratel-ai` on PyPI — the Python SDK binds the same Rust core via PyO3 (prebuilt `abi3` wheels), at full feature parity with `@ratel-ai/sdk`: `ToolRegistry`, `ToolCatalog`, gateway tool factories, `register_mcp_server`, and the core-owned trace schema. Binding strategy is locked in [ADR 0011](adr/0011-python-rust-binding-strategy.md). Originally slated for v0.5.x; landed early because the Rust core was already multi-language-ready — the FFI-binding strategy proven by the TS SDK ([ADR 0002](adr/0002-ts-rust-binding-strategy.md)) ported to PyO3 ([ADR 0011](adr/0011-python-rust-binding-strategy.md)).
6666

67+
## Usage analytics — shipped
68+
69+
A lean cloud-analytics client in both SDKs (`ratel-ai`, `@ratel-ai/sdk`): one `track()` per agent interaction ships a *usage rollup* — token spend by context source (skills / tools / history / memory / user_input), realized and potential Ratel savings, model, latency, cost — to Ratel's cloud at `POST /api/v1/events`, the exact shape the dashboard renders. Background, best-effort, never blocks or breaks the host app; absent an API key it is a no-op. The token / savings / cost maths live in the Rust core (`ratel-ai-core`) and bind identically into Python and TS, so the SDKs stay thin. `ToolCatalog(observe=True)` records the full-catalog-vs-top-K saving per search. Locked in [ADR 0013](adr/0013-observability-and-analytics.md).
70+
6771
## Out of scope (for now)
6872

6973
- **Hosted multi-tenant runtime.** Ratel is in-process by design; the v0.1.x server flavor is opt-in self-hosted, not a SaaS.

src/core/lib/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,6 @@ Built-in sinks:
6464
- `MemorySink``Vec`-backed for tests and embedder assertions (`snapshot()`, `drain()`).
6565
- `JsonlSink` — synchronous `O_APPEND` per event, mode `0600` on Unix.
6666

67-
Schema: `TraceEvent` is a tagged enum (search, index_churn, skill_search, skill_churn, skill_invoke, invoke_*, gateway_*, upstream_*, auth_*) wrapped in `TraceEnvelope { v, ts, session_id, ...event }`. The reliability profile is **query-log shaped** — best-effort, sampleable, lossy on backpressure. See ADR-0009 for the full rationale.
67+
Schema: `TraceEvent` is a tagged enum (search, index_churn, skill_search, skill_churn, skill_invoke, invoke_*, gateway_*, upstream_*, auth_*, plus the observability variants trace_root, observation_start, observation_end, generation, tokens_saved) wrapped in `TraceEnvelope { v, ts, session_id, ...event }`. The observability variants carry trace-tree identity and coarse token usage only — no prompt/output text and no `user_id` — so the on-disk JSONL stays PII-free ([ADR-0013](../../../docs/adr/0013-observability-and-analytics.md)); the rich payload lives in the host SDK's cloud stream. The reliability profile is **query-log shaped** — best-effort, sampleable, lossy on backpressure. See ADR-0009 for the full rationale.
6868

6969
The custom `TraceSink` trait lets embedders forward events to their own pipeline (HTTP, structured logger, ring buffer). The trait carries a `sample_rate()` knob (defaulting to `1.0`); the rate-limiter implementation is deferred to a later release.

0 commit comments

Comments
 (0)