fix(warmup): gate implausible prices and cap confidence at ingestion#15
fix(warmup): gate implausible prices and cap confidence at ingestion#15matheus1lva wants to merge 4 commits into
Conversation
The ingestion guard only stopped new garbage; rows already in the table (0, ~1e10) still outranked the Curve fallback when served, since ranking is source-based with no price check. Filter implausible prices in the read queries so stored garbage is never served and a good fallback wins. - queries: drop price<=0 / >MAX rows in batch+range reads (both branches) - format: clamp capConfidence to [0,1]; correct curve-confidence comment - add format unit tests; drop stray blank line in warmup
One-time cleanup of pre-guard garbage (price<=0, ~1e10) already in the table. Read path now filters these, so this only shrinks the table. Irreversible: down is a no-op.
Read path already filters implausible stored rows; the one-time DELETE is unnecessary.
murderteeth
left a comment
murderteeth
left a comment
There was a problem hiding this comment.
DefiLlama returns garbage for some legacy Curve LPs
do we know which LPs defi llama has a problem with? do we know all the addresses
Not all but the pattern have been weird, I don't remember which was the address that originated the first fix I did, but now was the stable 3 - all yearn vault tokens. This case was because the pool had such small liquidity that o think defi llama got confused. The gate is still valid despite |
can we get them all tho? my point is: if we can hardcode a list of legacy lp tokens that have bad prices, this PR becomes much simpler |
2 participants
Summary
DefiLlama returns garbage for some legacy Curve LPs —
0for a few, and ~1e10from its stale Curve-LP bug. Those values were entering the price table and
outranking the Curve fallback, so vaults priced off bad data. This guards both
the direct and Curve ingestion paths so only prices in
(0, 1_000_000]arestored, and the Curve fallback now wins when DefiLlama is unusable.
It also caps DefiLlama's confidence at 1 (it occasionally returns 1.01) and
adds
--chain/--tokenfilters to the warmup script for targeted backfills.Changes
src/format.ts: addisPlausiblePrice(finite,> 0,<= MAX_PLAUSIBLE_PRICE),capConfidence(clamp upper bound to 1), and constantsMAX_PLAUSIBLE_PRICE,CURVE_CONFIDENCE.scripts/warmup-prices.ts:warmDirectPrices; drop implausibleCurve prices in
warmCurveFallbackPrices. Track counts inguardedDirect/guardedCurveand log each skip.wrote one, so a guarded-out DefiLlama row no longer blocks the Curve price.
capConfidenceto DefiLlama writes; ship Curve writes atCURVE_CONFIDENCEwith the underlying symbol instead of nulls.--chain/--tokenparsing,filterVaults, and an error when afilter matches no vaults.
Testing
Not run. Reviewer should verify:
0or~1e10is skipped and the Curve fallback isstored instead.
--chain/--tokennarrow the vault set; an unmatched filter errors out.