Skip to content

Latest commit

 

History

History
558 lines (436 loc) · 23.5 KB

File metadata and controls

558 lines (436 loc) · 23.5 KB

CLAUDE.md — Candle

Crypto market screener and alert system. Backend-first. Data integrity over features. No premature optimization.


Project overview

Candle is a crypto market screener that fetches OHLCV data from multiple exchanges, computes technical indicators, evaluates configurable screening conditions, and delivers alerts via Telegram. Designed as a portfolio project with a clear path toward a deployable product.

Current phase: v1.1 — observability, refactor, frontend improvements + Polymarket maker bot (paper mode) Completed: Phase 1 (backend core) + Phase 2 (alerts + scheduling + Railway deploy) + Phase 3 (API + frontend dashboard)


Absolute rules — never break these

  • Never place real orders on any exchange. Read-only API keys only. Paper trading until this file explicitly states otherwise.
  • Never commit secrets. All credentials live in .env. Never hardcode them. Never log them, even partially.
  • Never skip error handling on exchange calls. Exchanges fail constantly. Every ccxt call must handle NetworkError, ExchangeError, and RateLimitExceeded.
  • Never modify the database schema directly. Use Alembic migrations exclusively. No raw ALTER TABLE, no editing models without a migration.
  • Never change the project structure without updating this file first.
  • Never use synchronous I/O inside async functions. Keep the async boundary clean.

Branching model

  • dev = development
  • main = production (Railway)

Do not propose changes directly targeting main. All changes must go through dev by Pull Request.


Tech stack

Backend (active)

Layer Technology Notes
Language Python 3.12+ Type hints on every function
Exchange connector ccxt 4.x Unified API — Binance, Kraken, Coinbase
Indicators pandas-ta Built on pandas DataFrames
ORM SQLAlchemy 2.x (async) Async session everywhere
Migrations Alembic Only way to touch the schema
Scheduler APScheduler Drives fetch + screen cycles
Alerts python-telegram-bot Async client, no blocking calls
Config pydantic-settings Typed config loaded from .env
API FastAPI + uvicorn REST API with API key auth
Rate limiting slowapi Per-IP limits on all endpoints
Testing pytest + pytest-asyncio 74 tests; real DB fixtures

Infrastructure

Layer Technology
Database PostgreSQL 15+
Local dev Docker + docker-compose
Deploy Railway (EU West)

Frontend (complete)

Layer Technology
Framework Next.js 14 (App Router)
Styling Tailwind CSS + shadcn/ui
Charts TradingView Lightweight Charts v5
Data fetch SWR (auto-refresh every 30 s)

Project structure

candle/
├── CLAUDE.md                    # This file — always read before doing anything
├── README.md
├── serve.py                     # Process entrypoint — scheduler or API mode
├── .env                         # Never commit
├── .env.example                 # Committed — all keys with placeholder values
├── docker-compose.yml           # PostgreSQL for local dev
├── pyproject.toml               # Single source of truth for deps and tooling
├── alembic.ini
│
├── candle/                      # Main Python package
│   ├── __init__.py
│   ├── config.py                # Single Settings instance via pydantic-settings
│   │
│   ├── data/                    # Fetching layer — talks to exchanges
│   │   ├── fetcher.py           # ccxt wrapper — fetch_ohlcv per exchange/pair
│   │   ├── normalizer.py        # Raw ccxt output → clean DataFrame
│   │   ├── exchange_factory.py  # Builds read-only exchange instances from config
│   │   └── bit2me.py            # Custom Bit2Me connector (not in ccxt)
│   │
│   ├── indicators/              # Pure functions. Input: DataFrame. Output: Series
│   │   ├── trend.py             # EMA, SMA, MACD
│   │   ├── momentum.py          # RSI, Stochastic
│   │   └── volume.py            # VWAP, OBV
│   │
│   ├── screener/                # Evaluation engine
│   │   ├── conditions.py        # Condition primitives (crossover, threshold, etc.)
│   │   ├── rules.py             # Rule dataclass — composes conditions with AND logic
│   │   └── engine.py            # Runs rules, builds alert messages with indicator values
│   │
│   ├── alerts/                  # Notification layer
│   │   └── telegram.py          # Formats and sends alert messages
│   │
│   ├── db/                      # Database layer
│   │   ├── models.py            # SQLAlchemy ORM models — data containers only
│   │   ├── session.py           # Async engine + session factory
│   │   └── repository.py        # All DB queries go here — no raw SQL elsewhere
│   │
│   ├── api/                     # REST API
│   │   ├── app.py               # FastAPI factory — routers, rate limiter, lifespan
│   │   ├── auth.py              # X-API-Key dependency (secrets.compare_digest)
│   │   ├── limiter.py           # Shared slowapi Limiter instance
│   │   ├── schemas.py           # Pydantic response models
│   │   └── routes/
│   │       ├── pairs.py         # GET /pairs, GET /pairs/{id}/candles
│   │       └── alerts.py        # GET /alerts
│   │
│   ├── scheduler/               # Task orchestration
│   │   └── jobs.py              # APScheduler job definitions (fetch + screen cycles)
│   │
│   └── polymarket/              # Polymarket maker bot (binary UP/DOWN markets)
│       ├── bot.py               # Main orchestrator — cycle per 5-min window
│       ├── config.py            # PolymarketConfig via pydantic-settings
│       ├── signal_engine.py     # Composite signal from Binance 1m klines + WS stream
│       ├── maker_strategy.py    # Entry pricing, model probability estimation
│       ├── paper_trading.py     # Fill simulation, trade resolution
│       ├── position_tracker.py  # In-memory position tracking, P&L calculation
│       ├── risk_manager.py      # Kill switches, exposure limits, cooldown
│       ├── market_discovery.py  # Active market lookup (deterministic slug)
│       ├── notifications.py     # Telegram trade alerts
│       ├── models.py            # PolymarketTrade, PolymarketDailyStats, PolymarketSkip ORM
│       ├── repository.py        # DB access for trades, daily stats, and skips
│       ├── skip_reasons.py      # Gate decision enum with version tracking
│       ├── redeemer.py           # On-chain conditional token redemption (Builder Relayer API)
│       └── cycle_skip_reasons.py  # Cycle-level skip reason constants (pre-trade)
│
├── frontend/                    # Next.js 14 dashboard
│   └── src/
│       ├── app/
│       │   ├── page.tsx         # Dashboard — pair cards with live price/RSI
│       │   ├── api/candle/      # Server-side proxy with path/param whitelist
│       │   ├── alerts/          # Alert history table
│       │   └── pairs/[id]/      # Pair detail with candlestick chart
│       ├── components/
│       │   ├── chart/           # TradingView Lightweight Charts wrapper
│       │   ├── pairs/           # PairCard, PairsList
│       │   └── alerts/          # AlertsTable with category badges
│       └── lib/hooks/           # SWR hooks: usePairs, useCandles, useAlerts
│
├── migrations/                  # Alembic migration files
│   └── versions/
│
├── scripts/
│   ├── seed.py                  # Seeds exchanges and initial trading pairs
│   └── seed_pairs.py            # Adds pairs idempotently (get-or-create)
│
├── docs/
│   ├── refactor-report.md       # Backend code review — prioritized quick wins
│   ├── security-audit.md        # Pre-production security audit (18 findings)
│   └── polymarket/              # Polymarket bot design docs and analysis
│       ├── polymarket-analysis-current-state.md  # System behavior analysis + technical audit
│       ├── polymarket-calibration-design.md      # Calibration layer design (schema, methodology)
│       ├── polymarket-deep-dive.md
│       ├── polymarket-schema-review.md
│       ├── polymarket-simulation-redesign.md
│       ├── entry-strategy-redesign.md
│       └── conditional-token-redeem.md   # Redeem problem analysis + Builder Relayer solution
│
└── tests/
    ├── conftest.py              # Shared fixtures (test DB, mock exchange, etc.)
    ├── test_fetcher.py
    ├── test_indicators.py
    ├── test_screener.py
    ├── test_alerts.py
    ├── test_api.py
    ├── test_paper_trading.py    # Paper trader resolve_trade delta_pct tests
    └── polymarket/              # Polymarket-specific tests
        ├── test_signal_engine.py
        ├── test_maker_strategy.py
        └── test_cycle_skips.py  # repository.save_skip + bot._record_skip

Database models (Phase 1)

User          id, email, hashed_password, telegram_chat_id, language, created_at
Exchange      id, name, slug (binance | kraken | coinbase)
TradingPair   id, exchange_id, symbol (BTC/USDT), timeframe (4h | 1d), active
Candle        id, pair_id, timestamp, open, high, low, close, volume
ScreenerRule  id, user_id, name, description, conditions (JSON), active
Alert         id, rule_id, pair_id, user_id, triggered_at, message, sent
  • User is the owner of rules and alerts. Seed admin user (id=1) backfilled for existing rows.
  • Exchange, TradingPair, Candle are global — market data is shared across users.
  • ScreenerRule and Alert are per-user — alerts route to user.telegram_chat_id.

All timestamps are UTC. No exceptions.


Polymarket maker bot

Binary prediction market bot trading BTC UP/DOWN 5-minute windows on Polymarket. Deployed on Railway in paper mode — no live capital.

Architecture

Binance 1m WS → signal_engine → maker_strategy → paper_trading → position_tracker
                                      ↓                              ↓
                                 shadow EV gate                  risk_manager
                                      ↓                              ↓
                                 repository (DB)              Telegram alerts

Database models (Polymarket)

PolymarketTrade       id, timestamp, mode, market_slug, asset, interval, direction,
                      confidence, token_id, entry_price, bet_size_usdc, shares,
                      outcome, pnl, window_open_price, window_close_price, delta_pct,
                      filled, created_at, resolved_at, model_probability,
                      ev_gate_passed, skip_reason, signal_snapshot (JSON)
PolymarketDailyStats  id, date, mode, total_trades, winning_trades, losing_trades,
                      total_pnl, max_drawdown, win_rate, avg_confidence
PolymarketSkip        id, timestamp, mode, asset, interval, skip_reason,
                      market_slug (nullable), detail (nullable)

outcome domain for PolymarketTrade: "win" | "loss" | "unfilled" | "pending"

  • "unfilled" — trade attempted, limit order not crossed (filled=False, pnl=0)
  • "pending" — initial state before resolution
  • "win" / "loss" — resolved filled trades

PolymarketSkip vs PolymarketTrade: A skip row means the bot never reached the order-placement stage — no trade row exists. An outcome="unfilled" trade row means the order was placed but not crossed.

Key config (Railway runtime)

Variable Value Purpose
POLY_MODE paper No live orders
POLY_ASSET btc BTC 5-min windows
POLY_INTERVAL 5 Window size in minutes
POLY_ENTRY_WINDOW_SECS 60 Signal computed 60s before close
POLY_MIN_CONFIDENCE 0.25 Minimum confidence to enter
POLY_TARGET_PRICE_MIN 0.50 Lower bound for entry price range
POLY_TARGET_PRICE_MAX 0.65 Upper bound for entry price range
POLY_MAX_ENTRY_PRICE 0.65 Hard cap on entry price (validated ≥ target_max)
POLY_CALIBRATION_SHRINKAGE 0.50 Shrinkage toward 0.50 prior

Current status

  • Signal engine: operational (delta 50%, RSI 20%, EMA 20%, volume 10%)
  • EV gate: shadow mode — computed and stored, not enforced
  • Calibration: pre-empirical (shrinkage is a guess, not data-driven)
  • Config validation: @model_validator rejects incoherent price ranges at startup
  • Startup logging: full effective config logged on boot (all 11 runtime fields)
  • Skip tracking: cycle-level skips persisted to polymarket_skips (6 reasons; see cycle_skip_reasons.py)
  • Daily risk reset: reset_daily() fires at UTC midnight, resetting daily_pnl only (kill switch and consecutive losses persist)
  • Analysis: docs/polymarket/polymarket-analysis-current-state.md (includes technical audit)
  • Next milestone: 200+ post-fix trades → confidence-bucketed win rate analysis

Supported exchanges

Exchange Slug Connector Notes
Binance binance ccxt Primary — deepest liquidity, USDT pairs
Kraken kraken ccxt EU-friendly, reliable API
Coinbase coinbase ccxt US pairs, good for BTC/ETH
Bit2Me bit2me custom EUR/USDC pairs, 2 req/s limit, max 288 candles

Exchange instances are read-only. API keys are optional for public OHLCV data. Bit2Me uses a custom connector (candle/data/bit2me.py) since it is not supported by ccxt. If keys are provided, they must only have read permissions.


Supported timeframes

1h · 4h · 1d

Start with 4h for swing screening. Anything below 1h is out of scope for Phase 1.


Screener conditions (Phase 1 primitives)

  • ema_crossover(fast, slow) — fast EMA crosses above slow EMA
  • rsi_range(min, max) — RSI within a given range
  • price_above_vwap() — close above VWAP
  • volume_spike(multiplier) — volume N× the rolling average

Conditions are composable with AND logic. OR logic comes in Phase 2.


Environment variables

# .env.example

# PostgreSQL
DATABASE_URL=postgresql+asyncpg://user:password@localhost:5432/candle

# API
API_KEY=                         # Shared secret for X-API-Key header. Empty = auth disabled (local dev only)

# Exchanges (all optional for public OHLCV data)
BINANCE_API_KEY=
BINANCE_API_SECRET=
KRAKEN_API_KEY=
KRAKEN_API_SECRET=
COINBASE_API_KEY=
COINBASE_API_SECRET=

# Telegram
TELEGRAM_BOT_TOKEN=
TELEGRAM_CHAT_ID=                    # market alerts — delivered to traders
TELEGRAM_ADMIN_CHAT_ID=              # operator alerts — job failures, heartbeats (can equal TELEGRAM_CHAT_ID in dev)

# Scheduler
FETCH_INTERVAL_MINUTES=60
SCREEN_INTERVAL_MINUTES=60
DEFAULT_TIMEFRAME=4h
ALERT_DEDUP_HOURS=4              # Suppress re-alerts for the same rule+pair within N hours

Language conventions

  • All code in English: variable names, function names, class names, comments
  • All docstrings in English
  • All log messages in English
  • Commit messages in English following Conventional Commits: feat:, fix:, chore:, refactor:, test:, docs:
  • README and internal technical docs in English
  • CLAUDE.md can be updated in Spanish by the developer

Coding conventions

  • Type hints on every function signature. No bare dict — use TypedDict or Pydantic models.
  • Async by default in data, db, and alerts layers. Sync only for pure indicator functions (CPU-bound, no I/O).
  • No business logic in models. Models are data containers. Logic lives in repositories or services.
  • Repository pattern for all DB access. No SQLAlchemy queries outside db/repository.py.
  • Indicators are pure functions. Same input always produces same output. No side effects. No DB calls. No logging inside indicator functions.
  • One responsibility per module. fetcher.py fetches. normalizer.py normalizes. They do not know about each other.
  • Fail loudly in development, fail gracefully in production. Use environment-aware error handling via the config.

What NOT to do — common mistakes to avoid

  • Do not use requests or any sync HTTP library. Use ccxt's async client.
  • Do not store raw ccxt responses in the database. Always normalize first.
  • Do not compute indicators inside the fetcher. Keep layers strictly separate.
  • Do not hardcode symbols or timeframes anywhere. They come from the DB or config.
  • Do not send a Telegram message directly from the screener. Go through the alerts layer.
  • Do not create a new DB session per query. Use the session factory from db/session.py.

Development workflow

# Start local DB
docker-compose up -d postgres

# Apply migrations
alembic upgrade head

# Run tests
pytest

# Run scheduler (fetches + screens on interval)
python serve.py

# Run API server (development)
python serve.py --api

Phase checklist

Phase 1 — Backend core

  • Project scaffolding + pyproject.toml
  • Docker-compose with PostgreSQL
  • Config via pydantic-settings
  • ccxt exchange factory
  • OHLCV fetcher for Binance / Kraken / Coinbase
  • DataFrame normalizer
  • EMA, RSI, VWAP indicators
  • Alembic models + initial migration
  • Repository layer (save candles, read candles)
  • Screener engine with 2 working rules
  • pytest suite for indicators and screener

Phase 2 — Alerts + scheduling

  • Telegram bot setup
  • Alert formatter and sender
  • APScheduler jobs wired to fetcher + screener
  • Alert persistence in DB
  • Deduplication (no re-alert for same condition within N hours)
  • Railway deploy (EU West, Dockerfile, alembic release command)

Phase 3 — API + frontend

  • FastAPI router for pairs, candles, alerts
  • Authentication (API key, simple)
  • Next.js project scaffolding
  • Price chart with indicators overlay
  • Alert history view
  • Dashboard with live price, change %, RSI per pair
  • Rich alert messages with real indicator values
  • Deploy frontend to Railway

v1.1 Roadmap

Observabilidad

  • Structured logging (JSON) en backend para Railway Log Explorer
  • Alertas de scheduler caído — send_error_alert en fetch_job y screen_job

Multi-user (seam)

  • User model con email, telegram_chat_id, language
  • user_id FK en ScreenerRule y Alert — reglas y alertas son por usuario
  • screen_job enruta alertas al telegram_chat_id del dueño de la regla
  • send_alert acepta chat_id explícito; fallback a TELEGRAM_CHAT_ID para modo single-user
  • JWT auth + registro — diferido hasta tener un segundo usuario real
  • Frontend login/signup — diferido
  • Suscripciones de pares por usuario (tabla user_pairs) — diferido

Refactor

  • Quick wins del docs/refactor-report.md: dead code eliminado, column names unificados, TelegramError re-export, seed scripts mergeados
  • Health check endpoint GET /health
  • Shared indicator computation (candle/indicators/compute.py)
  • Condition registry en _build_rule
  • Engine dispose en lifespan de FastAPI
  • Session URL resolution simplificada
  • Security headers middleware (X-Content-Type-Options, X-Frame-Options, Referrer-Policy)
  • Request audit log middleware (client IP, method, path, status code)
  • CORS middleware — diferido hasta tener clientes externos

Frontend

  • Mejorar diseño general del dashboard — layout, tipografía, dark mode consistente
  • Página de configuración: gestionar pares activos (activar/desactivar) y reglas del screener (activar/desactivar, editar umbrales)

Alertas Telegram enriquecidas

  • Mensajes estructurados: precio, RSI, VWAP, volumen, exchange, timestamp UTC

Tests

  • Tests E2E con Playwright — flujo completo: dashboard carga pares, pinchar par abre gráfico, tabla de alertas muestra entradas

MCP servers in use

MCP Purpose
filesystem Claude navigates and edits project files
github PRs, commits, issue tracking from conversation
postgresql Claude queries the live dev DB during development

Testing rules

  • Indicators must have at least 2 tests each: one happy path, one edge case
  • Never call real exchange APIs in tests — use fixtures in tests/fixtures/
  • Never use random data — fixtures must contain known signals with known outputs
  • A fixture is a real OHLCV response downloaded once with ccxt and saved as CSV or JSON
  • Integration tests use a separate test database (DATABASE_URL_TEST in .env)
  • Repository tests run against a real PostgreSQL test instance, not SQLite
  • Mock the Telegram client — verify it was called with the correct message, never send real messages in tests
  • A test that only checks "no exception raised" is not a test

Fixture strategy

tests/fixtures/
├── btc_4h_100.csv          # 100 real BTC/USDT 4h candles for indicator tests
├── btc_4h_crossover.csv    # Candles containing a known EMA crossover signal
├── btc_4h_overbought.csv   # Candles where RSI exceeds 70
└── raw_ccxt_binance.json   # Raw ccxt response saved once, used in normalizer tests

Generate fixtures once with a helper script (scripts/generate_fixtures.py). Never regenerate them automatically — fixtures must be stable and committed to git.


Trading System Constraints

You must follow the Trading System Validation Spec located at:

docs/polymarket/specs/polymarket_validation_spec.txt

We are in Phase 1 (Signal Validation).

Do NOT:

  • optimize parameters
  • change thresholds
  • modify signal logic
  • introduce new filters

Only:

  • analyze
  • add metrics
  • improve observability

Open questions

  • Authentication strategy if moving toward multi-user SaaS (current: single shared API key)
  • Whether to use Supabase instead of raw PostgreSQL for easier auth
  • Pricing model if monetizing

Last updated: 2026-04-06 — daily risk reset, config validation, startup logging, price range vars to config table