diff --git a/README.md b/README.md index a372812..d970505 100644 --- a/README.md +++ b/README.md @@ -1,11 +1,8 @@ -# poly-maker (v2) +# poly-maker A maker-only market-making bot for **Polymarket CLOB V2**, focused on political markets. Single async process, local-file config (no Google Sheets), typed and -tested. This is a ground-up rewrite of v1 — see [`docs/scoping/`](docs/scoping/00-overview.md) -for the design and [`docs/scoping/06-migration-plan.md`](docs/scoping/06-migration-plan.md) -for the plan. The original v1 code has been removed; see the git history (branch -`main`) if you need to reference it. +tested. > [!WARNING] > Market making on Polymarket is competitive and can lose money. This is a @@ -14,13 +11,13 @@ for the plan. The original v1 code has been removed; see the git history (branch ## What it does -- Discovers political markets via the **Gamma API** (seconds, not the v1 hour-long - crawl) and ranks them by reward + rebate income vs. volatility/spread risk. +- Discovers political markets via the **Gamma API** (seconds) and ranks them by + reward + rebate income vs. volatility/spread risk. - Maintains a live order book per token from the **market WebSocket**. -- Quotes **maker-only** (every order is post-only): a fair-value + inventory-skew - strategy that posts BUY-YES and BUY-NO as the canonical two-sided quote, with - live volatility/toxicity estimation and a regime machine that pulls quotes - during news events. See [`docs/scoping/04-strategy.md`](docs/scoping/04-strategy.md). +- Quotes **maker-only** — every order is post-only. Fair-value + inventory-skew + strategy that posts BUY-YES and BUY-NO as a two-sided quote, with live + volatility/toxicity estimation and a regime machine that pulls quotes during + news events (see [Strategy](#strategy)). - Reconciles a target quote set against live orders with churn tolerances; runs the exchange **heartbeat** dead-man switch; enforces risk caps and a daily-loss kill switch. @@ -39,13 +36,13 @@ uv run polymaker --help ## Configure ```bash -cp .env.example .env # then edit: PK + BROWSER_ADDRESS (same wallet as the UI) +cp .env.example .env # then edit: PK + BROWSER_ADDRESS ``` ### Which wallet address? -Polymarket's current **deposit-wallet** architecture (rolled out ~June 2026) -shows several addresses in the UI, and the labels are inconsistent. What matters: +Polymarket's current **deposit-wallet** architecture shows several addresses in +the UI, and the labels are inconsistent. What matters: - **`BROWSER_ADDRESS` = the funder** — the *smart-contract wallet that actually holds your pUSD and positions*. Depending on the account it may be shown as the @@ -61,8 +58,7 @@ Set `signature_type` in `config/config.toml` to match how the account was made: `3` = POLY_1271 deposit wallet (current default), `2` = older browser-wallet Gnosis Safe, `1` = email/magic proxy, `0` = plain EOA. A wrong type fails loudly (`polymaker doctor` / `livetest` report it). Approvals must have been granted -from the funding wallet — trading once in the UI does this. See -[`docs/scoping/03-api-layer.md`](docs/scoping/03-api-layer.md) §1/§9. +from the funding wallet — trading once in the UI does this. Everything else is TOML under [`config/`](config/): @@ -105,10 +101,41 @@ user WS ─▶ StateStore RiskManager Gamma ─▶ Catalog/scanner ─▶ SQLite periodic REST reconcile ┘ ``` -The strategy layer is a pure function `(book, inventory, params, clock) → -TargetQuotes` — deterministic and unit-tested. The engine owns all I/O and -state around it. Full component map in -[`docs/scoping/02-architecture.md`](docs/scoping/02-architecture.md). +One async event loop. The strategy layer is a pure function `(book, inventory, +params, clock) → TargetQuotes` — deterministic and unit-tested. The engine owns +all I/O and state around it; the `ExecutionGateway` wraps `py-clob-client-v2` +(which handles the V2 EIP-712 signing) and offloads its blocking calls to a +thread pool so the hot path never stalls. State (positions, orders, PnL, catalog) +lives in one SQLite file; raw WS/order events are journaled to `journal/` for +replay. + +## Strategy + +Maker-only, quoting both sides of each market as USDC-collateralized bids: + +- **Fair value** — depth-weighted microprice off the live book, nudged by an + EWMA of signed trade flow. +- **Quote construction** — reservation price `r = FV − skew(inventory)`; + half-spread `δ = base + c_vol·σ + c_tox·toxicity`. Post **BUY-YES at `r − δ`** + and **BUY-NO at `(1 − r) − δ`**. Because both legs are bids that sum below 1, + a filled pair merges back to USDC at locked edge `1 − p − q` — a maker-only + exit that never crosses the spread. +- **Inventory skew** — net position leans both quotes: long YES → bid YES lower, + bid NO higher (acquire the offsetting leg). Size tapers as inventory approaches + a soft cap, then the adding side is pulled entirely. +- **Volatility / toxicity** — realized-vol and per-fill markout (adverse + selection) EWMAs widen the spread and shrink size in markets that pick us off. +- **Regime machine** — per market: `QUIET` (farm rewards in-band), `TRENDING` + (lean + widen + half size), `EVENT` (sweep/jump detected → pull quotes, cool + off), `REDUCE_ONLY` (inventory cap / near end date → exits only), `HALTED` + (stale data / resolved / kill switch → cancel all). +- **Rewards + rebates** — quotes stay inside the liquidity-rewards band in QUIET; + the market selector also scores the new maker-rebate program (a share of taker + fees rebated to makers). +- **Risk** — per-market notional cap, neg-risk event-group worst-case cap, total + exposure cap, daily-loss kill switch, WS-staleness halt. + +Tune it all via profiles in `config/strategy.toml`. ## Develop @@ -119,20 +146,18 @@ uv run ruff check src tests # lint uv run mypy src # types (strict) ``` -## Status / roadmap +## Status -Implemented: config, catalog/scanner, order book + analytics, strategy (FV, +Implemented and live-verified end to end (auth → book → strategy → sign → post → +cancel): config, catalog/scanner, order book + analytics, strategy (FV, vol/toxicity, regime, quoting), state store + lifecycle, execution gateway + reconciler + heartbeat, market/user websockets, risk manager, merger (EOA path), -engine, CLI, paper mode, journal capture. 83 tests; the market-data path is -verified live. +engine, CLI, paper mode, journal capture. 83 tests; ruff + mypy strict clean. -Spike-gated (need the live wallet — see [`docs/scoping/03-api-layer.md`](docs/scoping/03-api-layer.md) §9): -V2 order signing with a Safe/proxy wallet (`livetest` is the probe), pUSD -wrap/allowance mechanics, exact user-WS field names, native Safe merge execution. - -Deferred (scoped, not built): replay backtester over captured journals, external -data feeds (polls/news/cross-venue — [`docs/scoping/07-future-external-data.md`](docs/scoping/07-future-external-data.md)). +Not yet built: a replay backtester over the captured journals, and external data +feeds (polls / news / cross-venue). Merging through a Safe/proxy wallet routes a +tx via the relayer and isn't wired yet — until then inventory exits via limit +sells rather than merging. ## License diff --git a/config/config.toml b/config/config.toml index 11469ce..431bb9b 100644 --- a/config/config.toml +++ b/config/config.toml @@ -1,5 +1,5 @@ # polymaker global config. Secrets live in .env, never here. -# See docs/scoping/05-config-and-ops.md. +# See the README. [wallet] # chain: Polygon mainnet @@ -12,7 +12,7 @@ chain_id = 137 # If your UI shows a "deposit address", you almost certainly have a deposit # wallet -> use 3. In every proxy case (1/2/3), BROWSER_ADDRESS is the DEPOSIT # address (the funder that holds your pUSD), NOT your signing EOA and NOT the -# deployer/factory. Known SDK auth bugs live here (docs 03 §9) — `polymaker +# deployer/factory. Known SDK auth bugs live here (see the README) — `polymaker # doctor` / `livetest` will tell you if you picked the wrong type. signature_type = 3 clob_host = "https://clob.polymarket.com" diff --git a/config/strategy.toml b/config/strategy.toml index 10ceecc..eefe25c 100644 --- a/config/strategy.toml +++ b/config/strategy.toml @@ -1,6 +1,6 @@ # Named strategy parameter profiles (replaces the v1 Hyperparameters sheet). # markets.toml maps each market to one of these profiles. -# See docs/scoping/04-strategy.md §8. +# See the README. [profiles.political-longdated] # --- fair value --- diff --git a/docs/scoping/00-overview.md b/docs/scoping/00-overview.md deleted file mode 100644 index 8e70d86..0000000 --- a/docs/scoping/00-overview.md +++ /dev/null @@ -1,72 +0,0 @@ -# Poly-Maker v2 — Scoping Overview - -**Date: 2026-07-05.** Scope for a ground-up v2 of this Polymarket market-making -bot, targeting political markets. - -## Why now (the forcing function) - -Polymarket cut over to **CLOB V2 on April 28, 2026** — new order signing, new -exchange contracts, pUSD collateral — and archived the `py-clob-client` -library this repo is built on. **v1 of this bot can no longer place an -order.** We are not refactoring a working system; we're rebuilding a dead one, -which frees us to fix everything at once. - -## Goals - -1. **CLOB V2 migration** — new client (`py-clob-client-v2`), V2 order signing, - pUSD collateral handling, post-only orders, heartbeat dead-man switch, - batch endpoints. → [03-api-layer.md](03-api-layer.md) -2. **Kill Google Sheets** — config, market selection, and dashboards become - local TOML files + SQLite + a `polymaker` CLI. A laptop with the repo and a - funded wallet is a complete deployment. → [05-config-and-ops.md](05-config-and-ops.md) -3. **Fast market discovery** — Gamma API keyset queries with server-side - politics/liquidity/reward filters replace the hour-long - crawl-every-orderbook scanner: seconds, one process. → [03](03-api-layer.md) §4, [05](05-config-and-ops.md) §2 -4. **Much better strategy** — fair-value + inventory-skew quoting, live - volatility/toxicity estimation, regime detection (knowing when *not* to - quote), rewards **and** the new maker-rebates program as explicit objective - terms, and a replay backtester + paper mode to prove it before capital. - → [04-strategy.md](04-strategy.md) -5. **Maker-only, always** — every order is post-only; exits are repriced - limits, the YES+NO merge loop, and size discipline. No taker orders, - period. → [04](04-strategy.md) §3, §6 -6. **Faster & simpler** — one async process (no threads, no Node.js - subprocess, no gc.collect() folklore), uvloop, event-to-order latency - target <50ms internal, Python 3.12+, uv end-to-end, typed and tested. - → [02-architecture.md](02-architecture.md) -7. **Political markets first, external data later** — v2 trades on - microstructure alone; the architecture reserves an explicit seat (SignalBus) - for polls/news/cross-venue feeds in a later phase. - → [07-future-external-data.md](07-future-external-data.md) - -## Reading order - -| Doc | Contents | -|---|---| -| [01-current-state.md](01-current-state.md) | Audit of v1: what exists, what's broken, what dies, what's worth keeping | -| [02-architecture.md](02-architecture.md) | v2 design: components, state machine, package layout, dependencies | -| [03-api-layer.md](03-api-layer.md) | CLOB V2 migration facts, client choice, endpoints, websockets, rate limits, on-chain/merge | -| [04-strategy.md](04-strategy.md) | The quoting engine: FV, vol/toxicity, regimes, rewards+rebates utility, risk, simulation | -| [05-config-and-ops.md](05-config-and-ops.md) | Local config schemas, scanner/CLI, observability, deployment | -| [06-migration-plan.md](06-migration-plan.md) | Phases 0–5, exit criteria, deletion ledger, risk register | -| [07-future-external-data.md](07-future-external-data.md) | Deferred feed layer, scoped so it's additive later | - -## Headline decisions (details in the docs) - -- **Build in place, delete v1 as we go** — v1 can't trade anyway; no parallel - run. (The v1 code was parked under `legacy/` during the port and has since - been deleted — it lives on in git history on `main`.) -- **Two de-risking spikes before building** (Phase 2): wallet auth on V2 with - our proxy-wallet setup (known SDK bugs around signature types), and live WS - payload capture (schemas changed post-2025 and docs are young). -- **BUY-YES + BUY-NO is the canonical two-sided quote** — USDC(pUSD)- - collateralized on both sides, pairs merge back to cash with locked edge, - and it satisfies the rewards program's two-sidedness rules. -- **Paper-trading gate before live capital**: ≥2 weeks, ≥10 markets, positive - PnL under a conservative fill model, markouts > −0.5 tick at +5m. - -## Open questions (tracked in 03 §9) - -WS ping/pong contract; `py-sdk` (beta unified SDK) maturity; pUSD -wrap/allowance/merge-collateral mechanics; current `price_change` payload -schema; rebate payout currency; signature-type-2 health in the v2 SDK. diff --git a/docs/scoping/01-current-state.md b/docs/scoping/01-current-state.md deleted file mode 100644 index a11e246..0000000 --- a/docs/scoping/01-current-state.md +++ /dev/null @@ -1,154 +0,0 @@ -# 01 — Current State Audit (v1) - -Honest inventory of what exists today, what's broken, and what dies in v2. -This is the baseline the rest of the scoping docs build on. - -## What v1 is - -A single-process Python market maker (~2,700 LOC) plus a separate market-scanner -script, glued together through a **Google Sheet** that acts as config store, -market database, and dashboard at the same time. - -``` -main.py entry point: REST polling thread + 2 websockets -trading.py the entire strategy (perform_trade, 470 lines) -update_markets.py scanner: crawls all markets, writes to Google Sheets hourly -update_stats.py writes account stats/earnings back to Google Sheets -poly_data/ client wrapper, global mutable state, WS handlers, book -poly_utils/ Google Sheets access (incl. a read-only CSV-export hack) -data_updater/ scanner internals (reward math, volatility fetch) -poly_stats/ earnings/positions summary -> Sheets -poly_merger/ Node.js subprocess to merge YES+NO via Gnosis Safe -positions/*.json ad-hoc risk-off state files written next to the code -``` - -### Data flow today - -1. `update_markets.py` (separate process, ideally separate IP) crawls - `clob.polymarket.com/sampling-markets` page by page, then fetches **one order - book per market** via REST plus one `prices-history` call per market to - compute volatility — thousands of sequential REST calls per scan, ~hourly. - Results are pasted into the "All Markets" / "Volatility Markets" sheets. -2. A human picks rows into the "Selected Markets" sheet and tunes the - "Hyperparameters" sheet. -3. `main.py` re-reads the sheet **every 30 seconds** (network call to Google in - the hot path), polls positions/orders REST every 5s in a background *thread*, - and maintains order books from the market websocket in the asyncio loop. -4. Every `book`/`price_change`/user event spawns `perform_trade(market)` as a - new asyncio task, serialized per market by an `asyncio.Lock`. - -## What's structurally wrong - -### Concurrency model -- **threading + asyncio mixed**: a daemon thread mutates `global_state` (dicts of - positions/orders) while asyncio tasks read them. No coherent locking - (`global_state.lock` exists but is essentially unused). -- **Blocking I/O inside the event loop**: `py-clob-client` and `requests` are - synchronous. Every order placement/cancel blocks the loop — while it's - signing + POSTing, the bot is blind to every other market's book updates. -- **`perform_trade` holds the per-market lock through a `gc.collect()` and a - hard-coded `await asyncio.sleep(2)`** (trading.py:470). Every quote decision - on a market therefore takes ≥2s and events pile up behind the lock. This - alone caps reactivity at ~0.5 Hz per market in a game where queue position - and reaction to sweeps decide profitability. -- Event storm handling is "spawn a task per WS message" — no coalescing, no - debounce; the lock queue grows unboundedly under load. - -### State management -- One module of **global mutable dicts** (`global_state.py`) shared across a - thread and dozens of tasks. Position tracking is a merge of three sources - (data-api REST, user WS fills, on-chain balance) reconciled with the - `performing` set + 15-second staleness hack — a hand-rolled, race-prone - answer to "did my fill land yet?". -- `set_order()` in `poly_data/data_utils.py:136` **overwrites both sides** of a - token's order record with a dict containing only the side that just updated — - the other side's `{price, size}` silently vanishes until the next 5s REST poll. -- Local order book is keyed by condition_id, stores only the YES token book, - derives the NO view by `1 - price`. Fine in principle, but book resync after - WS reconnect relies on the server re-sending a `book` snapshot; there's no - sequence-gap detection or hash check (the WS actually provides a book `hash`). - -### Google Sheets as infrastructure -- Config store, market DB, ops dashboard, and inter-process message bus are all - one spreadsheet. It's rate-limited (Sheets API quota), slow (hundreds of ms - per read), needs service-account credentials, and pulls a network dependency - into the trading loop. The "read-only mode" fallback in - `poly_utils/google_utils.py` literally guesses gid numbers 0–4 against a CSV - export URL. -- The scanner and the bot communicate *through* the sheet, so market metadata - in the hot path can be an hour stale (`row['best_bid']` used as a sanity - reference in trading.py:356 comes from the sheet). - -### Code quality -- Bare `except:` everywhere (~30 sites), `print()` as the only logging, no - tests at all, no types, dead code left as comments, `gc.collect()` sprinkled - as a superstition (7 call sites), `deets` reused as both the outcome list and - the book snapshot inside the same loop (trading.py:158/197). -- `data_updater/` duplicates `poly_utils` (two `google_utils.py`, two - `trading_utils.py` with different contents). -- Pinned to **Python 3.9** (.python-version = 3.9.18, EOL Oct 2025) and - `py-clob-client==0.28.0`. That client line is **dead**: Polymarket cut over - to CLOB V2 on 2026-04-28 (new order signing, new contracts) and archived - py-clob-client — **v1 of this bot cannot place an order at all anymore** - (see 03). This isn't a refactor of a working bot; it's a rebuild of a - non-functional one. -- A **Node.js subprocess** (`poly_merger/`) exists solely to call - `mergePositions` through a Gnosis Safe — a whole second runtime, package.json - and ethers v5 dependency for one contract call that web3.py can do. - -### Strategy weaknesses (why it loses money — detailed fix in 04-strategy.md) -- **Quote logic is penny-jumping**: `bid = best_bid + tick`, `ask = best_ask - - tick`, with a couple of size-based exceptions. No fair-value estimate, no - microprice, no inventory skew — position size only gates *whether* to quote, - never *where*. -- **Adverse selection is handled after the fact**: the only defenses are an - hourly 3-hour volatility number from the spreadsheet and a stop-loss that - **crosses the spread** (sells at best bid — a taker exit, exactly what we - want to ban) then sleeps the market for hours via a JSON file on disk. -- **Rewards are a filter, not an objective**: the scanner computes - reward-per-100 carefully, but the live quoter only checks "is my bid inside - the incentive band" as a boolean. Placement within the band (where the - reward-vs-fill-risk tradeoff actually lives) is never optimized. -- Hard-coded magic everywhere: 0.1–0.9 price band, 250 share absolute cap, - 0.005 reprice threshold, 15s staleness, 0.95/0.97/1.01 ratios, `max_size*2` - total-exposure allowance. -- Take-profit reprices only when 2% away from target; sell-side quote sits at - `avgPrice` floor (trading.py / trading_utils.py:136) — i.e. the bot refuses - to exit below cost, which turns losers into resolution lottery tickets. - -## What's worth keeping (conceptually) - -- **The two-token + merge insight**: buying YES and NO such that both fill and - merging the pair back to USDC is a maker-only exit that never crosses the - spread. v2 promotes this from an afterthought to a core mechanism. -- Book maintenance via `SortedDict` and the YES/NO mirror trick — cheap and - correct. -- The desired-vs-existing order diffing idea in `send_buy_order`/`send_sell_order` - (only cancel/replace on meaningful change) — right instinct, generalized into - a proper reconciler in v2. -- The scanner's reward-per-$100 estimator (`add_formula_params` implements the - official S(v,s) = ((v−s)/v)² scoring) — the math moves into v2's selector. -- The `performing` concept (in-flight fills gate REST reconciliation) — becomes - a real state machine instead of dicts of sets. - -## What dies in v2 - -| Component | Fate | -|---|---| -| Google Sheets (gspread, gspread-dataframe, google-auth, service accounts) | **Deleted.** Local files + SQLite (02/05) | -| `poly_merger/` Node.js + ethers + package.json | **Deleted.** Native Python Safe execution (03) | -| `update_stats.py`, `poly_stats/` (Sheets dashboards) | **Deleted.** Replaced by local status CLI/log (05) | -| `data_updater/` sampling-markets crawl + per-market book fetch | **Deleted.** Gamma API scanner, minutes → seconds (03/05) | -| `global_state.py` module-level dicts | **Deleted.** Typed state store owned by the engine (02) | -| Background REST polling thread | **Deleted.** Single asyncio loop; REST only for reconciliation (02) | -| `positions/*.json` risk files | **Deleted.** SQLite state (02) | -| Python 3.9 pin, py-clob-client 0.28 (archived, V1-only) | Upgraded: 3.12+, py-clob-client-v2 + own async HTTP (03) | -| `print()` logging, bare excepts, gc.collect() | Replaced: structured logging, real error policy (02/05) | - -## Repo facts for reference - -- uv is **already adopted** (uv.lock, pyproject.toml, hatchling build) — v2 - keeps uv and finishes the job: bump `requires-python` to ≥3.12, refresh all - pins, add ruff + mypy + pytest as dev group, `src/` layout. -- Git history is shallow and healthy; no CI exists; MIT licensed; README - already warns the v1 strategy is unprofitable. diff --git a/docs/scoping/02-architecture.md b/docs/scoping/02-architecture.md deleted file mode 100644 index d3853c9..0000000 --- a/docs/scoping/02-architecture.md +++ /dev/null @@ -1,214 +0,0 @@ -# 02 — v2 Architecture - -Single async Python process, local-file config, typed state, no threads, no -Google, no Node. Everything the bot needs to run lives in the repo directory. - -## Design principles - -1. **One event loop.** All I/O is async. Anything blocking (order signing is - CPU-trivial; EIP-712 signing is ~1ms) stays inline; anything slow-blocking - that can't be made async is banished to `asyncio.to_thread` — but the goal - is zero such call sites. -2. **Websocket-first, REST-reconcile.** The WS feeds are the source of truth - for books, orders, and fills in real time. REST is used for (a) startup - snapshots, (b) periodic drift reconciliation, (c) actions. Never poll REST - for something the WS already pushes. -3. **Desired-state reconciliation.** The strategy never "places an order"; it - emits a *target quote set* per market. The execution layer diffs targets - against live open orders and issues the minimal cancel/place set, applying - tolerances (don't churn for sub-tick / sub-10% size changes) and rate-limit - budgets. This is the v1 `should_cancel` instinct, made a first-class layer. -4. **Local-first ops.** Config = files in the repo. State = SQLite. Dashboards - = CLI. An operator with a laptop and the repo can run, inspect, and tune - everything with no external accounts beyond the Polymarket wallet. -5. **Deterministic and testable.** Strategy is a pure function - `(BookState, Inventory, Params, Clock) -> TargetQuotes`. That makes replay - backtesting and unit tests trivial and is the enabler for 04-strategy. - -## Process layout - -``` - ┌────────────────────────────────────────────┐ - │ polymaker run │ - │ │ - Gamma REST ──────▶│ MarketCatalog (refresh ~60s, metadata) │ - │ │ - CLOB market WS ──▶│ MarketDataService │ - │ └─ per-market OrderBook (seq/hash check) │ - CLOB user WS ────▶│ UserStream │ - │ └─ fills / order lifecycle events │ - │ │ - │ StateStore (positions, open orders, PnL) │ - │ ▲ WS events ▲ REST reconcile │ - │ │ - │ StrategyEngine │ - │ └─ per-market Quoter task (debounced) │ - │ emits TargetQuotes │ - │ │ - │ RiskManager (pre-trade gates, kill switch) │ - │ │ - │ ExecutionGateway │ - │ ├─ order signer (EIP-712, local) │ - │ ├─ REST client (async, rate budgeted) │ - │ └─ reconciler (target vs live diff) │ - │ │ - │ Merger (YES+NO -> USDC, web3.py native) │ - │ │ - │ Persistence: SQLite + JSONL event journal │ - │ Telemetry: structlog -> console/file │ - └────────────────────────────────────────────┘ -``` - -### Component responsibilities - -**MarketCatalog** — market metadata (tokens, tick size, neg-risk flag, reward -params, fees, end date) fetched from Gamma/CLOB REST (see 03). Refreshed on a -slow timer, cached in SQLite. Nothing in the hot path ever awaits this. - -**MarketDataService** — owns the market WS connection(s). Maintains one -`OrderBook` per subscribed market: `SortedDict` bids/asks (keep v1's approach), -YES-canonical with NO-side views derived. Handles: snapshot on subscribe, -`price_change` deltas, `tick_size_change`, book-hash verification when -available, and reconnect-with-resnapshot. Exposes sync accessors (microprice, -depth-within-band, imbalance) — pure reads, no awaits — plus an -`asyncio.Event`-style dirty flag per market that wakes the Quoter. - -**UserStream** — user WS channel: order placements/cancels/matches and trade -status transitions (MATCHED → MINED → CONFIRMED / FAILED). Feeds StateStore. -Replaces v1's `performing` dicts with an explicit per-order/per-trade state -machine (see below). - -**StateStore** — single owner of positions and open orders. In-memory, typed -(dataclasses), persisted to SQLite on change (WAL mode; writes are µs-cheap). -Three inputs, one arbitration rule: -- WS fill events apply immediately (optimistic), -- REST reconciliation (every ~30s, and on demand after anomalies) corrects - drift *only* for tokens with no in-flight trades, -- on-chain balance is consulted only by the Merger before merging. -This is v1's `performing`/`last_trade_update`/`avgOnly` logic, but as one -explicit component with tests instead of three dicts and a timeout. - -**StrategyEngine / Quoter** — one lightweight task per market. Loop: -``` -wait for (book dirty | fill | timer tick | param change), coalesced -debounce: recompute at most once per N ms (default 200ms), immediately on fill -compute TargetQuotes = strategy(book, inventory, params, clock) # pure -hand to RiskManager -> ExecutionGateway -``` -No lock held during I/O; the Quoter never awaits network calls itself. - -**RiskManager** — pre-trade checks (per-market notional cap, global exposure -cap, neg-risk group cap, max open order value, price sanity band vs recent -mid), plus global circuit breakers (drawdown kill switch, WS staleness halt, -API error-rate halt). Owns the "reduce-only" and "halt" market modes. Detailed -policy in 04-strategy.md §Risk. - -**ExecutionGateway** — the only component that talks to the CLOB REST API for -actions. Async HTTP (httpx), local EIP-712 signing (V2 order format), a -token-bucket rate budgeter tuned to documented API limits (03), batch -place/cancel endpoints, **post-only on every quote**, and the target-vs-live -reconciler with churn tolerances. Also owns the **exchange heartbeat** (03): -a ~5s keepalive; if the process dies or partitions, the exchange cancels all -our orders within ~10s — the dead-man switch a maker-only bot must have. -Every request/response is journaled. - -**Merger** — pure-Python replacement for poly_merger (03 §Merge). Triggered by -StateStore when min(YES,NO) ≥ merge threshold; runs as a low-priority task, -never blocks quoting. - -**Persistence** — `state.db` (SQLite): market catalog cache, positions, order -history, fills, PnL marks, risk events. `journal/*.jsonl`: append-only raw -event log (WS messages in, orders out) — this is also the capture format the -backtester replays (04 §Simulation). - -**Telemetry** — structlog JSON to file + human console; `polymaker status` -CLI reads SQLite for positions/PnL/quotes (05). Optional later: Prometheus -exporter. No Sheets dashboards. - -## Order/trade state machine - -Replaces `performing` + timestamps + 15s cleanup: - -``` -Order: DRAFT -> SIGNED -> POSTED -> LIVE -> {PARTIALLY_FILLED} -> DONE - └─ REJECTED └─ CANCELED -Trade: MATCHED -> MINED -> CONFIRMED - └────────────-> FAILED (position rolled back, reconcile forced) -``` -Timeouts are per-transition (e.g. MATCHED without MINED after T → force REST -reconcile for that token) instead of one global 15s sweep. - -## Package layout - -``` -poly-maker/ -├── pyproject.toml # uv-managed; python >=3.12 -├── uv.lock -├── config/ -│ ├── config.toml # bot settings: wallet env refs, global risk, engine knobs -│ ├── strategy.toml # named parameter profiles (replaces Hyperparameters sheet) -│ └── markets.toml # selected markets + per-market overrides (replaces Selected Markets) -├── src/polymaker/ -│ ├── cli.py # `polymaker` entrypoint: run/scan/markets/status/flatten/... -│ ├── config.py # pydantic-settings models, TOML load, hot-reload (watchfiles) -│ ├── catalog/ # Gamma/CLOB market discovery + scoring (05) -│ ├── marketdata/ # WS client, order book, book analytics -│ ├── userstream/ # user WS, event parsing -│ ├── state/ # StateStore, models, SQLite persistence -│ ├── strategy/ # pure quoting logic + parameter models (04) -│ ├── risk/ # RiskManager, circuit breakers -│ ├── execution/ # signer, REST client, rate budgeter, reconciler -│ ├── merge.py # native merge (03) -│ ├── journal.py # JSONL event journal -│ └── sim/ # replay backtester + paper-trading mode (04) -├── tests/ -└── docs/scoping/ -``` - -`poly_data`, `poly_utils`, `poly_stats`, `data_updater`, `poly_merger`, -`trading.py`, `main.py`, `update_markets.py`, `update_stats.py` are all -deleted once their replacements land (06 has the phasing). - -## Concurrency & performance notes - -- Python 3.12+ (faster asyncio, per-interpreter GIL groundwork irrelevant here - — one loop is enough; this workload is I/O bound with tiny CPU bursts). -- `uvloop` as the event loop (free ~2-4x loop throughput on macOS/Linux). -- WS message → book update → quoter wake is all sync code on the loop; the - only awaits in the hot path are the WS `recv` and the outbound HTTP. -- Signing: `py_order_utils`/eth-account key ops are ~1ms — sign inline, no - executor. -- Target: event-to-order-POST latency < 50ms internally (network to Polymarket - will dominate at ~50–150ms; colocating the box geographically near the CLOB - (AWS us-east) is an ops choice, noted in 05). -- Memory: no gc.collect() calls; books for ~50 markets are a few MB. - -## Error policy - -- No bare excepts. Each component defines recoverable (log + retry with - backoff) vs fatal (halt quoting, keep cancel capability, alert) errors. -- On any uncertainty about our own open-order/position state → freeze new - quotes for that market, force reconcile, resume. "When confused, cancel and - resync" is the universal recovery. -- On process crash: the exchange heartbeat has already cancelled our orders - within ~10s; startup sequence is cancel-all (belt) → REST snapshot → resume. - -## Dependency set (target) - -| Purpose | Package | -|---|---| -| HTTP | httpx (async) | -| WS | websockets (current) | -| Signing/order building | **py-clob-client-v2** (V1 client is archived/dead); direct py_order_utils + eth-account if we outgrow it (03) | -| Chain (merge, balances) | web3.py 7.x | -| Config/validation | pydantic v2 + pydantic-settings, tomllib | -| Hot reload | watchfiles | -| State | sqlite3 stdlib (+ tiny DAO layer; no ORM) | -| Logging | structlog | -| CLI | typer + rich | -| Loop | uvloop | -| Dev | ruff, mypy, pytest, pytest-asyncio | - -pandas/numpy leave the runtime hot path (they remain fine inside the scanner -and sim analysis). gspread, gspread-dataframe, google-auth, py-clob-client -(v1), and the entire Node/ethers runtime go; sortedcontainers stays (book). diff --git a/docs/scoping/03-api-layer.md b/docs/scoping/03-api-layer.md deleted file mode 100644 index 2117f30..0000000 --- a/docs/scoping/03-api-layer.md +++ /dev/null @@ -1,203 +0,0 @@ -# 03 — API Layer: V1 → V2 Migration - -Polymarket executed a **hard, non-backward-compatible cutover to CLOB V2 on -April 28, 2026**. V1-signed orders stopped working that day; the legacy -`py-clob-client` this repo pins (0.28.0; latest 0.34.6) was **archived May 11, -2026** and cannot trade against V2. There is no incremental path — the API -layer is a rewrite, which is exactly what this scoping assumes. - -Primary sources: [V2 migration guide](https://docs.polymarket.com/v2-migration), -[changelog](https://docs.polymarket.com/changelog), -[llms.txt doc index](https://docs.polymarket.com/llms.txt). - -## 1. What changed in CLOB V2 - -| Area | V1 (this repo) | V2 (target) | -|---|---|---| -| Base URL | `https://clob.polymarket.com` | same URL, new backend | -| Order struct | `nonce`, `feeRateBps`, `taker` fields | removed; adds `timestamp` (ms), `metadata`, `builder`; fees set by protocol at match time | -| EIP-712 domain | Exchange version "1" | version **"2"**, new verifying contracts | -| Exchange contracts | `0x4bFb41d5...` / neg-risk `0xC5d563A3...` | CTF Exchange V2 `0xE111180000d2663C0091e4f400237545B87B996B`; Neg Risk CTF Exchange V2 `0xe2222d279d744050d28e00520010520000310F59` | -| Collateral | USDC.e (`0x2791Bca1...`) | **pUSD** (ERC-20, 1:1 USDC-backed); API traders wrap via the Collateral Onramp contract and set pUSD allowances | -| Auth | L1 EIP-712 + L2 HMAC headers | **unchanged**; existing API keys carried over | -| Fees | fields existed, mostly zero | **makers pay zero**; taker-only fees `fee = C · feeRate · p·(1−p)` per category (Politics 1.00%, geopolitics/world-events free); live rate from each market's `feeSchedule` — never hardcode | -| Neg-risk | PartialCreateOrderOptions flag | fully supported; neg-risk orders sign against the neg-risk V2 contract (SDK handles) | - -### Consequences for us -- **Wallet/allowance setup is a new ops step**: wrap USDC→pUSD via the onramp, - approve pUSD to the V2 exchange (buys) and conditional tokens (sells). - Balance checks read pUSD, not USDC.e (v1's hardcoded USDC contract in - `polymarket_client.py:78` dies). -- **Signature-type risk**: v1 runs `signature_type=2` (Polymarket proxy/Safe - funder). `py-clob-client-v2` has a cluster of open auth bugs around - signature types 2/3 and Safe wallets (issues #70/#85/#87/#90/#91) and - "invalid order version" migration confusion (#92). **Phase 2 starts with a - wallet-auth spike**: place/cancel one order with our exact wallet setup - before building anything on top. Fallback: EOA wallet (signature_type=0). -- Fine ticks now exist (0.001, even 0.0025); `tick_size_change` events (fires - crossing 0.96/0.04) must be handled live or orders start rejecting. - -## 2. Client library choice - -| Option | Status | Call | -|---|---|---| -| `py-clob-client` 0.34.x | archived, V1-only | **dead — remove** | -| **`py-clob-client-v2`** (1.0.2, Jul 2026) | official successor, active | **use for Phase 2.** Sync-only → wrap calls we need in a thin async adapter, or use it only for signing/building and POST via httpx ourselves | -| `polymarket-client` (py-sdk) | official unified SDK, **beta**, sync+async | watch; adopt when stable if its async client is solid — re-evaluate at Phase 2 start | -| Direct: `py_order_utils` + eth-account + httpx | max control | fallback if the v2 client's sync-ness or Safe bugs block us; order signing is a well-documented EIP-712 struct | - -Decision: **start with `py-clob-client-v2` for order building/signing, own the -HTTP layer with httpx** (async, connection-pooled, our rate budgeter, our -retries). The signing code is pure CPU and composes fine with our loop. This -also keeps us one small step from going fully direct if the SDK disappoints. - -## 3. Endpoints v2 uses (and the v1 habits they replace) - -### Trading (CLOB, L2 auth) -- `POST /order` — single post-only/GTC/GTD order. -- **`POST /orders` (batch, ≤15)** — quote both tokens × layers in one request; - v1 placed orders one blocking call at a time. -- `DELETE /orders` (≤1,000 ids) — targeted mass cancel; replaces v1's - cancel-all-per-asset churn. -- `DELETE /cancel-market-orders`, `DELETE /cancel-all` — recovery paths - (startup, kill switch). -- **`POST /heartbeats`** (Jan 2026): opt-in dead-man switch — miss a ~10s - heartbeat window and the exchange cancels all our orders. **Mandatory for - v2**: a maker-only book of stale quotes after a crash/network partition is - the biggest tail risk we have. ExecutionGateway sends one every ~5s. -- **Order types**: GTC, GTD, FOK, FAK, and **Post-Only (Jan 2026, GTC/GTD - only)** — the enforcement mechanism for our maker-only mandate (04 §3.2); - every quote goes out post-only. - -### Market data (REST, no auth) -- `GET /book`, **`POST /books` (batch)**, `/midpoint(s)`, `/price(s)` — - snapshots at startup/resync. The scanner uses batch `/books` instead of v1's - one-book-per-market crawl. -- `GET /prices-history` — still exists; only used by the scanner for coarse - historical vol (live vol comes from our own WS-derived estimates). -- `GET /sampling-markets` — replaced by Gamma for discovery (below); CLOB - market metadata (`min_incentive_size`, `max_incentive_spread`, tick size, - neg_risk) still fetched per selected market as authoritative. - -### Rewards & earnings -- Liquidity-rewards program is live and mechanically the same family as v1: - score `S = ((v−s)/v)²`, adjusted midpoint (dust-filtered), two-sided - `Q_min = max(min(Q_one,Q_two), max(Q_one/c, Q_two/c))` with c=3.0, - per-minute sampling, daily payout, $1/market minimum, **double-sided - quoting required to score at all when mid < 0.10 or > 0.90**. - [docs](https://docs.polymarket.com/developers/market-makers/liquidity-rewards) -- **Maker Rebates Program (new, Jan 2026)**: ~20–25% of taker fees collected - in fee-enabled categories (politics 25%-ish) rebated daily to that market's - makers pro-rata. A second income stream v1 never knew about — enters the - strategy utility function (04 §4) and the market selector's scoring (05). -- Earnings actuals: rewards user endpoints (v1's `poly_stats` hit a - polymarket.com internal API with L2 headers; v2 uses the documented rewards - endpoints) — pulled daily into SQLite for calibration. - -### Relayer (only if we keep Safe-style wallets) -- Relayer `POST /submit` now returns `{transactionID, state}` — poll - `GET /transaction` for the hash. Affects the merge flow if executed via - proxy wallet (§6). - -## 4. Market discovery: Gamma API (the "better, faster market list") - -v1's scanner crawled `sampling-markets` page by page then fetched **one order -book + one price history per market** — thousands of sequential calls, ~hourly -cadence, results pasted into Google Sheets. v2 replaces the whole pipeline -with Gamma (`https://gamma-api.polymarket.com`, no auth): - -- **`GET /markets/keyset`** (Apr 2026, cursor-paginated, limit ≤100; the - offset endpoints are slated for deprecation — build keyset-first) with - server-side filters: `active`, `closed` (defaults false now), `tag_id` (+ - `related_tags`) for **politics**, `liquidity_num_min`, `volume_num_min`, - `end_date_min/max`, `rewards_min_size`. -- One response row already carries what v1 burned two extra calls per market - to compute: `bestBid`, `bestAsk`, `spread`, `liquidityNum`, `volumeNum` (+ - 24h/1wk/1mo), `rewardsMinSize`, `rewardsMaxSpread`, `feeSchedule`/`feesEnabled`, - `orderPriceMinTickSize`, `orderMinSize`, `negRisk`, `clobTokenIds`, - `conditionId`, `endDate`. -- Tag IDs are numeric — resolve once via `GET /tags`, cache in SQLite. -- A politics-filtered scan is **a handful of paginated requests (seconds)** - vs. v1's ~hour; run it every few minutes if we like. Rate limits are ample - (Gamma 4,000 req/10s global, `/markets` 300/10s). -- Depth/reward-optimality checks for the shortlist only: batch `POST /books` - on candidate tokens. -- Event grouping for neg-risk risk caps comes from `GET /events/keyset` - (markets nest their event; events give us the sibling-outcome structure). - -## 5. WebSockets - -Same hosts as v1 (`wss://ws-subscriptions-clob.polymarket.com/ws/{market,user}`) -— **URLs unchanged by the V2 cutover** — but the payloads moved: - -- **Market channel**: subscribe with - `{"assets_ids": [...], "type": "market", "custom_feature_enabled": true}`. - Events: `book` (snapshot), `price_change` (**schema changed Sept 2025 — - now includes best_bid/best_ask; re-derive the parser from current docs**), - `tick_size_change`, `last_trade_price` (includes `fee_rate_bps`), and with - the custom flag: **`best_bid_ask`, `new_market`, `market_resolved`**. - `market_resolved` is a free early-resolution alarm for the risk ladder - (04 §6). The old ~100-token subscription cap was removed (May 2025). -- **User channel**: auth payload as v1, but scope subscriptions with - `"markets": [condition_ids]`. `order` + `trade` events, trade status ladder - `MATCHED → MINED → CONFIRMED / RETRYING / FAILED` (maps directly onto the - 02 state machine). -- **Ping/pong**: reportedly server pings every 5s with a 10s pong deadline — - not confirmed on the official channel pages; **verify empirically** and make - keepalive parameters config, not constants. -- **RTDS** (`wss://ws-live-data.polymarket.com`, Sept 2025): separate - real-time data service — `activity` (all trades/matches platform-wide), - `comments`, crypto/equity prices, `clob_user` topic. Not needed for the - core loop; the `activity` topic is a cheap future enhancement for - cross-market flow signals (still Polymarket-internal data, so it doesn't - violate the no-external-feeds rule; noted in 07). - -## 6. On-chain layer (merge + balances) — Node.js dies - -- **Merger rewrite in Python** (web3.py 7.x, which the repo already ships): - `mergePositions` on the ConditionalTokens contract (plain markets) or the - NegRiskAdapter (neg-risk), exactly what `poly_merger/merge.js` does through - ethers v5. For proxy/Safe wallets, execute through the Safe with - `safe-eth-py`, or via Polymarket's relayer client — decide in the Phase 2 - wallet spike (same spike as §1 signature-type risk; if we land on an EOA, - the Safe code path disappears entirely). -- **pUSD questions to resolve in the spike** (docs are young): pUSD token - address; whether V2-era markets' CTF collateral is pUSD (merge returns pUSD) - and whether pre-V2 political markets still merge to USDC.e; onramp - wrap/unwrap mechanics and gas. The Merger and the balance sheet code treat - collateral as config-driven per market, not a constant. -- Balances: pUSD ERC-20 for cash, CTF `balanceOf` for positions (unchanged), - data-api `/positions` for cross-checks (rate limit 150/10s — reconcile use - only). - -## 7. Rate budget (per official limits, Jun 2026) - -Relevant ceilings: `POST /order` 5,000/10s burst; batch `POST /orders` -2,000/10s; `DELETE /orders` 2,000/10s; `cancel-market-orders` 1,500/10s; -books/prices 500–1,500/10s; Cloudflare throttles (queues) rather than 429s — -so the real risk is **silent latency injection**, not errors. - -For ~50 markets × 2 tokens × 2 layers ≈ 200 live orders, a full re-quote is -~14 batch calls — nothing. The ExecutionGateway still gets a token-bucket -budgeter (config: fraction of documented limits, default 25%) because -throttle-queueing would silently add latency exactly when the market is -moving; the budgeter surfaces pressure as a metric + shed-load policy -(skip reprices whose edge < threshold first) instead. - -## 8. Out of scope - -- **Polymarket US** (docs.polymarket.us): separate CFTC-regulated venue, - separate API/auth/SDKs. Not targeted; noted for the future in 07. -- Builder attribution (`builderCode`): not needed unless we monetize order - flow; field exists in the order struct if ever wanted. - -## 9. Open items to verify at implementation time - -1. WS ping/pong contract (empirical test). -2. `py-sdk` maturity re-check at Phase 2 start (async client + WS coverage). -3. pUSD address / onramp mechanics / merge collateral (wallet spike). -4. Exact current `price_change` + `best_bid_ask` payload schemas. -5. Maker-rebate payout currency (USDC vs pUSD) — accounting only. -6. Whether heartbeats interact with batch cancels/cancel-all on reconnect. -7. Signature type 2 (proxy/Safe) health in `py-clob-client-v2` (issue tracker - before the spike). diff --git a/docs/scoping/04-strategy.md b/docs/scoping/04-strategy.md deleted file mode 100644 index de1d339..0000000 --- a/docs/scoping/04-strategy.md +++ /dev/null @@ -1,292 +0,0 @@ -# 04 — Strategy v2: Maker-Only Quoting for Political Markets - -The v1 strategy is penny-jumping with an after-the-fact stop loss. v2 replaces -it with a quoting engine built around four ideas: - -1. a **fair value estimate** derived purely from market microstructure (no - external data in this phase — see 07 for the future feed layer), -2. **inventory-skewed, volatility-scaled quotes** that never cross the spread, -3. **liquidity rewards as an explicit term in the objective**, not a boolean - filter, -4. **regime detection** that decides when *not* to quote — which is where most - of the money is saved. - -Strategy code is a pure function `(BookState, Inventory, Params, Clock) -> -TargetQuotes`, unit-testable and replayable (see §9). - -## 0. The market we're fitting - -Political markets on Polymarket have a specific microstructure profile: - -- **Binary, bounded [0,1]**, usually inside **neg-risk event groups** (multi- - candidate events where Σ YES = 1 by construction). -- **Long-dated with punctuated equilibrium**: weeks of low-vol mean-reversion - interrupted by violent news repricing (debates, indictments, dropouts, - election nights). The flow between events is heavily retail/directional - (favorable for a maker); the flow *during* events is fast and toxic. -- **Early-resolution risk**: a candidate dropping out gates a market to 0 - instantly. There is no external feed to warn us in v2 — microstructure - (sweeps, book evaporation) is the alarm, and size caps are the survival - mechanism. -- Depth is deep at the touch on flagship markets, thin on down-ballot ones — - parameters must scale with observed book depth, not fixed constants. - -Everything below is designed against this profile. - -## 1. Fair value (FV) engine - -All inputs are local: the book we maintain from the WS and the trade tape. - -- **Microprice**: depth-weighted mid over the top K levels (K≈3), i.e. mid - pulled toward the thin side. Robust to size-1 dust orders via per-level - minimum-size filtering (keep v1's `find_best_price_with_size` idea, tuned by - observed depth instead of hardcoded 100/20 shares). -- **Flow adjustment**: EWMA of signed aggressor volume (from `last_trade_price` - /trade events) over ~1–5 min nudges FV in the direction of persistent flow. - Political markets trend on news; leaning with flow reduces adverse fills. -- **Cross-outcome consistency (neg-risk)**: Σ of YES microprices across the - event should ≈ 1. Persistent deviation → books dislocated (news repricing in - progress) → contributes to regime score (§5); large deviation is also the - trigger for the optional arb module (§7). -- **Clamping**: FV is smoothed (short EWMA) and clamped against jumps unless - the jump is confirmed by trades, not just quote flicker. - -Explicitly *not* an input in v2: polls, news, other venues (see 07). - -## 2. Volatility & toxicity engine - -Replaces the spreadsheet's hour-stale "3_hour volatility" with live estimates: - -- **Realized vol**: EWMA of squared FV returns at three horizons (~10s, ~1m, - ~15m), computed on the WS stream. σ_short drives spread width; the - short/long ratio drives regime detection. -- **Sweep detector**: single aggression that consumes ≥N levels or moves FV by - ≥X ticks → immediate event flag (§5). -- **Markout tracker** (self-tuning core): for every one of our fills, record - FV at +30s and +5m. Per-market EWMA of markout = realized adverse-selection - cost. Consistently negative markouts automatically widen that market's - spread / reduce its size via a toxicity multiplier. This is how the bot - learns "this market eats me" without any human or external data. - -## 3. Quote construction (maker-only) - -### 3.1 Reservation price and half-spread - -``` -skew = γ · σ_short · (q / q_max) # in price units -r = FV − skew # reservation price -δ = max(δ_min, c_vol·σ_short + c_tox·toxicity) -bid_raw = r − δ ; ask_raw = r + δ -``` -- `q` = net inventory in the YES token (NO inventory counts negative), - `q_max` = per-market cap. Long inventory pushes both quotes down: buy - cheaper, sell more eagerly. This single mechanism replaces v1's pile of - size-gating special cases. -- Fees: on CLOB V2 **makers pay zero** — no maker fee term in δ. Taker fees - (politics 1.00%, per-market `feeSchedule` from the catalog, 03) matter to - us twice: they fund the **maker rebates** income term (§4), and they enter - the cost model of the optional set-arb module (§7) only. -- All prices then snap to tick and go through **placement logic**: - -### 3.2 Placement: join, don't jump - -- Default: place at the better of (raw price, one tick behind the touch) — - i.e. **join the level or sit behind it**; jump the queue (improve the touch) - only when `FV − best_ask` / `best_bid − FV` edge exceeds tick + costs. v1 - penny-jumps unconditionally, which wins queue position precisely when the - price is about to be wrong. -- **Never cross**: candidate bid ≥ best_ask ⇒ reprice to best_ask − tick (and - symmetric for asks), validated against the live book at send time. Enforced - at the exchange too: **every quote is a Post-Only order** (added to CLOB - Jan 2026, GTC/GTD — 03), which rejects rather than takes. Maker-only is a - property of the system, not a convention. -- **Min edge vs FV**: bid ≤ FV − δ_edge_min always; we never pay through fair - value for queue position or rewards. -- **Layering**: optionally split size across 2–3 levels per side (touch-join + - deeper levels). Sweeps fill the deep levels at prices that already include - the move; layered in-band size also raises the reward score (§4). -- **Churn control**: reprice only if |target − live| > reprice_ticks or size - drift > 15%; batch cancel+place; respect the rate budget (03). Queue - position is an asset — don't burn it for sub-tick prettiness. - -### 3.3 Two-token quoting and the merge loop - -On Polymarket, BUY NO @ q is economically SELL YES @ 1−q, but it is -collateralized by USDC rather than by YES inventory. v2 makes the **BUY-YES + -BUY-NO pair the canonical two-sided quote**: - -- Both quotes are bids (USDC-collateralized) → we can quote two-sided with - zero inventory, from day one, on both sides of the reward band. -- When both fill: hold YES @ p and NO @ q with p + q < 1 → **merge** pair back - to 1 USDC. Realized edge = 1 − p − q, captured without ever selling. The - merge (native Python, 03) runs opportunistically in the background. -- When inventory accumulates on one leg, exits are dual-routed, still maker: - (a) keep/raise the opposite-token bid (merge exit), or - (b) place a SELL limit on the held token above FV. - The quoter maintains whichever route has the better expected time-to-fill × - price; often both, sized to not double-exit. - -### 3.4 Sizing - -- Base size per market from config; scaled down by toxicity, by inventory - utilization (q/q_max), and near end-date (§6). -- Respect `orderMinSize`; low-price multiplier logic from v1 (more shares when - price < 0.10) is replaced by sizing in **USDC notional**, constant per quote - regardless of price level. -- No hardcoded 0.1–0.9 price band: quote anywhere the reward band and risk - checks allow, but size shrinks near the boundaries where payoff asymmetry is - extreme (long-shot bias zone; near-certain markets pay ~nothing to make). - -## 4. Rewards and rebates as an objective - -Two live income programs (03 §3), both maker-side, both enter the objective: - -- **Liquidity rewards** (same family v1 targeted, confirmed live 2026): - per-market daily rate, sampled ~per minute; score `S = ((v − s)/v)²` per - order (s = distance from the dust-filtered *adjusted midpoint*, v = - max_spread), min-size gate, two-sided rule - `Q_min = max(min(Q_one, Q_two), max(Q_one/c, Q_two/c))` with c = 3.0, and — - important — **when mid < 0.10 or > 0.90 only double-sided liquidity scores - at all**. The formula already lives in v1's scanner (`add_formula_params`); - v2 moves it into the strategy as a live utility term. -- **Maker rebates** (new Jan 2026): ~25% of taker fees collected in politics - markets is redistributed daily to that market's makers pro-rata. Unlike - rewards (paid for *resting* in-band), rebates are paid for *being filled* — - they directly subsidize adverse selection and shift the optimizer toward - tighter quoting in fee-enabled markets with heavy taker flow. - -``` -utility(placement) = E[spread capture] + E[reward $/day at this placement] - + E[rebate | fill] − E[adverse selection | regime] - − inventory penalty -``` - -Consequences the optimizer discovers on its own: -- **Quiet regime** → sit deeper inside the band with more size (reward-dense, - low fill probability, low AS cost) — "farming posture". -- **Hot regime** → rewards can't compensate expected AS loss → widen beyond - the band or pull entirely. v1 could never do this; its reward check was a - one-way "must be inside band" constraint that *forced* it to quote tight - exactly when it shouldn't. -- **Two-sidedness**: the Qmin rule (and the hard double-sided requirement at - extreme mids) pays for balanced quoting — another reason the - BUY-YES/BUY-NO pair posture (§3.3) is the default; it keeps both sides - in-band even while carrying inventory. -- Income accounting: actual reward + rebate payouts pulled daily (03) and - attributed per market, so the utility model's estimates are calibrated - against reality instead of drifting. - -## 5. Regime detection (when not to quote) - -Per-market state machine, evaluated on every quoter wake: - -| Regime | Trigger (any) | Behavior | -|---|---|---| -| QUIET | default | farming posture: in-band, layered, full size | -| TRENDING | flow EWMA persistent one-sided; σ_short/σ_long elevated | lean FV with flow, widen δ, half size | -| EVENT | sweep detector; FV jump > J ticks; neg-risk Σ dislocation; book depth evaporates | **pull all quotes**, cooloff T_c (~30–120s), re-enter at 2–3× δ decaying back to normal | -| REDUCE_ONLY | inventory ≥ hard cap; end-date proximity; risk manager order | exit quotes only | -| HALTED | WS stale > T; market flagged closed/`acceptingOrders=false`; kill switch | cancel all, no quoting | - -The EVENT → re-enter-wide → decay pattern replaces v1's "stop-loss then sleep -4 hours via JSON file". Cooldowns are seconds-to-minutes and price-aware, not -hours and blind. - -## 6. Inventory, exits, and lifecycle risk — without taker orders - -Maker-only exit means **we cannot rely on being able to get out**. The -controls compensate: - -- **Soft cap `q_soft`**: skew (§3.1) already leans quotes; beyond q_soft the - adding side is pulled entirely, exit side gains urgency: exit price decays - from FV + δ toward one tick above best bid over τ_urgency (still maker, - never crossing). Urgency accelerates if FV drifts against the position. -- **Hard cap `q_max`**: reduce-only. Sizing (§3.4) makes approaching q_max - progressively harder, so the hard stop is rarely the active constraint. -- **No cost-basis anchoring**: v1 refused to ask below avgPrice, converting - losses into resolution lotteries. v2 prices exits off FV and urgency only; - cost basis is reporting, not strategy. -- **Merge always-on**: min(YES, NO) ≥ threshold → merge to USDC (frees - capital, realizes locked edge, zero market impact). -- **End-date ladder** (from catalog metadata): T−D days → size taper begins; - T−H hours → REDUCE_ONLY; T−S → HALTED + cancel. Political markets also - resolve early: the market WS now pushes **`market_resolved` / `new_market` - events** (03 §5) — instant halt on resolution — with catalog refresh - (closed/`acceptingOrders` flags) and the EVENT regime as backstops. -- **Per-market and global risk** (RiskManager, 02): market notional cap, - neg-risk **event-group worst-case-loss cap** (multiple candidates in one - event are one bet, not N), total exposure cap, daily realized-loss kill - switch (halts quoting, keeps cancel/exit capability), stale-data halt. - -## 7. Neg-risk structural opportunities (optional module, off by default) - -Political events are neg-risk groups; the structure occasionally gifts money: - -- **Set arbitrage**: if Σ best-asks of all outcomes < 1 − ε, buy the set - (maker where possible, but this is the one place taker math is ever - justified — behind a config flag, default off, consistent with the - maker-only policy). -- **NegRiskAdapter conversions**: NO(i) sets convert to YES(others) + USDC — - enables inventory rebalancing across outcomes without trading. Scoped in 03 - (§contracts); implementation phase 4+. -- Both are opportunistic add-ons; core PnL is quoting + rewards. Scoped now so - the state model (event-group awareness) is built in from day one. - -## 8. Parameters - -`config/strategy.toml` holds named profiles (replaces the Hyperparameters -sheet); `markets.toml` maps markets → profile + overrides (05 has schemas). -Illustrative: - -```toml -[profiles.political-longdated] -gamma = 0.5 # inventory risk aversion (skew strength) -delta_min_ticks = 2 -c_vol = 1.2 # spread per unit short-vol -c_tox = 2.0 # spread per unit toxicity score -q_max_usdc = 500 # hard inventory cap, notional -q_soft_frac = 0.6 -layers = 2 -reprice_ticks = 2 -debounce_ms = 200 -event_cooloff_s = 60 -end_date_taper_d = 7 -reduce_only_h = 24 -``` - -Every v1 magic number (0.005 reprice, 250 cap, 0.95/0.97/1.01, 15s, 0.1–0.9 -band, `max_size*2`) either becomes a named documented parameter or is deleted -by design. Hot-reload via watchfiles; changes apply on next quoter wake, no -restart. - -## 9. Simulation, paper trading, and measurement - -The strategy is only "much better" if we can prove it. Three rungs: - -1. **Replay backtest** (`polymaker sim replay`): the journal (02) captures raw - WS streams; the sim reconstructs books and runs the strategy with a queue- - position fill model — conservative mode fills our order only when the tape - prints *through* our price; optimistic mode fills pro-rata at the touch. - Truth lies between; both are reported. Reward income is computed from the - scoring formula on our simulated quotes. Start capturing journals from the - moment Phase 2 lands, so weeks of political-market data exist before the - strategy ships. -2. **Paper mode** (`polymaker run --paper`): full live pipeline, orders - journaled but not posted; fills simulated from the live tape. The gate for - live capital. -3. **Live KPIs** (05 §status): per-market markout curves (+30s/+5m), fill - ratio, two-sided in-band uptime (the reward KPI), reward $ vs estimate, - realized/unrealized PnL, inventory duration, EVENT-regime frequency. - -Acceptance criteria to go live (Phase 4 gate, see 06): paper-mode ≥ 2 weeks on -≥ 10 political markets with (a) positive net PnL including simulated rewards -under the **conservative** fill model, (b) average +5m markout > −0.5 tick, -(c) zero risk-limit breaches. - -## 10. Explicit non-goals in v2 (deferred to 07) - -- External data: polls, news, social monitors, other venues (Kalshi et al). -- Taker execution of any kind (except the flagged neg-risk set-arb, default off). -- Cross-event statistical arbitrage. -- Sports/crypto-specific logic — the engine is generic, but tuning and the - selector target political markets first. diff --git a/docs/scoping/05-config-and-ops.md b/docs/scoping/05-config-and-ops.md deleted file mode 100644 index dd4825a..0000000 --- a/docs/scoping/05-config-and-ops.md +++ /dev/null @@ -1,132 +0,0 @@ -# 05 — Local Config, Market Selection, and Operations - -Google Sheets currently plays four roles: config store, market database, -selection UI, and dashboard. v2 replaces each with a local equivalent. The -test: **a laptop with the repo, a `.env`, and a funded wallet is a complete -deployment.** No Google account, no service-account JSON, no second IP running -a scanner. - -## 1. Configuration files (replaces Hyperparameters + Selected Markets sheets) - -All TOML, all in `config/`, all validated by pydantic at load, all -hot-reloadable via watchfiles (applied at the next quoter wake; invalid edits -are rejected with a logged diff, keeping the last good config — an edit can -never crash the bot). - -### `config/config.toml` — engine & account -```toml -[wallet] -# secrets stay in .env: PK, BROWSER_ADDRESS; this file only references them -signature_type = 2 # revisit after Phase 2 wallet spike (03 §1) - -[engine] -debounce_ms = 200 -reconcile_interval_s = 30 -heartbeat = true # exchange dead-man switch (03 §3) -journal = true # raw WS/order journal for the backtester - -[risk] # global — per-market caps live in markets.toml -max_total_exposure_usdc = 5000 -max_event_group_loss_usdc = 1000 # neg-risk group worst-case -daily_loss_kill_usdc = 250 -ws_stale_halt_s = 10 -``` - -### `config/strategy.toml` — named parameter profiles -Schema in 04 §8. Profiles (`political-longdated`, `political-hot`, …) replace -the Hyperparameters sheet's `param_type` groups. - -### `config/markets.toml` — the trade list (replaces Selected Markets sheet) -```toml -[[markets]] -slug = "will-x-win-the-2028-democratic-nomination" -profile = "political-longdated" -q_max_usdc = 800 # optional per-market overrides of the profile - -[[markets]] -condition_id = "0x37a6de..." # slug or condition_id both accepted -profile = "political-hot" -enabled = false # keep the entry, stop quoting -``` -The bot resolves slugs → tokens/metadata via the catalog at startup and on -reload. Adding a market while running = edit file, save; the engine -subscribes, seeds the book, starts quoting. Removing/disabling = cancel that -market's orders, unsubscribe. This is the entire "market management UI". - -`.env` keeps only secrets: `PK`, `BROWSER_ADDRESS`, optional RPC URL override. -`SPREADSHEET_URL` and `credentials.json` are deleted. - -## 2. Market catalog & scanner (replaces data_updater + All/Volatility Markets sheets) - -`state.db` (SQLite) holds the market catalog; the scanner is a subcommand, not -a second deployment: - -- `polymaker scan` — Gamma keyset sweep with server-side filters (03 §4): - politics tag(s), active, min liquidity/volume, end-date window, rewards - present. Enriches the shortlist with batch `/books` depth and reward-density - estimates (the v1 `add_formula_params` math, kept) **plus the maker-rebate - estimate** (taker-fee volume × rebate share — 03 §3). Writes to SQLite. - Seconds per run; scheduled inside the bot process (e.g. every 15 min) — - the "run on a different IP" advice dies with the crawl that motivated it. -- `polymaker markets` — rank/browse the catalog in the terminal (rich table): - `polymaker markets --tag politics --min-reward 20 --max-vol-sum 20 --sort score`. - Score = expected daily income (rewards + rebates) vs. volatility & spread - risk — v1's `gm_reward_per_100` / `volatility_sum` composite, recalibrated. -- `polymaker markets add [--profile P]` — appends to `markets.toml` - (file remains the source of truth; the command is a convenience editor). -- Volatility for ranking: `prices-history` coarse estimate at scan time; - markets we quote get live WS-derived vol which supersedes it. - -## 3. Runtime CLI (replaces update_stats.py + Summary sheet + eyeballing prints) - -Single `polymaker` entrypoint (typer): - -| Command | Purpose | -|---|---| -| `polymaker run [--paper]` | start the engine (paper mode: full pipeline, no order POSTs) | -| `polymaker status` | positions, open quotes, inventory, PnL, regime per market (reads SQLite; works while the bot runs) | -| `polymaker pnl [--daily]` | realized/unrealized PnL, reward + rebate income vs. estimates | -| `polymaker flatten [market]` | reduce-only mode (maker exits) for one/all markets | -| `polymaker cancel-all` | panic button (also runs automatically at startup) | -| `polymaker scan` / `markets` | catalog (above) | -| `polymaker sim replay ...` | backtester (04 §9) | -| `polymaker doctor` | preflight: wallet auth spike checks, pUSD balance/allowances, WS reachability, clock skew | - -## 4. Observability - -- **structlog** JSON logs to `logs/` + human-readable console. Every order - decision logs its inputs (FV, σ, inventory, regime) — a quote is always - explainable after the fact. -- **Journal** (`journal/*.jsonl`): raw WS in / orders out. Feeds the - backtester and post-mortems. -- **SQLite** is the queryable operational record: fills, orders, PnL marks, - reward payouts, risk events. Any dashboard we want later (Grafana, a small - web page) reads this — no scoping dependency on it now. -- **Alerting (minimal, phase 4)**: kill-switch trips, WS-stale halts, - reconcile divergence, heartbeat failures → log at CRITICAL + optional - webhook URL in config (generic; Discord/Telegram/ntfy all take a POST). -- KPI set defined in 04 §9 (markouts, two-sided in-band uptime, fill ratio, - income vs. estimate). - -## 5. Deployment & runtime ops - -- **uv end-to-end** (already adopted; finish the job): `requires-python = - ">=3.12"`, refreshed lock, `uv run polymaker ...`, dev group with ruff, - mypy, pytest, pytest-asyncio. Delete the 3.9 pin. -- **Placement**: any always-on Linux box; latency to the CLOB (AWS us-east - region) is the one infra knob that matters for queue position — a small VPS - in us-east-1 beats a laptop on wifi. Not required for correctness. -- **Supervision**: systemd unit example in the README (`Restart=always`); - startup sequence is cancel-all → snapshot → resume, so a restart is always - safe. The exchange heartbeat covers the gap between death and restart. -- **CI (GitHub Actions)**: ruff + mypy + pytest on PR. No deploy pipeline - needed for a single-operator bot. -- **State backup**: `state.db` + `config/` are the whole world; journal grows - ~MBs/day per market — rotate + optionally compress with age. - -## 6. Security notes - -- `PK` only ever in `.env` (gitignored) / environment; never in TOML, logs, - or SQLite. Log wallet addresses truncated. -- The CLI never prints secrets; `doctor` verifies without echoing. -- Dependencies pinned via uv.lock; renovate/dependabot optional later. diff --git a/docs/scoping/06-migration-plan.md b/docs/scoping/06-migration-plan.md deleted file mode 100644 index 354c062..0000000 --- a/docs/scoping/06-migration-plan.md +++ /dev/null @@ -1,109 +0,0 @@ -# 06 — Migration Plan - -Phased so that every phase leaves the repo in a working, testable state, and -the risky unknowns (wallet auth on V2, pUSD) are attacked first. v1 cannot -trade anyway (CLOB V1 is gone — 03), so there is no parallel-run constraint: -we are building v2 in place, deleting v1 as replacements land. - -## Phase 0 — Repo hygiene & toolchain (small) - -- `src/polymaker/` package skeleton (02 layout), typer CLI stub. -- pyproject: `requires-python = ">=3.12"`, `.python-version` → 3.12.x, fresh - `uv lock`; dev group: ruff, mypy, pytest, pytest-asyncio; ruff replaces - black; GitHub Actions running lint+types+tests. -- Delete immediately (nothing depends on them working): `poly_stats/`, - `update_stats.py`, the read-only Sheets CSV hack, dead commented code. - The rest of the v1 code (`trading.py`/`main.py`/`data_updater/` …) was parked - under `legacy/` during the port and has now been deleted — it survives only in - git history on `main`. -- Remove deps: gspread, gspread-dataframe, google-auth. Add: httpx, pydantic, - structlog, typer, rich, watchfiles, uvloop, py-clob-client-v2. - -**Exit:** `uv run polymaker --help` works; CI green; no Google imports left. - -## Phase 1 — Catalog, config, scanner (replaces Sheets as data source) - -- pydantic config models + TOML loading + hot reload (05 §1). -- Gamma keyset client, tag resolution, SQLite catalog, market scoring - (rewards + rebates + vol), `polymaker scan` / `markets` / `markets add` - (05 §2). -- CLOB public data client (books batch, midpoints, prices-history) for - enrichment. - -**Exit:** politics markets scanned, ranked, and selected into `markets.toml` -in seconds, with no Google anywhere. (v1 bot is now un-runnable — fine, see -above.) - -## Phase 2 — Execution core (the V2 unknowns live here — do the spikes first) - -- **Spike A (days, throwaway code): wallet auth on V2.** Wrap USDC→pUSD, - set allowances, place + cancel one post-only order with our exact wallet - (signature_type 2), read it back on the user WS. Resolves 03 §9 items 3/7. - Decision output: keep proxy wallet vs. move to EOA; py-clob-client-v2 vs. - direct signing. -- **Spike B: WS contract.** Capture live `book`/`price_change`/`best_bid_ask` - /`tick_size_change` payloads (custom features on), confirm ping/pong, - write the parsers against reality. -- Then build: MarketDataService (books + analytics + resync), UserStream, - StateStore + order/trade state machine, ExecutionGateway (signer, httpx, - rate budgeter, heartbeats, batch ops, reconciler), Merger (native Python, - Safe-or-EOA per Spike A), journal. -- Integration test: on 1–2 quiet live markets with $10-size manual targets, - prove place/cancel/fill/merge/reconcile round-trips, then heartbeat-driven - auto-cancel on kill. - -**Exit:** engine can hold a hand-specified quote set live, survive WS drops, -crashes, and restarts safely. Journal capture running 24/7 from here on (the -backtester's data supply). - -## Phase 3 — Strategy engine (04) + simulation - -- Pure strategy module: FV, vol/toxicity, regime machine, quote construction, - reward/rebate utility, inventory/exit logic — with unit tests per component. -- Replay backtester over the journals accumulated since Phase 2; parameter - study for the political profiles. -- Paper mode (`run --paper`) end-to-end. -- Delete the parked v1 code — the port is done. (Done: `legacy/` removed.) - -**Exit:** paper trading ≥2 weeks, ≥10 political markets, meeting the -acceptance gates from 04 §9 (positive PnL under conservative fills, markouts -> −0.5 tick at +5m, zero risk breaches). - -## Phase 4 — Live rollout + ops hardening - -- RiskManager fully wired (caps, kill switches, event-group limits), alert - webhook, `status`/`pnl`/`flatten`/`doctor` polished. -- Go live: 2–3 markets at minimum size → widen coverage/size on weekly review - of the KPI set. us-east VPS placement. -- Reward/rebate reconciliation against actual daily payouts. - -**Exit:** stable unattended operation; income attribution (spread vs rewards -vs rebates) reported per market. - -## Phase 5 — Deferred (separately scoped) - -- External data feeds (07): polls, news, cross-venue. -- Neg-risk set-arb + NegRiskAdapter conversions (04 §7) if the structural - edge shows up in captured data. -- Optional: RTDS `activity` flow features, Prometheus/Grafana, Polymarket US - venue. - -## Deletion ledger (what's gone by the end of Phase 3) - -`trading.py`, `main.py`, `update_markets.py`, `update_stats.py`, `poly_data/`, -`poly_utils/`, `poly_stats/`, `data_updater/`, `poly_merger/` (entire Node -runtime), `positions/` runtime dir, `credentials.json` expectations, -`SPREADSHEET_URL`, Python 3.9 pin, gspread/gspread-dataframe/google-auth, -py-clob-client (v1), pandas-in-hot-path, all `print()` logging, all bare -excepts, all `gc.collect()` calls. - -## Risk register - -| Risk | Mitigation | -|---|---| -| V2 SDK auth bugs with proxy/Safe wallets | Spike A first; EOA fallback decided in days, not after building | -| pUSD mechanics undocumented corners | Spike A covers wrap/allowance/merge collateral; treat collateral as per-market config | -| WS payload drift (young V2 docs) | Spike B parses from captured reality; parsers tolerate unknown fields | -| Maker-only exits too slow in a news jump | Position caps sized for survivable worst case (04 §6); EVENT regime pulls quotes early; accept this as the strategy's core tradeoff | -| Rewards program parameters shift | All reward math data-driven from per-market API fields; utility recalibrated from actual payouts | -| Strategy underperforms | Paper-gate before capital; journal-driven iteration loop is the product as much as the bot | diff --git a/docs/scoping/07-future-external-data.md b/docs/scoping/07-future-external-data.md deleted file mode 100644 index c771d55..0000000 --- a/docs/scoping/07-future-external-data.md +++ /dev/null @@ -1,63 +0,0 @@ -# 07 — Future: External Data Feeds (deferred, scoped now) - -v2 trades political markets on **microstructure alone** (04 §1). That is a -deliberate constraint, not a belief that external data has no value — the -architecture reserves a seat for it so adding feeds later is additive, not a -refactor. This doc scopes the seat and the candidate feeds; **none of this is -built in v2.** - -## Architectural seat (built in v2, empty) - -The strategy signature already is `(BookState, Inventory, Params, Clock) -> -TargetQuotes`. The extension is one optional input: - -``` -SignalBus: name -> {value, confidence, as_of, ttl} -``` - -- Strategy reads signals as **FV adjusters and regime inputs** with explicit - staleness: a signal past its TTL contributes nothing (fail-safe to pure - microstructure — the bot must never be worse than v2 because a feed died). -- Feed adapters are separate async tasks (or even separate processes writing - to the bus via SQLite/socket) — never in the quote path; the hot loop only - reads a dict. -- Every signal is journaled like WS data, so the replay backtester (04 §9) - can grade any feed's marginal value before it touches live quotes. **A feed - earns its way in through replay evidence, or it doesn't ship.** - -## Candidate feeds for political markets (rough priority) - -1. **Event calendar (highest value / lowest effort)**: debates, primaries, - election nights, court dates, scheduled rulings. Even a hand-maintained - `calendar.toml` (date, market tags, severity) lets the regime machine - pre-widen/pull *before* scheduled volatility instead of reacting to the - first sweep. No API needed to start. -2. **Cross-venue prices**: Kalshi (public API), and Polymarket US as it - grows. Same-event price divergence is both an FV prior and a toxicity - warning (fast flow often arbs venues; if Kalshi moved and we didn't, our - quote is stale). Clean REST/WS, well-bounded work. -3. **Poll aggregates / forecast models**: RCP/538-style averages, Silver - Bulletin, model outputs. Slow-moving FV anchor for long-dated markets — - mostly useful as a *sanity band* (flag when market FV drifts far from - model FV) rather than a quoting signal. -4. **News/headline triggers**: wire headlines, X/Twitter firehose, Google - Trends spikes. Highest alpha, highest noise and engineering cost; - realistically an "instant EVENT-regime trigger" (pull quotes on keyword - burst), not a pricing input. Last in line. -5. **Polymarket-internal extras** (not really "external", could land earlier): - RTDS `activity` topic for platform-wide flow/toxicity features, comment - velocity per market as an attention proxy (03 §5). - -## Explicitly out of scope until then - -- Any taker execution driven by signals (feeds inform *maker* quoting only — - the maker-only policy survives the feed era). -- LLM/news-summarization pipelines, sentiment models. -- Trading the Polymarket US venue (separate API/regulatory surface — 03 §8). - -## Preconditions before starting this phase - -- v2 live and stable through at least one high-volatility political event. -- Journal + replay infrastructure proven (it is the evaluation harness here). -- KPI baseline established, so each feed's marginal contribution is measured - against a known control. diff --git a/src/polymaker/catalog/gamma.py b/src/polymaker/catalog/gamma.py index 02cd06f..a027dae 100644 --- a/src/polymaker/catalog/gamma.py +++ b/src/polymaker/catalog/gamma.py @@ -62,7 +62,7 @@ class GammaClient: Uses the offset `/markets` endpoint because it reliably supports `tag_id` filtering today. (Keyset is the go-forward per docs; switch when - it supports tag filtering. See docs/scoping/03-api-layer.md §4.) + it supports tag filtering. See the README.) """ offset = 0 for _ in range(max_pages): diff --git a/src/polymaker/doctor.py b/src/polymaker/doctor.py index ba0c2a5..5d1be46 100644 --- a/src/polymaker/doctor.py +++ b/src/polymaker/doctor.py @@ -4,7 +4,7 @@ Verifies the environment is ready to trade WITHOUT posting any order: config + secrets, CLOB/Gamma reachable, wallet auth (L1->L2 creds), collateral balance + positions ON THE FUNDER (deposit/developer wallet, where funds live), a live market-WS book frame, and an authenticated user-WS connection. This is -the gate before the live $5 round-trip (docs/scoping/06 Phase 2). +the gate before the live $5 round-trip (the README). """ from __future__ import annotations @@ -84,7 +84,7 @@ async def run_doctor(cfg: Config, console: Console) -> bool: except Exception as e: # noqa: BLE001 check("wallet auth (L2 creds derived)", False, str(e)) console.print(" [yellow]! signature-type mismatch? deposit wallets use sig_type=3 " - "(config.toml). See docs 03 §9.[/yellow]") + "(config.toml). See the README.[/yellow]") else: console.print(" [yellow]! skipping wallet checks (no secrets)[/yellow]") diff --git a/src/polymaker/domain.py b/src/polymaker/domain.py index feec9b0..39935c7 100644 --- a/src/polymaker/domain.py +++ b/src/polymaker/domain.py @@ -24,7 +24,7 @@ class Side(str, Enum): class Regime(str, Enum): - """Per-market quoting regime (see docs/scoping/04-strategy.md §5).""" + """Per-market quoting regime (see the README).""" QUIET = "QUIET" # farming posture: in-band, layered, full size TRENDING = "TRENDING" # persistent one-sided flow: lean + widen + half size diff --git a/src/polymaker/journal.py b/src/polymaker/journal.py index b704894..12d9b5e 100644 --- a/src/polymaker/journal.py +++ b/src/polymaker/journal.py @@ -1,6 +1,6 @@ """Append-only JSONL event journal. -Captures raw WS-in and orders-out so the replay backtester (docs 04 §9) can +Captures raw WS-in and orders-out so the replay backtester (see the README) can reconstruct books and re-run the strategy. Also the substrate for post-mortems. Cheap: one line per event, flushed, rotated by day. """ diff --git a/src/polymaker/livetest.py b/src/polymaker/livetest.py index 2358c12..50eaed4 100644 --- a/src/polymaker/livetest.py +++ b/src/polymaker/livetest.py @@ -1,4 +1,4 @@ -"""Live wallet round-trip test — the Phase-2 wallet spike (docs 06). +"""Live wallet round-trip test — the Phase-2 wallet spike (see the README). Proves the full V2 order path against the real exchange with minimal risk: places ONE post-only BUY well below the touch (so it rests and cannot fill), @@ -50,7 +50,7 @@ async def run_livetest(cfg: Config, console: Console, notional_usdc: float = 5.0 console.print(f" [green]✓[/green] wallet auth — address {gw.address[:12]}…") except Exception as e: # noqa: BLE001 console.print(f" [red]✗ wallet auth failed:[/red] {e}") - console.print(" [yellow]Auth/signature-type mismatch (docs 03 §9). If your account has a " + console.print(" [yellow]Auth/signature-type mismatch (see the README). If your account has a " "'deposit address', set signature_type=3 (POLY_1271) in config.toml and use the " "deposit address as BROWSER_ADDRESS. Errors like 'maker address not allowed, use " "the deposit wallet flow' or 'signer must be the API key address' mean the type is " diff --git a/src/polymaker/marketdata/parse.py b/src/polymaker/marketdata/parse.py index d245ba8..f64d055 100644 --- a/src/polymaker/marketdata/parse.py +++ b/src/polymaker/marketdata/parse.py @@ -1,7 +1,7 @@ """Pure parsers for market-WS wire messages -> structured updates. Kept separate from the socket so they're unit-testable against captured frames. -Verified against live frames on 2026-07-05 (docs/scoping/03-api-layer.md §5): +Verified against live frames on 2026-07-05 (the README): book: {market, asset_id, bids:[{price,size}], asks:[...], timestamp, hash, tick_size} price_change:{market, timestamp, price_changes:[{asset_id, price, size, side, hash}]} diff --git a/src/polymaker/merge.py b/src/polymaker/merge.py index 5b8b380..8a003f2 100644 --- a/src/polymaker/merge.py +++ b/src/polymaker/merge.py @@ -6,7 +6,7 @@ collateral (1 USDC/pUSD per pair) — a maker-only exit with zero market impact. Two execution paths: * EOA wallet (signature_type=0): direct contract call, fully implemented here. * Proxy/Safe wallet (signature_type 1/2): the merge tx must be routed through - the Safe. That path is gated on the Phase-2 wallet spike (docs 03 §6) — until + the Safe. That path is gated on the Phase-2 wallet spike (see the README) — until then merging is skipped (logged), and inventory is exited via limit sells instead. The bot is fully functional without it; merging just frees capital sooner. diff --git a/src/polymaker/risk/manager.py b/src/polymaker/risk/manager.py index cb03f68..50e1b4a 100644 --- a/src/polymaker/risk/manager.py +++ b/src/polymaker/risk/manager.py @@ -1,4 +1,4 @@ -"""RiskManager: pre-trade gates and circuit breakers (docs 04 §6, 02). +"""RiskManager: pre-trade gates and circuit breakers (see the README). Consulted by the engine before every quote set. Returns a per-market decision (size scale / reduce-only / halt) and owns the global kill switches. Position diff --git a/src/polymaker/state/store.py b/src/polymaker/state/store.py index 30a38bc..f8de001 100644 --- a/src/polymaker/state/store.py +++ b/src/polymaker/state/store.py @@ -1,7 +1,7 @@ """StateStore: the single owner of positions and open orders. Replaces v1's module-level global dicts + the `performing`/`last_trade_update` -races. Three inputs, one arbitration rule (docs/scoping/02-architecture.md): +races. Three inputs, one arbitration rule (the README): * WS fill events apply immediately (optimistic), * REST reconciliation corrects drift ONLY for tokens with no in-flight trades, diff --git a/src/polymaker/state/tracker.py b/src/polymaker/state/tracker.py index 1608aa0..6694bed 100644 --- a/src/polymaker/state/tracker.py +++ b/src/polymaker/state/tracker.py @@ -2,7 +2,7 @@ Consumes *normalized* user-stream events (the wire-format extraction lives in userstream/, so this is unit-testable with synthetic events) and drives the -state machine from docs/scoping/02-architecture.md: +state machine from the README: Trade: MATCHED -> MINED -> CONFIRMED └──────────-> FAILED (roll back the optimistic fill, reconcile) diff --git a/src/polymaker/strategy/quoting.py b/src/polymaker/strategy/quoting.py index 1700d45..2b2e3ac 100644 --- a/src/polymaker/strategy/quoting.py +++ b/src/polymaker/strategy/quoting.py @@ -3,7 +3,7 @@ This is the deterministic core of the strategy. No I/O, no wall-clock reads except values passed in. Everything here is exercised directly by unit tests. -Model (see docs/scoping/04-strategy.md): +Model (see the README): reservation r = FV - skew(inventory) half-spread δ = base + c_vol·σ + c_tox·toxicity (clamped to reward band in QUIET) YES entry bid = r - δ (BUY YES, USDC-collateralized) diff --git a/src/polymaker/strategy/regime.py b/src/polymaker/strategy/regime.py index 27ae00f..77ebf41 100644 --- a/src/polymaker/strategy/regime.py +++ b/src/polymaker/strategy/regime.py @@ -1,4 +1,4 @@ -"""Per-market regime decision (see docs/scoping/04-strategy.md §5). +"""Per-market regime decision (see the README). Priority order, highest first: HALTED kill switch / stale data / resolved / past halt-before window diff --git a/src/polymaker/userstream/parse.py b/src/polymaker/userstream/parse.py index edac694..98cde66 100644 --- a/src/polymaker/userstream/parse.py +++ b/src/polymaker/userstream/parse.py @@ -8,7 +8,7 @@ always the maker): * maker & taker on DIFFERENT outcomes -> a mint: we BUY the opposite token NOTE: exact field names must be reconfirmed in the Phase-2 wallet spike -(docs/scoping/03-api-layer.md §9); this is coded to the v1-observed shape. +(the README); this is coded to the v1-observed shape. """ from __future__ import annotations