btc-5m-edge-study/README.md

# BTC 5-Minute Binary Options — An Edge Feasibility Study

**Can a retail trader find a real, *capturable* edge in Polymarket's BTC up/down 5-minute binary options?**
I built a research system to answer that rigorously. The honest conclusion: **almost certainly not — and the reason why is more interesting than a "yes" would have been.**

*A market-microstructure research project in Python / asyncio: it reconstructs a Chainlink-style reference from 8 exchanges over live WebSockets, runs a family of XGBoost-gated strategies, and executes on Polymarket's CLOB. AI-assisted with Claude Code.*

---

## About this repository

This is a **reconstruction** — a runnable extraction of the strategy engine, the XGBoost models, and the execution layer from a larger private system I built and operated. It is provided **as-is and UNAUDITED**.

- **Paper mode runs with no keys** and reads live market data over WebSocket.
- The **execution layer places real orders with real funds** if you supply wallet credentials in `src/predictor/.env` — none are included (`.env.example` ships blank).
- If you intend to trade it: **audit the code yourself and accept the risk.** This is an engineering / research project, **not financial advice.**

Building it was AI-assisted with Claude Code: I designed the research questions, the market model, the strategy logic, and the architecture; Claude accelerated the implementation. The conclusions and the microstructure reasoning are mine.

---

## The market

Polymarket issues a fresh **BTC Up/Down** binary option every 5 minutes. At the close of each window it settles by a simple rule:

> If the **Chainlink BTC/USD reference price** at window close ≥ its value at the window **open** → **UP = $1.00, DOWN = $0.00**.
> Otherwise → **UP = $0.00, DOWN = $1.00**.

Each side trades as a token priced between **$0 and $1** on Polymarket's central limit order book (CLOB). The key fact that makes this analyzable:

> **A binary token's price *is* the market's implied probability of the outcome.**
> UP trading at **$0.60** means the market is pricing a **60%** chance BTC ends the window up.

So the whole game collapses to one question:

> **Can I estimate the true probability of "UP" more accurately than the market's price implies — *and act on it before the market reprices?***

That splits into three investigations. The first two ask *is there an edge?* The third asks *can you actually capture it?*

---

## Pillar 1 — Rebuilding the reference: 8 exchanges vs. the oracle

BTC does not have *one* price. It has a slightly different price on every exchange, updating at slightly different times — and settlement depends on the slowest of them all: the Chainlink oracle print.

So I rebuilt a **Chainlink-style reference in real time** by aggregating **8 exchanges**, while separately tracking the majors that move first and carry the most weight — **Coinbase, Binance, OKX**:

- **WebSocket trade streams:** Coinbase, Binance, OKX, Kraken, Bybit, Bitstamp
- **REST polls (1s — no free public trade WS):** Gemini, CryptoCompare
- **Polymarket itself, over two WebSockets:** the Chainlink price stream, and the CLOB order book (top 5 levels). REST is used only once per window, to discover the new market's token IDs.

The aggregation is deliberately not a naive average: sources stale by >2s are dropped, each venue gets a rolling bias correction, USDT-quoted venues (Binance/OKX/Bybit) are adjusted for the **USDT premium** (median USDT price − median USD price), and the final reference is a **lead-weighted mean** with a freshness bonus for sources that ticked in the last 500ms.

The question: does the **lead–lag between venues** create a *timing* edge? The measured hierarchy — the study's central empirical result:

```
1. The CLOB moves first      — market makers reprice the book ~4–5s before
                               Coinbase's own tape fully reflects the move
2. Coinbase & majors follow
3. The Chainlink oracle prints last — the settlement reference is the
                               slowest object in the room
```

There *is* a measurable lead–lag. The catch: the fastest thing in the market is the order book itself — which means the people you'd be racing have already moved.

---

## Pillar 2 — A directional signal from a rich parameter set

Beyond raw price, can BTC's behavior inside the window fold into a sharper directional estimate than the market's implied odds? The signal stack is deliberately wide — several families of parameters, not one:

- **Displacement & clock** — oracle move since window open (`pm_a`), distance to strike, time remaining
- **Cross-venue edge** — spot leaders vs. the oracle, both raw and EMA-smoothed (raw is faster; smoothed resists single-tick noise)
- **CLOB microstructure** — mid, spread, ask *velocity* (how fast makers are repricing), one-sided buy-volume imbalance
- **Momentum & intensity** — oracle-move acceleration, trade counts
- **ML layer** — XGBoost models scoring late-window outcomes (top features: own ask level, ask margin, |oracle move|, trade count, move acceleration, ask velocity)

The core intuition — and the cleanest math in the study:

> Late in the window, with a clear gap between spot and the strike and low remaining volatility, the outcome becomes highly predictable. To flip, the price would have to move a lot, fast, in the time left.

Modeling the price as a random walk to settlement, the probability of finishing on the current side of the strike is approximately:

```
P(UP) ≈ Φ(  gap  /  ( σ · √(time_remaining) )  )
```

- `gap` = current spot − the strike (the reference price at window open)
- `σ`   = short-horizon volatility
- `Φ`   = the normal CDF

Then the trade signal is simply the mispricing:

```
edge = P_estimate − P_market      # P_market = the token's current price
```

Trade only when `edge` is large **and** late in the window (small `time_remaining`), where the estimate is most reliable. That is exactly where a real edge, if one exists, would live.

### Strategy families: taker and maker

The signals feed two kinds of strategies, mirroring the two ways to trade a book:

- **Taker** (cross the spread when a signal fires): *edge reversal* — in the final seconds, a large cross-venue edge opposing the current leader → buy the reversal side; *CLOB momentum* — the book's ask rising fast means makers are repricing → follow them; *CLOB volume* — one side absorbing 5× the flow → follow it; *early lock* — oracle acceleration + volume doubling + book confirmation → buy the likely winner *early*, at ~0.67 instead of ~0.93, for 3–4× better payoff on the same thesis.
- **Maker** (rest passive limit bids and let the market come to you): cheap trailing-side bids with a GTD expiry — a $1 "option" on a late flip — and counter-consensus resting bids that only fill if a stop-loss cascade briefly dislocates the book. No fill, no risk.

Exits run a ladder of aggression: post the ask → step to the bid → cross at market.

One economics finding worth stating plainly: **low win-rate + cheap entry + high payoff beats high win-rate + expensive entry.** When market makers price accurately, the "safe" side's profit is compressed to zero — a 96%-win-rate strategy can still lose money at the odds you're offered.

---

## Machine learning — and what a weak model proved

To pressure-test whether the directional signal was more than eyeballed thresholds, I trained gradient-boosted trees (XGBoost) as the ML layer. Two model families (all shipped, trained, and loadable in `models/`):

- **Outcome / "lock" model** — 15 pre-trigger features (`ask`, `abs_pm_a`, `left`, `rv_5m`, `rv_15m`, `ask_margin`, `pm_a_accel`, `ask_vel`, `vol_ratio`, `total_vol`, `sign_changes`, …). Label: *does the current leader hold to settlement?* Used as a gate — only take the trade if the model's confidence clears a threshold.
- **Direction models** — a **separate 32-feature model per time-to-close slice** (models at 250s, 200s, …, 10s remaining), because late-window dynamics differ from early. Features span both the asset (`cb_a`, `bn_a`, `edge`, `rv_5m`, `funding`, `oi_chg`, `pm_a_z`) and the counterparty (`bid_ratio`, `ask_from_peak`, `vol_ratio`, `total_vol`).

**How I framed it, and read it honestly:** the label is the settlement outcome, the metric is AUC, and the result was **deliberately unimpressive — direction AUC ≈ 0.666** (0.50 is a coin flip; 0.70 is where a signal starts being useful). The lock gate had to be *lowered* from 0.90 to 0.75 before it would release enough trades to matter, and the lock strategy it gated still finished at a **96% win rate but negative PnL** — accurate predictions, wrong odds.

That weakness is the finding, not a failure. Two things fall out of the feature importances:

> **The most predictive features are the market maker's own quote** — `ask`, `ask_margin`, `ask_velocity` — not BTC's price dynamics. The model taught itself that the single best estimate of the outcome is *what the maker is already charging.* You cannot beat a signal that **is** the counterparty. A near-random AUC on BTC-derived features is direct evidence that the exploitable information isn't in the price — it's already priced.

**See it yourself:** `python ml/xgb_eval.py` loads the trained models and prints their feature importances (the maker's quote on top) plus the per-entry-price economics from real paper trades (every price bucket is net-negative).

**What I'd do next (unexplored):** every model here predicts the *outcome*. The open direction is to change the **label** — model the **counterparty's next action** (will the maker pull the quote, widen the spread, shade the book?) rather than the asset. That reframes the problem from *out-predicting BTC* (a race you lose to better infrastructure) to *detecting maker behavior* (where retail-observable microstructure might actually carry signal). The codebase has a seed of this — a `maker_bias` strategy built on "the maker knows the outcome, follow their bias" — but a proper counterparty model is the study's most interesting untested hypothesis.

---

## Pillar 3 — Execution: a faster manual trading interface

A correct signal is worthless if you can't act on it in time. Polymarket's native UI is **too slow** to place an order inside the seconds when an edge exists — and it's poor for building intuition about how the book behaves under time pressure.

So I built a **hotkey terminal interface** over the CLOB (`tools/quick_trade.py`): raw-keyboard mode where a single keystroke fires an order (buy up/down at the ask, sell at the bid, one-cent "lottery" limit bids), order parameters pre-warmed and cached so submission skips per-order lookups, prices streamed from a local feed instead of fetched per click — and every order prints its measured latency in milliseconds. Full product write-up: [docs/QUICK_TRADE_PRD.md](docs/QUICK_TRADE_PRD.md).

But building it surfaced the deeper execution problem — and it isn't clicking fast. **You don't buy; you post an order and hope.** Every trade is a little state machine:

```
quote (best ask) → post → { full fill | partial fill | no fill } → retry / cancel
```

On a thin binary book, partial fills and no-fills are the norm, not the exception. Each retry chases an ask that is *already moving* — the maker reprices the instant you show intent — so you either cross to a worse level (slippage, straight out of a margin that was pennies to begin with) or you miss and the window closes. Exits face the same gauntlet as a ladder of escalating aggression (post the ask → step to the bid → cross at market), with a circuit breaker that halts trading when round-trip latency spikes. That loop — quote → partial → retry → fill — routinely takes **longer than the edge itself exists**.

This pillar is where the thesis breaks: **the latency inside the fill lifecycle, plus the slippage from thin liquidity, eats essentially the entire edge.** A signal that's correct and a UI that's instant still lose money at the point of the fill.

---

## Operating reality — what actually broke

Running this against live funds is where the thesis stopped being theoretical. These failures are first-hand — and they are *why* an edge that looks real on paper doesn't survive contact with the chain:

- **Latency** — the round trip from signal to acknowledged order regularly outran the seconds an edge exists.
- **Quoted vs. filled** — the ask you decide on and the price you actually pay diverge, worst during the fast moves where the signal fires.
- **Can't buy / can't sell** — thin books mean orders partially fill or don't fill at all; exiting a position under time pressure is not guaranteed.
- **Opening-balance desync** — on-chain settled state and the bot's local view drift apart (a redemption not yet reflected, a fill counted twice), and a wrong balance quietly corrupts the next decision.
- **Fragile to change** — every strategy tweak surfaced a new on-chain edge case: nonce/gas, order expiry, reprice races, partial-fill accounting — none of it visible in backtest.

None of these are bugs to be fixed away; they are the structural tax of executing at retail on a chain-settled venue. Together they are the concrete form of the conclusion: the math can find an edge; the execution layer eats it.

---

## The honest conclusion: retail cannot win this

Even when the analysis surfaces a theoretical edge, **retail can't capture it.**

- The market makers consume the same exchange feeds this system does — and reprice the book in **milliseconds**. By the time a retail-latency system, let alone a human on hotkeys, detects the move and submits an order, the ask has already moved. The signal hierarchy in Pillar 1 is the proof: the book is *faster* than the exchanges it's pricing.
- The directional edge from Pillar 2 is real but thin, and it is **exactly what the market makers are also computing**, faster, with better infrastructure. Watching maker behavior over several weeks of tick data made this vivid: for a stretch, makers were slow to reprice late-window reversals — and within about three weeks they had adapted and that edge collapsed. The market *learns*.
- Pillar 3 and the operating reality above make the ceiling concrete: even a heavily optimized execution path is too slow, and the fill lifecycle plus thin-book slippage consumes what little edge survives.

**The math can find the edge. Market structure makes it uncapturable at retail speed and cost.** Compute, co-location, and latency consume the entire edge before it reaches you. This is structurally an **institutional** game — which is the single most useful finding of the study.

For anyone building consumer-facing trading products, that's the lesson worth carrying: **on any market where the edge is faster than the retail user, the honest product isn't "help them trade the edge" — it's designing for the game retail can actually win.**

---

## What's in the repo, and how to run it

```
tools/chainlink_predictor.py   the live engine — reconstructs the reference, runs the
                               strategies, prints signals + (paper) fills
tools/quick_trade.py           hotkey MANUAL trading terminal (live only) — see the PRD
src/predictor/                 the real engine: strategy.py (the ~4k-line core), collector,
                               feeds, orderbook, pnl — and the live executor
ml/                            XGBoost: offline model inspection, training scripts, a sample
models/                        the trained XGBoost models (lock + per-slice direction + hold7s)
docs/QUICK_TRADE_PRD.md        product write-up of the manual trading tool
```

Requires **Python 3.10+** (3.13 recommended). Set up once:

```bash
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
```

| What | Command | Wallet keys? |
|---|---|---|
| Strategy engine, **paper** — reads live data, simulates fills, prints records | `python tools/chainlink_predictor.py --shares 5` | no |
| Inspect the trained **XGBoost** models (offline, instant) | `python ml/xgb_eval.py` | no |
| Strategy engine, **live** — real automated orders | `python tools/chainlink_predictor.py --trade --shares 5` | yes |
| **Manual** trading terminal — real orders (run the engine in another terminal for prices) | `python tools/quick_trade.py` | yes |

**Paper mode** connects to Polymarket + 8 exchange WebSockets and needs nothing but internet and the market being open. (Binance blocks some regions with HTTP 451 — that source drops out and the reference uses the rest; non-fatal.)

**Live trading** needs your own wallet in `src/predictor/.env` (copy `.env.example` and fill it in) **and** a Polymarket CLOB client. The live path uses a client fork that is not on PyPI, so **live execution isn't reproducible out of the box; paper is.** No keys ship with this repo. Re-read [About this repository](#about-this-repository) before trading anything.

---

## System architecture

```
  Coinbase ── WS ──┐
  Binance ─── WS ──┤
  OKX ──────── WS ──┤  8-exchange reference
  Kraken ───── WS ──┼  (staleness-filtered,
  Bybit ────── WS ──┤   bias-corrected,        ┌─►  Strategy  ─►  Executor
  Bitstamp ─── WS ──┤   lead-weighted mean) ──►│   (P_estimate,   (risk gate, fills,
  Gemini ──── REST ─┤                          │    edge, maker    paper PnL, or live
  CryptoCompare REST┘                          │    + taker        orders on the CLOB)
  Polymarket ─ WS (Chainlink price) ──►  Merger ┘    signals)
  Polymarket ─ WS (CLOB order book) ──►  (one clock for many feeds)
```

- **Feeds** — independent async clients per venue (WebSocket where available, 1s REST polls otherwise); each normalizes into a common tick model.
- **Collector / Merger** — combines multi-source ticks into a single time-ordered market state (one clock for many feeds) and reconstructs the reference.
- **Strategy** — computes the reference, the gap, `P_estimate`, and the `edge`; runs the taker/maker strategies and the XGBoost gates; emits a signal only inside the gate (late window, edge threshold).
- **Executor** — simulates fills for paper mode, or (with `--trade` and your keys) signs and posts real orders to the CLOB and settles on-chain.

## Stack & authorship

Python · asyncio · aiohttp · numpy · XGBoost · web3 / py-clob-client (live path). Built **AI-assisted with Claude Code**: I designed the research questions, the market model, the strategy logic, and the architecture; Claude accelerated the implementation. The conclusions and the microstructure reasoning are mine. This public repo is a reconstruction of a larger private system I built and operated.