diff --git a/.gitignore b/.gitignore index 730dc55..fd5d030 100644 --- a/.gitignore +++ b/.gitignore @@ -478,3 +478,4 @@ logs/ positions/ data/ *.whl +session/ diff --git a/README.md b/README.md index 8c49adb..fcc721d 100644 --- a/README.md +++ b/README.md @@ -36,29 +36,11 @@ uv run polymaker --help ## Configure ```bash -cp .env.example .env # then edit: PK + BROWSER_ADDRESS +cp .env.example .env # then edit two values: ``` -### Which wallet address? - -Polymarket's current **deposit-wallet** architecture shows several addresses in -the UI, and the labels are inconsistent. What matters: - -- **`BROWSER_ADDRESS` = the funder** — the *smart-contract wallet that actually - holds your pUSD and positions*. Depending on the account it may be shown as the - "deposit" or "developer" address; the reliable test is which one holds the - money. `polymaker doctor` reads the balance so you can confirm. -- **`PK` = the private key of your signer** — the EOA (e.g. your MetaMask account) - that *owns/controls* that wallet. This is a **different** address than the - funder, and that's correct: your key signs on behalf of the wallet that holds - the funds. (A "deployer" / factory address, if shown, is Polymarket's shared - contract — ignore it.) - -Set `signature_type` in `config/config.toml` to match how the account was made: -`3` = POLY_1271 deposit wallet (current default), `2` = older browser-wallet -Gnosis Safe, `1` = email/magic proxy, `0` = plain EOA. A wrong type fails loudly -(`polymaker doctor` / `livetest` report it). Approvals must have been granted -from the funding wallet — trading once in the UI does this. +- `PK` — the private key of your signer wallet +- `BROWSER_ADDRESS` — your Polymarket address (shown on the profile / developer page) Everything else is TOML under [`config/`](config/): diff --git a/TIPS.md b/TIPS.md new file mode 100644 index 0000000..120cecc --- /dev/null +++ b/TIPS.md @@ -0,0 +1,132 @@ +# Operator's field guide + +Practical notes for running this bot live — where to look, what breaks, and what +to build next. Written from a real supervised session (Newsom 2028 + Romania PM), +so the failure modes below are ones that actually bit us, not hypotheticals. + +## Running & watching + +- **Run exactly ONE engine.** `uv run polymaker run`. If you background it, verify + with `pgrep -f "polymaker run"` and `grep -c engine_started ` — two + engines on the same wallet race each other and double-order. (The process tree + is zsh→uv→python, so ~3 procs but only **one** `engine_started` line.) +- **Capture the log** to a file (`… > live.log 2>&1`). Everything meaningful is a + structured line: `requote … regime=… fv=… place=… cancel=… tox=… flowz=…`, + `fill …`, `meta_refreshed`, `market_ws_dropped`, `market_halted_by_meta`. +- **Watch the stream, don't poll snapshots.** Follow the log and react the instant + something fires — a fill, `regime=EVENT/HALTED/REDUCE_ONLY`, `tox=0.1+`, or any + `Traceback/quoter_error/divergence`. Polling every N minutes misses the fill + + quick move that happens between checks. (See `session/watch.py` / `monitor.py` + from the session for a working pattern.) +- **Liveness ≠ quiet.** A silent log can mean "healthy and resting" OR "hung." Do a + real health probe periodically: open orders on the exchange, positions on-chain, + and that each order is inside the reward band. + +## Where it goes wrong (ranked by how much it cost us) + +1. **Adverse selection on thin/gapped books — the big one.** On a market with a + sparse book (e.g. a 14¢ air-pocket below the touch), someone can shove the + price, fill your resting bid, and leave you holding a directional bag. We rested + $75 / 159-share orders on Romania; a seller gapped the market 0.478→0.442 and + filled our whole bid → an oversized −$5 long we never wanted. + **→ On thin/manipulable markets, rest the *minimum* reward-qualifying size** + (the market's `rewardsMinSize`), not larger. A fill should be small and + disposable. Big size only belongs on deep books you can offload into. +2. **Orders that don't actually score.** To earn rewards an order must be **≥ + `rewardsMinSize`** shares **AND within `rewardsMaxSpread` of the midpoint**. + Below-min orders earn **zero** — easy to miss. `rewardsMinSize` also *changes* + (we saw 50→100 live); a stale catalog value silently mis-sizes you. Let the + engine refresh metadata from Gamma at startup, and rescan periodically. +3. **False regime signals on quiet markets.** Two we hit and fixed: + - *False HALT*: staleness measured "time since last book update," so a quiet + market halted itself into zero rewards. Gate on the **WS connection** liveness + instead (it pings every 5s). + - *False TRENDING*: on a market trading ~1×/hour, microprice jitter spikes the + short/long vol ratio → TRENDING → size halved → half the reward, for a trend + that doesn't exist. Raise `trend_vol_ratio` on thin markets. +4. **Churn.** Reprice/resize thresholds too tight → cancel/replace every few + seconds → you lose queue position and get sampled out of rewards. Make it + **sticky**: raise `reprice_ticks`, `resize_frac`, and the trend thresholds. + Resting > reacting for a reward farmer. +5. **Fine-tick illusion.** On 0.001-tick markets (prices like 19.3¢) per-share + spread is fractions of a cent — profit is **rewards + rebates**, not spread + capture. A "+$4 exit" is noise; don't let it set your strategy. +6. **Stale reads.** The positions API (`data-api`) lags; during a fast move it + showed +$1 while the real book was −$5. Trust the live book / on-chain, not the + position endpoint mid-move. + +## Getting out (exits & closing) — learned the hard way + +- **You can't cleanly exit a *large* position on a thin market.** This is the flip + side of the min-size rule: small fills unwind easily, big ones don't. To close + 159 Romania YES we either market-dumped through the gap (VWAP craters ~0.42, then + 0.33) or rested a limit near mid that **didn't fill as the market drifted away + from us**. We ended up eating ~2 ticks of slippage to get flat. If you can't exit + a size without moving the book, you never should have been that size. +- **Taker fees hit on market-order exits.** Maker fills (resting) pay zero, but + closing with a market/marketable order is a *taker* — the Newsom close reported + a gross 126.97 but only 122.86 landed (~$4.1 fee); Romania ~$1.5. Budget for it. +- **Don't dump into the gap.** On a gapped book, a plain market/FAK sell fills + straight through the air-pocket. Floor it: sell only into the near bids and stop + before the gap, even if it leaves a small tail to work off. +- **Separate the bot's trades from your own.** Our cash showed −$70, which looked + alarming — but the bot only lost ~$15.50; the rest was *manual World Cup sports + bets* on the same wallet. When tallying bot PnL, filter to the exact tokens the + bot traded (it already scopes untracked positions out of its own state/exposure — + do the same in your accounting). **Session result: Newsom −$5.27, Romania −$10.24 + (one bad adverse fill), total ≈ −$15.51** — over-sizing a thin market cost ~$10 + of that, which min-size would have made ~−$3. + +## Economics (set expectations) + +- **Liquidity rewards** = a *fixed* daily pool split by your Qmin share. Diminishing + returns — past ~a third of the pool you're fighting yourself. Sweet spot is a + small-to-mid size on a market with a real pool and light competition. +- **Maker rebates** = 25% (most markets) or 20% (high-fee) of taker fees, + **uncapped** and volume-driven. Modest on quiet markets, dominant on busy ones — + but you only earn them on orders that **fill**, so they come coupled with + inventory/adverse-selection risk. +- **Taker fee** = `rate × p(1-p)` per share. `rate` is 0.04 in the fee schedule and + the client library treats it as **4%** (≈2–3% of notional). **Verify against the + UI** — if it actually shows 0.4%, every rebate estimate is 10× too high. + +## Recommended next directions + +- **Backtest against historical L2 order-book data — the highest-leverage next + step.** Every parameter we tuned this session (churn thresholds, `trend_vol_ratio`, + event sensitivity, min-size, exit urgency) was fit by *intuition on live money*. + Instead: record the market WS feed (book snapshots + deltas + trade prints) to a + dataset, then replay it through the pure quoting/regime core (`strategy/quoting.py` + and `strategy/regime.py` are already I/O-free and deterministic — designed for + exactly this) to simulate fills, markouts, rewards, and PnL. Then sweep/optimize + params per market archetype (deep-liquid vs thin-gappy) offline. Model the two + things that actually decide profit: **fill probability** (are we at the touch when + a taker crosses?) and **adverse selection** (where's the price 30–60s after a + fill?). This turns the live losses above into a one-time data-collection cost. +- **Confirm the fee rate** (4% vs 0.4%) from a real taker fill / the UI — it 10×'s + all rebate math. +- **Per-fill markout logging** + a **reward-band watchdog** alert (fires if a + resting order drifts outside the band, i.e. stops scoring). We were half-blind to + toxicity until we added `tox`/`flowz` to the requote line — go further. +- **Refine rebate estimates** with each shortlisted market's *actual* `/trades` + volume (the scanner uses Gamma's 24h figure, which overstates CLOB flow). +- **Wire the alerts webhook** (`Alerter`) before any unattended run. +- **Market selection**: rank by reward pool + rebate pool, prefer *deep* books and + *light* competition; treat thin gapped books as min-size-only. +- **Exit tuning per market**; consider re-enabling the merge path for hedged + YES+NO pairs (currently gated off for deposit wallets). + +## Quick reference + +```bash +uv run polymaker scan # discover + rank markets -> markets.csv, state.db +uv run polymaker doctor # preflight: wallet, clock, WS, balances +uv run polymaker moneydoctor # live buy/sell/limit self-test (spends a little) +uv run polymaker run # start the maker (ONE instance) +uv run polymaker cancel-all # pull every resting order +``` + +Config lives in `config/*.toml`: `config.toml` (wallet/engine/risk), `strategy.toml` +(named profiles), `markets.toml` (trade list). A heartbeat dead-man switch cancels +all orders within ~10s if the engine dies — but that is a safety net, not a reason +to leave it unwatched on a thin market. diff --git a/session/monitor.py b/session/monitor.py deleted file mode 100644 index 36ea79a..0000000 --- a/session/monitor.py +++ /dev/null @@ -1,118 +0,0 @@ -"""One-shot health probe for the live MM session. Prints a compact status block -plus ALERT lines for anything that needs attention. Run each monitoring cycle.""" -from __future__ import annotations - -import asyncio -import json -import re -import subprocess -import httpx -from polymaker.config import Config -from polymaker.execution.gateway import ExecutionGateway - -LOG = "session/live.log" -NEWS_Y = "54533043819946592547517511176940999955633860128497669742211153063842200957669" -NEWS_N = "87854174148074652060467921081181402357467303721471806610111179101805869578687" -FUNDER = "0xb84ca5f197A73429F608842cD75ebbC7c578e169" - - -def logstats(cid: str) -> dict: - txt = open(LOG).read() - lines = [l for l in txt.splitlines() if f"cid={cid}" in l] - regimes = re.findall(rf"cid={cid}.*?regime=([A-Z_]+)", "\n".join(lines)) - cancels = sum(1 for l in lines if re.search(r"cancel=[1-9]", l)) - tox = [float(x) for x in re.findall(rf"cid={cid}.*?tox=([0-9.]+)", "\n".join(lines))] - return { - "reqs": len(lines), - "cancels": cancels, - "halts": sum(1 for r in regimes if r == "HALTED"), - "last_regime": regimes[-1] if regimes else "?", - "maxtox": max(tox) if tox else 0.0, - } - - -async def main() -> None: - alerts: list[str] = [] - engines = int(subprocess.run(["grep", "-c", "engine_started", LOG], - capture_output=True, text=True).stdout.strip() or 0) - procs = subprocess.run("pgrep -f 'polymaker run'", shell=True, - capture_output=True, text=True).stdout.split() - alive = len(procs) > 0 - if not alive: - alerts.append("BOT PROCESS DEAD") - - full = open(LOG).read() - errors = len(re.findall(r"Traceback|quoter_error|reconcile_error|task_died|divergence", full)) - fills = len(re.findall(r"\] fill ", full)) - if errors: - alerts.append(f"{errors} error/divergence lines in log") - - gw = ExecutionGateway(Config.load("config")) - await gw.connect() - naz = httpx.get("https://gamma-api.polymarket.com/markets", - params={"slug": "will-alexandru-nazare-be-the-next-prime-minister-of-romania"}, - timeout=15).json()[0] - NY, NN = json.loads(naz["clobTokenIds"]) - band = float(naz["rewardsMaxSpread"]) / 100.0 - - async def mid(tok: str) -> float: - b = httpx.get("https://clob.polymarket.com/book", params={"token_id": tok}, timeout=15).json() - bb = max((float(x["price"]) for x in b["bids"]), default=0) - ba = min((float(x["price"]) for x in b["asks"]), default=1) - return (bb + ba) / 2 - - naz_mid = await mid(NY) - news_mid = await mid(NEWS_Y) - oo = await gw.open_orders() - - def fmt(tok_yes, tok_no, m, label): - rows = [] - n_inband = 0 - for o in oo: - if o.token_id in (tok_yes, tok_no): - eff = o.price if o.token_id == tok_yes else round(1 - o.price, 3) - d = abs(eff - m) - inb = d <= band if label == "ROM" else d <= 0.055 - n_inband += inb - rows.append(f"{'Y' if o.token_id==tok_yes else 'N'}:{o.side.value[0]}{o.size:.0f}@{o.price}({d*100:.1f}c{'✓' if inb else '✗OUT'})") - return rows, n_inband - - rrows, rin = fmt(NY, NN, naz_mid, "ROM") - nrows, nin = fmt(NEWS_Y, NEWS_N, news_mid, "NEWS") - - # positions - pos = {} - try: - for p in httpx.get("https://data-api.polymarket.com/positions", - params={"user": FUNDER}, timeout=15).json(): - if p.get("asset") in (NEWS_Y, NY, NN): - pos[p["asset"]] = (float(p["size"]), float(p["avgPrice"]), float(p.get("curPrice", 0))) - except Exception: - pass - pusd = await gw.collateral_balance() - - rs, ns = logstats("0xabc341"), logstats("0x0f49db") - if not alive or engines != 1: - pass - if rs["halts"] > 0: - alerts.append(f"ROMANIA {rs['halts']} HALTs") - if rs["maxtox"] > 0.15 or ns["maxtox"] > 0.15: - alerts.append(f"TOXICITY spike ROM={rs['maxtox']} NEWS={ns['maxtox']}") - for rows, m, lbl in ((rrows, naz_mid, "ROM"), (nrows, news_mid, "NEWS")): - if any("OUT" in r for r in rows): - alerts.append(f"{lbl} order OUT OF BAND") - if NY in pos or NN in pos: - alerts.append(f"ROMANIA FILLED: {[(k[:6], round(v[0])) for k,v in pos.items() if k in (NY,NN)]}") - - ny = pos.get(NEWS_Y, (668, 0.1917, news_mid)) - news_unreal = ny[0] * (ny[2] - ny[1]) if ny[2] else 0.0 - - print(f"engines={engines} alive={alive} fills={fills} errors={errors} pUSD={pusd:.0f}") - print(f"NEWSOM mid={news_mid:.3f} pos={ny[0]:.0f}Y@{ny[1]:.4f} unreal=${news_unreal:+.2f} " - f"| {' '.join(nrows)} inband={nin} | reg={ns['last_regime']} reqs={ns['reqs']} cxl={ns['cancels']} halt={ns['halts']} tox={ns['maxtox']}") - print(f"ROMANIA mid={naz_mid:.3f} pos={'FILLED' if (NY in pos or NN in pos) else 'flat'} " - f"| {' '.join(rrows)} inband={rin} | reg={rs['last_regime']} reqs={rs['reqs']} cxl={rs['cancels']} halt={rs['halts']} tox={rs['maxtox']}") - print("ALERTS: " + (" || ".join(alerts) if alerts else "none — all nominal")) - - -asyncio.run(main()) diff --git a/session/watch.py b/session/watch.py deleted file mode 100644 index ebbb934..0000000 --- a/session/watch.py +++ /dev/null @@ -1,42 +0,0 @@ -"""Live event watcher: follow live.log and return the INSTANT a significant -event fires (fill / regime escalation / toxicity / error), streaming benign -notes as they pass. Blocks up to `maxwait` seconds when the tape is quiet, so -control returns periodically even with no events. This is event-driven watching -— no polling gaps — not interval snapshots.""" -import re -import sys -import time - -LOG = "session/live.log" -# events that demand my immediate attention -> return NOW -CRIT = re.compile( - r"\] fill |Traceback|quoter_error|reconcile_error|task_died|divergence|" - r"quarantine|market_blind|inflight_expired|regime=HALTED|regime=EVENT|" - r"regime=REDUCE_ONLY|tox=0\.[1-9]|market_halted_by_meta|risk_halt|_kill" -) -# worth surfacing but not alarming -> print and keep watching -NOTE = re.compile( - r"meta_refreshed|user_ws_reconnected|market_ws_dropped|position_forced|" - r"untracked_positions|book_drift|market_halted|pagination" -) - -maxwait = float(sys.argv[1]) if len(sys.argv) > 1 else 540.0 -f = open(LOG) -f.seek(0, 2) # tail from end -start = time.time() -fired = False -while time.time() - start < maxwait: - line = f.readline() - if not line: - time.sleep(0.4) - continue - if "HTTP Request" in line or "heartbeats" in line: - continue - if CRIT.search(line): - print("!! CRIT ", line.strip()[:230], flush=True) - fired = True - break - if NOTE.search(line): - print(".. note ", line.strip()[:190], flush=True) -elapsed = int(time.time() - start) -print(f"[{'CRITICAL — reacting' if fired else f'quiet {elapsed}s — re-arming'}]")