Good enough for the public
This commit is contained in:
Daniel Sapkota
@@ -478,3 +478,4 @@ logs/
|
||||
positions/
|
||||
data/
|
||||
*.whl
|
||||
session/
|
||||
|
||||
@@ -36,29 +36,11 @@ uv run polymaker --help
|
||||
## Configure
|
||||
|
||||
```bash
|
||||
cp .env.example .env # then edit: PK + BROWSER_ADDRESS
|
||||
cp .env.example .env # then edit two values:
|
||||
```
|
||||
|
||||
### Which wallet address?
|
||||
|
||||
Polymarket's current **deposit-wallet** architecture shows several addresses in
|
||||
the UI, and the labels are inconsistent. What matters:
|
||||
|
||||
- **`BROWSER_ADDRESS` = the funder** — the *smart-contract wallet that actually
|
||||
holds your pUSD and positions*. Depending on the account it may be shown as the
|
||||
"deposit" or "developer" address; the reliable test is which one holds the
|
||||
money. `polymaker doctor` reads the balance so you can confirm.
|
||||
- **`PK` = the private key of your signer** — the EOA (e.g. your MetaMask account)
|
||||
that *owns/controls* that wallet. This is a **different** address than the
|
||||
funder, and that's correct: your key signs on behalf of the wallet that holds
|
||||
the funds. (A "deployer" / factory address, if shown, is Polymarket's shared
|
||||
contract — ignore it.)
|
||||
|
||||
Set `signature_type` in `config/config.toml` to match how the account was made:
|
||||
`3` = POLY_1271 deposit wallet (current default), `2` = older browser-wallet
|
||||
Gnosis Safe, `1` = email/magic proxy, `0` = plain EOA. A wrong type fails loudly
|
||||
(`polymaker doctor` / `livetest` report it). Approvals must have been granted
|
||||
from the funding wallet — trading once in the UI does this.
|
||||
- `PK` — the private key of your signer wallet
|
||||
- `BROWSER_ADDRESS` — your Polymarket address (shown on the profile / developer page)
|
||||
|
||||
Everything else is TOML under [`config/`](config/):
|
||||
|
||||
|
||||
@@ -0,0 +1,132 @@
|
||||
# Operator's field guide
|
||||
|
||||
Practical notes for running this bot live — where to look, what breaks, and what
|
||||
to build next. Written from a real supervised session (Newsom 2028 + Romania PM),
|
||||
so the failure modes below are ones that actually bit us, not hypotheticals.
|
||||
|
||||
## Running & watching
|
||||
|
||||
- **Run exactly ONE engine.** `uv run polymaker run`. If you background it, verify
|
||||
with `pgrep -f "polymaker run"` and `grep -c engine_started <logfile>` — two
|
||||
engines on the same wallet race each other and double-order. (The process tree
|
||||
is zsh→uv→python, so ~3 procs but only **one** `engine_started` line.)
|
||||
- **Capture the log** to a file (`… > live.log 2>&1`). Everything meaningful is a
|
||||
structured line: `requote … regime=… fv=… place=… cancel=… tox=… flowz=…`,
|
||||
`fill …`, `meta_refreshed`, `market_ws_dropped`, `market_halted_by_meta`.
|
||||
- **Watch the stream, don't poll snapshots.** Follow the log and react the instant
|
||||
something fires — a fill, `regime=EVENT/HALTED/REDUCE_ONLY`, `tox=0.1+`, or any
|
||||
`Traceback/quoter_error/divergence`. Polling every N minutes misses the fill +
|
||||
quick move that happens between checks. (See `session/watch.py` / `monitor.py`
|
||||
from the session for a working pattern.)
|
||||
- **Liveness ≠ quiet.** A silent log can mean "healthy and resting" OR "hung." Do a
|
||||
real health probe periodically: open orders on the exchange, positions on-chain,
|
||||
and that each order is inside the reward band.
|
||||
|
||||
## Where it goes wrong (ranked by how much it cost us)
|
||||
|
||||
1. **Adverse selection on thin/gapped books — the big one.** On a market with a
|
||||
sparse book (e.g. a 14¢ air-pocket below the touch), someone can shove the
|
||||
price, fill your resting bid, and leave you holding a directional bag. We rested
|
||||
$75 / 159-share orders on Romania; a seller gapped the market 0.478→0.442 and
|
||||
filled our whole bid → an oversized −$5 long we never wanted.
|
||||
**→ On thin/manipulable markets, rest the *minimum* reward-qualifying size**
|
||||
(the market's `rewardsMinSize`), not larger. A fill should be small and
|
||||
disposable. Big size only belongs on deep books you can offload into.
|
||||
2. **Orders that don't actually score.** To earn rewards an order must be **≥
|
||||
`rewardsMinSize`** shares **AND within `rewardsMaxSpread` of the midpoint**.
|
||||
Below-min orders earn **zero** — easy to miss. `rewardsMinSize` also *changes*
|
||||
(we saw 50→100 live); a stale catalog value silently mis-sizes you. Let the
|
||||
engine refresh metadata from Gamma at startup, and rescan periodically.
|
||||
3. **False regime signals on quiet markets.** Two we hit and fixed:
|
||||
- *False HALT*: staleness measured "time since last book update," so a quiet
|
||||
market halted itself into zero rewards. Gate on the **WS connection** liveness
|
||||
instead (it pings every 5s).
|
||||
- *False TRENDING*: on a market trading ~1×/hour, microprice jitter spikes the
|
||||
short/long vol ratio → TRENDING → size halved → half the reward, for a trend
|
||||
that doesn't exist. Raise `trend_vol_ratio` on thin markets.
|
||||
4. **Churn.** Reprice/resize thresholds too tight → cancel/replace every few
|
||||
seconds → you lose queue position and get sampled out of rewards. Make it
|
||||
**sticky**: raise `reprice_ticks`, `resize_frac`, and the trend thresholds.
|
||||
Resting > reacting for a reward farmer.
|
||||
5. **Fine-tick illusion.** On 0.001-tick markets (prices like 19.3¢) per-share
|
||||
spread is fractions of a cent — profit is **rewards + rebates**, not spread
|
||||
capture. A "+$4 exit" is noise; don't let it set your strategy.
|
||||
6. **Stale reads.** The positions API (`data-api`) lags; during a fast move it
|
||||
showed +$1 while the real book was −$5. Trust the live book / on-chain, not the
|
||||
position endpoint mid-move.
|
||||
|
||||
## Getting out (exits & closing) — learned the hard way
|
||||
|
||||
- **You can't cleanly exit a *large* position on a thin market.** This is the flip
|
||||
side of the min-size rule: small fills unwind easily, big ones don't. To close
|
||||
159 Romania YES we either market-dumped through the gap (VWAP craters ~0.42, then
|
||||
0.33) or rested a limit near mid that **didn't fill as the market drifted away
|
||||
from us**. We ended up eating ~2 ticks of slippage to get flat. If you can't exit
|
||||
a size without moving the book, you never should have been that size.
|
||||
- **Taker fees hit on market-order exits.** Maker fills (resting) pay zero, but
|
||||
closing with a market/marketable order is a *taker* — the Newsom close reported
|
||||
a gross 126.97 but only 122.86 landed (~$4.1 fee); Romania ~$1.5. Budget for it.
|
||||
- **Don't dump into the gap.** On a gapped book, a plain market/FAK sell fills
|
||||
straight through the air-pocket. Floor it: sell only into the near bids and stop
|
||||
before the gap, even if it leaves a small tail to work off.
|
||||
- **Separate the bot's trades from your own.** Our cash showed −$70, which looked
|
||||
alarming — but the bot only lost ~$15.50; the rest was *manual World Cup sports
|
||||
bets* on the same wallet. When tallying bot PnL, filter to the exact tokens the
|
||||
bot traded (it already scopes untracked positions out of its own state/exposure —
|
||||
do the same in your accounting). **Session result: Newsom −$5.27, Romania −$10.24
|
||||
(one bad adverse fill), total ≈ −$15.51** — over-sizing a thin market cost ~$10
|
||||
of that, which min-size would have made ~−$3.
|
||||
|
||||
## Economics (set expectations)
|
||||
|
||||
- **Liquidity rewards** = a *fixed* daily pool split by your Qmin share. Diminishing
|
||||
returns — past ~a third of the pool you're fighting yourself. Sweet spot is a
|
||||
small-to-mid size on a market with a real pool and light competition.
|
||||
- **Maker rebates** = 25% (most markets) or 20% (high-fee) of taker fees,
|
||||
**uncapped** and volume-driven. Modest on quiet markets, dominant on busy ones —
|
||||
but you only earn them on orders that **fill**, so they come coupled with
|
||||
inventory/adverse-selection risk.
|
||||
- **Taker fee** = `rate × p(1-p)` per share. `rate` is 0.04 in the fee schedule and
|
||||
the client library treats it as **4%** (≈2–3% of notional). **Verify against the
|
||||
UI** — if it actually shows 0.4%, every rebate estimate is 10× too high.
|
||||
|
||||
## Recommended next directions
|
||||
|
||||
- **Backtest against historical L2 order-book data — the highest-leverage next
|
||||
step.** Every parameter we tuned this session (churn thresholds, `trend_vol_ratio`,
|
||||
event sensitivity, min-size, exit urgency) was fit by *intuition on live money*.
|
||||
Instead: record the market WS feed (book snapshots + deltas + trade prints) to a
|
||||
dataset, then replay it through the pure quoting/regime core (`strategy/quoting.py`
|
||||
and `strategy/regime.py` are already I/O-free and deterministic — designed for
|
||||
exactly this) to simulate fills, markouts, rewards, and PnL. Then sweep/optimize
|
||||
params per market archetype (deep-liquid vs thin-gappy) offline. Model the two
|
||||
things that actually decide profit: **fill probability** (are we at the touch when
|
||||
a taker crosses?) and **adverse selection** (where's the price 30–60s after a
|
||||
fill?). This turns the live losses above into a one-time data-collection cost.
|
||||
- **Confirm the fee rate** (4% vs 0.4%) from a real taker fill / the UI — it 10×'s
|
||||
all rebate math.
|
||||
- **Per-fill markout logging** + a **reward-band watchdog** alert (fires if a
|
||||
resting order drifts outside the band, i.e. stops scoring). We were half-blind to
|
||||
toxicity until we added `tox`/`flowz` to the requote line — go further.
|
||||
- **Refine rebate estimates** with each shortlisted market's *actual* `/trades`
|
||||
volume (the scanner uses Gamma's 24h figure, which overstates CLOB flow).
|
||||
- **Wire the alerts webhook** (`Alerter`) before any unattended run.
|
||||
- **Market selection**: rank by reward pool + rebate pool, prefer *deep* books and
|
||||
*light* competition; treat thin gapped books as min-size-only.
|
||||
- **Exit tuning per market**; consider re-enabling the merge path for hedged
|
||||
YES+NO pairs (currently gated off for deposit wallets).
|
||||
|
||||
## Quick reference
|
||||
|
||||
```bash
|
||||
uv run polymaker scan # discover + rank markets -> markets.csv, state.db
|
||||
uv run polymaker doctor # preflight: wallet, clock, WS, balances
|
||||
uv run polymaker moneydoctor # live buy/sell/limit self-test (spends a little)
|
||||
uv run polymaker run # start the maker (ONE instance)
|
||||
uv run polymaker cancel-all # pull every resting order
|
||||
```
|
||||
|
||||
Config lives in `config/*.toml`: `config.toml` (wallet/engine/risk), `strategy.toml`
|
||||
(named profiles), `markets.toml` (trade list). A heartbeat dead-man switch cancels
|
||||
all orders within ~10s if the engine dies — but that is a safety net, not a reason
|
||||
to leave it unwatched on a thin market.
|
||||
@@ -1,118 +0,0 @@
|
||||
"""One-shot health probe for the live MM session. Prints a compact status block
|
||||
plus ALERT lines for anything that needs attention. Run each monitoring cycle."""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
import httpx
|
||||
from polymaker.config import Config
|
||||
from polymaker.execution.gateway import ExecutionGateway
|
||||
|
||||
LOG = "session/live.log"
|
||||
NEWS_Y = "54533043819946592547517511176940999955633860128497669742211153063842200957669"
|
||||
NEWS_N = "87854174148074652060467921081181402357467303721471806610111179101805869578687"
|
||||
FUNDER = "0xb84ca5f197A73429F608842cD75ebbC7c578e169"
|
||||
|
||||
|
||||
def logstats(cid: str) -> dict:
|
||||
txt = open(LOG).read()
|
||||
lines = [l for l in txt.splitlines() if f"cid={cid}" in l]
|
||||
regimes = re.findall(rf"cid={cid}.*?regime=([A-Z_]+)", "\n".join(lines))
|
||||
cancels = sum(1 for l in lines if re.search(r"cancel=[1-9]", l))
|
||||
tox = [float(x) for x in re.findall(rf"cid={cid}.*?tox=([0-9.]+)", "\n".join(lines))]
|
||||
return {
|
||||
"reqs": len(lines),
|
||||
"cancels": cancels,
|
||||
"halts": sum(1 for r in regimes if r == "HALTED"),
|
||||
"last_regime": regimes[-1] if regimes else "?",
|
||||
"maxtox": max(tox) if tox else 0.0,
|
||||
}
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
alerts: list[str] = []
|
||||
engines = int(subprocess.run(["grep", "-c", "engine_started", LOG],
|
||||
capture_output=True, text=True).stdout.strip() or 0)
|
||||
procs = subprocess.run("pgrep -f 'polymaker run'", shell=True,
|
||||
capture_output=True, text=True).stdout.split()
|
||||
alive = len(procs) > 0
|
||||
if not alive:
|
||||
alerts.append("BOT PROCESS DEAD")
|
||||
|
||||
full = open(LOG).read()
|
||||
errors = len(re.findall(r"Traceback|quoter_error|reconcile_error|task_died|divergence", full))
|
||||
fills = len(re.findall(r"\] fill ", full))
|
||||
if errors:
|
||||
alerts.append(f"{errors} error/divergence lines in log")
|
||||
|
||||
gw = ExecutionGateway(Config.load("config"))
|
||||
await gw.connect()
|
||||
naz = httpx.get("https://gamma-api.polymarket.com/markets",
|
||||
params={"slug": "will-alexandru-nazare-be-the-next-prime-minister-of-romania"},
|
||||
timeout=15).json()[0]
|
||||
NY, NN = json.loads(naz["clobTokenIds"])
|
||||
band = float(naz["rewardsMaxSpread"]) / 100.0
|
||||
|
||||
async def mid(tok: str) -> float:
|
||||
b = httpx.get("https://clob.polymarket.com/book", params={"token_id": tok}, timeout=15).json()
|
||||
bb = max((float(x["price"]) for x in b["bids"]), default=0)
|
||||
ba = min((float(x["price"]) for x in b["asks"]), default=1)
|
||||
return (bb + ba) / 2
|
||||
|
||||
naz_mid = await mid(NY)
|
||||
news_mid = await mid(NEWS_Y)
|
||||
oo = await gw.open_orders()
|
||||
|
||||
def fmt(tok_yes, tok_no, m, label):
|
||||
rows = []
|
||||
n_inband = 0
|
||||
for o in oo:
|
||||
if o.token_id in (tok_yes, tok_no):
|
||||
eff = o.price if o.token_id == tok_yes else round(1 - o.price, 3)
|
||||
d = abs(eff - m)
|
||||
inb = d <= band if label == "ROM" else d <= 0.055
|
||||
n_inband += inb
|
||||
rows.append(f"{'Y' if o.token_id==tok_yes else 'N'}:{o.side.value[0]}{o.size:.0f}@{o.price}({d*100:.1f}c{'✓' if inb else '✗OUT'})")
|
||||
return rows, n_inband
|
||||
|
||||
rrows, rin = fmt(NY, NN, naz_mid, "ROM")
|
||||
nrows, nin = fmt(NEWS_Y, NEWS_N, news_mid, "NEWS")
|
||||
|
||||
# positions
|
||||
pos = {}
|
||||
try:
|
||||
for p in httpx.get("https://data-api.polymarket.com/positions",
|
||||
params={"user": FUNDER}, timeout=15).json():
|
||||
if p.get("asset") in (NEWS_Y, NY, NN):
|
||||
pos[p["asset"]] = (float(p["size"]), float(p["avgPrice"]), float(p.get("curPrice", 0)))
|
||||
except Exception:
|
||||
pass
|
||||
pusd = await gw.collateral_balance()
|
||||
|
||||
rs, ns = logstats("0xabc341"), logstats("0x0f49db")
|
||||
if not alive or engines != 1:
|
||||
pass
|
||||
if rs["halts"] > 0:
|
||||
alerts.append(f"ROMANIA {rs['halts']} HALTs")
|
||||
if rs["maxtox"] > 0.15 or ns["maxtox"] > 0.15:
|
||||
alerts.append(f"TOXICITY spike ROM={rs['maxtox']} NEWS={ns['maxtox']}")
|
||||
for rows, m, lbl in ((rrows, naz_mid, "ROM"), (nrows, news_mid, "NEWS")):
|
||||
if any("OUT" in r for r in rows):
|
||||
alerts.append(f"{lbl} order OUT OF BAND")
|
||||
if NY in pos or NN in pos:
|
||||
alerts.append(f"ROMANIA FILLED: {[(k[:6], round(v[0])) for k,v in pos.items() if k in (NY,NN)]}")
|
||||
|
||||
ny = pos.get(NEWS_Y, (668, 0.1917, news_mid))
|
||||
news_unreal = ny[0] * (ny[2] - ny[1]) if ny[2] else 0.0
|
||||
|
||||
print(f"engines={engines} alive={alive} fills={fills} errors={errors} pUSD={pusd:.0f}")
|
||||
print(f"NEWSOM mid={news_mid:.3f} pos={ny[0]:.0f}Y@{ny[1]:.4f} unreal=${news_unreal:+.2f} "
|
||||
f"| {' '.join(nrows)} inband={nin} | reg={ns['last_regime']} reqs={ns['reqs']} cxl={ns['cancels']} halt={ns['halts']} tox={ns['maxtox']}")
|
||||
print(f"ROMANIA mid={naz_mid:.3f} pos={'FILLED' if (NY in pos or NN in pos) else 'flat'} "
|
||||
f"| {' '.join(rrows)} inband={rin} | reg={rs['last_regime']} reqs={rs['reqs']} cxl={rs['cancels']} halt={rs['halts']} tox={rs['maxtox']}")
|
||||
print("ALERTS: " + (" || ".join(alerts) if alerts else "none — all nominal"))
|
||||
|
||||
|
||||
asyncio.run(main())
|
||||
@@ -1,42 +0,0 @@
|
||||
"""Live event watcher: follow live.log and return the INSTANT a significant
|
||||
event fires (fill / regime escalation / toxicity / error), streaming benign
|
||||
notes as they pass. Blocks up to `maxwait` seconds when the tape is quiet, so
|
||||
control returns periodically even with no events. This is event-driven watching
|
||||
— no polling gaps — not interval snapshots."""
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
|
||||
LOG = "session/live.log"
|
||||
# events that demand my immediate attention -> return NOW
|
||||
CRIT = re.compile(
|
||||
r"\] fill |Traceback|quoter_error|reconcile_error|task_died|divergence|"
|
||||
r"quarantine|market_blind|inflight_expired|regime=HALTED|regime=EVENT|"
|
||||
r"regime=REDUCE_ONLY|tox=0\.[1-9]|market_halted_by_meta|risk_halt|_kill"
|
||||
)
|
||||
# worth surfacing but not alarming -> print and keep watching
|
||||
NOTE = re.compile(
|
||||
r"meta_refreshed|user_ws_reconnected|market_ws_dropped|position_forced|"
|
||||
r"untracked_positions|book_drift|market_halted|pagination"
|
||||
)
|
||||
|
||||
maxwait = float(sys.argv[1]) if len(sys.argv) > 1 else 540.0
|
||||
f = open(LOG)
|
||||
f.seek(0, 2) # tail from end
|
||||
start = time.time()
|
||||
fired = False
|
||||
while time.time() - start < maxwait:
|
||||
line = f.readline()
|
||||
if not line:
|
||||
time.sleep(0.4)
|
||||
continue
|
||||
if "HTTP Request" in line or "heartbeats" in line:
|
||||
continue
|
||||
if CRIT.search(line):
|
||||
print("!! CRIT ", line.strip()[:230], flush=True)
|
||||
fired = True
|
||||
break
|
||||
if NOTE.search(line):
|
||||
print(".. note ", line.strip()[:190], flush=True)
|
||||
elapsed = int(time.time() - start)
|
||||
print(f"[{'CRITICAL — reacting' if fired else f'quiet {elapsed}s — re-arming'}]")
|
||||
Reference in New Issue
Block a user