gavindiaz/poly-maker
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #32 from warproxxx/v2-rewrite

Browse Source 
V2 rewrite
This commit is contained in:
warproxxx
2026-07-05 18:08:36 -04:00
committed by GitHub
parent bfacef317b b3f25e0fa7
commit 04179bc344
84 changed files with 6848 additions and 7964 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.env.example
+24 -6
View File
@@ -1,7 +1,25 @@
# Polymarket Authentication
PK=your_private_key_here
BROWSER_ADDRESS=your_wallet_address_here
# polymaker secrets. Copy to .env and fill in. NEVER commit .env.
#
# IMPORTANT: use the SAME wallet you use in the Polymarket browser UI, and make
# sure it has traded at least once through the UI so allowances are set.
# Google Sheets (for data_updater)
SPREADSHEET_URL=https://docs.google.com/spreadsheets/d/1Kt6yGY7CZpB75cLJJAdWo7LSp9Oz7pjqfuVWwgtn7Ns/edit?gid=97507557#gid=97507557
#replace with YOUR url
# Private key of the signing wallet (EOA that controls the funder).
PK=
# The funder = the smart-contract wallet that actually holds your pUSD and
# positions (NOT your signing EOA/MetaMask address above). Polymarket's UI labels
# this inconsistently (may show as "deposit" or "developer" address) — the one
# that holds the funds is the funder. `polymaker doctor` reads the balance so you
# can confirm you picked the right one. Set signature_type in config/config.toml
# to match how the account was made (new deposit wallets = 3).
BROWSER_ADDRESS=
# Optional: override the default public Polygon RPC with your own (Alchemy/Infura).
# POLYGON_RPC=
# Optional: webhook (Discord/Telegram/ntfy) POST URL for critical alerts.
# ALERT_WEBHOOK_URL=
# Optional: route outbound traffic through an SSH tunnel / proxy, e.g. to test
# from a colocated box. Standard env var; httpx and web3 pick it up automatically.
# ALL_PROXY=socks5://127.0.0.1:1080
.gitignore
+10
View File
@@ -467,3 +467,13 @@ cython_debug/
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
# polymaker v2
.venv/
state.db
state.db-*
journal/
logs/
positions/
data/
*.whl
.python-version
+1 -1
View File
@@ -1 +1 @@
3.9.18
3.12
README.md
+149 -145
View File
@@ -1,159 +1,163 @@
# Poly-Maker
# poly-maker
A market making bot for Polymarket prediction markets. This bot automates the process of providing liquidity to markets on Polymarket by maintaining orders on both sides of the book with configurable parameters. A summary of my experience running this bot is available [here](https://x.com/defiance_cr/status/1906774862254800934)
A maker-only market-making bot for **Polymarket CLOB V2**, focused on political
markets. Single async process, local-file config (no Google Sheets), typed and
tested.
## Overview
Poly-Maker is a comprehensive solution for automated market making on Polymarket. It includes:
- Real-time order book monitoring via WebSockets
- Position management with risk controls
- Customizable trade parameters fetched from Google Sheets
- Automated position merging functionality
- Sophisticated spread and price management
## Structure
The repository consists of several interconnected modules:
- `poly_data`: Core data management and market making logic
- `poly_merger`: Utility for merging positions (based on open-source Polymarket code)
- `poly_stats`: Account statistics tracking
- `poly_utils`: Shared utility functions
- `data_updater`: Separate module for collecting market information
## Requirements
- Python 3.9.10 or higher
- Node.js (for poly_merger)
- Google Sheets API credentials
- Polymarket account and API credentials
## Installation
This project uses UV for fast, reliable package management.
### Install UV
```bash
# macOS/Linux
curl -LsSf https://astral.sh/uv/install.sh | sh
# Windows
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
# Or with pip
pip install uv
```
### Install Dependencies
```bash
# Install all dependencies
uv sync
# Install with development dependencies (black, pytest)
uv sync --extra dev
```
### Quick Start
```bash
# Run the market maker (recommended)
uv run python main.py
# Update market data
uv run python update_markets.py
# Update statistics
uv run python update_stats.py
```
### Setup Steps
#### 1. Clone the repository
```bash
git clone https://github.com/yourusername/poly-maker.git
cd poly-maker
```
#### 2. Install Python dependencies
```bash
uv sync
```
#### 3. Install Node.js dependencies for the merger
```bash
cd poly_merger
npm install
cd ..
```
#### 4. Set up environment variables
```bash
cp .env.example .env
```
#### 5. Configure your credentials in `.env`
Edit the `.env` file with your credentials:
- `PK`: Your private key for Polymarket
- `BROWSER_ADDRESS`: Your wallet address
**Important:** Make sure your wallet has done at least one trade through the UI so that the permissions are proper.
#### 6. Set up Google Sheets integration
- Create a Google Service Account and download credentials to the main directory
- Copy the [sample Google Sheet](https://docs.google.com/spreadsheets/d/1Kt6yGY7CZpB75cLJJAdWo7LSp9Oz7pjqfuVWwgtn7Ns/edit?gid=1884499063#gid=1884499063)
- Add your Google service account to the sheet with edit permissions
- Update `SPREADSHEET_URL` in your `.env` file
#### 7. Update market data
Run the market data updater to fetch all available markets:
```bash
uv run python update_markets.py
```
This should run continuously in the background (preferably on a different IP than your trading bot).
- Add markets you want to trade to the "Selected Markets" sheet
- Select markets from the "Volatility Markets" sheet
- Configure parameters in the "Hyperparameters" sheet (default parameters that worked well in November are included)
#### 8. Start the market making bot
```bash
uv run python main.py
```
> [!WARNING]
> In today's market, this bot is not profitable and will lose money. Use it as a reference implementation for building your own market making strategies, not as a ready-to-deploy solution. Given the increased competition on Polymarket, I don't see a point in playing with this unless you're willing to dedicate a significant amount of time.
> Market making on Polymarket is competitive and can lose money. This is a
> reference implementation and a research harness, not a guaranteed-profitable
> product. Test in `--paper` mode first; go live with small size.
## Configuration
## What it does
The bot is configured via a Google Spreadsheet with several worksheets:
- Discovers political markets via the **Gamma API** (seconds) and ranks them by
reward + rebate income vs. volatility/spread risk.
- Maintains a live order book per token from the **market WebSocket**.
- Quotes **maker-only** — every order is post-only. Fair-value + inventory-skew
strategy that posts BUY-YES and BUY-NO as a two-sided quote, with live
volatility/toxicity estimation and a regime machine that pulls quotes during
news events (see [Strategy](#strategy)).
- Reconciles a target quote set against live orders with churn tolerances; runs
the exchange **heartbeat** dead-man switch; enforces risk caps and a daily-loss
kill switch.
- Config, market selection, and state are **local files + SQLite**. An operator
with the repo, a `.env`, and a funded wallet is a complete deployment.
- **Selected Markets**: Markets you want to trade
- **All Markets**: Database of all markets on Polymarket
- **Hyperparameters**: Configuration parameters for the trading logic
## Install
Uses [uv](https://docs.astral.sh/uv/) and Python 3.12+.
## Poly Merger
```bash
uv sync --extra dev # install deps + dev tools
uv run polymaker --help
```
The `poly_merger` module is a particularly powerful utility that handles position merging on Polymarket. It's built on open-source Polymarket code and provides a smooth way to consolidate positions, reducing gas fees and improving capital efficiency.
## Configure
## Important Notes
```bash
cp .env.example .env # then edit: PK + BROWSER_ADDRESS
```
- This code interacts with real markets and can potentially lose real money
- Test thoroughly with small amounts before deploying with significant capital
- The `data_updater` is technically a separate repository but is included here for convenience
### Which wallet address?
Polymarket's current **deposit-wallet** architecture shows several addresses in
the UI, and the labels are inconsistent. What matters:
- **`BROWSER_ADDRESS` = the funder** — the *smart-contract wallet that actually
holds your pUSD and positions*. Depending on the account it may be shown as the
"deposit" or "developer" address; the reliable test is which one holds the
money. `polymaker doctor` reads the balance so you can confirm.
- **`PK` = the private key of your signer** — the EOA (e.g. your MetaMask account)
that *owns/controls* that wallet. This is a **different** address than the
funder, and that's correct: your key signs on behalf of the wallet that holds
the funds. (A "deployer" / factory address, if shown, is Polymarket's shared
contract — ignore it.)
Set `signature_type` in `config/config.toml` to match how the account was made:
`3` = POLY_1271 deposit wallet (current default), `2` = older browser-wallet
Gnosis Safe, `1` = email/magic proxy, `0` = plain EOA. A wrong type fails loudly
(`polymaker doctor` / `livetest` report it). Approvals must have been granted
from the funding wallet — trading once in the UI does this.
Everything else is TOML under [`config/`](config/):
- `config.toml` — wallet/engine/risk/execution settings
- `strategy.toml` — named parameter profiles (`political-longdated`, `political-hot`)
- `markets.toml` — the trade list (populated via the CLI below)
## Use
```bash
# 1. discover + rank political markets (writes to state.db)
uv run polymaker scan
uv run polymaker markets
# 2. add markets to the trade list
uv run polymaker markets-add <slug> --profile political-longdated
# 3. dry run: full pipeline against the live feed, no orders posted
uv run polymaker run --paper
# 4. preflight the wallet before going live
uv run polymaker doctor
# 5. one safe live round-trip (~$5 post-only order, placed deep and cancelled)
uv run polymaker livetest
# 6. go live
uv run polymaker run
# ops
uv run polymaker status # positions / open orders
uv run polymaker cancel-all # panic button
```
## Architecture
```
market WS ─▶ OrderBook ─▶ (wake) ─▶ Quoter ─▶ strategy (pure) ─▶ reconcile ─▶ ExecutionGateway
user WS ─▶ StateStore RiskManager ┘ (post-only, heartbeat)
Gamma ─▶ Catalog/scanner ─▶ SQLite periodic REST reconcile ┘
```
One async event loop. The strategy layer is a pure function `(book, inventory,
params, clock) → TargetQuotes` — deterministic and unit-tested. The engine owns
all I/O and state around it; the `ExecutionGateway` wraps `py-clob-client-v2`
(which handles the V2 EIP-712 signing) and offloads its blocking calls to a
thread pool so the hot path never stalls. State (positions, orders, PnL, catalog)
lives in one SQLite file; raw WS/order events are journaled to `journal/` for
replay.
## Strategy
Maker-only, quoting both sides of each market as USDC-collateralized bids:
- **Fair value** — depth-weighted microprice off the live book, nudged by an
EWMA of signed trade flow.
- **Quote construction** — reservation price `r = FV − skew(inventory)`;
half-spread `δ = base + c_vol·σ + c_tox·toxicity`. Post **BUY-YES at `r − δ`**
and **BUY-NO at `(1 − r) − δ`**. Because both legs are bids that sum below 1,
a filled pair merges back to USDC at locked edge `1 − p − q` — a maker-only
exit that never crosses the spread.
- **Inventory skew** — net position leans both quotes: long YES → bid YES lower,
bid NO higher (acquire the offsetting leg). Size tapers as inventory approaches
a soft cap, then the adding side is pulled entirely.
- **Volatility / toxicity** — realized-vol and per-fill markout (adverse
selection) EWMAs widen the spread and shrink size in markets that pick us off.
- **Regime machine** — per market: `QUIET` (farm rewards in-band), `TRENDING`
(lean + widen + half size), `EVENT` (sweep/jump detected → pull quotes, cool
off), `REDUCE_ONLY` (inventory cap / near end date → exits only), `HALTED`
(stale data / resolved / kill switch → cancel all).
- **Rewards + rebates** — quotes stay inside the liquidity-rewards band in QUIET;
the market selector also scores the new maker-rebate program (a share of taker
fees rebated to makers).
- **Risk** — per-market notional cap, neg-risk event-group worst-case cap, total
exposure cap, daily-loss kill switch, WS-staleness halt.
Tune it all via profiles in `config/strategy.toml`.
## Develop
```bash
uv run pytest # unit suite (offline)
POLYMAKER_LIVE=1 uv run pytest tests/test_live_marketdata.py # live WS integration
uv run ruff check src tests # lint
uv run mypy src # types (strict)
```
## Status
Implemented and live-verified end to end (auth → book → strategy → sign → post →
cancel): config, catalog/scanner, order book + analytics, strategy (FV,
vol/toxicity, regime, quoting), state store + lifecycle, execution gateway +
reconciler + heartbeat, market/user websockets, risk manager, merger (EOA path),
engine, CLI, paper mode, journal capture. 83 tests; ruff + mypy strict clean.
Not yet built: a replay backtester over the captured journals, and external data
feeds (polls / news / cross-venue). Merging through a Safe/proxy wallet routes a
tx via the relayer and isn't wired yet — until then inventory exits via limit
sells rather than merging.
## License
config/config.toml
+48
View File
@@ -0,0 +1,48 @@
# polymaker global config. Secrets live in .env, never here.
# See the README.
[wallet]
# chain: Polygon mainnet
chain_id = 137
# signature_type — pick based on how your Polymarket account was created:
# 0 = EOA (funds held directly by your key; no proxy)
# 1 = email/magic proxy wallet
# 2 = older browser-wallet Gnosis Safe proxy
# 3 = POLY_1271 deposit wallet (the CURRENT architecture, ~June 2026+)
# If your UI shows a "deposit address", you almost certainly have a deposit
# wallet -> use 3. In every proxy case (1/2/3), BROWSER_ADDRESS is the DEPOSIT
# address (the funder that holds your pUSD), NOT your signing EOA and NOT the
# deployer/factory. Known SDK auth bugs live here (see the README) — `polymaker
# doctor` / `livetest` will tell you if you picked the wrong type.
signature_type = 3
clob_host = "https://clob.polymarket.com"
gamma_host = "https://gamma-api.polymarket.com"
data_api_host = "https://data-api.polymarket.com"
polygon_rpc = "https://polygon-rpc.com"
[engine]
debounce_ms = 200 # min gap between quote recomputes per market
reconcile_interval_s = 30 # REST drift reconciliation cadence
catalog_refresh_s = 900 # market catalog rescan cadence (15 min)
heartbeat = true # exchange dead-man switch
heartbeat_interval_s = 5
journal = true # append raw WS/orders to journal/ for backtest
loop = "uvloop" # "uvloop" | "asyncio"
[risk]
max_total_exposure_usdc = 5000.0 # sum of |position notional| + open buy notional
max_event_group_loss_usdc = 1000.0 # neg-risk group worst-case loss cap
max_market_notional_usdc = 800.0 # per-market position+orders notional cap
daily_loss_kill_usdc = 250.0 # realized daily loss -> halt new quotes
ws_stale_halt_s = 10.0 # no book updates for this long -> halt market
max_order_error_rate = 0.25 # rolling order-post error fraction -> halt
[execution]
rate_budget_fraction = 0.25 # fraction of documented API limits we allow
post_only = true # every quote is post-only (maker-only mandate)
max_orders_per_batch = 15 # exchange batch cap
[paths]
db = "state.db"
journal_dir = "journal"
log_dir = "logs"
config/markets.toml
+11
View File
@@ -0,0 +1,11 @@
# The trade list (replaces the v1 Selected Markets sheet).
# Each entry names a market by slug OR condition_id and a strategy profile.
# `polymaker markets add <slug>` appends here; edit freely, hot-reloaded live.
# Start empty — populate with `polymaker scan` then `polymaker markets`.
# Example (disabled) entry:
# [[markets]]
# slug = "will-the-democrats-win-the-2028-us-presidential-election"
# profile = "political-longdated"
# enabled = false
# q_max_usdc = 800 # optional per-market override of the profile value
config/strategy.toml
+66
View File
@@ -0,0 +1,66 @@
# Named strategy parameter profiles (replaces the v1 Hyperparameters sheet).
# markets.toml maps each market to one of these profiles.
# See the README.
[profiles.political-longdated]
# --- fair value ---
micro_levels = 3 # depth levels used for microprice
flow_ewma_halflife_s = 120 # signed-flow EWMA half-life
# --- spread / skew ---
gamma = 0.5 # inventory risk aversion (skew strength)
delta_min_ticks = 2 # minimum half-spread, in ticks
c_vol = 1.2 # half-spread added per unit short-horizon vol
c_tox = 2.0 # half-spread added per unit toxicity score
# --- vol horizons (seconds) ---
vol_short_halflife_s = 10
vol_long_halflife_s = 900
# --- sizing / inventory ---
base_size_usdc = 50.0 # notional per quote
q_max_usdc = 500.0 # hard inventory cap (notional)
q_soft_frac = 0.6 # soft cap as fraction of q_max
layers = 2 # price levels per side
layer_step_ticks = 2 # tick gap between layers
# --- placement / churn ---
reprice_ticks = 2 # only reprice if target moves this many ticks
resize_frac = 0.15 # or if size drifts this fraction
min_edge_ticks = 1 # never quote inside (FV +/- this) * tick
# --- regime ---
event_cooloff_s = 60
event_jump_ticks = 8 # FV jump over debounce that flags EVENT
event_sweep_levels = 3 # levels consumed in one print that flags EVENT
trend_flow_z = 1.5 # flow z-score that flags TRENDING
# --- lifecycle ---
end_date_taper_days = 7
reduce_only_hours = 24
halt_before_hours = 2
# --- exits ---
exit_urgency_s = 900 # time to walk exit from FV+delta to best-bid+tick
merge_min_size = 20.0 # min(YES,NO) shares to trigger a merge
[profiles.political-hot]
# tighter, defensive profile for high-volatility / event-prone markets
micro_levels = 3
flow_ewma_halflife_s = 60
gamma = 0.9
delta_min_ticks = 3
c_vol = 1.8
c_tox = 3.0
vol_short_halflife_s = 8
vol_long_halflife_s = 600
base_size_usdc = 30.0
q_max_usdc = 250.0
q_soft_frac = 0.5
layers = 2
layer_step_ticks = 3
reprice_ticks = 2
resize_frac = 0.15
min_edge_ticks = 1
event_cooloff_s = 120
event_jump_ticks = 6
event_sweep_levels = 2
trend_flow_z = 1.2
end_date_taper_days = 7
reduce_only_hours = 24
halt_before_hours = 2
exit_urgency_s = 600
merge_min_size = 20.0
data_updater/erc20ABI.json
-279
View File
@@ -1,279 +0,0 @@
[
{
"constant": true,
"inputs": [],
"name": "name",
"outputs": [
{
"name": "",
"type": "string"
}
],
"payable": false,
"stateMutability": "view",
"type": "function"
},
{
"constant": false,
"inputs": [
{
"name": "guy",
"type": "address"
},
{
"name": "wad",
"type": "uint256"
}
],
"name": "approve",
"outputs": [
{
"name": "",
"type": "bool"
}
],
"payable": false,
"stateMutability": "nonpayable",
"type": "function"
},
{
"constant": true,
"inputs": [],
"name": "totalSupply",
"outputs": [
{
"name": "",
"type": "uint256"
}
],
"payable": false,
"stateMutability": "view",
"type": "function"
},
{
"constant": false,
"inputs": [
{
"name": "src",
"type": "address"
},
{
"name": "dst",
"type": "address"
},
{
"name": "wad",
"type": "uint256"
}
],
"name": "transferFrom",
"outputs": [
{
"name": "",
"type": "bool"
}
],
"payable": false,
"stateMutability": "nonpayable",
"type": "function"
},
{
"constant": false,
"inputs": [
{
"name": "wad",
"type": "uint256"
}
],
"name": "withdraw",
"outputs": [],
"payable": false,
"stateMutability": "nonpayable",
"type": "function"
},
{
"constant": true,
"inputs": [],
"name": "decimals",
"outputs": [
{
"name": "",
"type": "uint8"
}
],
"payable": false,
"stateMutability": "view",
"type": "function"
},
{
"constant": true,
"inputs": [
{
"name": "",
"type": "address"
}
],
"name": "balanceOf",
"outputs": [
{
"name": "",
"type": "uint256"
}
],
"payable": false,
"stateMutability": "view",
"type": "function"
},
{
"constant": true,
"inputs": [],
"name": "symbol",
"outputs": [
{
"name": "",
"type": "string"
}
],
"payable": false,
"stateMutability": "view",
"type": "function"
},
{
"constant": false,
"inputs": [
{
"name": "dst",
"type": "address"
},
{
"name": "wad",
"type": "uint256"
}
],
"name": "transfer",
"outputs": [
{
"name": "",
"type": "bool"
}
],
"payable": false,
"stateMutability": "nonpayable",
"type": "function"
},
{
"constant": false,
"inputs": [],
"name": "deposit",
"outputs": [],
"payable": true,
"stateMutability": "payable",
"type": "function"
},
{
"constant": true,
"inputs": [
{
"name": "",
"type": "address"
},
{
"name": "",
"type": "address"
}
],
"name": "allowance",
"outputs": [
{
"name": "",
"type": "uint256"
}
],
"payable": false,
"stateMutability": "view",
"type": "function"
},
{
"payable": true,
"stateMutability": "payable",
"type": "fallback"
},
{
"anonymous": false,
"inputs": [
{
"indexed": true,
"name": "src",
"type": "address"
},
{
"indexed": true,
"name": "guy",
"type": "address"
},
{
"indexed": false,
"name": "wad",
"type": "uint256"
}
],
"name": "Approval",
"type": "event"
},
{
"anonymous": false,
"inputs": [
{
"indexed": true,
"name": "src",
"type": "address"
},
{
"indexed": true,
"name": "dst",
"type": "address"
},
{
"indexed": false,
"name": "wad",
"type": "uint256"
}
],
"name": "Transfer",
"type": "event"
},
{
"anonymous": false,
"inputs": [
{
"indexed": true,
"name": "dst",
"type": "address"
},
{
"indexed": false,
"name": "wad",
"type": "uint256"
}
],
"name": "Deposit",
"type": "event"
},
{
"anonymous": false,
"inputs": [
{
"indexed": true,
"name": "src",
"type": "address"
},
{
"indexed": false,
"name": "wad",
"type": "uint256"
}
],
"name": "Withdrawal",
"type": "event"
}
]
data_updater/find_markets.py
-338
View File
@@ -1,338 +0,0 @@
import pandas as pd
import numpy as np
import os
import requests
import time
import warnings
warnings.filterwarnings("ignore")
if not os.path.exists('data'):
os.makedirs('data')
def get_sel_df(spreadsheet, sheet_name='Selected Markets'):
try:
wk2 = spreadsheet.worksheet(sheet_name)
sel_df = pd.DataFrame(wk2.get_all_records())
sel_df = sel_df[sel_df['question'] != ""].reset_index(drop=True)
return sel_df
except:
return pd.DataFrame()
def get_all_markets(client):
cursor = ""
all_markets = []
while True:
try:
markets = client.get_sampling_markets(next_cursor = cursor)
markets_df = pd.DataFrame(markets['data'])
cursor = markets['next_cursor']
all_markets.append(markets_df)
if cursor is None:
break
except:
break
all_df = pd.concat(all_markets)
all_df = all_df.reset_index(drop=True)
return all_df
def get_bid_ask_range(ret, TICK_SIZE):
bid_from = ret['midpoint'] - ret['max_spread'] / 100
bid_to = ret['best_ask'] #Although bid to this high up will change bid_from because of changing midpoint, take optimistic approach
if bid_to == 0:
bid_to = ret['midpoint']
if bid_to - TICK_SIZE > ret['midpoint']:
bid_to = ret['best_bid'] + (TICK_SIZE + 0.1 * TICK_SIZE)
if bid_from > bid_to:
bid_from = bid_to - (TICK_SIZE + 0.1 * TICK_SIZE)
ask_to = ret['midpoint'] + ret['max_spread'] / 100
ask_from = ret['best_bid']
if ask_from == 0:
ask_from = ret['midpoint']
if ask_from + TICK_SIZE < ret['midpoint']:
ask_from = ret['best_ask'] - (TICK_SIZE + 0.1 * TICK_SIZE)
if ask_from > ask_to:
ask_to = ask_from + (TICK_SIZE + 0.1 * TICK_SIZE)
bid_from = round(bid_from, 3)
bid_to = round(bid_to, 3)
ask_from = round(ask_from, 3)
ask_to = round(ask_to, 3)
if bid_from < 0:
bid_from = 0
if ask_from < 0:
ask_from = 0
return bid_from, bid_to, ask_from, ask_to
def generate_numbers(start, end, TICK_SIZE):
# Calculate the starting point, rounding up to the next hundredth if not an exact multiple of TICK_SIZE
rounded_start = (int(start * 100) + 1) / 100 if start * 100 % 1 != 0 else start + TICK_SIZE
# Calculate the ending point, rounding down to the nearest hundredth
rounded_end = int(end * 100) / 100
# Generate numbers from rounded_start to rounded_end, ensuring they fall strictly within the original bounds
numbers = []
current = rounded_start
while current < end:
numbers.append(current)
current += TICK_SIZE
current = round(current, len(str(TICK_SIZE).split('.')[1])) # Rounding to avoid floating point imprecision
return numbers
def add_formula_params(curr_df, midpoint, v, daily_reward):
curr_df['s'] = (curr_df['price'] - midpoint).abs()
curr_df['S'] = ((v - curr_df['s']) / v) ** 2
curr_df['100'] = 1/curr_df['price'] * 100
curr_df['size'] = curr_df['size'] + curr_df['100']
curr_df['Q'] = curr_df['S'] * curr_df['size']
curr_df['reward_per_100'] = (curr_df['Q'] / curr_df['Q'].sum()) * daily_reward / 2 / curr_df['size'] * curr_df['100']
return curr_df
def process_single_row(row, client):
ret = {}
ret['question'] = row['question']
ret['neg_risk'] = row['neg_risk']
ret['answer1'] = row['tokens'][0]['outcome']
ret['answer2'] = row['tokens'][1]['outcome']
ret['min_size'] = row['rewards']['min_size']
ret['max_spread'] = row['rewards']['max_spread']
token1 = row['tokens'][0]['token_id']
token2 = row['tokens'][1]['token_id']
rate = 0
for rate_info in row['rewards']['rates']:
if rate_info['asset_address'].lower() == '0x2791Bca1f2de4661ED88A30C99A7a9449Aa84174'.lower():
rate = rate_info['rewards_daily_rate']
break
ret['rewards_daily_rate'] = rate
book = client.get_order_book(token1)
bids = pd.DataFrame()
asks = pd.DataFrame()
try:
bids = pd.DataFrame(book.bids).astype(float)
except:
pass
try:
asks = pd.DataFrame(book.asks).astype(float)
except:
pass
try:
ret['best_bid'] = bids.iloc[-1]['price']
except:
ret['best_bid'] = 0
try:
ret['best_ask'] = asks.iloc[-1]['price']
except:
ret['best_ask'] = 0
ret['midpoint'] = (ret['best_bid'] + ret['best_ask']) / 2
TICK_SIZE = row['minimum_tick_size']
ret['tick_size'] = TICK_SIZE
bid_from, bid_to, ask_from, ask_to = get_bid_ask_range(ret, TICK_SIZE)
v = round((ret['max_spread'] / 100), 2)
bids_df = pd.DataFrame()
bids_df['price'] = generate_numbers(bid_from, bid_to, TICK_SIZE)
asks_df = pd.DataFrame()
asks_df['price'] = generate_numbers(ask_from, ask_to, TICK_SIZE)
try:
bids_df = bids_df.merge(bids, on='price', how='left').fillna(0)
except:
bids_df = pd.DataFrame()
try:
asks_df = asks_df.merge(asks, on='price', how='left').fillna(0)
except:
asks_df = pd.DataFrame()
best_bid_reward = 0
ret_bid = pd.DataFrame()
try:
ret_bid = add_formula_params(bids_df, ret['midpoint'], v, rate)
best_bid_reward = round(ret_bid['reward_per_100'].max(), 2)
except:
pass
best_ask_reward = 0
ret_ask = pd.DataFrame()
try:
ret_ask = add_formula_params(asks_df, ret['midpoint'], v, rate)
best_ask_reward = round(ret_ask['reward_per_100'].max(), 2)
except:
pass
ret['bid_reward_per_100'] = best_bid_reward
ret['ask_reward_per_100'] = best_ask_reward
ret['sm_reward_per_100'] = round((best_bid_reward + best_ask_reward) / 2, 2)
ret['gm_reward_per_100'] = round((best_bid_reward * best_ask_reward) ** 0.5, 2)
ret['end_date_iso'] = row['end_date_iso']
ret['market_slug'] = row['market_slug']
ret['token1'] = token1
ret['token2'] = token2
ret['condition_id'] = row['condition_id']
return ret
def get_all_results(all_df, client, max_workers=5):
all_results = []
def process_with_progress(args):
idx, row = args
try:
return process_single_row(row, client)
except:
print("error fetching market")
return None
with concurrent.futures.ThreadPoolExecutor(max_workers=max_workers) as executor:
futures = [executor.submit(process_with_progress, (idx, row)) for idx, row in all_df.iterrows()]
for future in concurrent.futures.as_completed(futures):
result = future.result()
if result is not None:
all_results.append(result)
if len(all_results) % (max_workers * 2) == 0:
print(f'{len(all_results)} of {len(all_df)}')
return all_results
def get_combined_markets(new_df, new_markets, sel_df):
if len(sel_df) > 0:
old_markets = new_df[new_df['question'].isin(sel_df['question'])]
all_markets = pd.concat([old_markets, new_markets])
else:
all_markets = new_markets
all_markets = all_markets.drop_duplicates('question')
all_markets = all_markets.sort_values('gm_reward_per_100', ascending=False)
return all_markets
import concurrent.futures
def calculate_annualized_volatility(df, hours):
end_time = df['t'].max()
start_time = end_time - pd.Timedelta(hours=hours)
window_df = df[df['t'] >= start_time]
volatility = window_df['log_return'].std()
annualized_volatility = volatility * np.sqrt(60 * 24 * 252)
return round(annualized_volatility, 2)
def add_volatility(row):
res = requests.get(f'https://clob.polymarket.com/prices-history?interval=1m&market={row["token1"]}&fidelity=10')
price_df = pd.DataFrame(res.json()['history'])
price_df['t'] = pd.to_datetime(price_df['t'], unit='s')
price_df['p'] = price_df['p'].round(2)
price_df.to_csv(f'data/{row["token1"]}.csv', index=False)
price_df['log_return'] = np.log(price_df['p'] / price_df['p'].shift(1))
row_dict = row.copy()
stats = {
'1_hour': calculate_annualized_volatility(price_df, 1),
'3_hour': calculate_annualized_volatility(price_df, 3),
'6_hour': calculate_annualized_volatility(price_df, 6),
'12_hour': calculate_annualized_volatility(price_df, 12),
'24_hour': calculate_annualized_volatility(price_df, 24),
'7_day': calculate_annualized_volatility(price_df, 24 * 7),
'14_day': calculate_annualized_volatility(price_df, 24 * 14),
'30_day': calculate_annualized_volatility(price_df, 24 * 30),
'volatility_price': price_df['p'].iloc[-1]
}
new_dict = {**row_dict, **stats}
return new_dict
def add_volatility_to_df(df, max_workers=2):
results = []
df = df.reset_index(drop=True)
def process_volatility_with_progress(args):
idx, row = args
try:
ret = add_volatility(row.to_dict())
return ret
except:
print("Error fetching volatility")
return None
with concurrent.futures.ThreadPoolExecutor(max_workers=max_workers) as executor:
futures = [executor.submit(process_volatility_with_progress, (idx, row)) for idx, row in df.iterrows()]
for future in concurrent.futures.as_completed(futures):
result = future.result()
if result is not None:
results.append(result)
if len(results) % (max_workers * 2) == 0:
print(f'{len(results)} of {len(df)}')
return pd.DataFrame(results)
def get_markets(all_results, sel_df, maker_reward=1):
new_df = pd.DataFrame(all_results)
new_df['spread'] = abs(new_df['best_ask'] - new_df['best_bid'])
new_df = new_df.sort_values('rewards_daily_rate', ascending=False)
new_df[' '] = ''
new_df = new_df[['question', 'answer1', 'answer2', 'neg_risk', 'spread', 'best_bid', 'best_ask', 'rewards_daily_rate', 'bid_reward_per_100', 'ask_reward_per_100', 'gm_reward_per_100', 'sm_reward_per_100', 'min_size', 'max_spread', 'tick_size', 'market_slug', 'token1', 'token2', 'condition_id']]
new_df = new_df.replace([np.inf, -np.inf], 0)
all_data = new_df.copy()
s_df = new_df.copy()
making_markets = s_df[~new_df['question'].isin(sel_df['question'])]
making_markets = making_markets.sort_values('gm_reward_per_100', ascending=False)
making_markets = making_markets[making_markets['gm_reward_per_100'] >= maker_reward]
all_markets = get_combined_markets(new_df, making_markets, sel_df)
return all_data, all_markets
data_updater/google_utils.py
-96
View File
@@ -1,96 +0,0 @@
from google.oauth2.service_account import Credentials
import gspread
import os
import pandas as pd
import requests
import re
def get_spreadsheet(read_only=False):
"""
Get the main Google Spreadsheet using credentials and URL from environment variables
Args:
read_only (bool): If True, uses public CSV export when credentials are missing
"""
spreadsheet_url = os.getenv("SPREADSHEET_URL")
if not spreadsheet_url:
raise ValueError("SPREADSHEET_URL environment variable is not set")
# Check for credentials
if not os.path.exists('credentials.json'):
if read_only:
return ReadOnlySpreadsheet(spreadsheet_url)
else:
raise FileNotFoundError("credentials.json not found. Use read_only=True for read-only access.")
# Normal authenticated access
scope = ["https://spreadsheets.google.com/feeds", "https://www.googleapis.com/auth/drive"]
credentials = Credentials.from_service_account_file('credentials.json', scopes=scope)
client = gspread.authorize(credentials)
spreadsheet = client.open_by_url(spreadsheet_url)
return spreadsheet
class ReadOnlySpreadsheet:
"""Read-only wrapper for Google Sheets using public CSV export"""
def __init__(self, spreadsheet_url):
self.spreadsheet_url = spreadsheet_url
self.sheet_id = self._extract_sheet_id(spreadsheet_url)
def _extract_sheet_id(self, url):
"""Extract sheet ID from Google Sheets URL"""
match = re.search(r'/spreadsheets/d/([a-zA-Z0-9-_]+)', url)
if not match:
raise ValueError("Invalid Google Sheets URL")
return match.group(1)
def worksheet(self, title):
"""Return a read-only worksheet"""
return ReadOnlyWorksheet(self.sheet_id, title)
class ReadOnlyWorksheet:
"""Read-only worksheet that fetches data via CSV export"""
def __init__(self, sheet_id, title):
self.sheet_id = sheet_id
self.title = title
def get_all_records(self):
"""Get all records from the worksheet as a list of dictionaries"""
try:
# Use the public CSV export URL
csv_url = f"https://docs.google.com/spreadsheets/d/{self.sheet_id}/gviz/tq?tqx=out:csv&sheet={self.title}"
response = requests.get(csv_url, timeout=30)
response.raise_for_status()
# Read CSV data into DataFrame
from io import StringIO
df = pd.read_csv(StringIO(response.text))
# Convert to list of dictionaries (same format as gspread)
return df.to_dict('records')
except Exception as e:
print(f"Warning: Could not fetch data from sheet '{self.title}': {e}")
return []
def get_all_values(self):
"""Get all values from the worksheet as a list of lists"""
try:
csv_url = f"https://docs.google.com/spreadsheets/d/{self.sheet_id}/gviz/tq?tqx=out:csv&sheet={self.title}"
response = requests.get(csv_url, timeout=30)
response.raise_for_status()
# Read CSV and return as list of lists
from io import StringIO
df = pd.read_csv(StringIO(response.text))
# Include headers and convert to list of lists
headers = [df.columns.tolist()]
data = df.values.tolist()
return headers + data
except Exception as e:
print(f"Warning: Could not fetch data from sheet '{self.title}': {e}")
return []
data_updater/trading_utils.py
-137
View File
@@ -1,137 +0,0 @@
from py_clob_client.constants import POLYGON
from py_clob_client.client import ClobClient
from py_clob_client.clob_types import OrderArgs, BalanceAllowanceParams, AssetType
from py_clob_client.order_builder.constants import BUY
from web3 import Web3
from web3.middleware import ExtraDataToPOAMiddleware
import json
from dotenv import load_dotenv
load_dotenv()
import time
import os
MAX_INT = 2**256 - 1
def get_clob_client():
host = "https://clob.polymarket.com"
key = os.getenv("PK")
chain_id = POLYGON
if key is None:
print("Environment variable 'PK' cannot be found")
return None
try:
client = ClobClient(host, key=key, chain_id=chain_id)
api_creds = client.create_or_derive_api_creds()
client.set_api_creds(api_creds)
return client
except Exception as ex:
print("Error creating clob client")
print("________________")
print(ex)
return None
def approveContracts():
web3 = Web3(Web3.HTTPProvider("https://polygon-rpc.com"))
web3.middleware_onion.inject(ExtraDataToPOAMiddleware, layer=0)
wallet = web3.eth.account.from_key(os.getenv("PK"))
with open('erc20ABI.json', 'r') as file:
erc20_abi = json.load(file)
ctf_address = "0x4D97DCd97eC945f40cF65F87097ACe5EA0476045"
erc1155_set_approval = """[{"inputs": [{ "internalType": "address", "name": "operator", "type": "address" },{ "internalType": "bool", "name": "approved", "type": "bool" }],"name": "setApprovalForAll","outputs": [],"stateMutability": "nonpayable","type": "function"}]"""
usdc_contract = web3.eth.contract(address="0x2791Bca1f2de4661ED88A30C99A7a9449Aa84174", abi=erc20_abi) # usdc.e
ctf_contract = web3.eth.contract(address=ctf_address, abi=erc1155_set_approval)
for address in ['0x4bFb41d5B3570DeFd03C39a9A4D8dE6Bd8B8982E', '0xC5d563A36AE78145C45a50134d48A1215220f80a', '0xd91E80cF2E7be2e162c6513ceD06f1dD0dA35296']:
usdc_nonce = web3.eth.get_transaction_count( wallet.address )
raw_usdc_txn = usdc_contract.functions.approve(address, int(MAX_INT, 0)).build_transaction({
"chainId": 137,
"from": wallet.address,
"nonce": usdc_nonce
})
signed_usdc_txn = web3.eth.account.sign_transaction(raw_usdc_txn, private_key=os.getenv("PK"))
usdc_tx_receipt = web3.eth.wait_for_transaction_receipt(signed_usdc_txn, 600)
print(f'USDC Transaction for {address} returned {usdc_tx_receipt}')
time.sleep(1)
ctf_nonce = web3.eth.get_transaction_count( wallet.address )
raw_ctf_approval_txn = ctf_contract.functions.setApprovalForAll(address, True).build_transaction({
"chainId": 137,
"from": wallet.address,
"nonce": ctf_nonce
})
signed_ctf_approval_tx = web3.eth.account.sign_transaction(raw_ctf_approval_txn, private_key=os.getenv("PK"))
send_ctf_approval_tx = web3.eth.send_raw_transaction(signed_ctf_approval_tx.raw_transaction)
ctf_approval_tx_receipt = web3.eth.wait_for_transaction_receipt(send_ctf_approval_tx, 600)
print(f'CTF Transaction for {address} returned {ctf_approval_tx_receipt}')
time.sleep(1)
nonce = web3.eth.get_transaction_count( wallet.address )
raw_txn_2 = usdc_contract.functions.approve("0xC5d563A36AE78145C45a50134d48A1215220f80a", int(MAX_INT, 0)).build_transaction({
"chainId": 137,
"from": wallet.address,
"nonce": nonce
})
signed_txn_2 = web3.eth.account.sign_transaction(raw_txn_2, private_key=os.getenv("PK"))
send_txn_2 = web3.eth.send_raw_transaction(signed_txn_2.raw_transaction)
nonce = web3.eth.get_transaction_count( wallet.address )
raw_txn_3 = usdc_contract.functions.approve("0xd91E80cF2E7be2e162c6513ceD06f1dD0dA35296", int(MAX_INT, 0)).build_transaction({
"chainId": 137,
"from": wallet.address,
"nonce": nonce
})
signed_txn_3 = web3.eth.account.sign_transaction(raw_txn_3, private_key=os.getenv("PK"))
send_txn_3 = web3.eth.send_raw_transaction(signed_txn_3.raw_transaction)
def market_action( marketId, action, price, size ):
order_args = OrderArgs(
price=price,
size=size,
side=action,
token_id=marketId,
)
signed_order = get_clob_client().create_order(order_args)
try:
resp = get_clob_client().post_order(signed_order)
print(resp)
except Exception as ex:
print(ex)
pass
def get_position(marketId):
client = get_clob_client()
position_res = client.get_balance_allowance(
BalanceAllowanceParams(
asset_type=AssetType.CONDITIONAL,
token_id=marketId
)
)
orderBook = client.get_order_book(marketId)
price = float(orderBook.bids[-1].price)
shares = int(position_res['balance']) / 1e6
return shares * price
main.py
-115
View File
@@ -1,115 +0,0 @@
import gc # Garbage collection
import time # Time functions
import asyncio # Asynchronous I/O
import traceback # Exception handling
import threading # Thread management
from poly_data.polymarket_client import PolymarketClient
from poly_data.data_utils import update_markets, update_positions, update_orders
from poly_data.websocket_handlers import connect_market_websocket, connect_user_websocket
import poly_data.global_state as global_state
from poly_data.data_processing import remove_from_performing
from dotenv import load_dotenv
load_dotenv()
def update_once():
"""
Initialize the application state by fetching market data, positions, and orders.
"""
update_markets() # Get market information from Google Sheets
update_positions() # Get current positions from Polymarket
update_orders() # Get current orders from Polymarket
def remove_from_pending():
"""
Clean up stale trades that have been pending for too long (>15 seconds).
This prevents the system from getting stuck on trades that may have failed.
"""
try:
current_time = time.time()
# Iterate through all performing trades
for col in list(global_state.performing.keys()):
for trade_id in list(global_state.performing[col]):
try:
# If trade has been pending for more than 15 seconds, remove it
if current_time - global_state.performing_timestamps[col].get(trade_id, current_time) > 15:
print(f"Removing stale entry {trade_id} from {col} after 15 seconds")
remove_from_performing(col, trade_id)
print("After removing: ", global_state.performing, global_state.performing_timestamps)
except:
print("Error in remove_from_pending")
print(traceback.format_exc())
except:
print("Error in remove_from_pending")
print(traceback.format_exc())
def update_periodically():
"""
Background thread function that periodically updates market data, positions and orders.
- Positions and orders are updated every 5 seconds
- Market data is updated every 30 seconds (every 6 cycles)
- Stale pending trades are removed each cycle
"""
i = 1
while True:
time.sleep(5) # Update every 5 seconds
try:
# Clean up stale trades
remove_from_pending()
# Update positions and orders every cycle
update_positions(avgOnly=True) # Only update average price, not position size
update_orders()
# Update market data every 6th cycle (30 seconds)
if i % 6 == 0:
update_markets()
i = 1
gc.collect() # Force garbage collection to free memory
i += 1
except:
print("Error in update_periodically")
print(traceback.format_exc())
async def main():
"""
Main application entry point. Initializes client, data, and manages websocket connections.
"""
# Initialize client
global_state.client = PolymarketClient()
# Initialize state and fetch initial data
global_state.all_tokens = []
update_once()
print("After initial updates: ", global_state.orders, global_state.positions)
print("\n")
print(f'There are {len(global_state.df)} market, {len(global_state.positions)} positions and {len(global_state.orders)} orders. Starting positions: {global_state.positions}')
# Start background update thread
update_thread = threading.Thread(target=update_periodically, daemon=True)
update_thread.start()
# Main loop - maintain websocket connections
while True:
try:
# Connect to market and user websockets simultaneously
await asyncio.gather(
connect_market_websocket(global_state.all_tokens),
connect_user_websocket()
)
print("Reconnecting to the websocket")
except:
print("Error in main loop")
print(traceback.format_exc())
await asyncio.sleep(1)
gc.collect() # Clean up memory
if __name__ == "__main__":
asyncio.run(main())
poly_data/CONSTANTS.py
-3
View File
@@ -1,3 +0,0 @@
# Minimum position size to trigger position merging
# Positions smaller than this will be ignored to save on gas costs
MIN_MERGE_SIZE = 20
poly_data/abis.py
-3
View File
File diff suppressed because one or more lines are too long
poly_data/data_processing.py
-160
View File
@@ -1,160 +0,0 @@
import json
from sortedcontainers import SortedDict
import poly_data.global_state as global_state
import poly_data.CONSTANTS as CONSTANTS
from trading import perform_trade
import time
import asyncio
from poly_data.data_utils import set_position, set_order, update_positions
def process_book_data(asset, json_data):
global_state.all_data[asset] = {
'asset_id': json_data['asset_id'], # token_id for the Yes token
'bids': SortedDict(),
'asks': SortedDict()
}
global_state.all_data[asset]['bids'].update({float(entry['price']): float(entry['size']) for entry in json_data['bids']})
global_state.all_data[asset]['asks'].update({float(entry['price']): float(entry['size']) for entry in json_data['asks']})
def process_price_change(asset, side, price_level, new_size, asset_id=None):
# Skip updates for the No token to prevent duplicated updates
# Only process if asset_id matches the stored asset_id for this market
if asset_id and asset in global_state.all_data and asset_id != global_state.all_data[asset]['asset_id']:
return
if side == 'bids':
book = global_state.all_data[asset]['bids']
else:
book = global_state.all_data[asset]['asks']
if new_size == 0:
if price_level in book:
del book[price_level]
else:
book[price_level] = new_size
def process_data(json_datas, trade=True):
# Ensure input is always a list
if isinstance(json_datas, dict):
json_datas = [json_datas]
for json_data in json_datas:
event_type = json_data['event_type']
asset = json_data['market']
if event_type == 'book':
process_book_data(asset, json_data)
if trade:
asyncio.create_task(perform_trade(asset))
elif event_type == 'price_change':
for data in json_data['price_changes']:
side = 'bids' if data['side'] == 'BUY' else 'asks'
price_level = float(data['price'])
new_size = float(data['size'])
# Pass asset_id if available in the data
asset_id = data.get('asset_id', None)
process_price_change(asset, side, price_level, new_size, asset_id)
if trade:
asyncio.create_task(perform_trade(asset))
# pretty_print(f'Received book update for {asset}:', global_state.all_data[asset])
def add_to_performing(col, id):
if col not in global_state.performing:
global_state.performing[col] = set()
if col not in global_state.performing_timestamps:
global_state.performing_timestamps[col] = {}
# Add the trade ID and track its timestamp
global_state.performing[col].add(id)
global_state.performing_timestamps[col][id] = time.time()
def remove_from_performing(col, id):
if col in global_state.performing:
global_state.performing[col].discard(id)
if col in global_state.performing_timestamps:
global_state.performing_timestamps[col].pop(id, None)
def process_user_data(rows):
for row in rows:
market = row['market']
side = row['side'].lower()
token = row['asset_id']
if token in global_state.REVERSE_TOKENS:
col = token + "_" + side
if row['event_type'] == 'trade':
size = 0
price = 0
maker_outcome = ""
taker_outcome = row['outcome']
is_user_maker = False
for maker_order in row['maker_orders']:
if maker_order['maker_address'].lower() == global_state.client.browser_wallet.lower():
print("User is maker")
size = float(maker_order['matched_amount'])
price = float(maker_order['price'])
is_user_maker = True
maker_outcome = maker_order['outcome'] #this is curious
if maker_outcome == taker_outcome:
side = 'buy' if side == 'sell' else 'sell' #need to reverse as we reverse token too
else:
token = global_state.REVERSE_TOKENS[token]
if not is_user_maker:
size = float(row['size'])
price = float(row['price'])
print("User is taker")
print("TRADE EVENT FOR: ", row['market'], "ID: ", row['id'], "STATUS: ", row['status'], " SIDE: ", row['side'], " MAKER OUTCOME: ", maker_outcome, " TAKER OUTCOME: ", taker_outcome, " PROCESSED SIDE: ", side, " SIZE: ", size)
if row['status'] == 'CONFIRMED' or row['status'] == 'FAILED' :
if row['status'] == 'FAILED':
print(f"Trade failed for {token}, decreasing")
asyncio.create_task(asyncio.sleep(2))
update_positions()
else:
remove_from_performing(col, row['id'])
print("Confirmed. Performing is ", len(global_state.performing[col]))
print("Last trade update is ", global_state.last_trade_update)
print("Performing is ", global_state.performing)
print("Performing timestamps is ", global_state.performing_timestamps)
asyncio.create_task(perform_trade(market))
elif row['status'] == 'MATCHED':
add_to_performing(col, row['id'])
print("Matched. Performing is ", len(global_state.performing[col]))
set_position(token, side, size, price)
print("Position after matching is ", global_state.positions[str(token)])
print("Last trade update is ", global_state.last_trade_update)
print("Performing is ", global_state.performing)
print("Performing timestamps is ", global_state.performing_timestamps)
asyncio.create_task(perform_trade(market))
elif row['status'] == 'MINED':
remove_from_performing(col, row['id'])
elif row['event_type'] == 'order':
print("ORDER EVENT FOR: ", row['market'], " STATUS: ", row['status'], " TYPE: ", row['type'], " SIDE: ", side, " ORIGINAL SIZE: ", row['original_size'], " SIZE MATCHED: ", row['size_matched'])
set_order(token, side, float(row['original_size']) - float(row['size_matched']), row['price'])
asyncio.create_task(perform_trade(market))
else:
print(f"User date received for {market} but its not in")
poly_data/data_utils.py
-176
View File
@@ -1,176 +0,0 @@
import poly_data.global_state as global_state
from poly_data.utils import get_sheet_df
import time
import poly_data.global_state as global_state
#sth here seems to be removing the position
def update_positions(avgOnly=False):
pos_df = global_state.client.get_all_positions()
for idx, row in pos_df.iterrows():
asset = str(row['asset'])
if asset in global_state.positions:
position = global_state.positions[asset].copy()
else:
position = {'size': 0, 'avgPrice': 0}
position['avgPrice'] = row['avgPrice']
if not avgOnly:
position['size'] = row['size']
else:
for col in [f"{asset}_sell", f"{asset}_buy"]:
#need to review this
if col not in global_state.performing or not isinstance(global_state.performing[col], set) or len(global_state.performing[col]) == 0:
try:
old_size = position['size']
except:
old_size = 0
if asset in global_state.last_trade_update:
if time.time() - global_state.last_trade_update[asset] < 5:
print(f"Skipping update for {asset} because last trade update was less than 5 seconds ago")
continue
if old_size != row['size']:
print(f"No trades are pending. Updating position from {old_size} to {row['size']} and avgPrice to {row['avgPrice']} using API")
position['size'] = row['size']
else:
print(f"ALERT: Skipping update for {asset} because there are trades pending for {col} looking like {global_state.performing[col]}")
global_state.positions[asset] = position
def get_position(token):
token = str(token)
if token in global_state.positions:
return global_state.positions[token]
else:
return {'size': 0, 'avgPrice': 0}
def set_position(token, side, size, price, source='websocket'):
token = str(token)
size = float(size)
price = float(price)
global_state.last_trade_update[token] = time.time()
if side.lower() == 'sell':
size *= -1
if token in global_state.positions:
prev_price = global_state.positions[token]['avgPrice']
prev_size = global_state.positions[token]['size']
if size > 0:
if prev_size == 0:
# Starting a new position
avgPrice_new = price
else:
# Buying more; update average price
avgPrice_new = (prev_price * prev_size + price * size) / (prev_size + size)
elif size < 0:
# Selling; average price remains the same
avgPrice_new = prev_price
else:
# No change in position
avgPrice_new = prev_price
global_state.positions[token]['size'] += size
global_state.positions[token]['avgPrice'] = avgPrice_new
else:
global_state.positions[token] = {'size': size, 'avgPrice': price}
print(f"Updated position from {source}, set to ", global_state.positions[token])
def update_orders():
all_orders = global_state.client.get_all_orders()
orders = {}
if len(all_orders) > 0:
for token in all_orders['asset_id'].unique():
if token not in orders:
orders[str(token)] = {'buy': {'price': 0, 'size': 0}, 'sell': {'price': 0, 'size': 0}}
curr_orders = all_orders[all_orders['asset_id'] == str(token)]
if len(curr_orders) > 0:
sel_orders = {}
sel_orders['buy'] = curr_orders[curr_orders['side'] == 'BUY']
sel_orders['sell'] = curr_orders[curr_orders['side'] == 'SELL']
for type in ['buy', 'sell']:
curr = sel_orders[type]
if len(curr) > 1:
print("Multiple orders found, cancelling")
global_state.client.cancel_all_asset(token)
orders[str(token)] = {'buy': {'price': 0, 'size': 0}, 'sell': {'price': 0, 'size': 0}}
elif len(curr) == 1:
orders[str(token)][type]['price'] = float(curr.iloc[0]['price'])
orders[str(token)][type]['size'] = float(curr.iloc[0]['original_size'] - curr.iloc[0]['size_matched'])
global_state.orders = orders
def get_order(token):
token = str(token)
if token in global_state.orders:
if 'buy' not in global_state.orders[token]:
global_state.orders[token]['buy'] = {'price': 0, 'size': 0}
if 'sell' not in global_state.orders[token]:
global_state.orders[token]['sell'] = {'price': 0, 'size': 0}
return global_state.orders[token]
else:
return {'buy': {'price': 0, 'size': 0}, 'sell': {'price': 0, 'size': 0}}
def set_order(token, side, size, price):
curr = {}
curr = {side: {'price': 0, 'size': 0}}
curr[side]['size'] = float(size)
curr[side]['price'] = float(price)
global_state.orders[str(token)] = curr
print("Updated order, set to ", curr)
def update_markets():
received_df, received_params = get_sheet_df()
if len(received_df) > 0:
# Ensure multiplier column exists and fill NaN values with empty string
if 'multiplier' not in received_df.columns:
received_df['multiplier'] = ''
else:
received_df['multiplier'] = received_df['multiplier'].fillna('')
global_state.df, global_state.params = received_df.copy(), received_params
for _, row in global_state.df.iterrows():
for col in ['token1', 'token2']:
row[col] = str(row[col])
if row['token1'] not in global_state.all_tokens:
global_state.all_tokens.append(row['token1'])
if row['token1'] not in global_state.REVERSE_TOKENS:
global_state.REVERSE_TOKENS[row['token1']] = row['token2']
if row['token2'] not in global_state.REVERSE_TOKENS:
global_state.REVERSE_TOKENS[row['token2']] = row['token1']
for col2 in [f"{row['token1']}_buy", f"{row['token1']}_sell", f"{row['token2']}_buy", f"{row['token2']}_sell"]:
if col2 not in global_state.performing:
global_state.performing[col2] = set()
poly_data/global_state.py
-49
View File
@@ -1,49 +0,0 @@
import threading
import pandas as pd
# ============ Market Data ============
# List of all tokens being tracked
all_tokens = []
# Mapping between tokens in the same market (YES->NO, NO->YES)
REVERSE_TOKENS = {}
# Order book data for all markets
all_data = {}
# Market configuration data from Google Sheets
df = None
# ============ Client & Parameters ============
# Polymarket client instance
client = None
# Trading parameters from Google Sheets
params = {}
# Lock for thread-safe trading operations
lock = threading.Lock()
# ============ Trading State ============
# Tracks trades that have been matched but not yet mined
# Format: {"token_side": {trade_id1, trade_id2, ...}}
performing = {}
# Timestamps for when trades were added to performing
# Used to clear stale trades
performing_timestamps = {}
# Timestamps for when positions were last updated
last_trade_update = {}
# Current open orders for each token
# Format: {token_id: {'buy': {price, size}, 'sell': {price, size}}}
orders = {}
# Current positions for each token
# Format: {token_id: {'size': float, 'avgPrice': float}}
positions = {}
poly_data/polymarket_client.py
-320
View File
@@ -1,320 +0,0 @@
from dotenv import load_dotenv # Environment variable management
import os # Operating system interface
# Polymarket API client libraries
from py_clob_client.client import ClobClient
from py_clob_client.clob_types import OrderArgs, BalanceAllowanceParams, AssetType, PartialCreateOrderOptions
from py_clob_client.constants import POLYGON
# Web3 libraries for blockchain interaction
from web3 import Web3
from web3.middleware import ExtraDataToPOAMiddleware
from eth_account import Account
import requests # HTTP requests
import pandas as pd # Data analysis
import json # JSON processing
import subprocess # For calling external processes
from py_clob_client.clob_types import OpenOrderParams
# Smart contract ABIs
from poly_data.abis import NegRiskAdapterABI, ConditionalTokenABI, erc20_abi
# Load environment variables
load_dotenv()
class PolymarketClient:
"""
Client for interacting with Polymarket's API and smart contracts.
This class provides methods for:
- Creating and managing orders
- Querying order book data
- Checking balances and positions
- Merging positions
The client connects to both the Polymarket API and the Polygon blockchain.
"""
def __init__(self, pk='default') -> None:
"""
Initialize the Polymarket client with API and blockchain connections.
Args:
pk (str, optional): Private key identifier, defaults to 'default'
"""
host="https://clob.polymarket.com"
# Get credentials from environment variables
key=os.getenv("PK")
browser_address = os.getenv("BROWSER_ADDRESS")
# Don't print sensitive wallet information
print("Initializing Polymarket client...")
chain_id=POLYGON
self.browser_wallet=Web3.to_checksum_address(browser_address)
# Initialize the Polymarket API client
self.client = ClobClient(
host=host,
key=key,
chain_id=chain_id,
funder=self.browser_wallet,
signature_type=2
)
# Set up API credentials
self.creds = self.client.create_or_derive_api_creds()
self.client.set_api_creds(creds=self.creds)
# Initialize Web3 connection to Polygon
web3 = Web3(Web3.HTTPProvider("https://polygon-rpc.com"))
web3.middleware_onion.inject(ExtraDataToPOAMiddleware, layer=0)
# Set up USDC contract for balance checks
self.usdc_contract = web3.eth.contract(
address="0x2791Bca1f2de4661ED88A30C99A7a9449Aa84174",
abi=erc20_abi
)
# Store key contract addresses
self.addresses = {
'neg_risk_adapter': '0xd91E80cF2E7be2e162c6513ceD06f1dD0dA35296',
'collateral': '0x2791Bca1f2de4661ED88A30C99A7a9449Aa84174',
'conditional_tokens': '0x4D97DCd97eC945f40cF65F87097ACe5EA0476045'
}
# Initialize contract interfaces
self.neg_risk_adapter = web3.eth.contract(
address=self.addresses['neg_risk_adapter'],
abi=NegRiskAdapterABI
)
self.conditional_tokens = web3.eth.contract(
address=self.addresses['conditional_tokens'],
abi=ConditionalTokenABI
)
self.web3 = web3
def create_order(self, marketId, action, price, size, neg_risk=False):
"""
Create and submit a new order to the Polymarket order book.
Args:
marketId (str): ID of the market token to trade
action (str): "BUY" or "SELL"
price (float): Order price (0-1 range for prediction markets)
size (float): Order size in USDC
neg_risk (bool, optional): Whether this is a negative risk market. Defaults to False.
Returns:
dict: Response from the API containing order details, or empty dict on error
"""
# Create order parameters
order_args = OrderArgs(
token_id=str(marketId),
price=price,
size=size,
side=action
)
signed_order = None
# Handle regular vs negative risk markets differently
if neg_risk == False:
signed_order = self.client.create_order(order_args)
else:
signed_order = self.client.create_order(order_args, options=PartialCreateOrderOptions(neg_risk=True))
try:
# Submit the signed order to the API
resp = self.client.post_order(signed_order)
return resp
except Exception as ex:
print(ex)
return {}
def get_order_book(self, market):
"""
Get the current order book for a specific market.
Args:
market (str): Market ID to query
Returns:
tuple: (bids_df, asks_df) - DataFrames containing bid and ask orders
"""
orderBook = self.client.get_order_book(market)
return pd.DataFrame(orderBook.bids).astype(float), pd.DataFrame(orderBook.asks).astype(float)
def get_usdc_balance(self):
"""
Get the USDC balance of the connected wallet.
Returns:
float: USDC balance in decimal format
"""
return self.usdc_contract.functions.balanceOf(self.browser_wallet).call() / 10**6
def get_pos_balance(self):
"""
Get the total value of all positions for the connected wallet.
Returns:
float: Total position value in USDC
"""
res = requests.get(f'https://data-api.polymarket.com/value?user={self.browser_wallet}')
return float(res.json()['value'])
def get_total_balance(self):
"""
Get the combined value of USDC balance and all positions.
Returns:
float: Total account value in USDC
"""
return self.get_usdc_balance() + self.get_pos_balance()
def get_all_positions(self):
"""
Get all positions for the connected wallet across all markets.
Returns:
DataFrame: All positions with details like market, size, avgPrice
"""
res = requests.get(f'https://data-api.polymarket.com/positions?user={self.browser_wallet}')
return pd.DataFrame(res.json())
def get_raw_position(self, tokenId):
"""
Get the raw token balance for a specific market outcome token.
Args:
tokenId (int): Token ID to query
Returns:
int: Raw token amount (before decimal conversion)
"""
return int(self.conditional_tokens.functions.balanceOf(self.browser_wallet, int(tokenId)).call())
def get_position(self, tokenId):
"""
Get both raw and formatted position size for a token.
Args:
tokenId (int): Token ID to query
Returns:
tuple: (raw_position, shares) - Raw token amount and decimal shares
Shares less than 1 are treated as 0 to avoid dust amounts
"""
raw_position = self.get_raw_position(tokenId)
shares = float(raw_position / 1e6)
# Ignore very small positions (dust)
if shares < 1:
shares = 0
return raw_position, shares
def get_all_orders(self):
"""
Get all open orders for the connected wallet.
Returns:
DataFrame: All open orders with their details
"""
orders_df = pd.DataFrame(self.client.get_orders())
# Convert numeric columns to float
for col in ['original_size', 'size_matched', 'price']:
if col in orders_df.columns:
orders_df[col] = orders_df[col].astype(float)
return orders_df
def get_market_orders(self, market):
"""
Get all open orders for a specific market.
Args:
market (str): Market ID to query
Returns:
DataFrame: Open orders for the specified market
"""
orders_df = pd.DataFrame(self.client.get_orders(OpenOrderParams(
market=market,
)))
# Convert numeric columns to float
for col in ['original_size', 'size_matched', 'price']:
if col in orders_df.columns:
orders_df[col] = orders_df[col].astype(float)
return orders_df
def cancel_all_asset(self, asset_id):
"""
Cancel all orders for a specific asset token.
Args:
asset_id (str): Asset token ID
"""
self.client.cancel_market_orders(asset_id=str(asset_id))
def cancel_all_market(self, marketId):
"""
Cancel all orders in a specific market.
Args:
marketId (str): Market ID
"""
self.client.cancel_market_orders(market=marketId)
def merge_positions(self, amount_to_merge, condition_id, is_neg_risk_market):
"""
Merge positions in a market to recover collateral.
This function calls the external poly_merger Node.js script to execute
the merge operation on-chain. When you hold both YES and NO positions
in the same market, merging them recovers your USDC.
Args:
amount_to_merge (int): Raw token amount to merge (before decimal conversion)
condition_id (str): Market condition ID
is_neg_risk_market (bool): Whether this is a negative risk market
Returns:
str: Transaction hash or output from the merge script
Raises:
Exception: If the merge operation fails
"""
amount_to_merge_str = str(amount_to_merge)
# Prepare the command to run the JavaScript script
node_command = f'node poly_merger/merge.js {amount_to_merge_str} {condition_id} {"true" if is_neg_risk_market else "false"}'
print(node_command)
# Run the command and capture the output
result = subprocess.run(node_command, shell=True, capture_output=True, text=True)
# Check if there was an error
if result.returncode != 0:
print("Error:", result.stderr)
raise Exception(f"Error in merging positions: {result.stderr}")
print("Done merging")
# Return the transaction hash or output
return result.stdout
poly_data/trading_utils.py
-197
View File
@@ -1,197 +0,0 @@
import math
from poly_data.data_utils import update_positions
import poly_data.global_state as global_state
# def get_avgPrice(position, assetId):
# curr_global = global_state.all_positions[global_state.all_positions['asset'] == str(assetId)]
# api_position_size = 0
# api_avgPrice = 0
# if len(curr_global) > 0:
# c_row = curr_global.iloc[0]
# api_avgPrice = round(c_row['avgPrice'], 2)
# api_position_size = c_row['size']
# if position > 0:
# if abs((api_position_size - position)/position * 100) > 5:
# print("Updating global positions")
# update_positions()
# try:
# c_row = curr_global.iloc[0]
# api_avgPrice = round(c_row['avgPrice'], 2)
# api_position_size = c_row['size']
# except:
# return 0
# return api_avgPrice
def get_best_bid_ask_deets(market, name, size, deviation_threshold=0.05):
best_bid, best_bid_size, second_best_bid, second_best_bid_size, top_bid = find_best_price_with_size(global_state.all_data[market]['bids'], size, reverse=True)
best_ask, best_ask_size, second_best_ask, second_best_ask_size, top_ask = find_best_price_with_size(global_state.all_data[market]['asks'], size, reverse=False)
# Handle None values in mid_price calculation
if best_bid is not None and best_ask is not None:
mid_price = (best_bid + best_ask) / 2
bid_sum_within_n_percent = sum(size for price, size in global_state.all_data[market]['bids'].items() if best_bid <= price <= mid_price * (1 + deviation_threshold))
ask_sum_within_n_percent = sum(size for price, size in global_state.all_data[market]['asks'].items() if mid_price * (1 - deviation_threshold) <= price <= best_ask)
else:
mid_price = None
bid_sum_within_n_percent = 0
ask_sum_within_n_percent = 0
if name == 'token2':
# Handle None values before arithmetic operations
if all(x is not None for x in [best_bid, best_ask, second_best_bid, second_best_ask, top_bid, top_ask]):
best_bid, second_best_bid, top_bid, best_ask, second_best_ask, top_ask = 1 - best_ask, 1 - second_best_ask, 1 - top_ask, 1 - best_bid, 1 - second_best_bid, 1 - top_bid
best_bid_size, second_best_bid_size, best_ask_size, second_best_ask_size = best_ask_size, second_best_ask_size, best_bid_size, second_best_bid_size
bid_sum_within_n_percent, ask_sum_within_n_percent = ask_sum_within_n_percent, bid_sum_within_n_percent
else:
# Handle case where some prices are None - use available values or defaults
if best_bid is not None and best_ask is not None:
best_bid, best_ask = 1 - best_ask, 1 - best_bid
best_bid_size, best_ask_size = best_ask_size, best_bid_size
if second_best_bid is not None:
second_best_bid = 1 - second_best_bid
if second_best_ask is not None:
second_best_ask = 1 - second_best_ask
if top_bid is not None:
top_bid = 1 - top_bid
if top_ask is not None:
top_ask = 1 - top_ask
bid_sum_within_n_percent, ask_sum_within_n_percent = ask_sum_within_n_percent, bid_sum_within_n_percent
#return as dictionary
return {
'best_bid': best_bid,
'best_bid_size': best_bid_size,
'second_best_bid': second_best_bid,
'second_best_bid_size': second_best_bid_size,
'top_bid': top_bid,
'best_ask': best_ask,
'best_ask_size': best_ask_size,
'second_best_ask': second_best_ask,
'second_best_ask_size': second_best_ask_size,
'top_ask': top_ask,
'bid_sum_within_n_percent': bid_sum_within_n_percent,
'ask_sum_within_n_percent': ask_sum_within_n_percent
}
def find_best_price_with_size(price_dict, min_size, reverse=False):
lst = list(price_dict.items())
if reverse:
lst.reverse()
best_price, best_size = None, None
second_best_price, second_best_size = None, None
top_price = None
set_best = False
for price, size in lst:
if top_price is None:
top_price = price
if set_best:
second_best_price, second_best_size = price, size
break
if size > min_size:
if best_price is None:
best_price, best_size = price, size
set_best = True
return best_price, best_size, second_best_price, second_best_size, top_price
def get_order_prices(best_bid, best_bid_size, top_bid, best_ask, best_ask_size, top_ask, avgPrice, row):
bid_price = best_bid + row['tick_size']
ask_price = best_ask - row['tick_size']
if best_bid_size < row['min_size'] * 1.5:
bid_price = best_bid
if best_ask_size < 250 * 1.5:
ask_price = best_ask
if bid_price >= top_ask:
bid_price = top_bid
if ask_price <= top_bid:
ask_price = top_ask
if bid_price == ask_price:
bid_price = top_bid
ask_price = top_ask
# if ask_price <= avgPrice:
# if avgPrice - ask_price <= (row['max_spread']*1.7/100):
# ask_price = avgPrice
#temp for sleep
if ask_price <= avgPrice and avgPrice > 0:
ask_price = avgPrice
return bid_price, ask_price
def round_down(number, decimals):
factor = 10 ** decimals
return math.floor(number * factor) / factor
def round_up(number, decimals):
factor = 10 ** decimals
return math.ceil(number * factor) / factor
def get_buy_sell_amount(position, bid_price, row, other_token_position=0):
buy_amount = 0
sell_amount = 0
# Get max_size, defaulting to trade_size if not specified
max_size = row.get('max_size', row['trade_size'])
trade_size = row['trade_size']
# Calculate total exposure across both sides
total_exposure = position + other_token_position
# If we haven't reached max_size on either side, continue building
if position < max_size:
# Continue quoting trade_size amounts until we reach max_size
remaining_to_max = max_size - position
buy_amount = min(trade_size, remaining_to_max)
# Only sell if we have substantial position (to allow for exit when needed)
if position >= trade_size:
sell_amount = min(position, trade_size)
else:
sell_amount = 0
else:
# We've reached max_size, implement progressive exit strategy
# Always offer to sell trade_size amount when at max_size
sell_amount = min(position, trade_size)
# Continue quoting to buy if total exposure warrants it
if total_exposure < max_size * 2: # Allow some flexibility for market making
buy_amount = trade_size
else:
buy_amount = 0
# Ensure minimum order size compliance
if buy_amount > 0.7 * row['min_size'] and buy_amount < row['min_size']:
buy_amount = row['min_size']
# Apply multiplier for low-priced assets
if bid_price < 0.1 and buy_amount > 0:
multiplier = row.get('multiplier', '')
if multiplier != '':
print(f"Multiplying buy amount by {int(multiplier)}")
buy_amount = buy_amount * int(multiplier)
return buy_amount, sell_amount
poly_data/utils.py
-68
View File
@@ -1,68 +0,0 @@
import json
from poly_utils.google_utils import get_spreadsheet
import pandas as pd
import os
def pretty_print(txt, dic):
print("\n", txt, json.dumps(dic, indent=4))
def get_sheet_df(read_only=None):
"""
Get sheet data with optional read-only mode
Args:
read_only (bool): If None, auto-detects based on credentials availability
"""
all = 'All Markets'
sel = 'Selected Markets'
# Auto-detect read-only mode if not specified
if read_only is None:
creds_file = 'credentials.json' if os.path.exists('credentials.json') else '../credentials.json'
read_only = not os.path.exists(creds_file)
if read_only:
print("No credentials found, using read-only mode")
try:
spreadsheet = get_spreadsheet(read_only=read_only)
except FileNotFoundError:
print("No credentials found, falling back to read-only mode")
spreadsheet = get_spreadsheet(read_only=True)
wk = spreadsheet.worksheet(sel)
df = pd.DataFrame(wk.get_all_records())
df = df[df['question'] != ""].reset_index(drop=True)
wk2 = spreadsheet.worksheet(all)
df2 = pd.DataFrame(wk2.get_all_records())
df2 = df2[df2['question'] != ""].reset_index(drop=True)
result = df.merge(df2, on='question', how='inner')
wk_p = spreadsheet.worksheet('Hyperparameters')
records = wk_p.get_all_records()
hyperparams, current_type = {}, None
for r in records:
# Update current_type only when we have a non-empty type value
# Handle both string and NaN values from pandas
type_value = r['type']
if type_value and str(type_value).strip() and str(type_value) != 'nan':
current_type = str(type_value).strip()
# Skip rows where we don't have a current_type set
if current_type:
# Convert numeric values to appropriate types
value = r['value']
try:
# Try to convert to float if it's numeric
if isinstance(value, str) and value.replace('.', '').replace('-', '').isdigit():
value = float(value)
elif isinstance(value, (int, float)):
value = float(value)
except (ValueError, TypeError):
pass # Keep as string if conversion fails
hyperparams.setdefault(current_type, {})[r['param']] = value
return result, hyperparams
poly_data/websocket_handlers.py
-98
View File
@@ -1,98 +0,0 @@
import asyncio # Asynchronous I/O
import json # JSON handling
import websockets # WebSocket client
import traceback # Exception handling
from poly_data.data_processing import process_data, process_user_data
import poly_data.global_state as global_state
async def connect_market_websocket(chunk):
"""
Connect to Polymarket's market WebSocket API and process market updates.
This function:
1. Establishes a WebSocket connection to the Polymarket API
2. Subscribes to updates for a specified list of market tokens
3. Processes incoming order book and price updates
Args:
chunk (list): List of token IDs to subscribe to
Notes:
If the connection is lost, the function will exit and the main loop will
attempt to reconnect after a short delay.
"""
uri = "wss://ws-subscriptions-clob.polymarket.com/ws/market"
async with websockets.connect(uri, ping_interval=5, ping_timeout=None) as websocket:
# Prepare and send subscription message
message = {"assets_ids": chunk}
await websocket.send(json.dumps(message))
print("\n")
print(f"Sent market subscription message: {message}")
try:
# Process incoming market data indefinitely
while True:
message = await websocket.recv()
json_data = json.loads(message)
# Process order book updates and trigger trading as needed
process_data(json_data)
except websockets.ConnectionClosed:
print("Connection closed in market websocket")
print(traceback.format_exc())
except Exception as e:
print(f"Exception in market websocket: {e}")
print(traceback.format_exc())
finally:
# Brief delay before attempting to reconnect
await asyncio.sleep(5)
async def connect_user_websocket():
"""
Connect to Polymarket's user WebSocket API and process order/trade updates.
This function:
1. Establishes a WebSocket connection to the Polymarket user API
2. Authenticates using API credentials
3. Processes incoming order and trade updates for the user
Notes:
If the connection is lost, the function will exit and the main loop will
attempt to reconnect after a short delay.
"""
uri = "wss://ws-subscriptions-clob.polymarket.com/ws/user"
async with websockets.connect(uri, ping_interval=5, ping_timeout=None) as websocket:
# Prepare authentication message with API credentials
message = {
"type": "user",
"auth": {
"apiKey": global_state.client.client.creds.api_key,
"secret": global_state.client.client.creds.api_secret,
"passphrase": global_state.client.client.creds.api_passphrase
}
}
# Send authentication message
await websocket.send(json.dumps(message))
print("\n")
print(f"Sent user subscription message")
try:
# Process incoming user data indefinitely
while True:
message = await websocket.recv()
json_data = json.loads(message)
# Process trade and order updates
process_user_data(json_data)
except websockets.ConnectionClosed:
print("Connection closed in user websocket")
print(traceback.format_exc())
except Exception as e:
print(f"Exception in user websocket: {e}")
print(traceback.format_exc())
finally:
# Brief delay before attempting to reconnect
await asyncio.sleep(5)
poly_merger/README.md
-36
View File
@@ -1,36 +0,0 @@
# Poly-Merger
A utility for merging Polymarket positions efficiently. This tool helps in consolidating opposite positions in the same market, allowing you to:
1. Reduce gas costs
2. Free up capital
3. Simplify position management
## How It Works
The merger tool interacts with Polymarket's smart contracts to combine opposite positions in binary markets. When you hold both YES and NO shares in the same market, this tool will merge them to recover your USDC.
## Usage
The merger is invoked through the main Poly-Maker bot when position merging conditions are met, but you can also use it independently:
```
node merge.js [amount_to_merge] [condition_id] [is_neg_risk_market]
```
Example:
```
node merge.js 1000000 0xasdasda true
```
This would merge 1 USDC worth of opposing positions in market 0xasdasda, which is a negative risk market. 0xasdasda should be condition_id
## Prerequisites
- Node.js
- ethers.js v5.x
- A .env file with your Polygon network private key
## Notes
This implementation is based on open-source Polymarket code but has been optimized for automated market making operations.
poly_merger/merge.js
-140
View File
@@ -1,140 +0,0 @@
/**
* Poly-Merger: Position Merging Utility for Polymarket
*
* This script handles merging of YES and NO positions in Polymarket prediction markets
* to recover collateral. It works with both regular and negative risk markets.
*
* The merger supports Gnosis Safe wallets through the safe-helpers.js utility.
*
* Usage:
* node merge.js [amountToMerge] [conditionId] [isNegRiskMarket]
*
* Example:
* node merge.js 1000000 12345 true
*/
const { ethers } = require('ethers');
const { resolve } = require('path');
const { existsSync } = require('fs');
const { signAndExecuteSafeTransaction } = require('./safe-helpers');
const { safeAbi } = require('./safeAbi');
// Load environment variables
const localEnvPath = resolve(__dirname, '.env');
const parentEnvPath = resolve(__dirname, '../.env');
const envPath = existsSync(localEnvPath) ? localEnvPath : parentEnvPath;
require('dotenv').config({ path: envPath })
// Connect to Polygon network
const provider = new ethers.providers.JsonRpcProvider("https://polygon-rpc.com");
const privateKey = process.env.PK;
const wallet = new ethers.Wallet(privateKey, provider);
// Polymarket contract addresses
const addresses = {
// Adapter contract for negative risk markets
neg_risk_adapter: '0xd91E80cF2E7be2e162c6513ceD06f1dD0dA35296',
// USDC token contract on Polygon
collateral: '0x2791Bca1f2de4661ED88A30C99A7a9449Aa84174',
// Main conditional tokens contract for prediction markets
conditional_tokens: '0x4D97DCd97eC945f40cF65F87097ACe5EA0476045'
};
// Minimal ABIs for the contracts we interact with
const negRiskAdapterAbi = [
"function mergePositions(bytes32 conditionId, uint256 amount)"
];
const conditionalTokensAbi = [
"function mergePositions(address collateralToken, bytes32 parentCollectionId, bytes32 conditionId, uint256[] partition, uint256 amount)"
];
/**
* Merges YES and NO positions in a Polymarket prediction market to recover USDC collateral.
*
* This function handles both regular and negative risk markets via different contract calls.
* It uses the Gnosis Safe wallet infrastructure for secure transaction execution.
*
* @param {string|number} amountToMerge - Raw amount of tokens to merge (typically expressed in raw units, e.g., 1000000 = 1 USDC)
* @param {string|number} conditionId - The market's condition ID
* @param {boolean} isNegRiskMarket - Whether this is a negative risk market (uses different contract)
* @returns {string} The transaction hash of the merge operation
*/
async function mergePositions(amountToMerge, conditionId, isNegRiskMarket) {
// Log parameters for debugging
console.log(amountToMerge, conditionId, isNegRiskMarket);
// Prepare transaction parameters
const nonce = await provider.getTransactionCount(wallet.address);
const gasPrice = await provider.getGasPrice();
const gasLimit = 10000000; // Set high gas limit to ensure transaction completes
let tx;
// Different contract calls for different market types
if (isNegRiskMarket) {
// For negative risk markets, use the adapter contract
const negRiskAdapter = new ethers.Contract(addresses.neg_risk_adapter, negRiskAdapterAbi, wallet);
tx = await negRiskAdapter.populateTransaction.mergePositions(conditionId, amountToMerge);
} else {
// For regular markets, use the conditional tokens contract directly
const conditionalTokens = new ethers.Contract(addresses.conditional_tokens, conditionalTokensAbi, wallet);
tx = await conditionalTokens.populateTransaction.mergePositions(
addresses.collateral, // USDC contract
ethers.constants.HashZero, // Parent collection ID (0 for top-level markets)
conditionId, // Market ID
[1, 2], // Partition (indexes of outcomes to merge)
amountToMerge // Amount to merge
);
}
// Prepare full transaction object
const transaction = {
...tx,
chainId: 137, // Polygon chain ID
gasPrice: gasPrice,
gasLimit: gasLimit,
nonce: nonce
};
// Get the Safe address from environment variables
const safeAddress = process.env.BROWSER_ADDRESS;
const safe = new ethers.Contract(safeAddress, safeAbi, wallet);
// Execute the transaction through the Safe
console.log("Signing Transaction")
const txResponse = await signAndExecuteSafeTransaction(
wallet,
safe,
transaction.to,
transaction.data,
{
gasPrice: transaction.gasPrice,
gasLimit: transaction.gasLimit
}
);
console.log("Sent transaction. Waiting for response")
const txReceipt = await txResponse.wait();
console.log("merge positions " + txReceipt.transactionHash);
return txReceipt.transactionHash;
}
// Parse command line arguments
const args = process.argv.slice(2);
// Amount of tokens to merge (in raw units, e.g., 1000000 = 1 USDC)
const amountToMerge = args[0];
// The market's condition ID
const conditionId = args[1];
// Whether this is a negative risk market (true/false)
const isNegRiskMarket = args[2] === 'true';
// Execute the merge operation and handle any errors
mergePositions(amountToMerge, conditionId, isNegRiskMarket)
.catch(error => {
console.error("Error merging positions:", error);
process.exit(1);
});
poly_merger/package-lock.json
Generated
-1339
View File
File diff suppressed because it is too large Load Diff
poly_merger/package.json
-15
View File
@@ -1,15 +0,0 @@
{
"name": "poly-merger",
"version": "1.0.0",
"description": "Position merging utility for Polymarket",
"main": "merge.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "",
"license": "MIT",
"dependencies": {
"dotenv": "^16.4.5",
"ethers": "^5.6.3"
}
}
poly_merger/safe-helpers.js
-128
View File
@@ -1,128 +0,0 @@
const { BigNumber, ethers } = require('ethers');
function joinHexData(hexData) {
return `0x${hexData
.map(hex => {
const stripped = hex.replace(/^0x/, "");
return stripped.length % 2 === 0 ? stripped : "0" + stripped;
})
.join("")}`;
}
function abiEncodePacked(...params) {
return joinHexData(
params.map(({ type, value }) => {
const encoded = ethers.utils.defaultAbiCoder.encode([type], [value]);
if (type === "bytes" || type === "string") {
const bytesLength = parseInt(encoded.slice(66, 130), 16);
return encoded.slice(130, 130 + 2 * bytesLength);
}
let typeMatch = type.match(/^(?:u?int\d*|bytes\d+|address)\[\]$/);
if (typeMatch) {
return encoded.slice(130);
}
if (type.startsWith("bytes")) {
const bytesLength = parseInt(type.slice(5));
return encoded.slice(2, 2 + 2 * bytesLength);
}
typeMatch = type.match(/^u?int(\d*)$/);
if (typeMatch) {
if (typeMatch[1] !== "") {
const bytesLength = parseInt(typeMatch[1]) / 8;
return encoded.slice(-2 * bytesLength);
}
return encoded.slice(-64);
}
if (type === "address") {
return encoded.slice(-40);
}
throw new Error(`unsupported type ${type}`);
})
);
}
async function signTransactionHash(signer, message) {
const messageArray = ethers.utils.arrayify(message);
let sig = await signer.signMessage(messageArray);
let sigV = parseInt(sig.slice(-2), 16);
switch (sigV) {
case 0:
case 1:
sigV += 31;
break;
case 27:
case 28:
sigV += 4;
break;
default:
throw new Error("Invalid signature");
}
sig = sig.slice(0, -2) + sigV.toString(16);
return {
r: BigNumber.from("0x" + sig.slice(2, 66)).toString(),
s: BigNumber.from("0x" + sig.slice(66, 130)).toString(),
v: BigNumber.from("0x" + sig.slice(130, 132)).toString(),
};
}
async function signAndExecuteSafeTransaction(signer, safe, to, data, overrides = {}) {
const nonce = await safe.nonce();
console.log("Nonce for safe: ", nonce);
const value = "0";
const safeTxGas = "0";
const baseGas = "0";
const gasPrice = "0";
const gasToken = ethers.constants.AddressZero;
const refundReceiver = ethers.constants.AddressZero;
const operation = 0;
const txHash = await safe.getTransactionHash(
to,
value,
data,
operation,
safeTxGas,
baseGas,
gasPrice,
gasToken,
refundReceiver,
nonce
);
console.log("Transaction hash: ", txHash);
const rsvSignature = await signTransactionHash(signer, txHash);
const packedSig = abiEncodePacked(
{ type: "uint256", value: rsvSignature.r },
{ type: "uint256", value: rsvSignature.s },
{ type: "uint8", value: rsvSignature.v }
);
console.log("Executing transaction");
return safe.execTransaction(
to,
value,
data,
operation,
safeTxGas,
baseGas,
gasPrice,
gasToken,
refundReceiver,
packedSig,
overrides
);
}
module.exports = {
signAndExecuteSafeTransaction,
};
poly_merger/safeAbi.js
-1140
View File
File diff suppressed because it is too large Load Diff
poly_stats/account_stats.py
-137
View File
@@ -1,137 +0,0 @@
import pandas as pd
from py_clob_client.headers.headers import create_level_2_headers
from py_clob_client.clob_types import RequestArgs
from poly_utils.google_utils import get_spreadsheet
from gspread_dataframe import set_with_dataframe
import requests
import json
import os
from dotenv import load_dotenv
load_dotenv()
spreadsheet = get_spreadsheet()
def get_markets_df(wk_full):
markets_df = pd.DataFrame(wk_full.get_all_records())
markets_df = markets_df[['question', 'answer1', 'answer2', 'token1', 'token2']]
markets_df['token1'] = markets_df['token1'].astype(str)
markets_df['token2'] = markets_df['token2'].astype(str)
return markets_df
def get_all_orders(client):
orders = client.client.get_orders()
orders_df = pd.DataFrame(orders)
if len(orders_df) > 0:
orders_df['order_size'] = orders_df['original_size'].astype('float') - orders_df['size_matched'].astype('float')
orders_df = orders_df[['asset_id', 'order_size', 'side', 'price']]
orders_df = orders_df.rename(columns={'side': 'order_side', 'price': 'order_price'})
return orders_df
else:
return pd.DataFrame()
def get_all_positions(client):
try:
positions = client.get_all_positions()
positions = positions[['asset', 'size', 'avgPrice', 'curPrice', 'percentPnl']]
positions = positions.rename(columns={'size': 'position_size'})
return positions
except:
return pd.DataFrame()
def combine_dfs(orders_df, positions, markets_df, selected_df):
merged_df = orders_df.merge(positions, left_on=['asset_id'], right_on=['asset'], how='outer')
merged_df['asset_id'] = merged_df['asset_id'].combine_first(merged_df['asset'])
merged_df = merged_df.drop(columns='asset', axis=1)
merge_token1 = merged_df.merge(markets_df, left_on='asset_id', right_on='token1', how='inner')
merge_token1['merged_with'] = 'token1'
# Merge with token2
merge_token2 = merged_df.merge(markets_df, left_on='asset_id', right_on='token2', how='inner')
merge_token2['merged_with'] = 'token2'
# Combine the results
combined_df = pd.concat([merge_token1, merge_token2])
assert len(merged_df) == len(combined_df)
combined_df['answer'] = combined_df.apply(
lambda row: row['answer1'] if row['merged_with'] == 'token1' else row['answer2'], axis=1
)
combined_df = combined_df[['question', 'answer', 'order_size', 'order_side', 'order_price', 'position_size', 'avgPrice', 'curPrice']]
combined_df['order_side'] = combined_df['order_side'].fillna('')
combined_df = combined_df.fillna(0)
combined_df['marketInSelected'] = combined_df['question'].isin(selected_df['question'])
combined_df = combined_df.sort_values('question')
combined_df = combined_df.sort_values('marketInSelected')
return combined_df
def get_earnings(client):
args = RequestArgs(method='GET', request_path='/rewards/user/markets')
l2Headers = create_level_2_headers(client.signer, client.creds, args)
url = "https://polymarket.com/api/rewards/markets"
cursor = ''
markets = []
params = {
"l2Headers": json.dumps(l2Headers),
"orderBy": "earnings",
"position": "DESC",
"makerAddress": os.getenv('BROWSER_WALLET'),
"authenticationType": "eoa",
"nextCursor": cursor,
"requestPath": "/rewards/user/markets"
}
r = requests.get(url, params=params)
results = r.json()
data = pd.DataFrame(results['data'])
data['earnings'] = data['earnings'].apply(lambda x: x[0]['earnings'])
data = data[data['earnings'] > 0].reset_index(drop=True)
data = data[['question', 'earnings', 'earning_percentage']]
return data
def update_stats_once(client):
spreadsheet = get_spreadsheet()
wk_full = spreadsheet.worksheet('Full Markets')
wk_summary = spreadsheet.worksheet('Summary')
wk_sel = spreadsheet.worksheet('Selected Markets')
selected_df = pd.DataFrame(wk_sel.get_all_records())
markets_df = get_markets_df(wk_full)
print("Got spreadsheet...")
orders_df = get_all_orders(client)
print("Got Orders...")
positions = get_all_positions(client)
print("Got Positions...")
if len(positions) > 0 or len(orders_df) > 0:
combined_df = combine_dfs(orders_df, positions, markets_df, selected_df)
earnings = get_earnings(client.client)
print("Got Earnings...")
combined_df = combined_df.merge(earnings, on='question', how='left')
combined_df = combined_df.fillna(0)
combined_df = combined_df.round(2)
combined_df = combined_df.sort_values('earnings', ascending=False)
combined_df = combined_df[['question', 'answer', 'order_size', 'position_size', 'marketInSelected', 'earnings', 'earning_percentage']]
wk_summary.clear()
set_with_dataframe(wk_summary, combined_df, include_index=False, include_column_header=True, resize=True)
else:
print("Position or order is empty")
poly_utils/google_utils.py
-155
View File
@@ -1,155 +0,0 @@
from google.oauth2.service_account import Credentials
import gspread
import os
import pandas as pd
import requests
import re
from dotenv import load_dotenv
load_dotenv()
def get_spreadsheet(read_only=False):
"""
Get Google Spreadsheet with optional read-only mode.
Args:
read_only (bool): If True, uses public CSV export when credentials are missing
Returns:
Spreadsheet object or ReadOnlySpreadsheet wrapper for read-only mode
"""
spreadsheet_url = os.getenv("SPREADSHEET_URL")
if not spreadsheet_url:
raise ValueError("SPREADSHEET_URL environment variable is not set")
# Check for credentials
creds_file = 'credentials.json' if os.path.exists('credentials.json') else '../credentials.json'
if not os.path.exists(creds_file):
if read_only:
return ReadOnlySpreadsheet(spreadsheet_url)
else:
raise FileNotFoundError(f"Credentials file not found at {creds_file}. Use read_only=True for read-only access.")
# Normal authenticated access
scope = ["https://spreadsheets.google.com/feeds", "https://www.googleapis.com/auth/drive"]
credentials = Credentials.from_service_account_file(creds_file, scopes=scope)
client = gspread.authorize(credentials)
spreadsheet = client.open_by_url(spreadsheet_url)
return spreadsheet
class ReadOnlySpreadsheet:
"""Read-only wrapper for Google Sheets using public CSV export"""
def __init__(self, spreadsheet_url):
self.spreadsheet_url = spreadsheet_url
self.sheet_id = self._extract_sheet_id(spreadsheet_url)
def _extract_sheet_id(self, url):
"""Extract sheet ID from Google Sheets URL"""
match = re.search(r'/spreadsheets/d/([a-zA-Z0-9-_]+)', url)
if not match:
raise ValueError("Invalid Google Sheets URL")
return match.group(1)
def worksheet(self, title):
"""Return a read-only worksheet"""
return ReadOnlyWorksheet(self.sheet_id, title)
class ReadOnlyWorksheet:
"""Read-only worksheet that fetches data via CSV export"""
def __init__(self, sheet_id, title):
self.sheet_id = sheet_id
self.title = title
def get_all_records(self):
"""Get all records from the worksheet as a list of dictionaries"""
try:
# URL encode the sheet title to handle spaces and special characters
import urllib.parse
encoded_title = urllib.parse.quote(self.title)
# Map known sheet names to likely GID positions
# Based on the sheet order: Full Markets, All Markets, Volatility Markets, Selected Markets, Hyperparameters
sheet_gid_mapping = {
'Full Markets': 0,
'All Markets': 1,
'Volatility Markets': 2,
'Selected Markets': 3,
'Hyperparameters': 4
}
# Try multiple URL formats for accessing the sheet
urls_to_try = [
f"https://docs.google.com/spreadsheets/d/{self.sheet_id}/gviz/tq?tqx=out:csv&sheet={encoded_title}",
f"https://docs.google.com/spreadsheets/d/{self.sheet_id}/gviz/tq?tqx=out:csv&sheet={self.title}",
]
# Add GID-based URL if we know the likely position for this sheet
if self.title in sheet_gid_mapping:
gid = sheet_gid_mapping[self.title]
urls_to_try.append(f"https://docs.google.com/spreadsheets/d/{self.sheet_id}/export?format=csv&gid={gid}")
# Also try a few common GID positions as fallback
for gid in [0, 1, 2, 3, 4]:
urls_to_try.append(f"https://docs.google.com/spreadsheets/d/{self.sheet_id}/export?format=csv&gid={gid}")
for csv_url in urls_to_try:
try:
print(f"Trying to fetch sheet '{self.title}' from: {csv_url}")
response = requests.get(csv_url, timeout=30)
response.raise_for_status()
# Read CSV data into DataFrame
from io import StringIO
df = pd.read_csv(StringIO(response.text))
# Check if we got meaningful data (not empty or error response)
if not df.empty and len(df.columns) > 1:
# For Hyperparameters sheet, verify it has the expected columns
if self.title == 'Hyperparameters':
expected_cols = ['type', 'param', 'value']
if all(col in df.columns for col in expected_cols):
print(f"Successfully fetched {len(df)} hyperparameter records")
return df.to_dict('records')
else:
print(f"Sheet doesn't match Hyperparameters format. Columns: {list(df.columns)}")
continue
else:
print(f"Successfully fetched {len(df)} records from sheet '{self.title}'")
# Convert to list of dictionaries (same format as gspread)
return df.to_dict('records')
except Exception as url_error:
print(f"Failed with URL {csv_url}: {url_error}")
continue
print(f"All URL attempts failed for sheet '{self.title}'")
return []
except Exception as e:
print(f"Warning: Could not fetch data from sheet '{self.title}': {e}")
return []
def get_all_values(self):
"""Get all values from the worksheet as a list of lists"""
try:
csv_url = f"https://docs.google.com/spreadsheets/d/{self.sheet_id}/gviz/tq?tqx=out:csv&sheet={self.title}"
response = requests.get(csv_url, timeout=30)
response.raise_for_status()
# Read CSV and return as list of lists
from io import StringIO
df = pd.read_csv(StringIO(response.text))
# Include headers and convert to list of lists
headers = [df.columns.tolist()]
data = df.values.tolist()
return headers + data
except Exception as e:
print(f"Warning: Could not fetch data from sheet '{self.title}': {e}")
return []
pyproject.toml
+59 -24
View File
@@ -1,33 +1,39 @@
[project]
name = "poly-maker"
version = "0.1.0"
description = "A market making bot for Polymarket prediction markets"
name = "polymaker"
version = "2.0.0"
description = "Maker-only market-making bot for Polymarket (CLOB V2), local-file config, political markets"
readme = "README.md"
requires-python = ">=3.9.10"
requires-python = ">=3.12"
license = { text = "MIT" }
dependencies = [
"py-clob-client==0.28.0",
"python-dotenv==1.2.1",
"pandas==2.3.3",
"gspread==6.2.1",
"gspread-dataframe==4.0.0",
"sortedcontainers==2.4.0",
"eth-account==0.13.7",
"eth-utils==5.3.1",
"poly_eip712_structs==0.0.1",
"py_order_utils==0.3.2",
"requests==2.32.5",
"websockets==15.0.1",
"cryptography==46.0.3",
"google-auth==2.42.1",
"web3==7.14.0",
"py-clob-client-v2==1.0.2",
"web3>=7.14",
"httpx>=0.28",
"websockets>=15.0",
"pydantic>=2.10",
"pydantic-settings>=2.7",
"structlog>=25.1",
"typer>=0.15",
"rich>=14.0",
"watchfiles>=1.0",
"python-dotenv>=1.0",
"sortedcontainers>=2.4",
"uvloop>=0.21 ; sys_platform != 'win32'",
"socksio>=1.0",
"python-socks>=2.4",
]
[project.scripts]
polymaker = "polymaker.cli:app"
[project.optional-dependencies]
dev = [
"black==24.4.2",
"pytest==8.2.2",
"pytest>=8.3",
"pytest-asyncio>=0.25",
"respx>=0.22",
"ruff>=0.9",
"mypy>=1.14",
]
[build-system]
@@ -35,8 +41,37 @@ requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["poly_data", "poly_stats", "poly_utils", "data_updater"]
packages = ["src/polymaker"]
[tool.black]
[tool.ruff]
line-length = 100
target-version = ["py39"]
target-version = "py312"
src = ["src", "tests"]
[tool.ruff.lint]
select = ["E", "F", "I", "UP", "B", "SIM", "ASYNC"]
ignore = [
"E501", # line length handled by formatter
"UP042", # str+Enum is intentional (JSON-serializable, API-string valued)
"ASYNC109", # timeout params on client wrappers are fine
"SIM108", # ternary-vs-if is a readability call
]
[tool.mypy]
python_version = "3.12"
packages = ["polymaker"]
strict = true
warn_return_any = true
disallow_untyped_defs = true
# third-party libs without stubs
[[tool.mypy.overrides]]
module = ["py_clob_client_v2.*", "sortedcontainers.*", "web3.*", "eth_account.*"]
ignore_missing_imports = true
[tool.pytest.ini_options]
asyncio_mode = "auto"
testpaths = ["tests"]
filterwarnings = ["ignore::DeprecationWarning"]
[tool.uv]
package = true
src/polymaker/__init__.py
+3
View File
@@ -0,0 +1,3 @@
"""polymaker — maker-only market-making bot for Polymarket CLOB V2."""
__version__ = "2.0.0"
poly_data/__init__.py → src/polymaker/catalog/__init__.py
View File
src/polymaker/catalog/gamma.py
+187
View File
@@ -0,0 +1,187 @@
"""Async Gamma API client for market discovery.
Gamma (https://gamma-api.polymarket.com, no auth) returns everything the v1
scanner burned two extra REST calls per market to compute: best bid/ask,
liquidity, volume, reward params, fee schedule, tick size, tokens. We filter
server-side by the politics tag and liquidity/volume, so a full political-market
sweep is a handful of paginated requests.
"""
from __future__ import annotations
import json
from collections.abc import AsyncIterator
from typing import Any
import httpx
from polymaker.domain import MarketMeta, TokenMeta
from polymaker.logging import get_logger
log = get_logger("catalog.gamma")
POLITICS_TAG_SLUG = "politics"
class GammaClient:
"""Thin async wrapper over the Gamma REST endpoints we use."""
def __init__(self, host: str = "https://gamma-api.polymarket.com", timeout: float = 20.0) -> None:
self._host = host.rstrip("/")
self._client = httpx.AsyncClient(base_url=self._host, timeout=timeout)
async def __aenter__(self) -> GammaClient:
return self
async def __aexit__(self, *exc: object) -> None:
await self.aclose()
async def aclose(self) -> None:
await self._client.aclose()
async def resolve_tag_id(self, slug: str) -> str | None:
try:
r = await self._client.get(f"/tags/slug/{slug}")
r.raise_for_status()
return str(r.json()["id"])
except (httpx.HTTPError, KeyError, json.JSONDecodeError):
log.warning("tag_resolve_failed", slug=slug)
return None
async def iter_markets(
self,
*,
tag_id: str | None = None,
related_tags: bool = True,
min_liquidity: float = 0.0,
min_volume_24hr: float = 0.0,
limit: int = 100, # Gamma caps a page at 100 regardless of a higher ask
max_pages: int = 25,
) -> AsyncIterator[dict[str, Any]]:
"""Yield raw active/open market dicts, offset-paginated.
Uses the offset `/markets` endpoint because it reliably supports
`tag_id` filtering today. (Keyset is the go-forward per docs; switch when
it supports tag filtering. See the README.)
"""
offset = 0
for _ in range(max_pages):
params: dict[str, Any] = {
"limit": limit,
"offset": offset,
"active": "true",
"closed": "false",
"order": "volume24hr",
"ascending": "false",
}
if tag_id:
params["tag_id"] = tag_id
params["related_tags"] = "true" if related_tags else "false"
if min_liquidity > 0:
params["liquidity_num_min"] = min_liquidity
if min_volume_24hr > 0:
params["volume_num_min"] = min_volume_24hr
r = await self._client.get("/markets", params=params)
r.raise_for_status()
batch = r.json()
if not batch:
return
for m in batch:
yield m
if len(batch) < limit:
return
offset += limit
def parse_market(raw: dict[str, Any], reward_rates: dict[str, float] | None = None) -> MarketMeta | None:
"""Convert a Gamma market dict into our MarketMeta, or None if unusable."""
try:
if not raw.get("acceptingOrders", False):
return None
token_ids = _json_list(raw.get("clobTokenIds"))
outcomes = _json_list(raw.get("outcomes"))
if len(token_ids) != 2 or len(outcomes) != 2:
return None # only binary markets
condition_id = raw["conditionId"]
rate_map = reward_rates or {}
fee = raw.get("feeSchedule") or {}
taker_rate = float(fee.get("rate", 0.0) or 0.0)
event_id = None
events = raw.get("events") or []
if events:
event_id = str(events[0].get("id")) if events[0].get("id") is not None else None
return MarketMeta(
condition_id=condition_id,
question=raw.get("question", ""),
slug=raw.get("slug", ""),
tokens=(
TokenMeta(str(token_ids[0]), str(outcomes[0])),
TokenMeta(str(token_ids[1]), str(outcomes[1])),
),
tick_size=float(raw.get("orderPriceMinTickSize", 0.001) or 0.001),
neg_risk=bool(raw.get("negRisk", False)),
min_order_size=float(raw.get("orderMinSize", 5) or 5),
rewards_min_size=float(raw.get("rewardsMinSize", 0) or 0),
rewards_max_spread=float(raw.get("rewardsMaxSpread", 0) or 0),
rewards_daily_rate=float(rate_map.get(condition_id, 0.0)),
maker_fee_bps=0, # V2: makers pay zero
taker_fee_bps=int(round(taker_rate * 10000)),
fees_enabled=bool(raw.get("feesEnabled", False)),
rebate_rate=float(fee.get("rebateRate", 0.0) or 0.0),
end_date_iso=raw.get("endDate"),
event_id=event_id,
best_bid=float(raw.get("bestBid", 0) or 0),
best_ask=float(raw.get("bestAsk", 0) or 0),
liquidity_num=float(raw.get("liquidityNum", 0) or 0),
volume_num=float(raw.get("volumeNum", 0) or 0),
)
except (KeyError, ValueError, TypeError) as exc:
log.warning("parse_market_failed", err=str(exc), slug=raw.get("slug"))
return None
def _json_list(value: Any) -> list[Any]:
"""clobTokenIds / outcomes arrive as JSON-encoded strings."""
if value is None:
return []
if isinstance(value, list):
return value
try:
parsed = json.loads(value)
return parsed if isinstance(parsed, list) else []
except (json.JSONDecodeError, TypeError):
return []
async def fetch_reward_rates(
clob_host: str = "https://clob.polymarket.com", timeout: float = 20.0
) -> dict[str, float]:
"""Build {condition_id: daily USDC reward rate} from CLOB sampling-markets.
These are the rewards-enabled markets; the daily rate isn't on Gamma.
"""
usdc = "0x2791bca1f2de4661ed88a30c99a7a9449aa84174"
rates: dict[str, float] = {}
async with httpx.AsyncClient(base_url=clob_host.rstrip("/"), timeout=timeout) as client:
cursor = ""
for _ in range(50):
r = await client.get("/sampling-markets", params={"next_cursor": cursor})
r.raise_for_status()
data = r.json()
for m in data.get("data", []):
cid = m.get("condition_id")
rate = 0.0
for ri in (m.get("rewards") or {}).get("rates") or []:
if str(ri.get("asset_address", "")).lower() == usdc:
rate = float(ri.get("rewards_daily_rate", 0) or 0)
break
if cid:
rates[cid] = rate
cursor = data.get("next_cursor") or ""
if not cursor or cursor == "LTE=": # "LTE=" is the documented end sentinel
break
return rates
src/polymaker/catalog/scanner.py
+63
View File
@@ -0,0 +1,63 @@
"""The scanner: sweep Gamma for political markets, score, persist to SQLite.
Replaces the v1 data_updater (hour-long crawl of every order book, written to
Google Sheets). A politics-filtered sweep here is seconds and one process.
"""
from __future__ import annotations
from dataclasses import dataclass
from polymaker.catalog.gamma import (
POLITICS_TAG_SLUG,
GammaClient,
fetch_reward_rates,
parse_market,
)
from polymaker.catalog.scoring import score_market
from polymaker.catalog.store import CatalogStore
from polymaker.domain import MarketMeta
from polymaker.logging import get_logger
log = get_logger("catalog.scanner")
@dataclass(frozen=True, slots=True)
class ScanConfig:
tag_slug: str = POLITICS_TAG_SLUG
min_liquidity: float = 1000.0
min_volume_24hr: float = 0.0
rewards_only: bool = True # keep only markets in the liquidity-rewards program
gamma_host: str = "https://gamma-api.polymarket.com"
clob_host: str = "https://clob.polymarket.com"
async def run_scan(store: CatalogStore, cfg: ScanConfig) -> list[MarketMeta]:
"""Fetch, parse, filter, score, and persist. Returns the kept markets."""
reward_rates = await fetch_reward_rates(cfg.clob_host)
log.info("reward_rates_loaded", n=len(reward_rates))
kept: list[MarketMeta] = []
async with GammaClient(cfg.gamma_host) as gamma:
tag_id = store.cached_tag(cfg.tag_slug) or await gamma.resolve_tag_id(cfg.tag_slug)
if tag_id:
store.cache_tag(cfg.tag_slug, tag_id)
seen = 0
async for raw in gamma.iter_markets(
tag_id=tag_id,
min_liquidity=cfg.min_liquidity,
min_volume_24hr=cfg.min_volume_24hr,
):
seen += 1
meta = parse_market(raw, reward_rates)
if meta is None:
continue
if cfg.rewards_only and meta.rewards_daily_rate <= 0:
continue
kept.append(meta)
for m in kept:
store.upsert_market(m, score_market(m))
log.info("scan_complete", seen=seen, kept=len(kept), tag=cfg.tag_slug)
return kept
src/polymaker/catalog/scoring.py
+80
View File
@@ -0,0 +1,80 @@
"""Market attractiveness scoring for the scanner.
Combines the v1 reward-density intuition with the new maker-rebate income
stream and penalizes spread/extremes. Higher score = more attractive to make.
Pure functions over MarketMeta.
"""
from __future__ import annotations
from dataclasses import dataclass
from polymaker.domain import MarketMeta
@dataclass(frozen=True, slots=True)
class MarketScore:
condition_id: str
reward_density: float # est. reward $/day per $100 of two-sided liquidity
rebate_potential: float # est. daily rebate $ available to makers
spread: float
extremity: float # 0 = mid ~0.5 (good), 1 = near 0/1 (bad payoff asymmetry)
score: float
def _mid(m: MarketMeta) -> float:
if m.best_bid > 0 and m.best_ask > 0:
return (m.best_bid + m.best_ask) / 2.0
return 0.5
def reward_density(m: MarketMeta, quote_size_usdc: float = 100.0) -> float:
"""Rough reward $/day if we hold ~quote_size two-sided in-band.
The exact per-order S((v-s)/v)^2 scoring depends on live competition; for
ranking we use daily_rate scaled by how much of the (small) market our
typical size represents, capped. This mirrors v1's gm_reward_per_100 as a
relative ranking signal, not an absolute forecast.
"""
if m.rewards_daily_rate <= 0 or m.rewards_max_spread <= 0:
return 0.0
liq = max(m.liquidity_num, quote_size_usdc)
our_share = min(1.0, quote_size_usdc / liq)
return m.rewards_daily_rate * our_share
def rebate_potential(m: MarketMeta) -> float:
"""Est. daily maker-rebate pool: taker_fee_rate * rebate_rate * daily volume."""
if not m.fees_enabled or m.rebate_rate <= 0 or m.taker_fee_bps <= 0:
return 0.0
daily_vol = m.volume_num # best proxy available from catalog; refined live
taker_rate = m.taker_fee_bps / 10000.0
# taker fee peaks at p*(1-p); use mid as the representative point
mid = _mid(m)
fee_factor = mid * (1.0 - mid)
return daily_vol * taker_rate * fee_factor * m.rebate_rate * 0.01 # 1% daily-vol proxy
def extremity(m: MarketMeta) -> float:
"""0 near 0.5 (balanced), ->1 near the 0/1 boundary (skip these)."""
mid = _mid(m)
return min(1.0, abs(mid - 0.5) / 0.5)
def score_market(m: MarketMeta) -> MarketScore:
rd = reward_density(m)
rp = rebate_potential(m)
ext = extremity(m)
spread = max(0.0, m.best_ask - m.best_bid) if (m.best_bid and m.best_ask) else 1.0
# income terms are additive; extremity and wide spreads discount the score
income = rd + rp
penalty = (1.0 - 0.5 * ext) * (1.0 / (1.0 + spread * 20.0))
return MarketScore(
condition_id=m.condition_id,
reward_density=round(rd, 3),
rebate_potential=round(rp, 3),
spread=round(spread, 4),
extremity=round(ext, 3),
score=round(income * penalty, 4),
)
src/polymaker/catalog/store.py
+123
View File
@@ -0,0 +1,123 @@
"""SQLite persistence for the market catalog and scan results.
Replaces the v1 "All Markets" / "Volatility Markets" Google Sheets. One local
file (state.db), queryable by the CLI. WAL mode so the running bot and a
`polymaker markets` query don't block each other.
"""
from __future__ import annotations
import json
import sqlite3
import time
from dataclasses import asdict
from pathlib import Path
from polymaker.catalog.scoring import MarketScore, score_market
from polymaker.domain import MarketMeta, TokenMeta
_SCHEMA = """
CREATE TABLE IF NOT EXISTS markets (
condition_id TEXT PRIMARY KEY,
question TEXT,
slug TEXT,
meta_json TEXT NOT NULL,
score REAL DEFAULT 0,
score_json TEXT,
scanned_ts REAL NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_markets_score ON markets(score DESC);
CREATE INDEX IF NOT EXISTS idx_markets_slug ON markets(slug);
CREATE TABLE IF NOT EXISTS tags (
slug TEXT PRIMARY KEY,
tag_id TEXT NOT NULL,
ts REAL NOT NULL
);
"""
class CatalogStore:
"""Owns the markets/tags tables in state.db."""
def __init__(self, db_path: str | Path = "state.db") -> None:
self.path = str(db_path)
self._conn = sqlite3.connect(self.path)
self._conn.row_factory = sqlite3.Row
self._conn.execute("PRAGMA journal_mode=WAL")
self._conn.executescript(_SCHEMA)
self._conn.commit()
def close(self) -> None:
self._conn.close()
def upsert_market(self, meta: MarketMeta, score: MarketScore | None = None) -> None:
sc = score or score_market(meta)
self._conn.execute(
"""INSERT INTO markets(condition_id, question, slug, meta_json, score, score_json, scanned_ts)
VALUES(?,?,?,?,?,?,?)
ON CONFLICT(condition_id) DO UPDATE SET
question=excluded.question, slug=excluded.slug, meta_json=excluded.meta_json,
score=excluded.score, score_json=excluded.score_json, scanned_ts=excluded.scanned_ts""",
(
meta.condition_id,
meta.question,
meta.slug,
_dump_meta(meta),
sc.score,
json.dumps(asdict(sc)),
time.time(),
),
)
self._conn.commit()
def upsert_many(self, metas: list[MarketMeta]) -> int:
for m in metas:
self.upsert_market(m)
return len(metas)
def get(self, condition_id: str) -> MarketMeta | None:
row = self._conn.execute(
"SELECT meta_json FROM markets WHERE condition_id=?", (condition_id,)
).fetchone()
return _load_meta(row["meta_json"]) if row else None
def get_by_slug(self, slug: str) -> MarketMeta | None:
row = self._conn.execute(
"SELECT meta_json FROM markets WHERE slug=?", (slug,)
).fetchone()
return _load_meta(row["meta_json"]) if row else None
def top(self, limit: int = 50) -> list[tuple[MarketMeta, MarketScore]]:
rows = self._conn.execute(
"SELECT meta_json, score_json FROM markets ORDER BY score DESC LIMIT ?", (limit,)
).fetchall()
out = []
for row in rows:
meta = _load_meta(row["meta_json"])
sc = MarketScore(**json.loads(row["score_json"])) if row["score_json"] else score_market(meta)
out.append((meta, sc))
return out
def cache_tag(self, slug: str, tag_id: str) -> None:
self._conn.execute(
"INSERT OR REPLACE INTO tags(slug, tag_id, ts) VALUES(?,?,?)",
(slug, tag_id, time.time()),
)
self._conn.commit()
def cached_tag(self, slug: str) -> str | None:
row = self._conn.execute("SELECT tag_id FROM tags WHERE slug=?", (slug,)).fetchone()
return row["tag_id"] if row else None
def _dump_meta(meta: MarketMeta) -> str:
d = asdict(meta)
d["tokens"] = [asdict(t) for t in meta.tokens]
return json.dumps(d)
def _load_meta(blob: str) -> MarketMeta:
d = json.loads(blob)
d["tokens"] = tuple(TokenMeta(**t) for t in d["tokens"])
return MarketMeta(**d)
src/polymaker/cli.py
+213
View File
@@ -0,0 +1,213 @@
"""polymaker command-line interface.
polymaker scan sweep Gamma for political markets -> SQLite
polymaker markets rank/browse the catalog
polymaker markets-add <slug> append a market to config/markets.toml
polymaker status positions / open orders / PnL (reads SQLite)
polymaker doctor preflight: wallet auth, balances, WS reachability
polymaker run [--paper] start the market maker
polymaker cancel-all panic button
"""
from __future__ import annotations
import asyncio
from pathlib import Path
from typing import Any
import typer
from rich.console import Console
from rich.table import Table
from polymaker import __version__
from polymaker.config import Config
app = typer.Typer(
name="polymaker",
help="Maker-only market maker for Polymarket CLOB V2.",
no_args_is_help=True,
add_completion=False,
)
console = Console()
@app.command()
def version() -> None:
"""Print the polymaker version."""
console.print(f"polymaker {__version__}")
@app.command()
def scan(
config_dir: str = typer.Option("config", help="config directory"),
min_liquidity: float = typer.Option(1000.0, help="minimum market liquidity (USDC)"),
all_markets: bool = typer.Option(False, "--all", help="include non-rewards markets"),
) -> None:
"""Sweep Gamma for political markets, score, and persist to SQLite."""
from polymaker.catalog.scanner import ScanConfig, run_scan
from polymaker.catalog.store import CatalogStore
cfg = Config.load(config_dir)
store = CatalogStore(cfg.paths.db)
async def _go() -> int:
metas = await run_scan(store, ScanConfig(min_liquidity=min_liquidity, rewards_only=not all_markets))
return len(metas)
n = asyncio.run(_go())
console.print(f"[green]Scanned and stored {n} markets.[/green] Run [bold]polymaker markets[/bold] to browse.")
store.close()
@app.command()
def markets(
config_dir: str = typer.Option("config", help="config directory"),
limit: int = typer.Option(25, help="rows to show"),
) -> None:
"""Show the top scored markets from the catalog."""
from polymaker.catalog.store import CatalogStore
cfg = Config.load(config_dir)
store = CatalogStore(cfg.paths.db)
rows = store.top(limit)
if not rows:
console.print("[yellow]Catalog empty. Run `polymaker scan` first.[/yellow]")
raise typer.Exit()
table = Table(title="Political markets by score")
for col in ("score", "reward/day", "rebate/day", "spread", "tick", "neg", "question"):
table.add_column(col, justify="right" if col != "question" else "left")
for meta, sc in rows:
table.add_row(
f"{sc.score:.2f}", f"{meta.rewards_daily_rate:.0f}", f"{sc.rebate_potential:.0f}",
f"{sc.spread:.3f}", f"{meta.tick_size:g}", "Y" if meta.neg_risk else "-",
meta.question[:60],
)
console.print(table)
console.print("\nAdd one with: [bold]polymaker markets-add <slug>[/bold] (slugs are in the catalog)")
@app.command(name="markets-add")
def markets_add(
slug: str,
profile: str = typer.Option("political-longdated", help="strategy profile"),
config_dir: str = typer.Option("config", help="config directory"),
) -> None:
"""Append a market (by slug) to config/markets.toml."""
from polymaker.catalog.store import CatalogStore
cfg = Config.load(config_dir)
store = CatalogStore(cfg.paths.db)
meta = store.get_by_slug(slug)
store.close()
if meta is None:
console.print(f"[red]No market with slug {slug!r} in the catalog. Run `polymaker scan`.[/red]")
raise typer.Exit(1)
path = Path(config_dir) / "markets.toml"
block = f'\n[[markets]]\nslug = "{slug}"\nprofile = "{profile}"\nenabled = true\n'
with path.open("a") as fh:
fh.write(block)
console.print(f"[green]Added[/green] {meta.question[:60]!r} to {path}")
@app.command()
def status(config_dir: str = typer.Option("config", help="config directory")) -> None:
"""Show positions, open orders, and marks from the local state DB."""
from polymaker.state.store import StateStore
cfg = Config.load(config_dir)
store = StateStore(cfg.paths.db)
snap = store.snapshot()
console.print(f"[bold]Open orders:[/bold] {snap['open_orders']}")
positions: dict[str, Any] = snap["positions"] # type: ignore[assignment]
if not positions:
console.print("[dim]No open positions.[/dim]")
else:
table = Table(title="Positions")
table.add_column("token")
table.add_column("size", justify="right")
table.add_column("avg", justify="right")
for tok, p in positions.items():
table.add_row(tok[:16] + "…", f"{p['size']:.2f}", f"{p['avg_price']:.3f}")
console.print(table)
store.close()
@app.command()
def doctor(config_dir: str = typer.Option("config", help="config directory")) -> None:
"""Preflight checks: config, wallet auth, balance/allowance, WS reachability."""
from polymaker.doctor import run_doctor
cfg = Config.load(config_dir)
ok = asyncio.run(run_doctor(cfg, console))
raise typer.Exit(0 if ok else 1)
@app.command()
def run(
config_dir: str = typer.Option("config", help="config directory"),
paper: bool = typer.Option(False, "--paper", help="paper mode: full pipeline, no orders posted"),
) -> None:
"""Start the market maker."""
from polymaker.engine import Engine
from polymaker.logging import configure
cfg = Config.load(config_dir)
configure(json_file=Path(cfg.paths.log_dir) / ("paper.jsonl" if paper else "live.jsonl"))
if cfg.engine.loop == "uvloop":
try:
import uvloop
uvloop.install()
except Exception: # noqa: BLE001
pass
engine = Engine(cfg, paper=paper)
async def _go() -> None:
try:
await engine.run_forever()
except (KeyboardInterrupt, asyncio.CancelledError):
pass
finally:
await engine.shutdown()
console.print(f"[bold green]Starting polymaker[/bold green] ({'PAPER' if paper else 'LIVE'})…")
try:
asyncio.run(_go())
except KeyboardInterrupt:
console.print("\n[yellow]Stopped.[/yellow]")
@app.command()
def livetest(
config_dir: str = typer.Option("config", help="config directory"),
notional: float = typer.Option(5.0, help="order notional in USDC"),
) -> None:
"""Live wallet round-trip: place a deep post-only order and cancel it (~$5)."""
from polymaker.livetest import run_livetest
cfg = Config.load(config_dir)
ok = asyncio.run(run_livetest(cfg, console, notional))
raise typer.Exit(0 if ok else 1)
@app.command(name="cancel-all")
def cancel_all(config_dir: str = typer.Option("config", help="config directory")) -> None:
"""Cancel all open orders for the wallet (panic button)."""
from polymaker.execution.gateway import ExecutionGateway
cfg = Config.load(config_dir)
gw = ExecutionGateway(cfg)
async def _go() -> None:
await gw.connect()
await gw.cancel_all()
asyncio.run(_go())
console.print("[green]Sent cancel-all.[/green]")
if __name__ == "__main__":
app()
src/polymaker/config.py
+226
View File
@@ -0,0 +1,226 @@
"""Configuration: pydantic models over local TOML files + .env secrets.
Replaces the v1 Google Sheets config entirely. Three files under config/:
config.toml engine/wallet/risk/execution settings
strategy.toml named parameter profiles
markets.toml the trade list (market -> profile + overrides)
Secrets (private key, wallet address) come only from the environment / .env.
"""
from __future__ import annotations
import os
import tomllib
from pathlib import Path
from typing import Any
from dotenv import load_dotenv
from pydantic import BaseModel, ConfigDict, Field, model_validator
from pydantic_settings import BaseSettings, SettingsConfigDict
class WalletConfig(BaseModel):
chain_id: int = 137
signature_type: int = 2
clob_host: str = "https://clob.polymarket.com"
gamma_host: str = "https://gamma-api.polymarket.com"
data_api_host: str = "https://data-api.polymarket.com"
polygon_rpc: str = "https://polygon-rpc.com"
class EngineConfig(BaseModel):
debounce_ms: int = 200
reconcile_interval_s: float = 30.0
catalog_refresh_s: float = 900.0
heartbeat: bool = True
heartbeat_interval_s: float = 5.0
journal: bool = True
loop: str = "uvloop"
class RiskConfig(BaseModel):
max_total_exposure_usdc: float = 5000.0
max_event_group_loss_usdc: float = 1000.0
max_market_notional_usdc: float = 800.0
daily_loss_kill_usdc: float = 250.0
ws_stale_halt_s: float = 10.0
max_order_error_rate: float = 0.25
class ExecutionConfig(BaseModel):
rate_budget_fraction: float = 0.25
post_only: bool = True
max_orders_per_batch: int = 15
class PathsConfig(BaseModel):
db: str = "state.db"
journal_dir: str = "journal"
log_dir: str = "logs"
class StrategyProfile(BaseModel):
"""One named parameter set. Every knob the quoter uses lives here."""
model_config = ConfigDict(extra="forbid")
# fair value
micro_levels: int = 3
flow_ewma_halflife_s: float = 120.0
# spread / skew
gamma: float = 0.5
delta_min_ticks: int = 2
c_vol: float = 1.2
c_tox: float = 2.0
# vol horizons
vol_short_halflife_s: float = 10.0
vol_long_halflife_s: float = 900.0
# sizing / inventory
base_size_usdc: float = 50.0
q_max_usdc: float = 500.0
q_soft_frac: float = 0.6
layers: int = 2
layer_step_ticks: int = 2
# placement / churn
reprice_ticks: int = 2
resize_frac: float = 0.15
min_edge_ticks: int = 1
# regime
event_cooloff_s: float = 60.0
event_jump_ticks: int = 8
event_sweep_levels: int = 3
trend_flow_z: float = 1.5
# lifecycle
end_date_taper_days: float = 7.0
reduce_only_hours: float = 24.0
halt_before_hours: float = 2.0
# exits
exit_urgency_s: float = 900.0
merge_min_size: float = 20.0
def with_overrides(self, overrides: dict[str, Any]) -> StrategyProfile:
"""Return a copy with per-market override values applied."""
if not overrides:
return self
data = self.model_dump()
for k, v in overrides.items():
if k in data:
data[k] = v
return StrategyProfile(**data)
# Keys allowed on a market entry that are NOT profile overrides.
_MARKET_RESERVED = {"slug", "condition_id", "profile", "enabled"}
class MarketEntry(BaseModel):
"""One line of the trade list. Extra keys are treated as profile overrides."""
model_config = ConfigDict(extra="allow")
slug: str | None = None
condition_id: str | None = None
profile: str = "political-longdated"
enabled: bool = True
@model_validator(mode="after")
def _need_identifier(self) -> MarketEntry:
if not self.slug and not self.condition_id:
raise ValueError("market entry needs a slug or condition_id")
return self
@property
def overrides(self) -> dict[str, Any]:
extra = self.model_extra or {}
return {k: v for k, v in extra.items() if k not in _MARKET_RESERVED}
@property
def ref(self) -> str:
return self.slug or self.condition_id or "?"
class Secrets(BaseSettings):
"""Loaded from environment / .env. Never written to disk by us."""
model_config = SettingsConfigDict(env_file=".env", extra="ignore")
pk: str = Field(default="", alias="PK")
browser_address: str = Field(default="", alias="BROWSER_ADDRESS")
polygon_rpc: str | None = Field(default=None, alias="POLYGON_RPC")
alert_webhook_url: str | None = Field(default=None, alias="ALERT_WEBHOOK_URL")
@property
def has_wallet(self) -> bool:
return bool(self.pk and self.browser_address)
class Config(BaseModel):
"""Fully-resolved configuration tree."""
wallet: WalletConfig = WalletConfig()
engine: EngineConfig = EngineConfig()
risk: RiskConfig = RiskConfig()
execution: ExecutionConfig = ExecutionConfig()
paths: PathsConfig = PathsConfig()
profiles: dict[str, StrategyProfile] = {}
markets: list[MarketEntry] = []
secrets: Secrets = Field(default_factory=Secrets)
config_dir: Path = Path("config")
@property
def proxy(self) -> str | None:
# Standard proxy env var; ALL_PROXY lets you route through an SSH tunnel
# (e.g. simulate colocation during local testing). httpx and web3 honor
# it automatically once load_dotenv() has run.
return os.environ.get("ALL_PROXY") or os.environ.get("HTTPS_PROXY")
@property
def enabled_markets(self) -> list[MarketEntry]:
return [m for m in self.markets if m.enabled]
def profile_for(self, entry: MarketEntry) -> StrategyProfile:
base = self.profiles.get(entry.profile)
if base is None:
raise KeyError(f"unknown strategy profile: {entry.profile!r}")
return base.with_overrides(entry.overrides)
@classmethod
def load(cls, config_dir: str | Path = "config", *, load_env: bool = True) -> Config:
cdir = Path(config_dir)
if load_env:
load_dotenv()
main = _read_toml(cdir / "config.toml")
strat = _read_toml(cdir / "strategy.toml")
mkts = _read_toml(cdir / "markets.toml")
profiles = {
name: StrategyProfile(**params)
for name, params in (strat.get("profiles") or {}).items()
}
markets = [MarketEntry(**m) for m in (mkts.get("markets") or [])]
return cls(
wallet=WalletConfig(**main.get("wallet", {})),
engine=EngineConfig(**main.get("engine", {})),
risk=RiskConfig(**main.get("risk", {})),
execution=ExecutionConfig(**main.get("execution", {})),
paths=PathsConfig(**main.get("paths", {})),
profiles=profiles,
markets=markets,
secrets=Secrets(),
config_dir=cdir,
)
def reload_markets(self) -> Config:
"""Re-read markets.toml only (used by the hot-reload path)."""
mkts = _read_toml(self.config_dir / "markets.toml")
self.markets = [MarketEntry(**m) for m in (mkts.get("markets") or [])]
return self
def _read_toml(path: Path) -> dict[str, Any]:
if not path.exists():
return {}
with path.open("rb") as fh:
return tomllib.load(fh)
src/polymaker/doctor.py
+184
View File
@@ -0,0 +1,184 @@
"""Preflight checks for `polymaker doctor`.
Verifies the environment is ready to trade WITHOUT posting any order:
config + secrets, CLOB/Gamma reachable, wallet auth (L1->L2 creds), collateral
balance + positions ON THE FUNDER (deposit/developer wallet, where funds live),
a live market-WS book frame, and an authenticated user-WS connection. This is
the gate before the live $5 round-trip (the README).
"""
from __future__ import annotations
import asyncio
import json
from typing import Any
import httpx
import websockets
from rich.console import Console
from polymaker.config import Config
MARKET_WS = "wss://ws-subscriptions-clob.polymarket.com/ws/market"
USER_WS = "wss://ws-subscriptions-clob.polymarket.com/ws/user"
async def run_doctor(cfg: Config, console: Console) -> bool:
ok = True
def check(label: str, passed: bool, detail: str = "") -> None:
nonlocal ok
mark = "[green]✓[/green]" if passed else "[red]✗[/red]"
console.print(f" {mark} {label}" + (f" [dim]{detail}[/dim]" if detail else ""))
ok = ok and passed
console.print("[bold]polymaker doctor[/bold]")
# ── config + secrets ────────────────────────────────────────────────
check("config loads", True, f"{len(cfg.profiles)} profiles, {len(cfg.markets)} markets")
check("PK + BROWSER_ADDRESS set", cfg.secrets.has_wallet,
"put them in .env" if not cfg.secrets.has_wallet
else f"funder {cfg.secrets.browser_address[:10]}…, sig_type {cfg.wallet.signature_type}")
# ── REST reachability ───────────────────────────────────────────────
async with httpx.AsyncClient(timeout=10) as c:
try:
r = await c.get(f"{cfg.wallet.clob_host}/ok")
check("CLOB reachable", r.status_code == 200, cfg.wallet.clob_host)
except httpx.HTTPError as e:
check("CLOB reachable", False, str(e))
try:
r = await c.get(f"{cfg.wallet.gamma_host}/markets", params={"limit": 1})
check("Gamma reachable", r.status_code == 200, cfg.wallet.gamma_host)
except httpx.HTTPError as e:
check("Gamma reachable", False, str(e))
# ── wallet auth + balance + positions (on the FUNDER) ───────────────
creds: Any = None
funder = ""
held_tokens: list[str] = []
if cfg.secrets.has_wallet:
try:
from polymaker.execution.gateway import ExecutionGateway
gw = ExecutionGateway(cfg)
await gw.connect()
creds = gw.creds
funder = gw.funder
check("wallet auth (L2 creds derived)", bool(gw.creds),
f"signer {gw.address[:10]}… signs for funder {funder[:10]}…")
ba = await gw.balance_allowance()
bal = _extract_balance(ba)
check("collateral (pUSD) balance readable", bal is not None,
f"≈{bal:.2f} pUSD on funder {funder[:10]}…" if bal is not None else "check allowances")
if bal is not None and bal <= 0:
console.print(" [yellow]! balance is 0 — deposit USDC (mints pUSD) and set "
"allowances from the deposit wallet (trade once in the UI)[/yellow]")
positions = await gw.positions()
held_tokens = list(positions)
total_shares = sum(sz for sz, _ in positions.values())
check("positions readable (on funder)", True,
f"{len(positions)} positions, {total_shares:.0f} shares total")
except Exception as e: # noqa: BLE001
check("wallet auth (L2 creds derived)", False, str(e))
console.print(" [yellow]! signature-type mismatch? deposit wallets use sig_type=3 "
"(config.toml). See the README.[/yellow]")
else:
console.print(" [yellow]! skipping wallet checks (no secrets)[/yellow]")
if cfg.proxy:
console.print(f" [dim]· routing via proxy {cfg.proxy.split('@')[-1]}[/dim]")
# ── live market WS: receive an actual book frame ────────────────────
token = held_tokens[0] if held_tokens else await _top_political_token(cfg)
if token:
passed, detail = await _market_ws_book(token, cfg.proxy)
check("market WS live book frame", passed, detail)
else:
check("market WS live book frame", False, "no token to subscribe to")
# ── live user WS: authenticate ──────────────────────────────────────
if creds is not None:
markets = [cfg.markets[0].condition_id] if cfg.markets and cfg.markets[0].condition_id else []
passed, detail = await _user_ws_auth(creds, markets, cfg.proxy)
check("user WS authenticated", passed, detail)
else:
console.print(" [dim]· skipping user WS (needs wallet creds)[/dim]")
console.print(f"\n[bold]{'READY' if ok else 'NOT READY'}[/bold]")
return ok
async def _market_ws_book(token: str, proxy: str | None = None) -> tuple[bool, str]:
"""Subscribe to a token and confirm a real `book` frame arrives."""
kw: dict[str, Any] = {"ping_interval": 5, "ping_timeout": None, "open_timeout": 10}
if proxy:
kw["proxy"] = proxy
try:
async with websockets.connect(MARKET_WS, **kw) as ws:
await ws.send(json.dumps({"assets_ids": [token], "type": "market"}))
for _ in range(12):
raw = await asyncio.wait_for(ws.recv(), timeout=8)
data = json.loads(raw)
for m in data if isinstance(data, list) else [data]:
if isinstance(m, dict) and m.get("event_type") == "book":
nb, na = len(m.get("bids", [])), len(m.get("asks", []))
return True, f"book received: {nb} bids / {na} asks"
except Exception as e: # noqa: BLE001
return False, str(e)[:80]
return False, "no book frame within timeout"
async def _user_ws_auth(creds: Any, markets: list[str], proxy: str | None = None) -> tuple[bool, str]:
"""Authenticate on the user channel and confirm the server accepts it."""
kw: dict[str, Any] = {"ping_interval": 5, "ping_timeout": None, "open_timeout": 10}
if proxy:
kw["proxy"] = proxy
try:
async with websockets.connect(USER_WS, **kw) as ws:
await ws.send(json.dumps({
"type": "user",
"auth": {"apiKey": creds.api_key, "secret": creds.api_secret,
"passphrase": creds.api_passphrase},
"markets": markets,
}))
try:
raw = await asyncio.wait_for(ws.recv(), timeout=4)
low = raw.lower() if isinstance(raw, str) else ""
if "auth" in low and any(w in low for w in ("fail", "error", "invalid", "unauthor")):
return False, "auth rejected by server"
return True, "connected, receiving events"
except TimeoutError:
# no message but the socket stayed open => auth accepted, just idle
return True, "connected (idle — no events yet)"
except websockets.ConnectionClosed:
return False, "connection closed (auth likely rejected)"
except Exception as e: # noqa: BLE001
return False, str(e)[:80]
async def _top_political_token(cfg: Config) -> str | None:
try:
async with httpx.AsyncClient(timeout=15) as c:
r = await c.get(f"{cfg.wallet.gamma_host}/markets",
params={"limit": 1, "closed": "false", "tag_id": 2,
"order": "volume24hr", "ascending": "false"})
toks = json.loads(r.json()[0]["clobTokenIds"])
return str(toks[0])
except (httpx.HTTPError, KeyError, IndexError, ValueError):
return None
def _extract_balance(ba: dict[str, Any]) -> float | None:
if not isinstance(ba, dict):
return None
for k in ("balance", "collateral", "amount"):
if k in ba:
try:
v = float(ba[k])
return v / 1e6 if v > 1e6 else v
except (ValueError, TypeError):
return None
return None
src/polymaker/domain.py
+185
View File
@@ -0,0 +1,185 @@
"""Core domain types shared across polymaker.
These are plain, immutable-ish dataclasses and enums with no I/O. Everything the
strategy, execution, and state layers speak is defined here so the boundaries
between components are typed rather than dict-shaped (the v1 failure mode).
"""
from __future__ import annotations
import time
from dataclasses import dataclass, field
from enum import Enum
class Side(str, Enum):
"""Order side. Values match the CLOB API's string form."""
BUY = "BUY"
SELL = "SELL"
@property
def opposite(self) -> Side:
return Side.SELL if self is Side.BUY else Side.BUY
class Regime(str, Enum):
"""Per-market quoting regime (see the README)."""
QUIET = "QUIET" # farming posture: in-band, layered, full size
TRENDING = "TRENDING" # persistent one-sided flow: lean + widen + half size
EVENT = "EVENT" # sweep/jump detected: pull quotes, cool off
REDUCE_ONLY = "REDUCE_ONLY" # inventory cap / end-date: exit quotes only
HALTED = "HALTED" # stale data / resolved / kill switch: cancel all
class OrderState(str, Enum):
"""Lifecycle of one of our orders."""
DRAFT = "DRAFT"
POSTED = "POSTED"
LIVE = "LIVE"
PARTIALLY_FILLED = "PARTIALLY_FILLED"
CANCELED = "CANCELED"
REJECTED = "REJECTED"
DONE = "DONE"
class TradeState(str, Enum):
"""Lifecycle of an on-chain match, mirroring the user-WS status ladder."""
MATCHED = "MATCHED"
MINED = "MINED"
CONFIRMED = "CONFIRMED"
RETRYING = "RETRYING"
FAILED = "FAILED"
# ── Market metadata ────────────────────────────────────────────────────────
@dataclass(frozen=True, slots=True)
class TokenMeta:
token_id: str
outcome: str # e.g. "Yes" / "No" / candidate name
@dataclass(frozen=True, slots=True)
class MarketMeta:
"""Static-ish metadata for a tradable market, sourced from Gamma/CLOB."""
condition_id: str
question: str
slug: str
tokens: tuple[TokenMeta, TokenMeta]
tick_size: float
neg_risk: bool
min_order_size: float # exchange minimum order size (shares)
# liquidity-rewards params
rewards_min_size: float
rewards_max_spread: float # in cents (e.g. 3.0 == 3c band)
rewards_daily_rate: float
# fees
maker_fee_bps: int
taker_fee_bps: int
fees_enabled: bool
# lifecycle / grouping
end_date_iso: str | None
event_id: str | None # neg-risk event group; siblings share this
# fraction of taker fees rebated to makers (V2 maker rebates)
rebate_rate: float = 0.0
# market-data references (may be stale; not authoritative for quoting)
best_bid: float = 0.0
best_ask: float = 0.0
liquidity_num: float = 0.0
volume_num: float = 0.0
@property
def yes(self) -> TokenMeta:
return self.tokens[0]
@property
def no(self) -> TokenMeta:
return self.tokens[1]
def other_token(self, token_id: str) -> str:
a, b = self.tokens
return b.token_id if token_id == a.token_id else a.token_id
@property
def price_decimals(self) -> int:
"""Number of decimal places implied by the tick size."""
s = f"{self.tick_size:f}".rstrip("0")
return len(s.split(".")[1]) if "." in s else 0
# ── Live trading state ─────────────────────────────────────────────────────
@dataclass(slots=True)
class Position:
token_id: str
size: float = 0.0 # signed shares held (long only in practice; >= 0)
avg_price: float = 0.0
@property
def is_flat(self) -> bool:
return self.size <= 0.0
@dataclass(slots=True)
class OpenOrder:
"""One of our resting orders as we currently believe it exists."""
order_id: str
token_id: str
side: Side
price: float
size: float # remaining (original - matched)
state: OrderState = OrderState.LIVE
created_ts: float = field(default_factory=time.time)
@property
def notional(self) -> float:
return self.price * self.size
@dataclass(frozen=True, slots=True)
class Fill:
token_id: str
side: Side
price: float
size: float
trade_id: str
ts: float = field(default_factory=time.time)
is_maker: bool = True
# ── Strategy output ────────────────────────────────────────────────────────
@dataclass(frozen=True, slots=True)
class Quote:
"""One intended resting order the strategy wants live."""
token_id: str
side: Side
price: float
size: float
def key(self, price_decimals: int) -> tuple[str, Side, float]:
"""Identity used to match against live orders (side + rounded price)."""
return (self.token_id, self.side, round(self.price, price_decimals))
@dataclass(frozen=True, slots=True)
class TargetQuotes:
"""The full desired resting-order set for a market at a point in time."""
condition_id: str
regime: Regime
quotes: tuple[Quote, ...] = ()
@property
def is_empty(self) -> bool:
return len(self.quotes) == 0
src/polymaker/engine.py
+367
View File
@@ -0,0 +1,367 @@
"""Engine: wires every component into a single async event loop.
Data flow per market:
market WS -> OrderBook -> (wake) -> Quoter task -> strategy (pure) -> reconcile
-> ExecutionGateway ; user WS -> StateStore ; periodic REST reconcile + heartbeat.
One lightweight quoter task per market, woken by book/fill events and debounced.
The strategy layer is pure; the engine owns all the state and I/O around it.
"""
from __future__ import annotations
import asyncio
import contextlib
import time
from datetime import datetime
from typing import Any
from polymaker.catalog.gamma import GammaClient, fetch_reward_rates, parse_market
from polymaker.catalog.store import CatalogStore
from polymaker.config import Config, StrategyProfile
from polymaker.domain import Fill, MarketMeta
from polymaker.execution.gateway import ExecutionGateway
from polymaker.execution.reconciler import reconcile
from polymaker.journal import Journal
from polymaker.logging import get_logger
from polymaker.marketdata.parse import TradePrint
from polymaker.marketdata.service import MarketDataService
from polymaker.merge import Merger
from polymaker.risk.manager import RiskManager
from polymaker.state.store import StateStore
from polymaker.state.tracker import UserEventProcessor
from polymaker.strategy.estimators import (
FlowEstimator,
MarketEstimators,
MarkoutTracker,
VolEstimator,
)
from polymaker.strategy.quoting import QuoteInputs, compute_fair_value, construct_quotes
from polymaker.strategy.regime import RegimeInputs, RegimeMachine
from polymaker.userstream.client import UserStream
log = get_logger("engine")
class Engine:
def __init__(self, cfg: Config, *, paper: bool = False) -> None:
self.cfg = cfg
self.paper = paper
self._running = False
self.journal = Journal(cfg.paths.journal_dir, enabled=cfg.engine.journal,
day="paper" if paper else "live")
self.state = StateStore(cfg.paths.db)
self.catalog = CatalogStore(cfg.paths.db)
self.gateway = ExecutionGateway(cfg, self.journal, paper=paper)
self.risk = RiskManager(cfg.risk, self.state)
self.merger = Merger(cfg)
self.md = MarketDataService(on_dirty=self._on_dirty, on_trade=self._on_trade,
journal=self.journal, proxy=cfg.proxy)
self.user_proc = UserEventProcessor(self.state, on_change=self._wake_cid,
on_fill=self._on_fill)
self.user: UserStream | None = None
# per-market state
self.metas: dict[str, MarketMeta] = {}
self.profiles: dict[str, StrategyProfile] = {}
self.est: dict[str, MarketEstimators] = {}
self.regime_m: dict[str, RegimeMachine] = {}
self._dirty: dict[str, asyncio.Event] = {}
self._sweep: dict[str, bool] = {}
self._merging: set[str] = set()
self._token_cid: dict[str, str] = {}
self._tasks: list[asyncio.Task[Any]] = []
# ── lifecycle ───────────────────────────────────────────────────────
async def start(self) -> None:
self._running = True
await self.gateway.connect()
await self._resolve_markets()
if not self.metas:
log.warning("no_markets_selected", hint="add markets to config/markets.toml, run `polymaker scan`")
await self._startup_reconcile()
# subscribe feeds
self.md.set_markets([(cid, [m.yes.token_id, m.no.token_id]) for cid, m in self.metas.items()])
self.user = UserStream(
self.gateway.creds, self.gateway.address, self.user_proc,
other_token=self._other_token, condition_of_token=self._cid_of_token,
journal=self.journal, proxy=self.cfg.proxy,
)
self.user.set_markets(list(self.metas))
# launch tasks
self._tasks.append(asyncio.create_task(self.md.run(), name="market_ws"))
if not self.paper:
self._tasks.append(asyncio.create_task(self.user.run(), name="user_ws"))
self._tasks.append(asyncio.create_task(self._heartbeat_loop(), name="heartbeat"))
self._tasks.append(asyncio.create_task(self._reconcile_loop(), name="reconcile"))
for cid in self.metas:
self._tasks.append(asyncio.create_task(self._quoter(cid), name=f"quote:{cid[:8]}"))
self.risk.reset_day()
log.info("engine_started", markets=len(self.metas), paper=self.paper)
async def run_forever(self) -> None:
await self.start()
with contextlib.suppress(asyncio.CancelledError):
await asyncio.gather(*self._tasks)
async def shutdown(self) -> None:
self._running = False
log.info("engine_shutdown")
self.md.stop()
if self.user:
self.user.stop()
for t in self._tasks:
t.cancel()
with contextlib.suppress(Exception):
await self.gateway.cancel_all()
self.journal.close()
self.state.close()
self.catalog.close()
# ── market resolution ───────────────────────────────────────────────
async def _resolve_markets(self) -> None:
reward_rates: dict[str, float] | None = None
async with GammaClient(self.cfg.wallet.gamma_host) as gamma:
for entry in self.cfg.enabled_markets:
meta = self.catalog.get_by_slug(entry.slug) if entry.slug else None
if meta is None and entry.condition_id:
meta = self.catalog.get(entry.condition_id)
if meta is None: # fall back to a live Gamma fetch
if reward_rates is None:
reward_rates = await fetch_reward_rates(self.cfg.wallet.clob_host)
meta = await self._fetch_meta(gamma, entry.slug, entry.condition_id, reward_rates)
if meta is None:
log.warning("market_unresolved", ref=entry.ref)
continue
self.metas[meta.condition_id] = meta
self.profiles[meta.condition_id] = self.cfg.profile_for(entry)
self.est[meta.condition_id] = self._make_estimators(self.profiles[meta.condition_id])
self.regime_m[meta.condition_id] = RegimeMachine()
self._dirty[meta.condition_id] = asyncio.Event()
for tok in (meta.yes.token_id, meta.no.token_id):
self._token_cid[tok] = meta.condition_id
async def _fetch_meta(
self, gamma: GammaClient, slug: str | None, condition_id: str | None,
reward_rates: dict[str, float],
) -> MarketMeta | None:
tag_id = self.catalog.cached_tag("politics")
async for raw in gamma.iter_markets(tag_id=tag_id, max_pages=25):
if (slug and raw.get("slug") == slug) or (condition_id and raw.get("conditionId") == condition_id):
m = parse_market(raw, reward_rates)
if m:
self.catalog.upsert_market(m)
return m
return None
@staticmethod
def _make_estimators(p: StrategyProfile) -> MarketEstimators:
return MarketEstimators(
vol=VolEstimator(p.vol_short_halflife_s, p.vol_long_halflife_s),
flow=FlowEstimator(p.flow_ewma_halflife_s),
markout=MarkoutTracker(),
)
async def _startup_reconcile(self) -> None:
with contextlib.suppress(Exception):
await self.gateway.cancel_all() # clean slate; heartbeat covers crashes
positions = await self.gateway.positions()
if positions:
self.state.reconcile_positions(positions)
log.info("startup_positions", n=len(positions))
# ── callbacks ───────────────────────────────────────────────────────
def _on_dirty(self, condition_id: str, token_id: str) -> None:
ev = self._dirty.get(condition_id)
if ev is not None:
ev.set()
def _wake_cid(self, condition_id: str) -> None:
ev = self._dirty.get(condition_id)
if ev is not None:
ev.set()
def _on_trade(self, tp: TradePrint) -> None:
cid = self._token_cid.get(tp.asset_id)
if cid is None:
return
self.est[cid].flow.update(tp.aggressor, tp.size, tp.ts)
# crude sweep flag: a single print larger than 3x base size
base = self.profiles[cid].base_size_usdc / max(tp.price, 0.01)
if tp.size >= 3 * base:
self._sweep[cid] = True
def _on_fill(self, fill: Fill) -> None:
self.risk.note_fill(fill)
cid = self._token_cid.get(fill.token_id)
if cid is None:
return
est = self.est[cid]
fv = est.last_fv if est.last_fv is not None else fill.price
token_fv = fv if fill.token_id == self.metas[cid].yes.token_id else (1.0 - fv)
est.markout.record_fill(fill.side, token_fv, fill.ts)
# ── quoter ──────────────────────────────────────────────────────────
async def _quoter(self, cid: str) -> None:
debounce = self.cfg.engine.debounce_ms / 1000.0
ev = self._dirty[cid]
while self._running:
try:
await ev.wait()
await asyncio.sleep(debounce) # coalesce a burst of book updates
ev.clear()
await self._recompute(cid)
except asyncio.CancelledError:
break
except Exception as exc: # noqa: BLE001
log.error("quoter_error", cid=cid[:8], err=str(exc))
await asyncio.sleep(0.5)
async def _recompute(self, cid: str) -> None:
meta = self.metas[cid]
p = self.profiles[cid]
yes_book = self.md.book(meta.yes.token_id)
no_book = self.md.book(meta.no.token_id)
if yes_book is None or yes_book.is_empty:
return
now = time.time()
micro = yes_book.microprice(p.micro_levels)
if micro is None:
return
est = self.est[cid]
est.flow.decay_to(now)
fv = compute_fair_value(micro, est.flow.z, meta.tick_size)
prev_fv = est.last_fv
est.on_fair_value(fv, now)
self.risk.update_mark(meta.yes.token_id, fv)
self.risk.update_mark(meta.no.token_id, 1.0 - fv)
pos_yes = self.state.position(meta.yes.token_id)
pos_no = self.state.position(meta.no.token_id)
q_max = p.q_max_usdc
inv_util = abs(pos_yes.size - pos_no.size) * fv / q_max if q_max > 0 else 0.0
hours_to_end = _hours_to_end(meta.end_date_iso, now)
ws_stale = (now - self.md.last_update_ts(meta.yes.token_id)) > self.cfg.risk.ws_stale_halt_s
rd = self.risk.evaluate(meta, ws_stale=ws_stale,
event_group_cost=self._event_group_cost(meta))
regime = self.regime_m[cid].decide(
RegimeInputs(
now=now, tick=meta.tick_size, fv=fv, prev_fv=prev_fv,
vol_ratio=est.vol.ratio, flow_z=est.flow.z, inventory_util=inv_util,
hours_to_end=hours_to_end, sweep_flagged=self._sweep.pop(cid, False),
ws_stale=ws_stale, risk_halt=rd.halt, risk_reduce_only=rd.reduce_only,
),
p,
)
tq = construct_quotes(QuoteInputs(
meta=meta, regime=regime, fv=fv, vol_short=est.vol.short,
toxicity=est.markout.toxicity, yes_view=yes_book.view(),
no_view=(no_book.view() if no_book else _empty_view()),
pos_yes=pos_yes, pos_no=pos_no, profile=p, now=now,
risk_size_scale=rd.size_scale,
))
live = self.state.orders_for(meta.yes.token_id) + self.state.orders_for(meta.no.token_id)
plan = reconcile(tq, live, tick=meta.tick_size,
reprice_ticks=p.reprice_ticks, resize_frac=p.resize_frac)
if plan.is_noop:
self._maybe_merge(cid, meta, p, pos_yes.size, pos_no.size)
return
if plan.to_cancel:
await self.gateway.cancel(plan.to_cancel)
for oid in plan.to_cancel:
self.state.remove_order(oid)
if plan.to_place:
placed = await self.gateway.place(plan.to_place, meta)
self.risk.note_order_result(bool(placed) or not plan.to_place)
for o in placed:
self.state.upsert_order(o)
log.info("requote", cid=cid[:8], regime=regime.value, fv=round(fv, 4),
place=len(plan.to_place), cancel=len(plan.to_cancel),
pos_yes=round(pos_yes.size, 1), pos_no=round(pos_no.size, 1))
self._maybe_merge(cid, meta, p, pos_yes.size, pos_no.size)
def _maybe_merge(self, cid: str, meta: MarketMeta, p: StrategyProfile,
yes_size: float, no_size: float) -> None:
amount = min(yes_size, no_size)
if amount < p.merge_min_size or cid in self._merging or self.paper:
return
self._merging.add(cid)
self._tasks.append(asyncio.create_task(self._merge_task(cid, meta, amount)))
async def _merge_task(self, cid: str, meta: MarketMeta, amount: float) -> None:
try:
raw = int(amount * 1e6)
await asyncio.to_thread(self.merger.merge, meta.condition_id, raw, meta.neg_risk)
finally:
self._merging.discard(cid)
# ── background loops ────────────────────────────────────────────────
async def _heartbeat_loop(self) -> None:
if not self.cfg.engine.heartbeat:
return
while self._running:
await self.gateway.heartbeat()
await asyncio.sleep(self.cfg.engine.heartbeat_interval_s)
async def _reconcile_loop(self) -> None:
while self._running:
await asyncio.sleep(self.cfg.engine.reconcile_interval_s)
try:
positions = await self.gateway.positions()
if positions:
self.state.reconcile_positions(positions)
live = await self.gateway.open_orders()
if live or not self.paper:
by_token: dict[str, list[Any]] = {}
for o in live:
by_token.setdefault(o.token_id, []).append(o)
for tok, orders in by_token.items():
if self.state.inflight(tok) == 0:
self.state.replace_open_orders(tok, orders)
except Exception as exc: # noqa: BLE001
log.warning("reconcile_error", err=str(exc))
# ── helpers ─────────────────────────────────────────────────────────
def _other_token(self, token_id: str) -> str | None:
cid = self._token_cid.get(token_id)
return self.metas[cid].other_token(token_id) if cid else None
def _cid_of_token(self, token_id: str) -> str | None:
return self._token_cid.get(token_id)
def _event_group_cost(self, meta: MarketMeta) -> float:
if not meta.event_id:
return 0.0
cost = 0.0
for m in self.metas.values():
if m.event_id == meta.event_id:
for tok in (m.yes.token_id, m.no.token_id):
pos = self.state.position(tok)
cost += pos.size * pos.avg_price
return cost
def _hours_to_end(end_date_iso: str | None, now: float) -> float | None:
if not end_date_iso:
return None
try:
dt = datetime.fromisoformat(end_date_iso.replace("Z", "+00:00"))
return max(0.0, (dt.timestamp() - now) / 3600.0)
except (ValueError, TypeError):
return None
def _empty_view() -> Any:
from polymaker.marketdata.orderbook import BookView
return BookView(None, 0.0, None, 0.0, None, None, 0.0, 0.0)
poly_stats/__init__.py → src/polymaker/execution/__init__.py
View File
src/polymaker/execution/gateway.py
+287
View File
@@ -0,0 +1,287 @@
"""ExecutionGateway: the only component that sends actions to the CLOB.
Wraps the synchronous py-clob-client-v2 (which owns the hard V2 EIP-712 signing,
pUSD balance adjustment, and tick/fee caching) and offloads its blocking network
calls to a thread pool so the asyncio hot path never stalls. Every quote goes out
**post-only** (the maker-only mandate, enforced at the exchange).
A `paper=True` gateway shares the same path but fabricates order ids instead of
posting — so paper mode exercises the full pipeline.
"""
from __future__ import annotations
import asyncio
import itertools
import time
from dataclasses import asdict
from typing import Any
import httpx
from polymaker.config import Config
from polymaker.domain import MarketMeta, OpenOrder, OrderState, Quote, Side
from polymaker.execution.ratelimit import TokenBucket
from polymaker.journal import Journal
from polymaker.logging import get_logger
log = get_logger("execution.gateway")
def _tick_str(tick: float) -> str:
return f"{tick:g}"
class ExecutionGateway:
def __init__(
self,
cfg: Config,
journal: Journal | None = None,
*,
paper: bool = False,
) -> None:
self._cfg = cfg
self._paper = paper
self._journal = journal
self._client: Any = None # py_clob_client_v2.ClobClient
self._creds: Any = None
self._address: str = "" # signer EOA
self._funder: str = "" # funds/positions live here (proxy/deposit wallet)
self._data_host = cfg.wallet.data_api_host
# rate budgets: fraction of documented POST/DELETE ceilings (per second)
f = cfg.execution.rate_budget_fraction
self._order_bucket = TokenBucket(rate_per_s=200.0 * f, burst=500.0 * f)
self._cancel_bucket = TokenBucket(rate_per_s=200.0 * f, burst=500.0 * f)
self._paper_ids = itertools.count(1)
@property
def paper(self) -> bool:
return self._paper
@property
def creds(self) -> Any:
return self._creds
@property
def address(self) -> str:
"""The signing EOA address."""
return self._address
@property
def funder(self) -> str:
"""The address holding funds/positions (proxy/deposit wallet, or the EOA)."""
return self._funder or self._address
# ── lifecycle ───────────────────────────────────────────────────────
async def connect(self) -> None:
"""Build the client and derive L2 API creds (network). No-op fields in paper."""
sec = self._cfg.secrets
if self._paper and not sec.has_wallet:
# paper mode runs the full pipeline without a wallet (no orders posted)
self._address = sec.browser_address or "0xPAPER"
self._funder = sec.browser_address or self._address
log.info("gateway_connected", address=self._address[:10], paper=True)
return
if not sec.has_wallet:
raise RuntimeError("no wallet configured (set PK and BROWSER_ADDRESS in .env)")
def _build() -> tuple[Any, Any, str]:
from py_clob_client_v2.client import ClobClient
client = ClobClient(
host=self._cfg.wallet.clob_host,
chain_id=self._cfg.wallet.chain_id,
key=sec.pk,
signature_type=self._cfg.wallet.signature_type,
funder=sec.browser_address,
)
creds = client.create_or_derive_api_key()
client.set_api_creds(creds)
return client, creds, client.get_address()
self._client, self._creds, self._address = await asyncio.to_thread(_build)
# funds/positions live on the funder (proxy/deposit wallet); fall back to EOA
self._funder = sec.browser_address or self._address
log.info("gateway_connected", signer=self._address[:10], funder=self._funder[:10],
paper=self._paper)
# ── placement ───────────────────────────────────────────────────────
async def place(self, quotes: list[Quote], meta: MarketMeta) -> list[OpenOrder]:
if not quotes:
return []
await self._order_bucket.acquire(len(quotes))
ts = time.time()
self._journal_write("orders_out", [asdict(q) for q in quotes], ts)
if self._paper:
return [self._paper_order(q) for q in quotes]
def _place() -> list[OpenOrder]:
from py_clob_client_v2.clob_types import (
OrderArgsV2,
OrderType,
PartialCreateOrderOptions,
PostOrdersV2Args,
)
opts = PartialCreateOrderOptions(tick_size=_tick_str(meta.tick_size), neg_risk=meta.neg_risk)
args = []
for q in quotes:
signed = self._client.create_order(
OrderArgsV2(token_id=q.token_id, price=q.price, size=q.size, side=q.side.value),
options=opts,
)
args.append(PostOrdersV2Args(order=signed, orderType=OrderType.GTC))
resp = self._client.post_orders(args, post_only=self._cfg.execution.post_only)
return self._parse_place_response(resp, quotes)
try:
return await asyncio.to_thread(_place)
except Exception as exc: # noqa: BLE001 - surface + continue; engine handles error rate
log.error("place_failed", err=str(exc), n=len(quotes))
return []
def _paper_order(self, q: Quote) -> OpenOrder:
oid = f"paper-{next(self._paper_ids)}"
return OpenOrder(oid, q.token_id, q.side, q.price, q.size, OrderState.LIVE)
def _parse_place_response(self, resp: Any, quotes: list[Quote]) -> list[OpenOrder]:
"""Map a batch post response to OpenOrders. Tolerant of shape variants;
the user-WS order events + REST snapshot reconcile anything we miss."""
items = resp if isinstance(resp, list) else resp.get("orders", resp.get("data", []))
out: list[OpenOrder] = []
for q, item in zip(quotes, items if isinstance(items, list) else [], strict=False):
oid = _first(item, "orderID", "orderId", "order_id", "id", "hash")
if not oid:
log.warning("place_response_missing_id", item=str(item)[:120])
continue
out.append(OpenOrder(str(oid), q.token_id, q.side, q.price, q.size, OrderState.LIVE))
return out
# ── cancellation ────────────────────────────────────────────────────
async def cancel(self, order_ids: list[str]) -> None:
if not order_ids or self._paper:
return
await self._cancel_bucket.acquire(1)
def _cancel() -> None:
self._client.cancel_orders(order_ids)
try:
await asyncio.to_thread(_cancel)
except Exception as exc: # noqa: BLE001
log.error("cancel_failed", err=str(exc), n=len(order_ids))
async def cancel_asset(self, asset_id: str) -> None:
if self._paper:
return
def _cancel() -> None:
from py_clob_client_v2.clob_types import OrderMarketCancelParams
self._client.cancel_market_orders(OrderMarketCancelParams(asset_id=asset_id))
await asyncio.to_thread(_cancel)
async def cancel_all(self) -> None:
if self._paper or self._client is None:
return
await asyncio.to_thread(self._client.cancel_all)
log.info("cancel_all_sent")
# ── heartbeat (dead-man switch) ─────────────────────────────────────
async def heartbeat(self, hb_id: str = "") -> None:
if self._paper or self._client is None:
return
try:
await asyncio.to_thread(self._client.post_heartbeat, hb_id)
except Exception as exc: # noqa: BLE001
log.warning("heartbeat_failed", err=str(exc))
# ── reads ───────────────────────────────────────────────────────────
async def open_orders(self) -> list[OpenOrder]:
if self._paper or self._client is None:
return []
def _get() -> list[OpenOrder]:
raw = self._client.get_open_orders()
rows = raw if isinstance(raw, list) else raw.get("data", raw.get("orders", []))
out = []
for r in rows:
try:
side = Side(str(r["side"]).upper())
remaining = float(r.get("original_size", r.get("size", 0))) - float(
r.get("size_matched", 0)
)
out.append(
OpenOrder(
str(_first(r, "id", "orderID", "order_id")),
str(r["asset_id"]),
side,
float(r["price"]),
remaining,
OrderState.LIVE,
)
)
except (KeyError, ValueError, TypeError):
continue
return out
try:
return await asyncio.to_thread(_get)
except Exception as exc: # noqa: BLE001
log.warning("open_orders_failed", err=str(exc))
return []
async def positions(self) -> dict[str, tuple[float, float]]:
"""{token_id: (size, avg_price)} from the data API (reconcile use).
Queries the FUNDER (where positions live), not the signer EOA.
"""
user = self.funder
if not user or not user.startswith("0x") or user == "0xPAPER":
return {}
try:
async with httpx.AsyncClient(timeout=15.0) as c:
r = await c.get(f"{self._data_host}/positions", params={"user": user})
r.raise_for_status()
return {
str(p["asset"]): (float(p["size"]), float(p.get("avgPrice", 0)))
for p in r.json()
if float(p.get("size", 0)) > 0
}
except (httpx.HTTPError, KeyError, ValueError) as exc:
log.warning("positions_failed", err=str(exc))
return {}
async def balance_allowance(self) -> dict[str, Any]:
"""Collateral balance/allowance snapshot (for `doctor`)."""
if self._client is None:
return {}
def _get() -> dict[str, Any]:
from py_clob_client_v2.clob_types import AssetType, BalanceAllowanceParams
result: dict[str, Any] = self._client.get_balance_allowance(
BalanceAllowanceParams(asset_type=AssetType.COLLATERAL)
)
return result
try:
return await asyncio.to_thread(_get)
except Exception as exc: # noqa: BLE001
log.warning("balance_allowance_failed", err=str(exc))
return {}
def _journal_write(self, kind: str, payload: Any, ts: float) -> None:
if self._journal is not None:
self._journal.write(kind, payload, ts)
def _first(d: Any, *keys: str) -> Any:
if not isinstance(d, dict):
return None
for k in keys:
if k in d and d[k]:
return d[k]
return None
src/polymaker/execution/ratelimit.py
+42
View File
@@ -0,0 +1,42 @@
"""Async token-bucket rate budgeter.
The CLOB API throttles (queues) excess requests rather than 429-ing, so the real
risk is silent latency injection when the market is moving. We self-limit to a
fraction of the documented ceilings and expose pressure as a signal so the
engine can shed low-edge reprices first.
"""
from __future__ import annotations
import asyncio
import time
class TokenBucket:
def __init__(self, rate_per_s: float, burst: float | None = None) -> None:
self.rate = max(rate_per_s, 0.001)
self.capacity = burst if burst is not None else max(1.0, rate_per_s)
self._tokens = self.capacity
self._last = time.monotonic()
self._lock = asyncio.Lock()
def _refill(self) -> None:
now = time.monotonic()
self._tokens = min(self.capacity, self._tokens + (now - self._last) * self.rate)
self._last = now
async def acquire(self, n: float = 1.0) -> None:
async with self._lock:
while True:
self._refill()
if self._tokens >= n:
self._tokens -= n
return
deficit = n - self._tokens
await asyncio.sleep(deficit / self.rate)
@property
def pressure(self) -> float:
"""0 = plenty of budget, 1 = empty (callers about to wait)."""
self._refill()
return 1.0 - min(1.0, self._tokens / self.capacity)
src/polymaker/execution/reconciler.py
+64
View File
@@ -0,0 +1,64 @@
"""Pure reconciliation: desired TargetQuotes vs live orders -> minimal actions.
The strategy emits a target quote set; this computes the smallest cancel/place
set to reach it, applying churn tolerances so we don't burn queue position for
sub-tick or sub-threshold size changes (v1's should_cancel instinct, generalized).
No I/O — the gateway executes the returned plan.
"""
from __future__ import annotations
from collections import defaultdict
from dataclasses import dataclass, field
from polymaker.domain import OpenOrder, Quote, TargetQuotes
_EPS = 1e-9
@dataclass(frozen=True, slots=True)
class ReconcilePlan:
to_cancel: list[str] = field(default_factory=list) # order ids
to_place: list[Quote] = field(default_factory=list)
@property
def is_noop(self) -> bool:
return not self.to_cancel and not self.to_place
def reconcile(
targets: TargetQuotes,
live: list[OpenOrder],
*,
tick: float,
reprice_ticks: int,
resize_frac: float,
) -> ReconcilePlan:
"""Diff targets against live orders. Keep live orders that already satisfy a
target within tolerance; cancel the rest; place targets with no match."""
live_by_key: dict[tuple[str, str], list[OpenOrder]] = defaultdict(list)
for o in live:
live_by_key[(o.token_id, o.side.value)].append(o)
keep: set[str] = set()
to_place: list[Quote] = []
price_tol = reprice_ticks * tick + _EPS
for q in targets.quotes:
candidates = live_by_key.get((q.token_id, q.side.value), [])
match: OpenOrder | None = None
for o in candidates:
if o.order_id in keep:
continue
price_close = abs(o.price - q.price) <= price_tol
size_close = q.size <= 0 or abs(o.size - q.size) <= resize_frac * q.size + _EPS
if price_close and size_close:
match = o
break
if match is not None:
keep.add(match.order_id)
else:
to_place.append(q)
to_cancel = [o.order_id for o in live if o.order_id not in keep]
return ReconcilePlan(to_cancel=to_cancel, to_place=to_place)
src/polymaker/journal.py
+32
View File
@@ -0,0 +1,32 @@
"""Append-only JSONL event journal.
Captures raw WS-in and orders-out so the replay backtester (see the README) can
reconstruct books and re-run the strategy. Also the substrate for post-mortems.
Cheap: one line per event, flushed, rotated by day.
"""
from __future__ import annotations
import json
from pathlib import Path
from typing import Any
class Journal:
def __init__(self, directory: str | Path, *, enabled: bool = True, day: str = "live") -> None:
self.enabled = enabled
self._fh = None
if enabled:
d = Path(directory)
d.mkdir(parents=True, exist_ok=True)
self._fh = (d / f"{day}.jsonl").open("a", buffering=1)
def write(self, kind: str, payload: Any, ts: float) -> None:
if not self.enabled or self._fh is None:
return
self._fh.write(json.dumps({"ts": ts, "kind": kind, "data": payload}, default=str) + "\n")
def close(self) -> None:
if self._fh is not None:
self._fh.close()
self._fh = None
src/polymaker/livetest.py
+94
View File
@@ -0,0 +1,94 @@
"""Live wallet round-trip test — the Phase-2 wallet spike (see the README).
Proves the full V2 order path against the real exchange with minimal risk:
places ONE post-only BUY well below the touch (so it rests and cannot fill),
confirms it appears in open orders, then cancels it. Post-only guarantees it
never takes; the deep price + immediate cancel means ~zero economic risk.
This is where the known py-clob-client-v2 signature-type-2 (Safe/proxy) issues
would surface — the command reports each step so failures are diagnosable.
"""
from __future__ import annotations
import asyncio
from rich.console import Console
from polymaker.config import Config
from polymaker.domain import Quote, Side
from polymaker.execution.gateway import ExecutionGateway
async def run_livetest(cfg: Config, console: Console, notional_usdc: float = 5.0) -> bool:
from polymaker.catalog.store import CatalogStore
if not cfg.secrets.has_wallet:
console.print("[red]No wallet in .env. Set PK and BROWSER_ADDRESS first.[/red]")
return False
# pick a liquid market from the catalog (or fall back to a live scan)
store = CatalogStore(cfg.paths.db)
rows = store.top(20)
store.close()
if not rows:
console.print("[yellow]Catalog empty — run `polymaker scan` first.[/yellow]")
return False
# choose a market with a mid comfortably in (0.15, 0.85) so a deep bid is valid
meta = None
for m, _sc in rows:
mid = (m.best_bid + m.best_ask) / 2 if (m.best_bid and m.best_ask) else 0.0
if 0.15 < mid < 0.85:
meta = m
break
meta = meta or rows[0][0]
console.print(f"[bold]Live round-trip test[/bold] on: {meta.question[:60]}")
gw = ExecutionGateway(cfg)
try:
await gw.connect()
console.print(f" [green]✓[/green] wallet auth — address {gw.address[:12]}…")
except Exception as e: # noqa: BLE001
console.print(f" [red]✗ wallet auth failed:[/red] {e}")
console.print(" [yellow]Auth/signature-type mismatch (see the README). If your account has a "
"'deposit address', set signature_type=3 (POLY_1271) in config.toml and use the "
"deposit address as BROWSER_ADDRESS. Errors like 'maker address not allowed, use "
"the deposit wallet flow' or 'signer must be the API key address' mean the type is "
"wrong for this wallet.[/yellow]")
return False
ba = await gw.balance_allowance()
console.print(f" balance/allowance: {ba}")
# a deep resting price: well below best bid, snapped to tick, floored at 2 ticks
tick = meta.tick_size
best_bid = meta.best_bid or 0.30
price = max(2 * tick, round((best_bid - 0.10) / tick) * tick)
size = round(max(meta.min_order_size, notional_usdc / price), 2)
console.print(f" placing post-only BUY {size} @ {price} on YES token "
f"(~${price * size:.2f}, deep — will not fill)")
placed = await gw.place([Quote(meta.yes.token_id, Side.BUY, price, size)], meta)
if not placed:
console.print(" [red]✗ order not placed (see logs for the API error)[/red]")
return False
oid = placed[0].order_id
console.print(f" [green]✓[/green] placed — order id {oid[:16]}…")
await asyncio.sleep(2.0)
live = await gw.open_orders()
found = any(o.order_id == oid for o in live)
console.print(f" [{'green' if found else 'yellow'}]{'✓' if found else '?'}[/] "
f"read back open orders: {len(live)} live, ours {'present' if found else 'not seen yet'}")
await gw.cancel([oid])
console.print(" [green]✓[/green] cancel sent")
await asyncio.sleep(1.5)
after = await gw.open_orders()
still = any(o.order_id == oid for o in after)
console.print(f" [{'green' if not still else 'red'}]{'✓' if not still else '✗'}[/] "
f"order {'cancelled' if not still else 'STILL LIVE — cancel manually!'}")
ok = bool(placed) and not still
console.print(f"\n[bold]{'ROUND-TRIP OK' if ok else 'CHECK LOGS'}[/bold]")
return ok
src/polymaker/logging.py
+66
View File
@@ -0,0 +1,66 @@
"""structlog configuration: human console in dev, JSON to file in prod."""
from __future__ import annotations
import logging
import sys
from pathlib import Path
from typing import Any
import structlog
def configure(
*,
level: str = "INFO",
json_file: Path | None = None,
console: bool = True,
) -> None:
"""Set up structlog + stdlib logging once at process start."""
shared: list[Any] = [
structlog.contextvars.merge_contextvars,
structlog.processors.add_log_level,
structlog.processors.TimeStamper(fmt="iso", utc=True),
structlog.processors.StackInfoRenderer(),
structlog.processors.format_exc_info,
]
structlog.configure(
processors=[*shared, structlog.stdlib.ProcessorFormatter.wrap_for_formatter],
logger_factory=structlog.stdlib.LoggerFactory(),
wrapper_class=structlog.stdlib.BoundLogger,
cache_logger_on_first_use=True,
)
root = logging.getLogger()
root.handlers.clear()
root.setLevel(level)
if console:
ch = logging.StreamHandler(sys.stderr)
ch.setFormatter(
structlog.stdlib.ProcessorFormatter(
processors=[
structlog.stdlib.ProcessorFormatter.remove_processors_meta,
structlog.dev.ConsoleRenderer(colors=sys.stderr.isatty()),
]
)
)
root.addHandler(ch)
if json_file is not None:
json_file.parent.mkdir(parents=True, exist_ok=True)
fh = logging.FileHandler(json_file)
fh.setFormatter(
structlog.stdlib.ProcessorFormatter(
processors=[
structlog.stdlib.ProcessorFormatter.remove_processors_meta,
structlog.processors.JSONRenderer(),
]
)
)
root.addHandler(fh)
def get_logger(name: str) -> structlog.stdlib.BoundLogger:
return structlog.get_logger(name) # type: ignore[no-any-return]
poly_utils/__init__.py → src/polymaker/marketdata/__init__.py
View File
src/polymaker/marketdata/orderbook.py
+209
View File
@@ -0,0 +1,209 @@
"""Order book maintenance and analytics for a single market.
We keep the YES-token book canonical (bids/asks as SortedDicts keyed by price).
The NO-token view is derived by the identity no_price = 1 - yes_price, with
bids/asks swapped — so we only ever maintain one book per market.
All methods are synchronous and side-effect-free reads except the explicit
apply_* mutators. Nothing here does I/O; the WS layer drives it.
"""
from __future__ import annotations
from dataclasses import dataclass
from sortedcontainers import SortedDict
from polymaker.domain import Side
@dataclass(frozen=True, slots=True)
class BookLevel:
price: float
size: float
@dataclass(frozen=True, slots=True)
class BookView:
"""A resolved best/second/depth snapshot for one outcome token."""
best_bid: float | None
best_bid_size: float
best_ask: float | None
best_ask_size: float
second_bid: float | None
second_ask: float | None
bid_depth: float # summed size within band, bid side
ask_depth: float # summed size within band, ask side
@property
def mid(self) -> float | None:
if self.best_bid is None or self.best_ask is None:
return None
return (self.best_bid + self.best_ask) / 2.0
@property
def spread(self) -> float | None:
if self.best_bid is None or self.best_ask is None:
return None
return self.best_ask - self.best_bid
@property
def imbalance(self) -> float:
"""(bid_depth - ask_depth) / total in [-1, 1]; 0 if empty."""
total = self.bid_depth + self.ask_depth
return (self.bid_depth - self.ask_depth) / total if total > 0 else 0.0
class OrderBook:
"""YES-canonical L2 book for one market."""
__slots__ = ("bids", "asks", "tick_size", "last_update_ts", "book_hash")
def __init__(self, tick_size: float = 0.001) -> None:
# price -> size. bids and asks both ascending in price.
self.bids: SortedDict[float, float] = SortedDict()
self.asks: SortedDict[float, float] = SortedDict()
self.tick_size = tick_size
self.last_update_ts: float = 0.0
self.book_hash: str | None = None
# ── mutation ────────────────────────────────────────────────────────
def apply_snapshot(
self,
bids: list[tuple[float, float]],
asks: list[tuple[float, float]],
ts: float,
book_hash: str | None = None,
) -> None:
self.bids = SortedDict({p: s for p, s in bids if s > 0})
self.asks = SortedDict({p: s for p, s in asks if s > 0})
self.last_update_ts = ts
self.book_hash = book_hash
def apply_delta(self, side: Side, price: float, size: float, ts: float) -> None:
book = self.bids if side is Side.BUY else self.asks
if size <= 0:
book.pop(price, None)
else:
book[price] = size
self.last_update_ts = ts
def set_tick_size(self, tick_size: float) -> None:
self.tick_size = tick_size
@property
def is_empty(self) -> bool:
return len(self.bids) == 0 or len(self.asks) == 0
# ── raw best (YES side) ─────────────────────────────────────────────
def best_bid(self) -> BookLevel | None:
if not self.bids:
return None
p = self.bids.peekitem(-1) # highest bid
return BookLevel(p[0], p[1])
def best_ask(self) -> BookLevel | None:
if not self.asks:
return None
p = self.asks.peekitem(0) # lowest ask
return BookLevel(p[0], p[1])
# ── analytics ───────────────────────────────────────────────────────
def microprice(self, levels: int = 3) -> float | None:
"""Depth-weighted mid over the top `levels`, pulled toward the thin side.
Uses size at the opposite side as the weight for each price (standard
microprice intuition: price is dragged toward the side with less size).
Returns None if either side is empty.
"""
bb = self.best_bid()
ba = self.best_ask()
if bb is None or ba is None:
return None
bid_sz = self._top_size(self.bids, levels, from_high=True)
ask_sz = self._top_size(self.asks, levels, from_high=False)
total = bid_sz + ask_sz
if total <= 0:
return (bb.price + ba.price) / 2.0
# weight best_ask by bid size and best_bid by ask size
return (ba.price * bid_sz + bb.price * ask_sz) / total
def best_with_min_size(
self, side: Side, min_size: float
) -> tuple[float | None, float, float | None]:
"""First level (from the touch) with size > min_size.
Returns (price, size, top_price) where top_price is the actual touch
(used to detect dust at the front). Mirrors v1's find_best_price_with_size
but without the second-best bookkeeping the new strategy doesn't need.
"""
if side is Side.BUY:
items = reversed(self.bids.items()) # high -> low
else:
items = iter(self.asks.items()) # low -> high
top_price: float | None = None
for price, size in items:
if top_price is None:
top_price = price
if size > min_size:
return price, size, top_price
return None, 0.0, top_price
def depth_within(self, side: Side, lo: float, hi: float) -> float:
"""Sum of sizes with price in [lo, hi] on the given side."""
book = self.bids if side is Side.BUY else self.asks
# SortedDict.irange gives keys in [lo, hi]
return float(sum(book[p] for p in book.irange(lo, hi)))
def view(self, band_frac: float = 0.05, min_size: float = 0.0) -> BookView:
"""Resolved YES-side view with best/second and in-band depth."""
bb = self._nth_bid(0, min_size)
ba = self._nth_ask(0, min_size)
sb = self._nth_bid(1, min_size)
sa = self._nth_ask(1, min_size)
mid = None
bid_depth = ask_depth = 0.0
if bb is not None and ba is not None:
mid = (bb.price + ba.price) / 2.0
bid_depth = self.depth_within(Side.BUY, bb.price, mid * (1 + band_frac))
ask_depth = self.depth_within(Side.SELL, mid * (1 - band_frac), ba.price)
return BookView(
best_bid=bb.price if bb else None,
best_bid_size=bb.size if bb else 0.0,
best_ask=ba.price if ba else None,
best_ask_size=ba.size if ba else 0.0,
second_bid=sb.price if sb else None,
second_ask=sa.price if sa else None,
bid_depth=bid_depth,
ask_depth=ask_depth,
)
# ── internals ───────────────────────────────────────────────────────
def _nth_bid(self, n: int, min_size: float) -> BookLevel | None:
count = 0
for price in reversed(self.bids):
if self.bids[price] > min_size:
if count == n:
return BookLevel(price, self.bids[price])
count += 1
return None
def _nth_ask(self, n: int, min_size: float) -> BookLevel | None:
count = 0
for price in self.asks:
if self.asks[price] > min_size:
if count == n:
return BookLevel(price, self.asks[price])
count += 1
return None
@staticmethod
def _top_size(book: SortedDict[float, float], levels: int, *, from_high: bool) -> float:
keys = list(reversed(book)) if from_high else list(book)
return float(sum(book[k] for k in keys[:levels]))
def to_no_price(yes_price: float) -> float:
"""Convert a YES price to the equivalent NO price."""
return 1.0 - yes_price
src/polymaker/marketdata/parse.py
+136
View File
@@ -0,0 +1,136 @@
"""Pure parsers for market-WS wire messages -> structured updates.
Kept separate from the socket so they're unit-testable against captured frames.
Verified against live frames on 2026-07-05 (the README):
book: {market, asset_id, bids:[{price,size}], asks:[...], timestamp, hash, tick_size}
price_change:{market, timestamp, price_changes:[{asset_id, price, size, side, hash}]}
last_trade_price:{market, asset_id, price, size, side, timestamp, fee_rate_bps}
tick_size_change:{market, asset_id, old_tick_size, new_tick_size}
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Any
from polymaker.domain import Side
@dataclass(frozen=True, slots=True)
class BookUpdate:
asset_id: str
condition_id: str
bids: list[tuple[float, float]]
asks: list[tuple[float, float]]
ts: float
book_hash: str | None
tick_size: float | None
@dataclass(frozen=True, slots=True)
class PriceChange:
asset_id: str
condition_id: str
side: Side # BUY -> bid side, SELL -> ask side
price: float
size: float
ts: float
@dataclass(frozen=True, slots=True)
class TradePrint:
asset_id: str
condition_id: str
aggressor: Side
price: float
size: float
ts: float
@dataclass(frozen=True, slots=True)
class TickSizeChange:
asset_id: str
tick_size: float
def _ts(msg: dict[str, Any]) -> float:
raw = msg.get("timestamp")
if raw is None:
return 0.0
try:
v = float(raw)
return v / 1000.0 if v > 1e12 else v # ms -> s
except (ValueError, TypeError):
return 0.0
def _levels(items: Any) -> list[tuple[float, float]]:
out: list[tuple[float, float]] = []
for it in items or []:
try:
out.append((float(it["price"]), float(it["size"])))
except (KeyError, ValueError, TypeError):
continue
return out
def parse_book(msg: dict[str, Any]) -> BookUpdate | None:
try:
tick = msg.get("tick_size")
return BookUpdate(
asset_id=str(msg["asset_id"]),
condition_id=str(msg.get("market", "")),
bids=_levels(msg.get("bids")),
asks=_levels(msg.get("asks")),
ts=_ts(msg),
book_hash=msg.get("hash"),
tick_size=float(tick) if tick is not None else None,
)
except (KeyError, ValueError, TypeError):
return None
def parse_price_changes(msg: dict[str, Any]) -> list[PriceChange]:
out: list[PriceChange] = []
ts = _ts(msg)
cond = str(msg.get("market", ""))
for ch in msg.get("price_changes", []) or []:
try:
out.append(
PriceChange(
asset_id=str(ch["asset_id"]),
condition_id=cond,
side=Side(str(ch["side"]).upper()),
price=float(ch["price"]),
size=float(ch["size"]),
ts=ts,
)
)
except (KeyError, ValueError, TypeError):
continue
return out
def parse_last_trade(msg: dict[str, Any]) -> TradePrint | None:
try:
return TradePrint(
asset_id=str(msg["asset_id"]),
condition_id=str(msg.get("market", "")),
aggressor=Side(str(msg.get("side", "BUY")).upper()),
price=float(msg["price"]),
size=float(msg["size"]),
ts=_ts(msg),
)
except (KeyError, ValueError, TypeError):
return None
def parse_tick_size_change(msg: dict[str, Any]) -> TickSizeChange | None:
try:
tick = msg.get("new_tick_size", msg.get("tick_size"))
if tick is None:
return None
return TickSizeChange(asset_id=str(msg["asset_id"]), tick_size=float(tick))
except (KeyError, ValueError, TypeError):
return None
src/polymaker/marketdata/service.py
+185
View File
@@ -0,0 +1,185 @@
"""MarketDataService: owns the market WS, maintains a book per token.
Subscribes to every YES+NO token of the markets we quote and routes each frame
to that token's OrderBook. We do NOT set `custom_feature_enabled` (verified to
broaden the feed beyond our assets); resolution is detected via catalog flags.
On every book mutation it wakes the owning market's quoter via `on_dirty`, and
feeds trade prints to `on_trade` for the flow estimator. Reconnects re-snapshot
automatically because the server sends a fresh `book` on (re)subscribe.
"""
from __future__ import annotations
import asyncio
import json
from collections.abc import Callable
from typing import Any
import websockets
from polymaker.journal import Journal
from polymaker.logging import get_logger
from polymaker.marketdata.orderbook import BookView, OrderBook
from polymaker.marketdata.parse import (
TradePrint,
parse_book,
parse_last_trade,
parse_price_changes,
parse_tick_size_change,
)
log = get_logger("marketdata.service")
DirtyCb = Callable[[str, str], None] # (condition_id, token_id)
TradeCb = Callable[[TradePrint], None]
class MarketDataService:
def __init__(
self,
url: str = "wss://ws-subscriptions-clob.polymarket.com/ws/market",
*,
on_dirty: DirtyCb | None = None,
on_trade: TradeCb | None = None,
journal: Journal | None = None,
proxy: str | None = None,
) -> None:
self._url = url
self._on_dirty = on_dirty or (lambda _c, _t: None)
self._on_trade = on_trade or (lambda _tp: None)
self._journal = journal
self._proxy = proxy
self.books: dict[str, OrderBook] = {}
self._token_condition: dict[str, str] = {}
self._subs: list[str] = []
self._ws: Any = None
self._stop = asyncio.Event()
# ── subscription management ─────────────────────────────────────────
def set_markets(self, markets: list[tuple[str, list[str]]]) -> None:
"""markets = [(condition_id, [token_ids...])]. Rebuilds the desired set."""
subs: list[str] = []
for cond, tokens in markets:
for tok in tokens:
self._token_condition[tok] = cond
self.books.setdefault(tok, OrderBook())
subs.append(tok)
self._subs = subs
def view(self, token_id: str) -> BookView:
book = self.books.get(token_id)
return book.view() if book else _empty_view()
def book(self, token_id: str) -> OrderBook | None:
return self.books.get(token_id)
def last_update_ts(self, token_id: str) -> float:
b = self.books.get(token_id)
return b.last_update_ts if b else 0.0
# ── run loop ────────────────────────────────────────────────────────
async def run(self) -> None:
backoff = 1.0
while not self._stop.is_set():
try:
await self._connect_and_listen()
backoff = 1.0
except (websockets.ConnectionClosed, OSError) as exc:
log.warning("market_ws_dropped", err=str(exc), backoff=backoff)
except Exception as exc: # noqa: BLE001
log.error("market_ws_error", err=str(exc))
if self._stop.is_set():
break
await asyncio.sleep(backoff)
backoff = min(backoff * 2, 30.0)
async def _connect_and_listen(self) -> None:
if not self._subs:
await asyncio.sleep(1.0)
return
kwargs: dict[str, Any] = {"ping_interval": 5, "ping_timeout": None}
if self._proxy:
kwargs["proxy"] = self._proxy
async with websockets.connect(self._url, **kwargs) as ws:
self._ws = ws
await ws.send(json.dumps({"assets_ids": self._subs, "type": "market"}))
log.info("market_ws_subscribed", n=len(self._subs))
async for raw in ws:
self._handle(raw)
def stop(self) -> None:
self._stop.set()
# ── message handling ────────────────────────────────────────────────
def _handle(self, raw: str | bytes) -> None:
try:
data = json.loads(raw)
except (json.JSONDecodeError, TypeError):
return
for msg in data if isinstance(data, list) else [data]:
if not isinstance(msg, dict):
continue
self._dispatch(msg)
def _dispatch(self, msg: dict[str, Any]) -> None:
et = msg.get("event_type")
if et == "book":
self._on_book(msg)
elif et == "price_change":
self._on_price_change(msg)
elif et == "last_trade_price":
self._on_last_trade(msg)
elif et == "tick_size_change":
self._on_tick_change(msg)
def _on_book(self, msg: dict[str, Any]) -> None:
upd = parse_book(msg)
if upd is None or upd.asset_id not in self.books:
return
book = self.books[upd.asset_id]
if upd.tick_size:
book.set_tick_size(upd.tick_size)
book.apply_snapshot(upd.bids, upd.asks, upd.ts, upd.book_hash)
self._journal_write("book", msg, upd.ts)
self._wake(upd.asset_id)
def _on_price_change(self, msg: dict[str, Any]) -> None:
changes = parse_price_changes(msg)
touched: set[str] = set()
for ch in changes:
book = self.books.get(ch.asset_id)
if book is None:
continue
book.apply_delta(ch.side, ch.price, ch.size, ch.ts)
touched.add(ch.asset_id)
if changes:
self._journal_write("price_change", msg, changes[0].ts)
for tok in touched:
self._wake(tok)
def _on_last_trade(self, msg: dict[str, Any]) -> None:
tp = parse_last_trade(msg)
if tp is None or tp.asset_id not in self.books:
return
self._journal_write("last_trade_price", msg, tp.ts)
self._on_trade(tp)
def _on_tick_change(self, msg: dict[str, Any]) -> None:
tc = parse_tick_size_change(msg)
if tc and tc.asset_id in self.books:
self.books[tc.asset_id].set_tick_size(tc.tick_size)
log.info("tick_size_change", token=tc.asset_id[:12], tick=tc.tick_size)
def _wake(self, token_id: str) -> None:
cond = self._token_condition.get(token_id)
if cond:
self._on_dirty(cond, token_id)
def _journal_write(self, kind: str, payload: dict[str, Any], ts: float) -> None:
if self._journal is not None:
self._journal.write(kind, payload, ts)
def _empty_view() -> BookView:
return BookView(None, 0.0, None, 0.0, None, None, 0.0, 0.0)
src/polymaker/merge.py
+142
View File
@@ -0,0 +1,142 @@
"""Native Python position merger (replaces the Node.js poly_merger subprocess).
When we hold both YES and NO in the same market, merging the pair returns
collateral (1 USDC/pUSD per pair) — a maker-only exit with zero market impact.
Two execution paths:
* EOA wallet (signature_type=0): direct contract call, fully implemented here.
* Proxy/Safe wallet (signature_type 1/2): the merge tx must be routed through
the Safe. That path is gated on the Phase-2 wallet spike (see the README) — until
then merging is skipped (logged), and inventory is exited via limit sells
instead. The bot is fully functional without it; merging just frees capital
sooner.
The V2/pUSD collateral question (does the CTF collateral resolve to pUSD post-
migration?) is also spike-gated; addresses are config-driven, not baked in.
"""
from __future__ import annotations
from typing import Any
from polymaker.config import Config
from polymaker.logging import get_logger
log = get_logger("merge")
# Polygon mainnet contracts (pre-V2 defaults; confirm collateral in the spike).
CONDITIONAL_TOKENS = "0x4D97DCd97eC945f40cF65F87097ACe5EA0476045"
NEG_RISK_ADAPTER = "0xd91E80cF2E7be2e162c6513ceD06f1dD0dA35296"
USDC_COLLATERAL = "0x2791Bca1f2de4661ED88A30C99A7a9449Aa84174"
_CTF_ABI = [
{
"name": "mergePositions",
"type": "function",
"stateMutability": "nonpayable",
"inputs": [
{"name": "collateralToken", "type": "address"},
{"name": "parentCollectionId", "type": "bytes32"},
{"name": "conditionId", "type": "bytes32"},
{"name": "partition", "type": "uint256[]"},
{"name": "amount", "type": "uint256"},
],
"outputs": [],
}
]
_NEG_RISK_ABI = [
{
"name": "mergePositions",
"type": "function",
"stateMutability": "nonpayable",
"inputs": [
{"name": "conditionId", "type": "bytes32"},
{"name": "amount", "type": "uint256"},
],
"outputs": [],
}
]
class Merger:
def __init__(self, cfg: Config) -> None:
self._cfg = cfg
self._w3: Any = None
self._account: Any = None
def _ensure_web3(self) -> None:
if self._w3 is not None:
return
from eth_account import Account
from web3 import Web3
from web3.middleware import ExtraDataToPOAMiddleware
rpc = self._cfg.secrets.polygon_rpc or self._cfg.wallet.polygon_rpc
w3 = Web3(Web3.HTTPProvider(rpc))
w3.middleware_onion.inject(ExtraDataToPOAMiddleware, layer=0)
self._w3 = w3
self._account = Account.from_key(self._cfg.secrets.pk)
@property
def can_merge(self) -> bool:
"""EOA can merge directly today; Safe/proxy is spike-gated."""
return self._cfg.wallet.signature_type == 0
def merge(self, condition_id: str, amount_raw: int, neg_risk: bool) -> str | None:
"""Merge `amount_raw` (6-dec) YES+NO pairs. Returns tx hash or None."""
if amount_raw <= 0:
return None
if not self.can_merge:
log.info(
"merge_skipped_safe_wallet",
condition=condition_id[:12],
amount=amount_raw,
note="Safe merge is spike-gated; inventory exits via limit sells",
)
return None
try:
return self._merge_eoa(condition_id, amount_raw, neg_risk)
except Exception as exc: # noqa: BLE001
log.error("merge_failed", condition=condition_id[:12], err=str(exc))
return None
def _merge_eoa(self, condition_id: str, amount_raw: int, neg_risk: bool) -> str:
self._ensure_web3()
w3 = self._w3
addr = self._account.address
cond = _to_bytes32(condition_id)
if neg_risk:
c = w3.eth.contract(address=w3.to_checksum_address(NEG_RISK_ADAPTER), abi=_NEG_RISK_ABI)
fn = c.functions.mergePositions(cond, amount_raw)
else:
c = w3.eth.contract(address=w3.to_checksum_address(CONDITIONAL_TOKENS), abi=_CTF_ABI)
fn = c.functions.mergePositions(
w3.to_checksum_address(USDC_COLLATERAL),
b"\x00" * 32, # parent collection id (top-level market)
cond,
[1, 2], # partition: the two outcome slots
amount_raw,
)
tx = fn.build_transaction(
{
"from": addr,
"nonce": w3.eth.get_transaction_count(addr),
"chainId": self._cfg.wallet.chain_id,
"gas": 300_000,
"maxFeePerGas": w3.eth.gas_price * 2,
"maxPriorityFeePerGas": w3.to_wei(30, "gwei"),
}
)
signed = self._account.sign_transaction(tx)
tx_hash = w3.eth.send_raw_transaction(signed.raw_transaction)
receipt = w3.eth.wait_for_transaction_receipt(tx_hash, timeout=120)
h = str(receipt["transactionHash"].hex())
log.info("merge_sent", condition=condition_id[:12], amount=amount_raw, tx=h[:14])
return h
def _to_bytes32(hex_str: str) -> bytes:
s = hex_str[2:] if hex_str.startswith("0x") else hex_str
return bytes.fromhex(s.rjust(64, "0"))
src/polymaker/py.typed
View File
src/polymaker/risk/__init__.py
View File
src/polymaker/risk/manager.py
+146
View File
@@ -0,0 +1,146 @@
"""RiskManager: pre-trade gates and circuit breakers (see the README).
Consulted by the engine before every quote set. Returns a per-market decision
(size scale / reduce-only / halt) and owns the global kill switches. Position
and order data come from the StateStore; fair-value marks are pushed in by the
engine so PnL is always current.
"""
from __future__ import annotations
from dataclasses import dataclass
from polymaker.config import RiskConfig
from polymaker.domain import Fill, MarketMeta, Side
from polymaker.logging import get_logger
from polymaker.state.store import StateStore
log = get_logger("risk.manager")
@dataclass(frozen=True, slots=True)
class RiskDecision:
halt: bool # HALTED regime for this market
reduce_only: bool # REDUCE_ONLY regime for this market
size_scale: float # multiply quote sizes by this [0,1]
reason: str = ""
class RiskManager:
def __init__(self, cfg: RiskConfig, store: StateStore) -> None:
self._cfg = cfg
self._store = store
self._marks: dict[str, float] = {} # token_id -> fair value
self._net_cash = 0.0 # cumulative signed cash from fills (+sell, -buy)
self._day_start_equity = 0.0
self._killed = False
self._order_attempts = 0
self._order_errors = 0
# ── PnL bookkeeping ─────────────────────────────────────────────────
def note_fill(self, fill: Fill) -> None:
self._net_cash += (fill.price * fill.size) * (1 if fill.side is Side.SELL else -1)
def update_mark(self, token_id: str, fv: float) -> None:
self._marks[token_id] = fv
def _inventory_value(self) -> float:
total = 0.0
for tok, pos in self._store.positions.items():
if pos.size > 0:
total += pos.size * self._marks.get(tok, pos.avg_price)
return total
@property
def equity(self) -> float:
return self._net_cash + self._inventory_value()
@property
def daily_pnl(self) -> float:
return self.equity - self._day_start_equity
def reset_day(self) -> None:
self._day_start_equity = self.equity
# ── error-rate breaker ──────────────────────────────────────────────
def note_order_result(self, ok: bool) -> None:
self._order_attempts += 1
if not ok:
self._order_errors += 1
@property
def error_rate(self) -> float:
return self._order_errors / self._order_attempts if self._order_attempts >= 20 else 0.0
# ── global kill switch ──────────────────────────────────────────────
def global_halt(self) -> tuple[bool, str]:
if self._killed:
return True, "manual_kill"
if self.daily_pnl <= -self._cfg.daily_loss_kill_usdc:
return True, f"daily_loss {self.daily_pnl:.0f}"
if self.error_rate >= self._cfg.max_order_error_rate:
return True, f"error_rate {self.error_rate:.2f}"
return False, ""
def kill(self) -> None:
self._killed = True
log.critical("kill_switch_engaged")
# ── per-market evaluation ───────────────────────────────────────────
def evaluate(
self, meta: MarketMeta, *, ws_stale: bool, event_group_cost: float
) -> RiskDecision:
halted, why = self.global_halt()
if halted:
return RiskDecision(True, False, 0.0, why)
if ws_stale:
return RiskDecision(True, False, 0.0, "ws_stale")
market_notional = self._market_notional(meta)
total_exposure = self._total_exposure()
# hard caps -> reduce only
if market_notional >= self._cfg.max_market_notional_usdc:
return RiskDecision(False, True, 1.0, "market_cap")
if event_group_cost >= self._cfg.max_event_group_loss_usdc:
return RiskDecision(False, True, 1.0, "event_group_cap")
if total_exposure >= self._cfg.max_total_exposure_usdc:
return RiskDecision(False, True, 1.0, "total_exposure_cap")
# soft scaling: taper size as any cap is approached (worst-binding wins)
scale = min(
_headroom(market_notional, self._cfg.max_market_notional_usdc),
_headroom(total_exposure, self._cfg.max_total_exposure_usdc),
_headroom(event_group_cost, self._cfg.max_event_group_loss_usdc),
)
return RiskDecision(False, False, scale, "")
def _market_notional(self, meta: MarketMeta) -> float:
total = 0.0
for tok in (meta.yes.token_id, meta.no.token_id):
pos = self._store.position(tok)
total += pos.size * self._marks.get(tok, pos.avg_price or 0.5)
for o in self._store.orders_for(tok):
if o.side is Side.BUY:
total += o.notional
return total
def _total_exposure(self) -> float:
total = 0.0
for tok, pos in self._store.positions.items():
if pos.size > 0:
total += pos.size * self._marks.get(tok, pos.avg_price or 0.5)
for o in self._store.orders.values():
if o.side is Side.BUY:
total += o.notional
return total
def _headroom(current: float, cap: float) -> float:
"""1.0 well below the cap, tapering to 0 as we approach it (from 70%)."""
if cap <= 0:
return 1.0
frac = current / cap
if frac <= 0.7:
return 1.0
return max(0.0, (1.0 - frac) / 0.3)
src/polymaker/state/__init__.py
View File
src/polymaker/state/store.py
+176
View File
@@ -0,0 +1,176 @@
"""StateStore: the single owner of positions and open orders.
Replaces v1's module-level global dicts + the `performing`/`last_trade_update`
races. Three inputs, one arbitration rule (the README):
* WS fill events apply immediately (optimistic),
* REST reconciliation corrects drift ONLY for tokens with no in-flight trades,
* on-chain balances are consulted only by the merger.
In-memory + typed, mirrored to SQLite on change so a crash-restart resumes.
"""
from __future__ import annotations
import json
import sqlite3
import time
from pathlib import Path
from polymaker.domain import Fill, OpenOrder, OrderState, Position, Side
from polymaker.logging import get_logger
log = get_logger("state.store")
_SCHEMA = """
CREATE TABLE IF NOT EXISTS positions (
token_id TEXT PRIMARY KEY,
size REAL NOT NULL,
avg_price REAL NOT NULL,
updated_ts REAL NOT NULL
);
CREATE TABLE IF NOT EXISTS fills (
trade_id TEXT PRIMARY KEY,
token_id TEXT, side TEXT, price REAL, size REAL, is_maker INT, ts REAL
);
CREATE TABLE IF NOT EXISTS order_log (
order_id TEXT PRIMARY KEY,
token_id TEXT, side TEXT, price REAL, size REAL, state TEXT, ts REAL
);
"""
class StateStore:
"""Owns positions + open orders + a per-token in-flight guard."""
def __init__(self, db_path: str | Path = "state.db") -> None:
self._conn = sqlite3.connect(str(db_path))
self._conn.row_factory = sqlite3.Row
self._conn.execute("PRAGMA journal_mode=WAL")
self._conn.executescript(_SCHEMA)
self._conn.commit()
self.positions: dict[str, Position] = {}
# order_id -> OpenOrder
self.orders: dict[str, OpenOrder] = {}
# token_id -> count of in-flight (MATCHED-not-CONFIRMED) trades; guards reconcile
self._inflight: dict[str, int] = {}
self._last_fill_ts: dict[str, float] = {}
self._load()
def close(self) -> None:
self._conn.close()
# ── positions ───────────────────────────────────────────────────────
def position(self, token_id: str) -> Position:
return self.positions.get(token_id, Position(token_id))
def apply_fill(self, fill: Fill) -> None:
"""Apply a fill optimistically to inventory + avg price."""
pos = self.positions.setdefault(fill.token_id, Position(fill.token_id))
signed = fill.size if fill.side is Side.BUY else -fill.size
new_size = pos.size + signed
if fill.side is Side.BUY:
if pos.size <= 0:
pos.avg_price = fill.price
else:
pos.avg_price = (pos.avg_price * pos.size + fill.price * fill.size) / (
pos.size + fill.size
)
# selling leaves avg_price unchanged
pos.size = max(0.0, new_size)
if pos.size <= 0:
pos.avg_price = 0.0
self._last_fill_ts[fill.token_id] = fill.ts
self._persist_position(pos)
self._record_fill(fill)
log.info("fill", token=fill.token_id[:12], side=fill.side.value,
price=fill.price, size=fill.size, pos=round(pos.size, 2))
def set_position(self, token_id: str, size: float, avg_price: float) -> None:
pos = Position(token_id, max(0.0, size), avg_price if size > 0 else 0.0)
self.positions[token_id] = pos
self._persist_position(pos)
def reconcile_positions(self, api_positions: dict[str, tuple[float, float]]) -> None:
"""Overwrite sizes from REST, skipping tokens with in-flight trades or
a very recent fill (the optimistic value is more current there)."""
now = time.time()
for token_id, (size, avg) in api_positions.items():
if self._inflight.get(token_id, 0) > 0:
continue
if now - self._last_fill_ts.get(token_id, 0.0) < 5.0:
continue
self.set_position(token_id, size, avg)
# ── in-flight guard ─────────────────────────────────────────────────
def mark_inflight(self, token_id: str) -> None:
self._inflight[token_id] = self._inflight.get(token_id, 0) + 1
def clear_inflight(self, token_id: str) -> None:
if self._inflight.get(token_id, 0) > 0:
self._inflight[token_id] -= 1
def inflight(self, token_id: str) -> int:
return self._inflight.get(token_id, 0)
# ── orders ──────────────────────────────────────────────────────────
def orders_for(self, token_id: str) -> list[OpenOrder]:
return [o for o in self.orders.values() if o.token_id == token_id]
def upsert_order(self, order: OpenOrder) -> None:
if order.state in (OrderState.CANCELED, OrderState.DONE, OrderState.REJECTED):
self.orders.pop(order.order_id, None)
else:
self.orders[order.order_id] = order
self._persist_order(order)
def remove_order(self, order_id: str) -> None:
self.orders.pop(order_id, None)
def replace_open_orders(self, token_id: str, live: list[OpenOrder]) -> None:
"""Replace our view of a token's open orders from a REST snapshot."""
for oid in [o.order_id for o in self.orders.values() if o.token_id == token_id]:
self.orders.pop(oid, None)
for o in live:
self.orders[o.order_id] = o
# ── persistence ─────────────────────────────────────────────────────
def _persist_position(self, pos: Position) -> None:
self._conn.execute(
"INSERT OR REPLACE INTO positions(token_id,size,avg_price,updated_ts) VALUES(?,?,?,?)",
(pos.token_id, pos.size, pos.avg_price, time.time()),
)
self._conn.commit()
def _record_fill(self, f: Fill) -> None:
self._conn.execute(
"INSERT OR IGNORE INTO fills(trade_id,token_id,side,price,size,is_maker,ts) VALUES(?,?,?,?,?,?,?)",
(f.trade_id, f.token_id, f.side.value, f.price, f.size, int(f.is_maker), f.ts),
)
self._conn.commit()
def _persist_order(self, o: OpenOrder) -> None:
self._conn.execute(
"INSERT OR REPLACE INTO order_log(order_id,token_id,side,price,size,state,ts) VALUES(?,?,?,?,?,?,?)",
(o.order_id, o.token_id, o.side.value, o.price, o.size, o.state.value, time.time()),
)
self._conn.commit()
def _load(self) -> None:
for row in self._conn.execute("SELECT token_id,size,avg_price FROM positions"):
if row["size"] > 0:
self.positions[row["token_id"]] = Position(
row["token_id"], row["size"], row["avg_price"]
)
# ── reporting ───────────────────────────────────────────────────────
def snapshot(self) -> dict[str, object]:
return {
"positions": {k: json.loads(_pos_json(v)) for k, v in self.positions.items() if v.size > 0},
"open_orders": len(self.orders),
}
def _pos_json(p: Position) -> str:
return json.dumps({"size": round(p.size, 4), "avg_price": round(p.avg_price, 4)})
src/polymaker/state/tracker.py
+100
View File
@@ -0,0 +1,100 @@
"""Order/trade lifecycle processing over the StateStore.
Consumes *normalized* user-stream events (the wire-format extraction lives in
userstream/, so this is unit-testable with synthetic events) and drives the
state machine from the README:
Trade: MATCHED -> MINED -> CONFIRMED
└──────────-> FAILED (roll back the optimistic fill, reconcile)
Because we quote post-only, we are always the maker; `our_side` is our side of
each match. We apply the fill optimistically at MATCHED and reverse it on FAILED.
"""
from __future__ import annotations
from collections.abc import Callable
from dataclasses import dataclass
from polymaker.domain import Fill, OpenOrder, OrderState, Side, TradeState
from polymaker.logging import get_logger
from polymaker.state.store import StateStore
log = get_logger("state.tracker")
@dataclass(frozen=True, slots=True)
class TradeEvent:
token_id: str
our_side: Side
price: float
size: float
trade_id: str
status: TradeState
ts: float
@dataclass(frozen=True, slots=True)
class OrderEvent:
order_id: str
token_id: str
side: Side
price: float
remaining_size: float # original - matched
is_cancel: bool = False
class UserEventProcessor:
"""Applies normalized trade/order events to the store."""
def __init__(
self,
store: StateStore,
on_change: Callable[[str], None] | None = None,
on_fill: Callable[[Fill], None] | None = None,
) -> None:
self._store = store
self._on_change = on_change or (lambda _cid: None)
self._on_fill = on_fill or (lambda _fill: None)
# trade_id -> applied Fill, so FAILED can reverse exactly what we applied
self._applied: dict[str, Fill] = {}
def on_trade(self, ev: TradeEvent, condition_id: str) -> None:
if ev.status is TradeState.MATCHED:
if ev.trade_id in self._applied:
return # idempotent: already counted this match
fill = Fill(ev.token_id, ev.our_side, ev.price, ev.size, ev.trade_id, ev.ts, is_maker=True)
self._store.apply_fill(fill)
self._store.mark_inflight(ev.token_id)
self._applied[ev.trade_id] = fill
self._on_fill(fill)
self._on_change(condition_id)
elif ev.status in (TradeState.CONFIRMED, TradeState.MINED):
if ev.trade_id in self._applied and ev.status is TradeState.CONFIRMED:
self._store.clear_inflight(ev.token_id)
# keep the fill; it's now settled
self._applied.pop(ev.trade_id, None)
self._on_change(condition_id)
elif ev.status in (TradeState.FAILED, TradeState.RETRYING):
prior = self._applied.pop(ev.trade_id, None)
if prior is not None:
# reverse the optimistic fill
self._store.apply_fill(
Fill(prior.token_id, prior.side.opposite, prior.price, prior.size,
f"{prior.trade_id}:reverse", prior.ts, is_maker=True)
)
self._store.clear_inflight(ev.token_id)
log.warning("trade_failed_reversed", trade_id=ev.trade_id, token=ev.token_id[:12])
self._on_change(condition_id)
def on_order(self, ev: OrderEvent, condition_id: str) -> None:
if ev.is_cancel or ev.remaining_size <= 0:
self._store.remove_order(ev.order_id)
else:
state = OrderState.LIVE
self._store.upsert_order(
OpenOrder(ev.order_id, ev.token_id, ev.side, ev.price, ev.remaining_size, state)
)
self._on_change(condition_id)
src/polymaker/strategy/__init__.py
View File
src/polymaker/strategy/estimators.py
+195
View File
@@ -0,0 +1,195 @@
"""Online estimators driven by the live stream: EWMAs of vol, flow, toxicity.
All are time-decayed (half-life in seconds) so they behave correctly under
irregular event arrival — a burst of ticks and a quiet minute are weighted by
elapsed wall-clock, not by sample count. Pure state machines: feed them
observations with timestamps, read scalar summaries. No I/O.
"""
from __future__ import annotations
import math
from dataclasses import dataclass, field
from polymaker.domain import Side
class Ewma:
"""Time-decayed exponentially weighted mean.
On each update the prior weight decays by 0.5 ** (dt / halflife); a fresh
observation gets the remaining weight. The first observation seeds the mean.
"""
__slots__ = ("halflife", "_value", "_last_ts", "_initialized")
def __init__(self, halflife_s: float) -> None:
if halflife_s <= 0:
raise ValueError("halflife must be positive")
self.halflife = halflife_s
self._value = 0.0
self._last_ts = 0.0
self._initialized = False
def update(self, value: float, ts: float) -> float:
if not self._initialized:
self._value = value
self._last_ts = ts
self._initialized = True
return self._value
dt = max(0.0, ts - self._last_ts)
decay = 0.5 ** (dt / self.halflife)
self._value = decay * self._value + (1.0 - decay) * value
self._last_ts = ts
return self._value
def decay_to(self, ts: float) -> float:
"""Decay the stored value toward 0 as if observing 0 at `ts`.
Used to age out flow/vol during silence without a new observation.
"""
if self._initialized:
dt = max(0.0, ts - self._last_ts)
self._value *= 0.5 ** (dt / self.halflife)
self._last_ts = ts
return self._value
@property
def value(self) -> float:
return self._value
@property
def ready(self) -> bool:
return self._initialized
class VolEstimator:
"""Realized volatility at two horizons from fair-value changes."""
__slots__ = ("_short", "_long", "_last_fv", "_last_ts")
def __init__(self, short_halflife_s: float, long_halflife_s: float) -> None:
self._short = Ewma(short_halflife_s)
self._long = Ewma(long_halflife_s)
self._last_fv: float | None = None
self._last_ts = 0.0
def update(self, fv: float, ts: float) -> None:
if self._last_fv is not None:
r = fv - self._last_fv
sq = r * r
self._short.update(sq, ts)
self._long.update(sq, ts)
self._last_fv = fv
self._last_ts = ts
@property
def short(self) -> float:
return math.sqrt(max(0.0, self._short.value))
@property
def long(self) -> float:
return math.sqrt(max(0.0, self._long.value))
@property
def ratio(self) -> float:
"""short/long vol ratio; >1 means recent activity above baseline."""
lo = self.long
return self.short / lo if lo > 1e-9 else 1.0
class FlowEstimator:
"""Signed aggressor flow and its normalized strength (a crude z-score)."""
__slots__ = ("_signed", "_abs")
def __init__(self, halflife_s: float) -> None:
self._signed = Ewma(halflife_s)
self._abs = Ewma(halflife_s)
def update(self, aggressor: Side, size: float, ts: float) -> None:
signed = size if aggressor is Side.BUY else -size
self._signed.update(signed, ts)
self._abs.update(abs(size), ts)
def decay_to(self, ts: float) -> None:
self._signed.decay_to(ts)
self._abs.decay_to(ts)
@property
def signed(self) -> float:
return self._signed.value
@property
def z(self) -> float:
"""Signed flow normalized by average trade magnitude, in ~[-1, 1]+."""
denom = self._abs.value
return self._signed.value / denom if denom > 1e-9 else 0.0
@dataclass(slots=True)
class _PendingMarkout:
fv_at_fill: float
side: Side # our side of the fill (BUY => we bought => adverse if price falls)
due_ts: float
class MarkoutTracker:
"""Measures adverse selection: how fair value moves against us after fills.
For each fill we remember FV-at-fill and, after a horizon, compare to the
then-current FV. Signed so that a *positive* markout means the trade was
good (price moved in our favor) and negative means we got picked off. The
toxicity summary is the magnitude of recent adverse (negative) markout,
which the quoter turns into extra spread / less size.
"""
__slots__ = ("_horizon_s", "_pending", "_markout")
def __init__(self, horizon_s: float = 300.0, ewma_halflife_s: float = 1800.0) -> None:
self._horizon_s = horizon_s
self._pending: list[_PendingMarkout] = []
self._markout = Ewma(ewma_halflife_s)
def record_fill(self, side: Side, fv_at_fill: float, ts: float) -> None:
self._pending.append(_PendingMarkout(fv_at_fill, side, ts + self._horizon_s))
def evaluate(self, fv_now: float, ts: float) -> None:
"""Resolve any markouts whose horizon has elapsed."""
still: list[_PendingMarkout] = []
for p in self._pending:
if ts >= p.due_ts:
move = fv_now - p.fv_at_fill
# if we BOUGHT, a rise is good (+); if we SOLD, a fall is good (+)
signed = move if p.side is Side.BUY else -move
self._markout.update(signed, ts)
else:
still.append(p)
self._pending = still
@property
def markout(self) -> float:
return self._markout.value
@property
def toxicity(self) -> float:
"""Non-negative adverse-selection score (0 when fills are benign)."""
return max(0.0, -self._markout.value)
@dataclass(slots=True)
class MarketEstimators:
"""Bundle of the per-market online estimators the engine keeps."""
vol: VolEstimator
flow: FlowEstimator
markout: MarkoutTracker
last_fv: float | None = None
last_fv_ts: float = 0.0
fv_history: list[tuple[float, float]] = field(default_factory=list)
def on_fair_value(self, fv: float, ts: float) -> None:
self.vol.update(fv, ts)
self.markout.evaluate(fv, ts)
self.last_fv = fv
self.last_fv_ts = ts
src/polymaker/strategy/quoting.py
+203
View File
@@ -0,0 +1,203 @@
"""Pure quote construction: (market state, inventory, params) -> TargetQuotes.
This is the deterministic core of the strategy. No I/O, no wall-clock reads
except values passed in. Everything here is exercised directly by unit tests.
Model (see the README):
reservation r = FV - skew(inventory)
half-spread δ = base + c_vol·σ + c_tox·toxicity (clamped to reward band in QUIET)
YES entry bid = r - δ (BUY YES, USDC-collateralized)
NO entry bid = (1 - r) - δ (BUY NO; implied YES ask at r + δ)
exits = SELL limits on held inventory, walked toward the touch by urgency
The BUY-YES + BUY-NO pair is the canonical two-sided quote: both are bids, both
score rewards, and a filled pair merges back to USDC at locked edge 1 - p - q.
"""
from __future__ import annotations
import math
from dataclasses import dataclass
from polymaker.config import StrategyProfile
from polymaker.domain import MarketMeta, Position, Quote, Regime, Side, TargetQuotes
from polymaker.marketdata.orderbook import BookView
_EPS = 1e-9
def round_to_tick(price: float, tick: float, decimals: int, *, up: bool) -> float:
"""Snap a price to the tick grid, rounding up or down, clamped to (0,1)."""
n = price / tick
n = math.ceil(n - _EPS) if up else math.floor(n + _EPS)
p = round(n * tick, decimals)
return min(max(p, tick), 1.0 - tick)
def compute_fair_value(microprice: float, flow_z: float, tick: float, weight: float = 0.5) -> float:
"""Nudge the microprice by bounded signed flow. Clamped to (tick, 1-tick)."""
fv = microprice + weight * flow_z * tick
return min(max(fv, tick), 1.0 - tick)
@dataclass(frozen=True, slots=True)
class QuoteInputs:
meta: MarketMeta
regime: Regime
fv: float # YES fair value in (0,1)
vol_short: float
toxicity: float
yes_view: BookView
no_view: BookView
pos_yes: Position
pos_no: Position
profile: StrategyProfile
now: float
risk_size_scale: float = 1.0 # RiskManager may throttle size in [0,1]
yes_exit_urgency: float = 0.0 # [0,1]; engine raises with hold time / adverse drift
no_exit_urgency: float = 0.0
def construct_quotes(inp: QuoteInputs) -> TargetQuotes:
m = inp.meta
p = inp.profile
tick = m.tick_size
dec = m.price_decimals
cid = m.condition_id
if inp.regime in (Regime.EVENT, Regime.HALTED):
return TargetQuotes(cid, inp.regime, ())
quotes: list[Quote] = []
# ── inventory in YES-equivalent shares; holding NO is short YES ──────
net_shares = inp.pos_yes.size - inp.pos_no.size
q_max_shares = p.q_max_usdc / max(inp.fv, tick)
u = _clamp(net_shares / q_max_shares, -1.0, 1.0) if q_max_shares > 0 else 0.0
skew = p.gamma * inp.vol_short * u
# ── half-spread ─────────────────────────────────────────────────────
base = p.delta_min_ticks * tick
delta = base + p.c_vol * inp.vol_short + p.c_tox * inp.toxicity
reward_band = m.rewards_max_spread / 100.0
if inp.regime == Regime.QUIET and reward_band > 0:
delta = _clamp(delta, base, max(base, reward_band))
delta = max(delta, tick)
r = inp.fv - skew
yes_bid_target = r - delta
no_bid_target = (1.0 - r) - delta
# ── size scaling ────────────────────────────────────────────────────
regime_scale = 0.5 if inp.regime == Regime.TRENDING else 1.0
tox_scale = 1.0 / (1.0 + inp.toxicity * 10.0)
common_scale = regime_scale * tox_scale * _clamp(inp.risk_size_scale, 0.0, 1.0)
soft_cap = p.q_soft_frac # fraction of q_max at which the adding side pulls
add_yes = inp.regime not in (Regime.REDUCE_ONLY,) and u < soft_cap
add_no = inp.regime not in (Regime.REDUCE_ONLY,) and u > -soft_cap
# entry: BUY YES
if add_yes:
price = _place_bid(yes_bid_target, inp.yes_view, tick, dec, inp.fv, p.min_edge_ticks)
if price is not None:
_add_layers(quotes, m.yes.token_id, Side.BUY, price, tick, dec,
_size_shares(p.base_size_usdc, price, common_scale * (1 - max(u, 0.0)), m),
p.layers, p.layer_step_ticks, down=True)
# entry: BUY NO
if add_no:
no_fv = 1.0 - inp.fv
price = _place_bid(no_bid_target, inp.no_view, tick, dec, no_fv, p.min_edge_ticks)
if price is not None:
_add_layers(quotes, m.no.token_id, Side.BUY, price, tick, dec,
_size_shares(p.base_size_usdc, price, common_scale * (1 - max(-u, 0.0)), m),
p.layers, p.layer_step_ticks, down=True)
# ── exits: SELL held inventory (maker, never cross) ─────────────────
_maybe_exit(quotes, m.yes.token_id, inp.pos_yes, inp.fv, delta, inp.yes_view, tick, dec,
inp.yes_exit_urgency, m, inp.regime)
_maybe_exit(quotes, m.no.token_id, inp.pos_no, 1.0 - inp.fv, delta, inp.no_view, tick, dec,
inp.no_exit_urgency, m, inp.regime)
return TargetQuotes(cid, inp.regime, tuple(quotes))
# ── helpers ─────────────────────────────────────────────────────────────
def _clamp(x: float, lo: float, hi: float) -> float:
return min(max(x, lo), hi)
def _place_bid(
target: float, view: BookView, tick: float, dec: int, fv: float, min_edge_ticks: int
) -> float | None:
"""Position a BUY: join the touch or sit behind, never cross, keep min edge vs FV."""
price = target
# never bid above (FV - min_edge*tick): we don't pay through fair value
price = min(price, fv - min_edge_ticks * tick)
# join the queue rather than jump it (conservative maker default)
if view.best_bid is not None and price >= view.best_bid:
price = view.best_bid
# never cross the ask
if view.best_ask is not None and price >= view.best_ask:
price = view.best_ask - tick
p = round_to_tick(price, tick, dec, up=False)
if p <= 0 or p >= 1:
return None
return p
def _size_shares(base_usdc: float, price: float, scale: float, m: MarketMeta) -> float:
"""USDC-notional sizing -> shares, honoring exchange & reward minimums."""
shares = (base_usdc / max(price, m.tick_size)) * max(scale, 0.0)
if shares <= 0:
return 0.0
floor = max(m.min_order_size, m.rewards_min_size)
# round up small-but-real sizes to the reward min so they actually score
if 0.5 * floor <= shares < floor:
shares = floor
return round(shares, 2) if shares >= m.min_order_size else 0.0
def _add_layers(
quotes: list[Quote], token_id: str, side: Side, top_price: float, tick: float, dec: int,
total_size: float, layers: int, step_ticks: int, *, down: bool,
) -> None:
"""Split size across `layers` price levels stepping away from the touch."""
if total_size <= 0:
return
layers = max(1, layers)
per = round(total_size / layers, 2)
if per <= 0:
per = total_size
layers = 1
for i in range(layers):
offset = i * step_ticks * tick
price = top_price - offset if down else top_price + offset
price = round(price, dec)
if 0 < price < 1 and per > 0:
quotes.append(Quote(token_id, side, price, per))
def _maybe_exit(
quotes: list[Quote], token_id: str, pos: Position, token_fv: float, delta: float,
view: BookView, tick: float, dec: int, urgency: float, m: MarketMeta, regime: Regime,
) -> None:
if pos.size < m.min_order_size:
return
# target starts at fv + delta and walks toward best_bid + tick as urgency -> 1
passive = token_fv + delta
floor = (view.best_bid + tick) if view.best_bid is not None else passive
if regime == Regime.REDUCE_ONLY:
urgency = max(urgency, 0.5)
target = passive * (1.0 - urgency) + floor * urgency
# never cross down through the bid; never sell below best_bid
if view.best_bid is not None:
target = max(target, view.best_bid + tick)
price = round_to_tick(target, tick, dec, up=True)
size = round(pos.size, 2)
if 0 < price < 1 and size >= m.min_order_size:
quotes.append(Quote(token_id, Side.SELL, price, size))
src/polymaker/strategy/regime.py
+74
View File
@@ -0,0 +1,74 @@
"""Per-market regime decision (see the README).
Priority order, highest first:
HALTED kill switch / stale data / resolved / past halt-before window
EVENT active cooloff, or a fresh sweep / fair-value jump
REDUCE_ONLY inventory at hard cap, or inside the reduce-only end-date window
TRENDING persistent one-sided flow or elevated short/long vol
QUIET default farming posture
"""
from __future__ import annotations
from dataclasses import dataclass
from polymaker.config import StrategyProfile
from polymaker.domain import Regime
@dataclass(frozen=True, slots=True)
class RegimeInputs:
now: float
tick: float
fv: float
prev_fv: float | None
vol_ratio: float
flow_z: float
inventory_util: float # |net notional| / q_max, >=0
hours_to_end: float | None
sweep_flagged: bool = False
market_resolved: bool = False
ws_stale: bool = False
risk_halt: bool = False
risk_reduce_only: bool = False
class RegimeMachine:
"""Stateful regime decider for one market (tracks the EVENT cooloff)."""
__slots__ = ("_event_until",)
def __init__(self) -> None:
self._event_until: float = 0.0
def decide(self, inp: RegimeInputs, p: StrategyProfile) -> Regime:
# 1. hard halts
if inp.risk_halt or inp.ws_stale or inp.market_resolved:
return Regime.HALTED
if inp.hours_to_end is not None and inp.hours_to_end <= p.halt_before_hours:
return Regime.HALTED
# 2. events (sweep / jump / active cooloff)
jump_ticks = abs(inp.fv - inp.prev_fv) / inp.tick if inp.prev_fv is not None else 0.0
if inp.sweep_flagged or jump_ticks >= p.event_jump_ticks:
self._event_until = inp.now + p.event_cooloff_s
return Regime.EVENT
if inp.now < self._event_until:
return Regime.EVENT
# 3. reduce-only
if inp.risk_reduce_only or inp.inventory_util >= 1.0:
return Regime.REDUCE_ONLY
if inp.hours_to_end is not None and inp.hours_to_end <= p.reduce_only_hours:
return Regime.REDUCE_ONLY
# 4. trending
if abs(inp.flow_z) >= p.trend_flow_z or inp.vol_ratio >= 2.0:
return Regime.TRENDING
# 5. default
return Regime.QUIET
@property
def in_cooloff(self) -> bool:
return self._event_until > 0.0
src/polymaker/userstream/__init__.py
View File
src/polymaker/userstream/client.py
+126
View File
@@ -0,0 +1,126 @@
"""UserStream: authenticated user WS for our order/trade lifecycle events.
Subscribes with L2 creds and the condition_ids we trade; routes fills and order
updates into the StateStore via the UserEventProcessor. Reconnects with backoff.
"""
from __future__ import annotations
import asyncio
import json
from collections.abc import Callable
from typing import Any
import websockets
from polymaker.journal import Journal
from polymaker.logging import get_logger
from polymaker.state.tracker import UserEventProcessor
from polymaker.userstream.parse import normalize_order, normalize_trade
log = get_logger("userstream.client")
class UserStream:
def __init__(
self,
creds: Any,
our_address: str,
processor: UserEventProcessor,
*,
other_token: Callable[[str], str | None],
condition_of_token: Callable[[str], str | None],
url: str = "wss://ws-subscriptions-clob.polymarket.com/ws/user",
journal: Journal | None = None,
proxy: str | None = None,
) -> None:
self._creds = creds
self._address = our_address
self._proc = processor
self._other_token = other_token
self._condition_of_token = condition_of_token
self._url = url
self._journal = journal
self._proxy = proxy
self._markets: list[str] = []
self._stop = asyncio.Event()
def set_markets(self, condition_ids: list[str]) -> None:
self._markets = condition_ids
async def run(self) -> None:
backoff = 1.0
while not self._stop.is_set():
try:
await self._connect_and_listen()
backoff = 1.0
except (websockets.ConnectionClosed, OSError) as exc:
log.warning("user_ws_dropped", err=str(exc))
except Exception as exc: # noqa: BLE001
log.error("user_ws_error", err=str(exc))
if self._stop.is_set():
break
await asyncio.sleep(backoff)
backoff = min(backoff * 2, 30.0)
async def _connect_and_listen(self) -> None:
sub = {
"type": "user",
"auth": {
"apiKey": self._creds.api_key,
"secret": self._creds.api_secret,
"passphrase": self._creds.api_passphrase,
},
"markets": self._markets,
}
kwargs: dict[str, Any] = {"ping_interval": 5, "ping_timeout": None}
if self._proxy:
kwargs["proxy"] = self._proxy
async with websockets.connect(self._url, **kwargs) as ws:
await ws.send(json.dumps(sub))
log.info("user_ws_subscribed", markets=len(self._markets))
async for raw in ws:
self._handle(raw)
def stop(self) -> None:
self._stop.set()
def _handle(self, raw: str | bytes) -> None:
try:
data = json.loads(raw)
except (json.JSONDecodeError, TypeError):
return
for msg in data if isinstance(data, list) else [data]:
if not isinstance(msg, dict):
continue
et = msg.get("event_type")
if et == "trade":
self._on_trade(msg)
elif et == "order":
self._on_order(msg)
def _on_trade(self, msg: dict[str, Any]) -> None:
self._journal_write("user_trade", msg)
for ev in normalize_trade(msg, self._address, self._other_token):
cond = self._condition_of_token(ev.token_id) or str(msg.get("market", ""))
self._proc.on_trade(ev, cond)
def _on_order(self, msg: dict[str, Any]) -> None:
self._journal_write("user_order", msg)
ev = normalize_order(msg)
if ev is not None:
cond = self._condition_of_token(ev.token_id) or str(msg.get("market", ""))
self._proc.on_order(ev, cond)
def _journal_write(self, kind: str, payload: dict[str, Any]) -> None:
if self._journal is not None:
self._journal.write(kind, payload, _ts(payload))
def _ts(msg: dict[str, Any]) -> float:
raw = msg.get("timestamp")
try:
v = float(raw) # type: ignore[arg-type]
return v / 1000.0 if v > 1e12 else v
except (ValueError, TypeError):
return 0.0
src/polymaker/userstream/parse.py
+111
View File
@@ -0,0 +1,111 @@
"""Pure parsers for user-WS frames -> normalized trade/order events.
Fills come from `trade` events; open-order tracking from `order` events. The
maker/taker/mint side logic mirrors v1's proven handler (post-only means we are
always the maker):
* maker & taker on the SAME outcome -> we SELL the taker's asset (reverse side)
* maker & taker on DIFFERENT outcomes -> a mint: we BUY the opposite token
NOTE: exact field names must be reconfirmed in the Phase-2 wallet spike
(the README); this is coded to the v1-observed shape.
"""
from __future__ import annotations
from collections.abc import Callable
from typing import Any
from polymaker.domain import Side, TradeState
from polymaker.state.tracker import OrderEvent, TradeEvent
_STATUS = {
"MATCHED": TradeState.MATCHED,
"MINED": TradeState.MINED,
"CONFIRMED": TradeState.CONFIRMED,
"RETRYING": TradeState.RETRYING,
"FAILED": TradeState.FAILED,
}
def _ts(msg: dict[str, Any]) -> float:
raw = msg.get("timestamp")
try:
v = float(raw) # type: ignore[arg-type]
return v / 1000.0 if v > 1e12 else v
except (ValueError, TypeError):
return 0.0
def normalize_trade(
msg: dict[str, Any],
our_address: str,
other_token: Callable[[str], str | None],
) -> list[TradeEvent]:
"""Extract our maker fills from a `trade` event. Returns one TradeEvent per
matching maker order (usually one)."""
status = _STATUS.get(str(msg.get("status", "")).upper())
if status is None:
return []
taker_asset = str(msg.get("asset_id", ""))
taker_side = _side(msg.get("side"))
taker_outcome = msg.get("outcome")
ts = _ts(msg)
trade_id = str(msg.get("id", ""))
addr = our_address.lower()
out: list[TradeEvent] = []
for i, mo in enumerate(msg.get("maker_orders", []) or []):
if str(mo.get("maker_address", "")).lower() != addr:
continue
try:
size = float(mo.get("matched_amount", 0))
price = float(mo.get("price", 0))
except (ValueError, TypeError):
continue
if size <= 0:
continue
if mo.get("outcome") == taker_outcome:
token = taker_asset
our_side = taker_side.opposite
else:
token = other_token(taker_asset) or taker_asset
our_side = taker_side
out.append(
TradeEvent(
token_id=token,
our_side=our_side,
price=price,
size=size,
trade_id=f"{trade_id}:{i}" if len(msg.get('maker_orders', [])) > 1 else trade_id,
status=status,
ts=ts,
)
)
return out
def normalize_order(msg: dict[str, Any]) -> OrderEvent | None:
"""Map an `order` event to remaining-size tracking for the reconciler."""
try:
asset = str(msg["asset_id"])
side = _side(msg.get("side"))
original = float(msg.get("original_size", msg.get("size", 0)))
matched = float(msg.get("size_matched", 0))
remaining = original - matched
status = str(msg.get("status", "")).upper()
is_cancel = status in ("CANCELED", "CANCELLED") or msg.get("type") == "CANCELLATION"
return OrderEvent(
order_id=str(msg.get("id", "")),
token_id=asset,
side=side,
price=float(msg.get("price", 0)),
remaining_size=remaining,
is_cancel=is_cancel,
)
except (KeyError, ValueError, TypeError):
return None
def _side(value: object) -> Side:
return Side.SELL if str(value).upper() == "SELL" else Side.BUY
tests/__init__.py
View File
tests/conftest.py
+49
View File
@@ -0,0 +1,49 @@
"""Shared test fixtures."""
from __future__ import annotations
import pytest
from polymaker.config import StrategyProfile
from polymaker.domain import MarketMeta, TokenMeta
from polymaker.marketdata.orderbook import BookView
@pytest.fixture
def meta() -> MarketMeta:
return MarketMeta(
condition_id="0xcond",
question="Will X happen?",
slug="will-x-happen",
tokens=(TokenMeta("yes-token", "Yes"), TokenMeta("no-token", "No")),
tick_size=0.01,
neg_risk=False,
min_order_size=5.0,
rewards_min_size=10.0,
rewards_max_spread=3.0, # 3 cents
rewards_daily_rate=50.0,
maker_fee_bps=0,
taker_fee_bps=100,
fees_enabled=True,
end_date_iso="2028-11-07T00:00:00Z",
event_id="evt-1",
)
@pytest.fixture
def profile() -> StrategyProfile:
return StrategyProfile() # defaults
def view(bb: float | None, ba: float | None, bb_sz: float = 500, ba_sz: float = 500) -> BookView:
"""Construct a BookView for a token with a symmetric deep book."""
return BookView(
best_bid=bb,
best_bid_size=bb_sz,
best_ask=ba,
best_ask_size=ba_sz,
second_bid=(bb - 0.01) if bb else None,
second_ask=(ba + 0.01) if ba else None,
bid_depth=bb_sz,
ask_depth=ba_sz,
)
tests/test_catalog.py
+91
View File
@@ -0,0 +1,91 @@
"""Tests for market parsing, scoring, and the SQLite catalog store."""
from __future__ import annotations
import json
from polymaker.catalog.gamma import parse_market
from polymaker.catalog.scoring import score_market
from polymaker.catalog.store import CatalogStore
RAW = {
"conditionId": "0xabc",
"question": "Will candidate X win?",
"slug": "will-x-win",
"clobTokenIds": json.dumps(["tok-yes", "tok-no"]),
"outcomes": json.dumps(["Yes", "No"]),
"orderPriceMinTickSize": 0.01,
"orderMinSize": 5,
"negRisk": True,
"acceptingOrders": True,
"rewardsMinSize": 10,
"rewardsMaxSpread": 3.0,
"feesEnabled": True,
"feeSchedule": {"rate": 0.01, "takerOnly": True, "rebateRate": 0.25},
"bestBid": 0.48,
"bestAsk": 0.50,
"liquidityNum": 20000.0,
"volumeNum": 500000.0,
"endDate": "2028-11-07T00:00:00Z",
"events": [{"id": 999, "slug": "2028-election"}],
}
def test_parse_market_maps_fields():
m = parse_market(RAW, reward_rates={"0xabc": 42.0})
assert m is not None
assert m.condition_id == "0xabc"
assert m.yes.token_id == "tok-yes"
assert m.no.token_id == "tok-no"
assert m.tick_size == 0.01
assert m.neg_risk is True
assert m.rewards_daily_rate == 42.0
assert m.taker_fee_bps == 100 # 0.01 -> 100 bps
assert m.maker_fee_bps == 0 # V2 makers pay zero
assert m.rebate_rate == 0.25
assert m.event_id == "999"
def test_parse_market_rejects_non_binary_and_closed():
triple = {**RAW, "clobTokenIds": json.dumps(["a", "b", "c"]),
"outcomes": json.dumps(["A", "B", "C"])}
assert parse_market(triple) is None
not_accepting = {**RAW, "acceptingOrders": False}
assert parse_market(not_accepting) is None
def test_score_prefers_rewards_and_rebates():
good = parse_market(RAW, {"0xabc": 100.0})
poor = parse_market({**RAW, "conditionId": "0xdef", "rewardsMinSize": 0,
"rewardsMaxSpread": 0, "feesEnabled": False},
{"0xdef": 0.0})
assert score_market(good).score > score_market(poor).score
def test_score_penalizes_extremity():
balanced = parse_market(RAW, {"0xabc": 50.0})
extreme = parse_market({**RAW, "conditionId": "0xext", "bestBid": 0.96, "bestAsk": 0.98},
{"0xext": 50.0})
assert score_market(extreme).extremity > score_market(balanced).extremity
def test_store_roundtrip_and_top(tmp_path):
store = CatalogStore(tmp_path / "s.db")
m = parse_market(RAW, {"0xabc": 42.0})
store.upsert_market(m)
assert store.get("0xabc").condition_id == "0xabc"
assert store.get_by_slug("will-x-win").slug == "will-x-win"
top = store.top(10)
assert len(top) == 1 and top[0][0].condition_id == "0xabc"
# tokens survive the JSON round-trip as a 2-tuple
assert len(store.get("0xabc").tokens) == 2
store.close()
def test_store_upsert_is_idempotent(tmp_path):
store = CatalogStore(tmp_path / "s.db")
m = parse_market(RAW, {"0xabc": 42.0})
store.upsert_market(m)
store.upsert_market(m) # second time updates, not duplicates
assert len(store.top(10)) == 1
store.close()
tests/test_engine.py
+76
View File
@@ -0,0 +1,76 @@
"""Integration test: one full engine recompute cycle in paper mode (no network)."""
from __future__ import annotations
import asyncio
import time
from polymaker.config import Config, PathsConfig, StrategyProfile
from polymaker.domain import Side
from polymaker.engine import Engine
from polymaker.strategy.regime import RegimeMachine
def _engine_with_market(tmp_path, meta) -> Engine:
cfg = Config(paths=PathsConfig(db=str(tmp_path / "state.db"),
journal_dir=str(tmp_path / "j"),
log_dir=str(tmp_path / "l")))
cfg.engine.journal = False
eng = Engine(cfg, paper=True)
cid = meta.condition_id
# inject one market directly, bypassing network resolution
eng.metas[cid] = meta
eng.profiles[cid] = StrategyProfile()
eng.est[cid] = Engine._make_estimators(eng.profiles[cid])
eng.regime_m[cid] = RegimeMachine()
eng._dirty[cid] = asyncio.Event()
for tok in (meta.yes.token_id, meta.no.token_id):
eng._token_cid[tok] = cid
eng.md.set_markets([(cid, [meta.yes.token_id, meta.no.token_id])])
eng._running = True
return eng
def _feed_book(eng, meta):
now = time.time() # fresh ts so the ws_stale guard doesn't HALT the market
yb = eng.md.book(meta.yes.token_id)
yb.apply_snapshot(bids=[(0.48, 500), (0.49, 500)], asks=[(0.51, 500), (0.52, 500)], ts=now)
nb = eng.md.book(meta.no.token_id)
nb.apply_snapshot(bids=[(0.48, 500), (0.49, 500)], asks=[(0.51, 500), (0.52, 500)], ts=now)
async def test_recompute_places_two_sided_paper_quotes(tmp_path, meta):
eng = _engine_with_market(tmp_path, meta)
_feed_book(eng, meta)
await eng._recompute(meta.condition_id)
yes_orders = eng.state.orders_for(meta.yes.token_id)
no_orders = eng.state.orders_for(meta.no.token_id)
assert yes_orders, "no YES quotes placed"
assert no_orders, "no NO quotes placed"
# entry quotes are BUYs on both tokens (the canonical two-sided quote)
assert all(o.side is Side.BUY for o in yes_orders)
assert all(o.side is Side.BUY for o in no_orders)
eng.state.close()
eng.catalog.close()
async def test_recompute_is_idempotent_within_tolerance(tmp_path, meta):
eng = _engine_with_market(tmp_path, meta)
_feed_book(eng, meta)
await eng._recompute(meta.condition_id)
n_after_first = len(eng.state.orders)
# same book -> reconcile should be a no-op, order count unchanged
await eng._recompute(meta.condition_id)
assert len(eng.state.orders) == n_after_first
eng.state.close()
eng.catalog.close()
async def test_recompute_skips_when_book_empty(tmp_path, meta):
eng = _engine_with_market(tmp_path, meta)
# no book fed
await eng._recompute(meta.condition_id)
assert len(eng.state.orders) == 0
eng.state.close()
eng.catalog.close()
tests/test_estimators.py
+83
View File
@@ -0,0 +1,83 @@
"""Unit tests for the online estimators (vol, flow, markout/toxicity)."""
from __future__ import annotations
import pytest
from polymaker.domain import Side
from polymaker.strategy.estimators import (
Ewma,
FlowEstimator,
MarkoutTracker,
VolEstimator,
)
def test_ewma_seeds_then_decays():
e = Ewma(halflife_s=10.0)
assert not e.ready
e.update(1.0, ts=0.0)
assert e.value == 1.0
# after exactly one half-life, a 0 observation pulls the mean halfway
e.update(0.0, ts=10.0)
assert e.value == pytest.approx(0.5, abs=1e-9)
def test_ewma_decay_to_ages_value():
e = Ewma(halflife_s=10.0)
e.update(1.0, ts=0.0)
e.decay_to(ts=10.0) # one half-life of silence
assert e.value == pytest.approx(0.5, abs=1e-9)
def test_vol_estimator_rises_with_movement():
v = VolEstimator(short_halflife_s=5.0, long_halflife_s=100.0)
# quiet: tiny moves
fv, t = 0.5, 0.0
for _ in range(20):
t += 1.0
v.update(fv, t) # no change -> zero vol
assert v.short == pytest.approx(0.0, abs=1e-6)
# sudden jumps -> short vol jumps, ratio > 1
for step in (0.05, -0.04, 0.06):
t += 1.0
fv += step
v.update(fv, t)
assert v.short > 0.01
assert v.ratio > 1.0
def test_flow_estimator_sign_and_z():
f = FlowEstimator(halflife_s=10.0)
t = 0.0
for _ in range(5):
t += 1.0
f.update(Side.BUY, 100, t) # persistent buying
assert f.signed > 0
assert f.z > 0.5 # strongly one-sided
# now heavy selling flips the sign over time
for _ in range(10):
t += 1.0
f.update(Side.SELL, 200, t)
assert f.signed < 0
assert f.z < 0
def test_markout_toxicity_from_adverse_fills():
mt = MarkoutTracker(horizon_s=30.0, ewma_halflife_s=100.0)
# we BUY at fv=0.50; price then falls to 0.45 after the horizon -> adverse
mt.record_fill(Side.BUY, fv_at_fill=0.50, ts=0.0)
mt.evaluate(fv_now=0.50, ts=10.0) # before horizon: nothing resolves
assert mt.markout == 0.0
mt.evaluate(fv_now=0.45, ts=31.0) # after horizon: -0.05 markout
assert mt.markout < 0
assert mt.toxicity > 0
def test_markout_benign_fills_are_not_toxic():
mt = MarkoutTracker(horizon_s=30.0, ewma_halflife_s=100.0)
# we BUY at 0.50; price rises to 0.55 -> favorable, not toxic
mt.record_fill(Side.BUY, fv_at_fill=0.50, ts=0.0)
mt.evaluate(fv_now=0.55, ts=31.0)
assert mt.markout > 0
assert mt.toxicity == 0.0
tests/test_execution.py
+69
View File
@@ -0,0 +1,69 @@
"""Tests for the rate budgeter and the paper-mode gateway."""
from __future__ import annotations
import asyncio
import time
import pytest
from polymaker.config import Config
from polymaker.domain import Quote, Side
from polymaker.execution.gateway import ExecutionGateway, _tick_str
from polymaker.execution.ratelimit import TokenBucket
def test_tick_str_formats():
assert _tick_str(0.01) == "0.01"
assert _tick_str(0.001) == "0.001"
assert _tick_str(0.0025) == "0.0025"
assert _tick_str(0.1) == "0.1"
async def test_token_bucket_limits_rate():
bucket = TokenBucket(rate_per_s=100.0, burst=5.0)
start = time.monotonic()
# burst of 5 is instant; the next 5 must wait ~ (5/100)s = 50ms
for _ in range(10):
await bucket.acquire(1)
elapsed = time.monotonic() - start
assert elapsed >= 0.04 # had to wait for refill
async def test_token_bucket_pressure_rises_when_drained():
bucket = TokenBucket(rate_per_s=10.0, burst=10.0)
assert bucket.pressure == pytest.approx(0.0, abs=0.01)
for _ in range(10):
await bucket.acquire(1)
assert bucket.pressure > 0.8
async def test_paper_gateway_places_and_cancels_without_wallet(meta):
cfg = Config() # defaults, no secrets
gw = ExecutionGateway(cfg, paper=True)
quotes = [
Quote(meta.yes.token_id, Side.BUY, 0.49, 100),
Quote(meta.no.token_id, Side.BUY, 0.48, 100),
]
placed = await gw.place(quotes, meta)
assert len(placed) == 2
assert all(o.order_id.startswith("paper-") for o in placed)
# cancel is a no-op in paper mode but must not raise
await gw.cancel([o.order_id for o in placed])
assert await gw.open_orders() == []
async def test_paper_gateway_heartbeat_and_cancel_all_noop():
gw = ExecutionGateway(Config(), paper=True)
await gw.heartbeat("hb1")
await gw.cancel_all() # no client, must not raise
def test_gateway_requires_wallet_for_live_connect():
from polymaker.config import Secrets
# explicitly-empty secrets (don't read a real .env that may exist on disk)
cfg = Config(secrets=Secrets(_env_file=None))
gw = ExecutionGateway(cfg, paper=False)
with pytest.raises(RuntimeError, match="no wallet"):
asyncio.run(gw.connect())
tests/test_live_marketdata.py
+74
View File
@@ -0,0 +1,74 @@
"""Live integration test for the market WS (network-gated).
Run explicitly with: POLYMAKER_LIVE=1 uv run pytest tests/test_live_marketdata.py
Skipped by default so the unit suite stays offline and fast.
"""
from __future__ import annotations
import asyncio
import json
import os
import httpx
import pytest
import websockets
pytestmark = pytest.mark.skipif(
os.environ.get("POLYMAKER_LIVE") != "1", reason="live test; set POLYMAKER_LIVE=1"
)
async def _top_political_tokens() -> tuple[str, list[str]]:
async with httpx.AsyncClient(timeout=20) as c:
r = await c.get(
"https://gamma-api.polymarket.com/markets",
params={"limit": 1, "closed": "false", "tag_id": 2,
"order": "volume24hr", "ascending": "false"},
)
m = r.json()[0]
return m["conditionId"], json.loads(m["clobTokenIds"])
async def test_market_service_builds_a_live_book():
from polymaker.marketdata.service import MarketDataService
cond, tokens = await _top_political_tokens()
woken: list[tuple[str, str]] = []
svc = MarketDataService(on_dirty=lambda c, t: woken.append((c, t)))
svc.set_markets([(cond, tokens)])
task = asyncio.create_task(svc.run())
try:
# wait until at least one token has a two-sided book
for _ in range(60):
await asyncio.sleep(0.5)
if any(not svc.book(t).is_empty for t in tokens):
break
finally:
svc.stop()
task.cancel()
assert woken, "quoter was never woken by a book event"
live = [t for t in tokens if not svc.book(t).is_empty]
assert live, "no book was populated from the live feed"
book = svc.book(live[0])
assert book.best_bid() is not None and book.best_ask() is not None
assert book.best_bid().price < book.best_ask().price
async def test_live_market_ws_raw_frames():
"""Sanity check the wire format our parser targets hasn't drifted."""
_, tokens = await _top_political_tokens()
uri = "wss://ws-subscriptions-clob.polymarket.com/ws/market"
async with websockets.connect(uri, ping_interval=5, ping_timeout=None) as ws:
await ws.send(json.dumps({"assets_ids": tokens, "type": "market"}))
got_book = False
for _ in range(10):
raw = await asyncio.wait_for(ws.recv(), timeout=10)
data = json.loads(raw)
for msg in data if isinstance(data, list) else [data]:
if msg.get("event_type") == "book":
assert {"asset_id", "bids", "asks", "hash"} <= set(msg)
got_book = True
assert got_book
tests/test_marketdata.py
+110
View File
@@ -0,0 +1,110 @@
"""Tests for market-WS parsing and the book service routing."""
from __future__ import annotations
from polymaker.domain import Side
from polymaker.marketdata.parse import (
parse_book,
parse_last_trade,
parse_price_changes,
parse_tick_size_change,
)
from polymaker.marketdata.service import MarketDataService
# frames modeled on live captures (2026-07-05)
BOOK = {
"event_type": "book",
"market": "0xcond",
"asset_id": "yes-tok",
"timestamp": "1783270000000",
"hash": "abc123",
"tick_size": "0.01",
"bids": [{"price": "0.48", "size": "100"}, {"price": "0.49", "size": "200"}],
"asks": [{"price": "0.52", "size": "150"}, {"price": "0.51", "size": "80"}],
}
PRICE_CHANGE = {
"event_type": "price_change",
"market": "0xcond",
"timestamp": "1783270001000",
"price_changes": [
{"asset_id": "yes-tok", "price": "0.49", "size": "0", "side": "BUY", "hash": "h"},
{"asset_id": "yes-tok", "price": "0.50", "size": "300", "side": "BUY", "hash": "h"},
],
}
LAST_TRADE = {
"event_type": "last_trade_price",
"market": "0xcond",
"asset_id": "yes-tok",
"price": "0.50",
"size": "42",
"side": "BUY",
"timestamp": "1783270002000",
}
def test_parse_book_converts_ms_and_levels():
upd = parse_book(BOOK)
assert upd is not None
assert upd.asset_id == "yes-tok"
assert upd.condition_id == "0xcond"
assert (0.49, 200) in upd.bids
assert upd.ts == 1783270000.0 # ms -> s
assert upd.tick_size == 0.01
def test_parse_price_changes():
changes = parse_price_changes(PRICE_CHANGE)
assert len(changes) == 2
assert changes[0].side is Side.BUY
assert changes[1].price == 0.50 and changes[1].size == 300
def test_parse_last_trade_aggressor():
tp = parse_last_trade(LAST_TRADE)
assert tp is not None
assert tp.aggressor is Side.BUY
assert tp.size == 42
def test_parse_tick_size_change():
tc = parse_tick_size_change(
{"event_type": "tick_size_change", "asset_id": "yes-tok", "new_tick_size": "0.001"}
)
assert tc is not None and tc.tick_size == 0.001
def test_service_routes_book_and_wakes_quoter():
woken: list[tuple[str, str]] = []
svc = MarketDataService(on_dirty=lambda c, t: woken.append((c, t)))
svc.set_markets([("0xcond", ["yes-tok", "no-tok"])])
svc._dispatch(BOOK)
book = svc.book("yes-tok")
assert book is not None
assert book.best_bid().price == 0.49
assert book.best_ask().price == 0.51
assert woken == [("0xcond", "yes-tok")]
def test_service_applies_price_change_delta():
svc = MarketDataService()
svc.set_markets([("0xcond", ["yes-tok"])])
svc._dispatch(BOOK)
svc._dispatch(PRICE_CHANGE) # removes 0.49 bid, adds 0.50 bid
book = svc.book("yes-tok")
assert book.best_bid().price == 0.50
assert 0.49 not in book.bids
def test_service_ignores_unsubscribed_asset():
svc = MarketDataService()
svc.set_markets([("0xcond", ["yes-tok"])])
svc._dispatch({**BOOK, "asset_id": "stranger"})
assert svc.book("stranger") is None
def test_service_forwards_trades_for_flow():
trades = []
svc = MarketDataService(on_trade=trades.append)
svc.set_markets([("0xcond", ["yes-tok"])])
svc._dispatch(LAST_TRADE)
assert len(trades) == 1 and trades[0].aggressor is Side.BUY
tests/test_orderbook.py
+94
View File
@@ -0,0 +1,94 @@
"""Unit tests for the order book and its analytics."""
from __future__ import annotations
import pytest
from polymaker.domain import Side
from polymaker.marketdata.orderbook import OrderBook, to_no_price
def make_book() -> OrderBook:
ob = OrderBook(tick_size=0.01)
ob.apply_snapshot(
bids=[(0.40, 100), (0.41, 200), (0.42, 50)], # best bid 0.42
asks=[(0.45, 80), (0.46, 150), (0.44, 30)], # best ask 0.44
ts=1.0,
)
return ob
def test_best_bid_ask():
ob = make_book()
assert ob.best_bid().price == 0.42
assert ob.best_bid().size == 50
assert ob.best_ask().price == 0.44
assert ob.best_ask().size == 30
def test_apply_delta_add_and_remove():
ob = make_book()
ob.apply_delta(Side.BUY, 0.43, 25, ts=2.0)
assert ob.best_bid().price == 0.43
ob.apply_delta(Side.BUY, 0.43, 0, ts=3.0) # size 0 removes the level
assert ob.best_bid().price == 0.42
assert ob.last_update_ts == 3.0
def test_empty_book_views_are_none():
ob = OrderBook()
assert ob.best_bid() is None
assert ob.best_ask() is None
assert ob.microprice() is None
assert ob.is_empty
v = ob.view()
assert v.mid is None
assert v.spread is None
assert v.imbalance == 0.0
def test_microprice_pulls_toward_thin_side():
ob = OrderBook(tick_size=0.01)
# bid side much heavier than ask side -> microprice near the ask
ob.apply_snapshot(bids=[(0.40, 1000)], asks=[(0.42, 10)], ts=1.0)
mp = ob.microprice(levels=1)
assert mp is not None
assert 0.41 < mp <= 0.42 # dragged up toward the thin ask
# symmetric sizes -> mid
ob.apply_snapshot(bids=[(0.40, 100)], asks=[(0.42, 100)], ts=2.0)
assert ob.microprice(levels=1) == pytest.approx(0.41)
def test_best_with_min_size_skips_dust():
ob = OrderBook(tick_size=0.01)
# a dust order (size 1) sits at the touch; real size is one level back
ob.apply_snapshot(bids=[(0.42, 1), (0.41, 500)], asks=[(0.44, 1), (0.45, 500)], ts=1.0)
price, size, top = ob.best_with_min_size(Side.BUY, min_size=5)
assert price == 0.41 and size == 500
assert top == 0.42 # the dust touch is still reported as top
price, size, top = ob.best_with_min_size(Side.SELL, min_size=5)
assert price == 0.45 and size == 500
assert top == 0.44
def test_depth_within_band():
ob = make_book()
# bids at 0.40,0.41,0.42 all within [0.40, 0.42]
assert ob.depth_within(Side.BUY, 0.40, 0.42) == 350
assert ob.depth_within(Side.BUY, 0.415, 0.42) == 50
def test_view_second_levels_and_imbalance():
ob = make_book()
v = ob.view(min_size=0.0)
assert v.best_bid == 0.42
assert v.second_bid == 0.41
assert v.best_ask == 0.44
assert v.second_ask == 0.45
assert -1.0 <= v.imbalance <= 1.0
def test_no_price_mirror():
assert to_no_price(0.42) == pytest.approx(0.58)
assert to_no_price(0.0) == 1.0
assert to_no_price(1.0) == 0.0
tests/test_quoting.py
+187
View File
@@ -0,0 +1,187 @@
"""Unit tests for pure quote construction — the strategy's decision core."""
from __future__ import annotations
import pytest
from polymaker.domain import Position, Regime, Side
from polymaker.strategy.quoting import (
QuoteInputs,
compute_fair_value,
construct_quotes,
round_to_tick,
)
from tests.conftest import view
def _inputs(meta, profile, **over):
base = dict(
meta=meta,
regime=Regime.QUIET,
fv=0.50,
vol_short=0.0,
toxicity=0.0,
yes_view=view(0.49, 0.51),
no_view=view(0.49, 0.51),
pos_yes=Position("yes-token"),
pos_no=Position("no-token"),
profile=profile,
now=1000.0,
)
base.update(over)
return QuoteInputs(**base)
# ── round_to_tick ──────────────────────────────────────────────────────────
def test_round_to_tick_down_and_up():
assert round_to_tick(0.5049, 0.01, 2, up=False) == 0.50
assert round_to_tick(0.5051, 0.01, 2, up=True) == 0.51
# clamps inside (0,1)
assert round_to_tick(0.0, 0.01, 2, up=False) == 0.01
assert round_to_tick(1.0, 0.01, 2, up=True) == 0.99
def test_compute_fair_value_flow_nudge():
# positive flow nudges FV up, negative down, no flow = microprice
assert compute_fair_value(0.50, 0.0, 0.01) == pytest.approx(0.50)
assert compute_fair_value(0.50, 1.0, 0.01, weight=0.5) == pytest.approx(0.505)
assert compute_fair_value(0.50, -1.0, 0.01, weight=0.5) == pytest.approx(0.495)
# ── two-sided quoting ────────────────────────────────────────────────────────
def test_quiet_market_quotes_both_sides_as_bids(meta, profile):
tq = construct_quotes(_inputs(meta, profile))
assert tq.regime == Regime.QUIET
yes = [q for q in tq.quotes if q.token_id == "yes-token"]
no = [q for q in tq.quotes if q.token_id == "no-token"]
assert yes and no
# both entry quotes are BUYs (USDC-collateralized two-sided quote)
assert all(q.side == Side.BUY for q in yes)
assert all(q.side == Side.BUY for q in no)
def test_pair_prices_sum_below_one(meta, profile):
"""BUY YES @ p and BUY NO @ q must satisfy p + q < 1 (merge edge)."""
tq = construct_quotes(_inputs(meta, profile))
top_yes = max(q.price for q in tq.quotes if q.token_id == "yes-token")
top_no = max(q.price for q in tq.quotes if q.token_id == "no-token")
assert top_yes + top_no < 1.0
def test_never_bids_through_fair_value(meta, profile):
"""No BUY should ever sit at or above FV - min_edge (YES) / (1-FV)-min_edge (NO)."""
tq = construct_quotes(_inputs(meta, profile, fv=0.50))
edge = profile.min_edge_ticks * meta.tick_size
for q in tq.quotes:
if q.side == Side.BUY and q.token_id == "yes-token":
assert q.price <= 0.50 - edge + 1e-9
if q.side == Side.BUY and q.token_id == "no-token":
assert q.price <= 0.50 - edge + 1e-9 # NO fv is also 0.50 here
def test_layers_split_size(meta, profile):
tq = construct_quotes(_inputs(meta, profile))
yes = sorted((q for q in tq.quotes if q.token_id == "yes-token" and q.side == Side.BUY),
key=lambda q: -q.price)
assert len(yes) == profile.layers
# deeper layer is at a lower price
assert yes[0].price > yes[1].price
# ── inventory skew ──────────────────────────────────────────────────────────
def test_long_yes_inventory_skews_quotes_down(meta, profile):
"""Holding YES should lower the YES bid and raise the NO bid vs flat."""
flat = construct_quotes(_inputs(meta, profile, vol_short=0.02))
longy = construct_quotes(
_inputs(meta, profile, vol_short=0.02, pos_yes=Position("yes-token", 300, 0.5))
)
def top(tq, tok):
ps = [q.price for q in tq.quotes if q.token_id == tok and q.side == Side.BUY]
return max(ps) if ps else None
# YES bid should not be higher when long YES; NO bid should not be lower
assert top(longy, "yes-token") <= top(flat, "yes-token")
assert top(longy, "no-token") >= top(flat, "no-token")
def test_reduce_only_emits_only_exits(meta, profile):
tq = construct_quotes(
_inputs(
meta, profile, regime=Regime.REDUCE_ONLY,
pos_yes=Position("yes-token", 100, 0.5),
)
)
assert all(q.side == Side.SELL for q in tq.quotes)
assert any(q.token_id == "yes-token" for q in tq.quotes)
def test_event_and_halted_pull_all_quotes(meta, profile):
for regime in (Regime.EVENT, Regime.HALTED):
tq = construct_quotes(
_inputs(meta, profile, regime=regime, pos_yes=Position("yes-token", 100, 0.5))
)
assert tq.is_empty
# ── exits ────────────────────────────────────────────────────────────────────
def test_exit_sell_priced_above_fv_when_not_urgent(meta, profile):
tq = construct_quotes(
_inputs(meta, profile, pos_yes=Position("yes-token", 100, 0.4), yes_exit_urgency=0.0)
)
sells = [q for q in tq.quotes if q.side == Side.SELL and q.token_id == "yes-token"]
assert sells
assert sells[0].price >= 0.50 # at/above FV, a passive maker exit
def test_exit_never_below_best_bid(meta, profile):
tq = construct_quotes(
_inputs(
meta, profile,
pos_yes=Position("yes-token", 100, 0.4),
yes_view=view(0.49, 0.51),
yes_exit_urgency=1.0, # maximally urgent
)
)
sells = [q for q in tq.quotes if q.side == Side.SELL and q.token_id == "yes-token"]
assert sells
assert sells[0].price >= 0.49 # still a maker order, never crosses down
def test_no_exit_when_position_is_dust(meta, profile):
tq = construct_quotes(
_inputs(meta, profile, pos_yes=Position("yes-token", 1.0, 0.4)) # below min_order_size
)
assert not [q for q in tq.quotes if q.side == Side.SELL]
# ── spread widening ──────────────────────────────────────────────────────────
def test_toxicity_widens_spread(meta, profile):
"""Higher toxicity should push the YES bid lower (wider spread)."""
calm = construct_quotes(_inputs(meta, profile, regime=Regime.TRENDING, toxicity=0.0))
toxic = construct_quotes(_inputs(meta, profile, regime=Regime.TRENDING, toxicity=0.02))
def top_yes(tq):
ps = [q.price for q in tq.quotes if q.token_id == "yes-token" and q.side == Side.BUY]
return max(ps) if ps else None
assert top_yes(toxic) < top_yes(calm)
def test_quiet_regime_clamps_spread_to_reward_band(meta, profile):
"""In QUIET, even with high vol the bid stays within the reward band of FV."""
tq = construct_quotes(_inputs(meta, profile, regime=Regime.QUIET, vol_short=0.5))
band = meta.rewards_max_spread / 100.0 # 0.03
top_yes = max(q.price for q in tq.quotes if q.token_id == "yes-token" and q.side == Side.BUY)
# bid should be within (band + a tick of rounding) of FV
assert top_yes >= 0.50 - band - meta.tick_size
tests/test_regime.py
+74
View File
@@ -0,0 +1,74 @@
"""Unit tests for the regime state machine."""
from __future__ import annotations
from polymaker.config import StrategyProfile
from polymaker.domain import Regime
from polymaker.strategy.regime import RegimeInputs, RegimeMachine
def _inp(**over):
base = dict(
now=1000.0,
tick=0.01,
fv=0.50,
prev_fv=0.50,
vol_ratio=1.0,
flow_z=0.0,
inventory_util=0.0,
hours_to_end=1000.0,
)
base.update(over)
return RegimeInputs(**base)
def test_default_is_quiet():
p = StrategyProfile()
assert RegimeMachine().decide(_inp(), p) == Regime.QUIET
def test_halts_take_priority():
p = StrategyProfile()
m = RegimeMachine()
assert m.decide(_inp(risk_halt=True), p) == Regime.HALTED
assert m.decide(_inp(ws_stale=True), p) == Regime.HALTED
assert m.decide(_inp(market_resolved=True), p) == Regime.HALTED
assert m.decide(_inp(hours_to_end=1.0), p) == Regime.HALTED # inside halt window
def test_fv_jump_triggers_event_and_cooloff():
p = StrategyProfile() # event_jump_ticks=8, cooloff=60
m = RegimeMachine()
# jump of 0.10 = 10 ticks > 8 -> EVENT
assert m.decide(_inp(fv=0.60, prev_fv=0.50), p) == Regime.EVENT
# still in cooloff a moment later even without a jump
assert m.decide(_inp(now=1030.0, fv=0.60, prev_fv=0.60), p) == Regime.EVENT
# after cooloff expires, back to quiet
assert m.decide(_inp(now=1100.0, fv=0.60, prev_fv=0.60), p) == Regime.QUIET
def test_sweep_flag_triggers_event():
p = StrategyProfile()
assert RegimeMachine().decide(_inp(sweep_flagged=True), p) == Regime.EVENT
def test_reduce_only_from_inventory_and_enddate():
p = StrategyProfile() # reduce_only_hours=24
m = RegimeMachine()
assert m.decide(_inp(inventory_util=1.0), p) == Regime.REDUCE_ONLY
assert m.decide(_inp(risk_reduce_only=True), p) == Regime.REDUCE_ONLY
assert m.decide(_inp(hours_to_end=12.0), p) == Regime.REDUCE_ONLY
def test_trending_from_flow_and_vol():
p = StrategyProfile() # trend_flow_z=1.5
m = RegimeMachine()
assert m.decide(_inp(flow_z=2.0), p) == Regime.TRENDING
assert m.decide(_inp(vol_ratio=3.0), p) == Regime.TRENDING
def test_event_beats_reduce_only_and_trending():
p = StrategyProfile()
m = RegimeMachine()
r = m.decide(_inp(sweep_flagged=True, inventory_util=1.0, flow_z=5.0), p)
assert r == Regime.EVENT
tests/test_risk.py
+86
View File
@@ -0,0 +1,86 @@
"""Tests for the RiskManager gates and circuit breakers."""
from __future__ import annotations
from polymaker.config import RiskConfig
from polymaker.domain import Fill, Side
from polymaker.risk.manager import RiskManager
from polymaker.state.store import StateStore
def _rm(tmp_path, **over):
cfg = RiskConfig(**{
"max_total_exposure_usdc": 5000, "max_market_notional_usdc": 800,
"max_event_group_loss_usdc": 1000, "daily_loss_kill_usdc": 250,
**over,
})
store = StateStore(tmp_path / "s.db")
return RiskManager(cfg, store), store
def test_daily_loss_kill_switch(tmp_path, meta):
rm, store = _rm(tmp_path)
# buy 1000 shares @ 0.50 -> -500 cash, +1000 inventory
store.apply_fill(Fill(meta.yes.token_id, Side.BUY, 0.50, 1000, "t1"))
rm.note_fill(Fill(meta.yes.token_id, Side.BUY, 0.50, 1000, "t1"))
rm.update_mark(meta.yes.token_id, 0.50)
rm.reset_day()
assert rm.global_halt()[0] is False
# fair value collapses to 0.20 -> unrealized loss 300 > 250 kill
rm.update_mark(meta.yes.token_id, 0.20)
halted, why = rm.global_halt()
assert halted and "daily_loss" in why
store.close()
def test_market_cap_triggers_reduce_only(tmp_path, meta):
rm, store = _rm(tmp_path, max_market_notional_usdc=100)
store.apply_fill(Fill(meta.yes.token_id, Side.BUY, 0.50, 300, "t1")) # 150 notional > 100
rm.update_mark(meta.yes.token_id, 0.50)
rm.update_mark(meta.no.token_id, 0.50)
d = rm.evaluate(meta, ws_stale=False, event_group_cost=0.0)
assert d.reduce_only and d.reason == "market_cap"
store.close()
def test_ws_stale_halts_market(tmp_path, meta):
rm, store = _rm(tmp_path)
d = rm.evaluate(meta, ws_stale=True, event_group_cost=0.0)
assert d.halt and d.reason == "ws_stale"
store.close()
def test_size_scale_tapers_near_cap(tmp_path, meta):
rm, store = _rm(tmp_path, max_market_notional_usdc=100)
# 85 notional -> 85% of cap -> should scale below 1.0 but not reduce-only
store.apply_fill(Fill(meta.yes.token_id, Side.BUY, 0.50, 170, "t1")) # 85 notional
rm.update_mark(meta.yes.token_id, 0.50)
rm.update_mark(meta.no.token_id, 0.50)
d = rm.evaluate(meta, ws_stale=False, event_group_cost=0.0)
assert not d.reduce_only
assert 0.0 < d.size_scale < 1.0
store.close()
def test_event_group_cap(tmp_path, meta):
rm, store = _rm(tmp_path, max_event_group_loss_usdc=50)
d = rm.evaluate(meta, ws_stale=False, event_group_cost=60.0)
assert d.reduce_only and d.reason == "event_group_cap"
store.close()
def test_error_rate_breaker(tmp_path, meta):
rm, store = _rm(tmp_path, max_order_error_rate=0.25)
for _ in range(15):
rm.note_order_result(False)
for _ in range(10):
rm.note_order_result(True) # 15/25 = 0.6 > 0.25
assert rm.global_halt()[0] is True
store.close()
def test_manual_kill(tmp_path, meta):
rm, store = _rm(tmp_path)
rm.kill()
assert rm.global_halt() == (True, "manual_kill")
store.close()
tests/test_state.py
+171
View File
@@ -0,0 +1,171 @@
"""Tests for StateStore, the user-event tracker, and the reconciler."""
from __future__ import annotations
from polymaker.domain import (
Fill,
OpenOrder,
OrderState,
Quote,
Regime,
Side,
TargetQuotes,
TradeState,
)
from polymaker.execution.reconciler import reconcile
from polymaker.state.store import StateStore
from polymaker.state.tracker import OrderEvent, TradeEvent, UserEventProcessor
# ── StateStore ──────────────────────────────────────────────────────────────
def test_apply_fill_updates_size_and_avg(tmp_path):
s = StateStore(tmp_path / "s.db")
s.apply_fill(Fill("tok", Side.BUY, 0.50, 100, "t1"))
assert s.position("tok").size == 100
assert s.position("tok").avg_price == 0.50
# buy more at a higher price -> weighted avg
s.apply_fill(Fill("tok", Side.BUY, 0.60, 100, "t2"))
assert s.position("tok").size == 200
assert abs(s.position("tok").avg_price - 0.55) < 1e-9
# sell reduces size, avg unchanged
s.apply_fill(Fill("tok", Side.SELL, 0.70, 50, "t3"))
assert s.position("tok").size == 150
assert abs(s.position("tok").avg_price - 0.55) < 1e-9
s.close()
def test_sell_to_flat_resets_avg(tmp_path):
s = StateStore(tmp_path / "s.db")
s.apply_fill(Fill("tok", Side.BUY, 0.5, 100, "t1"))
s.apply_fill(Fill("tok", Side.SELL, 0.6, 100, "t2"))
assert s.position("tok").size == 0
assert s.position("tok").avg_price == 0.0
s.close()
def test_reconcile_positions_skips_inflight_and_recent(tmp_path):
s = StateStore(tmp_path / "s.db")
s.mark_inflight("tok")
s.reconcile_positions({"tok": (999.0, 0.9)}) # ignored: in-flight
assert s.position("tok").size == 0
s.clear_inflight("tok")
# still recent fill guard: simulate no recent fill by using a fresh token
s.reconcile_positions({"other": (42.0, 0.3)})
assert s.position("other").size == 42.0
s.close()
def test_state_persists_across_restart(tmp_path):
db = tmp_path / "s.db"
s = StateStore(db)
s.apply_fill(Fill("tok", Side.BUY, 0.5, 100, "t1"))
s.close()
s2 = StateStore(db)
assert s2.position("tok").size == 100
s2.close()
# ── UserEventProcessor ───────────────────────────────────────────────────────
def test_matched_then_confirmed(tmp_path):
s = StateStore(tmp_path / "s.db")
changed: list[str] = []
p = UserEventProcessor(s, on_change=changed.append)
p.on_trade(TradeEvent("tok", Side.BUY, 0.5, 100, "trade1", TradeState.MATCHED, 1.0), "cid")
assert s.position("tok").size == 100
assert s.inflight("tok") == 1
p.on_trade(TradeEvent("tok", Side.BUY, 0.5, 100, "trade1", TradeState.CONFIRMED, 2.0), "cid")
assert s.inflight("tok") == 0
assert s.position("tok").size == 100 # settled
assert changed == ["cid", "cid"]
s.close()
def test_matched_is_idempotent(tmp_path):
s = StateStore(tmp_path / "s.db")
p = UserEventProcessor(s)
ev = TradeEvent("tok", Side.BUY, 0.5, 100, "trade1", TradeState.MATCHED, 1.0)
p.on_trade(ev, "cid")
p.on_trade(ev, "cid") # duplicate MATCHED for same trade id
assert s.position("tok").size == 100 # not doubled
s.close()
def test_failed_trade_reverses_fill(tmp_path):
s = StateStore(tmp_path / "s.db")
p = UserEventProcessor(s)
p.on_trade(TradeEvent("tok", Side.BUY, 0.5, 100, "trade1", TradeState.MATCHED, 1.0), "cid")
assert s.position("tok").size == 100
p.on_trade(TradeEvent("tok", Side.BUY, 0.5, 100, "trade1", TradeState.FAILED, 2.0), "cid")
assert s.position("tok").size == 0 # rolled back
assert s.inflight("tok") == 0
s.close()
def test_order_event_upsert_and_cancel(tmp_path):
s = StateStore(tmp_path / "s.db")
p = UserEventProcessor(s)
p.on_order(OrderEvent("o1", "tok", Side.BUY, 0.49, 100), "cid")
assert len(s.orders_for("tok")) == 1
p.on_order(OrderEvent("o1", "tok", Side.BUY, 0.49, 0, is_cancel=True), "cid")
assert len(s.orders_for("tok")) == 0
s.close()
# ── reconciler ───────────────────────────────────────────────────────────────
def _live(order_id, token, side, price, size):
return OpenOrder(order_id, token, side, price, size, OrderState.LIVE)
def test_reconcile_places_when_no_live():
tq = TargetQuotes("cid", Regime.QUIET, (Quote("tok", Side.BUY, 0.49, 100),))
plan = reconcile(tq, [], tick=0.01, reprice_ticks=2, resize_frac=0.15)
assert len(plan.to_place) == 1
assert plan.to_cancel == []
def test_reconcile_keeps_close_order():
tq = TargetQuotes("cid", Regime.QUIET, (Quote("tok", Side.BUY, 0.49, 100),))
live = [_live("o1", "tok", Side.BUY, 0.49, 102)] # within tolerances
plan = reconcile(tq, live, tick=0.01, reprice_ticks=2, resize_frac=0.15)
assert plan.is_noop
def test_reconcile_reprices_when_far():
tq = TargetQuotes("cid", Regime.QUIET, (Quote("tok", Side.BUY, 0.45, 100),))
live = [_live("o1", "tok", Side.BUY, 0.49, 100)] # 4 ticks away > 2
plan = reconcile(tq, live, tick=0.01, reprice_ticks=2, resize_frac=0.15)
assert plan.to_cancel == ["o1"]
assert len(plan.to_place) == 1
def test_reconcile_resizes_when_size_drifts():
tq = TargetQuotes("cid", Regime.QUIET, (Quote("tok", Side.BUY, 0.49, 100),))
live = [_live("o1", "tok", Side.BUY, 0.49, 50)] # 50% smaller > 15%
plan = reconcile(tq, live, tick=0.01, reprice_ticks=2, resize_frac=0.15)
assert plan.to_cancel == ["o1"]
assert len(plan.to_place) == 1
def test_reconcile_cancels_all_when_target_empty():
tq = TargetQuotes("cid", Regime.EVENT, ())
live = [_live("o1", "tok", Side.BUY, 0.49, 100), _live("o2", "tok", Side.SELL, 0.55, 50)]
plan = reconcile(tq, live, tick=0.01, reprice_ticks=2, resize_frac=0.15)
assert set(plan.to_cancel) == {"o1", "o2"}
assert plan.to_place == []
def test_reconcile_matches_layers_one_to_one():
tq = TargetQuotes("cid", Regime.QUIET, (
Quote("tok", Side.BUY, 0.49, 100),
Quote("tok", Side.BUY, 0.47, 100),
))
live = [_live("o1", "tok", Side.BUY, 0.49, 100)] # only the top layer exists
plan = reconcile(tq, live, tick=0.01, reprice_ticks=2, resize_frac=0.15)
assert plan.to_cancel == []
assert len(plan.to_place) == 1 # only the missing deeper layer
assert plan.to_place[0].price == 0.47
tests/test_userstream_parse.py
+88
View File
@@ -0,0 +1,88 @@
"""Tests for user-WS normalization (maker fill extraction + order tracking).
Frames modeled on v1's observed shape; reconfirm field names in the wallet spike.
"""
from __future__ import annotations
from polymaker.domain import Side, TradeState
from polymaker.userstream.parse import normalize_order, normalize_trade
OUR = "0xMyWallet"
def _other(token: str) -> str | None:
return {"yes-tok": "no-tok", "no-tok": "yes-tok"}.get(token)
def test_maker_same_outcome_is_a_sell():
# taker BUYs YES; we are the maker on YES -> we SELL YES
msg = {
"event_type": "trade",
"market": "0xcond",
"asset_id": "yes-tok",
"side": "BUY",
"outcome": "Yes",
"status": "MATCHED",
"id": "trade1",
"timestamp": "1700000000000",
"maker_orders": [
{"maker_address": OUR, "matched_amount": "50", "price": "0.49", "outcome": "Yes"}
],
}
evs = normalize_trade(msg, OUR, _other)
assert len(evs) == 1
ev = evs[0]
assert ev.token_id == "yes-tok"
assert ev.our_side is Side.SELL
assert ev.size == 50 and ev.price == 0.49
assert ev.status is TradeState.MATCHED
def test_maker_different_outcome_is_a_mint_buy():
# taker BUYs YES; we are the maker on NO -> a mint: we BUY NO
msg = {
"event_type": "trade",
"market": "0xcond",
"asset_id": "yes-tok",
"side": "BUY",
"outcome": "Yes",
"status": "MATCHED",
"id": "trade2",
"timestamp": "1700000000000",
"maker_orders": [
{"maker_address": OUR, "matched_amount": "30", "price": "0.51", "outcome": "No"}
],
}
evs = normalize_trade(msg, OUR, _other)
assert len(evs) == 1
assert evs[0].token_id == "no-tok"
assert evs[0].our_side is Side.BUY
assert evs[0].size == 30
def test_ignores_maker_orders_that_are_not_ours():
msg = {
"event_type": "trade", "market": "0xcond", "asset_id": "yes-tok", "side": "BUY",
"outcome": "Yes", "status": "MATCHED", "id": "t", "timestamp": "1700000000000",
"maker_orders": [
{"maker_address": "0xSomeoneElse", "matched_amount": "50", "price": "0.49", "outcome": "Yes"}
],
}
assert normalize_trade(msg, OUR, _other) == []
def test_normalize_order_remaining_and_cancel():
ev = normalize_order({
"event_type": "order", "asset_id": "yes-tok", "side": "BUY", "price": "0.49",
"original_size": "100", "size_matched": "40", "status": "LIVE", "id": "o1",
})
assert ev is not None
assert ev.remaining_size == 60
assert ev.is_cancel is False
cancel = normalize_order({
"event_type": "order", "asset_id": "yes-tok", "side": "BUY", "price": "0.49",
"original_size": "100", "size_matched": "0", "status": "CANCELED", "id": "o1",
})
assert cancel is not None and cancel.is_cancel is True
trading.py
-471
View File
@@ -1,471 +0,0 @@
import gc # Garbage collection
import os # Operating system interface
import json # JSON handling
import asyncio # Asynchronous I/O
import traceback # Exception handling
import pandas as pd # Data analysis library
import math # Mathematical functions
import poly_data.global_state as global_state
import poly_data.CONSTANTS as CONSTANTS
# Import utility functions for trading
from poly_data.trading_utils import get_best_bid_ask_deets, get_order_prices, get_buy_sell_amount, round_down, round_up
from poly_data.data_utils import get_position, get_order, set_position
# Create directory for storing position risk information
if not os.path.exists('positions/'):
os.makedirs('positions/')
def send_buy_order(order):
"""
Create a BUY order for a specific token.
This function:
1. Cancels any existing orders for the token
2. Checks if the order price is within acceptable range
3. Creates a new buy order if conditions are met
Args:
order (dict): Order details including token, price, size, and market parameters
"""
client = global_state.client
# Only cancel existing orders if we need to make significant changes
existing_buy_size = order['orders']['buy']['size']
existing_buy_price = order['orders']['buy']['price']
# Cancel orders if price changed significantly or size needs major adjustment
price_diff = abs(existing_buy_price - order['price']) if existing_buy_price > 0 else float('inf')
size_diff = abs(existing_buy_size - order['size']) if existing_buy_size > 0 else float('inf')
should_cancel = (
price_diff > 0.005 or # Cancel if price diff > 0.5 cents
size_diff > order['size'] * 0.1 or # Cancel if size diff > 10%
existing_buy_size == 0 # Cancel if no existing buy order
)
if should_cancel and (existing_buy_size > 0 or order['orders']['sell']['size'] > 0):
print(f"Cancelling buy orders - price diff: {price_diff:.4f}, size diff: {size_diff:.1f}")
client.cancel_all_asset(order['token'])
elif not should_cancel:
print(f"Keeping existing buy orders - minor changes: price diff: {price_diff:.4f}, size diff: {size_diff:.1f}")
return # Don't place new order if existing one is fine
# Calculate minimum acceptable price based on market spread
incentive_start = order['mid_price'] - order['max_spread']/100
trade = True
# Don't place orders that are below incentive threshold
if order['price'] < incentive_start:
trade = False
if trade:
# Only place orders with prices between 0.1 and 0.9 to avoid extreme positions
if order['price'] >= 0.1 and order['price'] < 0.9:
print(f'Creating new order for {order["size"]} at {order["price"]}')
print(order['token'], 'BUY', order['price'], order['size'])
client.create_order(
order['token'],
'BUY',
order['price'],
order['size'],
True if order['neg_risk'] == 'TRUE' else False
)
else:
print("Not creating buy order because its outside acceptable price range (0.1-0.9)")
else:
print(f'Not creating new order because order price of {order["price"]} is less than incentive start price of {incentive_start}. Mid price is {order["mid_price"]}')
def send_sell_order(order):
"""
Create a SELL order for a specific token.
This function:
1. Cancels any existing orders for the token
2. Creates a new sell order with the specified parameters
Args:
order (dict): Order details including token, price, size, and market parameters
"""
client = global_state.client
# Only cancel existing orders if we need to make significant changes
existing_sell_size = order['orders']['sell']['size']
existing_sell_price = order['orders']['sell']['price']
# Cancel orders if price changed significantly or size needs major adjustment
price_diff = abs(existing_sell_price - order['price']) if existing_sell_price > 0 else float('inf')
size_diff = abs(existing_sell_size - order['size']) if existing_sell_size > 0 else float('inf')
should_cancel = (
price_diff > 0.005 or # Cancel if price diff > 0.5 cents
size_diff > order['size'] * 0.1 or # Cancel if size diff > 10%
existing_sell_size == 0 # Cancel if no existing sell order
)
if should_cancel and (existing_sell_size > 0 or order['orders']['buy']['size'] > 0):
print(f"Cancelling sell orders - price diff: {price_diff:.4f}, size diff: {size_diff:.1f}")
client.cancel_all_asset(order['token'])
elif not should_cancel:
print(f"Keeping existing sell orders - minor changes: price diff: {price_diff:.4f}, size diff: {size_diff:.1f}")
return # Don't place new order if existing one is fine
print(f'Creating new order for {order["size"]} at {order["price"]}')
client.create_order(
order['token'],
'SELL',
order['price'],
order['size'],
True if order['neg_risk'] == 'TRUE' else False
)
# Dictionary to store locks for each market to prevent concurrent trading on the same market
market_locks = {}
async def perform_trade(market):
"""
Main trading function that handles market making for a specific market.
This function:
1. Merges positions when possible to free up capital
2. Analyzes the market to determine optimal bid/ask prices
3. Manages buy and sell orders based on position size and market conditions
4. Implements risk management with stop-loss and take-profit logic
Args:
market (str): The market ID to trade on
"""
# Create a lock for this market if it doesn't exist
if market not in market_locks:
market_locks[market] = asyncio.Lock()
# Use lock to prevent concurrent trading on the same market
async with market_locks[market]:
try:
client = global_state.client
# Get market details from the configuration
row = global_state.df[global_state.df['condition_id'] == market].iloc[0]
# Determine decimal precision from tick size
round_length = len(str(row['tick_size']).split(".")[1])
# Get trading parameters for this market type
params = global_state.params[row['param_type']]
# Create a list with both outcomes for the market
deets = [
{'name': 'token1', 'token': row['token1'], 'answer': row['answer1']},
{'name': 'token2', 'token': row['token2'], 'answer': row['answer2']}
]
print(f"\n\n{pd.Timestamp.utcnow().tz_localize(None)}: {row['question']}")
# Get current positions for both outcomes
pos_1 = get_position(row['token1'])['size']
pos_2 = get_position(row['token2'])['size']
# ------- POSITION MERGING LOGIC -------
# Calculate if we have opposing positions that can be merged
amount_to_merge = min(pos_1, pos_2)
# Only merge if positions are above minimum threshold
if float(amount_to_merge) > CONSTANTS.MIN_MERGE_SIZE:
# Get exact position sizes from blockchain for merging
pos_1 = client.get_position(row['token1'])[0]
pos_2 = client.get_position(row['token2'])[0]
amount_to_merge = min(pos_1, pos_2)
scaled_amt = amount_to_merge / 10**6
if scaled_amt > CONSTANTS.MIN_MERGE_SIZE:
print(f"Position 1 is of size {pos_1} and Position 2 is of size {pos_2}. Merging positions")
# Execute the merge operation
client.merge_positions(amount_to_merge, market, row['neg_risk'] == 'TRUE')
# Update our local position tracking
set_position(row['token1'], 'SELL', scaled_amt, 0, 'merge')
set_position(row['token2'], 'SELL', scaled_amt, 0, 'merge')
# ------- TRADING LOGIC FOR EACH OUTCOME -------
# Loop through both outcomes in the market (YES and NO)
for detail in deets:
token = int(detail['token'])
# Get current orders for this token
orders = get_order(token)
# Get market depth and price information
deets = get_best_bid_ask_deets(market, detail['name'], 100, 0.1)
#if deet has None for one these values below, call it with min size of 20
if deets['best_bid'] is None or deets['best_ask'] is None or deets['best_bid_size'] is None or deets['best_ask_size'] is None:
deets = get_best_bid_ask_deets(market, detail['name'], 20, 0.1)
# Extract all order book details
best_bid = deets['best_bid']
best_bid_size = deets['best_bid_size']
second_best_bid = deets['second_best_bid']
second_best_bid_size = deets['second_best_bid_size']
top_bid = deets['top_bid']
best_ask = deets['best_ask']
best_ask_size = deets['best_ask_size']
second_best_ask = deets['second_best_ask']
second_best_ask_size = deets['second_best_ask_size']
top_ask = deets['top_ask']
# Round prices to appropriate precision
best_bid = round(best_bid, round_length)
best_ask = round(best_ask, round_length)
# Calculate ratio of buy vs sell liquidity in the market
try:
overall_ratio = (deets['bid_sum_within_n_percent']) / (deets['ask_sum_within_n_percent'])
except:
overall_ratio = 0
try:
second_best_bid = round(second_best_bid, round_length)
second_best_ask = round(second_best_ask, round_length)
except:
pass
top_bid = round(top_bid, round_length)
top_ask = round(top_ask, round_length)
# Get our current position and average price
pos = get_position(token)
position = pos['size']
avgPrice = pos['avgPrice']
position = round_down(position, 2)
# Calculate optimal bid and ask prices based on market conditions
bid_price, ask_price = get_order_prices(
best_bid, best_bid_size, top_bid, best_ask,
best_ask_size, top_ask, avgPrice, row
)
bid_price = round(bid_price, round_length)
ask_price = round(ask_price, round_length)
# Calculate mid price for reference
mid_price = (top_bid + top_ask) / 2
# Log market conditions for this outcome
print(f"\nFor {detail['answer']}. Orders: {orders} Position: {position}, "
f"avgPrice: {avgPrice}, Best Bid: {best_bid}, Best Ask: {best_ask}, "
f"Bid Price: {bid_price}, Ask Price: {ask_price}, Mid Price: {mid_price}")
# Get position for the opposite token to calculate total exposure
other_token = global_state.REVERSE_TOKENS[str(token)]
other_position = get_position(other_token)['size']
# Calculate how much to buy or sell based on our position
buy_amount, sell_amount = get_buy_sell_amount(position, bid_price, row, other_position)
# Get max_size for logging (same logic as in get_buy_sell_amount)
max_size = row.get('max_size', row['trade_size'])
# Prepare order object with all necessary information
order = {
"token": token,
"mid_price": mid_price,
"neg_risk": row['neg_risk'],
"max_spread": row['max_spread'],
'orders': orders,
'token_name': detail['name'],
'row': row
}
print(f"Position: {position}, Other Position: {other_position}, "
f"Trade Size: {row['trade_size']}, Max Size: {max_size}, "
f"buy_amount: {buy_amount}, sell_amount: {sell_amount}")
# File to store risk management information for this market
fname = 'positions/' + str(market) + '.json'
# ------- SELL ORDER LOGIC -------
if sell_amount > 0:
# Skip if we have no average price (no real position)
if avgPrice == 0:
print("Avg Price is 0. Skipping")
continue
order['size'] = sell_amount
order['price'] = ask_price
# Get fresh market data for risk assessment
n_deets = get_best_bid_ask_deets(market, detail['name'], 100, 0.1)
# Calculate current market price and spread
mid_price = round_up((n_deets['best_bid'] + n_deets['best_ask']) / 2, round_length)
spread = round(n_deets['best_ask'] - n_deets['best_bid'], 2)
# Calculate current profit/loss on position
pnl = (mid_price - avgPrice) / avgPrice * 100
print(f"Mid Price: {mid_price}, Spread: {spread}, PnL: {pnl}")
# Prepare risk details for tracking
risk_details = {
'time': str(pd.Timestamp.utcnow().tz_localize(None)),
'question': row['question']
}
try:
ratio = (n_deets['bid_sum_within_n_percent']) / (n_deets['ask_sum_within_n_percent'])
except:
ratio = 0
pos_to_sell = sell_amount # Amount to sell in risk-off scenario
# ------- STOP-LOSS LOGIC -------
# Trigger stop-loss if either:
# 1. PnL is below threshold and spread is tight enough to exit
# 2. Volatility is too high
if (pnl < params['stop_loss_threshold'] and spread <= params['spread_threshold']) or row['3_hour'] > params['volatility_threshold']:
risk_details['msg'] = (f"Selling {pos_to_sell} because spread is {spread} and pnl is {pnl} "
f"and ratio is {ratio} and 3 hour volatility is {row['3_hour']}")
print("Stop loss Triggered: ", risk_details['msg'])
# Sell at market best bid to ensure execution
order['size'] = pos_to_sell
order['price'] = n_deets['best_bid']
# Set period to avoid trading after stop-loss
risk_details['sleep_till'] = str(pd.Timestamp.utcnow().tz_localize(None) +
pd.Timedelta(hours=params['sleep_period']))
print("Risking off")
send_sell_order(order)
client.cancel_all_market(market)
# Save risk details to file
open(fname, 'w').write(json.dumps(risk_details))
continue
# ------- BUY ORDER LOGIC -------
# Get max_size, defaulting to trade_size if not specified
max_size = row.get('max_size', row['trade_size'])
# Only buy if:
# 1. Position is less than max_size (new logic)
# 2. Position is less than absolute cap (250)
# 3. Buy amount is above minimum size
if position < max_size and position < 250 and buy_amount > 0 and buy_amount >= row['min_size']:
# Get reference price from market data
sheet_value = row['best_bid']
if detail['name'] == 'token2':
sheet_value = 1 - row['best_ask']
sheet_value = round(sheet_value, round_length)
order['size'] = buy_amount
order['price'] = bid_price
# Check if price is far from reference
price_change = abs(order['price'] - sheet_value)
send_buy = True
# ------- RISK-OFF PERIOD CHECK -------
# If we're in a risk-off period (after stop-loss), don't buy
if os.path.isfile(fname):
risk_details = json.load(open(fname))
start_trading_at = pd.to_datetime(risk_details['sleep_till'])
current_time = pd.Timestamp.utcnow().tz_localize(None)
print(risk_details, current_time, start_trading_at)
if current_time < start_trading_at:
send_buy = False
print(f"Not sending a buy order because recently risked off. "
f"Risked off at {risk_details['time']}")
# Only proceed if we're not in risk-off period
if send_buy:
# Don't buy if volatility is high or price is far from reference
if row['3_hour'] > params['volatility_threshold'] or price_change >= 0.05:
print(f'3 Hour Volatility of {row["3_hour"]} is greater than max volatility of '
f'{params["volatility_threshold"]} or price of {order["price"]} is outside '
f'0.05 of {sheet_value}. Cancelling all orders')
client.cancel_all_asset(order['token'])
else:
# Check for reverse position (holding opposite outcome)
rev_token = global_state.REVERSE_TOKENS[str(token)]
rev_pos = get_position(rev_token)
# If we have significant opposing position, don't buy more
if rev_pos['size'] > row['min_size']:
print("Bypassing creation of new buy order because there is a reverse position")
if orders['buy']['size'] > CONSTANTS.MIN_MERGE_SIZE:
print("Cancelling buy orders because there is a reverse position")
client.cancel_all_asset(order['token'])
continue
# Check market buy/sell volume ratio
if overall_ratio < 0:
send_buy = False
print(f"Not sending a buy order because overall ratio is {overall_ratio}")
client.cancel_all_asset(order['token'])
else:
# Place new buy order if any of these conditions are met:
# 1. We can get a better price than current order
if best_bid > orders['buy']['price']:
print(f"Sending Buy Order for {token} because better price. "
f"Orders look like this: {orders['buy']}. Best Bid: {best_bid}")
send_buy_order(order)
# 2. Current position + orders is not enough to reach max_size
elif position + orders['buy']['size'] < 0.95 * max_size:
print(f"Sending Buy Order for {token} because not enough position + size")
send_buy_order(order)
# 3. Our current order is too large and needs to be resized
elif orders['buy']['size'] > order['size'] * 1.01:
print(f"Resending buy orders because open orders are too large")
send_buy_order(order)
# Commented out logic for cancelling orders when market conditions change
# elif best_bid_size < orders['buy']['size'] * 0.98 and abs(best_bid - second_best_bid) > 0.03:
# print(f"Cancelling buy orders because best size is less than 90% of open orders and spread is too large")
# global_state.client.cancel_all_asset(order['token'])
# ------- TAKE PROFIT / SELL ORDER MANAGEMENT -------
elif sell_amount > 0:
order['size'] = sell_amount
# Calculate take-profit price based on average cost
tp_price = round_up(avgPrice + (avgPrice * params['take_profit_threshold']/100), round_length)
order['price'] = round_up(tp_price if ask_price < tp_price else ask_price, round_length)
tp_price = float(tp_price)
order_price = float(orders['sell']['price'])
# Calculate % difference between current order and ideal price
diff = abs(order_price - tp_price)/tp_price * 100
# Update sell order if:
# 1. Current order price is significantly different from target
if diff > 2:
print(f"Sending Sell Order for {token} because better current order price of "
f"{order_price} is deviant from the tp_price of {tp_price} and diff is {diff}")
send_sell_order(order)
# 2. Current order size is too small for our position
elif orders['sell']['size'] < position * 0.97:
print(f"Sending Sell Order for {token} because not enough sell size. "
f"Position: {position}, Sell Size: {orders['sell']['size']}")
send_sell_order(order)
# Commented out additional conditions for updating sell orders
# elif orders['sell']['price'] < ask_price:
# print(f"Updating Sell Order for {token} because its not at the right price")
# send_sell_order(order)
# elif best_ask_size < orders['sell']['size'] * 0.98 and abs(best_ask - second_best_ask) > 0.03...:
# print(f"Cancelling sell orders because best size is less than 90% of open orders...")
# send_sell_order(order)
except Exception as ex:
print(f"Error performing trade for {market}: {ex}")
traceback.print_exc()
# Clean up memory and introduce a small delay
gc.collect()
await asyncio.sleep(2)
update_markets.py
-133
View File
@@ -1,133 +0,0 @@
import time
import pandas as pd
from data_updater.trading_utils import get_clob_client
from data_updater.google_utils import get_spreadsheet
from data_updater.find_markets import get_sel_df, get_all_markets, get_all_results, get_markets, add_volatility_to_df
from gspread_dataframe import set_with_dataframe
import traceback
# Initialize global variables
spreadsheet = get_spreadsheet()
client = get_clob_client()
wk_all = spreadsheet.worksheet("All Markets")
wk_vol = spreadsheet.worksheet("Volatility Markets")
sel_df = get_sel_df(spreadsheet, "Selected Markets")
def update_sheet(data, worksheet):
all_values = worksheet.get_all_values()
existing_num_rows = len(all_values)
existing_num_cols = len(all_values[0]) if all_values else 0
num_rows, num_cols = data.shape
max_rows = max(num_rows, existing_num_rows)
max_cols = max(num_cols, existing_num_cols)
# Create a DataFrame with the maximum size and fill it with empty strings
padded_data = pd.DataFrame('', index=range(max_rows), columns=range(max_cols))
# Update the padded DataFrame with the original data and its columns
padded_data.iloc[:num_rows, :num_cols] = data.values
padded_data.columns = list(data.columns) + [''] * (max_cols - num_cols)
# Update the sheet with the padded DataFrame, including column headers
set_with_dataframe(worksheet, padded_data, include_index=False, include_column_header=True, resize=True)
def sort_df(df):
# Calculate the mean and standard deviation for each column
mean_gm = df['gm_reward_per_100'].mean()
std_gm = df['gm_reward_per_100'].std()
mean_volatility = df['volatility_sum'].mean()
std_volatility = df['volatility_sum'].std()
# Standardize the columns
df['std_gm_reward_per_100'] = (df['gm_reward_per_100'] - mean_gm) / std_gm
df['std_volatility_sum'] = (df['volatility_sum'] - mean_volatility) / std_volatility
# Define a custom scoring function for best_bid and best_ask
def proximity_score(value):
if 0.1 <= value <= 0.25:
return (0.25 - value) / 0.15
elif 0.75 <= value <= 0.9:
return (value - 0.75) / 0.15
else:
return 0
df['bid_score'] = df['best_bid'].apply(proximity_score)
df['ask_score'] = df['best_ask'].apply(proximity_score)
# Create a composite score (higher is better for rewards, lower is better for volatility, with proximity scores)
df['composite_score'] = (
df['std_gm_reward_per_100'] -
df['std_volatility_sum'] +
df['bid_score'] +
df['ask_score']
)
# Sort by the composite score in descending order
sorted_df = df.sort_values(by='composite_score', ascending=False)
# Drop the intermediate columns used for calculation
sorted_df = sorted_df.drop(columns=['std_gm_reward_per_100', 'std_volatility_sum', 'bid_score', 'ask_score', 'composite_score'])
return sorted_df
def fetch_and_process_data():
global spreadsheet, client, wk_all, wk_vol, sel_df
spreadsheet = get_spreadsheet()
client = get_clob_client()
wk_all = spreadsheet.worksheet("All Markets")
wk_vol = spreadsheet.worksheet("Volatility Markets")
wk_full = spreadsheet.worksheet("Full Markets")
sel_df = get_sel_df(spreadsheet, "Selected Markets")
all_df = get_all_markets(client)
print("Got all Markets")
all_results = get_all_results(all_df, client)
print("Got all Results")
m_data, all_markets = get_markets(all_results, sel_df, maker_reward=0.75)
print("Got all orderbook")
print(f'{pd.to_datetime("now")}: Fetched all markets data of length {len(all_markets)}.')
new_df = add_volatility_to_df(all_markets)
new_df['volatility_sum'] = new_df['24_hour'] + new_df['7_day'] + new_df['14_day']
new_df = new_df.sort_values('volatility_sum', ascending=True)
new_df['volatilty/reward'] = ((new_df['gm_reward_per_100'] / new_df['volatility_sum']).round(2)).astype(str)
new_df = new_df[['question', 'answer1', 'answer2', 'spread', 'rewards_daily_rate', 'gm_reward_per_100', 'sm_reward_per_100', 'bid_reward_per_100', 'ask_reward_per_100', 'volatility_sum', 'volatilty/reward', 'min_size', '1_hour', '3_hour', '6_hour', '12_hour', '24_hour', '7_day', '30_day',
'best_bid', 'best_ask', 'volatility_price', 'max_spread', 'tick_size',
'neg_risk', 'market_slug', 'token1', 'token2', 'condition_id']]
volatility_df = new_df.copy()
volatility_df = volatility_df[new_df['volatility_sum'] < 20]
# volatility_df = sort_df(volatility_df)
volatility_df = volatility_df.sort_values('gm_reward_per_100', ascending=False)
new_df = new_df.sort_values('gm_reward_per_100', ascending=False)
print(f'{pd.to_datetime("now")}: Fetched select market of length {len(new_df)}.')
if len(new_df) > 50:
update_sheet(new_df, wk_all)
update_sheet(volatility_df, wk_vol)
update_sheet(m_data, wk_full)
else:
print(f'{pd.to_datetime("now")}: Not updating sheet because of length {len(new_df)}.')
if __name__ == "__main__":
while True:
try:
fetch_and_process_data()
time.sleep(60 * 60) # Sleep for an hour
except Exception as e:
traceback.print_exc()
print(str(e))
update_stats.py
-18
View File
@@ -1,18 +0,0 @@
from poly_data.polymarket_client import PolymarketClient
from poly_stats.account_stats import update_stats_once
import pandas as pd
import time
import traceback
client = PolymarketClient()
if __name__ == '__main__':
while True:
try:
update_stats_once(client)
except Exception as e:
traceback.print_exc()
print("Now sleeping\n")
time.sleep(60 * 60 * 3) #3 hours
uv.lock
Generated
+1209 -2037
View File
File diff suppressed because it is too large Load Diff