358 lines
24 KiB
Markdown
358 lines
24 KiB
Markdown
# Polymarket HFT Market Making System
|
|
|
|
_Python High-Frequency Trading Infrastructure for Prediction Markets_
|
|
|
|
[]()
|
|
[]()
|
|
[]()
|
|
[]()
|
|
|
|
## Overview
|
|
|
|
A sophisticated high-frequency trading (HFT) market making system built for Polymarket's prediction markets. This system uses real-time orderbook analysis, signal generation, and automated execution to provide liquidity while maintaining profitability through intelligent market making strategies.
|
|
|
|
**Key Features:**
|
|
|
|
- **Real-time Market Making**: Automated bid-ask spread management with dynamic pricing
|
|
- **Signal-Based Trading**: BPS (Basis Points Spread) threshold monitoring for entry signals
|
|
- **Intelligent Hedging**: Automatic position hedging to manage risk exposure
|
|
- **Performance Optimization**: CPU affinity control, GC management, and async architecture
|
|
- **Risk Management**: Position limits, profit margins, and automated trade execution
|
|
- **24/7 Operation**: Continuous monitoring with robust error handling and reconnection logic
|
|
|
|
---
|
|
|
|
## System Architecture
|
|
|
|
### Core Components
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ TRADING ENGINE │
|
|
├─────────────────────────────────────────────────────────┤
|
|
│ OrderBook │ Signal Generator │ Risk Manager │
|
|
│ ├─ WebSocket │ ├─ BPS Analysis │ ├─ Position │
|
|
│ ├─ Real-time │ ├─ Threshold │ │ Limits │
|
|
│ │ Updates │ │ Monitoring │ ├─ Profit │
|
|
│ └─ Price Data │ └─ Entry Signals │ │ Margins │
|
|
│ │ │ └─ Exposure │
|
|
├─────────────────────────────────────────────────────────┤
|
|
│ EXECUTION LAYER │
|
|
│ Market Maker │ Hedging System │ Monitoring │
|
|
│ ├─ Order │ ├─ Fill │ ├─ Logging │
|
|
│ │ Management │ │ Detection │ ├─ Performance │
|
|
│ ├─ Spread │ ├─ Automatic │ │ Metrics │
|
|
│ │ Calculation │ │ Hedging │ ├─ Health │
|
|
│ └─ P&L Tracking │ └─ State Mgmt │ │ Checks │
|
|
│ │ │ └─ Alerts │
|
|
└─────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Fast In-Memory Data Store
|
|
|
|
The system includes a custom high-performance in-memory database built in Rust for ultra-low latency data operations.
|
|
|
|
### Architecture
|
|
|
|
The in-memory store consists of two components:
|
|
|
|
1. **Rust Server** ([in_mem_db.rs](in_memory_db/in_mem_db.rs)): High-performance TCP server with thread-safe HashSet operations
|
|
2. **Python Client** ([utils.py](in_memory_db/utils.py)): Fast HTTP client interface for Python integration
|
|
|
|
### Performance Benefits
|
|
|
|
```rust
|
|
// Thread-safe operations with Arc<Mutex<HashSet<String>>>
|
|
struct InMemSetDB {
|
|
data: Arc<Mutex<HashSet<String>>>,
|
|
}
|
|
```
|
|
|
|
**Key Advantages:**
|
|
|
|
- **Ultra-low latency**: Rust-based operations for microsecond response times
|
|
- **Thread-safe**: Concurrent access with proper synchronization
|
|
- **Memory efficient**: Direct memory operations without disk I/O
|
|
- **Lightweight protocol**: Custom HTTP-like protocol minimizes overhead
|
|
- **Connection pooling**: Fast socket connections for rapid queries
|
|
|
|
### Python Integration
|
|
|
|
```python
|
|
from in_memory_db.utils import add_item, contains_item, clear_items, size
|
|
|
|
# Fast operations for trading state management
|
|
add_item("processed_order_123")
|
|
if not contains_item("processed_order_123"):
|
|
# Process order...
|
|
pass
|
|
|
|
# Performance monitoring
|
|
current_size = size()
|
|
clear_items() # Reset state
|
|
```
|
|
|
|
### Use Cases in Trading
|
|
|
|
- **Duplicate Prevention**: Track processed orders to prevent double-execution
|
|
- **State Management**: Fast lookups for order status and hedge tracking
|
|
- **Signal Filtering**: Cache processed market signals to avoid redundant trades
|
|
- **Performance Monitoring**: Real-time metrics storage and retrieval
|
|
|
|
### Starting the In-Memory DB
|
|
|
|
```bash
|
|
# Compile and run the Rust server
|
|
cd in_memory_db
|
|
rustc in_mem_db.rs
|
|
./in_mem_db
|
|
|
|
# Server starts on localhost:8080
|
|
# InMemSetDB started on http://127.0.0.1:8080
|
|
```
|
|
|
|
---
|
|
|
|
## Performance Engineering
|
|
|
|
### 1. Garbage Collection Management
|
|
|
|
```python
|
|
gc.disable() # Eliminates GC pauses during trading
|
|
```
|
|
|
|
**Why:** Python's garbage collector can cause trading delays. We disable it during active trading and manually trigger cleanup during quiet periods.
|
|
|
|
### 2. CPU Optimization
|
|
|
|
```python
|
|
def set_cpu_affinity():
|
|
affinity_cores = [cpu_count - 2, cpu_count - 1] # Use dedicated cores
|
|
process.cpu_affinity(affinity_cores)
|
|
process.nice(psutil.HIGH_PRIORITY_CLASS) # High priority
|
|
```
|
|
|
|
**Benefits:**
|
|
|
|
- Dedicated CPU cores for trading operations
|
|
- High process priority for consistent performance
|
|
- Reduced interference from other system processes
|
|
|
|
### 3. Async Architecture
|
|
|
|
```python
|
|
# WebSocket on separate thread
|
|
self.thread = threading.Thread(target=self._connect, daemon=True)
|
|
# Order processing with async
|
|
await asyncio.create_task(send_hedge(...))
|
|
```
|
|
|
|
**Advantages:**
|
|
|
|
- Non-blocking network operations
|
|
- Concurrent order processing
|
|
- Thread-safe state management
|
|
|
|
### 4. Efficient Data Processing
|
|
|
|
```python
|
|
def _on_message(self, ws, message):
|
|
data = json.loads(message) # Fast JSON parsing
|
|
if event_type == "price_change":
|
|
self._update_orderbook_incremental(data) # Efficient updates
|
|
```
|
|
|
|
---
|
|
|
|
## Trading Strategy
|
|
|
|
### Market Making Approach
|
|
|
|
The system employs a market making strategy that focuses on:
|
|
|
|
1. **Spread Analysis**: Monitors bid-ask spreads and identifies profitable opportunities
|
|
2. **BPS Threshold Trading**: Uses configurable basis point thresholds (default: 50 BPS) to trigger trades
|
|
3. **Automated Hedging**: Places hedge orders automatically to manage risk exposure
|
|
4. **Position Limits**: Enforces maximum trade limits (configurable via `MAX_TRADES`)
|
|
|
|
### Signal Generation
|
|
|
|
```python
|
|
# BPS-based signal detection
|
|
if current_spread_bps >= TRADING_BPS_THRESHOLD:
|
|
# Generate trading signal
|
|
place_anchor_and_hedge(market_data)
|
|
```
|
|
|
|
**Key Parameters:**
|
|
|
|
- `TRADING_BPS_THRESHOLD`: Minimum spread required to trigger a trade (50 BPS default)
|
|
- `PROFIT_MARGIN`: Minimum profit margin per trade (2% default)
|
|
- `MAX_TRADES`: Maximum concurrent positions (2 default)
|
|
|
|
### Risk Management
|
|
|
|
- **Position Sizing**: Controlled position sizes based on available capital
|
|
- **Automatic Hedging**: Every market making trade is automatically hedged
|
|
- **Market Time Windows**: Trades only during active market sessions
|
|
- **Stop-Loss Protection**: Built-in safeguards against adverse moves
|
|
|
|
---
|
|
|
|
## Monitoring & Logging
|
|
|
|
The system provides comprehensive logging and monitoring capabilities:
|
|
|
|
### Log Files
|
|
|
|
- **[logs/](logs/)**: Trading activity logs
|
|
|
|
### Performance Monitoring
|
|
|
|
- Real-time orderbook updates
|
|
- Trading signal generation logs
|
|
- P&L tracking and reporting
|
|
- System health monitoring
|
|
|
|
### Debug Information
|
|
|
|
Enable detailed logging by modifying the logger configuration in [utils/logger.py](utils/logger.py).
|
|
|
|
---
|
|
|
|
## License & Disclaimer
|
|
|
|
**License**: This project is for educational and research purposes only.
|
|
|
|
**Important Disclaimers:**
|
|
|
|
- This software is provided as-is for educational purposes
|
|
- Trading involves substantial risk of financial loss
|
|
- Users are responsible for compliance with applicable financial regulations
|
|
- Past performance does not guarantee future results
|
|
- The authors are not responsible for any financial losses incurred
|
|
- Use at your own risk and ensure proper testing before live trading
|
|
|
|
---
|
|
|
|
## Installation & Setup
|
|
|
|
### Prerequisites
|
|
|
|
```bash
|
|
# System Requirements
|
|
Python 3.11+
|
|
4+ CPU cores recommended
|
|
2GB+ RAM for orderbook processing
|
|
Stable internet connection
|
|
```
|
|
|
|
### Installation
|
|
|
|
1. **Clone the repository:**
|
|
|
|
```bash
|
|
git clone https://github.com/nawaz0x1/py_polymarket_hft_mm
|
|
cd py_polymarket_hft_mm
|
|
```
|
|
|
|
2. **Setup:**
|
|
|
|
```bash
|
|
./setup.sh
|
|
```
|
|
|
|
3. **Configuration:**
|
|
|
|
```bash
|
|
# Edit config.py with your settings
|
|
# Set up your Polymarket API credentials
|
|
# Configure trading parameters as needed
|
|
```
|
|
|
|
### Quick Start
|
|
|
|
```bash
|
|
./run.sh
|
|
```
|
|
|
|
### Configuration Options
|
|
|
|
Edit [config.py](config.py) to customize the system:
|
|
|
|
```python
|
|
# Trading Parameters
|
|
TRADING_BPS_THRESHOLD = 50 # BPS threshold for trade signals
|
|
PROFIT_MARGIN = 0.02 # Minimum profit margin (2%)
|
|
MAX_TRADES = 2 # Maximum concurrent trades
|
|
PLACE_OPPOSITE_ORDER = True # Enable automatic hedging
|
|
|
|
# Performance Settings
|
|
REQUEST_TIMEOUT = 5 # API request timeout
|
|
MARKET_SESSION_SECONDS = 900 # Market session duration
|
|
```
|
|
|
|
---
|
|
|
|
## Contact & Support
|
|
|
|
**Author**: Shah Nawaz Haider
|
|
**GitHub**: [@nawaz0x1](https://github.com/nawaz0x1)
|
|
**X (Twitter)**: [@nawaz0x1](https://x.com/nawaz0x1)
|
|
**LinkedIn**: [Shah Nawaz Haider](https://www.linkedin.com/in/nawazhaider/)
|
|
|
|
---
|
|
|
|
### Support Development
|
|
|
|
If you find this project useful, consider supporting further development:
|
|
|
|
**Crypto Donations:**
|
|
|
|
- **Ethereum (ETH):** `0x89ae2f064cf2cb06a5e66a8e9ea6b653dcb93cfa`
|
|
- **Solana (SOL):** `2V4g71bG6dJyqv4REZeZSCtiF4pQauDvRiLy8MDNjWNv`
|
|
|
|
Your support helps maintain and improve the trading algorithms!
|
|
|
|
---
|
|
|
|
## ⚠️ DISCLAIMER
|
|
|
|
**FOR EDUCATIONAL AND RESEARCH PURPOSES ONLY**
|
|
|
|
This software is provided strictly for educational, research, and demonstration purposes. By using this code, you acknowledge and agree to the following:
|
|
|
|
### Financial Risk Warning
|
|
- **HIGH RISK**: Trading and market making involve substantial risk of financial loss
|
|
- **NO GUARANTEES**: Past performance does not guarantee future results
|
|
- **CAPITAL LOSS**: You may lose some or all of your invested capital
|
|
- **MARKET VOLATILITY**: Prediction markets are highly volatile and unpredictable
|
|
|
|
### Legal and Regulatory Compliance
|
|
- **USER RESPONSIBILITY**: You are solely responsible for compliance with all applicable laws and regulations in your jurisdiction
|
|
- **NO LEGAL ADVICE**: This software does not constitute financial, legal, or investment advice
|
|
- **REGULATORY COMPLIANCE**: Ensure compliance with securities laws, derivatives regulations, and financial services requirements
|
|
- **JURISDICTION SPECIFIC**: Trading regulations vary by country and may prohibit certain activities
|
|
|
|
### Software Limitations
|
|
- **NO WARRANTY**: This software is provided "AS IS" without any warranties, express or implied
|
|
- **BUGS AND ERRORS**: The software may contain bugs, errors, or security vulnerabilities
|
|
- **NO SUPPORT**: No guarantee of maintenance, updates, or technical support
|
|
- **THIRD-PARTY DEPENDENCIES**: Relies on external APIs and services that may change or become unavailable
|
|
|
|
### Liability Disclaimer
|
|
- **NO LIABILITY**: The authors and contributors are not liable for any financial losses, damages, or consequences
|
|
- **USER ASSUMES RISK**: You use this software entirely at your own risk
|
|
- **INDEMNIFICATION**: You agree to indemnify and hold harmless the authors from any claims or damages
|
|
|
|
### Additional Warnings
|
|
- **TEST THOROUGHLY**: Always test extensively with small amounts before any live trading
|
|
- **MONITOR CONSTANTLY**: Automated trading systems require constant monitoring
|
|
- **TECHNICAL KNOWLEDGE**: Requires significant technical knowledge to operate safely
|
|
- **API CHANGES**: External API changes may break functionality without notice
|
|
|
|
**By using this software, you acknowledge that you have read, understood, and agree to these terms.**
|
|
|