py_polymarket_hft_mm/README.md

# Polymarket HFT Market Making System

_Python High-Frequency Trading Infrastructure for Prediction Markets_

[![Performance](https://img.shields.io/badge/Win%20Rate-99%25-brightgreen.svg)]()
[![Architecture](https://img.shields.io/badge/Architecture-Async%20%2B%20Threading-blue.svg)]()
[![Trading](https://img.shields.io/badge/Strategy-Market%20Making%20%2B%20Hedging-orange.svg)]()
[![License](https://img.shields.io/badge/License-Educational%20Use-green.svg)]()

## Overview

A sophisticated high-frequency trading (HFT) market making system built for Polymarket's prediction markets. This system uses real-time orderbook analysis, signal generation, and automated execution to provide liquidity while maintaining profitability through intelligent market making strategies.

**Key Features:**

- **Real-time Market Making**: Automated bid-ask spread management with dynamic pricing
- **Signal-Based Trading**: BPS (Basis Points Spread) threshold monitoring for entry signals
- **Intelligent Hedging**: Automatic position hedging to manage risk exposure
- **Performance Optimization**: CPU affinity control, GC management, and async architecture
- **Risk Management**: Position limits, profit margins, and automated trade execution
- **24/7 Operation**: Continuous monitoring with robust error handling and reconnection logic

---

## System Architecture

### Core Components

```
┌─────────────────────────────────────────────────────────┐
│                   TRADING ENGINE                        │
├─────────────────────────────────────────────────────────┤
│  OrderBook        │  Signal Generator │  Risk Manager   │
│  ├─ WebSocket     │  ├─ BPS Analysis  │  ├─ Position    │
│  ├─ Real-time     │  ├─ Threshold     │  │  Limits      │
│  │  Updates       │  │  Monitoring    │  ├─ Profit      │
│  └─ Price Data    │  └─ Entry Signals │  │  Margins     │
│                   │                   │  └─ Exposure    │
├─────────────────────────────────────────────────────────┤
│                 EXECUTION LAYER                         │
│  Market Maker     │  Hedging System   │  Monitoring     │
│  ├─ Order         │  ├─ Fill          │  ├─ Logging     │
│  │  Management    │  │  Detection     │  ├─ Performance │
│  ├─ Spread        │  ├─ Automatic     │  │  Metrics     │
│  │  Calculation   │  │  Hedging       │  ├─ Health      │
│  └─ P&L Tracking  │  └─ State Mgmt    │  │  Checks      │
│                   │                   │  └─ Alerts      │
└─────────────────────────────────────────────────────────┘
```

---

## Fast In-Memory Data Store

The system includes a custom high-performance in-memory database built in Rust for ultra-low latency data operations.

### Architecture

The in-memory store consists of two components:

1. **Rust Server** ([in_mem_db.rs](in_memory_db/in_mem_db.rs)): High-performance TCP server with thread-safe HashSet operations
2. **Python Client** ([utils.py](in_memory_db/utils.py)): Fast HTTP client interface for Python integration

### Performance Benefits

```rust
// Thread-safe operations with Arc<Mutex<HashSet<String>>>
struct InMemSetDB {
    data: Arc<Mutex<HashSet<String>>>,
}
```

**Key Advantages:**

- **Ultra-low latency**: Rust-based operations for microsecond response times
- **Thread-safe**: Concurrent access with proper synchronization
- **Memory efficient**: Direct memory operations without disk I/O
- **Lightweight protocol**: Custom HTTP-like protocol minimizes overhead
- **Connection pooling**: Fast socket connections for rapid queries

### Python Integration

```python
from in_memory_db.utils import add_item, contains_item, clear_items, size

# Fast operations for trading state management
add_item("processed_order_123")
if not contains_item("processed_order_123"):
    # Process order...
    pass

# Performance monitoring
current_size = size()
clear_items()  # Reset state
```

### Use Cases in Trading

- **Duplicate Prevention**: Track processed orders to prevent double-execution
- **State Management**: Fast lookups for order status and hedge tracking
- **Signal Filtering**: Cache processed market signals to avoid redundant trades
- **Performance Monitoring**: Real-time metrics storage and retrieval

### Starting the In-Memory DB

```bash
# Compile and run the Rust server
cd in_memory_db
rustc in_mem_db.rs
./in_mem_db

# Server starts on localhost:8080
# InMemSetDB started on http://127.0.0.1:8080
```

---

## Performance Engineering

### 1. Garbage Collection Management

```python
gc.disable()  # Eliminates GC pauses during trading
```

**Why:** Python's garbage collector can cause trading delays. We disable it during active trading and manually trigger cleanup during quiet periods.

### 2. CPU Optimization

```python
def set_cpu_affinity():
    affinity_cores = [cpu_count - 2, cpu_count - 1]  # Use dedicated cores
    process.cpu_affinity(affinity_cores)
    process.nice(psutil.HIGH_PRIORITY_CLASS)  # High priority
```

**Benefits:**

- Dedicated CPU cores for trading operations
- High process priority for consistent performance
- Reduced interference from other system processes

### 3. Async Architecture

```python
# WebSocket on separate thread
self.thread = threading.Thread(target=self._connect, daemon=True)
# Order processing with async
await asyncio.create_task(send_hedge(...))
```

**Advantages:**

- Non-blocking network operations
- Concurrent order processing
- Thread-safe state management

### 4. Efficient Data Processing

```python
def _on_message(self, ws, message):
    data = json.loads(message)  # Fast JSON parsing
    if event_type == "price_change":
        self._update_orderbook_incremental(data)  # Efficient updates
```

---

## Trading Strategy

### Market Making Approach

The system employs a market making strategy that focuses on:

1. **Spread Analysis**: Monitors bid-ask spreads and identifies profitable opportunities
2. **BPS Threshold Trading**: Uses configurable basis point thresholds (default: 50 BPS) to trigger trades
3. **Automated Hedging**: Places hedge orders automatically to manage risk exposure
4. **Position Limits**: Enforces maximum trade limits (configurable via `MAX_TRADES`)

### Signal Generation

```python
# BPS-based signal detection
if current_spread_bps >= TRADING_BPS_THRESHOLD:
    # Generate trading signal
    place_anchor_and_hedge(market_data)
```

**Key Parameters:**

- `TRADING_BPS_THRESHOLD`: Minimum spread required to trigger a trade (50 BPS default)
- `PROFIT_MARGIN`: Minimum profit margin per trade (2% default)
- `MAX_TRADES`: Maximum concurrent positions (2 default)

### Risk Management

- **Position Sizing**: Controlled position sizes based on available capital
- **Automatic Hedging**: Every market making trade is automatically hedged
- **Market Time Windows**: Trades only during active market sessions
- **Stop-Loss Protection**: Built-in safeguards against adverse moves

---

## Monitoring & Logging

The system provides comprehensive logging and monitoring capabilities:

### Log Files

- **[logs/](logs/)**: Trading activity logs

### Performance Monitoring

- Real-time orderbook updates
- Trading signal generation logs
- P&L tracking and reporting
- System health monitoring

### Debug Information

Enable detailed logging by modifying the logger configuration in [utils/logger.py](utils/logger.py).

---

## License & Disclaimer

**License**: This project is for educational and research purposes only.

**Important Disclaimers:**

- This software is provided as-is for educational purposes
- Trading involves substantial risk of financial loss
- Users are responsible for compliance with applicable financial regulations
- Past performance does not guarantee future results
- The authors are not responsible for any financial losses incurred
- Use at your own risk and ensure proper testing before live trading

---

## Installation & Setup

### Prerequisites

```bash
# System Requirements
Python 3.11+
4+ CPU cores recommended
2GB+ RAM for orderbook processing
Stable internet connection
```

### Installation

1. **Clone the repository:**

```bash
git clone https://github.com/nawaz0x1/py_polymarket_hft_mm
cd py_polymarket_hft_mm
```

2. **Setup:**

```bash
./setup.sh
```

3. **Configuration:**

```bash
# Edit config.py with your settings
# Set up your Polymarket API credentials
# Configure trading parameters as needed
```

### Quick Start

```bash
./run.sh
```

### Configuration Options

Edit [config.py](config.py) to customize the system:

```python
# Trading Parameters
TRADING_BPS_THRESHOLD = 50      # BPS threshold for trade signals
PROFIT_MARGIN = 0.02            # Minimum profit margin (2%)
MAX_TRADES = 2                  # Maximum concurrent trades
PLACE_OPPOSITE_ORDER = True     # Enable automatic hedging

# Performance Settings
REQUEST_TIMEOUT = 5             # API request timeout
MARKET_SESSION_SECONDS = 900    # Market session duration
```

---

## Contact & Support

**Author**: Shah Nawaz Haider  
**GitHub**: [@nawaz0x1](https://github.com/nawaz0x1)  
**X (Twitter)**: [@nawaz0x1](https://x.com/nawaz0x1)  
**LinkedIn**: [Shah Nawaz Haider](https://www.linkedin.com/in/nawazhaider/)

---

### Support Development

If you find this project useful, consider supporting further development:

**Crypto Donations:**

- **Ethereum (ETH):** `0x89ae2f064cf2cb06a5e66a8e9ea6b653dcb93cfa`
- **Solana (SOL):** `2V4g71bG6dJyqv4REZeZSCtiF4pQauDvRiLy8MDNjWNv`

Your support helps maintain and improve the trading algorithms!

---

## ⚠️ DISCLAIMER

**FOR EDUCATIONAL AND RESEARCH PURPOSES ONLY**

This software is provided strictly for educational, research, and demonstration purposes. By using this code, you acknowledge and agree to the following:

### Financial Risk Warning
- **HIGH RISK**: Trading and market making involve substantial risk of financial loss
- **NO GUARANTEES**: Past performance does not guarantee future results
- **CAPITAL LOSS**: You may lose some or all of your invested capital
- **MARKET VOLATILITY**: Prediction markets are highly volatile and unpredictable

### Legal and Regulatory Compliance
- **USER RESPONSIBILITY**: You are solely responsible for compliance with all applicable laws and regulations in your jurisdiction
- **NO LEGAL ADVICE**: This software does not constitute financial, legal, or investment advice
- **REGULATORY COMPLIANCE**: Ensure compliance with securities laws, derivatives regulations, and financial services requirements
- **JURISDICTION SPECIFIC**: Trading regulations vary by country and may prohibit certain activities

### Software Limitations
- **NO WARRANTY**: This software is provided "AS IS" without any warranties, express or implied
- **BUGS AND ERRORS**: The software may contain bugs, errors, or security vulnerabilities
- **NO SUPPORT**: No guarantee of maintenance, updates, or technical support
- **THIRD-PARTY DEPENDENCIES**: Relies on external APIs and services that may change or become unavailable

### Liability Disclaimer
- **NO LIABILITY**: The authors and contributors are not liable for any financial losses, damages, or consequences
- **USER ASSUMES RISK**: You use this software entirely at your own risk
- **INDEMNIFICATION**: You agree to indemnify and hold harmless the authors from any claims or damages

### Additional Warnings
- **TEST THOROUGHLY**: Always test extensively with small amounts before any live trading
- **MONITOR CONSTANTLY**: Automated trading systems require constant monitoring
- **TECHNICAL KNOWLEDGE**: Requires significant technical knowledge to operate safely
- **API CHANGES**: External API changes may break functionality without notice

**By using this software, you acknowledge that you have read, understood, and agree to these terms.**

�