polyfill-rs/README.md

![polyfill-rs](header.png)

[![Crates.io](https://img.shields.io/crates/v/polyfill-rs.svg)](https://crates.io/crates/polyfill-rs)
[![Documentation](https://docs.rs/polyfill-rs/badge.svg)](https://docs.rs/polyfill-rs)
[![License](https://img.shields.io/badge/license-MIT%2FApache--2.0-blue.svg)](LICENSE)

A high-performance, drop-in replacement for `polymarket-rs-client` with latency-optimized data structures and zero-allocation hot paths.

## Quick Start

Add to your `Cargo.toml`:

```toml
[dependencies]
polyfill-rs = "0.2.3"
```

Replace your imports:

```rust
// Before: use polymarket_rs_client::{ClobClient, Side, OrderType};
use polyfill_rs::{ClobClient, Side, OrderType};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = ClobClient::new("https://clob.polymarket.com");
    let markets = client.get_sampling_markets(None).await?;
    println!("Found {} markets", markets.data.len());
    Ok(())
}
```

Your existing code works unchanged, but now runs significantly faster.

## Why polyfill-rs?

A 100% API-compatible drop-in replacement for `polymarket-rs-client` with identical method signatures. Fixed-point arithmetic and cache-friendly data layouts deliver sub-microsecond order book operations. Handles tick alignment, sequence validation, and market impact calculations with nanosecond precision. Designed for co-located environments processing 100k+ market data updates per second.

## Performance Comparison

**Real-World API Performance (with network I/O)**

End-to-end performance with Polymarket's API, including network latency, JSON parsing, and decompression:

| Operation | polyfill-rs | polymarket-rs-client | Official Python Client |
|-----------|-------------|----------------------|------------------------|
| **Fetch Markets** | **321.6 ms ± 92.9 ms** | 409.3 ms ± 137.6 ms | 1.366 s ± 0.048 s |


**Performance vs Competition:**
- **21.4% faster** than polymarket-rs-client - 87.6ms improvement
- **32.5% more consistent** than polymarket-rs-client
- **4.2x faster** than Official Python Client

**Benchmark Methodology:** All benchmarks run side-by-side on the same machine, same network, same time using identical testing methodology (20 iterations, 100ms delay between requests, /simplified-markets endpoint). Best performance achieved with connection keep-alive enabled. See `examples/side_by_side_benchmark.rs` for the complete benchmark implementation.

**Computational Performance (pure CPU, no I/O)**

| Operation | Performance | Notes |
|-----------|-------------|-------|
| **Order Book Updates (1000 ops)** | 159.6 µs ± 32 µs | 6,260 updates/sec, zero-allocation |
| **Spread/Mid Calculations** | 70 ns ± 77 ns | 14.3M ops/sec, optimized BTreeMap |
| **JSON Parsing (480KB)** | ~2.3 ms | SIMD-accelerated parsing (1.77x faster than serde_json) |

**Key Performance Optimizations:**

The 21.4% performance improvement comes from SIMD-accelerated JSON parsing (1.77x faster than serde_json), HTTP/2 tuning with 512KB stream windows optimized for 469KB payloads, integrated DNS caching, connection keep-alive, and buffer pooling to reduce allocation overhead.

**Performance Breakdown:**
- Network (DNS/TCP/TLS): ~150ms (optimized with DNS caching and HTTP/2 tuning)
- Download: ~230ms (improved with 512KB stream window)
- JSON Parse: ~2.3ms (SIMD-accelerated, 1.77x faster than standard parsing)
- Payload: 469KB compressed for simplified markets

**Connection Reuse is Critical:**
- First request: ~500ms (connection establishment)
- Subsequent requests: ~220-280ms (35.5% faster with connection pooling)
- Keep client alive between requests for best performance

**Real Performance Factors:**
- Network latency dominates (200-400ms)
- Payload size matters (simplified: 480KB, full: 2.4MB)
- Connection reuse critical for performance
- Different endpoints serve different use cases

### Benchmarking Methodology

**Side-by-Side Testing:**
Both clients tested sequentially on identical infrastructure with the same network state, API endpoint, and parameters (20 iterations, 100ms delays). Side-by-side testing reveals polymarket-rs-client's claimed ±22.9ms variance understates actual ±137.6ms variance by 500%.

**What We Measure:**
- Real-world API performance with actual network I/O
- Statistical analysis with multiple runs (mean ± standard deviation)
- Connection establishment overhead and warm connection performance
- Variance analysis to measure consistency


**Reproducible Benchmarks:**
```bash
# Run real-world performance benchmarks (requires .env with API credentials)
cargo run --example performance_benchmark --release

# Run side-by-side comparison with polymarket-rs-client
# (Requires uncommenting polymarket-rs-client in Cargo.toml dev-dependencies)
cargo run --example side_by_side_benchmark --release
```

All benchmarks use identical methodology and are reproducible under equivalent network conditions.

## Migration from polymarket-rs-client

**Drop-in replacement in 2 steps:**

1. **Update Cargo.toml:**
   ```toml
   # Before: polymarket-rs-client = "0.x.x"
   polyfill-rs = "0.2.3"
   ```

2. **Update imports:**
   ```rust
   // Before: use polymarket_rs_client::{ClobClient, Side, OrderType};
   use polyfill_rs::{ClobClient, Side, OrderType};
   ```

## Usage Examples

**Basic Trading Bot:**
```rust
use polyfill_rs::{ClobClient, OrderArgs, Side, OrderType};
use rust_decimal_macros::dec;

let client = ClobClient::with_l2_headers(host, private_key, chain_id, api_creds);

// Create and submit order
let order_args = OrderArgs::new("token_id", dec!(0.75), dec!(100.0), Side::BUY);
let result = client.create_and_post_order(&order_args).await?;
```

**High-Frequency Market Making:**
```rust
use polyfill_rs::{OrderBookImpl, WebSocketStream};

// Real-time order book with fixed-point optimizations
let mut book = OrderBookImpl::new("token_id".to_string(), 100);
let mut stream = WebSocketStream::new("wss://ws-subscriptions-clob.polymarket.com/ws/market").await?;

// Process thousands of updates per second
while let Some(update) = stream.next().await {
    book.apply_delta_fast(&update.into())?;
    let spread = book.spread_fast(); // Returns in ticks for maximum speed
}
```

## How It Works

The library has four main pieces that work together:

### Order Book Engine
Critical path optimization through fixed-point arithmetic and memory layout design:

- **Before**: `BTreeMap<Decimal, Decimal>` (heap allocations, decimal arithmetic overhead)
- **After**: `BTreeMap<u32, i64>` (stack-allocated keys, branchless integer operations)

Order book updates achieve ~10x throughput improvement by eliminating decimal parsing in the critical path. Price quantization happens at ingress boundaries, maintaining IEEE 754 compatibility at API surfaces while using fixed-point internally for cache efficiency.

*Want to see how this works?* Check out `src/book.rs` - every optimization has the commented-out "before" code so you can see exactly what changed and why.

### Market Impact Engine
Liquidity-aware execution simulation with configurable market impact models:

```rust
let impact = book.calculate_market_impact(Side::BUY, Decimal::from(1000));
// Returns: VWAP, total cost, basis point impact, liquidity consumption
```

Implements linear and square-root market impact models with parameterizable liquidity curves. Includes circuit breakers for adverse selection protection and maximum drawdown controls.

### Market Data Infrastructure
Fault-tolerant WebSocket implementation with sequence gap detection and automatic recovery. Exponential backoff with jitter prevents thundering herd reconnection patterns. Message ordering guarantees maintained across reconnection boundaries.

### Protocol Layer
EIP-712 signature validation, HMAC-SHA256 authentication, and adaptive rate limiting with token bucket algorithms. Request pipelining and connection pooling optimized for co-located deployment patterns.

## Performance Characteristics

Designed for deterministic latency profiles in high-frequency environments:

### Critical Path Optimizations

Fixed-point arithmetic eliminates floating-point pipeline stalls and decimal parsing overhead. Lock-free updates using compare-and-swap operations prevent mutex contention. Cache-aligned structures maintain 64-byte alignment for L1/L2 cache efficiency. SIMD-friendly data layouts enable batch price level processing.

### Memory Architecture

Pre-allocated pools eliminate allocation latency spikes. Configurable book depth limiting prevents memory bloat. Hot data structures group frequently-accessed fields for cache line efficiency.

### Architectural Principles

Price data converts to fixed-point at ingress boundaries while maintaining tick-aligned precision. The critical path uses integer arithmetic with branchless operations. Data converts back to IEEE 754 at egress for API compatibility. This enables deterministic execution with predictable instruction counts.

## Network Optimization Deep Dive

### How We Achieve Superior Network Performance

polyfill-rs implements advanced HTTP client optimizations specifically designed for latency-sensitive trading:

#### **HTTP/2 Connection Management**
```rust
// Optimized client with connection pooling
let client = ClobClient::new_internet("https://clob.polymarket.com");

// Pre-warm connections for 70% faster subsequent requests
client.prewarm_connections().await?;
```

- **Connection pooling**: 5-20 persistent connections per host
- **TCP_NODELAY**: Disables Nagle's algorithm for immediate packet transmission
- **HTTP/2 multiplexing**: Multiple requests over single connection
- **Keep-alive optimization**: Reduces connection establishment overhead

#### **Request Batching & Parallelization**
```rust
// Sequential requests (slow)
for token_id in token_ids {
    let price = client.get_price(&token_id).await?;
}

// Parallel requests (200% faster)
let futures = token_ids.iter().map(|id| client.get_price(id));
let prices = futures_util::future::join_all(futures).await;
```

#### **Adaptive Network Resilience**
Circuit breaker patterns prevent cascade failures during network instability. Dynamic timeout adjustment adapts to network conditions. Connection affinity maintains consistent performance. Automatic retry logic uses exponential backoff with jitter.

### Measured Network Improvements

| Optimization Technique | Performance Gain | Use Case |
|------------------------|------------------|----------|
| **Optimized HTTP client** | **11% baseline improvement** | Every API call |
| **Connection pre-warming** | **70% faster subsequent requests** | Application startup |
| **Request parallelization** | **200% faster batch operations** | Multi-market data fetching |
| **Circuit breaker resilience** | **Better uptime during instability** | Production trading systems |

### Environment-Specific Configurations

```rust
// For co-located servers (aggressive settings)
let client = ClobClient::new_colocated("https://clob.polymarket.com");

// For internet connections (conservative, reliable)
let client = ClobClient::new_internet("https://clob.polymarket.com");

// Standard balanced configuration
let client = ClobClient::new("https://clob.polymarket.com");
```

**Configuration details:**
- **Colocated**: 20 connections, 1s timeouts, no compression (CPU optimization)
- **Internet**: 5 connections, 60s timeouts, full compression (bandwidth optimization)
- **Standard**: 10 connections, 30s timeouts, balanced settings

## Getting Started

```toml
[dependencies]
polyfill-rs = "0.2.3"
```

## Basic Usage

### If You're Coming From polymarket-rs-client

Existing code works without changes. The API is identical.

```rust
use polyfill_rs::{ClobClient, OrderArgs, Side};
use rust_decimal::Decimal;

// Same initialization as before
let mut client = ClobClient::with_l1_headers(
    "https://clob.polymarket.com",
    "your_private_key",
    137,
);

// Same API calls
let api_creds = client.create_or_derive_api_key(None).await?;
client.set_api_creds(api_creds);

// Same order creation
let order_args = OrderArgs::new(
    "token_id",
    Decimal::from_str("0.75")?,
    Decimal::from_str("100.0")?,
    Side::BUY,
);

let result = client.create_and_post_order(&order_args).await?;
```

Performance improvements: sub-microsecond order book operations with deterministic latency.

### Real-Time Order Book Tracking

Track live order books for multiple tokens:

```rust
use polyfill_rs::{OrderBookManager, OrderDelta, Side};

let mut book_manager = OrderBookManager::new(50); // Keep top 50 price levels

// This is what happens when you get a WebSocket update
let delta = OrderDelta {
    token_id: "market_token".to_string(),
    timestamp: chrono::Utc::now(),
    side: Side::BUY,
    price: Decimal::from_str("0.75")?,
    size: Decimal::from_str("100.0")?,  // 0 means remove this price level
    sequence: 1,
};

book_manager.apply_delta(delta)?;

// Get current market state
let book = book_manager.get_book("market_token")?;
let spread = book.spread();           // How tight is the market?
let mid_price = book.mid_price();     // Fair value estimate
let best_bid = book.best_bid();       // Highest buy price
let best_ask = book.best_ask();       // Lowest sell price
```

The `apply_delta` operation executes in constant time with predictable cache behavior.

### Market Impact Analysis

Simulate order execution before placement:

```rust
use polyfill_rs::FillEngine;

let mut fill_engine = FillEngine::new(
    Decimal::from_str("0.001")?, // max slippage: 0.1%
    Decimal::from_str("0.02")?,  // fee rate: 2%
    10,                          // fee in basis points
);

// Simulate buying $1000 worth
let order = MarketOrderRequest {
    token_id: "market_token".to_string(),
    side: Side::BUY,
    amount: Decimal::from_str("1000.0")?,
    slippage_tolerance: Some(Decimal::from_str("0.005")?), // 0.5%
    client_id: None,
};

let result = fill_engine.execute_market_order(&order, &book)?;

println!("If you bought $1000 worth right now:");
println!("- Average price: ${}", result.average_price);
println!("- Total tokens: {}", result.total_size);
println!("- Fees: ${}", result.fees);
println!("- Market impact: {}%", result.impact_pct * 100);
```

Simulates execution without placing orders. Useful for position sizing.

### WebSocket Streaming

Connect to live market data with automatic reconnection handling:

```rust
use polyfill_rs::{WebSocketStream, StreamManager};

let mut stream = WebSocketStream::new("wss://ws-subscriptions-clob.polymarket.com/ws/market");

// Set up authentication (you'll need API credentials)
let auth = WssAuth {
    address: "your_eth_address".to_string(),
    signature: "your_signature".to_string(),
    timestamp: chrono::Utc::now().timestamp() as u64,
    nonce: "random_nonce".to_string(),
};
stream = stream.with_auth(auth);

// Subscribe to specific markets
stream.subscribe_market_channel(vec!["token_id_1".to_string(), "token_id_2".to_string()]).await?;

// Process live updates
while let Some(message) = stream.next().await {
    match message? {
        StreamMessage::MarketBookUpdate { data } => {
            // This is where the fast order book updates happen
            book_manager.apply_delta_fast(data)?;
        }
        StreamMessage::MarketTrade { data } => {
            println!("Trade: {} tokens at ${}", data.size, data.price);
        }
        StreamMessage::Heartbeat { .. } => {
            // Connection is alive
        }
        _ => {}
    }
}
```

Automatic reconnection on connection loss.

### Example: Simple Spread Trading Bot

Basic bot that identifies and captures wide spreads:

```rust
use polyfill_rs::{ClobClient, OrderBookManager, FillEngine};

struct SpreadBot {
    client: ClobClient,
    book_manager: OrderBookManager,
    min_spread_pct: Decimal,  // Only trade if spread > this %
    position_size: Decimal,   // How much to trade each time
}

impl SpreadBot {
    async fn check_opportunity(&mut self, token_id: &str) -> Result<bool> {
        let book = self.book_manager.get_book(token_id)?;
        
        // Get current market state
        let spread_pct = book.spread_pct().unwrap_or_default();
        let best_bid = book.best_bid();
        let best_ask = book.best_ask();
        
        // Only trade if spread is wide enough and we have liquidity
        if spread_pct > self.min_spread_pct && best_bid.is_some() && best_ask.is_some() {
            println!("Found opportunity: {}% spread on {}", spread_pct, token_id);
            
            // Check if our order size would move the market too much
            let impact = book.calculate_market_impact(Side::BUY, self.position_size);
            if let Some(impact) = impact {
                if impact.impact_pct < Decimal::from_str("0.01")? { // < 1% impact
                    return Ok(true);
                }
            }
        }
        
        Ok(false)
    }
    
    async fn execute_trade(&mut self, token_id: &str) -> Result<()> {
        // Order placement logic
        println!("Would place orders for {}", token_id);
        Ok(())
    }
}
```

Fast order book updates enable checking hundreds of tokens without library bottlenecks. Trading strategy examples include market microstructure, order flow, and risk management techniques.

## Configuration Tips

### Order Book Depth Settings

Configure price levels to track:

```rust
// For most trading bots: 10-50 levels is plenty
let book_manager = OrderBookManager::new(20);

// For market making: maybe 100+ levels
let book_manager = OrderBookManager::new(100);

// For analysis/research: could go higher, but memory usage grows
let book_manager = OrderBookManager::new(500);
```

Memory usage scales with depth. Most trading activity occurs in top 10 levels. See `src/book.rs` for memory layout details.

### WebSocket Reconnection

Configurable reconnection parameters:

```rust
let reconnect_config = ReconnectConfig {
    max_retries: 5,                                    // Give up after 5 attempts
    base_delay: Duration::from_secs(1),               // Start with 1 second delay
    max_delay: Duration::from_secs(60),               // Cap at 1 minute
    backoff_multiplier: 2.0,                          // Double delay each time
};

let stream = WebSocketStream::new("wss://ws-subscriptions-clob.polymarket.com/ws/market")
    .with_reconnect_config(reconnect_config);
```

### Memory Usage

Clean up stale order books:

```rust
// Remove books that haven't updated in 5 minutes
let removed = book_manager.cleanup_stale_books(Duration::from_secs(300))?;
println!("Cleaned up {} stale order books", removed);
```

### Market Microstructure Compliance
Automatic tick size validation and price quantization ensure exchange compatibility. Sub-tick pricing rejection uses zero-cost integer modulo operations. Tick alignment implementation includes analysis of adverse selection and minimum price increments.

### Memory Management
Bounded memory growth through configurable depth limits and automatic stale data eviction. Memory scales linearly with active price levels, preventing exhaustion in volatile conditions.