mirror of
https://github.com/floor-licker/polyfill-rs.git
synced 2026-08-13 12:38:05 +00:00
docs: update README.md
This commit is contained in:
floor-licker
@@ -4,7 +4,8 @@
|
||||
[](https://docs.rs/polyfill-rs)
|
||||
[](LICENSE)
|
||||
|
||||
A high-performance, drop-in replacement for `polymarket-rs-client` with latency-optimized data structures and zero-allocation hot paths.
|
||||
A high-performance drop-in replacement for `polymarket-rs-client` with latency-optimized data structures and zero-allocation hot paths. A 100% API-compatible drop-in replacement for `polymarket-rs-client` with identical method signatures.
|
||||
|
||||
|
||||
## Quick Start
|
||||
|
||||
@@ -30,12 +31,6 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
}
|
||||
```
|
||||
|
||||
Your existing code works unchanged, but now runs significantly faster.
|
||||
|
||||
## Why polyfill-rs?
|
||||
|
||||
A 100% API-compatible drop-in replacement for `polymarket-rs-client` with identical method signatures. Fixed-point arithmetic and cache-friendly data layouts deliver sub-microsecond order book operations. Handles tick alignment, sequence validation, and market impact calculations with nanosecond precision. Designed for co-located environments processing 100k+ market data updates per second.
|
||||
|
||||
## Performance Comparison
|
||||
|
||||
**Real-World API Performance (with network I/O)**
|
||||
@@ -66,23 +61,6 @@ End-to-end performance with Polymarket's API, including network latency, JSON pa
|
||||
|
||||
The 21.4% performance improvement comes from SIMD-accelerated JSON parsing (1.77x faster than serde_json), HTTP/2 tuning with 512KB stream windows optimized for 469KB payloads, integrated DNS caching, connection keep-alive, and buffer pooling to reduce allocation overhead.
|
||||
|
||||
**Performance Breakdown:**
|
||||
- Network (DNS/TCP/TLS): ~150ms (optimized with DNS caching and HTTP/2 tuning)
|
||||
- Download: ~230ms (improved with 512KB stream window)
|
||||
- JSON Parse: ~2.3ms (SIMD-accelerated, 1.77x faster than standard parsing)
|
||||
- Payload: 469KB compressed for simplified markets
|
||||
|
||||
**Connection Reuse is Critical:**
|
||||
- First request: ~500ms (connection establishment)
|
||||
- Subsequent requests: ~220-280ms (35.5% faster with connection pooling)
|
||||
- Keep client alive between requests for best performance
|
||||
|
||||
**Real Performance Factors:**
|
||||
- Network latency dominates (200-400ms)
|
||||
- Payload size matters (simplified: 480KB, full: 2.4MB)
|
||||
- Connection reuse critical for performance
|
||||
- Different endpoints serve different use cases
|
||||
|
||||
### Benchmarking Methodology
|
||||
|
||||
**Side-by-Side Testing:**
|
||||
@@ -94,98 +72,6 @@ Both clients tested sequentially on identical infrastructure with the same netwo
|
||||
- Connection establishment overhead and warm connection performance
|
||||
- Variance analysis to measure consistency
|
||||
|
||||
|
||||
**Reproducible Benchmarks:**
|
||||
```bash
|
||||
# Run real-world performance benchmarks (requires .env with API credentials)
|
||||
cargo run --example performance_benchmark --release
|
||||
|
||||
# Run side-by-side comparison with polymarket-rs-client
|
||||
# (Requires uncommenting polymarket-rs-client in Cargo.toml dev-dependencies)
|
||||
cargo run --example side_by_side_benchmark --release
|
||||
```
|
||||
|
||||
All benchmarks use identical methodology and are reproducible under equivalent network conditions.
|
||||
|
||||
## Migration from polymarket-rs-client
|
||||
|
||||
**Drop-in replacement in 2 steps:**
|
||||
|
||||
1. **Update Cargo.toml:**
|
||||
```toml
|
||||
# Before: polymarket-rs-client = "0.x.x"
|
||||
polyfill-rs = "0.2.3"
|
||||
```
|
||||
|
||||
2. **Update imports:**
|
||||
```rust
|
||||
// Before: use polymarket_rs_client::{ClobClient, Side, OrderType};
|
||||
use polyfill_rs::{ClobClient, Side, OrderType};
|
||||
```
|
||||
|
||||
## Usage Examples
|
||||
|
||||
**Basic Trading Bot:**
|
||||
```rust
|
||||
use polyfill_rs::{ClobClient, OrderArgs, Side, OrderType};
|
||||
use rust_decimal_macros::dec;
|
||||
|
||||
let client = ClobClient::with_l2_headers(host, private_key, chain_id, api_creds);
|
||||
|
||||
// Create and submit order
|
||||
let order_args = OrderArgs::new("token_id", dec!(0.75), dec!(100.0), Side::BUY);
|
||||
let result = client.create_and_post_order(&order_args).await?;
|
||||
```
|
||||
|
||||
**High-Frequency Market Making:**
|
||||
```rust
|
||||
use polyfill_rs::{OrderBookImpl, WebSocketStream};
|
||||
|
||||
// Real-time order book with fixed-point optimizations
|
||||
let mut book = OrderBookImpl::new("token_id".to_string(), 100);
|
||||
let mut stream = WebSocketStream::new("wss://ws-subscriptions-clob.polymarket.com/ws/market").await?;
|
||||
|
||||
// Process thousands of updates per second
|
||||
while let Some(update) = stream.next().await {
|
||||
book.apply_delta_fast(&update.into())?;
|
||||
let spread = book.spread_fast(); // Returns in ticks for maximum speed
|
||||
}
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
The library has four main pieces that work together:
|
||||
|
||||
### Order Book Engine
|
||||
Critical path optimization through fixed-point arithmetic and memory layout design:
|
||||
|
||||
- **Before**: `BTreeMap<Decimal, Decimal>` (heap allocations, decimal arithmetic overhead)
|
||||
- **After**: `BTreeMap<u32, i64>` (stack-allocated keys, branchless integer operations)
|
||||
|
||||
Order book updates achieve ~10x throughput improvement by eliminating decimal parsing in the critical path. Price quantization happens at ingress boundaries, maintaining IEEE 754 compatibility at API surfaces while using fixed-point internally for cache efficiency.
|
||||
|
||||
*Want to see how this works?* Check out `src/book.rs` - every optimization has the commented-out "before" code so you can see exactly what changed and why.
|
||||
|
||||
### Market Impact Engine
|
||||
Liquidity-aware execution simulation with configurable market impact models:
|
||||
|
||||
```rust
|
||||
let impact = book.calculate_market_impact(Side::BUY, Decimal::from(1000));
|
||||
// Returns: VWAP, total cost, basis point impact, liquidity consumption
|
||||
```
|
||||
|
||||
Implements linear and square-root market impact models with parameterizable liquidity curves. Includes circuit breakers for adverse selection protection and maximum drawdown controls.
|
||||
|
||||
### Market Data Infrastructure
|
||||
Fault-tolerant WebSocket implementation with sequence gap detection and automatic recovery. Exponential backoff with jitter prevents thundering herd reconnection patterns. Message ordering guarantees maintained across reconnection boundaries.
|
||||
|
||||
### Protocol Layer
|
||||
EIP-712 signature validation, HMAC-SHA256 authentication, and adaptive rate limiting with token bucket algorithms. Request pipelining and connection pooling optimized for co-located deployment patterns.
|
||||
|
||||
## Performance Characteristics
|
||||
|
||||
Designed for deterministic latency profiles in high-frequency environments:
|
||||
|
||||
### Critical Path Optimizations
|
||||
|
||||
Fixed-point arithmetic eliminates floating-point pipeline stalls and decimal parsing overhead. Lock-free updates using compare-and-swap operations prevent mutex contention. Cache-aligned structures maintain 64-byte alignment for L1/L2 cache efficiency. SIMD-friendly data layouts enable batch price level processing.
|
||||
@@ -198,41 +84,6 @@ Pre-allocated pools eliminate allocation latency spikes. Configurable book depth
|
||||
|
||||
Price data converts to fixed-point at ingress boundaries while maintaining tick-aligned precision. The critical path uses integer arithmetic with branchless operations. Data converts back to IEEE 754 at egress for API compatibility. This enables deterministic execution with predictable instruction counts.
|
||||
|
||||
## Network Optimization Deep Dive
|
||||
|
||||
### How We Achieve Superior Network Performance
|
||||
|
||||
polyfill-rs implements advanced HTTP client optimizations specifically designed for latency-sensitive trading:
|
||||
|
||||
#### **HTTP/2 Connection Management**
|
||||
```rust
|
||||
// Optimized client with connection pooling
|
||||
let client = ClobClient::new_internet("https://clob.polymarket.com");
|
||||
|
||||
// Pre-warm connections for 70% faster subsequent requests
|
||||
client.prewarm_connections().await?;
|
||||
```
|
||||
|
||||
- **Connection pooling**: 5-20 persistent connections per host
|
||||
- **TCP_NODELAY**: Disables Nagle's algorithm for immediate packet transmission
|
||||
- **HTTP/2 multiplexing**: Multiple requests over single connection
|
||||
- **Keep-alive optimization**: Reduces connection establishment overhead
|
||||
|
||||
#### **Request Batching & Parallelization**
|
||||
```rust
|
||||
// Sequential requests (slow)
|
||||
for token_id in token_ids {
|
||||
let price = client.get_price(&token_id).await?;
|
||||
}
|
||||
|
||||
// Parallel requests (200% faster)
|
||||
let futures = token_ids.iter().map(|id| client.get_price(id));
|
||||
let prices = futures_util::future::join_all(futures).await;
|
||||
```
|
||||
|
||||
#### **Adaptive Network Resilience**
|
||||
Circuit breaker patterns prevent cascade failures during network instability. Dynamic timeout adjustment adapts to network conditions. Connection affinity maintains consistent performance. Automatic retry logic uses exponential backoff with jitter.
|
||||
|
||||
### Measured Network Improvements
|
||||
|
||||
| Optimization Technique | Performance Gain | Use Case |
|
||||
@@ -241,267 +92,3 @@ Circuit breaker patterns prevent cascade failures during network instability. Dy
|
||||
| **Connection pre-warming** | **70% faster subsequent requests** | Application startup |
|
||||
| **Request parallelization** | **200% faster batch operations** | Multi-market data fetching |
|
||||
| **Circuit breaker resilience** | **Better uptime during instability** | Production trading systems |
|
||||
|
||||
### Environment-Specific Configurations
|
||||
|
||||
```rust
|
||||
// For co-located servers (aggressive settings)
|
||||
let client = ClobClient::new_colocated("https://clob.polymarket.com");
|
||||
|
||||
// For internet connections (conservative, reliable)
|
||||
let client = ClobClient::new_internet("https://clob.polymarket.com");
|
||||
|
||||
// Standard balanced configuration
|
||||
let client = ClobClient::new("https://clob.polymarket.com");
|
||||
```
|
||||
|
||||
**Configuration details:**
|
||||
- **Colocated**: 20 connections, 1s timeouts, no compression (CPU optimization)
|
||||
- **Internet**: 5 connections, 60s timeouts, full compression (bandwidth optimization)
|
||||
- **Standard**: 10 connections, 30s timeouts, balanced settings
|
||||
|
||||
## Getting Started
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
polyfill-rs = "0.2.3"
|
||||
```
|
||||
|
||||
## Basic Usage
|
||||
|
||||
### If You're Coming From polymarket-rs-client
|
||||
|
||||
Existing code works without changes. The API is identical.
|
||||
|
||||
```rust
|
||||
use polyfill_rs::{ClobClient, OrderArgs, Side};
|
||||
use rust_decimal::Decimal;
|
||||
|
||||
// Same initialization as before
|
||||
let mut client = ClobClient::with_l1_headers(
|
||||
"https://clob.polymarket.com",
|
||||
"your_private_key",
|
||||
137,
|
||||
);
|
||||
|
||||
// Same API calls
|
||||
let api_creds = client.create_or_derive_api_key(None).await?;
|
||||
client.set_api_creds(api_creds);
|
||||
|
||||
// Same order creation
|
||||
let order_args = OrderArgs::new(
|
||||
"token_id",
|
||||
Decimal::from_str("0.75")?,
|
||||
Decimal::from_str("100.0")?,
|
||||
Side::BUY,
|
||||
);
|
||||
|
||||
let result = client.create_and_post_order(&order_args).await?;
|
||||
```
|
||||
|
||||
Performance improvements: sub-microsecond order book operations with deterministic latency.
|
||||
|
||||
### Real-Time Order Book Tracking
|
||||
|
||||
Track live order books for multiple tokens:
|
||||
|
||||
```rust
|
||||
use polyfill_rs::{OrderBookManager, OrderDelta, Side};
|
||||
|
||||
let mut book_manager = OrderBookManager::new(50); // Keep top 50 price levels
|
||||
|
||||
// This is what happens when you get a WebSocket update
|
||||
let delta = OrderDelta {
|
||||
token_id: "market_token".to_string(),
|
||||
timestamp: chrono::Utc::now(),
|
||||
side: Side::BUY,
|
||||
price: Decimal::from_str("0.75")?,
|
||||
size: Decimal::from_str("100.0")?, // 0 means remove this price level
|
||||
sequence: 1,
|
||||
};
|
||||
|
||||
book_manager.apply_delta(delta)?;
|
||||
|
||||
// Get current market state
|
||||
let book = book_manager.get_book("market_token")?;
|
||||
let spread = book.spread(); // How tight is the market?
|
||||
let mid_price = book.mid_price(); // Fair value estimate
|
||||
let best_bid = book.best_bid(); // Highest buy price
|
||||
let best_ask = book.best_ask(); // Lowest sell price
|
||||
```
|
||||
|
||||
The `apply_delta` operation executes in constant time with predictable cache behavior.
|
||||
|
||||
### Market Impact Analysis
|
||||
|
||||
Simulate order execution before placement:
|
||||
|
||||
```rust
|
||||
use polyfill_rs::FillEngine;
|
||||
|
||||
let mut fill_engine = FillEngine::new(
|
||||
Decimal::from_str("0.001")?, // max slippage: 0.1%
|
||||
Decimal::from_str("0.02")?, // fee rate: 2%
|
||||
10, // fee in basis points
|
||||
);
|
||||
|
||||
// Simulate buying $1000 worth
|
||||
let order = MarketOrderRequest {
|
||||
token_id: "market_token".to_string(),
|
||||
side: Side::BUY,
|
||||
amount: Decimal::from_str("1000.0")?,
|
||||
slippage_tolerance: Some(Decimal::from_str("0.005")?), // 0.5%
|
||||
client_id: None,
|
||||
};
|
||||
|
||||
let result = fill_engine.execute_market_order(&order, &book)?;
|
||||
|
||||
println!("If you bought $1000 worth right now:");
|
||||
println!("- Average price: ${}", result.average_price);
|
||||
println!("- Total tokens: {}", result.total_size);
|
||||
println!("- Fees: ${}", result.fees);
|
||||
println!("- Market impact: {}%", result.impact_pct * 100);
|
||||
```
|
||||
|
||||
Simulates execution without placing orders. Useful for position sizing.
|
||||
|
||||
### WebSocket Streaming
|
||||
|
||||
Connect to live market data with automatic reconnection handling:
|
||||
|
||||
```rust
|
||||
use polyfill_rs::{WebSocketStream, StreamManager};
|
||||
|
||||
let mut stream = WebSocketStream::new("wss://ws-subscriptions-clob.polymarket.com/ws/market");
|
||||
|
||||
// Set up authentication (you'll need API credentials)
|
||||
let auth = WssAuth {
|
||||
address: "your_eth_address".to_string(),
|
||||
signature: "your_signature".to_string(),
|
||||
timestamp: chrono::Utc::now().timestamp() as u64,
|
||||
nonce: "random_nonce".to_string(),
|
||||
};
|
||||
stream = stream.with_auth(auth);
|
||||
|
||||
// Subscribe to specific markets
|
||||
stream.subscribe_market_channel(vec!["token_id_1".to_string(), "token_id_2".to_string()]).await?;
|
||||
|
||||
// Process live updates
|
||||
while let Some(message) = stream.next().await {
|
||||
match message? {
|
||||
StreamMessage::MarketBookUpdate { data } => {
|
||||
// This is where the fast order book updates happen
|
||||
book_manager.apply_delta_fast(data)?;
|
||||
}
|
||||
StreamMessage::MarketTrade { data } => {
|
||||
println!("Trade: {} tokens at ${}", data.size, data.price);
|
||||
}
|
||||
StreamMessage::Heartbeat { .. } => {
|
||||
// Connection is alive
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Automatic reconnection on connection loss.
|
||||
|
||||
### Example: Simple Spread Trading Bot
|
||||
|
||||
Basic bot that identifies and captures wide spreads:
|
||||
|
||||
```rust
|
||||
use polyfill_rs::{ClobClient, OrderBookManager, FillEngine};
|
||||
|
||||
struct SpreadBot {
|
||||
client: ClobClient,
|
||||
book_manager: OrderBookManager,
|
||||
min_spread_pct: Decimal, // Only trade if spread > this %
|
||||
position_size: Decimal, // How much to trade each time
|
||||
}
|
||||
|
||||
impl SpreadBot {
|
||||
async fn check_opportunity(&mut self, token_id: &str) -> Result<bool> {
|
||||
let book = self.book_manager.get_book(token_id)?;
|
||||
|
||||
// Get current market state
|
||||
let spread_pct = book.spread_pct().unwrap_or_default();
|
||||
let best_bid = book.best_bid();
|
||||
let best_ask = book.best_ask();
|
||||
|
||||
// Only trade if spread is wide enough and we have liquidity
|
||||
if spread_pct > self.min_spread_pct && best_bid.is_some() && best_ask.is_some() {
|
||||
println!("Found opportunity: {}% spread on {}", spread_pct, token_id);
|
||||
|
||||
// Check if our order size would move the market too much
|
||||
let impact = book.calculate_market_impact(Side::BUY, self.position_size);
|
||||
if let Some(impact) = impact {
|
||||
if impact.impact_pct < Decimal::from_str("0.01")? { // < 1% impact
|
||||
return Ok(true);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Ok(false)
|
||||
}
|
||||
|
||||
async fn execute_trade(&mut self, token_id: &str) -> Result<()> {
|
||||
// Order placement logic
|
||||
println!("Would place orders for {}", token_id);
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Fast order book updates enable checking hundreds of tokens without library bottlenecks. Trading strategy examples include market microstructure, order flow, and risk management techniques.
|
||||
|
||||
## Configuration Tips
|
||||
|
||||
### Order Book Depth Settings
|
||||
|
||||
Configure price levels to track:
|
||||
|
||||
```rust
|
||||
// For most trading bots: 10-50 levels is plenty
|
||||
let book_manager = OrderBookManager::new(20);
|
||||
|
||||
// For market making: maybe 100+ levels
|
||||
let book_manager = OrderBookManager::new(100);
|
||||
|
||||
// For analysis/research: could go higher, but memory usage grows
|
||||
let book_manager = OrderBookManager::new(500);
|
||||
```
|
||||
|
||||
Memory usage scales with depth. Most trading activity occurs in top 10 levels. See `src/book.rs` for memory layout details.
|
||||
|
||||
### WebSocket Reconnection
|
||||
|
||||
Configurable reconnection parameters:
|
||||
|
||||
```rust
|
||||
let reconnect_config = ReconnectConfig {
|
||||
max_retries: 5, // Give up after 5 attempts
|
||||
base_delay: Duration::from_secs(1), // Start with 1 second delay
|
||||
max_delay: Duration::from_secs(60), // Cap at 1 minute
|
||||
backoff_multiplier: 2.0, // Double delay each time
|
||||
};
|
||||
|
||||
let stream = WebSocketStream::new("wss://ws-subscriptions-clob.polymarket.com/ws/market")
|
||||
.with_reconnect_config(reconnect_config);
|
||||
```
|
||||
|
||||
### Memory Usage
|
||||
|
||||
Clean up stale order books:
|
||||
|
||||
```rust
|
||||
// Remove books that haven't updated in 5 minutes
|
||||
let removed = book_manager.cleanup_stale_books(Duration::from_secs(300))?;
|
||||
println!("Cleaned up {} stale order books", removed);
|
||||
```
|
||||
|
||||
### Market Microstructure Compliance
|
||||
Automatic tick size validation and price quantization ensure exchange compatibility. Sub-tick pricing rejection uses zero-cost integer modulo operations. Tick alignment implementation includes analysis of adverse selection and minimum price increments.
|
||||
|
||||
### Memory Management
|
||||
Bounded memory growth through configurable depth limits and automatic stale data eviction. Memory scales linearly with active price levels, preventing exhaustion in volatile conditions.
|
||||
Reference in New Issue
Block a user