perf: Major event processing system refactor for improved performance

README.md

@@ -24,7 +24,7 @@ A lightweight Rust library for real-time event streaming from Solana DEX trading
 11. **Performance Monitoring**: Built-in performance metrics monitoring, including event processing speed, etc.
 12. **Memory Optimization**: Object pooling and caching mechanisms to reduce memory allocations
 13. **Flexible Configuration System**: Support for custom batch sizes, backpressure strategies, channel sizes, and other parameters
-14. **Preset Configurations**: Provides high-performance, low-latency, ordered processing, and other preset configurations
+14. **Preset Configurations**: Provides high-throughput, low-latency, and async processing preset configurations optimized for different use cases
 15. **Backpressure Handling**: Supports blocking, dropping, retrying, ordered, and other backpressure strategies
 16. **Runtime Configuration Updates**: Supports dynamic configuration parameter updates at runtime
 17. **Full Function Performance Monitoring**: All subscribe_events functions support performance monitoring, automatically collecting and reporting performance metrics
@@ -55,6 +55,91 @@ solana-streamer-sdk = { path = "./solana-streamer", version = "0.3.10" }
 solana-streamer-sdk = "0.3.10"
 ```
 
+## Configuration System
+
+### Preset Configurations
+
+The library provides three preset configurations optimized for different use cases:
+
+#### 1. High Throughput Configuration (`high_throughput()`)
+
+Optimized for high-concurrency scenarios, prioritizing throughput over latency:
+
+```rust
+let config = StreamClientConfig::high_throughput();
+// Or use convenience methods
+let grpc = YellowstoneGrpc::new_high_throughput(endpoint, token)?;
+let shred = ShredStreamGrpc::new_high_throughput(endpoint).await?;
+```
+
+**Features:**
+- **Backpressure Strategy**: Drop - drops messages during high load to avoid blocking
+- **Buffer Size**: 5,000 permits to handle burst traffic
+- **Use Case**: Scenarios where you need to process large volumes of data and can tolerate occasional message drops during peak loads
+
+#### 2. Low Latency Configuration (`low_latency()`)
+
+Optimized for real-time scenarios, prioritizing latency over throughput:
+
+```rust
+let config = StreamClientConfig::low_latency();
+// Or use convenience methods
+let grpc = YellowstoneGrpc::new_low_latency(endpoint, token)?;
+let shred = ShredStreamGrpc::new_low_latency(endpoint).await?;
+```
+
+**Features:**
+- **Backpressure Strategy**: Block - ensures no data loss
+- **Buffer Size**: 1 permit to minimize memory usage
+- **Immediate Processing**: No buffering, processes events immediately
+- **Use Case**: Scenarios where every millisecond counts and you cannot afford to lose any events, such as trading applications or real-time monitoring
+
+#### 3. Async Processing Configuration (`async_processing()`)
+
+Balances throughput and reliability:
+
+```rust
+let config = StreamClientConfig::async_processing();
+// Or use convenience methods
+let grpc = YellowstoneGrpc::new_async_processing(endpoint, token)?;
+let shred = ShredStreamGrpc::new_async_processing(endpoint).await?;
+```
+
+**Features:**
+- **Backpressure Strategy**: Async - non-blocking operation
+- **Buffer Size**: 5,000 permits for steady flow
+- **Fire-and-forget**: Async processing semantics
+- **Use Case**: Scenarios where you need sustained high throughput with eventual consistency, such as data ingestion pipelines or event streaming applications
+
+### Custom Configuration
+
+You can also create custom configurations:
+
+```rust
+let config = StreamClientConfig {
+    connection: ConnectionConfig {
+        connect_timeout: 30,
+        request_timeout: 120,
+        max_decoding_message_size: 20 * 1024 * 1024, // 20MB
+    },
+    backpressure: BackpressureConfig {
+        permits: 2000,
+        strategy: BackpressureStrategy::Block,
+    },
+    enable_metrics: true,
+};
+```
+
+### Configuration Selection Guide
+
+| Scenario | Recommended Config | Reason |
+|----------|-------------------|---------|
+| Trading Bots | `low_latency()` | Need fastest response time, cannot lose trading signals |
+| Data Analytics | `high_throughput()` | Need to process large amounts of historical data, can tolerate some data loss |
+| Event Stream Processing | `async_processing()` | Balance performance and reliability, suitable for continuous processing |
+| Real-time Monitoring | `low_latency()` | Need immediate response to anomalies |
+| Bulk Data Ingestion | `high_throughput()` | Prioritize overall throughput |
+
 ## Usage Examples
 
 ### Quick Start - Parse Transaction Events