Commit Graph

48 Commits

Author SHA1 Message Date
0xfnzero ad166ae62e fix: expose PumpFun create quote accounts 2026-06-12 01:08:33 +08:00
0xfnzero 590801db7c Add PumpFun trade filter and create ix_name support 2026-06-03 01:07:45 +08:00
0xfnzero 5a282bd84d Release solana-streamer-sdk v1.5.4 2026-05-28 04:58:18 +08:00
0xfnzero 108b9a616f Release solana-streamer-sdk v1.4.11 2026-05-25 03:38:28 +08:00
0xfnzero ea0f9b9145 Release solana-streamer-sdk v1.4.6 2026-05-18 01:53:25 +08:00
0xfnzero 92bac16fd4 Release solana-streamer-sdk v1.4.3
Route gRPC, ShredStream, and account parsing through sol-parser-sdk.

Remove local protocol parsing implementations.

Depend on sol-parser-sdk v0.4.8 from crates.io.
2026-05-16 06:45:35 +08:00
0xfnzero ab84249f4a feat: bridge streamer to sol-parser-sdk 2026-05-15 05:09:29 +08:00
Wood bdf984f2c1 chore: release v1.3.0
Made-with: Cursor
2026-03-09 00:41:28 +08:00
Wood 20a1048020 feat(pumpfun): parse 17th instruction account (account field)
- Add PumpFunTradeEvent.account: Option<Pubkey> for the 17th instruction
  account (index 16), labeled as 'Account' on block explorers.
- Fill account in parse_buy_instruction, parse_buy_exact_sol_in_instruction,
  and parse_sell_instruction when accounts.get(16) is present.
- Merge account from CPI log event into instruction event when merging.

Made-with: Cursor
2026-03-08 11:08:46 +08:00
Wood 7270372604 feat(shred): tx_index, Migrate without CPI, merge fix, doc limits
- Add tx_index to TransactionWithSlot (entry index), pass to parser
- Emit PumpFun Migrate even when no CPI (shred gets instruction-only)
- Merge: only overwrite Create/CreateV2 with CPI when value non-default
- Rename transaction_index -> tx_index across codebase
- Add docs/SHREDSTREAM_LIMITATIONS.md (ALT/CPI limits, field completeness)
- Comment shred path: static_account_keys only, no inner_instructions

Made-with: Cursor
2026-03-07 12:35:57 +08:00
Wood 04df6db0b0 fix(pumpfun): add account length check for create_v2 to avoid panic
- Require accounts.len() >= 16 in parse_create_v2_token_instruction
- Require accounts.len() >= 14 in parse_create_token_instruction

Made-with: Cursor
2026-03-07 12:11:50 +08:00
Wood b281a4be52 idl: sync pump_amm; parser account comments; low-latency: sequential inner parse (no thread::scope)
Made-with: Cursor
2026-03-07 11:45:40 +08:00
Estereg c2f2113211 feat(pumpfun): align IDL for V2, Mayhem & Cashback
Update BondingCurve & Global structs, fix account mappings (Create/Buy/Sell), improve trade event merging, add missing fields to create events, and fix create_v2 user mapping.
2026-02-21 09:51:02 +00:00
Estereg 8e99720ce3 fix(pumpfun): fix trade event log size mapping
Adjusted PUMPFUN_TRADE_EVENT_LOG_SIZE from 274 to 250 bytes in the PumpFun event parser to match the actual IDL structure. This resolves an issue where inner instruction events were not parsed correctly, causing trade fields (amounts, reserves, fees) to appear as zeros.
2026-02-21 08:57:31 +00:00
Wood 308a1b594e Align PumpFun with IDL and sol-parser-sdk; fix warnings; use English comments
- Sync IDL from sol-parser-sdk (pumpfun, pump_amm, add raydium_amm_v4)
- PumpFun CreateEvent: add is_cashback_enabled to CreateToken/CreateV2, parse in log decode and merge
- PumpFun TradeEvent: add ix_name, mayhem_mode, cashback_fee_basis_points, cashback, is_cashback_coin; fix Borsh size 250->274 for fixed fields; parse extension after 274
- Fix deprecation: enable solana-entry feature agave-unstable-api
- Remove unused import wide::CmpEq in simd_utils
- Replace all Chinese comments in modified code with English

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-20 00:29:01 +08:00
Wood da550e0c3a fix: correct parameter order for buy_exact_sol_in and align Raydium CPMM field names with IDL
This commit addresses two critical issues:

1. PumpFun buy_exact_sol_in Parameter Order Fix
   - Fixed incorrect parameter parsing in buy_exact_sol_in instruction
   - Created dedicated parse_buy_exact_sol_in_instruction function
   - Correct order: spendable_sol_in (SOL) first, min_tokens_out (token) second
   - Previous bug: reused parse_buy_instruction which has reversed order
   - Impact: Fixes ~4-5% of PumpFun trades that were being parsed incorrectly
   - Issue reported by community: SOL and token amounts were swapped

2. Raydium CPMM Field Naming Alignment
   - Updated PoolState struct field names to match IDL specification exactly
   - Changed token0_vault -> token_0_vault (and similar for all token fields)
   - Changed mint0_decimals -> mint_0_decimals
   - Changed protocol_fees_token0 -> protocol_fees_token_0
   - Ensures consistency with official IDL from solana-program-idls repo
   - Improves code maintainability and readability

3. IDL Files Sync
   - Added latest IDL files from https://github.com/0xfnzero/solana-program-idls
   - Includes: pumpfun, pump_amm, raydium_cpmm, raydium_clmm, raydium_pool_v4
   - Also added: meteora_amm, meteora_damm_v2, meteora_dlmm, orca_whirlpool, raydium_launchpad
   - All IDL files stored in idl/ directory for reference

Files Modified:
- src/streaming/event_parser/protocols/pumpfun/parser.rs
- src/streaming/event_parser/protocols/raydium_cpmm/types.rs
- src/streaming/event_parser/protocols/raydium_cpmm/events.rs
- src/streaming/event_parser/protocols/raydium_cpmm/parser.rs

Testing:
-  Compiles successfully (cargo build --release)
-  All field name updates verified
-  Parameter order matches IDL specification

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-23 16:47:55 +08:00
Wood ce2931046d fix: sync all protocol structures with latest IDL definitions
This commit fixes multiple critical issues found when comparing SDK structures
with the official IDL repository (https://github.com/0xfnzero/solana-program-idls):

1. PumpSwap GlobalConfig
   - Add missing reserved_fee_recipients: [Pubkey; 7] field
   - Update GLOBAL_CONFIG_SIZE constant

2. PumpFun Global
   - Add missing reserved_fee_recipients: [Pubkey; 7] field
   - Update GLOBAL_SIZE constant

3. Raydium CPMM AmmConfig
   - Add missing creator_fee_rate: u64 field
   - Adjust padding from [u64; 16] to [u64; 15]

4. Raydium CPMM PoolState
   - Add creator_fee_on: u8 field
   - Add enable_creator_fee: bool field
   - Add padding1: [u8; 6] field
   - Add creator_fees_token_0: u64 field
   - Add creator_fees_token_1: u64 field
   - Adjust padding from [u64; 31] to [u64; 28]

5. Bonk (Raydium Launchpad) PoolState
   - Add AmmCreatorFeeOn enum with borsh discriminant support
   - Add token_program_flag: u8 field
   - Add amm_creator_fee_on: AmmCreatorFeeOn field
   - Add platform_vesting_share: u64 field
   - Fix padding from [u64; 8] to [u8; 54] with serde_big_array
   - Update POOL_STATE_SIZE constant

6. Bonk (Raydium Launchpad) PlatformConfig
   - Add BondingCurveParam and PlatformCurveParam structures
   - Fix name from Vec<u8> to [u8; 64] with serde_big_array
   - Fix web from Vec<u8> to [u8; 256] with serde_big_array
   - Fix img from Vec<u8> to [u8; 256] with serde_big_array
   - Add cpswap_config: Pubkey field
   - Add creator_fee_rate: u64 field
   - Add transfer_fee_extension_auth: Pubkey field
   - Add platform_vesting_wallet: Pubkey field
   - Add platform_vesting_scale: u64 field
   - Add platform_cp_creator: Pubkey field
   - Fix padding from Vec<u8> to [u8; 108] with serde_big_array
   - Add curve_params: Vec<PlatformCurveParam> field
   - Update PLATFORM_CONFIG_SIZE constant

7. PumpSwap buy_exact_quote_in parameter order bug
   - Create separate parse_buy_exact_quote_in_instruction function
   - Fix parameter parsing order: spendable_quote_in (SOL) first, then min_base_amount_out (token)
   - Previous bug: buy_exact_quote_in incorrectly reused parse_buy_instruction which has reversed parameter order
   - Add detailed comments explaining the parameter order difference

All changes have been verified against the official IDL files and compile successfully.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-23 15:40:26 +08:00
vibes ed8831cc8a feat: Add support for buy_exact_sol_in and buy_exact_quote_in instructions
PumpFun and PumpSwap have additional buy instruction variants that were not being parsed:

- PumpFun: `buy_exact_sol_in` (discriminator: [56, 252, 116, 8, 158, 223, 205, 95])
- PumpSwap: `buy_exact_quote_in` (discriminator: [198, 46, 21, 82, 180, 217, 232, 112])

These discriminators are from the official PumpFun IDL:
https://github.com/pump-fun/pump-public-docs/tree/main/idl

Without this fix, approximately 4-5% of trades were being missed.
2026-01-06 10:14:06 +00:00
ysq 6e3d2c23d1 feat: support Pump.fun V2 (Mayhem Mode) and update protocol structures for Nov 11 breaking changes 2025-11-09 23:59:56 +08:00
ysq 9e4f71dbb2 feat: support Pump.fun V2 (Mayhem Mode) and update protocol structures for Nov 11 breaking changes 2025-11-09 23:30:48 +08:00
ysq 819b71ce1f refactor: unify event parsing API with callback ownership transfer 2025-11-04 22:49:40 +08:00
ysq 4c0e29865f refactor: replace static CONFIGS with dynamic event dispatcher 2025-11-03 23:58:19 +08:00
ysq fd180935f8 refactor: rename UnifiedEvent to DexEvent 2025-10-12 22:00:15 +08:00
ysq e21f2cfd17 refactor: convert EventParser to static methods and optimize caching
- Replace instance-based EventParser with static method design
  - Introduce ParserCache module for protocol config caching
  - Simplify EventProcessor to pure functional handlers
  - Optimize metrics: remove min/max tracking, keep last + avg only
  - Reduce atomic operations in hot path for better performance
2025-10-12 17:20:50 +08:00
ysq 312a675a40 refactor: restructure event handling from trait objects to enum pattern
- Convert UnifiedEvent from trait object (Box<dyn UnifiedEvent>) to concrete enum type
  - Remove match_event! macro in favor of native match expressions
  - Simplify event metadata access: event.event_type() -> event.metadata().event_type
  - Unify event callback signatures: Fn(Box<dyn UnifiedEvent>) -> Fn(UnifiedEvent)
  - Update all example code to use the new enum-based event system
  - Optimize type system to reduce runtime overhead and improve type safety

  Affected scope:
  - Core event parser and processor
  - All protocol event definitions (PumpFun, PumpSwap, Bonk, Raydium series, etc.)
  - gRPC and Shred streaming modules
  - All example code
2025-10-10 18:19:44 +08:00
ysq 317e990ed2 feat: upgrade to v0.4.10 2025-09-14 23:26:21 +08:00
ysq a76563314f refactor: overhaul event parser architecture for better performance
- Centralize event parsing logic and remove factory pattern
- Add high-performance clock module to reduce latency
- Consolidate protocol parsers with unified interface
- Introduce account pubkey caching and streamline traits
- Remove deprecated macro-based and multi-protocol implementations
2025-09-14 01:13:11 +08:00
ioxde 8ec1457858 feat(event_parser): add requires_inner_instruction field to skip incomplete events
- Add requires_inner_instruction field to GenericEventParseConfig
  - Skip events that require but lack inner instruction data during processing
  - Set PumpFun migrate events to require inner instructions (prevents parsing failed migrations)
2025-09-08 20:19:13 -07:00
ysq b71a83bf66 perf: add object pooling and enhanced metrics for streaming optimization
- Implement gRPC/shred connection pools for memory efficiency
- Add atomic processing time stats with auto-calibration clock
- Optimize event parsers and protocol handlers
- Refactor metrics system for advanced performance monitoring
2025-09-02 17:27:27 +08:00
ysq 74781e5cbe perf: implement SIMD-accelerated event processing and optimize streaming performance
- Add SIMD utilities for fast byte array comparison and discriminator matching
- Optimize event processor with batch processing and memory pool
- Refactor global state management with concurrent data structures
- Remove deprecated batch processing module
- Enhance metrics collection with reduced overhead
- Improve parser efficiency across all protocol implementations
- Add performance benchmarking dependencies (criterion, wide)
- Update documentation and examples for new architecture

Performance improvements:
- SIMD-accelerated byte operations for instruction parsing
- Concurrent HashMap (DashMap) for better multi-threading
- Optimized memory allocation patterns
- Reduced lock contention in event processing pipeline

Breaking changes: Removed batch.rs module, updated parser interfaces
2025-08-31 22:18:18 +08:00
ysq 218abd3aa4 perf: Major event processing system refactor for improved performance 2025-08-28 14:49:56 +08:00
ysq 9ea4dab4df perf: Major event processing system refactor for improved performance 2025-08-27 21:31:31 +08:00
ysq dee683396d perf: Refactor event processing system for better performance 2025-08-26 17:57:39 +08:00
ysq e6912a8915 fix: Upgrade to v0.3.8 and fix parsing security issues
- Upgrade to version 0.3.8
- Add data length validation to prevent out-of-bounds access
- Fix boundary check issues in all protocol parsers
2025-08-22 17:58:57 +08:00
ysq 372bb765af feat: enhance protocol event parsers and bump to v0.3.5
- Add creator_fee support to Bonk events
- Improve PumpFun/PumpSwap event parsing
- Update documentation and common types
2025-08-19 21:13:22 +08:00
ysq 915209abd8 fix: use fixed-size slices for protocol data parsing
- Improve borsh deserialization safety across all protocol decoders
- Add MarketState support for raydium_amm_v4
- Prevent buffer overflow in data decoding operations
2025-08-19 17:37:17 +08:00
ysq 89f4a85a11 feat: upgrade to v0.3.1 with enhanced PumpFun event parser 2025-08-13 23:48:02 +08:00
ysq aee8921ad5 feat: Add account event parser and protocol type definitions
- Add account event parser (account_event_parser.rs) for account-level event handling
- Introduce type definitions for pumpfun, pumpswap, raydium_amm_v4, raydium_clmm, raydium_cpmm protocols
- Enhance event processor to support account event processing alongside transactions
- Improve subscription system with separate transaction and account filters
- Optimize parser configuration using Option types for cleaner code structure
- Update examples and documentation to reflect new capabilities

Changes include:
- 6 new types.rs files for protocol-specific data structures
- Updated event processor for account, transaction, and block meta events
- Refactored subscription manager with account filtering support
- Enhanced metrics and batch processing logic
2025-08-13 23:30:59 +08:00
ysq 5d1efa3c1e feat: upgrade to v0.2.3 2025-08-12 10:31:55 +08:00
ysq 8e3d64a09c feat: Add Raydium AMM V4 support and enhance protocol parsers
- Add complete Raydium AMM V4 protocol integration
- Enhance PumpFun, CLMM, CPMM event parsing
- Refactor parser architecture for better extensibility
- Update documentation and examples
2025-08-11 22:02:09 +08:00
ysq 97faba9406 feat: Major refactor with batch processing and performance optimization
- Refactor streaming architecture with modular grpc components
- Add batch processing system with backpressure strategies
- Implement performance monitoring and metrics collection
- Add multi-protocol parser with block metadata support
- Enhance configuration system with preset profiles
- Add parse_tx_events example and update documentation
- Optimize yellowstone_grpc client complexity
2025-08-11 01:04:16 +08:00
ysq 87465cbfdb feat: upgrade to Solana 2.3.x and fix event parsing
- Bump version to 0.1.11
- Upgrade Solana deps to 2.3.x, yellowstone-grpc to 8.0.0
- Fix inner instruction indexing and timestamp handling
- Improve PumpFun/PumpSwap parser program ID support
2025-08-05 18:23:01 +08:00
wood 4c90931e28 events optimization 2025-08-03 05:37:04 +08:00
ysq 2cbab93346 feat: upgrade to v0.1.8 and add volume accumulator support
- Bump version from 0.1.7 to 0.1.8
- Update version references in README.md and README_CN.md
- Add global_volume_accumulator and user_volume_accumulator fields to PumpFunTradeEvent
- Fix account index mapping in trade parser (creator_vault index adjusted from 8 to 9)

This update introduces volume accumulator support for PumpFun protocol event parsing,
aligning with the latest protocol account structure changes.
2025-08-01 22:56:31 +08:00
ysq 0a4be48b99 feat: add event processing time tracking (v0.1.6)
- Add processing duration monitoring
- Update all protocol parser interfaces
- Improve PumpFun dev address detection
- Bump version to 0.1.6
2025-07-29 14:48:43 +08:00
ysq 2e864dd14e feat(api): Upgrade to v0.1.5 with improved event subscription API
This update includes the following improvements:
- Version upgraded to 0.1.5
- Added subscribe_events_v2 API with more granular account filtering
- Marked original subscribe_events API as deprecated
- Fixed bug
2025-07-26 17:11:28 +08:00
wood 3b89ca381c add block_time and block_time_ms 2025-07-25 15:32:05 +08:00
ysq 9e34a01874 feat: refactor to multi-protocol event streaming system
Major architectural refactor, upgrading from simple logging system to comprehensive multi-protocol Solana DEX event streaming system:

 New Features:
- Support for 5 DEX protocols: PumpFun, PumpSwap, Bonk, Raydium CPMM, Raydium CLMM
- Implement unified event interface (UnifiedEvent trait) and event factory pattern
- Add dual streaming support: Yellowstone gRPC and ShredStream
- Add Chinese documentation (README_CN.md)

🏗️ Architectural Improvements:
- Refactor event parsing system with modular design
- Implement protocol-specific parsers and event types
- Optimize dependency management, update Cargo.toml
- Remove legacy logging modules, clean up redundant code

📊 Statistics:
- Added 46 files, 4511 lines of code
- Removed 1381 lines of legacy code
- Net addition of 3130 lines of code

Tech Stack:
- Rust async/await for asynchronous processing
- Protocol Buffers support
- Multi-protocol event parsing
- High-performance event stream subscription
2025-07-19 23:46:42 +08:00