mirror of
https://github.com/chainstacklabs/pumpfun-bonkfun-bot.git
synced 2026-11-02 06:16:10 +00:00
feat(monitoring): add shreds listener for pre-execution token detection (#220)
* feat(monitoring): add shreds listener for pre-execution token detection Adds a `shreds` listener backed by geyser's SubscribeDeshred, which delivers transactions when entries form from shreds, before execution. pump.fun only; bots/bot-sniper-5-shreds.yaml ships disabled. With no TransactionStatusMeta there is no CreateEvent, so the listener decodes the create instruction and resolves what the buy needs itself. It marks its tokens state_from_event, because there is no curve to read before the create executes: trading on the instruction or not trading at all are the only options. Two fixes to the shared instruction-decode path, which geyser and blocks also reach when logs are absent: - `user` is account 5 on create_v2, not 7. create_v2 drops the metaplex accounts legacy create carries, so index 7 is the token program and every address derived from the wallet was wrong. - A holder-reward coin's creator is PDA(["holder-rewards", mint]). The program ignores args.creator on those coins and writes that PDA into BondingCurve.creator, so creator_vault must derive from it or the buy is rejected on a seeds constraint. Subscribes on the mint authority rather than the program id: it appears in create/create_v2 and in no trade instruction, which keeps the stream to creations instead of every pump.fun transaction. Verified live on mainnet with extreme_fast_mode on and off and both the time_based and tp_sl exits, each ending with the wallet holding SOL only. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: drop reported measurement statistics from prose Removes sample counts, traffic percentages, latency percentiles, window durations and "measured on <date>" claims from docstrings, comments, docs and bot YAML. The fact each measurement supported is kept and stated plainly. Numbers that are load-bearing rather than reporting stay: protocol and packet limits, the CU figures used to tune compute budgets, and dates pinning on-chain or dependency upgrades. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Anton Sauchyk
co-authored by
Claude Opus 5
Claude Opus 5
parent
1e2de43292
commit
2ba40c0771
@@ -172,8 +172,8 @@ What the 13.0.0-rc4 protos changed, all verified live on 2026-09-23:
|
||||
v0 and v1 alike. The field is set only for v1 and carries the inline budget
|
||||
(priority fee in **total lamports**, CU limit, loaded-accounts data size,
|
||||
heap) that v1 moved off the ComputeBudget instructions — so a v1 coin has no
|
||||
ComputeBudget instructions to read a budget from at all. Measured share of
|
||||
pump.fun traffic: 660 of 4,215 transactions (15.7%) in a 90s window.
|
||||
ComputeBudget instructions to read a budget from at all. A meaningful share
|
||||
of pump.fun traffic is v1.
|
||||
- Protos older than this **skipped `config` as an unknown field rather than
|
||||
failing**, so the stale stubs degraded in silence: frames still decoded, only
|
||||
the budget went missing. `verify_transaction_v1.py` now pins the geyser route
|
||||
@@ -187,32 +187,36 @@ What the 13.0.0-rc4 protos changed, all verified live on 2026-09-23:
|
||||
debugging a v1 gap.
|
||||
|
||||
`SubscribeDeshred` is a separate RPC delivering transactions **before
|
||||
execution**, when entries form from shreds. It is **deliberately not wired into
|
||||
the bot**, for two reasons, the second of which is disqualifying.
|
||||
execution**, when entries form from shreds. It backs the **`shreds` listener**
|
||||
(`monitoring/universal_shreds_listener.py`, pump.fun only,
|
||||
`bots/bot-sniper-5-shreds.yaml`), which is off by default and carries one
|
||||
permanent cost.
|
||||
|
||||
Raced against `Subscribe` on the pump.fun filter, 300s on 2026-09-23: identical
|
||||
signature sets, no orphans either way. On creates deshred is first 92% of the
|
||||
time, median **+6.4ms**, p90 +21.5ms; on general traffic 79%, median +4.0ms.
|
||||
`tools/compare_deshred_latency.py` reproduces it — the far tail swings a lot
|
||||
between runs, the median does not.
|
||||
Raced against `Subscribe` on the pump.fun filter, deshred arrives first on most
|
||||
creates by a small margin; `tools/compare_deshred_latency.py` reproduces the
|
||||
comparison.
|
||||
|
||||
- There is no `TransactionStatusMeta`, so no `meta.log_messages`, so no
|
||||
CreateEvent. That leaves `state_from_event` unset on every coin and silently
|
||||
demotes `extreme_fast_mode` to a curve refresh.
|
||||
CreateEvent — the listener decodes the create instruction, which is the
|
||||
fallback route everywhere else and the only route here. It sets
|
||||
`state_from_event` itself so `extreme_fast_mode` submits with zero RPC:
|
||||
waiting is not the safer option, because the curve account does not exist
|
||||
yet, so a refresh can only time out and skip the coin.
|
||||
- Outcomes are unknown at detection — nothing has executed, so a create that
|
||||
goes on to revert is delivered exactly like one that lands.
|
||||
- **A coin created through a router is invisible on it.** The create arrives as
|
||||
a CPI, and inner instructions are produced *by* execution, so a pre-execution
|
||||
stream never carries them. Measured 4 of 118 creates (3.4%) in one window and
|
||||
3 of 101 in another — every one present on the stream and undetectable, not
|
||||
dropped, so no decoding recovers them. Note this inverts the CPI note under
|
||||
*Listener and decoder pitfalls*: **trades** are overwhelmingly inner
|
||||
instructions, **creates** are overwhelmingly top-level (114:4), which is why
|
||||
deshred looks nearly complete until you count.
|
||||
stream never carries them — every one present on the stream and undetectable,
|
||||
not dropped, so no decoding recovers them. Note this inverts the CPI note
|
||||
under *Listener and decoder pitfalls*: **trades** are overwhelmingly inner
|
||||
instructions, **creates** are overwhelmingly top-level, which is why deshred
|
||||
looks nearly complete until you count.
|
||||
|
||||
Trading a permanent few-percent miss rate on new coins for a 6ms median is not
|
||||
a trade this bot should take.
|
||||
`cookbook/pumpfun/listen/pumpfun_listen_tokens_deshred.py` demonstrates the
|
||||
stream and that trade-off; it decodes instructions because it has no logs to
|
||||
route on.
|
||||
That miss rate is the whole trade: a few percent of coins never seen, against a
|
||||
head start on the rest. `geyser` remains the default for that reason; pick
|
||||
`shreds` deliberately.
|
||||
`cookbook/pumpfun/listen/pumpfun_listen_tokens_deshred.py` demonstrates the raw
|
||||
stream, and `tests/regression/verify_shreds_listener.py` pins the listener.
|
||||
|
||||
Where the Solana libraries live, since solana-py 0.40 moved most of them
|
||||
(upgraded 2026-09-23 from solders 0.26 / solana 0.36.6):
|
||||
@@ -243,6 +247,7 @@ name individual scripts to run a subset.
|
||||
| `verify_curve_account_sizes.py` | 125/151/256-byte curves all decode, and nothing filters on account length |
|
||||
| `verify_create_v2_optional_args.py` | omitted trailing option-typed `create_v2` args decode as unset; mandatory args still fail |
|
||||
| `verify_transaction_v1.py` | every reader asks `maxSupportedTransactionVersion: 1`, a v1 `create_v2` is detected from its logs alone with the envelope made unreadable, the envelope decode still works as the fallback, and no cookbook/tools listener detects by opening the envelope; same two routes pinned again over geyser, on a committed v1 `SubscribeUpdate`, plus the inline v1 budget staying readable |
|
||||
| `verify_shreds_listener.py` | pre-execution creates decode from the instruction alone: `user` at `create_v2` account 5, a holder-reward coin's creator derived as `PDA(["holder-rewards", mint])`, truncated trailing args decoding as not-holder-reward, lookup-table accounts resolved, and nothing reading a `meta` the stream has no field for |
|
||||
| `verify_block_null_guard.py` | a `blockSubscribe` frame with `value.block: null` is skipped, not logged as an error |
|
||||
| `verify_listener_cancellation.py` | a cancelled WebSocket listener stops, even when `websockets` reports cancellation as `AssertionError` |
|
||||
| `verify_pumpportal_buy_path.py` | curve derived from the mint, unreadable curve skips the buy, curve+mint read in one slot-consistent batch |
|
||||
@@ -289,7 +294,7 @@ The reasoning behind each lives in the verifier named beside it.
|
||||
they read the same for every transaction version; the byte decode is the
|
||||
fallback. The repo now runs solders 0.29, which *does* read a v1 envelope
|
||||
(0.26, 0.27.1 and 0.28 all raise `ValueError: io error: unexpected end of
|
||||
file` on the committed v1 fixture, measured 2026-09-22) — that does not make
|
||||
file` on the committed v1 fixture) — that does not make
|
||||
the log route redundant, because the next version byte will be unreadable in
|
||||
its turn. `verify_transaction_v1.py` pins both routes separately.
|
||||
- The bot **sends** v0 transactions (`MessageV0.try_compile` +
|
||||
@@ -322,8 +327,8 @@ The reasoning behind each lives in the verifier named beside it.
|
||||
a wrong power of ten inflates the price *and* the slippage cap in the same
|
||||
direction, so they compound into an overspend instead of cancelling. pump.fun's
|
||||
`QuoteControl` registry (PDA `["quote-control"]`) admits mints at 6, 8 and 9
|
||||
decimals — 79 of the 170 admitted on 2026-09-22 are tokenized equities, and
|
||||
coins paired with them trade live (8 of 124 curves in a 75s sample that day).
|
||||
decimals — many of the admitted mints are tokenized equities, and coins
|
||||
paired with them trade live.
|
||||
- `trade.curve_refresh_budget` (seconds, default 2.0) bounds the pre-buy curve
|
||||
read in `extreme_fast_mode`; when it expires the token is **skipped**, because a
|
||||
buy built from listener-guessed defaults reverts with `NotAuthorized` (6000),
|
||||
@@ -331,9 +336,15 @@ The reasoning behind each lives in the verifier named beside it.
|
||||
sell path keeps the opposite fallback — proceed with cached values — since
|
||||
skipping a sell strands the position.
|
||||
- The refresh is skipped entirely when `TokenInfo.state_from_event` is set, i.e.
|
||||
the listener parsed the **CreateEvent**. Instruction-parsed `TokenInfo`
|
||||
deliberately does not set it: `args.creator` is user-supplied and post-2026-04-28
|
||||
may differ from the canonical `BC.creator`. `trade.trust_create_event: false`
|
||||
the listener resolved creator/flags/quote_mint for itself. The **CreateEvent**
|
||||
parsers set it. Instruction-parsed `TokenInfo` does not, because
|
||||
`args.creator` is user-supplied and post-2026-04-28 may differ from the
|
||||
canonical `BC.creator` — but that gap is now understood rather than merely
|
||||
feared, and it has exactly two causes: a **holder-reward** coin (derivable
|
||||
from the mint, see below) and a post-launch `migrate_bonding_curve_creator`
|
||||
(which happens *after* create, so it cannot affect a snipe). The `shreds`
|
||||
listener sets the flag itself on that basis; it has no curve to read and no
|
||||
choice. `trade.trust_create_event: false`
|
||||
forces the refresh back on; PumpPortal payloads always refresh.
|
||||
|
||||
**Selling**
|
||||
@@ -393,9 +404,8 @@ offline and only visible after a couple of minutes against mainnet.
|
||||
was reported as `claim_cashback`. Note `buy_exact_sol_in` is also 18 accounts
|
||||
on chain, same as legacy `buy`.
|
||||
- **Walk `meta.innerInstructions`, not just `message.instructions`.** Most trades
|
||||
reach the program as a CPI from a router or aggregator: in 40 consecutive
|
||||
pump.fun transactions there was **1 top-level** pump instruction against
|
||||
**8 inner** ones. Anchor's event-CPI prefix (`e445a52e51cb9a1d`) accounts for a
|
||||
reach the program as a CPI from a router or aggregator: top-level pump
|
||||
instructions are heavily outnumbered by inner ones. Anchor's event-CPI prefix (`e445a52e51cb9a1d`) accounts for a
|
||||
good share of the inner instructions; the event's own discriminator follows it.
|
||||
- **`getProgramAccounts` over the whole pump program is rejected** by current
|
||||
providers: *"Too many accounts requested (10000001 pubkeys) … use
|
||||
@@ -566,9 +576,11 @@ The IDLs under `idl/` are vendored verbatim from `github.com/pump-fun/pump-publi
|
||||
"omitted fee bps" getTransaction fixture sends `is_cashback_enabled` only
|
||||
(1 byte), and the "with fee bps" getTransaction fixture sends
|
||||
`is_cashback_enabled` + `creator_fee_bps` (9 bytes) — counted directly from
|
||||
each fixture's instruction data, 2026-09-15. `is_holder_reward` (the
|
||||
three-trailing-arg form) is legal per the IDL but has not been observed on
|
||||
the wire — UNVERIFIED whether it is ever sent. A decoder that reads a
|
||||
each fixture's instruction data, 2026-09-15. The three-trailing-arg form
|
||||
**is** sent. Because the args are positional, a shorter form cannot have set
|
||||
a later one — so a truncated instruction is proof the coin is not
|
||||
holder-reward, not merely silence about it, which is what lets a
|
||||
pre-execution listener classify every coin. A decoder that reads a
|
||||
fixed number of trailing bytes raises `IndexError` on the shorter forms.
|
||||
Decode trailing args defensively and report a missing one as unset —
|
||||
`utils/idl_parser.py` does this for trailing option-typed args since #184
|
||||
@@ -602,7 +614,16 @@ holders instead of paid to a creator wallet. On such a coin
|
||||
`BondingCurve.creator` holds a pump.fun address rather than the actual
|
||||
creator's — that's expected, and the `creator_vault` derivation is unchanged
|
||||
and still correct either way, since it derives from whatever `creator` the
|
||||
curve carries. **No trade instruction changed**: `buy`, `sell`, `buy_v2`,
|
||||
curve carries.
|
||||
|
||||
**That substituted address is `PDA(["holder-rewards", mint])` under the pump
|
||||
program** — the same account the IDL declares for `holder_rewards` on
|
||||
`distribute_fee_to_holders`, which is how the ordinary `creator-vault` route
|
||||
delivers the fee to the holder pool without a second payout path. It depends on
|
||||
nothing but the mint, so `args.creator` being unusable on these coins costs no
|
||||
RPC call to work around: `PumpFunAddresses.find_holder_reward_creator` derives
|
||||
it. A meaningful share of launches are holder-reward, so this is not an edge
|
||||
case. **No trade instruction changed**: `buy`, `sell`, `buy_v2`,
|
||||
`sell_v2` and the PumpSwap instructions take identical accounts and arguments
|
||||
whether or not a coin is a holder-reward coin. `Global.is_holder_reward_enabled`
|
||||
can switch creation off globally.
|
||||
@@ -643,15 +664,14 @@ interface pump.fun maintains.
|
||||
checks this against committed fixtures, and `--live` re-checks it against the
|
||||
real feed. The bonk **trade** path past detection is still unverified — see
|
||||
issue #201.
|
||||
- **PumpPortal does not report coins created in a transaction v1.** Measured
|
||||
2026-09-16 across two windows: 0 of 6 v1 creates pushed, against near-complete
|
||||
coverage of v0 creates (104 of 105 in one 240s window). It is a third-party
|
||||
feed, so no local change recovers them — the coin is never sent. This compounds
|
||||
- **PumpPortal does not report coins created in a transaction v1**, while its
|
||||
coverage of v0 creates is near-complete. It is a third-party feed, so no local change recovers them — the coin is never sent. This compounds
|
||||
the thin-bonk-payload gap below, and it means `pumpportal` is a sampling feed
|
||||
now, not a complete one. `bots/bot-sniper-4-pp.yaml` carries the same warning.
|
||||
- `src/config_loader.py` (repo root of the package, not under `core/`) validates the platform/listener pairing before startup:
|
||||
pump.fun supports `logs`, `blocks`, `geyser`, `pumpportal`; letsbonk.fun
|
||||
supports `blocks`, `geyser`, `pumpportal` — **not `logs`**. Adding a listener
|
||||
pump.fun supports `logs`, `blocks`, `geyser`, `shreds`, `pumpportal`;
|
||||
letsbonk.fun supports `blocks`, `geyser`, `pumpportal` — **not `logs`**, and
|
||||
**not `shreds`** (its instruction parse path is unverified, see #201). Adding a listener
|
||||
means updating `PLATFORM_LISTENER_COMPATIBILITY` there too.
|
||||
- Bots with `separate_process: true` run in their own process. One log file per
|
||||
bot instance.
|
||||
|
||||
@@ -58,7 +58,7 @@ Fill in `.env`:
|
||||
| `SOLANA_NODE_RPC_ENDPOINT` | HTTPS RPC endpoint |
|
||||
| `SOLANA_NODE_WSS_ENDPOINT` | WebSocket endpoint (for `logs` / `blocks` listeners) |
|
||||
| `SOLANA_PRIVATE_KEY` | Base58 private key of the trading wallet |
|
||||
| `GEYSER_ENDPOINT`, `GEYSER_API_TOKEN`, `GEYSER_AUTH_TYPE` | Only for the `geyser` listener |
|
||||
| `GEYSER_ENDPOINT`, `GEYSER_API_TOKEN`, `GEYSER_AUTH_TYPE` | For the `geyser` and `shreds` listeners (same endpoint, different RPC) |
|
||||
|
||||
Public RPC nodes will not work for this workload — see [throughput](#throughput-and-rate-limits) below.
|
||||
|
||||
@@ -72,10 +72,11 @@ Each YAML file in `bots/` is one bot instance. They ship with commented defaults
|
||||
| `bot-sniper-2-logs.yaml` | `logs` — `logsSubscribe`, supported everywhere | `pump_fun` |
|
||||
| `bot-sniper-3-blocks.yaml` | `blocks` — `blockSubscribe`, not supported by every provider | `pump_fun` |
|
||||
| `bot-sniper-4-pp.yaml` | `pumpportal` — third-party aggregator, misses some coins | `lets_bonk` |
|
||||
| `bot-sniper-5-shreds.yaml` | `shreds` — pre-execution, ahead of `geyser`, cannot see router-created coins | `pump_fun` |
|
||||
|
||||
Set `platform: "pump_fun"` or `platform: "lets_bonk"`. pump.fun supports all four listeners; letsbonk.fun supports `blocks`, `geyser`, and `pumpportal` but **not** `logs`. The bot validates the pairing at startup and refuses to run an invalid one.
|
||||
Set `platform: "pump_fun"` or `platform: "lets_bonk"`. pump.fun supports all five listeners; letsbonk.fun supports `blocks`, `geyser`, and `pumpportal` but **not** `logs` or `shreds`. The bot validates the pairing at startup and refuses to run an invalid one.
|
||||
|
||||
`pumpportal` is a third-party feed and only reports what it indexes. As of 2026-09-16 it does not push coins whose creation landed in a Solana transaction v1 (a format live since 2026-09-15), so it sees a sample rather than everything. `geyser` and `logs` read the chain directly and are unaffected; `blocks` needs `maxSupportedTransactionVersion: 1`, which it now sends.
|
||||
`pumpportal` is a third-party feed and only reports what it indexes. It does not push coins whose creation landed in a Solana transaction v1 (a format live since 2026-09-15), so it sees a sample rather than everything. `geyser` and `logs` read the chain directly and are unaffected; `blocks` needs `maxSupportedTransactionVersion: 1`, which it now sends.
|
||||
|
||||
Set `enabled: false` to keep a config around without running it. Every bot with `enabled: true` starts when you run the bot.
|
||||
|
||||
|
||||
@@ -91,9 +91,9 @@ filters:
|
||||
match_string: null # Only process tokens with this string in name/symbol
|
||||
bro_address: null # Only trade tokens created by this user address
|
||||
# PumpPortal is a third-party feed, so it only reports what it indexes.
|
||||
# Measured 2026-09-16: it reported 0 of 6 coins whose create landed in a
|
||||
# Solana transaction v1 (live since 2026-09-15), against near-complete
|
||||
# coverage of v0 creates. Nothing in this repo can recover those — the
|
||||
# It does not report coins whose create landed in a Solana transaction v1
|
||||
# (live since 2026-09-15), while its coverage of v0 creates is
|
||||
# near-complete. Nothing in this repo can recover those — the
|
||||
# coin is simply never pushed. Use "geyser" or "logs" if you need every
|
||||
# coin; see tests/regression/verify_transaction_v1.py.
|
||||
listener_type: "pumpportal" # Method for detecting new tokens: "logs", "blocks", "geyser", or "pumpportal"
|
||||
|
||||
@@ -0,0 +1,152 @@
|
||||
# Shreds mode: reads transactions from the geyser SubscribeDeshred stream
|
||||
# BEFORE they execute, which is earlier than the ordinary geyser listener.
|
||||
#
|
||||
# A few notes:
|
||||
#
|
||||
# 1. A coin created through a router is INVISIBLE. The create then arrives as
|
||||
# a CPI, and inner instructions are produced by execution, so no
|
||||
# pre-execution stream carries them. No setting recovers those coins -- if
|
||||
# you need every coin, use listener_type "geyser".
|
||||
# 2. Outcomes are unknown at detection. Nothing has executed, so a create that
|
||||
# goes on to revert looks exactly like one that lands, and the buy against
|
||||
# it costs a fee for nothing.
|
||||
#
|
||||
# extreme_fast_mode below is deliberately true: with it off the bot waits for
|
||||
# the curve to stabilize, which spends the latency advantage and leaves only
|
||||
# the costs above.
|
||||
|
||||
# This file defines comprehensive parameters and settings for the trading bot.
|
||||
# Carefully review and adjust values to match your trading strategy and risk tolerance.
|
||||
|
||||
# Bot identification and connection settings
|
||||
name: "bot-sniper-5"
|
||||
env_file: ".env"
|
||||
rpc_endpoint: "${SOLANA_NODE_RPC_ENDPOINT}"
|
||||
wss_endpoint: "${SOLANA_NODE_WSS_ENDPOINT}"
|
||||
private_key: "${SOLANA_PRIVATE_KEY}"
|
||||
|
||||
enabled: false # You can turn off the bot w/o removing its config
|
||||
separate_process: true
|
||||
|
||||
# Options: "pump_fun" (default), "lets_bonk"
|
||||
platform: "pump_fun"
|
||||
|
||||
# Geyser connection. Shreds mode uses the same endpoint and credentials as the
|
||||
# geyser listener -- SubscribeDeshred is a separate RPC on it, not a separate
|
||||
# service.
|
||||
geyser:
|
||||
endpoint: "${GEYSER_ENDPOINT}"
|
||||
api_token: "${GEYSER_API_TOKEN}"
|
||||
auth_type: "x-token" # or "basic"
|
||||
|
||||
# Trading parameters
|
||||
# Control trade execution: amount of SOL per trade and acceptable price deviation
|
||||
trade:
|
||||
buy_amount: 0.0001 # Amount of SOL to spend when buying (in SOL)
|
||||
buy_slippage: 0.3 # Maximum acceptable price deviation (0.3 = 30%)
|
||||
sell_slippage: 0.3
|
||||
|
||||
# Spend amounts for coins paired against a non-SOL quote asset.
|
||||
# pump.fun now supports quote mints other than SOL (USDC first). Amounts are
|
||||
# in that mint's own whole units, so "usdc: 1.0" means one USDC — it is NOT
|
||||
# comparable to buy_amount above. Keys accept the aliases "sol"/"usdc" or a
|
||||
# raw base58 mint address. A coin whose quote mint has no entry here is
|
||||
# skipped rather than bought with a SOL-sized amount.
|
||||
# SOL always uses buy_amount above and needs no entry.
|
||||
#quote_amounts:
|
||||
# usdc: 1.0
|
||||
|
||||
# Exit strategy configuration
|
||||
exit_strategy: "time_based" # Options: "time_based", "tp_sl", "manual"
|
||||
#take_profit_percentage: 0.1 # Take profit at 10% gain (0.1 = 10%)
|
||||
#stop_loss_percentage: 0.1 # Stop loss at 10% loss (0.1 = 10%)
|
||||
max_hold_time: 5 # Maximum hold time in seconds for TP/SL strategy, for time_based - see wait_after_buy
|
||||
#price_check_interval: 2 # Check price every 2 seconds
|
||||
# A tp/sl exit sell that reverts on-chain (slippage, curve moved under it) is
|
||||
# retried on the next price check, re-reading the price first so the slippage
|
||||
# floor matches the market. After this many attempts the bot gives up and
|
||||
# leaves the position open - tokens are still held.
|
||||
#max_exit_sell_attempts: 3
|
||||
|
||||
# EXTREME FAST mode configuration
|
||||
# When enabled, skips waiting for the bonding curve to stabilize and RPC price check.
|
||||
# The bot buys the specified number of tokens directly, making the process faster but less precise.
|
||||
# For the shreds listener the buy is built entirely from the create
|
||||
# instruction — ZERO RPC calls between detection and submission. The curve
|
||||
# account does not exist yet at that point, so this is not merely faster:
|
||||
# it is the only way shreds mode can trade at all (see trust_create_event).
|
||||
extreme_fast_mode: true
|
||||
extreme_fast_token_amount: 20 # Amount of tokens to buy
|
||||
|
||||
# Pre-buy curve refresh (issue #170). Unused in shreds mode while
|
||||
# trust_create_event is true, and kept here for when you switch this bot to
|
||||
# another listener.
|
||||
curve_refresh_budget: 2.0
|
||||
|
||||
# Skip the pre-buy curve read when the listener resolved the creator,
|
||||
# mayhem/cashback flags and quote_mint itself. Setting this false does NOT
|
||||
# make shreds mode safer — the bonding curve has not been created yet, so
|
||||
# the refresh can only time out and skip the coin. Leave it true here and
|
||||
# use listener_type "geyser" if you want the curve read.
|
||||
trust_create_event: true
|
||||
|
||||
# Priority fee configuration
|
||||
# Manage transaction speed and cost on the Solana network.
|
||||
# Note: dynamic mode requires an additional RPC call, which slows down the buying process.
|
||||
priority_fees:
|
||||
enable_dynamic: false # Use latest transactions to estimate required fee (getRecentPrioritizationFees)
|
||||
enable_fixed: true # Use fixed amount below
|
||||
fixed_amount: 1_000_000 # Base fee in microlamports
|
||||
extra_percentage: 0.0 # Percentage increase on riority fee regardless of the calculation method (0.1 = 10%)
|
||||
hard_cap: 1_000_000 # Maximum allowable fee in microlamports to prevent excessive spending
|
||||
|
||||
# Compute unit limits for transaction processing
|
||||
# Operation-specific defaults are used if not specified: buy=100K, sell=60K
|
||||
compute_units:
|
||||
# Override default CU limits for this platform
|
||||
# buy: 100_000 # Buy operations (ATA creation + trading)
|
||||
# sell: 60_000 # Sell operations (just trading)
|
||||
|
||||
# Account data size optimization (reduces CU cost and improves tx priority)
|
||||
# Reduces CU cost from 16k to ~128 CU by limiting loaded account data.
|
||||
# Default is 64MB (16k CU). Setting to 512KB significantly reduces overhead.
|
||||
# Note: Savings don't show in "consumed CU" but improve tx priority/cost.
|
||||
# Note (Nov 23, 2025): with data size set to 512KB, transactions fail - increasing to 12.5MB resolves the issue.
|
||||
# Reference: https://www.anza.xyz/blog/cu-optimization-with-setloadedaccountsdatasizelimit
|
||||
# account_data_size: 12_500_000 # Disabled: causes MaxLoadedAccountsDataSizeExceeded with Token-2022
|
||||
|
||||
# Filters for token selection
|
||||
filters:
|
||||
match_string: null # Only process tokens with this string in name/symbol
|
||||
bro_address: null # Only trade tokens created by this user address
|
||||
# Restrict which quote assets to trade. Omit (or set null) to allow any quote
|
||||
# mint that has an amount configured under trade.quote_amounts.
|
||||
# Accepts "sol"/"usdc" aliases or raw mint addresses.
|
||||
#allowed_quote_mints: ["sol"]
|
||||
listener_type: "shreds" # Pre-execution detection; see the header of this file.
|
||||
# Supported on pump_fun only. Other options: "logs", "blocks", "geyser", "pumpportal".
|
||||
max_token_age: 0.001 # Maximum token age in seconds for processing
|
||||
marry_mode: false # Only buy tokens, skip selling
|
||||
yolo_mode: false # Continuously trade tokens
|
||||
|
||||
# Retry and timeout settings
|
||||
retries:
|
||||
max_attempts: 1 # Number of attempts for transaction submission
|
||||
wait_after_creation: 15 # Seconds to wait after token creation (only if EXTREME FAST is disabled)
|
||||
wait_after_buy: 5 # Holding period after buy transaction
|
||||
wait_before_new_token: 15 # Pause between token trades
|
||||
|
||||
# Token and account management
|
||||
cleanup:
|
||||
# Cleanup mode determines when to manage token accounts. Options:
|
||||
# "disabled": no cleanup will occur.
|
||||
# "on_fail": only clean up if a buy transaction fails.
|
||||
# "after_sell": clean up after selling.
|
||||
# "post_session": clean up all empty accounts after a trading session ends.
|
||||
mode: "post_session"
|
||||
force_close_with_burn: false # Force burning remaining tokens before closing account
|
||||
with_priority_fee: false # Use priority fees for cleanup transactions
|
||||
|
||||
# Node provider configuration
|
||||
node:
|
||||
max_rps: 25 # Maximum requests per second
|
||||
@@ -12,8 +12,8 @@ Two things this example exists to show, because both are easy to get wrong:
|
||||
|
||||
2. Read `meta.innerInstructions` as well as `message.instructions`. Most pump.fun
|
||||
trades reach the program as a CPI from an aggregator or router, so a decoder
|
||||
that only walks the top level sees almost nothing — in a sample of 40 consecutive
|
||||
pump.fun transactions there was 1 top-level pump instruction against 8 inner ones.
|
||||
that only walks the top level sees almost nothing — top-level pump
|
||||
instructions are heavily outnumbered by inner ones.
|
||||
|
||||
3. Accept whichever encoding the response was captured in. `getTransaction`
|
||||
answers in `jsonParsed`, `json` or `base64` depending on what was asked for,
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"_comment": "One pump.fun create_v2 as delivered by geyser SubscribeDeshred (pre-execution). There is no TransactionStatusMeta on this stream, so there are no log messages and no CreateEvent: the create instruction is the only source.",
|
||||
"mint": "9AyUC2BSCmQfuCjB1V77yNYyS8YHXsaq2uvVbXw9pump",
|
||||
"is_holder_reward": true,
|
||||
"create_v2_trailing_arg_bytes": 10,
|
||||
"loaded_lookup_table_accounts": 13,
|
||||
"subscribe_update_base64": "CgFmEoEOCvgNCkDW7GH1ZzFw3wu5/OH88QRU5X0J2ZW1XqOk8jDO0IQJletGISKblHoYPLxePOBAs1bE5HmvY9ilM64cjfYuSUIBGvMJCkDW7GH1ZzFw3wu5/OH88QRU5X0J2ZW1XqOk8jDO0IQJletGISKblHoYPLxePOBAs1bE5HmvY9ilM64cjfYuSUIBCkBQiwSylScjScxgqkphp89clMozxGflyNZT0nkh1FumFbmwyAU7x7d7ojJPY77EXDV+ET5/F+UPqh0VHAp+rFoMEuwICgQIAhgJEiDB/PHN5Ck25qFYzs1KMHvXgZVVB/K/doRH/i1w1pQ/YxIgeWuSeRzAi1v9ZaHYU9v40ukB6dKe8RBEUkeJO5M58Y8SIAeA9I4+XqfNEKiLXbUZgonyxA/ua7cQhB88Rd77a9uIEiA/px6nlYLHqYxMtdlwx5O8DxGx9LiGgA5x9E+SBYUhKhIgRuC8GHq8YcIareXyNDCyPzxxk1NBIrzTjdT05vWo0+gSIEtoRBm0Sh1GbbxjTLGJwaoAD9lsHiN0ds95bv3AporyEiBh0O+XAX1aDvuzLysS4nfDuX8JIZRf2qo6jZjkWVsE+RIge92l5IAzkZvGSnPk5uI1yRJrS6YMsdAspf5wULcF2mESIIEd127HqTmeyzeqIDYwgVsgwIJHrzyXyIykLg6Rf0wfEiDJ9MOoyWb72oi2V2lLUjoibHwYa12gi6grDWhPF+lcpxIgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAASIAFW4PaTZlrPRNsVaL8XW6pRicuX9dL/O2VdK7b9bRiwEiADBkZv5SEXMv/srbpyw5vnvIzlu8X3EmssQ5s6QAAAABIgBHnVW/IxwG7udMVuzmgVB/2xst6j9I5RArHNola8E48SIAbd9uHXZaGT2cvhRs7reawctIXtX1s3kTqM9YV+/wCpEiBY6n2uAJq/JZn175WcnSk5FDty0P/UR+xgddr5f50QIhIgb+tgX91tNg5Dg5Kj0nE0Gxg00NYMgKYkWXL4LlW5GqsSIIyXJY9OJInxuz0QKRSODYMLWhOZ2v8QhASOe9jb6fhZEiC0P/on9df2SnTAmx8pWHneSwmrNt/J3VFLMhqns4zl6BoglJzPAlCXmx/aezUkw9DpRX7+0wZTebudz7fikccDSkciCQgMGgUCwFwVACKvAQgLEhQBGgYCHAAKGxETGBQFAx0LEAQODxqUAdaQTOxfizG0BAAAAE5PTEEEAAAATk9MQU0AAABodHRwczovL2Zhc3QtaXBmcy5jb20vYXN5bmMvOUF5VUMyQlNDbVFmdUNqQjFWNzd5Tll5UzhZSFhzYXEydXZWYlh3OXB1bXAuanNvbsH88c3kKTbmoVjOzUowe9eBlVUH8r92hEf+LXDWlD9jAAABAAAAAAAAAAEiFAgKEgIACRoMAgAAAAAtMQEAAAAAIg0IERIGAAcAGQoOGgEBIhQIChICAAcaDAIAAAAA8gUqAQAAACIICA4SAQcaAREiDQgREgYACAAQCg4aAQEiQAgNEhMABwgZEA4ODRINHw4WHhUXBwgAGie7ZPrMMcSvFADyBSoBAAAADChj6CUAAADuAgAAAAABAAAAaRAnAAEiCggOEgMHAAAaAQkoATIuCiAMTgRReRLiWDL8AJ4vhn3nREHkmFuQXwvfM8VDleBH9BICDlcaBhUfAxYFBDIrCiB7IaeWrpHAP+GCFcNSxU02N7P1poWPoqe4jdJSPogCTRIDo6akGgIFBCIgBSrl16fapySm6rCnKVSRhVrUoGcWYGdMTgNFWYA9ZaMiIKKLX9JqtHmmqcxsv2sLI+thiFo3HgEgrKkTvu89E4p4IiA9Z3j8PnBLHuBw4PQ5pAJz+B+D/Iprn9RrTnVxHQV9ySIgyeescdhqphOtM/Hz0DDKGcc0N3+S39MIc0rY285uXnQiIOlgqrTFZbt0mmx1QG3X7T34DEX2DaFTmYQ7mSJEGdNdKiAAra6ifbPG89BOXPCiAp8D4zmPCFOV7+/vFx/GSvgXQSogBpuIV/6rgYT7aH9jRhjANdrEOdwa6ztVmKDwAAAAAAEqIAbFwc5jjSVn0mRosF65UdGijcxuEjSCtcZ1FJdw5ivyKiAG3fbh7nWP3hhCXbzkbM3athr8TYO5DSf+vfko2KGL/CogOoZeae4PVIDKvPZjV+TcLxjVjUXB6nSJ+zcj2Xk8cqYqIKzxNusB/BxOiD0jyLWESrWaN/Zq3VfF6aw7U+BZ01xkKiBBV7BYDzHF/ORKYlgtvPnXjudZQ6CEo5OzUDaNIomTCCogS9lJxDYCwz8gd5DtFqNSTKG5l1zxIaKpDP/sffi2is0wgAc4oAcQ4sC51gEqCwjJyM/VBhCPt8UI"
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"_comment": "One pump.fun create_v2 as delivered by geyser SubscribeDeshred (pre-execution). There is no TransactionStatusMeta on this stream, so there are no log messages and no CreateEvent: the create instruction is the only source.",
|
||||
"mint": "CJPAzeEsYpjNV4Jqi22f1GR1duRzCxbDfmLSWpFQpump",
|
||||
"is_holder_reward": false,
|
||||
"create_v2_trailing_arg_bytes": 10,
|
||||
"loaded_lookup_table_accounts": 17,
|
||||
"subscribe_update_base64": "CgFmEqwOCqMOCkBXhKZjuSxElbDEc0cKLM8NrF7kOa1ubEbTWFtEMretZTrP31R2EXHdG4pD3QXqeTuS/Xw9BJ/nFgM4mpAY0E4JGpYJCkBXhKZjuSxElbDEc0cKLM8NrF7kOa1ubEbTWFtEMretZTrP31R2EXHdG4pD3QXqeTuS/Xw9BJ/nFgM4mpAY0E4JCkBkE295NBLbtDGXowQs9kdg0tE52QfyA7Sm1W5u1Z+jOIkXpvxC6CRgu2pZPdtOysiJ3VSjWtpAllZTZahR5hQAEo8ICgQIAhgFEiBh7uw65deFVC7QAEBy/+n/sIS57XMPDSj+76oOTqLTgRIgp+SAEISNTfW4TGOVTUVhJJJOU0sAEX0HJrMHaYvymz8SICVALN+0ygQq9BDL7yqIch2i026ZyOT/EVm3sSaSyGnREiAu7nuJKk74pEqr+XDMDjfq9yZ2dRcMHWmc0xJ0SWxUPxIgTfAzVJLibHvMgUlbg7czTnb5RzhDuM1pC5ZRc3NCxRgSIHk13yLqlNyRXjPsH81mRUFmLmqiRvL5r7Hqob0dUvDfEiB6xDJG1mNXg8eblQqpZ1HuDZhbYFstucF+5qWC7dBI4hIgi0Y7dbssC2mnDKDjnEQOs3YfuFH9CltdVdCEObEg+UESIJCvMvE6lBalbKfWXujtoNHSf8/HPrWqLqe8fAnrmph9EiCbyyx/uaVGJA6595kStd1vIxNcCRcjL821y3aPwHSBfhIgrWUIqcg27H6UZT9FcJqP7O0jbf1/qThqLHQ9hpM1zN4SIMkhnahQUX0E+zsjH/l2Uv0lyPVEZXFuOcIqEyz2LVWqEiD2Al9NNfPlMYLaKooDFWRYUq1orYQNLLIu1NEl2b51lxIg/ln71TN/H2X/sSk4IaKgfnllE3gRwX0TSsTmgTta/bQSIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEiABVuD2k2Zaz0TbFWi/F1uqUYnLl/XS/ztlXSu2/W0YsBIgAwZGb+UhFzL/7K26csOb57yM5bvF9xJrLEObOkAAAAASIIyXJY9OJInxuz0QKRSODYMLWhOZ2v8QhASOe9jb6fhZEiDT4fmVLTvYo/j/PzMwZ40u1XG0D3/2eM/nLWoHf2vWVxog4i4R8FX1nm6efwrgpJE3A6+xRJ+jEwzqQevxewd2X3UiDAgQEgEeGgUCIKEHACINCBAaCQMQJwAAAAAAACKmAQgPEhABGwUNIAAOHRETGRQGByIPGo8B1pBM7F+LMbQJAAAAdGFjIG5heW4gBAAAAE5heW5DAAAAaHR0cHM6Ly9pcGZzLmlvL2lwZnMvUW1lQ2c5alRwTVkyR25Yc2d6ZDZnekpOWFVGMjVBelRaN1FpeDlZU2VkVEFyeWHu7Drl14VULtAAQHL/6f+whLntcw8NKP7vqg5OotOBAAAAAAAAAAAAAAAiDQgREgYAAgABDh0aAQEiOQgPEhsgARodHBEXFhgVBQ0KAAIMCwgSIwkEIR8OIg8aGLgX7mFnxdM9cEFSLpEJAAD//////////yIUCA4SAgADGgwCAAAAECcAAAAAAAAoATI3CiD8Qp4gXoenQOKszYWtvBGwxRGBR1TLTn7zyMKm5cKiShIGEBNVPRoiGgsSfJ4MC6APAQ4DBCIgBSrl16fapySm6rCnKVSRhVrUoGcWYGdMTgNFWYA9ZaMiIKKLX9JqtHmmqcxsv2sLI+thiFo3HgEgrKkTvu89E4p4IiClxpp/KZBRuALoEUgWKRdBf1lz5ugSm1m5QPxmnJz0QSIgp1eKehRYM7MDTzEN0iZy79qBPI9y4VG2s8SKZQpIIOIiIK0R5qT8KUSk+oJRvvgVQm4b+yjGtmRmd2B8atn1ZqZGIiDFS5a1yTGUHkbqS+Lg4xEndE/Gt0z7RV7+r4vVcXks7SogAK2uon2zxvPQTlzwogKfA+M5jwhTle/v7xcfxkr4F0EqIAabiFf+q4GE+2h/Y0YYwDXaxDncGus7VZig8AAAAAABKiAGxcHOY40lZ9JkaLBeuVHRoo3MbhI0grXGdRSXcOYr8iogBt324ddloZPZy+FGzut5rBy0he1fWzeROoz1hX7/AKkqIAbd9uHudY/eGEJdvORszdq2GvxNg7kNJ/69+SjYoYv8KiAK8cNDIYjKOmNRNaE6GJUazr0p5qwtrqYL8lyHXEAL8yogDDX/qQVajlaNqPe8B1YVJ0zxySykH0AAnFFqpBTCfHAqIDqGXmnuD1SAyrz2Y1fk3C8Y1Y1Fwep0ifs3I9l5PHKmKiBvmrSk8ZWNwKnJTD+3LAeZWEPtpIXjok8QxpOZ+BmUDyogrPE26wH8HE6IPSPItYRKtZo39mrdV8XprDtT4FnTXGQqIPoJEaVIY0EtYx9OB4cDKWwDXw0TM6DZyIONc7cQ/m4tMKAGOOAGENPAudYBKgwIxMjP1QYQq+zO2AM="
|
||||
}
|
||||
@@ -61,30 +61,26 @@ client-side `MIN_CURVE_LEN` floor discards anything shorter than the
|
||||
smallest real struct, so a stray short/legacy account still can't reach the
|
||||
decoder.
|
||||
|
||||
**Bandwidth trade-off, measured 2026-09-15 directly over Geyser** (not
|
||||
borrowed from the WebSocket script's measurement — Geyser reports slot and
|
||||
signature per update and its named-group model batches differently, so it
|
||||
gets its own number). Two *separate* concurrent gRPC streams over the same
|
||||
120s window, so both see the identical trade activity without the
|
||||
sequential-window volume-swing problem (a first attempt comparing sequential
|
||||
windows was unreliable for exactly that reason — see
|
||||
`pumpfun_watch_graduating_programsubscribe.py`'s module docstring): one stream subscribed with
|
||||
this script's old two named groups (`datasize` 125 and 151), the other with
|
||||
a single unfiltered group. The filtered stream took in 6,754 updates /
|
||||
2,340,863 proto bytes; the unfiltered stream took in 6,761 updates /
|
||||
2,337,304 proto bytes — a 1.001x update ratio and a 0.998x byte ratio,
|
||||
i.e. no measurable cost. The 7-update difference was the 256-byte curve
|
||||
(`EJpNsfxnTB6mtVdzrTcgQ9xfywobHSSsUtu1Gh1GFvEg`) that no enumerated length
|
||||
could ever match; the filtered stream structurally cannot see it at all.
|
||||
`getAccountInfo` on that same curve, run directly the same day, confirms it:
|
||||
still 256 bytes, discriminator intact, and it decodes cleanly through this
|
||||
repo's own IDL-driven decoder — the bytes past the documented fields are
|
||||
zero padding.
|
||||
**Bandwidth trade-off, checked directly over Geyser** (not borrowed from the
|
||||
WebSocket script's comparison — Geyser reports slot and signature per update
|
||||
and its named-group model batches differently, so it gets its own). Two
|
||||
*separate* concurrent gRPC streams over the same window, so both see the
|
||||
identical trade activity without the sequential-window volume-swing problem
|
||||
(a first attempt comparing sequential windows was unreliable for exactly that
|
||||
reason — see `pumpfun_watch_graduating_programsubscribe.py`'s module
|
||||
docstring): one stream subscribed with this script's old two named groups
|
||||
(`datasize` 125 and 151), the other with a single unfiltered group. They took
|
||||
in effectively the same volume, i.e. filtering bought nothing. The difference
|
||||
was the 256-byte curve (`EJpNsfxnTB6mtVdzrTcgQ9xfywobHSSsUtu1Gh1GFvEg`) that
|
||||
no enumerated length could ever match; the filtered stream structurally
|
||||
cannot see it at all. `getAccountInfo` on that same curve confirms it: still
|
||||
256 bytes, discriminator intact, and it decodes cleanly through this repo's
|
||||
own IDL-driven decoder — the bytes past the documented fields are zero
|
||||
padding.
|
||||
|
||||
UNVERIFIED here: whether the original 49-byte layout (no `creator` field) is
|
||||
still written anywhere. None of the 6,761 unfiltered updates in the
|
||||
measurement above were that length, but that is one 120s window, not proof
|
||||
of absence.
|
||||
still written anywhere. Nothing in the unfiltered stream was that length, but
|
||||
that is one window, not proof of absence.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
|
||||
@@ -69,32 +69,26 @@ elsewhere) returned 256 bytes, discriminator matching, owned by the pump
|
||||
program, and it decoded cleanly through this repo's own IDL-driven decoder
|
||||
(`PumpFunCurveManager._decode_curve_state_with_idl`) with sane reserves —
|
||||
everything past the documented fields is zero padding. Second, over the
|
||||
wire: a 120s `programSubscribe` window with this script's own filters
|
||||
(discriminator + `complete = false`, no `dataSize`) took in 1,066 updates —
|
||||
207 at 125 bytes, 853 at 151, and 6 at that same 256 length, among ordinary
|
||||
traffic. None of the 1,066 were the legacy 49-byte layout (no `creator`
|
||||
field). UNVERIFIED: whether that layout still has any `complete = false`
|
||||
accounts left on chain, and whether anything still writes to them — not
|
||||
re-measured here.
|
||||
wire: a `programSubscribe` window with this script's own filters
|
||||
(discriminator + `complete = false`, no `dataSize`) saw all three lengths —
|
||||
125, 151 and that same 256 — among ordinary traffic, the 256 being rare.
|
||||
Nothing in it was the legacy 49-byte layout (no `creator` field).
|
||||
UNVERIFIED: whether that layout still has any `complete = false` accounts
|
||||
left on chain, and whether anything still writes to them — not re-measured
|
||||
here.
|
||||
|
||||
**Bandwidth trade-off, measured 2026-09-15.** A first attempt ran the
|
||||
`dataSize`-filtered shape and the unfiltered shape back to back, 90s each,
|
||||
and looked like unfiltered cost *less* (0.68x) — that was noise: pump.fun
|
||||
trading volume swings a lot minute to minute, and two sequential windows just
|
||||
land on different volume. Rerun with all three filter groups (`dataSize
|
||||
125`, `dataSize 151`, and no `dataSize`) subscribed **simultaneously on one
|
||||
connection**, so all three watch the identical trade stream over the same
|
||||
120s: the two enumerated lengths together took in 1,060 updates / 589,279
|
||||
bytes; the unfiltered subscription took in 1,066 updates / 593,509 bytes —
|
||||
6 extra updates, 4,230 extra bytes, all of it the 256-byte curve neither
|
||||
enumerated length can match. That is a **1.006x update ratio / 1.007x byte
|
||||
ratio** — under 1% either way, not the double subscription's worth intuition
|
||||
might suggest, because in this trade window virtually every update already
|
||||
lands on one of the two dataSize-filtered lengths (125 or 151), and 256 is
|
||||
rare. Dropping the filter is effectively free here; if a resize-happy period
|
||||
ever shifts that mix, the cost scales with however much traffic sits outside
|
||||
the two filtered lengths (125/151, i.e. the rarer 256-byte curves and
|
||||
beyond), not with total volume.
|
||||
**Bandwidth trade-off.** A first attempt ran the `dataSize`-filtered shape
|
||||
and the unfiltered shape back to back and made unfiltered look *cheaper* —
|
||||
that was noise: pump.fun trading volume swings a lot minute to minute, and
|
||||
two sequential windows just land on different volume. Rerun with all three
|
||||
filter groups (`dataSize 125`, `dataSize 151`, and no `dataSize`) subscribed
|
||||
**simultaneously on one connection**, so all three watch the identical trade
|
||||
stream: the unfiltered subscription cost barely more than the two enumerated
|
||||
lengths together, and the difference was entirely the 256-byte curve neither
|
||||
enumerated length can match. Dropping the filter is effectively free here,
|
||||
because virtually every update already lands on 125 or 151 and 256 is rare;
|
||||
if a resize-happy period ever shifts that mix, the cost scales with however
|
||||
much traffic sits outside the two filtered lengths, not with total volume.
|
||||
|
||||
UNVERIFIED: a curve was once observed going from 125 to 151 bytes, with
|
||||
several 125-byte trades logged in between, suggesting `extend_account` ran as
|
||||
|
||||
@@ -9,22 +9,19 @@ executed stream. The two are different RPCs on the same endpoint and the trade-o
|
||||
between them is the point of this example.
|
||||
|
||||
`SubscribeDeshred` delivers a transaction when entries are formed from shreds,
|
||||
BEFORE any execution. Measured head-to-head against `Subscribe` on the pump.fun
|
||||
program, 300s on 2026-09-23: identical signature sets, no orphans either way.
|
||||
On token creations -- the number that matters here -- deshred arrived first in
|
||||
92% of cases, median +6.4ms, p75 +10.1ms, p90 +21.5ms. General traffic does
|
||||
worse (79% first, median +4.0ms), and the far tail is volatile between runs.
|
||||
`tools/compare_deshred_latency.py` reproduces all of this.
|
||||
BEFORE any execution. Raced head-to-head against `Subscribe` on the pump.fun
|
||||
program it carries the same signatures with no orphans either way, and on token
|
||||
creations -- the case that matters here -- it usually arrives first, by a small
|
||||
margin. `tools/compare_deshred_latency.py` reproduces that comparison.
|
||||
|
||||
What you give up for those milliseconds:
|
||||
|
||||
- **It cannot see a coin created through a router, at all.** The create then
|
||||
reaches the program as a CPI, and inner instructions are produced *by*
|
||||
execution, so a pre-execution stream never carries them: the transaction
|
||||
arrives on deshred with the create invisible. Measured 4 of 118 creates
|
||||
(3.4%) in one 300s window and 3 of 101 in another -- every one of them
|
||||
present on the stream and undetectable, not dropped. No decoding recovers
|
||||
them, and this script misses them too. The executed listeners walk
|
||||
arrives on deshred with the create invisible -- present on the stream and
|
||||
undetectable, not dropped. No decoding recovers them, and this script
|
||||
misses them too. The executed listeners walk
|
||||
`meta.innerInstructions` and do not have this blind spot.
|
||||
- No TransactionStatusMeta, so no meta.log_messages, so no CreateEvent. This
|
||||
script therefore decodes the create instruction, which is the fallback route
|
||||
|
||||
@@ -8,14 +8,11 @@ Usage:
|
||||
The mirror of `pumpfun_buy_token_v2.py`. It reads how many tokens you hold, reads the curve
|
||||
for a price, and sells the lot with a slippage floor underneath.
|
||||
|
||||
Two things to know before you run it:
|
||||
One thing to know before you run it:
|
||||
|
||||
- **The floor is yours to set.** `min_sol_output` is computed here from the price
|
||||
and the slippage, and the program enforces only that number. Passing a floor
|
||||
derived from a stale price is how a sell goes through at a price you did not intend.
|
||||
- **A dust position may not be sellable at all.** Below a few thousand tokens the
|
||||
output rounds to zero and the program rejects the sell with error 6003
|
||||
(TooLittleSolReceived), whatever slippage you allow.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
|
||||
@@ -80,7 +80,7 @@ CONFIG_VALIDATION_RULES = [
|
||||
|
||||
# Valid values for enum-like fields
|
||||
VALID_VALUES = {
|
||||
"filters.listener_type": ["logs", "blocks", "geyser", "pumpportal"],
|
||||
"filters.listener_type": ["logs", "blocks", "geyser", "shreds", "pumpportal"],
|
||||
"cleanup.mode": ["disabled", "on_fail", "after_sell", "post_session"],
|
||||
"trade.exit_strategy": ["time_based", "tp_sl", "manual"],
|
||||
"platform": ["pump_fun", "lets_bonk"],
|
||||
@@ -88,7 +88,7 @@ VALID_VALUES = {
|
||||
|
||||
# Platform-specific listener compatibility
|
||||
PLATFORM_LISTENER_COMPATIBILITY = {
|
||||
Platform.PUMP_FUN: ["logs", "blocks", "geyser", "pumpportal"],
|
||||
Platform.PUMP_FUN: ["logs", "blocks", "geyser", "shreds", "pumpportal"],
|
||||
Platform.LETS_BONK: ["blocks", "geyser", "pumpportal"],
|
||||
}
|
||||
|
||||
|
||||
+20
-7
@@ -93,13 +93,9 @@ class TokenInfo:
|
||||
# buy_v2/sell_v2 account lists are unchanged — so this is informational and
|
||||
# available for filtering. creator_fee_bps is non-zero only on custom-pair
|
||||
# coins; SOL- and USDC-paired coins use the standard fee schedule (bps 0).
|
||||
# Verified live on 2026-09-15: 21 SOL-paired CreateEvents (bps 0 on all;
|
||||
# the 90-second capture window saw 25 coins total, 4 of them custom-pair)
|
||||
# and 10 USDC-paired bonding curves — located via Dune's indexed
|
||||
# pumpdotfun_solana.pump_evt_createevent table, then read directly by
|
||||
# deriving each curve PDA and calling PumpFunCurveManager.get_pool_state —
|
||||
# also bps 0 on all. Every custom-pair coin observed in the same window
|
||||
# showed a nonzero value (30, 125, 1, 300 bps).
|
||||
# Checked live against SOL-paired CreateEvents and USDC-paired bonding
|
||||
# curves, which carry bps 0; every custom-pair coin checked alongside them
|
||||
# carried a nonzero value.
|
||||
is_holder_reward: bool = False
|
||||
creator_fee_bps: int = 0
|
||||
|
||||
@@ -453,6 +449,23 @@ class EventParser(ABC):
|
||||
"""
|
||||
pass
|
||||
|
||||
def get_creation_filter_accounts(self) -> list[Pubkey]:
|
||||
"""Get accounts to subscribe on when only creations are wanted.
|
||||
|
||||
A subscription filtered on the program id sees every transaction the
|
||||
program handles, which is overwhelmingly trades. Where a platform has
|
||||
an account that only its creation instructions touch, naming it here
|
||||
moves that filtering server-side. The pre-execution shreds listener
|
||||
needs this: the unfiltered firehose makes a Python consumer lag, and a
|
||||
lagging deshred stream is terminated by the server rather than slowed.
|
||||
|
||||
Defaults to the program id, which is correct but unselective.
|
||||
|
||||
Returns:
|
||||
Accounts a creation transaction is guaranteed to mention
|
||||
"""
|
||||
return [self.get_program_id()]
|
||||
|
||||
@abstractmethod
|
||||
def get_instruction_discriminators(self) -> list[bytes]:
|
||||
"""Get instruction discriminators for token creation.
|
||||
|
||||
@@ -25,10 +25,11 @@ class ListenerFactory:
|
||||
"""Create a token listener based on the specified type.
|
||||
|
||||
Args:
|
||||
listener_type: Type of listener ('logs', 'blocks', 'geyser', or 'pumpportal')
|
||||
listener_type: Type of listener ('logs', 'blocks', 'geyser',
|
||||
'shreds' or 'pumpportal')
|
||||
wss_endpoint: WebSocket endpoint URL (for logs/blocks listeners)
|
||||
geyser_endpoint: Geyser gRPC endpoint URL (for geyser listener)
|
||||
geyser_api_token: Geyser API token (for geyser listener)
|
||||
geyser_endpoint: Geyser gRPC endpoint URL (geyser/shreds listeners)
|
||||
geyser_api_token: Geyser API token (geyser/shreds listeners)
|
||||
geyser_auth_type: Geyser authentication type
|
||||
pumpportal_url: PumpPortal WebSocket URL (for pumpportal listener)
|
||||
platforms: List of platforms to monitor (if None, monitor all)
|
||||
@@ -58,6 +59,23 @@ class ListenerFactory:
|
||||
logger.info("Created Universal Geyser listener for token monitoring")
|
||||
return listener
|
||||
|
||||
elif listener_type == "shreds":
|
||||
if not geyser_endpoint or not geyser_api_token:
|
||||
raise ValueError(
|
||||
"Geyser endpoint and API token are required for shreds listener"
|
||||
)
|
||||
|
||||
from monitoring.universal_shreds_listener import UniversalShredsListener
|
||||
|
||||
listener = UniversalShredsListener(
|
||||
geyser_endpoint=geyser_endpoint,
|
||||
geyser_api_token=geyser_api_token,
|
||||
geyser_auth_type=geyser_auth_type,
|
||||
platforms=platforms,
|
||||
)
|
||||
logger.info("Created Universal Shreds listener for token monitoring")
|
||||
return listener
|
||||
|
||||
elif listener_type == "logs":
|
||||
if not wss_endpoint:
|
||||
raise ValueError("WebSocket endpoint is required for logs listener")
|
||||
@@ -124,7 +142,8 @@ class ListenerFactory:
|
||||
else:
|
||||
raise ValueError(
|
||||
f"Invalid listener type '{listener_type}'. "
|
||||
f"Must be one of: 'logs', 'blocks', 'geyser', 'pumpportal'"
|
||||
f"Must be one of: 'logs', 'blocks', 'geyser', 'shreds', "
|
||||
f"'pumpportal'"
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
@@ -134,7 +153,7 @@ class ListenerFactory:
|
||||
Returns:
|
||||
List of supported listener type strings
|
||||
"""
|
||||
return ["logs", "blocks", "geyser", "pumpportal"]
|
||||
return ["logs", "blocks", "geyser", "shreds", "pumpportal"]
|
||||
|
||||
@staticmethod
|
||||
def get_platform_compatible_listeners(platform: Platform) -> list[str]:
|
||||
@@ -147,7 +166,7 @@ class ListenerFactory:
|
||||
List of compatible listener types
|
||||
"""
|
||||
if platform == Platform.PUMP_FUN:
|
||||
return ["logs", "blocks", "geyser", "pumpportal"]
|
||||
return ["logs", "blocks", "geyser", "shreds", "pumpportal"]
|
||||
elif platform == Platform.LETS_BONK:
|
||||
return ["blocks", "geyser", "pumpportal"] # Added pumpportal support
|
||||
else:
|
||||
|
||||
@@ -183,9 +183,8 @@ class UniversalBlockListener(BaseTokenListener):
|
||||
# live since epoch 1035, 2026-09-15. A version the
|
||||
# subscription does not accept does not just skip
|
||||
# that transaction: the RPC nulls out `value.block`
|
||||
# for the whole frame. Measured against mainnet on
|
||||
# 2026-09-16, 60s each: `0` delivered 1 block and
|
||||
# 177 nulls, `1` delivered 78 blocks and no nulls.
|
||||
# for the whole frame, leaving this listener
|
||||
# almost entirely blind.
|
||||
"maxSupportedTransactionVersion": 1,
|
||||
},
|
||||
],
|
||||
|
||||
@@ -0,0 +1,360 @@
|
||||
"""Universal shreds listener: pre-execution token detection over SubscribeDeshred.
|
||||
|
||||
`SubscribeDeshred` delivers a transaction when entries are formed from shreds,
|
||||
before any execution. That is earlier than `Subscribe`, and it costs two things
|
||||
that shape everything in this module.
|
||||
|
||||
**There is no TransactionStatusMeta.** No `meta.log_messages`, so no CreateEvent,
|
||||
so the log route every other listener prefers does not exist here. This listener
|
||||
decodes the create instruction, which is the fallback route elsewhere and the
|
||||
only route here. It also means no success or failure: nothing has executed, so a
|
||||
create that goes on to revert is delivered exactly like one that lands.
|
||||
|
||||
**A coin created through a router is invisible.** The create then reaches the
|
||||
program as a CPI, and inner instructions are produced *by* execution, so a
|
||||
pre-execution stream never carries them -- the transaction arrives with the
|
||||
create undetectable, not dropped. No decoding recovers them. This is the
|
||||
standing cost of the mode and the reason it is not the default; see
|
||||
bots/bot-sniper-5-shreds.yaml.
|
||||
|
||||
Note this inverts the usual CPI advice in AGENTS.md: *trades* are overwhelmingly
|
||||
inner instructions, *creates* are overwhelmingly top-level, which is why walking
|
||||
only top-level instructions here is right rather than a shortcut.
|
||||
|
||||
Address lookup tables are already resolved and reported on the update itself, so
|
||||
instruction account indices resolve as they do on the executed stream.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from collections.abc import Awaitable, Callable
|
||||
|
||||
import grpc
|
||||
from solders.pubkey import Pubkey
|
||||
|
||||
from geyser.generated import geyser_pb2, geyser_pb2_grpc
|
||||
from interfaces.core import EventParser, Platform, TokenInfo
|
||||
from monitoring.base_listener import BaseTokenListener
|
||||
from platforms import platform_factory
|
||||
from utils.logger import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
# Seconds to wait before retrying a dropped stream, matching the geyser listener.
|
||||
_STREAM_RETRY_SECONDS = 5
|
||||
_CONNECTION_RETRY_SECONDS = 10
|
||||
|
||||
|
||||
class UniversalShredsListener(BaseTokenListener):
|
||||
"""Pre-execution token listener over the geyser deshred stream."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
geyser_endpoint: str,
|
||||
geyser_api_token: str,
|
||||
geyser_auth_type: str,
|
||||
platforms: list[Platform] | None = None,
|
||||
):
|
||||
"""Initialize the shreds listener.
|
||||
|
||||
Args:
|
||||
geyser_endpoint: Geyser gRPC endpoint URL
|
||||
geyser_api_token: Geyser API token
|
||||
geyser_auth_type: Either "x-token" or "basic"
|
||||
platforms: Platforms to monitor; all supported platforms if None
|
||||
|
||||
Raises:
|
||||
ValueError: If geyser_auth_type is not a supported value
|
||||
"""
|
||||
super().__init__()
|
||||
self.geyser_endpoint = geyser_endpoint
|
||||
self.geyser_api_token = geyser_api_token
|
||||
|
||||
valid_auth_types = {"x-token", "basic"}
|
||||
self.auth_type: str = (geyser_auth_type or "x-token").lower()
|
||||
if self.auth_type not in valid_auth_types:
|
||||
raise ValueError(
|
||||
f"Unsupported auth_type={self.auth_type!r}. "
|
||||
f"Expected one of {valid_auth_types}"
|
||||
)
|
||||
|
||||
if platforms is None:
|
||||
self.platforms = platform_factory.get_supported_platforms()
|
||||
else:
|
||||
self.platforms = platforms
|
||||
|
||||
self.platform_parsers: dict[Platform, EventParser] = {}
|
||||
self.platform_program_ids: dict[bytes, EventParser] = {}
|
||||
self.create_discriminators: dict[bytes, tuple[bytes, ...]] = {}
|
||||
# Flattened across platforms for the str.startswith prefix test.
|
||||
self.all_discriminators: tuple[bytes, ...] = ()
|
||||
|
||||
for platform in self.platforms:
|
||||
try:
|
||||
from core.client import SolanaClient
|
||||
|
||||
class DummyClient(SolanaClient):
|
||||
"""Client stub that skips the blockhash updater.
|
||||
|
||||
Parsers only decode here; nothing in this listener reaches
|
||||
the network through the client.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self.rpc_endpoint = "http://dummy"
|
||||
self._client = None
|
||||
self._cached_blockhash = None
|
||||
self._blockhash_lock = None
|
||||
self._blockhash_updater_task = None
|
||||
|
||||
implementations = platform_factory.create_for_platform(
|
||||
platform, DummyClient()
|
||||
)
|
||||
parser = implementations.event_parser
|
||||
self.platform_parsers[platform] = parser
|
||||
self.platform_program_ids[bytes(parser.get_program_id())] = parser
|
||||
# Cheap prefix gate, see _process_update: the deshred stream
|
||||
# carries every pump transaction pre-execution, so most
|
||||
# instructions reaching us are trades, not creates.
|
||||
self.create_discriminators[bytes(parser.get_program_id())] = tuple(
|
||||
parser.get_instruction_discriminators()
|
||||
)
|
||||
self.all_discriminators += tuple(
|
||||
parser.get_instruction_discriminators()
|
||||
)
|
||||
|
||||
logger.info(
|
||||
f"Registered platform {platform.value} with program ID "
|
||||
f"{parser.get_program_id()}"
|
||||
)
|
||||
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning(f"Could not register platform {platform.value}: {e}")
|
||||
|
||||
async def _create_geyser_connection(self):
|
||||
"""Establish a secure connection to the Geyser endpoint.
|
||||
|
||||
Returns:
|
||||
A (stub, channel) pair; the caller owns closing the channel
|
||||
"""
|
||||
if self.auth_type == "x-token":
|
||||
auth = grpc.metadata_call_credentials(
|
||||
lambda _, callback: callback(
|
||||
(("x-token", self.geyser_api_token),), None
|
||||
)
|
||||
)
|
||||
else:
|
||||
auth = grpc.metadata_call_credentials(
|
||||
lambda _, callback: callback(
|
||||
(("authorization", f"Basic {self.geyser_api_token}"),), None
|
||||
)
|
||||
)
|
||||
creds = grpc.composite_channel_credentials(grpc.ssl_channel_credentials(), auth)
|
||||
channel = grpc.aio.secure_channel(self.geyser_endpoint, creds)
|
||||
|
||||
return geyser_pb2_grpc.GeyserStub(channel), channel
|
||||
|
||||
def _create_subscription_request(self) -> geyser_pb2.SubscribeDeshredRequest:
|
||||
"""Build the deshred subscription request for all monitored platforms.
|
||||
|
||||
This is its own request type, not a SubscribeRequest: the deshred stream
|
||||
carries no commitment level, because commitment describes execution and
|
||||
nothing has executed. There is no `failed` filter for the same reason.
|
||||
|
||||
Returns:
|
||||
The request to open the stream with
|
||||
"""
|
||||
request = geyser_pb2.SubscribeDeshredRequest()
|
||||
|
||||
for program_id, parser in self.platform_program_ids.items():
|
||||
filter_name = f"platform_filter_{Pubkey.from_bytes(program_id)}"
|
||||
deshred_filter = request.deshred_transactions[filter_name]
|
||||
# Not the program id: that matches every trade too, and the volume
|
||||
# makes a Python consumer lag until the server ends the stream.
|
||||
for account in parser.get_creation_filter_accounts():
|
||||
deshred_filter.account_include.append(str(account))
|
||||
deshred_filter.vote = False
|
||||
|
||||
return request
|
||||
|
||||
async def listen_for_tokens(
|
||||
self,
|
||||
token_callback: Callable[[TokenInfo], Awaitable[None]],
|
||||
match_string: str | None = None,
|
||||
creator_address: str | None = None,
|
||||
) -> None:
|
||||
"""Listen for new token creations on the pre-execution deshred stream.
|
||||
|
||||
Args:
|
||||
token_callback: Called with each detected token
|
||||
match_string: Optional substring to require in the name or symbol
|
||||
creator_address: Optional creator address to filter by
|
||||
"""
|
||||
if not self.platform_parsers:
|
||||
logger.error("No platform parsers available. Cannot listen for tokens.")
|
||||
return
|
||||
|
||||
while True:
|
||||
try:
|
||||
stub, channel = await self._create_geyser_connection()
|
||||
request = self._create_subscription_request()
|
||||
|
||||
logger.info(f"Connected to deshred stream: {self.geyser_endpoint}")
|
||||
logger.info(
|
||||
f"Monitoring platforms: {[p.value for p in self.platforms]}"
|
||||
)
|
||||
logger.warning(
|
||||
"Shreds mode reads transactions before execution: coins "
|
||||
"created through a router arrive as CPIs and cannot be "
|
||||
"detected, and a create that later reverts is "
|
||||
"indistinguishable from one that lands."
|
||||
)
|
||||
|
||||
try:
|
||||
async for update in stub.SubscribeDeshred(iter([request])):
|
||||
token_info = self._process_update(update)
|
||||
if not token_info:
|
||||
continue
|
||||
|
||||
logger.info(
|
||||
f"New token detected pre-execution: {token_info.name} "
|
||||
f"({token_info.symbol}) on {token_info.platform.value}"
|
||||
)
|
||||
|
||||
if match_string and not (
|
||||
match_string.lower() in token_info.name.lower()
|
||||
or match_string.lower() in token_info.symbol.lower()
|
||||
):
|
||||
logger.info(
|
||||
f"Token does not match filter '{match_string}'. "
|
||||
"Skipping..."
|
||||
)
|
||||
continue
|
||||
|
||||
if creator_address and str(token_info.user) != creator_address:
|
||||
logger.info(
|
||||
f"Token not created by {creator_address}. Skipping..."
|
||||
)
|
||||
continue
|
||||
|
||||
await token_callback(token_info)
|
||||
|
||||
except Exception as e:
|
||||
if isinstance(e, grpc.aio.AioRpcError):
|
||||
logger.exception(f"gRPC error: {e.details()}")
|
||||
else:
|
||||
logger.exception("Deshred stream error occurred")
|
||||
await asyncio.sleep(_STREAM_RETRY_SECONDS)
|
||||
|
||||
finally:
|
||||
await channel.close()
|
||||
|
||||
except Exception:
|
||||
logger.exception("Deshred connection error")
|
||||
logger.info(f"Reconnecting in {_CONNECTION_RETRY_SECONDS} seconds...")
|
||||
await asyncio.sleep(_CONNECTION_RETRY_SECONDS)
|
||||
|
||||
@staticmethod
|
||||
def _resolve_account_keys(transaction) -> list[bytes]:
|
||||
"""Build the full account-key table for one deshred transaction.
|
||||
|
||||
A v0 transaction indexes accounts past the end of `message.account_keys`
|
||||
when it uses an address lookup table. The deshred stream resolves those
|
||||
and reports them on the update itself rather than under a meta, in this
|
||||
order: static keys, writable loaded, read-only loaded. Ignoring them
|
||||
raises IndexError on the first coin that uses a lookup table.
|
||||
|
||||
Args:
|
||||
transaction: A geyser SubscribeUpdateDeshredTransactionInfo
|
||||
|
||||
Returns:
|
||||
Account keys as raw 32-byte values, indexable by an instruction
|
||||
"""
|
||||
keys = list(transaction.transaction.message.account_keys)
|
||||
keys.extend(transaction.loaded_writable_addresses)
|
||||
keys.extend(transaction.loaded_readonly_addresses)
|
||||
return keys
|
||||
|
||||
def _process_update(self, update) -> TokenInfo | None:
|
||||
"""Extract token creation info from one deshred update.
|
||||
|
||||
Only top-level instructions are walked, because inner instructions do
|
||||
not exist before execution. Each instruction is routed to its own
|
||||
platform's parser by program id rather than offered to every parser,
|
||||
so an unrelated program's instruction is never decoded speculatively.
|
||||
|
||||
Args:
|
||||
update: A geyser SubscribeUpdate from the deshred stream
|
||||
|
||||
Returns:
|
||||
TokenInfo for a detected creation, None otherwise
|
||||
"""
|
||||
try:
|
||||
if not update.HasField("deshred_transaction"):
|
||||
return None
|
||||
|
||||
transaction = update.deshred_transaction.transaction
|
||||
message = getattr(transaction.transaction, "message", None)
|
||||
if message is None:
|
||||
return None
|
||||
|
||||
static_keys = transaction.transaction.message.account_keys
|
||||
keys = None
|
||||
|
||||
for instruction in message.instructions:
|
||||
# Cheapest test first. The account filter already narrows the
|
||||
# stream to creations, but a creation transaction still carries
|
||||
# ComputeBudget, system and token instructions, and this avoids
|
||||
# both the key-table copy and a full IDL decode for each.
|
||||
if not bytes(instruction.data).startswith(self.all_discriminators):
|
||||
continue
|
||||
|
||||
program_index = instruction.program_id_index
|
||||
if program_index >= len(static_keys):
|
||||
continue
|
||||
|
||||
parser = self.platform_program_ids.get(
|
||||
bytes(static_keys[program_index])
|
||||
)
|
||||
if parser is None:
|
||||
continue
|
||||
|
||||
# Resolve lookup-table accounts only once a real candidate is
|
||||
# in hand; the instruction's own accounts may index into them.
|
||||
if keys is None:
|
||||
keys = self._resolve_account_keys(transaction)
|
||||
|
||||
token_info = parser.parse_token_creation_from_instruction(
|
||||
instruction.data, instruction.accounts, keys
|
||||
)
|
||||
if token_info:
|
||||
return self._mark_trusted(token_info)
|
||||
|
||||
return None
|
||||
|
||||
except Exception:
|
||||
logger.exception("Error processing deshred update")
|
||||
return None
|
||||
|
||||
@staticmethod
|
||||
def _mark_trusted(token_info: TokenInfo) -> TokenInfo:
|
||||
"""Mark an instruction-parsed TokenInfo as safe to trade without a refresh.
|
||||
|
||||
Every other listener leaves `state_from_event` False on instruction-parsed
|
||||
tokens, because `args.creator` is user-supplied and the program may write
|
||||
something else into `BondingCurve.creator`. That reservation has exactly
|
||||
one cause, and the parser now resolves it: a holder-reward coin takes its
|
||||
creator from a PDA of the mint, which is derivable without reading
|
||||
anything.
|
||||
|
||||
Waiting for the curve here would not be a safer fallback but a broken one:
|
||||
the account does not exist yet, so the only alternatives are trading on
|
||||
this data or not trading at all.
|
||||
|
||||
Args:
|
||||
token_info: Token parsed from a create instruction
|
||||
|
||||
Returns:
|
||||
The same TokenInfo, marked so extreme_fast_mode skips the curve read
|
||||
"""
|
||||
token_info.state_from_event = True
|
||||
return token_info
|
||||
@@ -30,6 +30,12 @@ class PumpFunAddresses:
|
||||
GLOBAL: Final[Pubkey] = Pubkey.from_string(
|
||||
"4wTV1YmiEkRvAtNtsSGPtUrqRYQMe5SKy2uB4Jjaxnjf"
|
||||
)
|
||||
# PDA ["mint-authority"], constant. Account 1 of create/create_v2 and
|
||||
# absent from every trade instruction, which makes it the selective filter
|
||||
# for a creations-only subscription.
|
||||
MINT_AUTHORITY: Final[Pubkey] = Pubkey.from_string(
|
||||
"TSLvdd1pWpHVjahSpsvCXUbgwsL3JAcvokwaKt1eokM"
|
||||
)
|
||||
EVENT_AUTHORITY: Final[Pubkey] = Pubkey.from_string(
|
||||
"Ce6TQqeHC9p8KetsN6JsjHK7UTZk7nasjjnr7XxXp9F1"
|
||||
)
|
||||
@@ -121,6 +127,35 @@ class PumpFunAddresses:
|
||||
)
|
||||
return derived_address
|
||||
|
||||
@staticmethod
|
||||
def find_holder_reward_creator(mint: Pubkey) -> Pubkey:
|
||||
"""Derive the creator a holder-reward coin's bonding curve carries.
|
||||
|
||||
On a holder-reward coin the creator fee is shared out to holders rather
|
||||
than paid to the launcher, and pump.fun implements that by reusing the
|
||||
ordinary fee route: the program ignores the `creator` passed to
|
||||
`create_v2` and writes this PDA into `BondingCurve.creator`, so
|
||||
`creator-vault` derives onto the holder-reward distribution account on
|
||||
its own. Seeds are the ones the IDL declares for the `holder_rewards`
|
||||
account on `distribute_fee_to_holders`.
|
||||
|
||||
This is what lets a listener holding only the create instruction — no
|
||||
CreateEvent, and no curve to read because nothing has executed — still
|
||||
build a correct buy: the substituted creator depends on nothing but the
|
||||
mint.
|
||||
|
||||
Args:
|
||||
mint: Coin mint address
|
||||
|
||||
Returns:
|
||||
The address BondingCurve.creator holds for a holder-reward coin
|
||||
"""
|
||||
derived_address, _ = Pubkey.find_program_address(
|
||||
[b"holder-rewards", bytes(mint)],
|
||||
PumpFunAddresses.PROGRAM,
|
||||
)
|
||||
return derived_address
|
||||
|
||||
@staticmethod
|
||||
def find_global_volume_accumulator() -> Pubkey:
|
||||
"""
|
||||
@@ -333,6 +368,20 @@ class PumpFunAddressProvider(AddressProvider):
|
||||
)
|
||||
return creator_vault
|
||||
|
||||
def derive_holder_reward_creator(self, mint: Pubkey) -> Pubkey:
|
||||
"""Derive the creator a holder-reward coin's bonding curve carries.
|
||||
|
||||
See PumpFunAddresses.find_holder_reward_creator for why this differs
|
||||
from the creator passed to create_v2.
|
||||
|
||||
Args:
|
||||
mint: Coin mint address
|
||||
|
||||
Returns:
|
||||
The address BondingCurve.creator holds for a holder-reward coin
|
||||
"""
|
||||
return PumpFunAddresses.find_holder_reward_creator(mint)
|
||||
|
||||
def derive_global_volume_accumulator(self) -> Pubkey:
|
||||
"""Derive the global volume accumulator PDA.
|
||||
|
||||
|
||||
@@ -31,10 +31,37 @@ logger = get_logger(__name__)
|
||||
# pump-fun public docs repository.
|
||||
_CREATE_V2_QUOTE_MINT_ACCOUNT_INDEX = 16
|
||||
|
||||
# Index of the `user` account, which differs per instruction version: create_v2
|
||||
# dropped the four metaplex accounts legacy `create` carries between the global
|
||||
# account and the user, so the same wallet sits four places earlier. Reading
|
||||
# create_v2's account 7 yields the token program instead.
|
||||
_CREATE_USER_ACCOUNT_INDEX = 7
|
||||
_CREATE_V2_USER_ACCOUNT_INDEX = 5
|
||||
|
||||
# Length of a Solana public key in bytes.
|
||||
PUBKEY_BYTE_LENGTH = 32
|
||||
|
||||
|
||||
def _option_bool(value: object) -> bool:
|
||||
"""Read an Anchor OptionBool argument.
|
||||
|
||||
OptionBool is a single-field struct with no presence tag, so the IDL parser
|
||||
decodes it as {"field_0": bool}; a trailing argument the caller omitted
|
||||
arrives as None. Omission is not ambiguity: create_v2's trailing arguments
|
||||
are positional, so a truncated instruction cannot have set a later one, and
|
||||
reporting it unset is correct rather than a guess.
|
||||
|
||||
Args:
|
||||
value: Decoded argument, a dict, a bool, or None when absent
|
||||
|
||||
Returns:
|
||||
The flag's value, False when the argument was not sent
|
||||
"""
|
||||
if isinstance(value, dict):
|
||||
return bool(value.get("field_0", False))
|
||||
return bool(value) if value is not None else False
|
||||
|
||||
|
||||
def _coerce_pubkey(value: object) -> Pubkey | None:
|
||||
"""Coerce a decoded IDL pubkey field into a Pubkey.
|
||||
|
||||
@@ -398,21 +425,38 @@ class PumpFunEventParser(EventParser):
|
||||
|
||||
args = decoded.get("args", {})
|
||||
|
||||
# Extract account information based on IDL account order
|
||||
# Extract account information based on IDL account order. `user`
|
||||
# sits at a different index per instruction version: create_v2
|
||||
# dropped the four metaplex accounts legacy `create` carries, so
|
||||
# its account 7 is the token program, not the launcher's wallet.
|
||||
mint = get_account_key(0)
|
||||
bonding_curve = get_account_key(2)
|
||||
associated_bonding_curve = get_account_key(3)
|
||||
user = get_account_key(7)
|
||||
user = get_account_key(
|
||||
_CREATE_V2_USER_ACCOUNT_INDEX
|
||||
if is_create_v2
|
||||
else _CREATE_USER_ACCOUNT_INDEX
|
||||
)
|
||||
|
||||
if not all([mint, bonding_curve, associated_bonding_curve, user]):
|
||||
return None
|
||||
|
||||
# Create creator vault
|
||||
creator = (
|
||||
Pubkey.from_string(args.get("creator", str(user)))
|
||||
if args.get("creator")
|
||||
else user
|
||||
)
|
||||
# Holder-reward flag first: it decides where the creator comes from.
|
||||
# Safe to trust from instruction args even though `creator` is not —
|
||||
# the program rejects the request when the feature is globally
|
||||
# disabled, so a landed create carrying `true` is proof. The args
|
||||
# are positional, so a truncated trailing form cannot have set this
|
||||
# and reads as False, which is correct by construction.
|
||||
is_holder_reward = _option_bool(args.get("is_holder_reward"))
|
||||
|
||||
# On a holder-reward coin the program ignores args.creator and
|
||||
# writes its own per-coin PDA into BondingCurve.creator, so
|
||||
# creator_vault must derive from that instead. args.creator is
|
||||
# otherwise copied to the curve verbatim at create time.
|
||||
if is_holder_reward:
|
||||
creator = PumpFunAddresses.find_holder_reward_creator(mint)
|
||||
else:
|
||||
creator = _coerce_pubkey(args.get("creator")) or user
|
||||
creator_vault = self._derive_creator_vault(creator)
|
||||
|
||||
# Determine token program based on instruction type
|
||||
@@ -422,35 +466,11 @@ class PumpFunEventParser(EventParser):
|
||||
else SystemAddresses.TOKEN_PROGRAM
|
||||
)
|
||||
|
||||
# Extract cashback flag from OptionBool struct (decoded as
|
||||
# {"field_0": bool}). Cashback creation was deprecated 2026-09-15
|
||||
# (create_v2 rejects a new true here with 6082 CashbackDeprecated),
|
||||
# but older create_v2 instructions still carry it and existing
|
||||
# cashback coins still trade, so it's still decoded here.
|
||||
is_cashback_raw = args.get("is_cashback_enabled")
|
||||
is_cashback = (
|
||||
is_cashback_raw.get("field_0", False)
|
||||
if isinstance(is_cashback_raw, dict)
|
||||
else bool(is_cashback_raw)
|
||||
if is_cashback_raw is not None
|
||||
else False
|
||||
)
|
||||
|
||||
# Extract the holder-reward flag from its OptionBool struct, same
|
||||
# wrapper shape as is_cashback_enabled above. Unlike `creator`,
|
||||
# this one is safe to trust from instruction args: a succeeded
|
||||
# create_v2 carrying `true` here means the coin genuinely is a
|
||||
# holder-reward coin -- the program itself rejects the request
|
||||
# when the feature is globally disabled, so a landed transaction
|
||||
# is proof, not merely an unverified claim.
|
||||
is_holder_reward_raw = args.get("is_holder_reward")
|
||||
is_holder_reward = (
|
||||
is_holder_reward_raw.get("field_0", False)
|
||||
if isinstance(is_holder_reward_raw, dict)
|
||||
else bool(is_holder_reward_raw)
|
||||
if is_holder_reward_raw is not None
|
||||
else False
|
||||
)
|
||||
# Cashback creation was deprecated 2026-09-15 (create_v2 rejects a
|
||||
# new true here with 6082 CashbackDeprecated), but older create_v2
|
||||
# instructions still carry it and existing cashback coins still
|
||||
# trade, so it's still decoded here.
|
||||
is_cashback = _option_bool(args.get("is_cashback_enabled"))
|
||||
|
||||
# `creator_fee_bps` is deliberately left at TokenInfo's default
|
||||
# (0) rather than decoded from args here. Upstream documents it
|
||||
@@ -559,13 +579,35 @@ class PumpFunEventParser(EventParser):
|
||||
"""
|
||||
return PumpFunAddresses.PROGRAM
|
||||
|
||||
def get_creation_filter_accounts(self) -> list[Pubkey]:
|
||||
"""Subscribe on the mint authority rather than the whole program.
|
||||
|
||||
`mint_authority` (PDA ["mint-authority"], a constant) is account 1 of
|
||||
both `create` and `create_v2`, and appears in no trade instruction --
|
||||
buy_v2/sell_v2 never reference it. Filtering on it therefore yields
|
||||
creations only, server-side, instead of every pump.fun transaction.
|
||||
|
||||
Returns:
|
||||
The pump.fun mint authority
|
||||
"""
|
||||
return [PumpFunAddresses.MINT_AUTHORITY]
|
||||
|
||||
def get_instruction_discriminators(self) -> list[bytes]:
|
||||
"""Get instruction discriminators for token creation.
|
||||
|
||||
Both creation instructions, newest first: `create_v2` is what pump.fun
|
||||
issues now, and legacy `create` still appears. A caller uses these to
|
||||
reject an instruction before paying for a full IDL decode, so leaving
|
||||
`create_v2` out would make the gate reject every current coin.
|
||||
|
||||
Returns:
|
||||
List of discriminator bytes to match
|
||||
"""
|
||||
return [self._create_instruction_discriminator_bytes]
|
||||
discriminators = []
|
||||
if self._create_v2_instruction_discriminator_bytes:
|
||||
discriminators.append(self._create_v2_instruction_discriminator_bytes)
|
||||
discriminators.append(self._create_instruction_discriminator_bytes)
|
||||
return discriminators
|
||||
|
||||
def get_event_discriminators(self) -> list[bytes]:
|
||||
"""Get event discriminators for token creation.
|
||||
|
||||
@@ -70,9 +70,8 @@ async def _read_pool_state_with_retry(
|
||||
can momentarily serve a slot that predates it — both surface as "account
|
||||
not found". Reading at `processed` and retrying costs a handful of RPC
|
||||
calls, which is far cheaper than trading on stale account data. Issue #170
|
||||
measured individual reads on a load-balanced endpoint lagging several
|
||||
seconds behind a fast listener, hence a time budget rather than a fixed
|
||||
attempt count.
|
||||
found individual reads on a load-balanced endpoint lagging well behind a
|
||||
fast listener, hence a time budget rather than a fixed attempt count.
|
||||
|
||||
When `mint` is given and the curve manager supports it, the curve and the
|
||||
mint are read in one slot-consistent batch so the mint's owning token
|
||||
|
||||
@@ -16,8 +16,8 @@ connection whose frame state was now corrupt, so every later read tripped
|
||||
identical ERROR lines, and `_wait_for_token` never returning, so the bot hung
|
||||
after detecting a token instead of buying it.
|
||||
|
||||
Observed live: a blocks-listener run produced 86 consecutive AssertionErrors
|
||||
starting on the line after "Found token", and never exited.
|
||||
Observed live: a blocks-listener run produced a continuous stream of
|
||||
AssertionErrors starting on the line after "Found token", and never exited.
|
||||
|
||||
Offline machine checks, no network and no funds moved. A stub websocket raises
|
||||
what the real library raises, against the real listeners:
|
||||
|
||||
@@ -8,11 +8,11 @@ built, which showed up only as a repeating
|
||||
|
||||
Missing required fields in PumpPortal LetsBonk token data
|
||||
|
||||
Captured from the live feed on 2026-09-16: over 90 seconds it carried 24 pump
|
||||
creates and 7 bonk creates, and the two payloads are not the same shape. A bonk
|
||||
Captured from the live feed, which carries both pump creates and bonk creates,
|
||||
and the two payloads are not the same shape. A bonk
|
||||
create carries no `name`, no `symbol` and no `uri`, where a pump create has all
|
||||
three. Requiring name and symbol therefore rejected every bonk token that ever
|
||||
arrived - 100% of them, which matches the field report on the original run.
|
||||
arrived, which matches the field report on the original run.
|
||||
|
||||
Only `mint` and `traderPublicKey` cannot be derived from something else, so only
|
||||
those two are required now. `name` and `symbol` are used for logging and for the
|
||||
|
||||
@@ -12,8 +12,8 @@ The pair is read *positionally* from the end, and `pool-v2` only belongs to a
|
||||
`Pubkey::default()` otherwise (upstream PUMP_SWAP_CREATOR_FEE_README.md). Sending
|
||||
`pool-v2` on a non-canonical pool shifts the pair by one, so the program reads the
|
||||
pool-v2 PDA as the buyback recipient and rejects it with
|
||||
`BuybackFeeRecipientNotAuthorized` (6053). That was every non-canonical pool: 5 of
|
||||
5 sampled failed before the fix, 10 of 10 simulate cleanly after it (2026-09-23).
|
||||
`BuybackFeeRecipientNotAuthorized` (6053). That was every non-canonical pool
|
||||
before the fix; they simulate cleanly after it.
|
||||
|
||||
Unblocking those pools then exposed a second bug they had been hiding. The scripts
|
||||
hardcoded `TOKEN_DECIMALS = 6`, "standard for most pump.fun tokens" — true for
|
||||
|
||||
@@ -1,10 +1,9 @@
|
||||
"""Verify no trade path prices a coin before resolving its quote mint.
|
||||
|
||||
pump.fun's quote assets are not just SOL and USDC. The `QuoteControl` registry
|
||||
(PDA `["quote-control"]`) admits mints at 6, 8 and 9 decimals — 79 of the 170
|
||||
admitted on 2026-09-22 are tokenized equities, 8 decimals for Backed's xStocks
|
||||
and 6 for Backpack Securities — and coins paired with them trade live: a 75s
|
||||
sample of curve updates that day caught 8 stock-paired curves out of 124.
|
||||
(PDA `["quote-control"]`) admits mints at 6, 8 and 9 decimals — many of them
|
||||
tokenized equities, 8 decimals for Backed's xStocks and 6 for Backpack
|
||||
Securities — and coins paired with them trade live.
|
||||
|
||||
`quote_units()` used to default to 9 decimals for an unresolved mint. Every
|
||||
cookbook trade script called it *before* `resolve_quote_token_program()` warmed
|
||||
|
||||
@@ -0,0 +1,521 @@
|
||||
"""Offline verifier for the shreds (pre-execution) listener.
|
||||
|
||||
Guards the listener added for shreds mode and the instruction-decode path it
|
||||
depends on. Nothing here touches the network or moves funds; both fixtures are
|
||||
committed geyser `SubscribeDeshred` frames captured from mainnet.
|
||||
|
||||
The bugs and claims it pins:
|
||||
|
||||
1. **`user` is account 5 on `create_v2`, not 7.** `create_v2` dropped the four
|
||||
metaplex accounts legacy `create` carries, so the launcher's wallet sits four
|
||||
places earlier. Reading index 7 yields the Token-2022 program, and every
|
||||
address derived from the wallet is then wrong. This was dormant while only
|
||||
geyser/blocks used the instruction path as a fallback behind `log_messages`;
|
||||
shreds has no logs, so it runs on every coin.
|
||||
|
||||
2. **A holder-reward coin's creator comes from the mint, not from the args.**
|
||||
The program ignores `args.creator` on those coins and writes
|
||||
`PDA(["holder-rewards", mint])` into `BondingCurve.creator`, so
|
||||
`creator_vault` must derive from the PDA. Passing the wrong vault gets the
|
||||
buy rejected on a seeds constraint. Without this the shreds listener could
|
||||
not trade holder-reward coins at all, because there is no curve to read
|
||||
before execution.
|
||||
|
||||
3. **The trailing-arg form cannot lie about holder rewards.** `create_v2`'s
|
||||
trailing arguments are positional, so an instruction that stops short cannot
|
||||
have set a later one. A truncated form must therefore decode as *not* a
|
||||
holder-reward coin rather than raising or guessing.
|
||||
|
||||
4. **Lookup-table accounts are resolved before indexing.** The deshred stream
|
||||
reports them on the update as `loaded_writable_addresses` then
|
||||
`loaded_readonly_addresses`, in that order. Both fixtures genuinely use a
|
||||
lookup table, so a listener that ignores them fails here rather than on the
|
||||
first live coin.
|
||||
|
||||
5. **Nothing on this path reads `meta`.** There is no `TransactionStatusMeta` on
|
||||
a pre-execution stream, so a reader that reaches for `meta.log_messages` gets
|
||||
silence, not an error — the failure would be a listener that quietly detects
|
||||
nothing.
|
||||
|
||||
6. **The listener marks its tokens `state_from_event`.** That is what lets
|
||||
`extreme_fast_mode` submit without a curve read. Waiting instead is not a
|
||||
safer fallback here: the account does not exist yet.
|
||||
|
||||
Usage:
|
||||
uv run tests/regression/verify_shreds_listener.py
|
||||
"""
|
||||
|
||||
# The listener's public entry point opens a gRPC stream, so these checks drive
|
||||
# the decode helpers directly. Reaching past the public API is the point here.
|
||||
# ruff: noqa: SLF001
|
||||
|
||||
import ast
|
||||
import base64
|
||||
import json
|
||||
import struct
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[2]
|
||||
sys.path.insert(0, str(PROJECT_ROOT / "src"))
|
||||
sys.path.insert(0, str(PROJECT_ROOT))
|
||||
|
||||
from solders.pubkey import Pubkey # noqa: E402
|
||||
|
||||
from core.pubkeys import SystemAddresses # noqa: E402
|
||||
from geyser.generated import geyser_pb2 # noqa: E402
|
||||
from interfaces.core import Platform # noqa: E402
|
||||
from monitoring.universal_shreds_listener import UniversalShredsListener # noqa: E402
|
||||
from platforms.pumpfun.address_provider import PumpFunAddresses # noqa: E402
|
||||
|
||||
FIXTURE_DIR = PROJECT_ROOT / "cookbook" / "pumpfun" / "decode"
|
||||
HOLDER_REWARD_FIXTURE = FIXTURE_DIR / "raw_create_v2_holder_reward_from_deshred.json"
|
||||
ORDINARY_FIXTURE = FIXTURE_DIR / "raw_create_v2_ordinary_from_deshred.json"
|
||||
|
||||
LISTENER_SOURCE = PROJECT_ROOT / "src" / "monitoring" / "universal_shreds_listener.py"
|
||||
|
||||
# create_v2 account indices, from idl/pump_fun_idl.json.
|
||||
CREATE_V2_MINT_INDEX = 0
|
||||
CREATE_V2_BONDING_CURVE_INDEX = 2
|
||||
CREATE_V2_USER_INDEX = 5
|
||||
# The index the code used to read, kept so the check states what it rules out.
|
||||
LEGACY_CREATE_USER_INDEX = 7
|
||||
|
||||
CREATE_V2_DISCRIMINATOR = bytes([214, 144, 76, 236, 95, 139, 49, 180])
|
||||
|
||||
|
||||
def _load(path: Path) -> dict:
|
||||
"""Read one committed fixture.
|
||||
|
||||
Args:
|
||||
path: Fixture file path
|
||||
|
||||
Returns:
|
||||
The parsed fixture document
|
||||
"""
|
||||
with path.open() as handle:
|
||||
return json.load(handle)
|
||||
|
||||
|
||||
def _update(fixture: dict) -> geyser_pb2.SubscribeUpdateDeshred:
|
||||
"""Rebuild the geyser update from a fixture.
|
||||
|
||||
Args:
|
||||
fixture: Parsed fixture document
|
||||
|
||||
Returns:
|
||||
The decoded SubscribeUpdateDeshred protobuf
|
||||
"""
|
||||
update = geyser_pb2.SubscribeUpdateDeshred()
|
||||
update.ParseFromString(base64.b64decode(fixture["subscribe_update_base64"]))
|
||||
return update
|
||||
|
||||
|
||||
def _listener() -> UniversalShredsListener:
|
||||
"""Build the listener without connecting to anything.
|
||||
|
||||
Returns:
|
||||
A pump.fun-only shreds listener
|
||||
"""
|
||||
return UniversalShredsListener(
|
||||
geyser_endpoint="unused.invalid:443",
|
||||
geyser_api_token="unused", # noqa: S106 — never sent; nothing connects
|
||||
geyser_auth_type="x-token",
|
||||
platforms=[Platform.PUMP_FUN],
|
||||
)
|
||||
|
||||
|
||||
def _account_keys(update: geyser_pb2.SubscribeUpdateDeshred) -> list[bytes]:
|
||||
"""Resolve the full account table for a deshred update.
|
||||
|
||||
Args:
|
||||
update: A deshred SubscribeUpdateDeshred
|
||||
|
||||
Returns:
|
||||
Static keys followed by writable then read-only loaded addresses
|
||||
"""
|
||||
transaction = update.deshred_transaction.transaction
|
||||
keys = list(transaction.transaction.message.account_keys)
|
||||
keys.extend(transaction.loaded_writable_addresses)
|
||||
keys.extend(transaction.loaded_readonly_addresses)
|
||||
return keys
|
||||
|
||||
|
||||
def _create_instruction(update: geyser_pb2.SubscribeUpdateDeshred) -> object | None:
|
||||
"""Find the create_v2 instruction in a deshred update.
|
||||
|
||||
Args:
|
||||
update: A deshred SubscribeUpdateDeshred
|
||||
|
||||
Returns:
|
||||
The instruction, or None if the fixture carries no create_v2
|
||||
"""
|
||||
message = update.deshred_transaction.transaction.transaction.message
|
||||
for instruction in message.instructions:
|
||||
if bytes(instruction.data).startswith(CREATE_V2_DISCRIMINATOR):
|
||||
return instruction
|
||||
return None
|
||||
|
||||
|
||||
def check_fixtures_are_pre_execution() -> bool:
|
||||
"""Both fixtures must be real deshred frames with no execution metadata.
|
||||
|
||||
A fixture carrying logs would make every other check here meaningless: the
|
||||
parser would take the CreateEvent route that shreds mode does not have.
|
||||
|
||||
Returns:
|
||||
True if both fixtures are pre-execution frames
|
||||
"""
|
||||
ok = True
|
||||
for path in (HOLDER_REWARD_FIXTURE, ORDINARY_FIXTURE):
|
||||
update = _update(_load(path))
|
||||
if not update.HasField("deshred_transaction"):
|
||||
print(f" {path.name} is not a deshred update")
|
||||
ok = False
|
||||
continue
|
||||
transaction = update.deshred_transaction.transaction
|
||||
# Not "is meta unset" but "there is no meta field to set": the deshred
|
||||
# message has no TransactionStatusMeta, which is what makes the
|
||||
# CreateEvent route structurally unavailable rather than merely absent
|
||||
# from these two captures.
|
||||
fields = {field.name for field in transaction.DESCRIPTOR.fields}
|
||||
if "meta" in fields:
|
||||
print(
|
||||
f" {path.name}: the deshred message gained a meta field; "
|
||||
"the CreateEvent route may now be available and this listener's "
|
||||
"instruction-only design should be revisited"
|
||||
)
|
||||
ok = False
|
||||
if _create_instruction(update) is None:
|
||||
print(f" {path.name} carries no top-level create_v2")
|
||||
ok = False
|
||||
return ok
|
||||
|
||||
|
||||
def check_fixtures_use_lookup_tables() -> bool:
|
||||
"""The fixtures must actually need lookup-table resolution.
|
||||
|
||||
Otherwise check_lookup_table_accounts_resolved proves nothing: a listener
|
||||
that ignored loaded addresses would pass it by accident.
|
||||
|
||||
Returns:
|
||||
True if both fixtures load accounts from a lookup table
|
||||
"""
|
||||
ok = True
|
||||
for path in (HOLDER_REWARD_FIXTURE, ORDINARY_FIXTURE):
|
||||
update = _update(_load(path))
|
||||
transaction = update.deshred_transaction.transaction
|
||||
loaded = len(transaction.loaded_writable_addresses) + len(
|
||||
transaction.loaded_readonly_addresses
|
||||
)
|
||||
if loaded == 0:
|
||||
print(f" {path.name} loads no lookup-table accounts")
|
||||
ok = False
|
||||
return ok
|
||||
|
||||
|
||||
def check_lookup_table_accounts_resolved() -> bool:
|
||||
"""Account indices must resolve past the static keys, in the reported order.
|
||||
|
||||
Args:
|
||||
None
|
||||
|
||||
Returns:
|
||||
True if the resolved table is static + writable + readonly
|
||||
"""
|
||||
update = _update(_load(ORDINARY_FIXTURE))
|
||||
transaction = update.deshred_transaction.transaction
|
||||
static = list(transaction.transaction.message.account_keys)
|
||||
resolved = UniversalShredsListener._resolve_account_keys(transaction)
|
||||
|
||||
expected = (
|
||||
static
|
||||
+ list(transaction.loaded_writable_addresses)
|
||||
+ list(transaction.loaded_readonly_addresses)
|
||||
)
|
||||
if [bytes(k) for k in resolved] != [bytes(k) for k in expected]:
|
||||
print(" resolved account table does not match static+writable+readonly")
|
||||
return False
|
||||
|
||||
instruction = _create_instruction(update)
|
||||
highest = max(instruction.accounts)
|
||||
if highest < len(static):
|
||||
print(" create_v2 indexes no loaded account; the check is vacuous")
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def check_user_account_index() -> bool:
|
||||
"""`user` must come from account 5, not the legacy index 7.
|
||||
|
||||
Returns:
|
||||
True if the parsed user is the wallet at index 5
|
||||
"""
|
||||
ok = True
|
||||
for path in (HOLDER_REWARD_FIXTURE, ORDINARY_FIXTURE):
|
||||
update = _update(_load(path))
|
||||
keys = _account_keys(update)
|
||||
instruction = _create_instruction(update)
|
||||
accounts = list(instruction.accounts)
|
||||
|
||||
expected_user = Pubkey.from_bytes(keys[accounts[CREATE_V2_USER_INDEX]])
|
||||
wrong_user = Pubkey.from_bytes(keys[accounts[LEGACY_CREATE_USER_INDEX]])
|
||||
|
||||
token_info = _listener()._process_update(update)
|
||||
if token_info is None:
|
||||
print(f" {path.name} produced no TokenInfo")
|
||||
ok = False
|
||||
continue
|
||||
|
||||
if token_info.user != expected_user:
|
||||
print(
|
||||
f" {path.name}: user is {token_info.user}, "
|
||||
f"expected account 5 ({expected_user})"
|
||||
)
|
||||
ok = False
|
||||
|
||||
# State what the old index actually pointed at, so a regression is
|
||||
# recognisable rather than just unequal.
|
||||
if wrong_user != SystemAddresses.TOKEN_2022_PROGRAM:
|
||||
print(
|
||||
f" {path.name}: account 7 is {wrong_user}, expected the "
|
||||
"Token-2022 program — the fixture or the IDL layout moved"
|
||||
)
|
||||
ok = False
|
||||
if token_info.user == wrong_user:
|
||||
print(f" {path.name}: user was read from the legacy index 7")
|
||||
ok = False
|
||||
return ok
|
||||
|
||||
|
||||
def check_holder_reward_creator_derived() -> bool:
|
||||
"""A holder-reward coin's creator must be the mint's holder-rewards PDA.
|
||||
|
||||
Returns:
|
||||
True if creator and creator_vault derive from the PDA, not from args
|
||||
"""
|
||||
fixture = _load(HOLDER_REWARD_FIXTURE)
|
||||
if not fixture.get("is_holder_reward"):
|
||||
print(" the holder-reward fixture is not flagged holder-reward")
|
||||
return False
|
||||
|
||||
update = _update(fixture)
|
||||
token_info = _listener()._process_update(update)
|
||||
if token_info is None:
|
||||
print(" holder-reward fixture produced no TokenInfo")
|
||||
return False
|
||||
|
||||
expected_creator = PumpFunAddresses.find_holder_reward_creator(token_info.mint)
|
||||
expected_vault, _ = Pubkey.find_program_address(
|
||||
[b"creator-vault", bytes(expected_creator)], PumpFunAddresses.PROGRAM
|
||||
)
|
||||
# The substitution is only meaningful if it differs from what the
|
||||
# instruction claims; otherwise the fixture proves nothing.
|
||||
args_creator = _args_creator(_create_instruction(update))
|
||||
|
||||
ok = True
|
||||
if not token_info.is_holder_reward:
|
||||
print(" holder-reward flag not decoded from the instruction")
|
||||
ok = False
|
||||
if token_info.creator != expected_creator:
|
||||
print(
|
||||
f" creator is {token_info.creator}, expected the holder-rewards "
|
||||
f"PDA {expected_creator}"
|
||||
)
|
||||
ok = False
|
||||
if token_info.creator_vault != expected_vault:
|
||||
print(
|
||||
f" creator_vault is {token_info.creator_vault}, expected "
|
||||
f"{expected_vault}"
|
||||
)
|
||||
ok = False
|
||||
if token_info.creator == args_creator:
|
||||
print(
|
||||
" creator equals args.creator; the fixture does not exercise "
|
||||
"the substitution"
|
||||
)
|
||||
ok = False
|
||||
return ok
|
||||
|
||||
|
||||
def _args_creator(instruction: object) -> Pubkey:
|
||||
"""Read the `creator` argument out of a create_v2 instruction.
|
||||
|
||||
Args:
|
||||
instruction: A create_v2 instruction
|
||||
|
||||
Returns:
|
||||
The creator the launcher passed
|
||||
"""
|
||||
data = bytes(instruction.data)
|
||||
offset = 8
|
||||
for _ in range(3): # name, symbol, uri
|
||||
length = struct.unpack_from("<I", data, offset)[0]
|
||||
offset += 4 + length
|
||||
return Pubkey.from_bytes(data[offset : offset + 32])
|
||||
|
||||
|
||||
def check_ordinary_creator_from_args() -> bool:
|
||||
"""An ordinary coin's creator is the one the instruction carries.
|
||||
|
||||
Returns:
|
||||
True if creator matches args.creator for a non-holder-reward coin
|
||||
"""
|
||||
fixture = _load(ORDINARY_FIXTURE)
|
||||
if fixture.get("is_holder_reward"):
|
||||
print(" the ordinary fixture is flagged holder-reward")
|
||||
return False
|
||||
|
||||
update = _update(fixture)
|
||||
token_info = _listener()._process_update(update)
|
||||
if token_info is None:
|
||||
print(" ordinary fixture produced no TokenInfo")
|
||||
return False
|
||||
|
||||
if token_info.is_holder_reward:
|
||||
print(" ordinary coin decoded as holder-reward")
|
||||
return False
|
||||
|
||||
expected = _args_creator(_create_instruction(update))
|
||||
if token_info.creator != expected:
|
||||
print(f" creator is {token_info.creator}, expected {expected}")
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def check_truncated_trailing_args_are_not_holder_reward() -> bool:
|
||||
"""A create_v2 that stops short cannot be a holder-reward coin.
|
||||
|
||||
The trailing arguments are positional, so reaching argument 8 means sending
|
||||
6 and 7 first. Truncating the real instruction must therefore decode as a
|
||||
plain coin — never raise, and never leave the flag set.
|
||||
|
||||
Returns:
|
||||
True if every truncated form decodes as a non-holder-reward coin
|
||||
"""
|
||||
update = _update(_load(HOLDER_REWARD_FIXTURE))
|
||||
instruction = _create_instruction(update)
|
||||
data = bytearray(instruction.data)
|
||||
|
||||
# Trailing args are the last 10 bytes: is_cashback_enabled (1),
|
||||
# creator_fee_bps (8), is_holder_reward (1).
|
||||
listener = _listener()
|
||||
for dropped in (1, 9, 10):
|
||||
truncated = bytes(data[: len(data) - dropped])
|
||||
instruction.data = truncated
|
||||
try:
|
||||
token_info = listener._process_update(update)
|
||||
except Exception as error: # noqa: BLE001
|
||||
print(f" dropping {dropped} trailing bytes raised {error!r}")
|
||||
return False
|
||||
if token_info is None:
|
||||
print(f" dropping {dropped} trailing bytes produced no TokenInfo")
|
||||
return False
|
||||
if token_info.is_holder_reward:
|
||||
print(
|
||||
f" dropping {dropped} trailing bytes still decoded as holder-reward"
|
||||
)
|
||||
return False
|
||||
# With the flag unset the creator must fall back to the args, not the PDA.
|
||||
if token_info.creator != _args_creator(instruction):
|
||||
print(
|
||||
f" dropping {dropped} trailing bytes did not fall back to "
|
||||
"args.creator"
|
||||
)
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def check_tokens_marked_trusted() -> bool:
|
||||
"""The listener must mark its tokens so extreme_fast_mode skips the curve read.
|
||||
|
||||
Returns:
|
||||
True if both fixtures produce state_from_event tokens with a quote mint
|
||||
"""
|
||||
ok = True
|
||||
for path in (HOLDER_REWARD_FIXTURE, ORDINARY_FIXTURE):
|
||||
token_info = _listener()._process_update(_update(_load(path)))
|
||||
if token_info is None:
|
||||
print(f" {path.name} produced no TokenInfo")
|
||||
ok = False
|
||||
continue
|
||||
if not token_info.state_from_event:
|
||||
print(f" {path.name}: state_from_event not set")
|
||||
ok = False
|
||||
# _can_skip_refresh also requires a resolved quote mint.
|
||||
if token_info.quote_mint is None:
|
||||
print(f" {path.name}: quote_mint unresolved, the buy would refresh")
|
||||
ok = False
|
||||
return ok
|
||||
|
||||
|
||||
def check_listener_never_reads_meta() -> bool:
|
||||
"""The listener must not reach for execution metadata that cannot exist.
|
||||
|
||||
Returns:
|
||||
True if the listener source references no meta field
|
||||
"""
|
||||
tree = ast.parse(LISTENER_SOURCE.read_text())
|
||||
|
||||
# Walk attribute access rather than scanning text, so a mention inside a
|
||||
# docstring or comment -- this module is full of them -- is not a finding.
|
||||
banned = {"meta", "log_messages", "logMessages"}
|
||||
offenders = [
|
||||
f"line {node.lineno}: .{node.attr}"
|
||||
for node in ast.walk(tree)
|
||||
if isinstance(node, ast.Attribute) and node.attr in banned
|
||||
]
|
||||
|
||||
if offenders:
|
||||
print(" listener reads execution metadata that deshred never carries:")
|
||||
for line in offenders:
|
||||
print(f" {line}")
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
CHECKS = (
|
||||
("fixtures are pre-execution deshred frames", check_fixtures_are_pre_execution),
|
||||
("fixtures genuinely use lookup tables", check_fixtures_use_lookup_tables),
|
||||
("lookup-table accounts resolved in order", check_lookup_table_accounts_resolved),
|
||||
("user read from create_v2 account 5", check_user_account_index),
|
||||
("holder-reward creator derived from mint", check_holder_reward_creator_derived),
|
||||
("ordinary creator taken from args", check_ordinary_creator_from_args),
|
||||
(
|
||||
"truncated trailing args are not holder-reward",
|
||||
check_truncated_trailing_args_are_not_holder_reward,
|
||||
),
|
||||
("tokens marked state_from_event", check_tokens_marked_trusted),
|
||||
("listener never reads meta", check_listener_never_reads_meta),
|
||||
)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
"""Run every check.
|
||||
|
||||
Returns:
|
||||
0 if all checks pass, 1 otherwise
|
||||
"""
|
||||
print("Verifying the shreds (pre-execution) listener\n")
|
||||
failures = 0
|
||||
for name, check in CHECKS:
|
||||
try:
|
||||
passed = check()
|
||||
except Exception as error: # noqa: BLE001
|
||||
print(f" [FAIL] {name}")
|
||||
print(f" raised {error!r}")
|
||||
failures += 1
|
||||
continue
|
||||
print(f" [{'PASS' if passed else 'FAIL'}] {name}")
|
||||
failures += not passed
|
||||
|
||||
print()
|
||||
if failures:
|
||||
print(f"{failures} check(s) failed")
|
||||
return 1
|
||||
print("All checks passed")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -7,10 +7,9 @@ v1 traffic. Two things break a reader that ignores it.
|
||||
|
||||
`maxSupportedTransactionVersion` is a whole-frame setting, not a per-transaction
|
||||
one. Asking `blockSubscribe` for 0 does not skip the v1 transactions in a block,
|
||||
it nulls out `value.block` for the entire notification. Measured against mainnet
|
||||
on 2026-09-16, 60s per run: `0` delivered 1 block and 177 nulls, `1` delivered
|
||||
78 blocks and no nulls — the blocks listener was roughly 99% blind, silently,
|
||||
because a null frame looks exactly like a skipped slot.
|
||||
it nulls out `value.block` for the entire notification. Asking for `0` leaves
|
||||
the blocks listener almost entirely blind, silently, because a null frame looks
|
||||
exactly like a skipped slot.
|
||||
|
||||
solders could not deserialize a v1 transaction (they begin with byte 129 and
|
||||
put the signatures at the tail) until 0.29. That was survivable, because the RPC
|
||||
|
||||
@@ -16,9 +16,6 @@ stream on the same endpoint and reports two things:
|
||||
amount of decoding recovers them. This classifies every create as
|
||||
top-level or inner to size that blind spot.
|
||||
|
||||
Measured 2026-09-23 over 300s: creates led by a 6.4ms median (92% of the time),
|
||||
while 4 of 118 creates were CPI-shaped and therefore undetectable on deshred.
|
||||
|
||||
Usage:
|
||||
uv run tools/compare_deshred_latency.py
|
||||
uv run tools/compare_deshred_latency.py --duration 600
|
||||
|
||||
@@ -45,7 +45,7 @@ EXTREME_FAST_TOKEN_AMOUNT = 20
|
||||
HOLD_SECONDS = 5
|
||||
PRIORITY_FEE = 1_000_000
|
||||
# This harness verifies plumbing, not profitability. Brand-new coins routinely
|
||||
# lose 90% of their curve SOL within seconds of creation, and a realistic
|
||||
# lose most of their curve SOL within seconds of creation, and a realistic
|
||||
# sell_slippage of 0.3 then makes the program reject the sell with
|
||||
# TooLittleSolReceived (6003) — a correct guard, but it stops us proving the
|
||||
# 26-account sell_v2 actually lands. Accept almost any payout instead; the
|
||||
|
||||
Reference in New Issue
Block a user