mirror of
https://github.com/chainstacklabs/pumpfun-bonkfun-bot.git
synced 2026-08-16 00:38:05 +00:00
fix(learning-examples): repair crashing listeners and decoders, prune obsolete paths, unify naming (#181)
Audited every script in learning-examples/ against mainnet. Broken — found by running them, all invisible offline: - listen_geyser.py crashed with IndexError after ~11 coins: it never resolved v0 address-lookup-table accounts, which geyser reports in meta.loaded_writable_addresses / loaded_readonly_addresses. Resolving them removes the crash and brings detections level with the WebSocket listeners, 35 coins each per 150 s. - compare_listeners.py logged 13,090,862 error lines / 888 MB in 150 s and never printed its own 30-second report: the inner recv() loop caught ConnectionClosed in a broad handler that only logged, so every following recv() raised at once and the outer reconnect handler was unreachable. Now 12 KB and exit 0. Same shape fixed in compare_migration_listeners.py, listen_blocksubscribe.py and extract_blocksubscribe_transactions.py; the last two also gained the reconnect loop their siblings already had. - decode_from_gettransaction.py matched instructions on account count instead of discriminator, reporting a real 19-account create_v2 as claim_cashback with every account under the wrong name. It also walked only top-level instructions, and in 40 consecutive pump.fun transactions there was 1 top-level pump instruction against 8 inner ones. - decode_from_blocksubscribe.py crashed on every real create_v2: on chain the trailing args are variable length, 0001 in one tx and 00 in another, so is_cashback_enabled can be absent entirely. - poll_bonding_curve_progress.py polled a hardcoded dead mint and took no argv. Obsolete: - Delete listen_blocksubscribe_old_raydium.py. Seven minutes on mainnet produced 0 initialize2 events while the wrapper listener caught 3 real migrations. - Delete the duplicate geyser stubs and protos under listen-new-tokens/. The protos were byte-identical to src/geyser/proto and the stubs had drifted; both geyser examples now import src.geyser.generated. - Recapture all four fixtures. The old ones were from Aug 2024 and included a 49-byte pre-creator bonding curve. Behind the protocol: - fetch_price.py, get_bonding_curve_status.py, poll_bonding_curve_progress.py and decode_from_getaccountinfo.py never read quote_mint and scaled by a hardcoded 1e9. Against a live USDC-paired curve the price was off by 1000x. - get_pumpswap_pools.py stopped parsing at coin_creator and missed the i128 virtual_quote_reserves. Live pools carry 17.5845 SOL of them, which under-prices by 3.5-23.9% when ignored. Duplication and naming: - Merge manual_buy_cu_optimized.py into manual_buy.py --cu-optimized. The deleted file's docstring said 512 KB while its code used 16 MB; simulation confirms 512 KB and 4 MB both fail MaxLoadedAccountsDataSizeExceeded on Token-2022 mints, so 16 MB is the correct value. - Merge listen_logsubscribe_abc.py into listen_logsubscribe.py. Its ATA derivation hardcoded the legacy token program, so every Associated BC it printed for a Token2022 coin was an address that does not exist on chain. Fixed on merge and cross-checked 59/59 against on-chain accounts. - Remove 19 dead symbols. BREAKING_FEE_RECIPIENTS is still live in the PumpSwap scripts and stays there. - Normalize naming: kebab-case directories, RPC method names as one lowercase token, scripts verb-first. Rules documented in CLAUDE.md. get_graduating_tokens.py is knowingly left broken: getProgramAccounts over the whole pump program is now rejected by providers and it needs a getProgramAccountsV2 rewrite, which belongs in its own PR. Verified: both offline gates pass, all 41 examples parse, every read-only script exercised on mainnet against SOL- and USDC-paired coins, no new ruff findings (427 -> 413). No script that spends real funds was run. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
5c1fb7eb28
commit
a33f4fd035
@@ -38,6 +38,23 @@ registry in `platforms/__init__.py`. Listeners and the trader are
|
||||
platform-agnostic (`Universal*`); anything platform-shaped belongs under
|
||||
`platforms/<name>/`.
|
||||
|
||||
### Naming inside `learning-examples/`
|
||||
|
||||
- **Directories are kebab-case** (`bonding-curve-progress`, `listen-new-tokens`,
|
||||
`copy-trading`). A single-token product name stays one word (`pumpswap`).
|
||||
- **Files are snake_case, verb first** — `fetch_price.py`, `decode_from_*.py`,
|
||||
`extract_blocksubscribe_transactions.py`, `verify_*.py`, `simulate_*.py`.
|
||||
Exceptions are the shared helper modules `pump_v2.py` and `tx_status.py`, which
|
||||
are libraries rather than runnable scripts.
|
||||
- **RPC and service names are lowercased into one token**, never camelCase:
|
||||
`blocksubscribe`, `logsubscribe`, `programsubscribe`, `getaccountinfo`,
|
||||
`gettransaction`, `pumpportal`. So `decode_from_gettransaction.py`, not
|
||||
`decode_from_getTransaction.py`.
|
||||
- Fixtures are `raw_<what>_from_<method>.json` next to the script that reads
|
||||
them, under the same rules.
|
||||
- `live_*` marks a script that spends real funds; `simulate_*` and `verify_*`
|
||||
never do.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
@@ -67,8 +84,11 @@ commented-out code). Type-hint public functions, Google-style docstrings, and
|
||||
Python 3.11+ (`requires-python = ">=3.11"`, matching ruff's target). Runtime
|
||||
deps are declared in `[project.dependencies]`; `ruff` and `grpcio-tools` live in
|
||||
`[dependency-groups] dev`, which `uv sync` installs by default. `grpcio-tools`
|
||||
is protoc — needed only to regenerate the `geyser_pb2` stubs from `proto/`, never
|
||||
at runtime.
|
||||
is protoc — needed only to regenerate the `geyser_pb2` stubs in
|
||||
`src/geyser/generated/` from `src/geyser/proto/`, never at runtime. That is the
|
||||
**only** copy: the geyser examples reach it by putting the repo root on
|
||||
`sys.path` and importing `src.geyser.generated`. Don't add a second copy under
|
||||
`learning-examples/` — the last one drifted out of sync with the protos.
|
||||
|
||||
### Verifying pump.fun v2 trade instructions
|
||||
|
||||
@@ -121,6 +141,47 @@ with `BuybackFeeRecipientMissing` (6062) printed as confirmed buys.
|
||||
and `str()` on it is empty, so the caller logs a blank reason. A slow
|
||||
`getAccountInfo` is enough to take down a whole listener run this way.
|
||||
|
||||
### Listener and decoder pitfalls
|
||||
|
||||
Each of these was a live bug in `learning-examples/`, all of them invisible
|
||||
offline and only visible after a couple of minutes against mainnet.
|
||||
|
||||
- **A `while True: recv()` loop must break out on `websockets.ConnectionClosed`.**
|
||||
Catching it in a broad `except Exception` that only logs makes the next `recv()`
|
||||
raise immediately, forever: `compare_listeners.py` produced **13,090,862 error
|
||||
lines / 888 MB in 150 s** and never reached its own 30-second report. The outer
|
||||
reconnect handler with its `sleep` is unreachable in that shape. A narrow
|
||||
`except TimeoutError` or `except json.JSONDecodeError` is fine to swallow —
|
||||
those are per-message, not per-connection.
|
||||
- **Resolve v0 lookup-table accounts before indexing them.** An instruction's
|
||||
account indices can point past `message.account_keys` into the address lookup
|
||||
table, which geyser reports in `meta.loaded_writable_addresses` then
|
||||
`loaded_readonly_addresses` (that order). Ignoring them crashed the geyser
|
||||
example with `IndexError` after ~11 coins in 150 s; resolving them removed the
|
||||
crash and brought its detection count level with the WebSocket listeners
|
||||
(35 coins each over the same window).
|
||||
- **Identify an instruction by its 8-byte discriminator, never by account count.**
|
||||
Several pump.fun instructions share a count, so counting mislabels them and
|
||||
then prints every account under the wrong name — a real 19-account `create_v2`
|
||||
was reported as `claim_cashback`. Note `buy_exact_sol_in` is also 18 accounts
|
||||
on chain, same as legacy `buy`.
|
||||
- **Walk `meta.innerInstructions`, not just `message.instructions`.** Most trades
|
||||
reach the program as a CPI from a router or aggregator: in 40 consecutive
|
||||
pump.fun transactions there was **1 top-level** pump instruction against
|
||||
**8 inner** ones. Anchor's event-CPI prefix (`e445a52e51cb9a1d`) accounts for a
|
||||
good share of the inner instructions; the event's own discriminator follows it.
|
||||
- **`getProgramAccounts` over the whole pump program is rejected** by current
|
||||
providers: *"Too many accounts requested (10000001 pubkeys) … use
|
||||
getProgramAccountsV2 with pagination"*. It still works against pump-amm, which
|
||||
is small enough. `bonding-curve-progress/get_graduating_tokens.py` is knowingly
|
||||
broken on this and needs the V2 pagination rewrite.
|
||||
- **`SetLoadedAccountsDataSizeLimit` must stay generous: 16 MB, not 512 KB.**
|
||||
Verified by simulation on a Token-2022 mint with extensions — 512 KB and 4 MB
|
||||
both fail `MaxLoadedAccountsDataSizeExceeded` with `unitsConsumed=0` (never
|
||||
executed), while 16 MB reaches the buy instruction and is still 4x under the
|
||||
64 MB default. solders has no builder for it; encode `struct.pack("<BI", 4, n)`
|
||||
against the compute-budget program.
|
||||
|
||||
## Pump.fun protocol notes (gotchas)
|
||||
|
||||
The IDLs under `idl/` are vendored verbatim from `github.com/pump-fun/pump-public-docs`
|
||||
@@ -185,11 +246,24 @@ The IDLs under `idl/` are vendored verbatim from `github.com/pump-fun/pump-publi
|
||||
symbol (str), uri (str), creator (pubkey), is_mayhem_mode (bool),
|
||||
is_cashback_enabled (OptionBool 1B)`. `OptionBool` is a struct wrapping a single
|
||||
bool — serialized as 1 byte, not 2.
|
||||
- **`is_cashback_enabled` can be absent from the wire entirely.** Two mainnet
|
||||
`create_v2` instructions carry `0001` and `00` after `creator`: one sends both
|
||||
trailing args, the other omits the last. A decoder that reads a fixed number of
|
||||
trailing bytes raises `IndexError` on roughly half of all coins. Decode
|
||||
trailing args defensively and report a missing one as unset.
|
||||
- `create_v2` accounts 1-16 are in the IDL; accounts **17-19 are optional
|
||||
remaining accounts** (`quote_mint`, `associated_quote_bonding_curve`,
|
||||
`quote_token_program`) appended only for a non-SOL quote mint. All three or
|
||||
none. This is the only way to read a new coin's quote asset from the
|
||||
instruction rather than the event.
|
||||
`quote_token_program`). All three or none. This is the only way to read a new
|
||||
coin's quote asset from the instruction rather than the event. In practice they
|
||||
are appended for **SOL-paired coins too**, carrying wrapped SOL — a live
|
||||
SOL-paired `create_v2` was observed with 19 accounts and account 17 = WSOL — so
|
||||
do not treat a 19-account `create_v2` as proof of a non-SOL quote asset. Read
|
||||
`quote_mint` off the curve instead.
|
||||
- The **associated bonding curve is an ordinary ATA**, so its address depends on
|
||||
which token program owns the mint: Token2022 for `create_v2` coins, SPL Token
|
||||
for legacy `create`. Deriving with the wrong program returns a valid-looking
|
||||
address that does not exist on chain. Verified: curve `3jJ83ND…` derives to
|
||||
`Cd4iC3Jn…` under Token2022 (matches chain) and `AhNzZsBp…` under SPL Token.
|
||||
- `extreme_fast_mode` skips the curve-state price fetch but still refreshes
|
||||
mayhem/cashback/creator/**quote_mint** from chain, because the wrong quote mint
|
||||
means spending the wrong balance entirely. Event parsers also populate
|
||||
|
||||
Reference in New Issue
Block a user