feat(monitoring): add shreds listener for pre-execution token detection (#220)

* feat(monitoring): add shreds listener for pre-execution token detection

Adds a `shreds` listener backed by geyser's SubscribeDeshred, which delivers
transactions when entries form from shreds, before execution. pump.fun only;
bots/bot-sniper-5-shreds.yaml ships disabled.

With no TransactionStatusMeta there is no CreateEvent, so the listener decodes
the create instruction and resolves what the buy needs itself. It marks its
tokens state_from_event, because there is no curve to read before the create
executes: trading on the instruction or not trading at all are the only
options.

Two fixes to the shared instruction-decode path, which geyser and blocks also
reach when logs are absent:

- `user` is account 5 on create_v2, not 7. create_v2 drops the metaplex
  accounts legacy create carries, so index 7 is the token program and every
  address derived from the wallet was wrong.
- A holder-reward coin's creator is PDA(["holder-rewards", mint]). The program
  ignores args.creator on those coins and writes that PDA into
  BondingCurve.creator, so creator_vault must derive from it or the buy is
  rejected on a seeds constraint.

Subscribes on the mint authority rather than the program id: it appears in
create/create_v2 and in no trade instruction, which keeps the stream to
creations instead of every pump.fun transaction.

Verified live on mainnet with extreme_fast_mode on and off and both the
time_based and tp_sl exits, each ending with the wallet holding SOL only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: drop reported measurement statistics from prose

Removes sample counts, traffic percentages, latency percentiles, window
durations and "measured on <date>" claims from docstrings, comments, docs and
bot YAML. The fact each measurement supported is kept and stated plainly.

Numbers that are load-bearing rather than reporting stay: protocol and packet
limits, the CU figures used to tune compute budgets, and dates pinning on-chain
or dependency upgrades.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Anton Sauchyk
2026-09-23 17:39:33 +02:00
committed by GitHub
co-authored by Claude Opus 5
parent 1e2de43292
commit 2ba40c0771
27 changed files with 1358 additions and 188 deletions
@@ -12,8 +12,8 @@ Two things this example exists to show, because both are easy to get wrong:
2. Read `meta.innerInstructions` as well as `message.instructions`. Most pump.fun
trades reach the program as a CPI from an aggregator or router, so a decoder
that only walks the top level sees almost nothing — in a sample of 40 consecutive
pump.fun transactions there was 1 top-level pump instruction against 8 inner ones.
that only walks the top level sees almost nothing — top-level pump
instructions are heavily outnumbered by inner ones.
3. Accept whichever encoding the response was captured in. `getTransaction`
answers in `jsonParsed`, `json` or `base64` depending on what was asked for,
@@ -0,0 +1,8 @@
{
"_comment": "One pump.fun create_v2 as delivered by geyser SubscribeDeshred (pre-execution). There is no TransactionStatusMeta on this stream, so there are no log messages and no CreateEvent: the create instruction is the only source.",
"mint": "9AyUC2BSCmQfuCjB1V77yNYyS8YHXsaq2uvVbXw9pump",
"is_holder_reward": true,
"create_v2_trailing_arg_bytes": 10,
"loaded_lookup_table_accounts": 13,
"subscribe_update_base64": "CgFmEoEOCvgNCkDW7GH1ZzFw3wu5/OH88QRU5X0J2ZW1XqOk8jDO0IQJletGISKblHoYPLxePOBAs1bE5HmvY9ilM64cjfYuSUIBGvMJCkDW7GH1ZzFw3wu5/OH88QRU5X0J2ZW1XqOk8jDO0IQJletGISKblHoYPLxePOBAs1bE5HmvY9ilM64cjfYuSUIBCkBQiwSylScjScxgqkphp89clMozxGflyNZT0nkh1FumFbmwyAU7x7d7ojJPY77EXDV+ET5/F+UPqh0VHAp+rFoMEuwICgQIAhgJEiDB/PHN5Ck25qFYzs1KMHvXgZVVB/K/doRH/i1w1pQ/YxIgeWuSeRzAi1v9ZaHYU9v40ukB6dKe8RBEUkeJO5M58Y8SIAeA9I4+XqfNEKiLXbUZgonyxA/ua7cQhB88Rd77a9uIEiA/px6nlYLHqYxMtdlwx5O8DxGx9LiGgA5x9E+SBYUhKhIgRuC8GHq8YcIareXyNDCyPzxxk1NBIrzTjdT05vWo0+gSIEtoRBm0Sh1GbbxjTLGJwaoAD9lsHiN0ds95bv3AporyEiBh0O+XAX1aDvuzLysS4nfDuX8JIZRf2qo6jZjkWVsE+RIge92l5IAzkZvGSnPk5uI1yRJrS6YMsdAspf5wULcF2mESIIEd127HqTmeyzeqIDYwgVsgwIJHrzyXyIykLg6Rf0wfEiDJ9MOoyWb72oi2V2lLUjoibHwYa12gi6grDWhPF+lcpxIgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAASIAFW4PaTZlrPRNsVaL8XW6pRicuX9dL/O2VdK7b9bRiwEiADBkZv5SEXMv/srbpyw5vnvIzlu8X3EmssQ5s6QAAAABIgBHnVW/IxwG7udMVuzmgVB/2xst6j9I5RArHNola8E48SIAbd9uHXZaGT2cvhRs7reawctIXtX1s3kTqM9YV+/wCpEiBY6n2uAJq/JZn175WcnSk5FDty0P/UR+xgddr5f50QIhIgb+tgX91tNg5Dg5Kj0nE0Gxg00NYMgKYkWXL4LlW5GqsSIIyXJY9OJInxuz0QKRSODYMLWhOZ2v8QhASOe9jb6fhZEiC0P/on9df2SnTAmx8pWHneSwmrNt/J3VFLMhqns4zl6BoglJzPAlCXmx/aezUkw9DpRX7+0wZTebudz7fikccDSkciCQgMGgUCwFwVACKvAQgLEhQBGgYCHAAKGxETGBQFAx0LEAQODxqUAdaQTOxfizG0BAAAAE5PTEEEAAAATk9MQU0AAABodHRwczovL2Zhc3QtaXBmcy5jb20vYXN5bmMvOUF5VUMyQlNDbVFmdUNqQjFWNzd5Tll5UzhZSFhzYXEydXZWYlh3OXB1bXAuanNvbsH88c3kKTbmoVjOzUowe9eBlVUH8r92hEf+LXDWlD9jAAABAAAAAAAAAAEiFAgKEgIACRoMAgAAAAAtMQEAAAAAIg0IERIGAAcAGQoOGgEBIhQIChICAAcaDAIAAAAA8gUqAQAAACIICA4SAQcaAREiDQgREgYACAAQCg4aAQEiQAgNEhMABwgZEA4ODRINHw4WHhUXBwgAGie7ZPrMMcSvFADyBSoBAAAADChj6CUAAADuAgAAAAABAAAAaRAnAAEiCggOEgMHAAAaAQkoATIuCiAMTgRReRLiWDL8AJ4vhn3nREHkmFuQXwvfM8VDleBH9BICDlcaBhUfAxYFBDIrCiB7IaeWrpHAP+GCFcNSxU02N7P1poWPoqe4jdJSPogCTRIDo6akGgIFBCIgBSrl16fapySm6rCnKVSRhVrUoGcWYGdMTgNFWYA9ZaMiIKKLX9JqtHmmqcxsv2sLI+thiFo3HgEgrKkTvu89E4p4IiA9Z3j8PnBLHuBw4PQ5pAJz+B+D/Iprn9RrTnVxHQV9ySIgyeescdhqphOtM/Hz0DDKGcc0N3+S39MIc0rY285uXnQiIOlgqrTFZbt0mmx1QG3X7T34DEX2DaFTmYQ7mSJEGdNdKiAAra6ifbPG89BOXPCiAp8D4zmPCFOV7+/vFx/GSvgXQSogBpuIV/6rgYT7aH9jRhjANdrEOdwa6ztVmKDwAAAAAAEqIAbFwc5jjSVn0mRosF65UdGijcxuEjSCtcZ1FJdw5ivyKiAG3fbh7nWP3hhCXbzkbM3athr8TYO5DSf+vfko2KGL/CogOoZeae4PVIDKvPZjV+TcLxjVjUXB6nSJ+zcj2Xk8cqYqIKzxNusB/BxOiD0jyLWESrWaN/Zq3VfF6aw7U+BZ01xkKiBBV7BYDzHF/ORKYlgtvPnXjudZQ6CEo5OzUDaNIomTCCogS9lJxDYCwz8gd5DtFqNSTKG5l1zxIaKpDP/sffi2is0wgAc4oAcQ4sC51gEqCwjJyM/VBhCPt8UI"
}
@@ -0,0 +1,8 @@
{
"_comment": "One pump.fun create_v2 as delivered by geyser SubscribeDeshred (pre-execution). There is no TransactionStatusMeta on this stream, so there are no log messages and no CreateEvent: the create instruction is the only source.",
"mint": "CJPAzeEsYpjNV4Jqi22f1GR1duRzCxbDfmLSWpFQpump",
"is_holder_reward": false,
"create_v2_trailing_arg_bytes": 10,
"loaded_lookup_table_accounts": 17,
"subscribe_update_base64": "CgFmEqwOCqMOCkBXhKZjuSxElbDEc0cKLM8NrF7kOa1ubEbTWFtEMretZTrP31R2EXHdG4pD3QXqeTuS/Xw9BJ/nFgM4mpAY0E4JGpYJCkBXhKZjuSxElbDEc0cKLM8NrF7kOa1ubEbTWFtEMretZTrP31R2EXHdG4pD3QXqeTuS/Xw9BJ/nFgM4mpAY0E4JCkBkE295NBLbtDGXowQs9kdg0tE52QfyA7Sm1W5u1Z+jOIkXpvxC6CRgu2pZPdtOysiJ3VSjWtpAllZTZahR5hQAEo8ICgQIAhgFEiBh7uw65deFVC7QAEBy/+n/sIS57XMPDSj+76oOTqLTgRIgp+SAEISNTfW4TGOVTUVhJJJOU0sAEX0HJrMHaYvymz8SICVALN+0ygQq9BDL7yqIch2i026ZyOT/EVm3sSaSyGnREiAu7nuJKk74pEqr+XDMDjfq9yZ2dRcMHWmc0xJ0SWxUPxIgTfAzVJLibHvMgUlbg7czTnb5RzhDuM1pC5ZRc3NCxRgSIHk13yLqlNyRXjPsH81mRUFmLmqiRvL5r7Hqob0dUvDfEiB6xDJG1mNXg8eblQqpZ1HuDZhbYFstucF+5qWC7dBI4hIgi0Y7dbssC2mnDKDjnEQOs3YfuFH9CltdVdCEObEg+UESIJCvMvE6lBalbKfWXujtoNHSf8/HPrWqLqe8fAnrmph9EiCbyyx/uaVGJA6595kStd1vIxNcCRcjL821y3aPwHSBfhIgrWUIqcg27H6UZT9FcJqP7O0jbf1/qThqLHQ9hpM1zN4SIMkhnahQUX0E+zsjH/l2Uv0lyPVEZXFuOcIqEyz2LVWqEiD2Al9NNfPlMYLaKooDFWRYUq1orYQNLLIu1NEl2b51lxIg/ln71TN/H2X/sSk4IaKgfnllE3gRwX0TSsTmgTta/bQSIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEiABVuD2k2Zaz0TbFWi/F1uqUYnLl/XS/ztlXSu2/W0YsBIgAwZGb+UhFzL/7K26csOb57yM5bvF9xJrLEObOkAAAAASIIyXJY9OJInxuz0QKRSODYMLWhOZ2v8QhASOe9jb6fhZEiDT4fmVLTvYo/j/PzMwZ40u1XG0D3/2eM/nLWoHf2vWVxog4i4R8FX1nm6efwrgpJE3A6+xRJ+jEwzqQevxewd2X3UiDAgQEgEeGgUCIKEHACINCBAaCQMQJwAAAAAAACKmAQgPEhABGwUNIAAOHRETGRQGByIPGo8B1pBM7F+LMbQJAAAAdGFjIG5heW4gBAAAAE5heW5DAAAAaHR0cHM6Ly9pcGZzLmlvL2lwZnMvUW1lQ2c5alRwTVkyR25Yc2d6ZDZnekpOWFVGMjVBelRaN1FpeDlZU2VkVEFyeWHu7Drl14VULtAAQHL/6f+whLntcw8NKP7vqg5OotOBAAAAAAAAAAAAAAAiDQgREgYAAgABDh0aAQEiOQgPEhsgARodHBEXFhgVBQ0KAAIMCwgSIwkEIR8OIg8aGLgX7mFnxdM9cEFSLpEJAAD//////////yIUCA4SAgADGgwCAAAAECcAAAAAAAAoATI3CiD8Qp4gXoenQOKszYWtvBGwxRGBR1TLTn7zyMKm5cKiShIGEBNVPRoiGgsSfJ4MC6APAQ4DBCIgBSrl16fapySm6rCnKVSRhVrUoGcWYGdMTgNFWYA9ZaMiIKKLX9JqtHmmqcxsv2sLI+thiFo3HgEgrKkTvu89E4p4IiClxpp/KZBRuALoEUgWKRdBf1lz5ugSm1m5QPxmnJz0QSIgp1eKehRYM7MDTzEN0iZy79qBPI9y4VG2s8SKZQpIIOIiIK0R5qT8KUSk+oJRvvgVQm4b+yjGtmRmd2B8atn1ZqZGIiDFS5a1yTGUHkbqS+Lg4xEndE/Gt0z7RV7+r4vVcXks7SogAK2uon2zxvPQTlzwogKfA+M5jwhTle/v7xcfxkr4F0EqIAabiFf+q4GE+2h/Y0YYwDXaxDncGus7VZig8AAAAAABKiAGxcHOY40lZ9JkaLBeuVHRoo3MbhI0grXGdRSXcOYr8iogBt324ddloZPZy+FGzut5rBy0he1fWzeROoz1hX7/AKkqIAbd9uHudY/eGEJdvORszdq2GvxNg7kNJ/69+SjYoYv8KiAK8cNDIYjKOmNRNaE6GJUazr0p5qwtrqYL8lyHXEAL8yogDDX/qQVajlaNqPe8B1YVJ0zxySykH0AAnFFqpBTCfHAqIDqGXmnuD1SAyrz2Y1fk3C8Y1Y1Fwep0ifs3I9l5PHKmKiBvmrSk8ZWNwKnJTD+3LAeZWEPtpIXjok8QxpOZ+BmUDyogrPE26wH8HE6IPSPItYRKtZo39mrdV8XprDtT4FnTXGQqIPoJEaVIY0EtYx9OB4cDKWwDXw0TM6DZyIONc7cQ/m4tMKAGOOAGENPAudYBKgwIxMjP1QYQq+zO2AM="
}
@@ -61,30 +61,26 @@ client-side `MIN_CURVE_LEN` floor discards anything shorter than the
smallest real struct, so a stray short/legacy account still can't reach the
decoder.
**Bandwidth trade-off, measured 2026-09-15 directly over Geyser** (not
borrowed from the WebSocket script's measurement — Geyser reports slot and
signature per update and its named-group model batches differently, so it
gets its own number). Two *separate* concurrent gRPC streams over the same
120s window, so both see the identical trade activity without the
sequential-window volume-swing problem (a first attempt comparing sequential
windows was unreliable for exactly that reason — see
`pumpfun_watch_graduating_programsubscribe.py`'s module docstring): one stream subscribed with
this script's old two named groups (`datasize` 125 and 151), the other with
a single unfiltered group. The filtered stream took in 6,754 updates /
2,340,863 proto bytes; the unfiltered stream took in 6,761 updates /
2,337,304 proto bytes — a 1.001x update ratio and a 0.998x byte ratio,
i.e. no measurable cost. The 7-update difference was the 256-byte curve
(`EJpNsfxnTB6mtVdzrTcgQ9xfywobHSSsUtu1Gh1GFvEg`) that no enumerated length
could ever match; the filtered stream structurally cannot see it at all.
`getAccountInfo` on that same curve, run directly the same day, confirms it:
still 256 bytes, discriminator intact, and it decodes cleanly through this
repo's own IDL-driven decoder — the bytes past the documented fields are
zero padding.
**Bandwidth trade-off, checked directly over Geyser** (not borrowed from the
WebSocket script's comparison — Geyser reports slot and signature per update
and its named-group model batches differently, so it gets its own). Two
*separate* concurrent gRPC streams over the same window, so both see the
identical trade activity without the sequential-window volume-swing problem
(a first attempt comparing sequential windows was unreliable for exactly that
reason — see `pumpfun_watch_graduating_programsubscribe.py`'s module
docstring): one stream subscribed with this script's old two named groups
(`datasize` 125 and 151), the other with a single unfiltered group. They took
in effectively the same volume, i.e. filtering bought nothing. The difference
was the 256-byte curve (`EJpNsfxnTB6mtVdzrTcgQ9xfywobHSSsUtu1Gh1GFvEg`) that
no enumerated length could ever match; the filtered stream structurally
cannot see it at all. `getAccountInfo` on that same curve confirms it: still
256 bytes, discriminator intact, and it decodes cleanly through this repo's
own IDL-driven decoder — the bytes past the documented fields are zero
padding.
UNVERIFIED here: whether the original 49-byte layout (no `creator` field) is
still written anywhere. None of the 6,761 unfiltered updates in the
measurement above were that length, but that is one 120s window, not proof
of absence.
still written anywhere. Nothing in the unfiltered stream was that length, but
that is one window, not proof of absence.
"""
import argparse
@@ -69,32 +69,26 @@ elsewhere) returned 256 bytes, discriminator matching, owned by the pump
program, and it decoded cleanly through this repo's own IDL-driven decoder
(`PumpFunCurveManager._decode_curve_state_with_idl`) with sane reserves —
everything past the documented fields is zero padding. Second, over the
wire: a 120s `programSubscribe` window with this script's own filters
(discriminator + `complete = false`, no `dataSize`) took in 1,066 updates —
207 at 125 bytes, 853 at 151, and 6 at that same 256 length, among ordinary
traffic. None of the 1,066 were the legacy 49-byte layout (no `creator`
field). UNVERIFIED: whether that layout still has any `complete = false`
accounts left on chain, and whether anything still writes to them — not
re-measured here.
wire: a `programSubscribe` window with this script's own filters
(discriminator + `complete = false`, no `dataSize`) saw all three lengths —
125, 151 and that same 256 — among ordinary traffic, the 256 being rare.
Nothing in it was the legacy 49-byte layout (no `creator` field).
UNVERIFIED: whether that layout still has any `complete = false` accounts
left on chain, and whether anything still writes to them — not re-measured
here.
**Bandwidth trade-off, measured 2026-09-15.** A first attempt ran the
`dataSize`-filtered shape and the unfiltered shape back to back, 90s each,
and looked like unfiltered cost *less* (0.68x) — that was noise: pump.fun
trading volume swings a lot minute to minute, and two sequential windows just
land on different volume. Rerun with all three filter groups (`dataSize
125`, `dataSize 151`, and no `dataSize`) subscribed **simultaneously on one
connection**, so all three watch the identical trade stream over the same
120s: the two enumerated lengths together took in 1,060 updates / 589,279
bytes; the unfiltered subscription took in 1,066 updates / 593,509 bytes —
6 extra updates, 4,230 extra bytes, all of it the 256-byte curve neither
enumerated length can match. That is a **1.006x update ratio / 1.007x byte
ratio** — under 1% either way, not the double subscription's worth intuition
might suggest, because in this trade window virtually every update already
lands on one of the two dataSize-filtered lengths (125 or 151), and 256 is
rare. Dropping the filter is effectively free here; if a resize-happy period
ever shifts that mix, the cost scales with however much traffic sits outside
the two filtered lengths (125/151, i.e. the rarer 256-byte curves and
beyond), not with total volume.
**Bandwidth trade-off.** A first attempt ran the `dataSize`-filtered shape
and the unfiltered shape back to back and made unfiltered look *cheaper* —
that was noise: pump.fun trading volume swings a lot minute to minute, and
two sequential windows just land on different volume. Rerun with all three
filter groups (`dataSize 125`, `dataSize 151`, and no `dataSize`) subscribed
**simultaneously on one connection**, so all three watch the identical trade
stream: the unfiltered subscription cost barely more than the two enumerated
lengths together, and the difference was entirely the 256-byte curve neither
enumerated length can match. Dropping the filter is effectively free here,
because virtually every update already lands on 125 or 151 and 256 is rare;
if a resize-happy period ever shifts that mix, the cost scales with however
much traffic sits outside the two filtered lengths, not with total volume.
UNVERIFIED: a curve was once observed going from 125 to 151 bytes, with
several 125-byte trades logged in between, suggesting `extend_account` ran as
@@ -9,22 +9,19 @@ executed stream. The two are different RPCs on the same endpoint and the trade-o
between them is the point of this example.
`SubscribeDeshred` delivers a transaction when entries are formed from shreds,
BEFORE any execution. Measured head-to-head against `Subscribe` on the pump.fun
program, 300s on 2026-09-23: identical signature sets, no orphans either way.
On token creations -- the number that matters here -- deshred arrived first in
92% of cases, median +6.4ms, p75 +10.1ms, p90 +21.5ms. General traffic does
worse (79% first, median +4.0ms), and the far tail is volatile between runs.
`tools/compare_deshred_latency.py` reproduces all of this.
BEFORE any execution. Raced head-to-head against `Subscribe` on the pump.fun
program it carries the same signatures with no orphans either way, and on token
creations -- the case that matters here -- it usually arrives first, by a small
margin. `tools/compare_deshred_latency.py` reproduces that comparison.
What you give up for those milliseconds:
- **It cannot see a coin created through a router, at all.** The create then
reaches the program as a CPI, and inner instructions are produced *by*
execution, so a pre-execution stream never carries them: the transaction
arrives on deshred with the create invisible. Measured 4 of 118 creates
(3.4%) in one 300s window and 3 of 101 in another -- every one of them
present on the stream and undetectable, not dropped. No decoding recovers
them, and this script misses them too. The executed listeners walk
arrives on deshred with the create invisible -- present on the stream and
undetectable, not dropped. No decoding recovers them, and this script
misses them too. The executed listeners walk
`meta.innerInstructions` and do not have this blind spot.
- No TransactionStatusMeta, so no meta.log_messages, so no CreateEvent. This
script therefore decodes the create instruction, which is the fallback route
@@ -8,14 +8,11 @@ Usage:
The mirror of `pumpfun_buy_token_v2.py`. It reads how many tokens you hold, reads the curve
for a price, and sells the lot with a slippage floor underneath.
Two things to know before you run it:
One thing to know before you run it:
- **The floor is yours to set.** `min_sol_output` is computed here from the price
and the slippage, and the program enforces only that number. Passing a floor
derived from a stale price is how a sell goes through at a price you did not intend.
- **A dust position may not be sellable at all.** Below a few thousand tokens the
output rounds to zero and the program rejects the sell with error 6003
(TooLittleSolReceived), whatever slippage you allow.
"""
import argparse