feat(geyser): upgrade to yellowstone proto 13.0.0-rc4 and read the transaction v1 config (#219)

The vendored geyser protos predated transaction v1 (SIMD-0385, live since
2026-09-15). Message.config -- the inline compute budget that v1 moved off the
ComputeBudget instructions -- is field 7, which the old stubs had no definition
for. Protobuf skips an unknown field rather than failing, so the gap was
silent: frames still decoded and only the budget went missing.

Pin to yellowstone v16.0.0-rc10+solana.4.3.0 (commit 301e9cf, proto
13.0.0-rc4), the build the endpoint itself reports from GetVersion. Not master:
master has diverged and is missing block_footer, bank_id and VATDebit.
Regenerate the stubs (gencode 6.31.1 -> 7.35.1) and rewrite the emitted
relative imports to the absolute form this repo requires.

Fields adopted: Message.config and TransactionStatusMeta.cost_units. Over
geyser there is no version number on the wire and `versioned` is true for v0
and v1 alike, so config presence is the version test; the listener and the
geyser example both report it. CommitmentLevel lost members 3-6 to a new
SlotStatus enum -- the one breaking change, and nothing here sent anything but
PROCESSED.

Adds a cookbook example for SubscribeDeshred, the pre-execution stream, and
tools/compare_deshred_latency.py to measure it. Deshred is deliberately not
wired into the bot: it carries no TransactionStatusMeta, so no CreateEvent and
no state_from_event, and a router-created coin is structurally invisible on it
because the create is an inner instruction and inner instructions are produced
by execution. Measured 4 of 118 creates missed in a 300s window, against a
6.4ms median lead on the creates it does see.

verify_transaction_v1.py gains four geyser checks on a committed v1
SubscribeUpdate, pinning the log route and the instruction fallback separately,
as the block listener already does.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

cookbook/README.md

@@ -90,12 +90,15 @@ Not specific to any of the launchpads below.
 |---|---|
 | [`pumpfun/listen/pumpfun_listen_tokens_logsubscribe.py`](pumpfun/listen/pumpfun_listen_tokens_logsubscribe.py) | New coins over `logsSubscribe` — works on every provider |
 | [`pumpfun/listen/pumpfun_listen_tokens_blocksubscribe.py`](pumpfun/listen/pumpfun_listen_tokens_blocksubscribe.py) | New coins over `blockSubscribe` — whole blocks, slower, not on every provider |
-| [`pumpfun/listen/pumpfun_listen_tokens_geyser.py`](pumpfun/listen/pumpfun_listen_tokens_geyser.py) | New coins over Geyser gRPC — the fastest of the four |
+| [`pumpfun/listen/pumpfun_listen_tokens_geyser.py`](pumpfun/listen/pumpfun_listen_tokens_geyser.py) | New coins over Geyser gRPC — the fastest of the four executed-stream listeners |
+| [`pumpfun/listen/pumpfun_listen_tokens_deshred.py`](pumpfun/listen/pumpfun_listen_tokens_deshred.py) | New coins over Geyser `SubscribeDeshred` — ~6ms earlier, but pre-execution: no logs, no CreateEvent, no outcome, and router-created coins are invisible |
 | [`pumpfun/listen/pumpfun_listen_tokens_pumpportal.py`](pumpfun/listen/pumpfun_listen_tokens_pumpportal.py) | New coins from PumpPortal's feed — third party, misses some coins |
 | [`pumpfun/listen/pumpfun_listen_wallet_trades.py`](pumpfun/listen/pumpfun_listen_wallet_trades.py) | One wallet's bonding-curve buys and sells — copy trading |
 | [`pumpfun/listen/pumpfun_capture_transactions_blocksubscribe.py`](pumpfun/listen/pumpfun_capture_transactions_blocksubscribe.py) | Save live transactions to disk, to build a fixture |
 
 Racing the four listeners against each other is `tools/compare_listeners.py`.
+Racing the deshred stream against the executed one — and sizing its blind spot —
+is `tools/compare_deshred_latency.py`.
 
 ### Read state
 

cookbook/pumpfun/listen/pumpfun_listen_tokens_deshred.py

@@ -0,0 +1,305 @@
+"""Monitors Solana for new Pump.fun token creations from the pre-execution deshred stream.
+Decodes 'create' instructions the moment entries are formed from shreds, before the transaction runs.
+
+Usage:
+    uv run cookbook/pumpfun/listen/pumpfun_listen_tokens_deshred.py
+
+This is the companion to pumpfun_listen_tokens_geyser.py, which reads the ordinary
+executed stream. The two are different RPCs on the same endpoint and the trade-off
+between them is the point of this example.
+
+`SubscribeDeshred` delivers a transaction when entries are formed from shreds,
+BEFORE any execution. Measured head-to-head against `Subscribe` on the pump.fun
+program, 300s on 2026-09-23: identical signature sets, no orphans either way.
+On token creations -- the number that matters here -- deshred arrived first in
+92% of cases, median +6.4ms, p75 +10.1ms, p90 +21.5ms. General traffic does
+worse (79% first, median +4.0ms), and the far tail is volatile between runs.
+`tools/compare_deshred_latency.py` reproduces all of this.
+
+What you give up for those milliseconds:
+
+  - **It cannot see a coin created through a router, at all.** The create then
+    reaches the program as a CPI, and inner instructions are produced *by*
+    execution, so a pre-execution stream never carries them: the transaction
+    arrives on deshred with the create invisible. Measured 4 of 118 creates
+    (3.4%) in one 300s window and 3 of 101 in another -- every one of them
+    present on the stream and undetectable, not dropped. No decoding recovers
+    them, and this script misses them too. The executed listeners walk
+    `meta.innerInstructions` and do not have this blind spot.
+  - No TransactionStatusMeta, so no meta.log_messages, so no CreateEvent. This
+    script therefore decodes the create instruction, which is the fallback route
+    everywhere else in this repo. The creator it prints is the instruction's
+    `args.creator`, which is user-supplied and since 2026-04-28 may differ from
+    the canonical `BondingCurve.creator`. Read the curve if you need the real one.
+  - No success or failure, by construction -- nothing has executed yet. The
+    executed stream filters with `failed = False`; here you see every shredded
+    transaction, landed or reverted.
+
+So this is a latency demo, not a better listener: a few milliseconds earlier on
+most coins, in exchange for never seeing a few percent of them.
+
+Address lookup tables are already resolved, reported as loaded_writable_addresses
+then loaded_readonly_addresses (that order), so instruction account indices
+resolve the same way they do on the executed stream.
+
+Geyser gRPC Reference:
+https://docs.triton.one/project-yellowstone/dragons-mouth-grpc-subscriptions
+
+Authentication: Supports both Basic and X-Token authentication methods.
+Configure via GEYSER_ENDPOINT, GEYSER_API_TOKEN, and AUTH_TYPE variables.
+"""
+
+import asyncio
+import os
+import struct
+import sys
+from pathlib import Path
+
+import base58
+import grpc
+from dotenv import load_dotenv
+from solders.pubkey import Pubkey
+
+# The geyser stubs are generated once, into src/geyser/generated. Reuse them rather
+# than keeping a second copy here that drifts out of sync with proto/.
+sys.path.insert(0, str(Path(__file__).resolve().parents[3]))
+from src.geyser.generated import geyser_pb2, geyser_pb2_grpc
+
+load_dotenv()
+
+
+GEYSER_ENDPOINT = os.getenv("GEYSER_ENDPOINT")
+GEYSER_API_TOKEN = os.getenv("GEYSER_API_TOKEN")
+# Authentication type: "x-token" or "basic"
+AUTH_TYPE = "x-token"
+
+PUMP_PROGRAM_ID = Pubkey.from_string("6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P")
+
+# Instruction discriminators (8-byte identifiers for instruction types)
+# Calculated using the first 8 bytes of sha256("global:create") for legacy Create
+# and sha256("global:createV2") for Token2022 CreateV2
+# Run cookbook/solana/anchor_calculate_discriminator.py to recompute them.
+PUMP_CREATE_PREFIX = struct.pack("<Q", 8576854823835016728)
+PUMP_CREATE_V2_PREFIX = bytes([214, 144, 76, 236, 95, 139, 49, 180])
+
+
+def print_token_info(token_data: dict, signature: str, slot: int) -> None:
+    """Print token information in a consistent, user-friendly format.
+
+    Args:
+        token_data: Dictionary containing token fields
+        signature: Transaction signature
+        slot: Slot the entry was shredded in
+    """
+    print("\n" + "=" * 80)
+    print("🎯 NEW TOKEN DETECTED (pre-execution)")
+    print("=" * 80)
+    print(f"Name:             {token_data.get('name', 'N/A')}")
+    print(f"Symbol:           {token_data.get('symbol', 'N/A')}")
+    print(f"Mint:             {token_data.get('mint', 'N/A')}")
+    print(f"Bonding Curve:    {token_data.get('bonding_curve', 'N/A')}")
+    print(f"Associated BC:    {token_data.get('associated_bonding_curve', 'N/A')}")
+    print(f"User:             {token_data.get('user', 'N/A')}")
+    # From the instruction args, not the curve: may differ from BondingCurve.creator.
+    print(f"Creator (args):   {token_data.get('creator', 'N/A')}")
+    print(f"Token Standard:   {token_data.get('token_standard', 'N/A')}")
+    print(f"Mayhem Mode:      {token_data.get('is_mayhem_mode', False)}")
+    print(f"URI:              {token_data.get('uri', 'N/A')}")
+    print(f"Tx version:       {token_data.get('tx_version', 'N/A')}")
+    print(f"Slot:             {slot}")
+    print(f"Signature:        {signature}")
+    print("Outcome:          unknown - nothing has executed yet")
+    print("=" * 80 + "\n")
+
+
+async def create_geyser_connection() -> geyser_pb2_grpc.GeyserStub:
+    """Establish a secure connection to the Geyser endpoint using the configured auth type.
+
+    Returns:
+        A Geyser stub bound to an authenticated channel
+    """
+    if AUTH_TYPE == "x-token":
+        auth = grpc.metadata_call_credentials(
+            lambda _, callback: callback((("x-token", GEYSER_API_TOKEN),), None)
+        )
+    else:  # Default to basic auth
+        auth = grpc.metadata_call_credentials(
+            lambda _, callback: callback(
+                (("authorization", f"Basic {GEYSER_API_TOKEN}"),), None
+            )
+        )
+
+    creds = grpc.composite_channel_credentials(grpc.ssl_channel_credentials(), auth)
+    channel = grpc.aio.secure_channel(GEYSER_ENDPOINT, creds)
+    return geyser_pb2_grpc.GeyserStub(channel)
+
+
+def create_subscription_request() -> geyser_pb2.SubscribeDeshredRequest:
+    """Create a deshred subscription request for Pump.fun transactions.
+
+    This is its own request type, not a SubscribeRequest: the deshred stream has
+    no commitment level, because commitment describes execution and nothing has
+    executed. There is also no `failed` filter for the same reason.
+
+    Returns:
+        The request to open the stream with
+    """
+    request = geyser_pb2.SubscribeDeshredRequest()
+    request.deshred_transactions["pump_filter"].account_include.append(
+        str(PUMP_PROGRAM_ID)
+    )
+    request.deshred_transactions["pump_filter"].vote = False
+    return request
+
+
+def resolve_account_keys(
+    tx: geyser_pb2.SubscribeUpdateDeshredTransactionInfo,
+) -> list[bytes]:
+    """Build the full account-key table for one deshred transaction.
+
+    A v0 transaction indexes accounts past the end of `message.account_keys` when
+    it uses an address lookup table. The deshred stream resolves those for you and
+    reports them on the update itself rather than under a meta, in this exact
+    order: static keys, then writable loaded, then read-only loaded.
+
+    Args:
+        tx: A geyser `SubscribeUpdateDeshredTransactionInfo`
+
+    Returns:
+        Account keys as raw 32-byte values, indexable by an instruction's account list
+    """
+    keys = list(tx.transaction.message.account_keys)
+    keys.extend(tx.loaded_writable_addresses)
+    keys.extend(tx.loaded_readonly_addresses)
+    return keys
+
+
+def decode_create_instruction(
+    ix_data: bytes, keys: list[bytes], accounts: bytes, *, is_v2: bool
+) -> dict:
+    """Decode a create or create_v2 instruction from its raw data.
+
+    Args:
+        ix_data: Instruction data, starting with the 8-byte discriminator
+        keys: Full account-key table from `resolve_account_keys`
+        accounts: The instruction's account indices
+        is_v2: True for `create_v2` (Token2022), False for legacy `create`
+
+    Returns:
+        Dictionary of decoded token fields
+    """
+    offset = 8  # Skip past the 8-byte discriminator prefix
+
+    def get_account_key(index: int) -> str:
+        if index >= len(accounts):
+            return "N/A"
+        account_index = accounts[index]
+        if account_index >= len(keys):
+            return "N/A"
+        return base58.b58encode(keys[account_index]).decode()
+
+    def read_string() -> str:
+        nonlocal offset
+        length = struct.unpack_from("<I", ix_data, offset)[0]
+        offset += 4
+        value = ix_data[offset : offset + length].decode()
+        offset += length
+        return value
+
+    def read_pubkey() -> str:
+        nonlocal offset
+        value = base58.b58encode(ix_data[offset : offset + 32]).decode("utf-8")
+        offset += 32
+        return value
+
+    name = read_string()
+    symbol = read_string()
+    uri = read_string()
+    creator = read_pubkey()
+
+    if not is_v2:
+        return {
+            "name": name,
+            "symbol": symbol,
+            "uri": uri,
+            "creator": creator,
+            "mint": get_account_key(0),
+            "bonding_curve": get_account_key(2),
+            "associated_bonding_curve": get_account_key(3),
+            "user": get_account_key(7),
+            "token_standard": "legacy",
+            "is_mayhem_mode": False,
+        }
+
+    # create_v2 trailing args are positional and may be truncated on the wire:
+    # read defensively and report a missing one as unset rather than raising.
+    is_mayhem_mode = False
+    if offset < len(ix_data):
+        is_mayhem_mode = bool(ix_data[offset])
+        offset += 1
+
+    return {
+        "name": name,
+        "symbol": symbol,
+        "uri": uri,
+        "creator": creator,
+        "mint": get_account_key(0),
+        "bonding_curve": get_account_key(2),
+        "associated_bonding_curve": get_account_key(3),
+        "user": get_account_key(5),
+        "token_standard": "token2022",
+        "is_mayhem_mode": is_mayhem_mode,
+    }
+
+
+def describe_version(message: geyser_pb2.Message) -> str:
+    """Name the transaction format a message is in.
+
+    `config` is set only for transaction v1 (SIMD-0385), so its presence is what
+    separates v1 from v0 -- the `versioned` flag is true for both.
+
+    Args:
+        message: The transaction's message
+
+    Returns:
+        "v1", "v0" or "legacy"
+    """
+    if message.HasField("config"):
+        return "v1"
+    return "v0" if message.versioned else "legacy"
+
+
+async def monitor_pump() -> None:
+    """Monitor the deshred stream for new Pump.fun token creations."""
+    print(f"Starting Pump.fun deshred monitor using {AUTH_TYPE.upper()} authentication")
+    print("Reading pre-execution transactions: outcomes are not yet known.\n")
+    stub = await create_geyser_connection()
+    request = create_subscription_request()
+
+    async for update in stub.SubscribeDeshred(iter([request])):
+        if not update.HasField("deshred_transaction"):
+            continue
+
+        tx = update.deshred_transaction.transaction
+        msg = getattr(tx.transaction, "message", None)
+        if msg is None:
+            continue
+
+        keys = resolve_account_keys(tx)
+
+        for ix in msg.instructions:
+            is_create = ix.data.startswith(PUMP_CREATE_PREFIX)
+            is_create_v2 = ix.data.startswith(PUMP_CREATE_V2_PREFIX)
+            if not (is_create or is_create_v2):
+                continue
+
+            info = decode_create_instruction(
+                ix.data, keys, ix.accounts, is_v2=is_create_v2
+            )
+            info["tx_version"] = describe_version(msg)
+            signature = base58.b58encode(bytes(tx.signature)).decode()
+            print_token_info(info, signature, update.deshred_transaction.slot)
+
+
+if __name__ == "__main__":
+    asyncio.run(monitor_pump())

cookbook/pumpfun/listen/pumpfun_listen_tokens_geyser.py

@@ -1,5 +1,6 @@
 """Monitors Solana for new Pump.fun token creations using Geyser gRPC.
-Decodes 'create' instructions to extract and display token details (name, symbol, mint, bonding curve).
+Decodes 'create' instructions to extract and display token details (name, symbol, mint, bonding curve),
+and reports which transaction format each coin was created in, with its inline budget for v1.
 
 Usage:
     uv run cookbook/pumpfun/listen/pumpfun_listen_tokens_geyser.py
@@ -51,13 +52,14 @@ PUMP_CREATE_PREFIX = struct.pack("<Q", 8576854823835016728)
 PUMP_CREATE_V2_PREFIX = bytes([214, 144, 76, 236, 95, 139, 49, 180])
 
 
-def print_token_info(token_data, signature=None):
+def print_token_info(token_data, signature=None, envelope: dict | None = None):
     """
     Print token information in a consistent, user-friendly format.
 
     Args:
         token_data: Dictionary containing token fields
         signature: Optional transaction signature
+        envelope: Optional envelope summary from `describe_envelope`
     """
     print("\n" + "=" * 80)
     print("🎯 NEW TOKEN DETECTED")
@@ -82,6 +84,12 @@ def print_token_info(token_data, signature=None):
         print(f"URI:              {token_data['uri']}")
     if signature:
         print(f"Signature:        {signature}")
+    if envelope:
+        print(f"Tx version:       {envelope['version']}")
+        if envelope["config"]:
+            print(f"Tx config:        {envelope['config']}")
+        if envelope["cost_units"] is not None:
+            print(f"Cost units:       {envelope['cost_units']}")
 
     print("=" * 80 + "\n")
 
@@ -138,6 +146,55 @@ def resolve_account_keys(
     return keys
 
 
+def describe_envelope(tx: geyser_pb2.SubscribeUpdateTransactionInfo) -> dict:
+    """Summarise which transaction format a coin was created in, and its budget.
+
+    Transaction v1 (SIMD-0385, live since 2026-09-15) carries its compute budget
+    inline on the message as `config`, instead of as separate ComputeBudget
+    instructions. Geyser sets that field only for v1, so its presence is how you
+    tell a v1 transaction from a legacy or v0 one here: the `versioned` flag is
+    true for both v0 and v1 and cannot separate them.
+
+    Args:
+        tx: A geyser `SubscribeUpdateTransactionInfo`
+
+    Returns:
+        Dict with the detected `version`, a readable `config` string (empty for
+        legacy/v0) and `cost_units` when the validator reported it
+    """
+    msg = tx.transaction.message
+    is_v1 = msg.HasField("config")
+
+    config = ""
+    if is_v1:
+        cfg = msg.config
+        parts = []
+        # Every field is optional, and an unset one means zero rather than a
+        # default: a v1 transaction that omits the compute unit limit budgets
+        # 0 CU and dies at account loading.
+        if cfg.HasField("priority_fee"):
+            # v1 states a total in lamports, not micro-lamports per CU as v0 does.
+            parts.append(f"priority_fee={cfg.priority_fee} lamports")
+        if cfg.HasField("compute_unit_limit"):
+            parts.append(f"cu_limit={cfg.compute_unit_limit}")
+        if cfg.HasField("loaded_accounts_data_size_limit"):
+            parts.append(f"data_size={cfg.loaded_accounts_data_size_limit}")
+        if cfg.HasField("heap_size"):
+            parts.append(f"heap={cfg.heap_size}")
+        config = ", ".join(parts) or "all fields unset"
+
+    meta = getattr(tx, "meta", None)
+    cost_units = None
+    if meta is not None and meta.HasField("cost_units"):
+        cost_units = meta.cost_units
+
+    return {
+        "version": "v1" if is_v1 else ("v0" if msg.versioned else "legacy"),
+        "config": config,
+        "cost_units": cost_units,
+    }
+
+
 def decode_create_instruction(ix_data: bytes, keys, accounts) -> dict:
     """Decode a legacy create instruction (Metaplex) from transaction data."""
     # Skip past the 8-byte discriminator prefix
@@ -303,7 +360,11 @@ async def monitor_pump():
             ).decode()
 
             # Print token information in consistent format
-            print_token_info(info, signature=signature)
+            print_token_info(
+                info,
+                signature=signature,
+                envelope=describe_envelope(update.transaction.transaction),
+            )
 
 
 if __name__ == "__main__":