Compare commits

..
Author SHA1 Message Date
pselamyandClaude Opus 4.6 b1b88a3dd0 fix: lint/format fixes for eth_getLogs chunking
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-14 19:14:16 +00:00
schrodinger01andpselamy 8ebcd6b4c6 fix(profiler): 0x-prefix the Transfer event topic for strict RPC providers
`HexBytes.hex()` returns a bare hex string with no `0x` prefix. publicnode
tolerates that, but drpc — which we use as the failover RPC — rejects it
outright with `invalid argument 0: hex string without 0x prefix`, and every
single eth_getLogs chunk in the funding trace fails. Once the primary is
flipped to unhealthy by any other call, the entire funding subsystem
silently produces zero rows in funding_transfers.

Switch to a precomputed `TRANSFER_EVENT_TOPIC` constant that always carries
the `0x` prefix, and add a regression test that asserts the topic shape
sent to eth_getLogs.
2026-06-14 19:13:14 +00:00
schrodinger01andpselamy ff1e23a0d3 fix(profiler): cap default lookback at 80k blocks + early-break on pruned
Field-test of the chunking fix on a public Polygon RPC (publicnode)
revealed a second wall behind the first: after a chunk request lands
outside the provider's archive horizon, every subsequent chunk fails
with the same error:

  {'code': -32701, 'message': 'History has been pruned for this block.
   To remove restrictions, order a dedicated full node here: ...'}

publicnode empirically retains roughly the most recent 100_000 blocks
(~55 hours) of log history. Surveying other public free-tier RPCs:

  drpc.org      — archive, but rejects ranges >= ~1_000 blocks
  llamarpc      — empty responses on archive ranges
  ankr          — now requires API key
  blockpi/onfin — block-range limits 50–500
  1rpc.io/matic — limited to 50 blocks

Two changes to make funding traces actually return data on a public
RPC instead of swallowing 140 pruned-history warnings per wallet:

1. Lower DEFAULT_MAX_LOOKBACK_BLOCKS from 1_300_000 to 80_000. Fresh
   wallets — the population this signal exists to flag — are by
   definition new, so a ~44 hour window covers their entire funding
   history. Older wallets lose archive coverage on free RPCs but
   they're not what the fresh-wallet signal scores on anyway.

2. Detect pruned-history errors by message substring and short-circuit
   the chunk walk. Walking further back is futile once we're past the
   cutoff; bailing early avoids burning RPC quota on chunks that are
   guaranteed to fail.

Both knobs remain constructor parameters — deployments behind a paid
archive node can dial DEFAULT_MAX_LOOKBACK_BLOCKS back up.

Two new tests:
- test_get_transfer_logs_breaks_on_pruned_history: pruned error on
  chunk #2 must keep chunk #3 from ever being issued
- test_get_transfer_logs_default_lookback_fits_pruned_horizon:
  regression guard pinning the default at <= 100_000 so a future
  refactor doesn't silently re-introduce the unusable default
2026-06-14 19:13:14 +00:00
schrodinger01andpselamy 82f3e8eb6a fix(profiler): chunk eth_getLogs into <=10k-block windows
Public Polygon RPC providers (publicnode, ankr, llamarpc) cap eth_getLogs
at 10_000 blocks per request. The funding tracer was calling get_logs
with from_block=0 / to_block="latest", so every funding chain trace
failed in production with:

    {'code': -32701, 'message': 'exceed maximum block range: 10000'}

Resolve the symbolic range to concrete bounds (default lookback ~30 days
of Polygon blocks) and walk the window in 9_000-block chunks, oldest
first, stopping early once `limit` matches are gathered. Walking
oldest-first preserves the "first transfer" semantics the funding tracer
already relies on.

Includes 4 new tests:
- chunks_large_ranges: regression guard that no single window exceeds
  the cap
- stops_when_limit_hit_mid_walk: short-circuits once enough hits
- skips_failing_chunk: a flaky window doesn't tank the whole trace
- resolves_latest_via_block_number: from_block=0 + to_block="latest"
  resolves to the last max_lookback_blocks

The 3 pre-existing _get_transfer_logs tests now pass explicit numeric
ranges so they don't go through the latest-resolution path; coverage of
that path is moved to the new dedicated test.
2026-06-14 19:13:14 +00:00
7 changed files with 443 additions and 604 deletions
@@ -1,205 +0,0 @@
"""Gamma API client for Polymarket market volume / liquidity data.
The CLOB API does not expose 24h volume or liquidity. The public Gamma API
(https://gamma-api.polymarket.com) does, with no auth required. This module
fetches the volume/liquidity snapshot keyed by condition_id so the
size_anomaly detector can do real ratio math instead of falling back to
the niche-base 0.2 confidence floor.
"""
from __future__ import annotations
import asyncio
import logging
from dataclasses import dataclass
from decimal import Decimal, InvalidOperation
import httpx
logger = logging.getLogger(__name__)
DEFAULT_HOST = "https://gamma-api.polymarket.com"
DEFAULT_TIMEOUT_SECONDS = 15.0
# Gamma /markets enforces a server-side max of 100 per page even when a
# higher `limit` is sent. Using 100 lines our page size up with the actual
# response so pagination doesn't bail out after the first page.
DEFAULT_PAGE_LIMIT = 100
# Gamma also caps `offset` around 10000 for this collection. Combined with
# the 100/page limit that gives ~10k markets max, sequential — way too slow
# at default sync interval. We sort by 24h volume desc and only walk the
# top N pages, since markets with zero recent volume don't need a real
# ratio anyway (the niche path handles them).
DEFAULT_MAX_PAGES = 50 # 50 * 100 = 5000 most-traded markets per sync
DEFAULT_PAGE_CONCURRENCY = 5
DEFAULT_MAX_RETRIES = 3
DEFAULT_RETRY_BASE_DELAY_SECONDS = 1.0
@dataclass(frozen=True)
class GammaMarketStats:
"""Volume / liquidity snapshot for a single market from gamma-api."""
condition_id: str
daily_volume: Decimal | None
weekly_volume: Decimal | None
monthly_volume: Decimal | None
total_volume: Decimal | None
liquidity: Decimal | None
def _to_decimal(value: object) -> Decimal | None:
if value is None or value == "":
return None
try:
return Decimal(str(value))
except (InvalidOperation, ValueError, TypeError):
return None
def _parse_market(raw: dict[str, object]) -> GammaMarketStats | None:
cid = raw.get("conditionId")
if not cid or not isinstance(cid, str):
return None
return GammaMarketStats(
condition_id=cid,
daily_volume=_to_decimal(raw.get("volume24hr")),
weekly_volume=_to_decimal(raw.get("volume1wk")),
monthly_volume=_to_decimal(raw.get("volume1mo")),
total_volume=_to_decimal(raw.get("volumeNum") or raw.get("volume")),
liquidity=_to_decimal(raw.get("liquidityNum") or raw.get("liquidity")),
)
class GammaClientError(Exception):
"""Raised when gamma-api returns an unrecoverable error."""
class GammaClient:
"""Async client for the public gamma-api markets endpoint.
Provides batched, paginated reads of every active market with their
24h/weekly/monthly volume and current liquidity. Designed to be called
from MarketMetadataSync once per sync interval; results are merged into
Redis-cached MarketMetadata objects.
"""
def __init__(
self,
*,
host: str = DEFAULT_HOST,
timeout_seconds: float = DEFAULT_TIMEOUT_SECONDS,
page_limit: int = DEFAULT_PAGE_LIMIT,
max_pages: int = DEFAULT_MAX_PAGES,
page_concurrency: int = DEFAULT_PAGE_CONCURRENCY,
max_retries: int = DEFAULT_MAX_RETRIES,
retry_base_delay_seconds: float = DEFAULT_RETRY_BASE_DELAY_SECONDS,
) -> None:
self._host = host.rstrip("/")
self._timeout = timeout_seconds
self._page_limit = page_limit
self._max_pages = max_pages
self._page_concurrency = page_concurrency
self._max_retries = max_retries
self._retry_base = retry_base_delay_seconds
async def _get_with_retry(
self,
client: httpx.AsyncClient,
path: str,
params: dict[str, object],
) -> list[dict[str, object]]:
last_exc: Exception | None = None
delay = self._retry_base
for attempt in range(self._max_retries):
try:
resp = await client.get(path, params=params)
resp.raise_for_status()
payload = resp.json()
if not isinstance(payload, list):
raise GammaClientError(
f"Unexpected gamma response shape for {path}: {type(payload).__name__}"
)
return payload
except (httpx.HTTPError, ValueError) as exc:
last_exc = exc
logger.warning(
"gamma %s attempt %d/%d failed: %s",
path,
attempt + 1,
self._max_retries,
exc,
)
if attempt < self._max_retries - 1:
await asyncio.sleep(delay)
delay *= 2
raise GammaClientError(
f"gamma {path} failed after {self._max_retries} attempts: {last_exc}"
)
async def get_active_market_stats(self) -> dict[str, GammaMarketStats]:
"""Fetch volume/liquidity for the most-traded active markets.
Walks up to `max_pages` pages of `page_limit` markets each, sorted
by 24h volume descending, with bounded concurrency. Markets beyond
that window have effectively zero recent volume — the size_anomaly
niche path handles them without needing a ratio.
Returns:
Mapping condition_id -> GammaMarketStats.
"""
results: dict[str, GammaMarketStats] = {}
sem = asyncio.Semaphore(self._page_concurrency)
stop = asyncio.Event()
async with httpx.AsyncClient(
base_url=self._host,
timeout=self._timeout,
headers={"User-Agent": "polymarket-insider-tracker/0.1"},
) as client:
async def fetch_page(page_index: int) -> list[dict[str, object]]:
if stop.is_set():
return []
params = {
"limit": self._page_limit,
"offset": page_index * self._page_limit,
"active": "true",
"closed": "false",
"order": "volume24hr",
"ascending": "false",
}
async with sem:
if stop.is_set():
return []
try:
return await self._get_with_retry(client, "/markets", params)
except GammaClientError as exc:
# Gamma rejects offsets past its hard cap with a
# validation error; treat that as a clean stop.
logger.debug("gamma stop at page %d: %s", page_index, exc)
stop.set()
return []
tasks = [asyncio.create_task(fetch_page(i)) for i in range(self._max_pages)]
pages = await asyncio.gather(*tasks)
empty_streak = 0
for page in pages:
if not page:
empty_streak += 1
continue
empty_streak = 0
for raw in page:
if not isinstance(raw, dict):
continue
parsed = _parse_market(raw)
if parsed is not None:
results[parsed.condition_id] = parsed
if len(page) < self._page_limit:
# short page — we walked past the end of the active set
empty_streak += 1
if empty_streak >= 2:
break
logger.info("gamma sync: fetched stats for %d active markets", len(results))
return results
@@ -9,14 +9,13 @@ import contextlib
import json
import logging
from collections.abc import Callable
from dataclasses import dataclass, replace
from dataclasses import dataclass
from datetime import UTC, datetime
from enum import StrEnum
from redis.asyncio import Redis
from .clob_client import ClobClient
from .gamma_client import GammaClient, GammaClientError, GammaMarketStats
from .models import MarketMetadata
logger = logging.getLogger(__name__)
@@ -92,7 +91,6 @@ class MarketMetadataSync:
redis: Redis,
clob_client: ClobClient,
*,
gamma_client: GammaClient | None = None,
sync_interval_seconds: int = DEFAULT_SYNC_INTERVAL_SECONDS,
cache_ttl_seconds: int = DEFAULT_CACHE_TTL_SECONDS,
key_prefix: str = DEFAULT_REDIS_KEY_PREFIX,
@@ -104,8 +102,6 @@ class MarketMetadataSync:
Args:
redis: Redis async client for caching.
clob_client: CLOB API client for fetching markets.
gamma_client: Optional gamma-api client for volume/liquidity
enrichment. Defaults to a fresh GammaClient() instance.
sync_interval_seconds: Interval between syncs (default: 300 / 5 min).
cache_ttl_seconds: TTL for cached entries (default: 600 / 10 min).
key_prefix: Redis key prefix for market data.
@@ -114,7 +110,6 @@ class MarketMetadataSync:
"""
self._redis = redis
self._clob = clob_client
self._gamma = gamma_client or GammaClient()
self._sync_interval = sync_interval_seconds
self._cache_ttl = cache_ttl_seconds
self._key_prefix = key_prefix
@@ -221,18 +216,6 @@ class MarketMetadataSync:
self._set_state(SyncState.ERROR)
# Continue running - will retry on next interval
async def _fetch_gamma_stats(self) -> dict[str, GammaMarketStats]:
"""Fetch volume/liquidity stats from gamma-api.
Returns an empty dict on failure so a degraded gamma endpoint
does not stop CLOB metadata from being cached.
"""
try:
return await self._gamma.get_active_market_stats()
except (GammaClientError, Exception) as e:
logger.warning("gamma stats fetch failed (continuing without volume): %s", e)
return {}
async def _sync_all_markets(self) -> None:
"""Fetch all markets and cache them in Redis."""
self._set_state(SyncState.SYNCING)
@@ -240,27 +223,14 @@ class MarketMetadataSync:
self._stats.total_syncs += 1
try:
# Fetch CLOB markets and gamma volume snapshot in parallel
markets, gamma_stats = await asyncio.gather(
asyncio.to_thread(self._clob.get_markets, True),
self._fetch_gamma_stats(),
)
# Fetch markets from CLOB API (runs in thread pool for sync API)
markets = await asyncio.to_thread(self._clob.get_markets, True)
# Cache each market in Redis, enriched with gamma volume/liquidity
# Cache each market in Redis
cached_count = 0
enriched_count = 0
for market in markets:
try:
metadata = MarketMetadata.from_market(market)
stats = gamma_stats.get(metadata.condition_id)
if stats is not None:
metadata = replace(
metadata,
daily_volume=stats.daily_volume,
weekly_volume=stats.weekly_volume,
liquidity=stats.liquidity,
)
enriched_count += 1
await self._cache_market(metadata)
cached_count += 1
except Exception as e:
@@ -276,10 +246,7 @@ class MarketMetadataSync:
self._set_state(SyncState.IDLE)
logger.info(
"Synced %d markets (%d enriched with gamma volume) in %.2fs",
cached_count,
enriched_count,
self._stats.last_sync_duration_seconds,
f"Synced {cached_count} markets in {self._stats.last_sync_duration_seconds:.2f}s"
)
# Notify callback
@@ -399,12 +399,6 @@ class MarketMetadata:
# Derived metadata
category: str = "other"
# Liquidity/volume snapshot (from gamma-api). All optional — older
# cache entries and CLOB-only sync results may not have these.
daily_volume: Decimal | None = None
weekly_volume: Decimal | None = None
liquidity: Decimal | None = None
# Cache metadata
last_updated: datetime = field(default_factory=lambda: datetime.now(UTC))
@@ -452,9 +446,6 @@ class MarketMetadata:
"active": self.active,
"closed": self.closed,
"category": self.category,
"daily_volume": str(self.daily_volume) if self.daily_volume is not None else None,
"weekly_volume": str(self.weekly_volume) if self.weekly_volume is not None else None,
"liquidity": str(self.liquidity) if self.liquidity is not None else None,
"last_updated": self.last_updated.isoformat(),
}
@@ -486,15 +477,6 @@ class MarketMetadata:
else:
last_updated = datetime.now(UTC)
def _opt_dec(key: str) -> Decimal | None:
raw = data.get(key)
if raw is None or raw == "":
return None
try:
return Decimal(str(raw))
except (ValueError, ArithmeticError):
return None
return cls(
condition_id=str(data["condition_id"]),
question=str(data.get("question", "")),
@@ -504,8 +486,5 @@ class MarketMetadata:
active=bool(data.get("active", True)),
closed=bool(data.get("closed", False)),
category=str(data.get("category", "other")),
daily_volume=_opt_dec("daily_volume"),
weekly_volume=_opt_dec("weekly_volume"),
liquidity=_opt_dec("liquidity"),
last_updated=last_updated,
)
@@ -26,8 +26,46 @@ logger = logging.getLogger(__name__)
USDC_BRIDGED = "0x2791Bca1f2de4661ED88A30C99A7a9449Aa84174"
USDC_NATIVE = "0x3c499c542cEF5E3811e1192ce70d8cC03d5c3359"
# ERC20 Transfer event signature
# ERC20 Transfer event signature. ``HexBytes.hex()`` returns a *bare* hex
# string without the ``0x`` prefix; publicnode tolerates that, but stricter
# providers (e.g. drpc — which we use as the fallback) reject it with
# ``invalid argument 0: hex string without 0x prefix``. Always pass the
# 0x-prefixed form to ``eth_getLogs``.
TRANSFER_EVENT_SIGNATURE = AsyncWeb3.keccak(text="Transfer(address,address,uint256)")
TRANSFER_EVENT_TOPIC = "0x" + TRANSFER_EVENT_SIGNATURE.hex().removeprefix("0x")
# eth_getLogs block-range chunking. Most public Polygon RPCs (publicnode, ankr,
# llamarpc) cap the range at 10_000 blocks per call; pick a window slightly
# under the cap so off-by-one differences between providers don't trip us up.
DEFAULT_CHUNK_SIZE_BLOCKS = 9_000
# Polygon block time is ~2.0s. publicnode (the most common free RPC) prunes
# log history aggressively — empirically only ~100k blocks (~55 hours) are
# served before requests start returning "History has been pruned". We default
# to 80k blocks (~44 hours), which is more than enough for fresh-wallet
# funding traces (those wallets are by definition new) and fits comfortably
# inside what most public providers retain.
DEFAULT_MAX_LOOKBACK_BLOCKS = 80_000
# Substrings that, when present in an RPC error, indicate the chunk we just
# asked for is outside the provider's archive horizon. Walking further back
# is futile, so we stop the trace early instead of hammering every chunk.
_PRUNED_HISTORY_MARKERS: tuple[str, ...] = (
"history has been pruned",
"missing trie node",
"older than",
)
def _is_pruned_history_error(err: BaseException) -> bool:
"""Return True if the RPC error indicates pruned history.
Public Polygon nodes only retain a recent slice of log history. When we
walk back through that slice in chunks and hit the cutoff, every further
chunk will fail with the same message — so we stop early instead of
burning quota on guaranteed failures.
"""
text = str(err).lower()
return any(marker in text for marker in _PRUNED_HISTORY_MARKERS)
class FundingTracer:
@@ -50,6 +88,8 @@ class FundingTracer:
*,
max_hops: int = 3,
usdc_addresses: list[str] | None = None,
chunk_size_blocks: int = DEFAULT_CHUNK_SIZE_BLOCKS,
max_lookback_blocks: int = DEFAULT_MAX_LOOKBACK_BLOCKS,
) -> None:
"""Initialize the funding tracer.
@@ -58,6 +98,11 @@ class FundingTracer:
entity_registry: Registry for entity classification. Creates default if None.
max_hops: Maximum hops to trace back (default 3).
usdc_addresses: USDC contract addresses to track. Uses defaults if None.
chunk_size_blocks: Block window size per eth_getLogs call. Public
Polygon RPCs cap at 10_000 blocks; default leaves a safety margin.
max_lookback_blocks: How far back to scan when caller passes
``from_block=0``. Default ~44 hours at 2s block time, which
fits inside the pruned-history horizon of most public RPCs.
"""
self.polygon_client = polygon_client
self.entity_registry = entity_registry or EntityRegistry()
@@ -65,6 +110,8 @@ class FundingTracer:
self._usdc_addresses = [
addr.lower() for addr in (usdc_addresses or [USDC_BRIDGED, USDC_NATIVE])
]
self._chunk_size_blocks = chunk_size_blocks
self._max_lookback_blocks = max_lookback_blocks
async def trace(
self,
@@ -209,46 +256,144 @@ class FundingTracer:
) -> list[dict[str, Any]]:
"""Get ERC20 Transfer event logs.
Public Polygon RPCs (publicnode, ankr, llamarpc) cap ``eth_getLogs`` at
10_000 blocks per call. To work around this we resolve the requested
range into a concrete block window (defaulting to the last
``max_lookback_blocks`` when caller passes ``from_block=0``) and walk
the window in chunks of ``chunk_size_blocks``, oldest-first, stopping
once ``limit`` matches are collected. Walking oldest-first preserves
the "first transfer" semantics expected by the funding chain tracer.
If a chunk comes back with a "history has been pruned" style error
the rest of the walk is short-circuited — every subsequent chunk
would hit the same archive cutoff and there's no point burning quota
on guaranteed failures.
Args:
to_address: Filter by recipient address.
token_address: ERC20 token contract address.
limit: Maximum logs to return.
from_block: Starting block number.
to_block: Ending block number.
from_block: Starting block number (0 means
``latest - max_lookback_blocks``).
to_block: Ending block number ("latest" resolves to current head).
Returns:
List of log dictionaries.
List of log dictionaries, oldest first, capped at ``limit``.
"""
# Pad address to 32 bytes for topic filter
padded_to = "0x" + to_address.lower().replace("0x", "").zfill(64)
topics = [
TRANSFER_EVENT_TOPIC, # Transfer event (must be 0x-prefixed for drpc)
None, # from (any)
padded_to, # to (target address)
]
contract_address = AsyncWeb3.to_checksum_address(token_address)
start_block, end_block = await self._resolve_block_range(from_block, to_block)
if start_block > end_block:
return []
results: list[dict[str, Any]] = []
chunk_size = max(1, self._chunk_size_blocks)
chunk_start = start_block
while chunk_start <= end_block:
chunk_end = min(chunk_start + chunk_size - 1, end_block)
try:
chunk_logs = await self._fetch_logs_chunk(
contract_address=contract_address,
topics=topics,
from_block=chunk_start,
to_block=chunk_end,
)
except Exception as e:
if _is_pruned_history_error(e):
# The provider has dropped this slice of history. Walking
# further back will hit the same wall on every chunk;
# stop now and return what we already have.
logger.info(
"eth_getLogs chunk %d-%d outside archive horizon for %s; stopping trace",
chunk_start,
chunk_end,
to_address,
)
break
logger.warning(
"eth_getLogs chunk %d-%d failed for %s: %s",
chunk_start,
chunk_end,
to_address,
e,
)
# Skip this window and keep walking — partial data is better
# than aborting the whole trace on a single flaky chunk.
chunk_start = chunk_end + 1
continue
for log in chunk_logs:
results.append(dict(log))
if len(results) >= limit:
return results
chunk_start = chunk_end + 1
return results
async def _resolve_block_range(
self,
from_block: int | str,
to_block: int | str,
) -> tuple[int, int]:
"""Resolve symbolic block params to concrete numeric bounds.
``from_block=0`` (the historical default) is rewritten to
``latest - max_lookback_blocks`` so we don't try to scan all of Polygon.
"""
w3 = self._select_w3()
if isinstance(to_block, str):
await self.polygon_client._rate_limiter.acquire()
head = int(await w3.eth.block_number)
end = head
else:
end = int(to_block)
if isinstance(from_block, str):
# Treat any symbolic from-block (e.g. "earliest") as "go back
# max_lookback_blocks from end"; that's what callers actually want.
start = max(0, end - self._max_lookback_blocks)
elif from_block == 0:
start = max(0, end - self._max_lookback_blocks)
else:
start = int(from_block)
return start, end
async def _fetch_logs_chunk(
self,
contract_address: str,
topics: list[Any],
from_block: int,
to_block: int,
) -> list[Any]:
"""Issue a single bounded ``eth_getLogs`` call."""
await self.polygon_client._rate_limiter.acquire()
# Use the web3 instance from polygon client
w3 = (
self.polygon_client._w3
if self.polygon_client._primary_healthy
else (self.polygon_client._w3_fallback or self.polygon_client._w3)
)
# Get logs with Transfer event filtering by recipient
w3 = self._select_w3()
# Note: web3 typing is overly restrictive for block params
logs = await w3.eth.get_logs(
return await w3.eth.get_logs(
{
"address": AsyncWeb3.to_checksum_address(token_address),
"topics": [
TRANSFER_EVENT_SIGNATURE.hex(), # Transfer event
None, # from (any)
padded_to, # to (target address)
],
"address": contract_address,
"topics": topics,
"fromBlock": from_block, # type: ignore[typeddict-item]
"toBlock": to_block, # type: ignore[typeddict-item]
}
)
# Convert to list of dicts and limit
result = [dict(log) for log in logs[:limit]]
return result
def _select_w3(self) -> AsyncWeb3:
"""Pick primary or fallback web3 instance based on health."""
if self.polygon_client._primary_healthy:
return self.polygon_client._w3
return self.polygon_client._w3_fallback or self.polygon_client._w3
async def _log_to_funding_transfer(
self,
-249
View File
@@ -1,249 +0,0 @@
"""Tests for the gamma-api client."""
from __future__ import annotations
from decimal import Decimal
import httpx
import pytest
from polymarket_insider_tracker.ingestor import gamma_client as gamma_module
from polymarket_insider_tracker.ingestor.gamma_client import (
GammaClient,
GammaClientError,
GammaMarketStats,
_parse_market,
)
class TestParseMarket:
def test_parses_full_payload(self) -> None:
raw = {
"conditionId": "0xabc",
"volume24hr": "12345.67",
"volume1wk": "100000",
"volume1mo": "500000",
"volumeNum": "999999.5",
"liquidityNum": "42000",
}
stats = _parse_market(raw)
assert stats is not None
assert stats.condition_id == "0xabc"
assert stats.daily_volume == Decimal("12345.67")
assert stats.weekly_volume == Decimal("100000")
assert stats.monthly_volume == Decimal("500000")
assert stats.total_volume == Decimal("999999.5")
assert stats.liquidity == Decimal("42000")
def test_falls_back_to_alternative_keys(self) -> None:
raw = {
"conditionId": "0x1",
"volume24hr": "1",
"volume": "777",
"liquidity": "55",
}
stats = _parse_market(raw)
assert stats is not None
assert stats.total_volume == Decimal("777")
assert stats.liquidity == Decimal("55")
def test_handles_missing_numeric_fields(self) -> None:
stats = _parse_market({"conditionId": "0x2"})
assert stats is not None
assert stats.daily_volume is None
assert stats.liquidity is None
def test_drops_garbage_decimals(self) -> None:
stats = _parse_market(
{"conditionId": "0x3", "volume24hr": "not-a-number", "liquidityNum": ""}
)
assert stats is not None
assert stats.daily_volume is None
assert stats.liquidity is None
def test_rejects_missing_condition_id(self) -> None:
assert _parse_market({"volume24hr": "1"}) is None
assert _parse_market({"conditionId": ""}) is None
assert _parse_market({"conditionId": 123}) is None # type: ignore[arg-type]
def _make_client(
_handler: httpx.MockTransport,
*,
page_limit: int = 100,
max_pages: int = 5,
page_concurrency: int = 5,
max_retries: int = 1,
) -> GammaClient:
"""Build a GammaClient that constructs httpx.AsyncClient with the given transport.
GammaClient creates its own AsyncClient inside `get_active_market_stats`,
so we monkeypatch the AsyncClient factory in the module to inject the mock
transport.
"""
return GammaClient(
page_limit=page_limit,
max_pages=max_pages,
page_concurrency=page_concurrency,
max_retries=max_retries,
retry_base_delay_seconds=0.0,
)
@pytest.fixture
def patch_async_client(monkeypatch: pytest.MonkeyPatch):
"""Replace the AsyncClient used by gamma_client with one bound to a MockTransport."""
def _apply(handler: httpx.MockTransport) -> None:
original = gamma_module.httpx.AsyncClient
def factory(*args: object, **kwargs: object) -> httpx.AsyncClient:
kwargs["transport"] = handler # type: ignore[index]
return original(*args, **kwargs) # type: ignore[arg-type]
monkeypatch.setattr(gamma_module.httpx, "AsyncClient", factory)
return _apply
@pytest.mark.asyncio
async def test_get_active_market_stats_single_page(patch_async_client) -> None:
page_one = [
{"conditionId": "0xa", "volume24hr": "100", "liquidityNum": "10"},
{"conditionId": "0xb", "volume24hr": "200", "liquidityNum": "20"},
]
calls: list[dict[str, str]] = []
def handler(request: httpx.Request) -> httpx.Response:
calls.append(dict(request.url.params))
offset = int(request.url.params.get("offset", "0"))
if offset == 0:
return httpx.Response(200, json=page_one)
return httpx.Response(200, json=[])
transport = httpx.MockTransport(handler)
patch_async_client(transport)
client = _make_client(transport, page_limit=2, max_pages=3)
result = await client.get_active_market_stats()
assert set(result.keys()) == {"0xa", "0xb"}
assert isinstance(result["0xa"], GammaMarketStats)
assert result["0xa"].daily_volume == Decimal("100")
assert calls[0]["limit"] == "2"
assert calls[0]["order"] == "volume24hr"
assert calls[0]["ascending"] == "false"
@pytest.mark.asyncio
async def test_get_active_market_stats_short_page_stops(patch_async_client) -> None:
"""A page shorter than page_limit signals end-of-data after a small empty streak."""
page_zero = [{"conditionId": f"0x{i}", "volume24hr": str(i)} for i in range(5)]
page_one_short = [{"conditionId": "0xshort", "volume24hr": "1"}]
def handler(request: httpx.Request) -> httpx.Response:
offset = int(request.url.params.get("offset", "0"))
if offset == 0:
return httpx.Response(200, json=page_zero)
if offset == 5:
return httpx.Response(200, json=page_one_short)
return httpx.Response(200, json=[])
transport = httpx.MockTransport(handler)
patch_async_client(transport)
client = _make_client(transport, page_limit=5, max_pages=10)
result = await client.get_active_market_stats()
assert "0xshort" in result
assert len(result) == 6
@pytest.mark.asyncio
async def test_get_active_market_stats_offset_cap_clean_stop(
patch_async_client,
) -> None:
"""Gamma rejects offsets past its hard cap; that error is swallowed cleanly."""
def handler(request: httpx.Request) -> httpx.Response:
offset = int(request.url.params.get("offset", "0"))
if offset == 0:
return httpx.Response(200, json=[{"conditionId": "0xa", "volume24hr": "1"}])
return httpx.Response(400, json={"error": "offset too large"})
transport = httpx.MockTransport(handler)
patch_async_client(transport)
client = _make_client(transport, page_limit=1, max_pages=4, max_retries=1)
result = await client.get_active_market_stats()
assert "0xa" in result
@pytest.mark.asyncio
async def test_get_with_retry_recovers_after_transient_error(
patch_async_client,
) -> None:
"""Transient HTTP errors retry up to max_retries before giving up."""
state = {"attempts": 0}
def handler(request: httpx.Request) -> httpx.Response:
offset = int(request.url.params.get("offset", "0"))
if offset == 0:
state["attempts"] += 1
if state["attempts"] < 2:
return httpx.Response(503, json={"error": "transient"})
return httpx.Response(200, json=[{"conditionId": "0xrecover", "volume24hr": "1"}])
return httpx.Response(200, json=[])
transport = httpx.MockTransport(handler)
patch_async_client(transport)
client = _make_client(transport, page_limit=1, max_pages=2, max_retries=3)
result = await client.get_active_market_stats()
assert "0xrecover" in result
assert state["attempts"] == 2
@pytest.mark.asyncio
async def test_unexpected_response_shape_is_handled(patch_async_client) -> None:
"""A non-list payload becomes a clean stop, not a crash."""
def handler(_request: httpx.Request) -> httpx.Response:
return httpx.Response(200, json={"unexpected": "shape"})
transport = httpx.MockTransport(handler)
patch_async_client(transport)
client = _make_client(transport, page_limit=1, max_pages=2, max_retries=1)
result = await client.get_active_market_stats()
assert result == {}
@pytest.mark.asyncio
async def test_skips_non_dict_entries(patch_async_client) -> None:
"""Defensive: server returning mixed-type list items shouldn't crash."""
def handler(request: httpx.Request) -> httpx.Response:
offset = int(request.url.params.get("offset", "0"))
if offset == 0:
return httpx.Response(
200,
json=[
{"conditionId": "0xa", "volume24hr": "1"},
"garbage",
None,
42,
],
)
return httpx.Response(200, json=[])
transport = httpx.MockTransport(handler)
patch_async_client(transport)
client = _make_client(transport, page_limit=4, max_pages=2)
result = await client.get_active_market_stats()
assert list(result.keys()) == ["0xa"]
def test_gamma_client_error_inherits_exception() -> None:
assert issubclass(GammaClientError, Exception)
+26 -67
View File
@@ -8,7 +8,6 @@ from unittest.mock import AsyncMock, MagicMock
import pytest
from polymarket_insider_tracker.ingestor.clob_client import ClobClient
from polymarket_insider_tracker.ingestor.gamma_client import GammaClient
from polymarket_insider_tracker.ingestor.metadata_sync import (
DEFAULT_CACHE_TTL_SECONDS,
DEFAULT_REDIS_KEY_PREFIX,
@@ -73,18 +72,6 @@ def mock_clob(sample_market: Market) -> MagicMock:
return clob
@pytest.fixture
def mock_gamma() -> MagicMock:
"""Create a mock GammaClient that returns empty volume stats.
Without this, MarketMetadataSync would instantiate a default
GammaClient and hit the real gamma-api over HTTP during unit tests.
"""
gamma = MagicMock(spec=GammaClient)
gamma.get_active_market_stats = AsyncMock(return_value={})
return gamma
class TestDeriveCategory:
"""Tests for the derive_category function."""
@@ -208,9 +195,9 @@ class TestSyncStats:
class TestMarketMetadataSync:
"""Tests for the MarketMetadataSync class."""
def test_init(self, mock_redis: AsyncMock, mock_clob: MagicMock, mock_gamma: MagicMock) -> None:
def test_init(self, mock_redis: AsyncMock, mock_clob: MagicMock) -> None:
"""Test initialization."""
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob, gamma_client=mock_gamma)
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob)
assert sync.state == SyncState.STOPPED
assert sync.stats.total_syncs == 0
@@ -218,14 +205,11 @@ class TestMarketMetadataSync:
assert sync._cache_ttl == DEFAULT_CACHE_TTL_SECONDS
assert sync._key_prefix == DEFAULT_REDIS_KEY_PREFIX
def test_init_custom_config(
self, mock_redis: AsyncMock, mock_clob: MagicMock, mock_gamma: MagicMock
) -> None:
def test_init_custom_config(self, mock_redis: AsyncMock, mock_clob: MagicMock) -> None:
"""Test initialization with custom config."""
sync = MarketMetadataSync(
redis=mock_redis,
clob_client=mock_clob,
gamma_client=mock_gamma,
sync_interval_seconds=60,
cache_ttl_seconds=120,
key_prefix="custom:",
@@ -236,11 +220,9 @@ class TestMarketMetadataSync:
assert sync._key_prefix == "custom:"
@pytest.mark.asyncio
async def test_start_stop(
self, mock_redis: AsyncMock, mock_clob: MagicMock, mock_gamma: MagicMock
) -> None:
async def test_start_stop(self, mock_redis: AsyncMock, mock_clob: MagicMock) -> None:
"""Test starting and stopping the sync service."""
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob, gamma_client=mock_gamma)
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob)
# Start
await sync.start()
@@ -254,10 +236,10 @@ class TestMarketMetadataSync:
@pytest.mark.asyncio
async def test_start_performs_initial_sync(
self, mock_redis: AsyncMock, mock_clob: MagicMock, mock_gamma: MagicMock
self, mock_redis: AsyncMock, mock_clob: MagicMock
) -> None:
"""Test that start performs an initial sync."""
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob, gamma_client=mock_gamma)
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob)
await sync.start()
@@ -270,12 +252,10 @@ class TestMarketMetadataSync:
await sync.stop()
@pytest.mark.asyncio
async def test_start_failure(
self, mock_redis: AsyncMock, mock_clob: MagicMock, mock_gamma: MagicMock
) -> None:
async def test_start_failure(self, mock_redis: AsyncMock, mock_clob: MagicMock) -> None:
"""Test start failure handling."""
mock_clob.get_markets.side_effect = Exception("API error")
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob, gamma_client=mock_gamma)
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob)
with pytest.raises(MetadataSyncError, match="initial sync failed"):
await sync.start()
@@ -288,7 +268,6 @@ class TestMarketMetadataSync:
self,
mock_redis: AsyncMock,
mock_clob: MagicMock,
mock_gamma: MagicMock,
sample_metadata: MarketMetadata,
) -> None:
"""Test get_market with cache hit."""
@@ -296,7 +275,7 @@ class TestMarketMetadataSync:
cached_data = json.dumps(sample_metadata.to_dict())
mock_redis.get = AsyncMock(return_value=cached_data)
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob, gamma_client=mock_gamma)
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob)
await sync.start()
result = await sync.get_market("cond123")
@@ -309,14 +288,12 @@ class TestMarketMetadataSync:
await sync.stop()
@pytest.mark.asyncio
async def test_get_market_cache_miss(
self, mock_redis: AsyncMock, mock_clob: MagicMock, mock_gamma: MagicMock
) -> None:
async def test_get_market_cache_miss(self, mock_redis: AsyncMock, mock_clob: MagicMock) -> None:
"""Test get_market with cache miss."""
# Setup cache miss
mock_redis.get = AsyncMock(return_value=None)
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob, gamma_client=mock_gamma)
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob)
await sync.start()
result = await sync.get_market("cond123")
@@ -331,14 +308,12 @@ class TestMarketMetadataSync:
await sync.stop()
@pytest.mark.asyncio
async def test_get_market_not_found(
self, mock_redis: AsyncMock, mock_clob: MagicMock, mock_gamma: MagicMock
) -> None:
async def test_get_market_not_found(self, mock_redis: AsyncMock, mock_clob: MagicMock) -> None:
"""Test get_market when market doesn't exist."""
mock_redis.get = AsyncMock(return_value=None)
mock_clob.get_market.return_value = None
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob, gamma_client=mock_gamma)
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob)
await sync.start()
result = await sync.get_market("nonexistent")
@@ -348,11 +323,9 @@ class TestMarketMetadataSync:
await sync.stop()
@pytest.mark.asyncio
async def test_invalidate_market(
self, mock_redis: AsyncMock, mock_clob: MagicMock, mock_gamma: MagicMock
) -> None:
async def test_invalidate_market(self, mock_redis: AsyncMock, mock_clob: MagicMock) -> None:
"""Test cache invalidation."""
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob, gamma_client=mock_gamma)
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob)
await sync.start()
result = await sync.invalidate_market("cond123")
@@ -363,11 +336,9 @@ class TestMarketMetadataSync:
await sync.stop()
@pytest.mark.asyncio
async def test_force_sync(
self, mock_redis: AsyncMock, mock_clob: MagicMock, mock_gamma: MagicMock
) -> None:
async def test_force_sync(self, mock_redis: AsyncMock, mock_clob: MagicMock) -> None:
"""Test forced sync."""
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob, gamma_client=mock_gamma)
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob)
await sync.start()
# Initial sync
@@ -382,9 +353,7 @@ class TestMarketMetadataSync:
await sync.stop()
@pytest.mark.asyncio
async def test_state_change_callback(
self, mock_redis: AsyncMock, mock_clob: MagicMock, mock_gamma: MagicMock
) -> None:
async def test_state_change_callback(self, mock_redis: AsyncMock, mock_clob: MagicMock) -> None:
"""Test state change callback."""
states: list[SyncState] = []
@@ -394,7 +363,6 @@ class TestMarketMetadataSync:
sync = MarketMetadataSync(
redis=mock_redis,
clob_client=mock_clob,
gamma_client=mock_gamma,
on_state_change=on_state_change,
)
@@ -409,7 +377,7 @@ class TestMarketMetadataSync:
@pytest.mark.asyncio
async def test_sync_complete_callback(
self, mock_redis: AsyncMock, mock_clob: MagicMock, mock_gamma: MagicMock
self, mock_redis: AsyncMock, mock_clob: MagicMock
) -> None:
"""Test sync complete callback."""
sync_stats: list[SyncStats] = []
@@ -420,7 +388,6 @@ class TestMarketMetadataSync:
sync = MarketMetadataSync(
redis=mock_redis,
clob_client=mock_clob,
gamma_client=mock_gamma,
on_sync_complete=on_sync_complete,
)
@@ -434,11 +401,7 @@ class TestMarketMetadataSync:
@pytest.mark.asyncio
async def test_get_markets_by_category(
self,
mock_redis: AsyncMock,
mock_clob: MagicMock,
mock_gamma: MagicMock,
sample_metadata: MarketMetadata,
self, mock_redis: AsyncMock, mock_clob: MagicMock, sample_metadata: MarketMetadata
) -> None:
"""Test getting markets by category."""
# Setup scan to return keys
@@ -449,7 +412,7 @@ class TestMarketMetadataSync:
cached_data = json.dumps(sample_metadata.to_dict())
mock_redis.get = AsyncMock(return_value=cached_data)
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob, gamma_client=mock_gamma)
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob)
# Don't start to avoid initial sync complexity
sync._state = SyncState.IDLE
@@ -459,11 +422,9 @@ class TestMarketMetadataSync:
assert results[0].category == "crypto"
@pytest.mark.asyncio
async def test_cannot_start_twice(
self, mock_redis: AsyncMock, mock_clob: MagicMock, mock_gamma: MagicMock
) -> None:
async def test_cannot_start_twice(self, mock_redis: AsyncMock, mock_clob: MagicMock) -> None:
"""Test that starting twice doesn't double-start."""
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob, gamma_client=mock_gamma)
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob)
await sync.start()
await sync.start() # Should be a no-op
@@ -473,11 +434,9 @@ class TestMarketMetadataSync:
await sync.stop()
@pytest.mark.asyncio
async def test_stop_when_stopped(
self, mock_redis: AsyncMock, mock_clob: MagicMock, mock_gamma: MagicMock
) -> None:
async def test_stop_when_stopped(self, mock_redis: AsyncMock, mock_clob: MagicMock) -> None:
"""Test stopping when already stopped."""
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob, gamma_client=mock_gamma)
sync = MarketMetadataSync(redis=mock_redis, clob_client=mock_clob)
await sync.stop() # Should be a no-op
+244 -1
View File
@@ -327,6 +327,10 @@ class TestGetTransferLogs:
await funding_tracer._get_transfer_logs(
to_address=TEST_WALLET,
token_address=USDC_BRIDGED,
# Explicit numeric range so we stay inside one chunk and skip
# the "latest" → block_number resolution path.
from_block=1,
to_block=8_000,
)
mock_w3.eth.get_logs.assert_called_once()
@@ -334,10 +338,15 @@ class TestGetTransferLogs:
# Verify topics structure
assert len(call_args["topics"]) == 3
assert call_args["topics"][0] == TRANSFER_EVENT_SIGNATURE.hex()
# The Transfer event topic must be 0x-prefixed; drpc rejects bare hex.
assert call_args["topics"][0] == "0x" + TRANSFER_EVENT_SIGNATURE.hex().removeprefix("0x")
assert call_args["topics"][0].startswith("0x")
assert call_args["topics"][1] is None # from (any)
# to address should be padded to 32 bytes
assert call_args["topics"][2].endswith(TEST_WALLET.lower().replace("0x", ""))
# And the chunk bounds match what we asked for.
assert call_args["fromBlock"] == 1
assert call_args["toBlock"] == 8_000
@pytest.mark.asyncio
async def test_get_transfer_logs_respects_limit(
@@ -355,6 +364,8 @@ class TestGetTransferLogs:
to_address=TEST_WALLET,
token_address=USDC_BRIDGED,
limit=3,
from_block=1,
to_block=8_000,
)
assert len(result) == 3
@@ -374,10 +385,242 @@ class TestGetTransferLogs:
await funding_tracer._get_transfer_logs(
to_address=TEST_WALLET,
token_address=USDC_BRIDGED,
from_block=1,
to_block=8_000,
)
mock_fallback.eth.get_logs.assert_called_once()
@pytest.mark.asyncio
async def test_get_transfer_logs_chunks_large_ranges(
self,
funding_tracer: FundingTracer,
mock_polygon_client: MagicMock,
) -> None:
"""Ranges wider than chunk_size are split into multiple eth_getLogs calls.
This is the regression guard for the publicnode 10_000-block cap that
was rejecting every funding trace before chunking landed.
"""
mock_w3 = MagicMock()
mock_w3.eth.get_logs = AsyncMock(return_value=[])
mock_polygon_client._w3 = mock_w3
# 25_000 blocks at 9_000-per-chunk → 3 calls (9000 + 9000 + 7001).
await funding_tracer._get_transfer_logs(
to_address=TEST_WALLET,
token_address=USDC_BRIDGED,
from_block=1_000_000,
to_block=1_025_000,
)
assert mock_w3.eth.get_logs.call_count == 3
windows = [call[0][0] for call in mock_w3.eth.get_logs.call_args_list]
assert windows[0]["fromBlock"] == 1_000_000
assert windows[0]["toBlock"] == 1_008_999
assert windows[1]["fromBlock"] == 1_009_000
assert windows[1]["toBlock"] == 1_017_999
assert windows[2]["fromBlock"] == 1_018_000
assert windows[2]["toBlock"] == 1_025_000
# No window exceeds the chunk size — that's what RPC providers reject.
for win in windows:
assert win["toBlock"] - win["fromBlock"] + 1 <= 9_000
@pytest.mark.asyncio
async def test_get_transfer_logs_stops_when_limit_hit_mid_walk(
self,
funding_tracer: FundingTracer,
mock_polygon_client: MagicMock,
) -> None:
"""Walking should stop as soon as ``limit`` matches are gathered."""
mock_w3 = MagicMock()
# First chunk yields 5 logs, more than the limit, so subsequent chunks
# must not be queried.
mock_w3.eth.get_logs = AsyncMock(return_value=[MagicMock() for _ in range(5)])
mock_polygon_client._w3 = mock_w3
result = await funding_tracer._get_transfer_logs(
to_address=TEST_WALLET,
token_address=USDC_BRIDGED,
limit=2,
from_block=1_000_000,
to_block=1_025_000,
)
assert len(result) == 2
mock_w3.eth.get_logs.assert_called_once()
@pytest.mark.asyncio
async def test_get_transfer_logs_skips_failing_chunk(
self,
funding_tracer: FundingTracer,
mock_polygon_client: MagicMock,
) -> None:
"""A flaky chunk must not abort the whole trace — we move on."""
mock_w3 = MagicMock()
good_log = MagicMock()
responses: list[Any] = [
RuntimeError("RPC hiccup"),
[good_log],
]
async def fake_get_logs(_params: dict[str, Any]) -> list[Any]:
outcome = responses.pop(0)
if isinstance(outcome, BaseException):
raise outcome
return outcome
mock_w3.eth.get_logs = AsyncMock(side_effect=fake_get_logs)
mock_polygon_client._w3 = mock_w3
result = await funding_tracer._get_transfer_logs(
to_address=TEST_WALLET,
token_address=USDC_BRIDGED,
from_block=1_000_000,
to_block=1_018_000, # forces 3 chunks; we exercise chunks 1+2
)
# The error chunk is skipped; the second chunk contributes one log.
assert result == [dict(good_log)]
assert mock_w3.eth.get_logs.call_count >= 2
@pytest.mark.asyncio
async def test_get_transfer_logs_resolves_latest_via_block_number(
self,
funding_tracer: FundingTracer,
mock_polygon_client: MagicMock,
) -> None:
"""``to_block='latest'`` should resolve via ``eth.block_number``.
And ``from_block=0`` should not become a full-history scan it must
be clamped to ``latest - max_lookback_blocks``.
"""
async def _block_number_coro() -> int:
return 5_000
mock_eth = MagicMock()
mock_eth.get_logs = AsyncMock(return_value=[])
# Property-style awaitable: web3.py exposes block_number as a property
# returning a coroutine, so each access must yield a fresh awaitable.
type(mock_eth).block_number = property( # type: ignore[misc]
lambda _self: _block_number_coro()
)
mock_w3 = MagicMock()
mock_w3.eth = mock_eth
mock_polygon_client._w3 = mock_w3
await funding_tracer._get_transfer_logs(
to_address=TEST_WALLET,
token_address=USDC_BRIDGED,
)
# block_number=5000 < chunk_size, so it's one chunk that bottoms at 0.
mock_eth.get_logs.assert_called_once()
call_args = mock_eth.get_logs.call_args[0][0]
assert call_args["fromBlock"] == 0
assert call_args["toBlock"] == 5_000
@pytest.mark.asyncio
async def test_get_transfer_logs_breaks_on_pruned_history(
self,
funding_tracer: FundingTracer,
mock_polygon_client: MagicMock,
) -> None:
"""A pruned-history error must short-circuit the whole walk.
Public Polygon RPCs prune log history. Once we walk past the cutoff,
every subsequent chunk will raise the same error keep walking and
we just burn quota on guaranteed failures. The first such error must
end the walk and return whatever we already collected.
"""
mock_w3 = MagicMock()
good_log = MagicMock()
responses: list[Any] = [
[good_log],
RuntimeError(
"{'code': -32701, 'message': 'History has been pruned for "
"this block. To remove restrictions, order a dedicated full "
"node here: https://www.allnodes.com/pol/host'}"
),
# If the early-break logic is missing, this third chunk would
# also be requested. The test asserts it isn't.
[MagicMock()],
]
async def fake_get_logs(_params: dict[str, Any]) -> list[Any]:
outcome = responses.pop(0)
if isinstance(outcome, BaseException):
raise outcome
return outcome
mock_w3.eth.get_logs = AsyncMock(side_effect=fake_get_logs)
mock_polygon_client._w3 = mock_w3
# 3 chunks total. The pruned error fires on chunk #2; chunk #3 must
# never be issued.
result = await funding_tracer._get_transfer_logs(
to_address=TEST_WALLET,
token_address=USDC_BRIDGED,
from_block=1_000_000,
to_block=1_027_000,
)
assert result == [dict(good_log)]
assert mock_w3.eth.get_logs.call_count == 2
@pytest.mark.asyncio
async def test_get_transfer_logs_default_lookback_fits_pruned_horizon(
self,
) -> None:
"""Default ``max_lookback_blocks`` must stay inside what public RPCs serve.
publicnode prunes after ~100k blocks. If we default to 1.3M, every
funding trace blows through the archive horizon and produces nothing
but pruned-history warnings. Pin the default at <= 100k as a
regression guard.
"""
from polymarket_insider_tracker.profiler.funding import (
DEFAULT_MAX_LOOKBACK_BLOCKS,
)
assert DEFAULT_MAX_LOOKBACK_BLOCKS <= 100_000
@pytest.mark.asyncio
async def test_get_transfer_logs_topic_is_0x_prefixed(
self,
funding_tracer: FundingTracer,
mock_polygon_client: MagicMock,
) -> None:
"""The Transfer event topic passed to ``eth_getLogs`` must begin with ``0x``.
``HexBytes.hex()`` returns a bare hex string. publicnode tolerates
that, but stricter providers like drpc (our fallback) reject it with
``invalid argument 0: hex string without 0x prefix`` and every chunk
in the trace fails. This guards against regressing back to the
bare-hex form.
"""
mock_w3 = MagicMock()
mock_w3.eth.get_logs = AsyncMock(return_value=[])
mock_polygon_client._w3 = mock_w3
await funding_tracer._get_transfer_logs(
to_address=TEST_WALLET,
token_address=USDC_BRIDGED,
from_block=1,
to_block=8_000,
)
topics = mock_w3.eth.get_logs.call_args[0][0]["topics"]
assert topics[0].startswith("0x")
# And the topic also has to be 32 bytes (64 hex chars) as required by
# the JSON-RPC spec.
assert len(topics[0]) == 2 + 64
# The padded `to` topic was already 0x-prefixed; double-check that
# didn't regress either.
assert topics[2].startswith("0x")
class TestLogToFundingTransfer:
"""Tests for _log_to_funding_transfer method."""