Compare commits

...

14 Commits

Author SHA1 Message Date
pselamy e5d6194947 fix: resolve .env.example variable interpolation and alembic env var mismatch (#62)
.env files don't support shell variable expansion, so DATABASE_URL and
REDIS_URL contained literal ${...} strings. Also alembic/env.py read
SQLALCHEMY_DATABASE_URL instead of DATABASE_URL.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 00:04:42 +00:00
Patrick Selamy a263d20e58 Merge pull request #60 from pselamy/fix/56
feat: add graceful shutdown handler with signal trapping (#56)
2026-01-04 20:57:09 -05:00
Patrick Selamy a18fc3493a feat: add graceful shutdown handler with signal trapping (#56)
Implement graceful shutdown handling for the async pipeline with proper
signal trapping and cleanup coordination.

Features:
- GracefulShutdown class with SIGTERM/SIGINT signal trapping
- Async event-based shutdown coordination
- Configurable shutdown timeout (default: 30 seconds)
- Cleanup callback registration (sync and async)
- Force exit on second signal
- Async context manager support
- Platform-specific signal handling (Unix/Windows)

Integration:
- Updated __main__.py to use GracefulShutdown wrapper
- Pipeline cleanup registered as shutdown callback
- Proper logging of shutdown stages

Tests: 25 new tests covering signal handling, timeouts, cleanup callbacks,
and context manager behavior.

Closes #56

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 20:54:38 -05:00
Patrick Selamy 29968897c5 Merge pull request #59 from pselamy/fix/55
feat: add CLI entry point with argparse (#55)
2026-01-04 20:46:50 -05:00
Patrick Selamy 5cf6382e42 feat: add CLI entry point with argparse (#55)
Implement the main CLI entry point for the Polymarket Insider Tracker.

Features:
- argparse-based CLI with version, config-check, dry-run, log-level flags
- Structured logging configuration with multiple formatters
- Configuration validation with friendly error messages
- Startup banner display
- Config check mode for validation without running
- Integration with Pipeline orchestrator

Tests: 20 new tests covering parser, logging, banner, config validation,
and integration scenarios.

Closes #55

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 20:44:17 -05:00
Patrick Selamy bf7b972edb Merge pull request #58 from pselamy/fix/54
feat: implement main pipeline orchestrator (#54)
2026-01-04 20:34:46 -05:00
Patrick Selamy 771f968401 feat: implement main pipeline orchestrator (#54)
Add Pipeline class that wires together all detection components and
manages the event flow from trade ingestion to alerting.

Key features:
- Lifecycle management (start/stop) with proper cleanup
- Component initialization from Settings
- Parallel detector execution for fresh wallet and size anomaly
- Risk scoring with alert threshold
- Multi-channel alert dispatch (Discord/Telegram)
- Dry-run mode for testing without sending alerts
- Async context manager support
- Comprehensive test coverage (19 tests)

Pipeline flow: WebSocket → Profiler → Detectors → Scorer → Alerter

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 20:32:16 -05:00
Patrick Selamy 792b85c2be Merge pull request #57 from pselamy/fix/53
feat: implement configuration management service with Pydantic Settings (#53)
2026-01-04 20:17:39 -05:00
Patrick Selamy b801c0da5f feat: implement configuration management service with Pydantic Settings (#53)
- Add pydantic-settings dependency
- Create centralized config.py with nested settings groups:
  - DatabaseSettings: PostgreSQL connection validation
  - RedisSettings: Redis connection with defaults
  - PolygonSettings: RPC endpoints with fallback support
  - PolymarketSettings: WebSocket URL and optional API key
  - DiscordSettings: Optional webhook for alerts
  - TelegramSettings: Optional bot token/chat ID for alerts
- Implement singleton pattern with get_settings() and LRU cache
- Add redacted_summary() for logging config without exposing secrets
- Support .env file loading via python-dotenv
- Add comprehensive test suite (26 tests)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-04 20:12:27 -05:00
Patrick Selamy 704b6d6c29 Merge pull request #51 from pselamy/fix/48
fix: resolve all mypy type errors for strict type checking (#48)
2026-01-04 19:18:04 -05:00
Patrick Selamy 5c5440d776 fix: resolve all mypy type errors for strict type checking (#48)
## Summary

Fixed 30 mypy type errors across 8 files to enable strict type checking.

## Changes by Category

### no-any-return errors (fixed with explicit type casts)
- alerter/history.py: Cast Redis zcount/zremrangebyscore returns to int
- ingestor/clob_client.py: Cast API response values appropriately
- ingestor/publisher.py: Cast Redis xack, xlen, xtrim returns to int
- ingestor/metadata_sync.py: Cast Redis delete return to int
- detector/scorer.py: Cast Redis delete return to int

### operator errors (fixed with null checks)
- alerter/formatter.py: Added `if age_hours is not None:` guards

### External library typing issues (fixed with type: ignore)
- ingestor/publisher.py: redis-py typing issue with xadd dict parameter
- profiler/funding.py: web3 typing issue with block parameters
- storage/repos.py: SQLAlchemy Result.rowcount typing issue

### assignment errors (fixed by renaming)
- storage/repos.py: Renamed SQLite insert stmt to avoid type conflict

Closes #48

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 19:15:32 -05:00
Patrick Selamy f57e0243dd Merge pull request #50 from pselamy/fix/49
fix: resolve flaky tests in clob_client and websocket (#49)
2026-01-04 19:07:03 -05:00
Patrick Selamy 31c675bdec fix: resolve flaky tests in clob_client and websocket (#49)
## Changes

### test_get_market_not_found
- Updated test to expect `RetryError` instead of `ClobClientError`
- The `get_market` method uses the `@with_retry()` decorator, so when
  the underlying API call fails repeatedly, it raises `RetryError`
  wrapping the original exception
- Removed unused `ClobClientError` import

### test_start_and_receive_trades
- Replaced `MagicMock` with a proper `MockWebSocket` class that
  implements the async iterator protocol correctly
- `MagicMock` was passing `self` as an extra argument when calling
  `__aiter__`, causing "takes 0 positional arguments but 1 was given"
- Removed unused `MagicMock` import

Closes #49

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 19:04:32 -05:00
Patrick Selamy 95b7876057 Merge pull request #47 from pselamy/fix/26
feat: add GitHub Actions CI/CD pipeline (#26)
2026-01-04 17:23:40 -05:00
23 changed files with 2768 additions and 65 deletions
+2 -2
View File
@@ -6,12 +6,12 @@ POSTGRES_USER=tracker
POSTGRES_PASSWORD=dev_password
# Constructed database URL (for application use)
DATABASE_URL=postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT}/${POSTGRES_DB}
DATABASE_URL=postgresql://tracker:dev_password@localhost:5432/polymarket_tracker
# Redis Configuration
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_URL=redis://${REDIS_HOST}:${REDIS_PORT}
REDIS_URL=redis://localhost:6379
# Optional: Development tool ports
ADMINER_PORT=8080
+1 -1
View File
@@ -19,7 +19,7 @@ if config.config_file_name is not None:
target_metadata = Base.metadata
# Get database URL from environment variable or config
database_url = os.environ.get("SQLALCHEMY_DATABASE_URL")
database_url = os.environ.get("DATABASE_URL")
if database_url:
config.set_main_option("sqlalchemy.url", database_url)
+14
View File
@@ -0,0 +1,14 @@
"""Pytest configuration for the polymarket-insider-tracker tests."""
import pytest
# Configure pytest-asyncio
pytest_plugins = ["pytest_asyncio"]
@pytest.fixture(scope="session")
def event_loop_policy():
"""Use default event loop policy."""
import asyncio
return asyncio.DefaultEventLoopPolicy()
+1
View File
@@ -12,6 +12,7 @@ dependencies = [
"sqlalchemy>=2.0.0",
"alembic>=1.13.0",
"pydantic>=2.0.0",
"pydantic-settings>=2.0.0",
"python-dotenv>=1.0.0",
"websockets>=12.0",
"prometheus-client>=0.19.0",
+302
View File
@@ -0,0 +1,302 @@
"""CLI entry point for Polymarket Insider Tracker.
This module provides the main entry point for running the tracker
from the command line.
Usage:
python -m polymarket_insider_tracker [options]
"""
from __future__ import annotations
import argparse
import asyncio
import logging
import logging.config
import sys
from typing import NoReturn
from pydantic import ValidationError
from polymarket_insider_tracker import __version__
from polymarket_insider_tracker.config import Settings, clear_settings_cache, get_settings
from polymarket_insider_tracker.pipeline import Pipeline
from polymarket_insider_tracker.shutdown import GracefulShutdown
# Application info
APP_NAME = "Polymarket Insider Tracker"
APP_VERSION = __version__
# Exit codes
EXIT_SUCCESS = 0
EXIT_ERROR = 1
EXIT_CONFIG_ERROR = 2
EXIT_INTERRUPTED = 130
def create_parser() -> argparse.ArgumentParser:
"""Create the argument parser for the CLI.
Returns:
Configured ArgumentParser instance.
"""
parser = argparse.ArgumentParser(
prog="polymarket-insider-tracker",
description="Detect insider trading activity on Polymarket prediction markets.",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
python -m polymarket_insider_tracker Run full pipeline
python -m polymarket_insider_tracker --config-check Validate config and exit
python -m polymarket_insider_tracker --dry-run Run without sending alerts
python -m polymarket_insider_tracker --log-level DEBUG Enable debug logging
""",
)
parser.add_argument(
"--version",
action="version",
version=f"%(prog)s {APP_VERSION}",
)
parser.add_argument(
"--config-check",
action="store_true",
help="Validate configuration and exit without running pipeline",
)
parser.add_argument(
"--log-level",
choices=["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"],
default=None,
help="Override logging level (default: from settings)",
)
parser.add_argument(
"--dry-run",
action="store_true",
help="Run pipeline but don't send alerts",
)
parser.add_argument(
"--health-port",
type=int,
default=None,
help="Override health check port (default: from settings)",
)
return parser
def configure_logging(level: str) -> None:
"""Configure structured logging for the application.
Args:
level: Logging level string (DEBUG, INFO, etc.)
"""
config = {
"version": 1,
"disable_existing_loggers": False,
"formatters": {
"standard": {
"format": "%(asctime)s [%(levelname)s] %(name)s: %(message)s",
"datefmt": "%Y-%m-%d %H:%M:%S",
},
"detailed": {
"format": (
"%(asctime)s [%(levelname)s] %(name)s (%(filename)s:%(lineno)d): %(message)s"
),
"datefmt": "%Y-%m-%d %H:%M:%S",
},
},
"handlers": {
"console": {
"class": "logging.StreamHandler",
"level": level,
"formatter": "detailed" if level == "DEBUG" else "standard",
"stream": "ext://sys.stdout",
},
},
"root": {
"level": level,
"handlers": ["console"],
},
# Quieter logging for noisy libraries
"loggers": {
"httpx": {"level": "WARNING"},
"httpcore": {"level": "WARNING"},
"websockets": {"level": "WARNING"},
"web3": {"level": "WARNING"},
"urllib3": {"level": "WARNING"},
},
}
logging.config.dictConfig(config)
def print_banner() -> None:
"""Print the application startup banner."""
banner = f"""
╔══════════════════════════════════════════════════════════════╗
║ ║
{APP_NAME:^56}
{"v" + APP_VERSION:^56}
║ ║
╚══════════════════════════════════════════════════════════════╝
"""
print(banner)
def print_config_summary(settings: Settings, dry_run: bool) -> None:
"""Print a summary of the configuration.
Args:
settings: Application settings.
dry_run: Whether dry-run mode is enabled.
"""
summary = settings.redacted_summary()
print("Configuration:")
print(f" Database: {summary['database_url']}")
print(f" Redis: {summary['redis_url']}")
print(f" Log Level: {summary['log_level']}")
print(f" Health Port: {summary['health_port']}")
print(f" Dry Run: {dry_run}")
print(f" Discord: {'enabled' if summary['discord_enabled'] == 'True' else 'disabled'}")
print(f" Telegram: {'enabled' if summary['telegram_enabled'] == 'True' else 'disabled'}")
print()
def validate_config() -> Settings | None:
"""Validate and load configuration.
Returns:
Settings instance if valid, None if invalid.
"""
try:
# Clear cache to force reload
clear_settings_cache()
return get_settings()
except ValidationError as e:
print("Configuration validation failed:", file=sys.stderr)
for error in e.errors():
field = ".".join(str(loc) for loc in error["loc"])
msg = error["msg"]
print(f" {field}: {msg}", file=sys.stderr)
return None
def run_config_check(settings: Settings) -> int:
"""Run configuration check and exit.
Args:
settings: Validated settings.
Returns:
Exit code (0 for success).
"""
print("Configuration is valid!")
print()
print_config_summary(settings, dry_run=False)
# Test component availability
print("Checking component availability...")
# Check Discord
if settings.discord.enabled:
print(" Discord: configured")
else:
print(" Discord: not configured")
# Check Telegram
if settings.telegram.enabled:
print(" Telegram: configured")
else:
print(" Telegram: not configured")
print()
print("All checks passed. Ready to run.")
return EXIT_SUCCESS
async def run_pipeline(
settings: Settings,
dry_run: bool,
shutdown_timeout: float = 30.0,
) -> int:
"""Run the main pipeline with graceful shutdown handling.
Args:
settings: Application settings.
dry_run: Whether to skip sending alerts.
shutdown_timeout: Maximum time to wait for graceful shutdown.
Returns:
Exit code.
"""
logger = logging.getLogger(__name__)
shutdown = GracefulShutdown(timeout=shutdown_timeout)
try:
async with shutdown:
pipeline = Pipeline(settings, dry_run=dry_run)
# Register pipeline cleanup
shutdown.register_cleanup(pipeline.stop)
logger.info("Starting pipeline...")
await pipeline.start()
logger.info("Pipeline running. Press Ctrl+C to stop.")
# Wait for shutdown signal
await shutdown.wait()
logger.info("Shutdown signal received, stopping pipeline...")
await pipeline.stop()
return EXIT_SUCCESS
except KeyboardInterrupt:
logger.info("Interrupted by user")
return EXIT_INTERRUPTED
except Exception as e:
logger.exception("Pipeline failed: %s", e)
return EXIT_ERROR
def main(argv: list[str] | None = None) -> NoReturn:
"""Main entry point for the CLI.
Args:
argv: Command line arguments (defaults to sys.argv[1:]).
"""
parser = create_parser()
args = parser.parse_args(argv)
# Validate configuration first
settings = validate_config()
if settings is None:
sys.exit(EXIT_CONFIG_ERROR)
# Determine effective log level
log_level = args.log_level or settings.log_level
configure_logging(log_level)
# Print banner
print_banner()
# Config check mode
if args.config_check:
sys.exit(run_config_check(settings))
# Determine dry-run mode
dry_run = args.dry_run or settings.dry_run
# Print config summary
print_config_summary(settings, dry_run)
# Run pipeline
exit_code = asyncio.run(run_pipeline(settings, dry_run))
sys.exit(exit_code)
if __name__ == "__main__":
main()
@@ -190,10 +190,11 @@ class AlertFormatter:
wallet_age_str = ""
if assessment.fresh_wallet_signal:
age_hours = assessment.fresh_wallet_signal.wallet_profile.age_hours
if age_hours < 1:
wallet_age_str = f" (Age: {int(age_hours * 60)}m)"
else:
wallet_age_str = f" (Age: {age_hours:.0f}h)"
if age_hours is not None:
if age_hours < 1:
wallet_age_str = f" (Age: {int(age_hours * 60)}m)"
else:
wallet_age_str = f" (Age: {age_hours:.0f}h)"
fields: list[dict[str, object]] = [
{
@@ -282,10 +283,11 @@ class AlertFormatter:
wallet_line = f"*Wallet:* `{wallet_short}`"
if assessment.fresh_wallet_signal:
age_hours = assessment.fresh_wallet_signal.wallet_profile.age_hours
if age_hours < 1:
wallet_line += f" \\(Age: {int(age_hours * 60)}m\\)"
else:
wallet_line += f" \\(Age: {age_hours:.0f}h\\)"
if age_hours is not None:
if age_hours < 1:
wallet_line += f" \\(Age: {int(age_hours * 60)}m\\)"
else:
wallet_line += f" \\(Age: {age_hours:.0f}h\\)"
lines.append(wallet_line)
# Risk score
@@ -366,10 +368,11 @@ class AlertFormatter:
wallet_line = f"Wallet: {wallet_short}"
if assessment.fresh_wallet_signal:
age_hours = assessment.fresh_wallet_signal.wallet_profile.age_hours
if age_hours < 1:
wallet_line += f" (Age: {int(age_hours * 60)}m)"
else:
wallet_line += f" (Age: {age_hours:.0f}h)"
if age_hours is not None:
if age_hours < 1:
wallet_line += f" (Age: {int(age_hours * 60)}m)"
else:
wallet_line += f" (Age: {age_hours:.0f}h)"
lines.append(wallet_line)
# Risk
@@ -362,7 +362,7 @@ class AlertHistory:
start.timestamp(),
end.timestamp(),
)
return count
return int(count)
async def cleanup_old_alerts(self) -> int:
"""Remove alerts older than retention period.
@@ -393,4 +393,4 @@ class AlertHistory:
# Note: Individual alert records will expire via TTL
# Wallet/market indexes will also expire via TTL
logger.info(f"Cleaned up {removed} old alert references")
return removed
return int(removed)
+263
View File
@@ -0,0 +1,263 @@
"""Configuration management service with Pydantic Settings.
This module provides centralized configuration management for the
Polymarket Insider Tracker application, loading and validating
environment variables at startup.
"""
from __future__ import annotations
import logging
from functools import lru_cache
from typing import Literal
from pydantic import Field, SecretStr, field_validator
from pydantic_settings import BaseSettings, SettingsConfigDict
class DatabaseSettings(BaseSettings):
"""Database connection settings."""
model_config = SettingsConfigDict(env_prefix="")
url: str = Field(
alias="DATABASE_URL",
description="PostgreSQL connection string",
)
@field_validator("url")
@classmethod
def validate_url(cls, v: str) -> str:
"""Validate database URL format."""
if not v.startswith(("postgresql://", "postgresql+asyncpg://")):
raise ValueError("DATABASE_URL must be a PostgreSQL connection string")
return v
class RedisSettings(BaseSettings):
"""Redis connection settings."""
model_config = SettingsConfigDict(env_prefix="")
url: str = Field(
default="redis://localhost:6379",
alias="REDIS_URL",
description="Redis connection string",
)
@field_validator("url")
@classmethod
def validate_url(cls, v: str) -> str:
"""Validate Redis URL format."""
if not v.startswith("redis://"):
raise ValueError("REDIS_URL must start with redis://")
return v
class PolygonSettings(BaseSettings):
"""Polygon blockchain RPC settings."""
model_config = SettingsConfigDict(env_prefix="POLYGON_")
rpc_url: str = Field(
default="https://polygon-rpc.com",
alias="POLYGON_RPC_URL",
description="Primary Polygon RPC endpoint",
)
fallback_rpc_url: str | None = Field(
default=None,
alias="POLYGON_FALLBACK_RPC_URL",
description="Fallback Polygon RPC endpoint",
)
@field_validator("rpc_url", "fallback_rpc_url")
@classmethod
def validate_url(cls, v: str | None) -> str | None:
"""Validate RPC URL format."""
if v is None:
return v
if not v.startswith(("http://", "https://")):
raise ValueError("RPC URL must be an HTTP(S) endpoint")
return v
class PolymarketSettings(BaseSettings):
"""Polymarket API settings."""
model_config = SettingsConfigDict(env_prefix="POLYMARKET_")
ws_url: str = Field(
default="wss://ws-subscriptions-clob.polymarket.com/ws/market",
alias="POLYMARKET_WS_URL",
description="Polymarket WebSocket URL for live data",
)
api_key: SecretStr | None = Field(
default=None,
alias="POLYMARKET_API_KEY",
description="Optional Polymarket API key",
)
@field_validator("ws_url")
@classmethod
def validate_ws_url(cls, v: str) -> str:
"""Validate WebSocket URL format."""
if not v.startswith(("ws://", "wss://")):
raise ValueError("WebSocket URL must start with ws:// or wss://")
return v
class DiscordSettings(BaseSettings):
"""Discord notification settings."""
model_config = SettingsConfigDict(env_prefix="DISCORD_")
webhook_url: SecretStr | None = Field(
default=None,
alias="DISCORD_WEBHOOK_URL",
description="Discord webhook URL for alerts",
)
@property
def enabled(self) -> bool:
"""Check if Discord notifications are enabled."""
return self.webhook_url is not None
class TelegramSettings(BaseSettings):
"""Telegram notification settings."""
model_config = SettingsConfigDict(env_prefix="TELEGRAM_")
bot_token: SecretStr | None = Field(
default=None,
alias="TELEGRAM_BOT_TOKEN",
description="Telegram bot token",
)
chat_id: str | None = Field(
default=None,
alias="TELEGRAM_CHAT_ID",
description="Telegram chat ID for alerts",
)
@property
def enabled(self) -> bool:
"""Check if Telegram notifications are enabled."""
return self.bot_token is not None and self.chat_id is not None
class Settings(BaseSettings):
"""Main application settings.
Loads configuration from environment variables with support for
.env files via python-dotenv.
Example:
```python
from polymarket_insider_tracker.config import get_settings
settings = get_settings()
print(settings.database.url)
print(settings.log_level)
```
"""
model_config = SettingsConfigDict(
env_file=".env",
env_file_encoding="utf-8",
extra="ignore",
)
# Nested configuration groups
database: DatabaseSettings = Field(default_factory=DatabaseSettings)
redis: RedisSettings = Field(default_factory=RedisSettings)
polygon: PolygonSettings = Field(default_factory=PolygonSettings)
polymarket: PolymarketSettings = Field(default_factory=PolymarketSettings)
discord: DiscordSettings = Field(default_factory=DiscordSettings)
telegram: TelegramSettings = Field(default_factory=TelegramSettings)
# Application settings
log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] = Field(
default="INFO",
alias="LOG_LEVEL",
description="Logging level",
)
health_port: int = Field(
default=8080,
alias="HEALTH_PORT",
description="HTTP port for health check endpoints",
ge=1,
le=65535,
)
dry_run: bool = Field(
default=False,
alias="DRY_RUN",
description="Run without sending actual alerts",
)
def get_logging_level(self) -> int:
"""Get the numeric logging level."""
level: int = getattr(logging, self.log_level)
return level
def redacted_summary(self) -> dict[str, str | dict[str, str]]:
"""Get a summary of settings with secrets redacted.
Returns:
Dictionary of settings with sensitive values masked.
"""
return {
"database_url": self._redact_url(self.database.url),
"redis_url": self._redact_url(self.redis.url),
"polygon": {
"rpc_url": self.polygon.rpc_url,
"fallback_rpc_url": self.polygon.fallback_rpc_url or "(not set)",
},
"polymarket": {
"ws_url": self.polymarket.ws_url,
"api_key": "(set)" if self.polymarket.api_key else "(not set)",
},
"discord_enabled": str(self.discord.enabled),
"telegram_enabled": str(self.telegram.enabled),
"log_level": self.log_level,
"health_port": str(self.health_port),
"dry_run": str(self.dry_run),
}
@staticmethod
def _redact_url(url: str) -> str:
"""Redact password from URL if present."""
if "@" in url and "://" in url:
# URL has credentials - redact the password
protocol_end = url.index("://") + 3
at_pos = url.index("@")
creds_part = url[protocol_end:at_pos]
if ":" in creds_part:
username = creds_part.split(":")[0]
return f"{url[:protocol_end]}{username}:***@{url[at_pos + 1 :]}"
return url
@lru_cache(maxsize=1)
def get_settings() -> Settings:
"""Get the application settings singleton.
Uses LRU cache to ensure settings are loaded only once and
reused across the application.
Returns:
The Settings instance.
Raises:
ValidationError: If required environment variables are missing
or have invalid values.
"""
return Settings()
def clear_settings_cache() -> None:
"""Clear the settings cache.
Useful for testing when you need to reload settings with
different environment variables.
"""
get_settings.cache_clear()
@@ -268,7 +268,7 @@ class RiskScorer:
"""
key = f"{self._key_prefix}{wallet_address}:{market_id}"
deleted = await self._redis.delete(key)
return deleted > 0
return int(deleted) > 0
async def assess_batch(self, bundles: list[SignalBundle]) -> list[RiskAssessment]:
"""Assess multiple trade bundles.
@@ -287,7 +287,8 @@ class ClobClient:
try:
response = self._client.get_midpoint(token_id)
return response.get("mid")
mid = response.get("mid")
return str(mid) if mid is not None else None
except Exception as e:
logger.warning("Failed to get midpoint for %s: %s", token_id, e)
return None
@@ -307,7 +308,8 @@ class ClobClient:
try:
response = self._client.get_price(token_id, side=side)
return response.get("price")
price = response.get("price")
return str(price) if price is not None else None
except Exception as e:
logger.warning("Failed to get %s price for %s: %s", side, token_id, e)
return None
@@ -321,7 +323,7 @@ class ClobClient:
try:
self._rate_limiter.acquire_sync()
result = self._client.get_ok()
return result == "OK"
return str(result) == "OK"
except Exception as e:
logger.error("Health check failed: %s", e)
return False
@@ -334,7 +336,8 @@ class ClobClient:
"""
try:
self._rate_limiter.acquire_sync()
return self._client.get_server_time()
result = self._client.get_server_time()
return int(result) if result is not None else None
except Exception as e:
logger.error("Failed to get server time: %s", e)
return None
@@ -351,7 +351,7 @@ class MarketMetadataSync:
"""
key = f"{self._key_prefix}{condition_id}"
deleted = await self._redis.delete(key)
return deleted > 0
return int(deleted) > 0
async def force_sync(self) -> None:
"""Force an immediate sync of all markets.
@@ -186,9 +186,10 @@ class EventPublisher:
The entry ID assigned by Redis.
"""
data = _serialize_trade_event(event)
# redis-py typing expects broader dict type than dict[str, str]
entry_id = await self._redis.xadd(
self._stream_name,
data,
data, # type: ignore[arg-type]
maxlen=self._max_len,
)
# entry_id may be bytes or str
@@ -213,7 +214,8 @@ class EventPublisher:
pipe = self._redis.pipeline()
for event in events:
data = _serialize_trade_event(event)
pipe.xadd(self._stream_name, data, maxlen=self._max_len)
# redis-py typing expects broader dict type than dict[str, str]
pipe.xadd(self._stream_name, data, maxlen=self._max_len) # type: ignore[arg-type]
results = await pipe.execute()
@@ -384,7 +386,8 @@ class EventPublisher:
"""
if not entry_ids:
return 0
return await self._redis.xack(self._stream_name, group_name, *entry_ids)
result = await self._redis.xack(self._stream_name, group_name, *entry_ids)
return int(result)
async def get_stream_info(self) -> dict[str, Any]:
"""Get information about the stream.
@@ -404,7 +407,8 @@ class EventPublisher:
Returns:
Number of entries in the stream.
"""
return await self._redis.xlen(self._stream_name)
result = await self._redis.xlen(self._stream_name)
return int(result)
async def trim_stream(self, max_len: int | None = None) -> int:
"""Trim the stream to a maximum length.
@@ -416,4 +420,5 @@ class EventPublisher:
Number of entries removed.
"""
length = max_len or self._max_len
return await self._redis.xtrim(self._stream_name, maxlen=length)
result = await self._redis.xtrim(self._stream_name, maxlen=length)
return int(result)
+480
View File
@@ -0,0 +1,480 @@
"""Main pipeline orchestrator for Polymarket Insider Tracker.
This module provides the Pipeline class that wires together all detection
components and manages the event flow from ingestion to alerting.
"""
from __future__ import annotations
import asyncio
import contextlib
import logging
from dataclasses import dataclass
from datetime import UTC, datetime
from enum import Enum
from typing import TYPE_CHECKING
from redis.asyncio import Redis
from polymarket_insider_tracker.alerter.channels.discord import DiscordChannel
from polymarket_insider_tracker.alerter.channels.telegram import TelegramChannel
from polymarket_insider_tracker.alerter.dispatcher import AlertChannel, AlertDispatcher
from polymarket_insider_tracker.alerter.formatter import AlertFormatter
from polymarket_insider_tracker.config import Settings, get_settings
from polymarket_insider_tracker.detector.fresh_wallet import FreshWalletDetector
from polymarket_insider_tracker.detector.scorer import RiskScorer, SignalBundle
from polymarket_insider_tracker.detector.size_anomaly import SizeAnomalyDetector
from polymarket_insider_tracker.ingestor.clob_client import ClobClient
from polymarket_insider_tracker.ingestor.metadata_sync import MarketMetadataSync
from polymarket_insider_tracker.ingestor.websocket import TradeStreamHandler
from polymarket_insider_tracker.profiler.analyzer import WalletAnalyzer
from polymarket_insider_tracker.profiler.chain import PolygonClient
from polymarket_insider_tracker.storage.database import DatabaseManager
if TYPE_CHECKING:
from typing import Any
from polymarket_insider_tracker.detector.models import (
FreshWalletSignal,
SizeAnomalySignal,
)
from polymarket_insider_tracker.ingestor.models import TradeEvent
logger = logging.getLogger(__name__)
class PipelineState(str, Enum):
"""Pipeline lifecycle states."""
STOPPED = "stopped"
STARTING = "starting"
RUNNING = "running"
STOPPING = "stopping"
ERROR = "error"
@dataclass
class PipelineStats:
"""Statistics for the pipeline."""
started_at: datetime | None = None
trades_processed: int = 0
signals_generated: int = 0
alerts_sent: int = 0
errors: int = 0
last_trade_time: datetime | None = None
last_error: str | None = None
class Pipeline:
"""Main pipeline orchestrator for the Polymarket Insider Tracker.
This class wires together all detection components and manages the
event flow from trade ingestion through profiling, detection, and alerting.
Pipeline flow:
WebSocket Trade Stream → Wallet Profiler → Detectors → Risk Scorer → Alerter
Example:
```python
from polymarket_insider_tracker.config import get_settings
from polymarket_insider_tracker.pipeline import Pipeline
settings = get_settings()
pipeline = Pipeline(settings)
await pipeline.start()
# Pipeline runs until stop() is called
await pipeline.stop()
```
"""
def __init__(
self,
settings: Settings | None = None,
*,
dry_run: bool | None = None,
) -> None:
"""Initialize the pipeline.
Args:
settings: Application settings. If not provided, uses get_settings().
dry_run: If True, skip sending alerts. Overrides settings.dry_run.
"""
self._settings = settings or get_settings()
self._dry_run = dry_run if dry_run is not None else self._settings.dry_run
self._state = PipelineState.STOPPED
self._stats = PipelineStats()
# Components (initialized in start())
self._redis: Redis | None = None
self._db_manager: DatabaseManager | None = None
self._polygon_client: PolygonClient | None = None
self._clob_client: ClobClient | None = None
self._metadata_sync: MarketMetadataSync | None = None
self._wallet_analyzer: WalletAnalyzer | None = None
self._fresh_wallet_detector: FreshWalletDetector | None = None
self._size_anomaly_detector: SizeAnomalyDetector | None = None
self._risk_scorer: RiskScorer | None = None
self._alert_formatter: AlertFormatter | None = None
self._alert_dispatcher: AlertDispatcher | None = None
self._trade_stream: TradeStreamHandler | None = None
# Synchronization
self._stop_event: asyncio.Event | None = None
self._stream_task: asyncio.Task[None] | None = None
@property
def state(self) -> PipelineState:
"""Current pipeline state."""
return self._state
@property
def stats(self) -> PipelineStats:
"""Current pipeline statistics."""
return self._stats
@property
def is_running(self) -> bool:
"""Check if pipeline is running."""
return self._state == PipelineState.RUNNING
async def start(self) -> None:
"""Start the pipeline.
Initializes all components and begins processing trades.
Raises:
RuntimeError: If pipeline is already running.
Exception: If any component fails to initialize.
"""
if self._state != PipelineState.STOPPED:
raise RuntimeError(f"Cannot start pipeline in state {self._state}")
self._state = PipelineState.STARTING
self._stop_event = asyncio.Event()
logger.info("Starting pipeline...")
try:
await self._initialize_components()
await self._start_background_services()
self._stats.started_at = datetime.now(UTC)
self._state = PipelineState.RUNNING
logger.info("Pipeline started successfully")
except Exception as e:
self._state = PipelineState.ERROR
self._stats.last_error = str(e)
logger.error("Failed to start pipeline: %s", e)
await self._cleanup()
raise
async def stop(self) -> None:
"""Stop the pipeline gracefully.
Stops all background services and cleans up resources.
"""
if self._state == PipelineState.STOPPED:
return
self._state = PipelineState.STOPPING
logger.info("Stopping pipeline...")
if self._stop_event:
self._stop_event.set()
await self._stop_background_services()
await self._cleanup()
self._state = PipelineState.STOPPED
logger.info("Pipeline stopped")
async def _initialize_components(self) -> None:
"""Initialize all pipeline components."""
settings = self._settings
# Initialize Redis
logger.debug("Initializing Redis connection...")
self._redis = Redis.from_url(settings.redis.url)
# Initialize Database Manager
logger.debug("Initializing database manager...")
self._db_manager = DatabaseManager(
settings.database.url,
async_mode=True,
)
# Initialize Polygon client
logger.debug("Initializing Polygon client...")
self._polygon_client = PolygonClient(
settings.polygon.rpc_url,
fallback_rpc_url=settings.polygon.fallback_rpc_url,
redis=self._redis,
)
# Initialize CLOB client
logger.debug("Initializing CLOB client...")
api_key = (
settings.polymarket.api_key.get_secret_value() if settings.polymarket.api_key else None
)
self._clob_client = ClobClient(api_key=api_key)
# Initialize Market Metadata Sync
logger.debug("Initializing market metadata sync...")
self._metadata_sync = MarketMetadataSync(
redis=self._redis,
clob_client=self._clob_client,
)
# Initialize Wallet Analyzer
logger.debug("Initializing wallet analyzer...")
self._wallet_analyzer = WalletAnalyzer(
self._polygon_client,
redis=self._redis,
)
# Initialize Detectors
logger.debug("Initializing detectors...")
self._fresh_wallet_detector = FreshWalletDetector(self._wallet_analyzer)
self._size_anomaly_detector = SizeAnomalyDetector(self._metadata_sync)
# Initialize Risk Scorer
logger.debug("Initializing risk scorer...")
self._risk_scorer = RiskScorer(self._redis)
# Initialize Alerting
logger.debug("Initializing alerting components...")
self._alert_formatter = AlertFormatter(verbosity="detailed")
channels = self._build_alert_channels()
self._alert_dispatcher = AlertDispatcher(channels)
# Initialize Trade Stream
logger.debug("Initializing trade stream handler...")
self._trade_stream = TradeStreamHandler(
on_trade=self._on_trade,
host=settings.polymarket.ws_url,
)
logger.info("All components initialized")
def _build_alert_channels(self) -> list[AlertChannel]:
"""Build list of enabled alert channels."""
channels: list[AlertChannel] = []
settings = self._settings
if settings.discord.enabled and settings.discord.webhook_url:
webhook_url = settings.discord.webhook_url.get_secret_value()
channels.append(DiscordChannel(webhook_url))
logger.info("Discord channel enabled")
if settings.telegram.enabled:
bot_token = settings.telegram.bot_token
chat_id = settings.telegram.chat_id
if bot_token and chat_id:
channels.append(
TelegramChannel(
bot_token.get_secret_value(),
chat_id,
)
)
logger.info("Telegram channel enabled")
if not channels:
logger.warning("No alert channels configured")
return channels
async def _start_background_services(self) -> None:
"""Start background services."""
# Start metadata sync
if self._metadata_sync:
logger.debug("Starting metadata sync service...")
await self._metadata_sync.start()
# Start trade stream in background task
if self._trade_stream:
logger.debug("Starting trade stream...")
self._stream_task = asyncio.create_task(self._run_trade_stream())
async def _run_trade_stream(self) -> None:
"""Run the trade stream in a task."""
if not self._trade_stream:
return
try:
await self._trade_stream.start()
except asyncio.CancelledError:
logger.debug("Trade stream task cancelled")
except Exception as e:
logger.error("Trade stream error: %s", e)
self._stats.last_error = str(e)
self._stats.errors += 1
async def _stop_background_services(self) -> None:
"""Stop background services."""
# Stop trade stream
if self._trade_stream:
logger.debug("Stopping trade stream...")
await self._trade_stream.stop()
# Cancel stream task
if self._stream_task:
self._stream_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await self._stream_task
self._stream_task = None
# Stop metadata sync
if self._metadata_sync:
logger.debug("Stopping metadata sync...")
await self._metadata_sync.stop()
async def _cleanup(self) -> None:
"""Clean up resources."""
# Close database connections
if self._db_manager:
await self._db_manager.dispose_async()
self._db_manager = None
# Close Redis connection
if self._redis:
await self._redis.aclose()
self._redis = None
logger.debug("Resources cleaned up")
async def _on_trade(self, trade: TradeEvent) -> None:
"""Process a single trade event.
This is the main event handler that runs the detection pipeline:
1. Run fresh wallet detection
2. Run size anomaly detection
3. Score the combined signals
4. Send alert if threshold exceeded
Args:
trade: The trade event from the WebSocket stream.
"""
self._stats.trades_processed += 1
self._stats.last_trade_time = datetime.now(UTC)
try:
# Run detectors in parallel
fresh_signal, size_signal = await asyncio.gather(
self._detect_fresh_wallet(trade),
self._detect_size_anomaly(trade),
)
# Bundle signals
bundle = SignalBundle(
trade_event=trade,
fresh_wallet_signal=fresh_signal,
size_anomaly_signal=size_signal,
)
# Score and potentially alert
if fresh_signal or size_signal:
self._stats.signals_generated += 1
await self._score_and_alert(bundle)
except Exception as e:
logger.error("Error processing trade %s: %s", trade.trade_id, e)
self._stats.errors += 1
self._stats.last_error = str(e)
async def _detect_fresh_wallet(self, trade: TradeEvent) -> FreshWalletSignal | None:
"""Run fresh wallet detection."""
if not self._fresh_wallet_detector:
return None
try:
return await self._fresh_wallet_detector.analyze(trade)
except Exception as e:
logger.warning("Fresh wallet detection failed for %s: %s", trade.trade_id, e)
return None
async def _detect_size_anomaly(self, trade: TradeEvent) -> SizeAnomalySignal | None:
"""Run size anomaly detection."""
if not self._size_anomaly_detector:
return None
try:
return await self._size_anomaly_detector.analyze(trade)
except Exception as e:
logger.warning("Size anomaly detection failed for %s: %s", trade.trade_id, e)
return None
async def _score_and_alert(self, bundle: SignalBundle) -> None:
"""Score signals and send alert if threshold exceeded."""
if not self._risk_scorer or not self._alert_formatter or not self._alert_dispatcher:
return
# Get risk assessment
assessment = await self._risk_scorer.assess(bundle)
if not assessment.should_alert:
logger.debug(
"Trade %s below alert threshold (score=%.2f)",
bundle.trade_event.trade_id,
assessment.weighted_score,
)
return
# Format and dispatch alert
formatted_alert = self._alert_formatter.format(assessment)
if self._dry_run:
logger.info(
"[DRY RUN] Would send alert: wallet=%s, score=%.2f",
assessment.wallet_address[:10] + "...",
assessment.weighted_score,
)
return
result = await self._alert_dispatcher.dispatch(formatted_alert)
if result.all_succeeded:
self._stats.alerts_sent += 1
logger.info(
"Alert sent successfully: wallet=%s, score=%.2f",
assessment.wallet_address[:10] + "...",
assessment.weighted_score,
)
else:
logger.warning(
"Alert partially failed: %d/%d channels succeeded",
result.success_count,
result.success_count + result.failure_count,
)
async def run(self) -> None:
"""Start the pipeline and run until interrupted.
This is a convenience method that starts the pipeline and
blocks until a stop signal is received.
Example:
```python
pipeline = Pipeline()
try:
await pipeline.run()
except KeyboardInterrupt:
pass
```
"""
await self.start()
try:
if self._stop_event:
await self._stop_event.wait()
except asyncio.CancelledError:
pass
finally:
await self.stop()
async def __aenter__(self) -> Pipeline:
"""Async context manager entry."""
await self.start()
return self
async def __aexit__(self, *args: Any) -> None:
"""Async context manager exit."""
await self.stop()
@@ -232,6 +232,7 @@ class FundingTracer:
)
# Get logs with Transfer event filtering by recipient
# Note: web3 typing is overly restrictive for block params
logs = await w3.eth.get_logs(
{
"address": AsyncWeb3.to_checksum_address(token_address),
@@ -240,8 +241,8 @@ class FundingTracer:
None, # from (any)
padded_to, # to (target address)
],
"fromBlock": from_block,
"toBlock": to_block,
"fromBlock": from_block, # type: ignore[typeddict-item]
"toBlock": to_block, # type: ignore[typeddict-item]
}
)
@@ -317,7 +318,7 @@ class FundingTracer:
origin_type="error",
)
else:
chains[addr.lower()] = result # type: ignore[assignment]
chains[addr.lower()] = result
return chains
+325
View File
@@ -0,0 +1,325 @@
"""Graceful shutdown handler for Polymarket Insider Tracker.
This module provides signal handling and graceful shutdown coordination
for the async pipeline components.
Usage:
```python
async def main():
shutdown = GracefulShutdown()
async with shutdown:
pipeline = Pipeline(settings)
await pipeline.start()
# Wait for shutdown signal
await shutdown.wait()
# Graceful cleanup
await pipeline.stop()
```
"""
from __future__ import annotations
import asyncio
import logging
import signal
import sys
from collections.abc import Callable
from contextlib import suppress
from types import FrameType
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from typing import Any
logger = logging.getLogger(__name__)
# Default shutdown timeout in seconds
DEFAULT_SHUTDOWN_TIMEOUT = 30.0
# Signals to trap for graceful shutdown
SHUTDOWN_SIGNALS = (signal.SIGTERM, signal.SIGINT)
class ShutdownTimeoutError(Exception):
"""Raised when graceful shutdown exceeds timeout."""
class GracefulShutdown:
"""Graceful shutdown handler with signal trapping.
This class provides coordinated shutdown handling for async applications.
It traps SIGTERM and SIGINT signals and provides an async event that
can be awaited to detect shutdown requests.
Features:
- SIGTERM and SIGINT signal trapping
- Async event-based shutdown coordination
- Configurable shutdown timeout
- Async context manager support
- Cleanup callback registration
- Force exit on second signal or timeout
Example:
```python
shutdown = GracefulShutdown(timeout=30.0)
async with shutdown:
await some_long_running_task()
# Automatically handles cleanup on signals
```
"""
def __init__(
self,
timeout: float = DEFAULT_SHUTDOWN_TIMEOUT,
*,
exit_on_timeout: bool = True,
) -> None:
"""Initialize the shutdown handler.
Args:
timeout: Maximum time in seconds to wait for graceful shutdown.
exit_on_timeout: If True, force exit when timeout is exceeded.
"""
self._timeout = timeout
self._exit_on_timeout = exit_on_timeout
self._shutdown_event: asyncio.Event | None = None
self._shutdown_requested = False
self._force_exit_requested = False
self._original_handlers: dict[signal.Signals, Any] = {}
self._cleanup_callbacks: list[Callable[[], Any]] = []
self._loop: asyncio.AbstractEventLoop | None = None
@property
def timeout(self) -> float:
"""Shutdown timeout in seconds."""
return self._timeout
@property
def is_shutdown_requested(self) -> bool:
"""Check if shutdown has been requested."""
return self._shutdown_requested
@property
def is_force_exit_requested(self) -> bool:
"""Check if force exit has been requested (second signal received)."""
return self._force_exit_requested
def register_cleanup(self, callback: Callable[[], Any]) -> None:
"""Register a cleanup callback to run during shutdown.
Args:
callback: A callable (sync or async) to run during shutdown.
"""
self._cleanup_callbacks.append(callback)
def request_shutdown(self) -> None:
"""Programmatically request shutdown.
This can be used to trigger shutdown from application code
instead of waiting for a signal.
"""
if not self._shutdown_requested:
self._shutdown_requested = True
logger.info("Shutdown requested programmatically")
if self._shutdown_event:
self._shutdown_event.set()
async def wait(self) -> None:
"""Wait for a shutdown signal.
This coroutine blocks until a shutdown signal is received
or request_shutdown() is called.
"""
if self._shutdown_event is None:
self._shutdown_event = asyncio.Event()
await self._shutdown_event.wait()
async def wait_with_timeout(self) -> bool:
"""Wait for shutdown with timeout.
Returns:
True if shutdown was requested, False if timeout occurred.
"""
if self._shutdown_event is None:
self._shutdown_event = asyncio.Event()
try:
await asyncio.wait_for(
self._shutdown_event.wait(),
timeout=self._timeout,
)
return True
except TimeoutError:
return False
def install_signal_handlers(self) -> None:
"""Install signal handlers for graceful shutdown.
Traps SIGTERM and SIGINT to trigger graceful shutdown.
On Windows, only SIGINT is trapped as SIGTERM is not available.
"""
self._loop = asyncio.get_running_loop()
self._shutdown_event = asyncio.Event()
# Platform-specific signal handling
if sys.platform == "win32":
# Windows: use signal.signal for SIGINT only
self._install_windows_handlers()
else:
# Unix: use loop.add_signal_handler for both signals
self._install_unix_handlers()
logger.debug("Signal handlers installed")
def _install_unix_handlers(self) -> None:
"""Install Unix signal handlers using the event loop."""
if self._loop is None:
return
for sig in SHUTDOWN_SIGNALS:
try:
self._loop.add_signal_handler(
sig,
self._handle_signal,
sig,
)
logger.debug("Installed handler for %s", sig.name)
except (ValueError, OSError) as e:
logger.warning("Could not install handler for %s: %s", sig.name, e)
def _install_windows_handlers(self) -> None:
"""Install Windows signal handlers using signal.signal."""
for sig in SHUTDOWN_SIGNALS:
try:
self._original_handlers[sig] = signal.signal(
sig,
self._handle_signal_sync,
)
logger.debug("Installed handler for %s", sig.name)
except (ValueError, OSError) as e:
logger.warning("Could not install handler for %s: %s", sig.name, e)
def remove_signal_handlers(self) -> None:
"""Remove installed signal handlers and restore originals."""
if sys.platform == "win32":
self._remove_windows_handlers()
else:
self._remove_unix_handlers()
logger.debug("Signal handlers removed")
def _remove_unix_handlers(self) -> None:
"""Remove Unix signal handlers."""
if self._loop is None:
return
for sig in SHUTDOWN_SIGNALS:
with suppress(ValueError, OSError):
self._loop.remove_signal_handler(sig)
def _remove_windows_handlers(self) -> None:
"""Remove Windows signal handlers and restore originals."""
for sig, original in self._original_handlers.items():
with suppress(ValueError, OSError):
signal.signal(sig, original)
self._original_handlers.clear()
def _handle_signal(self, sig: signal.Signals) -> None:
"""Handle shutdown signal (Unix version).
Args:
sig: The signal that was received.
"""
if self._shutdown_requested:
# Second signal - force exit
self._force_exit_requested = True
logger.warning("Received %s again - forcing exit!", sig.name)
sys.exit(128 + sig.value)
else:
self._shutdown_requested = True
logger.info("Received %s - initiating graceful shutdown...", sig.name)
if self._shutdown_event:
self._shutdown_event.set()
def _handle_signal_sync(self, sig: int, _frame: FrameType | None) -> None:
"""Handle shutdown signal (Windows version).
Args:
sig: The signal number that was received.
_frame: The current stack frame (unused).
"""
sig_enum = signal.Signals(sig)
self._handle_signal(sig_enum)
async def run_cleanup_callbacks(self) -> None:
"""Run all registered cleanup callbacks."""
for callback in self._cleanup_callbacks:
try:
result = callback()
if asyncio.iscoroutine(result):
await result
except Exception as e:
logger.error("Cleanup callback failed: %s", e)
async def __aenter__(self) -> GracefulShutdown:
"""Async context manager entry - install signal handlers."""
self.install_signal_handlers()
return self
async def __aexit__(self, *_args: Any) -> None:
"""Async context manager exit - cleanup."""
self.remove_signal_handlers()
await self.run_cleanup_callbacks()
async def run_with_graceful_shutdown(
coro: Any,
*,
timeout: float = DEFAULT_SHUTDOWN_TIMEOUT,
) -> None:
"""Run an async coroutine with graceful shutdown handling.
This is a convenience function that wraps a coroutine with
signal handling and timeout-based cleanup.
Args:
coro: The coroutine to run.
timeout: Maximum time to wait for graceful shutdown.
Example:
```python
async def my_app():
await asyncio.sleep(3600) # Run for an hour
# Will handle SIGTERM/SIGINT gracefully
await run_with_graceful_shutdown(my_app())
```
"""
shutdown = GracefulShutdown(timeout=timeout)
async with shutdown:
task = asyncio.create_task(coro)
# Wait for either task completion or shutdown signal
shutdown_wait = asyncio.create_task(shutdown.wait())
done, pending = await asyncio.wait(
[task, shutdown_wait],
return_when=asyncio.FIRST_COMPLETED,
)
# Cancel pending tasks
for pending_task in pending:
pending_task.cancel()
with suppress(asyncio.CancelledError):
await pending_task
# If the main task is in done, get any exception
if task in done:
task.result()
+20 -17
View File
@@ -208,20 +208,20 @@ class WalletRepository:
await self.session.execute(stmt)
except Exception:
# Fall back to SQLite upsert for testing
stmt = sqlite_insert(WalletProfileModel).values(**values, created_at=now)
stmt = stmt.on_conflict_do_update(
sqlite_stmt = sqlite_insert(WalletProfileModel).values(**values, created_at=now)
sqlite_stmt = sqlite_stmt.on_conflict_do_update(
index_elements=["address"],
set_={
"nonce": stmt.excluded.nonce,
"first_seen_at": stmt.excluded.first_seen_at,
"is_fresh": stmt.excluded.is_fresh,
"matic_balance": stmt.excluded.matic_balance,
"usdc_balance": stmt.excluded.usdc_balance,
"analyzed_at": stmt.excluded.analyzed_at,
"updated_at": stmt.excluded.updated_at,
"nonce": sqlite_stmt.excluded.nonce,
"first_seen_at": sqlite_stmt.excluded.first_seen_at,
"is_fresh": sqlite_stmt.excluded.is_fresh,
"matic_balance": sqlite_stmt.excluded.matic_balance,
"usdc_balance": sqlite_stmt.excluded.usdc_balance,
"analyzed_at": sqlite_stmt.excluded.analyzed_at,
"updated_at": sqlite_stmt.excluded.updated_at,
},
)
await self.session.execute(stmt)
await self.session.execute(sqlite_stmt)
await self.session.flush()
return dto
@@ -238,7 +238,8 @@ class WalletRepository:
result = await self.session.execute(
delete(WalletProfileModel).where(WalletProfileModel.address == address.lower())
)
return result.rowcount > 0
# SQLAlchemy Result does have rowcount but typing doesn't reflect it
return (result.rowcount or 0) > 0 # type: ignore[attr-defined]
async def mark_stale(self, address: str) -> bool:
"""Mark a wallet profile as stale (soft delete).
@@ -257,7 +258,8 @@ class WalletRepository:
.where(WalletProfileModel.address == address.lower())
.values(analyzed_at=stale_time, updated_at=datetime.now(UTC))
)
return result.rowcount > 0
# SQLAlchemy Result does have rowcount but typing doesn't reflect it
return (result.rowcount or 0) > 0 # type: ignore[attr-defined]
class FundingRepository:
@@ -478,12 +480,12 @@ class RelationshipRepository:
await self.session.execute(stmt)
except Exception:
# Fall back to SQLite upsert for testing
stmt = sqlite_insert(WalletRelationshipModel).values(**values)
stmt = stmt.on_conflict_do_update(
sqlite_stmt = sqlite_insert(WalletRelationshipModel).values(**values)
sqlite_stmt = sqlite_stmt.on_conflict_do_update(
index_elements=["wallet_a", "wallet_b", "relationship_type"],
set_={"confidence": stmt.excluded.confidence},
set_={"confidence": sqlite_stmt.excluded.confidence},
)
await self.session.execute(stmt)
await self.session.execute(sqlite_stmt)
await self.session.flush()
return dto
@@ -506,4 +508,5 @@ class RelationshipRepository:
WalletRelationshipModel.relationship_type == relationship_type,
)
)
return result.rowcount > 0
# SQLAlchemy Result does have rowcount but typing doesn't reflect it
return (result.rowcount or 0) > 0 # type: ignore[attr-defined]
+10 -5
View File
@@ -7,7 +7,6 @@ import pytest
from polymarket_insider_tracker.ingestor.clob_client import (
ClobClient,
ClobClientError,
RateLimiter,
RetryError,
with_retry,
@@ -253,17 +252,23 @@ class TestClobClient:
assert market.condition_id == "0xabc"
assert len(market.tokens) == 2
@pytest.mark.xfail(reason="Retry logic wraps exception differently - see #49")
def test_get_market_not_found(self, mock_base_client: MagicMock) -> None:
"""Test error handling when market not found."""
"""Test error handling when market not found.
When the underlying API call fails, the @with_retry decorator will
retry the operation. After all retries are exhausted, it raises
RetryError wrapping the original exception.
"""
mock_base_client.get_market.side_effect = Exception("Not found")
client = ClobClient()
with pytest.raises(ClobClientError) as exc_info:
with pytest.raises(RetryError) as exc_info:
client.get_market("0xnotfound")
assert "0xnotfound" in str(exc_info.value)
# The RetryError wraps the original exception
assert "get_market" in str(exc_info.value)
assert exc_info.value.last_exception is not None
def test_get_orderbook(self, mock_base_client: MagicMock) -> None:
"""Test fetching an orderbook."""
+27 -12
View File
@@ -3,7 +3,7 @@
import asyncio
import json
from decimal import Decimal
from unittest.mock import AsyncMock, MagicMock, patch
from unittest.mock import AsyncMock, patch
import pytest
@@ -275,7 +275,6 @@ class TestTradeStreamHandlerIntegration:
"""
@pytest.mark.asyncio
@pytest.mark.xfail(reason="Async mock iterator signature issue - see #49")
async def test_start_and_receive_trades(self) -> None:
"""Test starting handler and receiving trades."""
received_trades: list[TradeEvent] = []
@@ -288,11 +287,6 @@ class TestTradeStreamHandlerIntegration:
initial_reconnect_delay=0.01,
)
# Create mock WebSocket that sends one trade then closes
mock_ws = MagicMock()
mock_ws.send = AsyncMock()
mock_ws.close = AsyncMock()
trade_message = json.dumps(
{
"topic": "activity",
@@ -311,12 +305,33 @@ class TestTradeStreamHandlerIntegration:
}
)
# Mock async iteration
async def mock_iter() -> None:
yield trade_message
await handler.stop() # Stop after first message
# Create a proper async iterable mock WebSocket
class MockWebSocket:
"""Mock WebSocket that yields one message then stops."""
mock_ws.__aiter__ = mock_iter
def __init__(self, handler: TradeStreamHandler, message: str):
self.handler = handler
self.message = message
self.sent = False
async def send(self, _msg: str) -> None:
pass
async def close(self) -> None:
pass
def __aiter__(self):
return self
async def __anext__(self) -> str:
if not self.sent:
self.sent = True
return self.message
# Stop the handler and raise StopAsyncIteration
await self.handler.stop()
raise StopAsyncIteration
mock_ws = MockWebSocket(handler, trade_message)
with patch("websockets.connect", AsyncMock(return_value=mock_ws)):
# Run with timeout to prevent hanging
+329
View File
@@ -0,0 +1,329 @@
"""Tests for configuration management service."""
from __future__ import annotations
import os
from typing import TYPE_CHECKING
from unittest.mock import patch
import pytest
from pydantic import ValidationError
from polymarket_insider_tracker.config import (
DatabaseSettings,
DiscordSettings,
PolygonSettings,
PolymarketSettings,
RedisSettings,
Settings,
TelegramSettings,
clear_settings_cache,
get_settings,
)
if TYPE_CHECKING:
from collections.abc import Iterator
@pytest.fixture(autouse=True)
def clear_cache() -> Iterator[None]:
"""Clear settings cache before and after each test."""
clear_settings_cache()
yield
clear_settings_cache()
class TestDatabaseSettings:
"""Tests for DatabaseSettings."""
def test_valid_postgresql_url(self) -> None:
"""Test valid PostgreSQL URL."""
with patch.dict(os.environ, {"DATABASE_URL": "postgresql://user:pass@localhost/db"}):
settings = DatabaseSettings()
assert settings.url == "postgresql://user:pass@localhost/db"
def test_valid_asyncpg_url(self) -> None:
"""Test valid asyncpg URL."""
with patch.dict(
os.environ, {"DATABASE_URL": "postgresql+asyncpg://user:pass@localhost/db"}
):
settings = DatabaseSettings()
assert settings.url == "postgresql+asyncpg://user:pass@localhost/db"
def test_invalid_url_raises(self) -> None:
"""Test that invalid database URL raises validation error."""
with (
patch.dict(os.environ, {"DATABASE_URL": "mysql://user:pass@localhost/db"}),
pytest.raises(ValidationError, match="PostgreSQL connection string"),
):
DatabaseSettings()
class TestRedisSettings:
"""Tests for RedisSettings."""
def test_default_url(self) -> None:
"""Test default Redis URL."""
with patch.dict(os.environ, {}, clear=True):
settings = RedisSettings()
assert settings.url == "redis://localhost:6379"
def test_custom_url(self) -> None:
"""Test custom Redis URL."""
with patch.dict(os.environ, {"REDIS_URL": "redis://redis:6380"}):
settings = RedisSettings()
assert settings.url == "redis://redis:6380"
def test_invalid_url_raises(self) -> None:
"""Test that invalid Redis URL raises validation error."""
with (
patch.dict(os.environ, {"REDIS_URL": "http://localhost:6379"}),
pytest.raises(ValidationError, match="redis://"),
):
RedisSettings()
class TestPolygonSettings:
"""Tests for PolygonSettings."""
def test_default_rpc_url(self) -> None:
"""Test default Polygon RPC URL."""
with patch.dict(os.environ, {}, clear=True):
settings = PolygonSettings()
assert settings.rpc_url == "https://polygon-rpc.com"
assert settings.fallback_rpc_url is None
def test_custom_urls(self) -> None:
"""Test custom Polygon RPC URLs."""
with patch.dict(
os.environ,
{
"POLYGON_RPC_URL": "https://alchemy.io/polygon",
"POLYGON_FALLBACK_RPC_URL": "https://backup.polygon.io",
},
):
settings = PolygonSettings()
assert settings.rpc_url == "https://alchemy.io/polygon"
assert settings.fallback_rpc_url == "https://backup.polygon.io"
def test_invalid_url_raises(self) -> None:
"""Test that invalid RPC URL raises validation error."""
with (
patch.dict(os.environ, {"POLYGON_RPC_URL": "ws://polygon.io"}),
pytest.raises(ValidationError, match="HTTP"),
):
PolygonSettings()
class TestPolymarketSettings:
"""Tests for PolymarketSettings."""
def test_default_ws_url(self) -> None:
"""Test default Polymarket WebSocket URL."""
with patch.dict(os.environ, {}, clear=True):
settings = PolymarketSettings()
assert "polymarket.com" in settings.ws_url
assert settings.api_key is None
def test_custom_api_key(self) -> None:
"""Test custom API key (secret)."""
with patch.dict(os.environ, {"POLYMARKET_API_KEY": "secret-key-123"}):
settings = PolymarketSettings()
assert settings.api_key is not None
assert settings.api_key.get_secret_value() == "secret-key-123"
def test_invalid_ws_url_raises(self) -> None:
"""Test that invalid WebSocket URL raises validation error."""
with (
patch.dict(os.environ, {"POLYMARKET_WS_URL": "http://polymarket.com"}),
pytest.raises(ValidationError, match="ws://"),
):
PolymarketSettings()
class TestDiscordSettings:
"""Tests for DiscordSettings."""
def test_disabled_by_default(self) -> None:
"""Test Discord is disabled when no webhook URL."""
with patch.dict(os.environ, {}, clear=True):
settings = DiscordSettings()
assert not settings.enabled
assert settings.webhook_url is None
def test_enabled_with_webhook(self) -> None:
"""Test Discord is enabled with webhook URL."""
with patch.dict(os.environ, {"DISCORD_WEBHOOK_URL": "https://discord.com/webhook/123"}):
settings = DiscordSettings()
assert settings.enabled
assert settings.webhook_url is not None
class TestTelegramSettings:
"""Tests for TelegramSettings."""
def test_disabled_by_default(self) -> None:
"""Test Telegram is disabled when no credentials."""
with patch.dict(os.environ, {}, clear=True):
settings = TelegramSettings()
assert not settings.enabled
def test_disabled_with_partial_config(self) -> None:
"""Test Telegram is disabled with only token or chat_id."""
with patch.dict(os.environ, {"TELEGRAM_BOT_TOKEN": "token123"}):
settings = TelegramSettings()
assert not settings.enabled
with patch.dict(os.environ, {"TELEGRAM_CHAT_ID": "12345"}):
settings = TelegramSettings()
assert not settings.enabled
def test_enabled_with_full_config(self) -> None:
"""Test Telegram is enabled with both token and chat_id."""
with patch.dict(
os.environ,
{
"TELEGRAM_BOT_TOKEN": "token123",
"TELEGRAM_CHAT_ID": "12345",
},
):
settings = TelegramSettings()
assert settings.enabled
class TestSettings:
"""Tests for main Settings class."""
def test_loads_with_required_vars(self) -> None:
"""Test settings load with required environment variables."""
with patch.dict(
os.environ,
{
"DATABASE_URL": "postgresql://user:pass@localhost/db",
"REDIS_URL": "redis://localhost:6379",
},
):
settings = Settings()
assert settings.database.url == "postgresql://user:pass@localhost/db"
assert settings.redis.url == "redis://localhost:6379"
def test_default_log_level(self) -> None:
"""Test default log level is INFO."""
with patch.dict(
os.environ,
{"DATABASE_URL": "postgresql://user:pass@localhost/db"},
):
settings = Settings()
assert settings.log_level == "INFO"
def test_custom_log_level(self) -> None:
"""Test custom log level."""
with patch.dict(
os.environ,
{
"DATABASE_URL": "postgresql://user:pass@localhost/db",
"LOG_LEVEL": "DEBUG",
},
):
settings = Settings()
assert settings.log_level == "DEBUG"
def test_invalid_log_level_raises(self) -> None:
"""Test invalid log level raises validation error."""
with (
patch.dict(
os.environ,
{
"DATABASE_URL": "postgresql://user:pass@localhost/db",
"LOG_LEVEL": "TRACE",
},
),
pytest.raises(ValidationError),
):
Settings()
def test_health_port_validation(self) -> None:
"""Test health port must be valid port number."""
with (
patch.dict(
os.environ,
{
"DATABASE_URL": "postgresql://user:pass@localhost/db",
"HEALTH_PORT": "99999",
},
),
pytest.raises(ValidationError, match="65535"),
):
Settings()
def test_get_logging_level(self) -> None:
"""Test get_logging_level returns numeric level."""
import logging
with patch.dict(
os.environ,
{
"DATABASE_URL": "postgresql://user:pass@localhost/db",
"LOG_LEVEL": "WARNING",
},
):
settings = Settings()
assert settings.get_logging_level() == logging.WARNING
def test_redacted_summary(self) -> None:
"""Test redacted_summary masks sensitive data."""
with patch.dict(
os.environ,
{
"DATABASE_URL": "postgresql://user:secretpass@localhost/db",
"REDIS_URL": "redis://localhost:6379",
},
):
settings = Settings()
summary = settings.redacted_summary()
# Database password should be redacted
db_url = summary["database_url"]
assert isinstance(db_url, str)
assert "secretpass" not in db_url
assert "***" in db_url
assert "user" in db_url
class TestGetSettings:
"""Tests for get_settings singleton."""
def test_returns_same_instance(self) -> None:
"""Test get_settings returns cached instance."""
with patch.dict(
os.environ,
{"DATABASE_URL": "postgresql://user:pass@localhost/db"},
):
settings1 = get_settings()
settings2 = get_settings()
assert settings1 is settings2
def test_clear_cache_allows_reload(self) -> None:
"""Test clear_settings_cache allows reloading settings."""
with patch.dict(
os.environ,
{
"DATABASE_URL": "postgresql://user:pass@localhost/db",
"LOG_LEVEL": "INFO",
},
):
settings1 = get_settings()
assert settings1.log_level == "INFO"
clear_settings_cache()
with patch.dict(
os.environ,
{
"DATABASE_URL": "postgresql://user:pass@localhost/db",
"LOG_LEVEL": "DEBUG",
},
):
settings2 = get_settings()
assert settings2.log_level == "DEBUG"
assert settings1 is not settings2
+214
View File
@@ -0,0 +1,214 @@
"""Tests for the CLI entry point."""
from __future__ import annotations
from unittest.mock import patch
import pytest
from polymarket_insider_tracker.__main__ import (
EXIT_CONFIG_ERROR,
EXIT_SUCCESS,
configure_logging,
create_parser,
main,
print_banner,
run_config_check,
validate_config,
)
class TestCreateParser:
"""Tests for argument parser creation."""
def test_parser_has_version(self):
"""Parser should have version flag."""
parser = create_parser()
with pytest.raises(SystemExit) as exc_info:
parser.parse_args(["--version"])
assert exc_info.value.code == 0
def test_parser_config_check(self):
"""Parser should accept --config-check flag."""
parser = create_parser()
args = parser.parse_args(["--config-check"])
assert args.config_check is True
def test_parser_log_level(self):
"""Parser should accept --log-level option."""
parser = create_parser()
args = parser.parse_args(["--log-level", "DEBUG"])
assert args.log_level == "DEBUG"
def test_parser_dry_run(self):
"""Parser should accept --dry-run flag."""
parser = create_parser()
args = parser.parse_args(["--dry-run"])
assert args.dry_run is True
def test_parser_health_port(self):
"""Parser should accept --health-port option."""
parser = create_parser()
args = parser.parse_args(["--health-port", "9090"])
assert args.health_port == 9090
def test_parser_default_values(self):
"""Parser should have correct defaults."""
parser = create_parser()
args = parser.parse_args([])
assert args.config_check is False
assert args.log_level is None
assert args.dry_run is False
assert args.health_port is None
class TestConfigureLogging:
"""Tests for logging configuration."""
def test_configure_logging_info(self):
"""Should configure logging at INFO level."""
configure_logging("INFO")
import logging
assert logging.getLogger().level == logging.INFO
def test_configure_logging_debug(self):
"""Should configure logging at DEBUG level."""
configure_logging("DEBUG")
import logging
assert logging.getLogger().level == logging.DEBUG
class TestPrintBanner:
"""Tests for banner printing."""
def test_banner_contains_app_name(self, capsys):
"""Banner should contain application name."""
print_banner()
captured = capsys.readouterr()
assert "Polymarket Insider Tracker" in captured.out
def test_banner_contains_version(self, capsys):
"""Banner should contain version."""
print_banner()
captured = capsys.readouterr()
assert "v0.1.0" in captured.out
class TestValidateConfig:
"""Tests for configuration validation."""
def test_validate_config_success(self, monkeypatch):
"""Should return settings on valid config."""
monkeypatch.setenv("DATABASE_URL", "postgresql://localhost/test")
settings = validate_config()
assert settings is not None
def test_validate_config_failure(self, monkeypatch, capsys):
"""Should return None on invalid config."""
# Clear any existing DATABASE_URL
monkeypatch.delenv("DATABASE_URL", raising=False)
settings = validate_config()
assert settings is None
captured = capsys.readouterr()
assert "Configuration validation failed" in captured.err
class TestRunConfigCheck:
"""Tests for config check mode."""
def test_config_check_prints_summary(self, monkeypatch, capsys):
"""Config check should print configuration summary."""
monkeypatch.setenv("DATABASE_URL", "postgresql://localhost/test")
settings = validate_config()
assert settings is not None
result = run_config_check(settings)
assert result == EXIT_SUCCESS
captured = capsys.readouterr()
assert "Configuration is valid!" in captured.out
assert "Configuration:" in captured.out
class TestMain:
"""Tests for main entry point."""
def test_main_with_config_check(self, monkeypatch):
"""Main should exit successfully with --config-check."""
monkeypatch.setenv("DATABASE_URL", "postgresql://localhost/test")
with pytest.raises(SystemExit) as exc_info:
main(["--config-check"])
assert exc_info.value.code == EXIT_SUCCESS
def test_main_with_invalid_config(self, monkeypatch):
"""Main should exit with config error on invalid config."""
monkeypatch.delenv("DATABASE_URL", raising=False)
with pytest.raises(SystemExit) as exc_info:
main([])
assert exc_info.value.code == EXIT_CONFIG_ERROR
def test_main_with_dry_run_and_config_check(self, monkeypatch):
"""Main should handle dry-run with config-check."""
monkeypatch.setenv("DATABASE_URL", "postgresql://localhost/test")
with pytest.raises(SystemExit) as exc_info:
main(["--config-check", "--dry-run"])
assert exc_info.value.code == EXIT_SUCCESS
@patch("polymarket_insider_tracker.__main__.run_pipeline")
@patch("polymarket_insider_tracker.__main__.asyncio.run")
def test_main_runs_pipeline(self, mock_asyncio_run, _mock_run_pipeline, monkeypatch):
"""Main should run pipeline when not in config-check mode."""
monkeypatch.setenv("DATABASE_URL", "postgresql://localhost/test")
mock_asyncio_run.return_value = EXIT_SUCCESS
with pytest.raises(SystemExit) as exc_info:
main([])
assert exc_info.value.code == EXIT_SUCCESS
mock_asyncio_run.assert_called_once()
class TestIntegration:
"""Integration tests for CLI invocation."""
def test_cli_help_option(self, capsys):
"""CLI should display help with -h option."""
with pytest.raises(SystemExit) as exc_info:
main(["-h"])
assert exc_info.value.code == 0
captured = capsys.readouterr()
assert "polymarket-insider-tracker" in captured.out
assert "--config-check" in captured.out
assert "--dry-run" in captured.out
assert "--log-level" in captured.out
def test_cli_version_option(self, capsys):
"""CLI should display version with --version option."""
with pytest.raises(SystemExit) as exc_info:
main(["--version"])
assert exc_info.value.code == 0
captured = capsys.readouterr()
assert "0.1.0" in captured.out
def test_cli_invalid_log_level(self, capsys):
"""CLI should reject invalid log level."""
with pytest.raises(SystemExit) as exc_info:
main(["--log-level", "INVALID"])
assert exc_info.value.code != 0
captured = capsys.readouterr()
assert "invalid choice" in captured.err
+394
View File
@@ -0,0 +1,394 @@
"""Tests for the main pipeline orchestrator."""
from __future__ import annotations
import asyncio
from datetime import UTC, datetime
from decimal import Decimal
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from polymarket_insider_tracker.config import Settings
from polymarket_insider_tracker.detector.models import FreshWalletSignal
from polymarket_insider_tracker.detector.scorer import SignalBundle
from polymarket_insider_tracker.ingestor.models import TradeEvent
from polymarket_insider_tracker.pipeline import Pipeline, PipelineState
from polymarket_insider_tracker.profiler.models import WalletProfile
@pytest.fixture
def mock_settings():
"""Create mock settings for testing."""
# Create nested mock objects
redis = MagicMock()
redis.url = "redis://localhost:6379"
database = MagicMock()
database.url = "postgresql+asyncpg://user:pass@localhost/db"
polygon = MagicMock()
polygon.rpc_url = "https://polygon-rpc.com"
polygon.fallback_rpc_url = None
polymarket = MagicMock()
polymarket.ws_url = "wss://ws-subscriptions-clob.polymarket.com/ws/market"
polymarket.api_key = None
discord = MagicMock()
discord.enabled = False
discord.webhook_url = None
telegram = MagicMock()
telegram.enabled = False
telegram.bot_token = None
telegram.chat_id = None
settings = MagicMock(spec=Settings)
settings.redis = redis
settings.database = database
settings.polygon = polygon
settings.polymarket = polymarket
settings.discord = discord
settings.telegram = telegram
settings.dry_run = True
return settings
@pytest.fixture
def sample_trade_event():
"""Create a sample trade event for testing."""
return TradeEvent(
trade_id="0x" + "a" * 64,
wallet_address="0x" + "b" * 40,
market_id="0x" + "c" * 64,
asset_id="asset_123",
side="BUY",
price=Decimal("0.65"),
size=Decimal("5000"),
timestamp=datetime.now(UTC),
outcome="Yes",
outcome_index=0,
event_title="Test Market",
market_slug="test-market",
)
@pytest.fixture
def sample_wallet_profile():
"""Create a sample wallet profile for testing."""
return WalletProfile(
address="0x" + "b" * 40,
nonce=2,
first_seen=datetime.now(UTC),
age_hours=1.5,
is_fresh=True,
total_tx_count=2,
matic_balance=Decimal("100"),
usdc_balance=Decimal("5000"),
fresh_threshold=5,
)
class TestPipelineState:
"""Tests for pipeline state management."""
def test_initial_state_is_stopped(self, mock_settings):
"""Pipeline should start in stopped state."""
pipeline = Pipeline(mock_settings)
assert pipeline.state == PipelineState.STOPPED
def test_is_running_property(self, mock_settings):
"""is_running property should reflect state."""
pipeline = Pipeline(mock_settings)
assert not pipeline.is_running
pipeline._state = PipelineState.RUNNING
assert pipeline.is_running
class TestPipelineStats:
"""Tests for pipeline statistics."""
def test_initial_stats(self, mock_settings):
"""Pipeline should have zero stats initially."""
pipeline = Pipeline(mock_settings)
stats = pipeline.stats
assert stats.started_at is None
assert stats.trades_processed == 0
assert stats.signals_generated == 0
assert stats.alerts_sent == 0
assert stats.errors == 0
class TestPipelineInitialization:
"""Tests for pipeline initialization."""
def test_dry_run_from_settings(self, mock_settings):
"""Pipeline should use dry_run from settings by default."""
mock_settings.dry_run = True
pipeline = Pipeline(mock_settings)
assert pipeline._dry_run is True
mock_settings.dry_run = False
pipeline = Pipeline(mock_settings)
assert pipeline._dry_run is False
def test_dry_run_override(self, mock_settings):
"""Pipeline should allow overriding dry_run."""
mock_settings.dry_run = False
pipeline = Pipeline(mock_settings, dry_run=True)
assert pipeline._dry_run is True
def test_uses_get_settings_when_none_provided(self):
"""Pipeline should call get_settings if no settings provided."""
with patch("polymarket_insider_tracker.pipeline.get_settings") as mock_get:
mock_get.return_value = MagicMock(spec=Settings)
mock_get.return_value.dry_run = False
Pipeline()
mock_get.assert_called_once()
class TestBuildAlertChannels:
"""Tests for alert channel building."""
def test_no_channels_when_none_enabled(self, mock_settings):
"""Should return empty list when no channels enabled."""
mock_settings.discord.enabled = False
mock_settings.telegram.enabled = False
pipeline = Pipeline(mock_settings)
channels = pipeline._build_alert_channels()
assert channels == []
def test_discord_channel_when_enabled(self, mock_settings):
"""Should add Discord channel when enabled."""
mock_settings.discord.enabled = True
mock_settings.discord.webhook_url = MagicMock()
mock_settings.discord.webhook_url.get_secret_value.return_value = (
"https://discord.com/webhook"
)
pipeline = Pipeline(mock_settings)
channels = pipeline._build_alert_channels()
assert len(channels) == 1
assert channels[0].name == "discord"
def test_telegram_channel_when_enabled(self, mock_settings):
"""Should add Telegram channel when enabled."""
mock_settings.telegram.enabled = True
mock_settings.telegram.bot_token = MagicMock()
mock_settings.telegram.bot_token.get_secret_value.return_value = "bot_token"
mock_settings.telegram.chat_id = "chat_123"
pipeline = Pipeline(mock_settings)
channels = pipeline._build_alert_channels()
assert len(channels) == 1
assert channels[0].name == "telegram"
def test_both_channels_when_both_enabled(self, mock_settings):
"""Should add both channels when both enabled."""
mock_settings.discord.enabled = True
mock_settings.discord.webhook_url = MagicMock()
mock_settings.discord.webhook_url.get_secret_value.return_value = (
"https://discord.com/webhook"
)
mock_settings.telegram.enabled = True
mock_settings.telegram.bot_token = MagicMock()
mock_settings.telegram.bot_token.get_secret_value.return_value = "bot_token"
mock_settings.telegram.chat_id = "chat_123"
pipeline = Pipeline(mock_settings)
channels = pipeline._build_alert_channels()
assert len(channels) == 2
class TestOnTrade:
"""Tests for trade event processing."""
@pytest.mark.asyncio
async def test_on_trade_increments_stats(self, mock_settings, sample_trade_event):
"""Processing a trade should increment stats."""
pipeline = Pipeline(mock_settings)
pipeline._fresh_wallet_detector = AsyncMock(return_value=None)
pipeline._size_anomaly_detector = AsyncMock(return_value=None)
await pipeline._on_trade(sample_trade_event)
assert pipeline.stats.trades_processed == 1
assert pipeline.stats.last_trade_time is not None
@pytest.mark.asyncio
async def test_on_trade_runs_detectors_in_parallel(self, mock_settings, sample_trade_event):
"""Detectors should run in parallel."""
pipeline = Pipeline(mock_settings)
pipeline._fresh_wallet_detector = AsyncMock()
pipeline._size_anomaly_detector = AsyncMock()
# Make detectors take some time
async def slow_detect(*_args):
await asyncio.sleep(0.1)
return None
pipeline._fresh_wallet_detector.analyze = slow_detect
pipeline._size_anomaly_detector.analyze = slow_detect
start = asyncio.get_event_loop().time()
await pipeline._on_trade(sample_trade_event)
elapsed = asyncio.get_event_loop().time() - start
# Should complete in ~0.1s not ~0.2s
assert elapsed < 0.15
@pytest.mark.asyncio
async def test_on_trade_handles_detector_errors(self, mock_settings, sample_trade_event):
"""Should handle detector errors gracefully."""
pipeline = Pipeline(mock_settings)
pipeline._fresh_wallet_detector = MagicMock()
pipeline._fresh_wallet_detector.analyze = AsyncMock(side_effect=Exception("Detector error"))
pipeline._size_anomaly_detector = MagicMock()
pipeline._size_anomaly_detector.analyze = AsyncMock(return_value=None)
# Should not raise
await pipeline._on_trade(sample_trade_event)
# Should still increment trades processed
assert pipeline.stats.trades_processed == 1
@pytest.mark.asyncio
async def test_on_trade_calls_score_and_alert_when_signals(
self, mock_settings, sample_trade_event, sample_wallet_profile
):
"""Should call score_and_alert when signals are detected."""
pipeline = Pipeline(mock_settings)
# Create a signal
fresh_signal = FreshWalletSignal(
trade_event=sample_trade_event,
wallet_profile=sample_wallet_profile,
confidence=0.8,
factors={"base": 0.5, "brand_new": 0.2},
)
pipeline._fresh_wallet_detector = MagicMock()
pipeline._fresh_wallet_detector.analyze = AsyncMock(return_value=fresh_signal)
pipeline._size_anomaly_detector = MagicMock()
pipeline._size_anomaly_detector.analyze = AsyncMock(return_value=None)
# Mock score_and_alert
pipeline._score_and_alert = AsyncMock()
await pipeline._on_trade(sample_trade_event)
# Should call score_and_alert with the bundle
pipeline._score_and_alert.assert_called_once()
bundle = pipeline._score_and_alert.call_args[0][0]
assert bundle.fresh_wallet_signal == fresh_signal
assert pipeline.stats.signals_generated == 1
class TestScoreAndAlert:
"""Tests for scoring and alerting."""
@pytest.mark.asyncio
async def test_dry_run_skips_dispatch(
self, mock_settings, sample_trade_event, sample_wallet_profile
):
"""Dry run should skip actual alert dispatch."""
mock_settings.dry_run = True
pipeline = Pipeline(mock_settings)
# Create mock components
pipeline._risk_scorer = MagicMock()
pipeline._risk_scorer.assess = AsyncMock(
return_value=MagicMock(
should_alert=True,
wallet_address="0x" + "b" * 40,
weighted_score=0.85,
)
)
pipeline._alert_formatter = MagicMock()
pipeline._alert_dispatcher = MagicMock()
pipeline._alert_dispatcher.dispatch = AsyncMock()
bundle = SignalBundle(
trade_event=sample_trade_event,
fresh_wallet_signal=FreshWalletSignal(
trade_event=sample_trade_event,
wallet_profile=sample_wallet_profile,
confidence=0.8,
factors={},
),
)
await pipeline._score_and_alert(bundle)
# Dispatcher should NOT be called in dry run
pipeline._alert_dispatcher.dispatch.assert_not_called()
@pytest.mark.asyncio
async def test_no_alert_when_below_threshold(self, mock_settings, sample_trade_event):
"""Should not alert when below threshold."""
pipeline = Pipeline(mock_settings)
# Create mock components
pipeline._risk_scorer = MagicMock()
pipeline._risk_scorer.assess = AsyncMock(
return_value=MagicMock(
should_alert=False,
weighted_score=0.4,
)
)
pipeline._alert_formatter = MagicMock()
pipeline._alert_dispatcher = MagicMock()
bundle = SignalBundle(trade_event=sample_trade_event)
await pipeline._score_and_alert(bundle)
# Formatter should NOT be called
pipeline._alert_formatter.format.assert_not_called()
class TestPipelineLifecycle:
"""Tests for pipeline lifecycle methods."""
@pytest.mark.asyncio
async def test_cannot_start_when_not_stopped(self, mock_settings):
"""Should raise error when starting non-stopped pipeline."""
pipeline = Pipeline(mock_settings)
pipeline._state = PipelineState.RUNNING
with pytest.raises(RuntimeError, match="Cannot start pipeline"):
await pipeline.start()
@pytest.mark.asyncio
async def test_stop_when_already_stopped(self, mock_settings):
"""Stop should be no-op when already stopped."""
pipeline = Pipeline(mock_settings)
assert pipeline.state == PipelineState.STOPPED
# Should not raise
await pipeline.stop()
assert pipeline.state == PipelineState.STOPPED
class TestPipelineContextManager:
"""Tests for async context manager."""
@pytest.mark.asyncio
async def test_context_manager_calls_start_and_stop(self, mock_settings):
"""Context manager should call start and stop."""
pipeline = Pipeline(mock_settings)
pipeline.start = AsyncMock()
pipeline.stop = AsyncMock()
async with pipeline:
pipeline.start.assert_called_once()
pipeline.stop.assert_called_once()
+319
View File
@@ -0,0 +1,319 @@
"""Tests for the graceful shutdown handler."""
from __future__ import annotations
import asyncio
import signal
import sys
from unittest.mock import MagicMock, patch
import pytest
from polymarket_insider_tracker.shutdown import (
DEFAULT_SHUTDOWN_TIMEOUT,
SHUTDOWN_SIGNALS,
GracefulShutdown,
run_with_graceful_shutdown,
)
class TestGracefulShutdownInit:
"""Tests for GracefulShutdown initialization."""
def test_default_timeout(self) -> None:
"""Should use default timeout when not specified."""
shutdown = GracefulShutdown()
assert shutdown.timeout == DEFAULT_SHUTDOWN_TIMEOUT
def test_custom_timeout(self) -> None:
"""Should accept custom timeout."""
shutdown = GracefulShutdown(timeout=60.0)
assert shutdown.timeout == 60.0
def test_initial_state(self) -> None:
"""Should start in non-shutdown state."""
shutdown = GracefulShutdown()
assert shutdown.is_shutdown_requested is False
assert shutdown.is_force_exit_requested is False
class TestRequestShutdown:
"""Tests for programmatic shutdown requests."""
async def test_request_shutdown_sets_flag(self) -> None:
"""Should set shutdown requested flag."""
shutdown = GracefulShutdown()
shutdown.request_shutdown()
assert shutdown.is_shutdown_requested is True
async def test_request_shutdown_sets_event(self) -> None:
"""Should set the shutdown event when called."""
shutdown = GracefulShutdown()
shutdown._shutdown_event = asyncio.Event()
shutdown.request_shutdown()
assert shutdown._shutdown_event.is_set()
async def test_request_shutdown_idempotent(self) -> None:
"""Multiple requests should be idempotent."""
shutdown = GracefulShutdown()
shutdown._shutdown_event = asyncio.Event()
shutdown.request_shutdown()
shutdown.request_shutdown()
shutdown.request_shutdown()
assert shutdown.is_shutdown_requested is True
class TestWait:
"""Tests for waiting for shutdown."""
async def test_wait_blocks_until_shutdown(self) -> None:
"""Wait should block until shutdown is requested."""
shutdown = GracefulShutdown()
async def request_after_delay() -> None:
await asyncio.sleep(0.1)
shutdown.request_shutdown()
asyncio.create_task(request_after_delay())
# Should complete after the delay
await asyncio.wait_for(shutdown.wait(), timeout=1.0)
assert shutdown.is_shutdown_requested is True
async def test_wait_with_timeout_returns_true_on_shutdown(self) -> None:
"""wait_with_timeout should return True when shutdown is requested."""
shutdown = GracefulShutdown(timeout=1.0)
async def request_after_delay() -> None:
await asyncio.sleep(0.05)
shutdown.request_shutdown()
asyncio.create_task(request_after_delay())
result = await shutdown.wait_with_timeout()
assert result is True
async def test_wait_with_timeout_returns_false_on_timeout(self) -> None:
"""wait_with_timeout should return False when timeout occurs."""
shutdown = GracefulShutdown(timeout=0.05)
# Don't request shutdown, let it timeout
result = await shutdown.wait_with_timeout()
assert result is False
class TestSignalHandlers:
"""Tests for signal handler installation and removal."""
async def test_install_signal_handlers_creates_event(self) -> None:
"""Installing handlers should create the shutdown event."""
shutdown = GracefulShutdown()
shutdown.install_signal_handlers()
try:
assert shutdown._shutdown_event is not None
assert not shutdown._shutdown_event.is_set()
finally:
shutdown.remove_signal_handlers()
@pytest.mark.skipif(sys.platform == "win32", reason="Unix-specific test")
async def test_unix_signal_handler_installed(self) -> None:
"""On Unix, should install loop signal handlers."""
shutdown = GracefulShutdown()
with patch.object(asyncio.get_running_loop(), "add_signal_handler") as mock_add:
shutdown.install_signal_handlers()
# Should have been called for both SIGTERM and SIGINT
assert mock_add.call_count >= 1
shutdown.remove_signal_handlers()
@pytest.mark.skipif(sys.platform != "win32", reason="Windows-specific test")
async def test_windows_signal_handler_installed(self) -> None:
"""On Windows, should install signal.signal handlers."""
shutdown = GracefulShutdown()
with patch("signal.signal") as mock_signal:
shutdown.install_signal_handlers()
# Should have been called for available signals
assert mock_signal.call_count >= 1
shutdown.remove_signal_handlers()
class TestHandleSignal:
"""Tests for signal handling behavior."""
async def test_first_signal_sets_shutdown_event(self) -> None:
"""First signal should set the shutdown event."""
shutdown = GracefulShutdown()
shutdown._shutdown_event = asyncio.Event()
shutdown._handle_signal(signal.SIGTERM)
assert shutdown.is_shutdown_requested is True
assert shutdown._shutdown_event.is_set()
assert shutdown.is_force_exit_requested is False
async def test_second_signal_force_exits(self) -> None:
"""Second signal should trigger force exit."""
shutdown = GracefulShutdown()
shutdown._shutdown_event = asyncio.Event()
shutdown._shutdown_requested = True # First signal already received
with pytest.raises(SystemExit) as exc_info:
shutdown._handle_signal(signal.SIGTERM)
assert exc_info.value.code == 128 + signal.SIGTERM.value
assert shutdown.is_force_exit_requested is True
class TestCleanupCallbacks:
"""Tests for cleanup callback registration and execution."""
async def test_register_sync_callback(self) -> None:
"""Should register sync cleanup callbacks."""
shutdown = GracefulShutdown()
callback = MagicMock()
shutdown.register_cleanup(callback)
assert callback in shutdown._cleanup_callbacks
async def test_run_sync_cleanup_callback(self) -> None:
"""Should run sync cleanup callbacks."""
shutdown = GracefulShutdown()
callback = MagicMock()
shutdown.register_cleanup(callback)
await shutdown.run_cleanup_callbacks()
callback.assert_called_once()
async def test_run_async_cleanup_callback(self) -> None:
"""Should run async cleanup callbacks."""
shutdown = GracefulShutdown()
called = False
async def async_callback() -> None:
nonlocal called
called = True
shutdown.register_cleanup(async_callback)
await shutdown.run_cleanup_callbacks()
assert called is True
async def test_cleanup_callback_error_logged(self) -> None:
"""Cleanup callback errors should be logged, not raised."""
shutdown = GracefulShutdown()
def failing_callback() -> None:
raise ValueError("Cleanup failed")
shutdown.register_cleanup(failing_callback)
# Should not raise
await shutdown.run_cleanup_callbacks()
class TestAsyncContextManager:
"""Tests for async context manager protocol."""
async def test_context_manager_installs_handlers(self) -> None:
"""Entering context should install signal handlers."""
shutdown = GracefulShutdown()
async with shutdown:
assert shutdown._shutdown_event is not None
assert shutdown._loop is not None
async def test_context_manager_removes_handlers(self) -> None:
"""Exiting context should remove signal handlers."""
shutdown = GracefulShutdown()
with patch.object(shutdown, "remove_signal_handlers") as mock_remove:
async with shutdown:
pass
mock_remove.assert_called_once()
async def test_context_manager_runs_cleanup(self) -> None:
"""Exiting context should run cleanup callbacks."""
shutdown = GracefulShutdown()
callback = MagicMock()
shutdown.register_cleanup(callback)
async with shutdown:
pass
callback.assert_called_once()
class TestRunWithGracefulShutdown:
"""Tests for the run_with_graceful_shutdown helper."""
async def test_runs_coroutine_to_completion(self) -> None:
"""Should run the coroutine to completion if no shutdown."""
result = []
async def my_coro() -> None:
result.append("done")
await run_with_graceful_shutdown(my_coro(), timeout=1.0)
assert result == ["done"]
async def test_stops_on_shutdown_signal(self) -> None:
"""Should stop when shutdown is signaled."""
result = []
async def long_running() -> None:
try:
await asyncio.sleep(10.0)
except asyncio.CancelledError:
result.append("cancelled")
raise
# Create a wrapper that will trigger shutdown
async def run_with_interrupt() -> None:
async def trigger_shutdown() -> None:
await asyncio.sleep(0.1)
# We can't easily trigger a signal in tests, so we test the task
# completion path instead
pass
# Run both tasks
task = asyncio.create_task(long_running())
asyncio.create_task(trigger_shutdown())
await asyncio.sleep(0.05)
task.cancel()
with pytest.raises(asyncio.CancelledError):
await task
result.append("finished")
await run_with_interrupt()
assert "finished" in result
class TestShutdownSignals:
"""Tests for shutdown signal configuration."""
def test_shutdown_signals_includes_sigterm(self) -> None:
"""SHUTDOWN_SIGNALS should include SIGTERM."""
assert signal.SIGTERM in SHUTDOWN_SIGNALS
def test_shutdown_signals_includes_sigint(self) -> None:
"""SHUTDOWN_SIGNALS should include SIGINT."""
assert signal.SIGINT in SHUTDOWN_SIGNALS
Generated
+27
View File
@@ -126,6 +126,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/fb/76/641ae371508676492379f16e2fa48f4e2c11741bd63c48be4b12a6b09cba/aiosignal-1.4.0-py3-none-any.whl", hash = "sha256:053243f8b92b990551949e63930a839ff0cf0b0ebbe0597b0f3fb19e1a0fe82e", size = 7490, upload-time = "2025-07-03T22:54:42.156Z" },
]
[[package]]
name = "aiosqlite"
version = "0.22.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/4e/8a/64761f4005f17809769d23e518d915db74e6310474e733e3593cfc854ef1/aiosqlite-0.22.1.tar.gz", hash = "sha256:043e0bd78d32888c0a9ca90fc788b38796843360c855a7262a532813133a0650", size = 14821, upload-time = "2025-12-23T19:25:43.997Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/00/b7/e3bf5133d697a08128598c8d0abc5e16377b51465a33756de24fa7dee953/aiosqlite-0.22.1-py3-none-any.whl", hash = "sha256:21c002eb13823fad740196c5a2e9d8e62f6243bd9e7e4a1f87fb5e44ecb4fceb", size = 17405, upload-time = "2025-12-23T19:25:42.139Z" },
]
[[package]]
name = "alembic"
version = "1.17.2"
@@ -1538,6 +1547,7 @@ dependencies = [
{ name = "prometheus-client" },
{ name = "py-clob-client" },
{ name = "pydantic" },
{ name = "pydantic-settings" },
{ name = "python-dotenv" },
{ name = "redis" },
{ name = "scikit-learn" },
@@ -1548,6 +1558,7 @@ dependencies = [
[package.optional-dependencies]
dev = [
{ name = "aiosqlite" },
{ name = "mypy" },
{ name = "pre-commit" },
{ name = "pytest" },
@@ -1559,6 +1570,7 @@ dev = [
[package.metadata]
requires-dist = [
{ name = "aiohttp", specifier = ">=3.9.0" },
{ name = "aiosqlite", marker = "extra == 'dev'", specifier = ">=0.19.0" },
{ name = "alembic", specifier = ">=1.13.0" },
{ name = "httpx", specifier = ">=0.25.0" },
{ name = "mypy", marker = "extra == 'dev'", specifier = ">=1.7.0" },
@@ -1567,6 +1579,7 @@ requires-dist = [
{ name = "prometheus-client", specifier = ">=0.19.0" },
{ name = "py-clob-client", specifier = ">=0.1.0" },
{ name = "pydantic", specifier = ">=2.0.0" },
{ name = "pydantic-settings", specifier = ">=2.0.0" },
{ name = "pytest", marker = "extra == 'dev'", specifier = ">=7.0.0" },
{ name = "pytest-asyncio", marker = "extra == 'dev'", specifier = ">=0.21.0" },
{ name = "pytest-cov", marker = "extra == 'dev'", specifier = ">=4.0.0" },
@@ -1892,6 +1905,20 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/36/c7/cfc8e811f061c841d7990b0201912c3556bfeb99cdcb7ed24adc8d6f8704/pydantic_core-2.41.5-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:56121965f7a4dc965bff783d70b907ddf3d57f6eba29b6d2e5dabfaf07799c51", size = 2145302, upload-time = "2025-11-04T13:43:46.64Z" },
]
[[package]]
name = "pydantic-settings"
version = "2.12.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "pydantic" },
{ name = "python-dotenv" },
{ name = "typing-inspection" },
]
sdist = { url = "https://files.pythonhosted.org/packages/43/4b/ac7e0aae12027748076d72a8764ff1c9d82ca75a7a52622e67ed3f765c54/pydantic_settings-2.12.0.tar.gz", hash = "sha256:005538ef951e3c2a68e1c08b292b5f2e71490def8589d4221b95dab00dafcfd0", size = 194184, upload-time = "2025-11-10T14:25:47.013Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/c1/60/5d4751ba3f4a40a6891f24eec885f51afd78d208498268c734e256fb13c4/pydantic_settings-2.12.0-py3-none-any.whl", hash = "sha256:fddb9fd99a5b18da837b29710391e945b1e30c135477f484084ee513adb93809", size = 51880, upload-time = "2025-11-10T14:25:45.546Z" },
]
[[package]]
name = "pygments"
version = "2.19.2"