mirror of
https://github.com/mauricioabh/arbpulse.git
synced 2026-07-27 23:57:43 +00:00
feat(observability): gate Sentry tracing spans behind SENTRY_TRACING
Spans are now created and exported to Sentry only when SENTRY_TRACING is enabled (default off), keeping 24/7 operation within the free-tier span quota. Error monitoring stays always-on. Includes OpenSpec change gate-sentry-spans (proposal, design, specs, tasks). Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-07-16
|
||||
@@ -0,0 +1,34 @@
|
||||
## Context
|
||||
|
||||
`src/instrumentation/otel.ts` builds a tracer that, when `SENTRY_DSN` is set, registers a `NodeTracerProvider` with `SentrySpanProcessor`/`SentryPropagator`, so every `withSpan(...)` call on the hot path creates and exports a span to Sentry. `src/instrumentation/sentry.ts` sets `tracesSampleRate` unconditionally. Running 24/7 on a VPS multiplies span volume against Sentry's free-tier quota (5M spans/month), while error events stay small. We want tracing to remain a debugging option but be off by default.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Add a single boolean env var (`SENTRY_TRACING`, default `false`) that turns span creation/export on or off.
|
||||
- Default behavior emits zero spans and zero hot-path span allocation.
|
||||
- Preserve error monitoring exactly as-is regardless of the flag.
|
||||
- No dependency changes; no API/wire changes.
|
||||
|
||||
**Non-Goals:**
|
||||
- Removing OpenTelemetry or the `withSpan` abstraction.
|
||||
- Making `tracesSampleRate` itself env-configurable (kept as current 0.1 prod / 1 dev when enabled).
|
||||
- Changing which operations are wrapped.
|
||||
|
||||
## Decisions
|
||||
|
||||
- **Boolean gate over sample-rate gate.** A boolean `SENTRY_TRACING` that makes the tracer a no-op avoids hot-path span allocation entirely when off (aligns with the "no unnecessary alloc on the hot path" convention). A `tracesSampleRate=0` approach would still register the provider and create spans that are dropped later — more overhead. Alternative considered and rejected.
|
||||
- **Gate lives in `otel.ts` init.** `initOpenTelemetry()` returns the no-op `trace.getTracer("arbpulse")` unless both `SENTRY_DSN` and `SENTRY_TRACING` are truthy. This keeps `withSpan`'s signature and all call sites unchanged; its `catch` still calls `Sentry.captureException`, so errors are captured even with a no-op span.
|
||||
- **Conditional `tracesSampleRate` in `sentry.ts`.** Only set `tracesSampleRate` when tracing is enabled; omit it otherwise so Sentry performance is fully off. A shared helper `isTracingEnabled()` reads the flag so both files agree.
|
||||
- **Parsing.** Reuse the existing truthy convention (`"1"` or `"true"`, case-insensitive) used by `config.ts`'s `bool()` for consistency; implement locally in the instrumentation module since it initializes before `config` import.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- [Someone enables tracing in prod and hits the quota] → Documented in `.env.example`/README that it is off by default and why; enabling is an explicit opt-in.
|
||||
- [Flag read in two files could drift] → Centralize in one `isTracingEnabled()` helper exported from the instrumentation layer.
|
||||
- [No-op tracer still wraps hot path in a promise] → Accepted: `withSpan` remains async as today; the change only removes span export, matching prior behavior when `SENTRY_DSN` was unset.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
- Deploy with `SENTRY_TRACING` unset → tracing off (safe default), errors still reported.
|
||||
- To debug, set `SENTRY_TRACING=true` (with `SENTRY_DSN`) temporarily; unset to roll back. No data migration.
|
||||
@@ -0,0 +1,25 @@
|
||||
## Why
|
||||
|
||||
Arb Pulse is moving to a 24/7 VPS (Hetzner), where the engine runs continuously and emits OpenTelemetry spans on the hot path (`ws.message` → `orderbook.process` → `arbitrage.evaluate` → `sse.broadcast`). At that volume the app can approach or exceed Sentry's free-tier span quota (5M spans/month), while error monitoring stays well within limits. We want to keep tracing available for debugging but off by default so continuous operation is free-tier safe.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Introduce an environment variable (`SENTRY_TRACING`, boolean, default `false`) that gates OpenTelemetry span creation and export to Sentry.
|
||||
- When `SENTRY_TRACING` is disabled (default), the tracer is a no-op: no spans are created or exported, and no hot-path span allocation occurs. Error capture (`Sentry.captureException`, `unhandledRejection`, `uncaughtException`) remains fully active.
|
||||
- When `SENTRY_TRACING` is enabled **and** `SENTRY_DSN` is set, spans are created and exported to Sentry as today, and `tracesSampleRate` is applied.
|
||||
- Document the new variable in `.env.example` and `README.md`.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `observability`: error monitoring and optional distributed tracing behavior — how Sentry error capture is always on, and how OpenTelemetry span emission is conditioned on an environment flag.
|
||||
|
||||
### Modified Capabilities
|
||||
<!-- None: no existing specs under openspec/specs/. -->
|
||||
|
||||
## Impact
|
||||
|
||||
- Code: `src/instrumentation/otel.ts` (gate span provider/export), `src/instrumentation/sentry.ts` (conditional `tracesSampleRate`), possibly `src/instrumentation/index.ts`.
|
||||
- Config/docs: `.env.example`, `README.md` (observability section + env var table).
|
||||
- Dependencies: none added or removed (OTel packages stay for the enabled path).
|
||||
- Behavior: default runtime emits zero spans; error monitoring unchanged. No API or wire-format changes.
|
||||
@@ -0,0 +1,43 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Error monitoring is always active
|
||||
|
||||
The system SHALL report runtime errors to Sentry whenever `SENTRY_DSN` is configured, independent of any tracing configuration. This includes REST, WebSocket feed, and SSE errors captured via `Sentry.captureException`, plus process-level `unhandledRejection` and `uncaughtException` handlers.
|
||||
|
||||
#### Scenario: Error captured with DSN set
|
||||
|
||||
- **WHEN** `SENTRY_DSN` is set and a handled error occurs on the WebSocket, REST, or SSE path
|
||||
- **THEN** the error is sent to Sentry as an error event
|
||||
|
||||
#### Scenario: No DSN configured
|
||||
|
||||
- **WHEN** `SENTRY_DSN` is empty or unset
|
||||
- **THEN** Sentry is not initialized and no events are sent, and the application continues running normally
|
||||
|
||||
### Requirement: Tracing spans are gated by an environment flag
|
||||
|
||||
The system SHALL only create and export OpenTelemetry spans when the `SENTRY_TRACING` environment variable is enabled AND `SENTRY_DSN` is set. The flag SHALL default to disabled so that continuous 24/7 operation emits zero spans by default.
|
||||
|
||||
#### Scenario: Tracing disabled by default
|
||||
|
||||
- **WHEN** `SENTRY_TRACING` is unset or falsy (default)
|
||||
- **THEN** the tracer is a no-op, no spans are created on the hot path, and no spans are exported to Sentry
|
||||
|
||||
#### Scenario: Tracing enabled with DSN
|
||||
|
||||
- **WHEN** `SENTRY_TRACING` is truthy and `SENTRY_DSN` is set
|
||||
- **THEN** hot-path spans (`ws.message`, `orderbook.process`, `arbitrage.evaluate`, `sse.broadcast`) are created and exported to Sentry, and `tracesSampleRate` is applied
|
||||
|
||||
#### Scenario: Tracing enabled without DSN
|
||||
|
||||
- **WHEN** `SENTRY_TRACING` is truthy but `SENTRY_DSN` is empty or unset
|
||||
- **THEN** the tracer is a no-op and no spans are exported, since export requires a DSN
|
||||
|
||||
### Requirement: Error capture is preserved when tracing is disabled
|
||||
|
||||
The system SHALL continue to report exceptions thrown inside hot-path work wrappers to Sentry even when tracing is disabled, so disabling spans never reduces error visibility.
|
||||
|
||||
#### Scenario: Exception in hot-path wrapper with tracing off
|
||||
|
||||
- **WHEN** `SENTRY_TRACING` is disabled and code wrapped by the hot-path span helper throws
|
||||
- **THEN** the exception is still captured and reported to Sentry (as an error), and re-thrown to preserve existing control flow
|
||||
@@ -0,0 +1,16 @@
|
||||
## 1. Tracing gate
|
||||
|
||||
- [x] 1.1 Add `isTracingEnabled()` helper (reads `SENTRY_TRACING`, truthy = `"1"`/`"true"` case-insensitive) exported from the instrumentation layer
|
||||
- [x] 1.2 In `src/instrumentation/otel.ts`, return the no-op tracer unless both `SENTRY_DSN` and `SENTRY_TRACING` are truthy (keep `withSpan` signature and its `Sentry.captureException` in catch)
|
||||
- [x] 1.3 In `src/instrumentation/sentry.ts`, only set `tracesSampleRate` when tracing is enabled (omit it otherwise)
|
||||
|
||||
## 2. Documentation
|
||||
|
||||
- [x] 2.1 Add `SENTRY_TRACING` (default `false`) to `.env.example` with a note it gates spans for free-tier safety
|
||||
- [x] 2.2 Update the README observability section to describe the flag and default-off behavior
|
||||
|
||||
## 3. Verification
|
||||
|
||||
- [x] 3.1 `npm run typecheck` passes
|
||||
- [x] 3.2 `npm test` passes
|
||||
- [x] 3.3 Manually confirm: with `SENTRY_TRACING` unset, no spans are emitted; error capture path unaffected
|
||||
@@ -0,0 +1,20 @@
|
||||
schema: spec-driven
|
||||
|
||||
# Project context (optional)
|
||||
# This is shown to AI when creating artifacts.
|
||||
# Add your tech stack, conventions, style guides, domain knowledge, etc.
|
||||
# Example:
|
||||
# context: |
|
||||
# Tech stack: TypeScript, React, Node.js
|
||||
# We use conventional commits
|
||||
# Domain: e-commerce platform
|
||||
|
||||
# Per-artifact rules (optional)
|
||||
# Add custom rules for specific artifacts.
|
||||
# Example:
|
||||
# rules:
|
||||
# proposal:
|
||||
# - Keep proposals under 500 words
|
||||
# - Always include a "Non-goals" section
|
||||
# tasks:
|
||||
# - Break tasks into chunks of max 2 hours
|
||||
Reference in New Issue
Block a user