Initial commit: fx_quant_engine and options_quant_engine scaffold

This commit is contained in:
Pramit Dutta
2026-03-30 17:20:45 +05:30
commit bfd32becc7
114 changed files with 4974 additions and 0 deletions
+53
View File
@@ -0,0 +1,53 @@
name: fx-quant-engine-ci
on:
push:
paths:
- "fx_quant_engine/**"
- "config/**"
- "tests/**"
- "scripts/**"
- "pyproject.toml"
pull_request:
paths:
- "fx_quant_engine/**"
- "config/**"
- "tests/**"
- "scripts/**"
- "pyproject.toml"
workflow_dispatch:
jobs:
lint-test-build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -e '.[dev]'
- name: Lint
run: ruff check .
- name: Test
run: pytest
- name: Generate sample run artifact
run: python scripts/run_engine.py
- name: Upload run artifacts
uses: actions/upload-artifact@v4
with:
name: fx-engine-runs
path: examples/runs/*.json
if-no-files-found: warn
retention-days: 30
+44
View File
@@ -0,0 +1,44 @@
name: fx-quant-engine-release
on:
push:
tags:
- "v*"
workflow_dispatch:
jobs:
build-release-artifacts:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install build tools
run: |
python -m pip install --upgrade pip
pip install build twine
- name: Build wheel and sdist
run: python -m build
- name: Verify artifacts
run: twine check dist/*
- name: Upload release distribution artifacts
uses: actions/upload-artifact@v4
with:
name: fx-quant-engine-dist
path: dist/*
retention-days: 90
- name: Create GitHub release
uses: softprops/action-gh-release@v2
with:
files: dist/*
generate_release_notes: true
+32
View File
@@ -0,0 +1,32 @@
# macOS
.DS_Store
# Python
__pycache__/
*.py[cod]
*.pyo
*.pyd
*.so
*.egg-info/
*.egg
.pytest_cache/
.coverage
htmlcov/
.mypy_cache/
.ruff_cache/
# Virtual environments
.venv/
venv/
# Build artifacts
build/
dist/
# Local run artifacts
examples/runs/
options_quant_engine/examples/runs/
# IDE
.vscode/
.idea/
+193
View File
@@ -0,0 +1,193 @@
# fx_quant_engine
A professional, modular, explainable, and production-ready starter repository for an FX quant signal engine focused on India-first constraints with global extensibility.
## What This Engine Does
- Ingests FX/macro/rate data from multiple pluggable adapters
- Uses configurable source priority and fallback logic per asset
- Engineers economically meaningful features through a registry-based pipeline
- Detects interpretable market regimes
- Generates directional and relative-value FX signals
- Separates signal strength from confidence
- Applies risk overlays and outputs action recommendations
- Supports evaluation and realistic backtest scaffolding
- Exports integration hooks for cross-engine multi-asset platforms
## Asset Universe
Primary INR pairs:
- USDINR
- EURINR
- GBPINR
- JPYINR
Secondary G10 pairs:
- EURUSD
- GBPUSD
- USDJPY
- AUDUSD
- USDCAD
- USDCHF
## Repository Structure
- fx_quant_engine/ingestion: adapters + fallback router
- fx_quant_engine/preprocessing: data cleaning/alignment
- fx_quant_engine/features: registry + modular feature pipeline
- fx_quant_engine/regime: explainable regime detection
- fx_quant_engine/models: model scoring and extension points
- fx_quant_engine/signals: directional/RV signal + confidence engine
- fx_quant_engine/risk: risk overlays and action mapping
- fx_quant_engine/evaluation: signal quality diagnostics
- fx_quant_engine/backtest: no-lookahead pair/portfolio simulation
- fx_quant_engine/outputs: machine-readable + trader-readable payloads
- fx_quant_engine/integration: cross-engine score exports
- config: YAML-driven behavior
- scripts: runnable examples
- tests: unit tests
- examples: sample payloads and run artifacts
## Data Adapter Architecture
Base interface:
```python
class BaseDataAdapter:
def fetch_price_data(self, asset, start, end):
pass
def fetch_macro_data(self, key, start, end):
pass
def fetch_rate_data(self, asset, start, end):
pass
def health_check(self):
pass
```
Included adapters:
- BreezeAdapter (ICICI Breeze)
- ZerodhaAdapter (optional via config)
- NSEAdapter
- RBIAdapter
- FreeFXAdapter
- MockAdapter
Live API wiring uses strict vendor-specific parsers with schema validation and supports:
- FXE_HTTP_TIMEOUT_SEC
- FXE_HTTP_MAX_ATTEMPTS
- FXE_HTTP_BACKOFF_SEC
- BREEZE_LIVE_ENABLED, BREEZE_BASE_URL, BREEZE_API_KEY, BREEZE_API_SECRET
- ZERODHA_LIVE_ENABLED, ZERODHA_BASE_URL, ZERODHA_API_KEY, ZERODHA_ACCESS_TOKEN
- NSE_LIVE_ENABLED, NSE_BASE_URL
- RBI_LIVE_ENABLED, RBI_BASE_URL
Router safeguards include adapter-level circuit breakers and source cooldown windows.
## Config-Driven Files
- config/universe.yaml
- config/data_sources.yaml
- config/features.yaml
- config/regimes.yaml
- config/models.yaml
- config/risk.yaml
- config/output.yaml
## Quickstart
1. Create and activate a Python 3.10+ environment.
1. Install package and dev dependencies:
```bash
pip install -e '.[dev]'
```
1. Run tests:
```bash
pytest
```
1. Run engine example:
```bash
python scripts/run_engine.py
```
The script saves artifacts to examples/runs.
## Signal Output Schema
Each signal includes:
- direction
- signal strength
- confidence (separate module)
- regime context
- expected volatility
- key drivers
- risk flags
- recommended action
- position sizing multiplier
See examples/sample_signal.json.
## Relative Value Spread Model
Relative-value generation uses:
- explicit hedge ratio estimation from aligned log prices
- spread z-score component
- pair-level momentum and carry differentials
- risk-overlayed RV signal output with `hedge_ratio` and `spread_zscore`
## Release Workflow
Automated release workflow is available in `.github/workflows/release.yml`.
- triggers on version tags like `v0.1.0`
- builds wheel + source distribution
- validates artifacts via twine
- publishes GitHub release assets with retention policy
## Build Phases Coverage
Phase 1:
- repo structure
- config system
- ingestion + preprocessing
- basic features
Phase 2:
- regime detection
- signal engine
- confidence engine
- outputs
Phase 3:
- risk layer
- evaluation framework
- backtesting scaffolding
Phase 4 starter:
- model extension hooks
- integration hooks
- tests
- docs
## Notes
- No credentials are hardcoded.
- Live API integration can be added by replacing adapter internals while preserving interfaces.
- Mock pathways are included for deterministic testing and offline development.
+54
View File
@@ -0,0 +1,54 @@
enabled_sources:
breeze: true
zerodha: false
nse: true
rbi: true
freefx: true
mock: true
source_priority:
default: [nse, rbi, freefx, mock]
g10: [freefx, mock]
asset_source_map:
USDINR: [nse, breeze, rbi, mock]
EURINR: [nse, breeze, freefx, mock]
GBPINR: [nse, freefx, mock]
JPYINR: [nse, freefx, mock]
EURUSD: [freefx, mock]
GBPUSD: [freefx, mock]
USDJPY: [freefx, mock]
AUDUSD: [freefx, mock]
USDCAD: [freefx, mock]
USDCHF: [freefx, mock]
reliability_tags:
breeze: medium
zerodha: medium
nse: high
rbi: high
freefx: medium
mock: low
latency_tags_ms:
breeze: 350
zerodha: 250
nse: 500
rbi: 800
freefx: 700
mock: 1
live_api:
breeze_live_enabled: false
zerodha_live_enabled: false
nse_live_enabled: false
rbi_live_enabled: false
http_controls:
timeout_sec: 5
max_attempts: 3
backoff_sec: 0.5
circuit_breaker:
failure_threshold: 3
cooldown_seconds: 120
+28
View File
@@ -0,0 +1,28 @@
features:
returns: true
moving_averages: true
breakouts: true
trend_strength: true
momentum_multi_horizon: true
momentum_persistence: true
momentum_acceleration: true
mean_reversion_zscore: true
bollinger_position: true
realized_vol: true
atr_proxy: true
vol_regime_proxy: true
carry_proxy: true
usd_strength_score: true
risk_on_off_proxy: true
inr_crude_proxy: true
em_stress_proxy: true
inr_vol_clustering: true
rbi_intervention_proxy: true
cross_asset_bond_proxy: true
cross_asset_commodity_proxy: true
cross_asset_equity_proxy: true
windows:
short: 5
medium: 20
long: 60
+22
View File
@@ -0,0 +1,22 @@
directional:
model_weights:
trend: 0.35
momentum: 0.25
mean_reversion: 0.15
carry: 0.10
macro: 0.15
relative_value:
lookback: 60
spread_zscore_threshold: 1.0
model_weights:
spread: 0.5
momentum_diff: 0.3
carry_diff: 0.2
confidence:
agreement_weight: 0.25
regime_stability_weight: 0.20
data_quality_weight: 0.20
volatility_penalty_weight: 0.20
event_penalty_weight: 0.15
+5
View File
@@ -0,0 +1,5 @@
output:
include_dashboard_payload: true
include_trader_summary: true
precision: 4
engine_name: fx_quant_engine
+15
View File
@@ -0,0 +1,15 @@
trend:
slope_threshold: 0.0005
mr_band: 0.003
volatility:
low_quantile: 0.3
high_quantile: 0.7
stress_quantile: 0.9
dollar:
neutral_band: 0.2
risk:
stress_vol_threshold: 0.75
risk_off_equity_threshold: -0.01
+13
View File
@@ -0,0 +1,13 @@
risk:
target_vol: 0.10
max_gross_exposure: 1.0
max_single_asset_exposure: 0.25
stressed_regime_multiplier: 0.5
illiquid_market_multiplier: 0.6
data_reliability_penalty:
high: 1.0
medium: 0.85
low: 0.6
market_hours:
inr_derivatives_open_utc: "03:45"
inr_derivatives_close_utc: "10:00"
+12
View File
@@ -0,0 +1,12 @@
primary_inr_pairs:
- USDINR
- EURINR
- GBPINR
- JPYINR
secondary_g10_pairs:
- EURUSD
- GBPUSD
- USDJPY
- AUDUSD
- USDCAD
- USDCHF
+28
View File
@@ -0,0 +1,28 @@
{
"engine": "fx_quant_engine",
"timestamp": "2026-03-30T10:00:00",
"asset": "USDINR",
"signal_type": "directional",
"signal_direction": "long_usd",
"signal_strength": 0.74,
"confidence": 0.68,
"regime": {
"trend": "trend",
"volatility": "normal",
"dollar": "strengthening",
"risk": "risk_off",
"combined": "usd_dominant",
"stability": 0.71
},
"expected_volatility": 0.089,
"risk_flags": ["outside_primary_market_hours"],
"drivers": {
"trend": 0.51,
"momentum": 0.36,
"mean_reversion": -0.12,
"carry": 0.09,
"macro": -0.07
},
"recommended_action": "enter_small",
"position_sizing_multiplier": 0.65
}
+5
View File
@@ -0,0 +1,5 @@
"""FX Quant Engine package."""
from fx_quant_engine.engine import FXQuantEngine
__all__ = ["FXQuantEngine"]
+3
View File
@@ -0,0 +1,3 @@
from fx_quant_engine.backtest.engine import BacktestEngine
__all__ = ["BacktestEngine"]
+54
View File
@@ -0,0 +1,54 @@
from __future__ import annotations
import numpy as np
import pandas as pd
from fx_quant_engine.schemas import BacktestResult
class BacktestEngine:
def __init__(self, transaction_cost_bps: float = 1.5, slippage_bps: float = 1.0) -> None:
self.tc = transaction_cost_bps / 10000.0
self.slippage = slippage_bps / 10000.0
def simulate_pair(self, prices: pd.Series, signal_strength: pd.Series) -> pd.Series:
returns = prices.pct_change().fillna(0.0)
position = signal_strength.shift(1).fillna(0.0) # no lookahead
traded = position.diff().abs().fillna(0.0)
net = position * returns - traded * (self.tc + self.slippage)
return net
def simulate_portfolio(self, pair_returns: dict[str, pd.Series]) -> BacktestResult:
if not pair_returns:
return BacktestResult({}, 0.0, 0.0, 0.0, {})
df = pd.DataFrame(pair_returns).fillna(0.0)
port = df.mean(axis=1)
equity = (1.0 + port).cumprod()
dd = equity / equity.cummax() - 1.0
pair_level = {k: float(v.mean() * 252.0) for k, v in pair_returns.items()}
metrics = {
"annualized_return": float(port.mean() * 252.0),
"annualized_vol": float(port.std() * np.sqrt(252.0)),
"sharpe": float((port.mean() / (port.std() + 1e-9)) * np.sqrt(252.0)),
}
return BacktestResult(
pair_level_returns=pair_level,
portfolio_return=float((equity.iloc[-1] - 1.0) if len(equity) else 0.0),
max_drawdown=float(dd.min() if len(dd) else 0.0),
turnover=float(df.diff().abs().sum().sum()),
metrics=metrics,
)
def walk_forward(self, prices: pd.Series, signal_strength: pd.Series, train_window: int = 120, test_window: int = 20) -> pd.Series:
out = []
idx = prices.index
i = train_window
while i < len(idx):
end = min(i + test_window, len(idx))
segment = self.simulate_pair(prices.iloc[:end], signal_strength.iloc[:end])
out.append(segment.iloc[i:end])
i += test_window
return pd.concat(out).sort_index() if out else pd.Series(dtype=float)
+214
View File
@@ -0,0 +1,214 @@
from __future__ import annotations
from datetime import datetime
from pathlib import Path
from typing import Any
import numpy as np
from fx_quant_engine.features.pipeline import FeaturePipeline
from fx_quant_engine.ingestion.adapters import (
BreezeAdapter,
FreeFXAdapter,
MockAdapter,
NSEAdapter,
RBIAdapter,
ZerodhaAdapter,
)
from fx_quant_engine.ingestion.router import DataSourceRouter
from fx_quant_engine.integration.hooks import IntegrationHooks
from fx_quant_engine.models.ensemble import EnsembleModel
from fx_quant_engine.models.relative_value import RelativeValueModel
from fx_quant_engine.outputs.formatter import OutputFormatter
from fx_quant_engine.preprocessing.cleaning import preprocess_market_data
from fx_quant_engine.regime.engine import RegimeEngine
from fx_quant_engine.risk.engine import RiskEngine
from fx_quant_engine.schemas import SignalPayload
from fx_quant_engine.signals.confidence import ConfidenceEngine
from fx_quant_engine.signals.engine import SignalEngine
from fx_quant_engine.utils.config import load_all_configs
class FXQuantEngine:
def __init__(self, config_dir: str | Path = "config") -> None:
self.config = load_all_configs(config_dir)
self.adapters = {
"breeze": BreezeAdapter(),
"zerodha": ZerodhaAdapter(),
"nse": NSEAdapter(),
"rbi": RBIAdapter(),
"freefx": FreeFXAdapter(),
"mock": MockAdapter(),
}
self.router = DataSourceRouter(self.adapters, self.config["data_sources"])
self.features = FeaturePipeline(self.config["features"])
self.regimes = RegimeEngine(self.config["regimes"])
self.models = EnsembleModel(self.config["models"])
self.rv_model = RelativeValueModel(
lookback=int(self.config["models"].get("relative_value", {}).get("lookback", 60))
)
self.confidence = ConfidenceEngine(self.config["models"])
self.signals = SignalEngine(
engine_name=self.config["output"]["output"].get("engine_name", "fx_quant_engine")
)
self.risk = RiskEngine(self.config["risk"])
self.output = OutputFormatter(self.config["output"])
self.integration = IntegrationHooks()
def _asset_context(self, asset: str, start: datetime, end: datetime) -> dict[str, Any]:
px_res = self.router.fetch_price_data(asset, start, end)
macro_res = self.router.fetch_macro_data("macro_proxy", start, end)
rate_res = self.router.fetch_rate_data(asset, start, end)
if not px_res.success or px_res.data.empty:
raise RuntimeError(f"No price data available for {asset}: {px_res.error}")
df = px_res.data.join(macro_res.data, how="left").join(rate_res.data, how="left")
if f"{asset}_rate" in df.columns:
df["short_rate"] = df[f"{asset}_rate"]
df["usd_rate"] = 0.045
df["equity_proxy"] = df["close"].pct_change().rolling(5).mean().fillna(0.0)
df["crude_proxy"] = (1.0 + df["close"].pct_change().fillna(0.0)).cumprod()
df["emfx_proxy"] = df["close"].pct_change().fillna(0.0)
clean = preprocess_market_data(df)
feat = self.features.transform(clean)
regime = self.regimes.detect(feat)
return {
"price": px_res,
"macro": macro_res,
"rate": rate_res,
"clean": clean,
"features": feat,
"regime": regime,
}
def run_asset(self, asset: str, start: datetime, end: datetime) -> dict[str, Any]:
ctx = self._asset_context(asset, start, end)
px_res = ctx["price"]
feat = ctx["features"]
regime = ctx["regime"]
row = feat.iloc[-1]
score, model_drivers, model_agreement = self.models.directional_score(row)
feature_agreement = float((row > 0).mean())
data_quality = 1.0 if px_res.reliability == "high" else 0.8 if px_res.reliability == "medium" else 0.6
volatility = float(min(1.0, row.get("realized_vol", 0.0)))
event_risk = float(min(1.0, row.get("rbi_intervention_proxy", 0.0)))
conf = self.confidence.compute(
feature_agreement=feature_agreement,
model_agreement=model_agreement,
regime_stability=regime.stability,
data_quality=data_quality,
volatility=volatility,
event_risk=event_risk,
)
sig: SignalPayload = self.signals.directional_signal(
asset=asset,
feature_row=row,
regime=regime,
score=score,
confidence=conf,
drivers=model_drivers,
expected_volatility=float(row.get("realized_vol", 0.0)),
)
sig = self.risk.apply(
signal=sig,
regime=regime,
reliability=px_res.reliability,
liquidity_ok=True,
event_risk=event_risk,
)
payload = self.output.to_json_payload(sig)
summary = self.output.trader_summary(sig)
dashboard = self.output.dashboard_payload(sig)
hooks = self.integration.export_scores(feat, regime)
return {
"signal": payload,
"summary": summary,
"dashboard": dashboard,
"integration_hooks": hooks,
"source_usage": [u.__dict__ for u in self.router.usage_log],
}
def run_relative_value(self, asset_a: str, asset_b: str, start: datetime, end: datetime) -> dict[str, Any]:
ctx_a = self._asset_context(asset_a, start, end)
ctx_b = self._asset_context(asset_b, start, end)
feat_a = ctx_a["features"]
feat_b = ctx_b["features"]
rv = self.rv_model.generate(
asset_a=asset_a,
asset_b=asset_b,
prices_a=ctx_a["clean"]["close"],
prices_b=ctx_b["clean"]["close"],
features_a=feat_a,
features_b=feat_b,
)
regime = ctx_a["regime"] if ctx_a["regime"].stability >= ctx_b["regime"].stability else ctx_b["regime"]
driver_vals = [v for k, v in rv.drivers.items() if k != "hedge_ratio"]
signs = [np.sign(v) for v in driver_vals if abs(v) > 1e-12]
model_agreement = float(abs(sum(signs)) / len(signs)) if signs else 0.0
feature_agreement = float((feat_a.iloc[-1] - feat_b.iloc[-1] > 0).mean())
rel_map = {"high": 1.0, "medium": 0.8, "low": 0.6}
q_a = rel_map.get(ctx_a["price"].reliability, 0.6)
q_b = rel_map.get(ctx_b["price"].reliability, 0.6)
data_quality = float((q_a + q_b) / 2.0)
volatility = float(min(1.0, rv.expected_volatility))
event_risk = float(
min(
1.0,
(
float(feat_a.iloc[-1].get("rbi_intervention_proxy", 0.0))
+ float(feat_b.iloc[-1].get("rbi_intervention_proxy", 0.0))
)
/ 2.0,
)
)
conf = self.confidence.compute(
feature_agreement=feature_agreement,
model_agreement=model_agreement,
regime_stability=regime.stability,
data_quality=data_quality,
volatility=volatility,
event_risk=event_risk,
)
sig = self.signals.relative_value_signal(
long_asset=rv.long_asset,
short_asset=rv.short_asset,
regime=regime,
score=rv.score,
confidence=conf,
drivers=rv.drivers,
expected_volatility=rv.expected_volatility,
)
reliability = "high" if min(q_a, q_b) >= 1.0 else "medium" if min(q_a, q_b) >= 0.8 else "low"
sig = self.risk.apply(
signal=sig,
regime=regime,
reliability=reliability,
liquidity_ok=True,
event_risk=event_risk,
)
return {
"relative_value_pair": f"{asset_a}:{asset_b}",
"long": rv.long_asset,
"short": rv.short_asset,
"spread_score": rv.score,
"hedge_ratio": rv.hedge_ratio,
"spread_zscore": rv.spread_zscore,
"signal": self.output.to_json_payload(sig),
"summary": self.output.trader_summary(sig),
"dashboard": self.output.dashboard_payload(sig),
}
+3
View File
@@ -0,0 +1,3 @@
from fx_quant_engine.evaluation.engine import Evaluator
__all__ = ["Evaluator"]
+67
View File
@@ -0,0 +1,67 @@
from __future__ import annotations
from datetime import datetime, timezone
import numpy as np
import pandas as pd
from fx_quant_engine.schemas import EvaluationReport
class Evaluator:
def evaluate(
self,
signals: pd.DataFrame,
forward_returns: pd.Series,
regimes: pd.Series,
feature_importance: dict[str, float],
) -> EvaluationReport:
if signals.empty or forward_returns.empty:
return EvaluationReport(
generated_at=datetime.now(timezone.utc).isoformat(),
hit_rate=0.0,
avg_forward_return=0.0,
regime_performance={},
signal_decay={},
feature_importance=feature_importance,
drift_flags=["insufficient_data"],
calibration_score=0.0,
)
aligned = signals.join(forward_returns.rename("fwd"), how="inner")
aligned = aligned.join(regimes.rename("regime"), how="left").fillna("unknown")
pred = np.sign(aligned["signal_strength"] * aligned.get("signal_sign", 1.0))
realized = np.sign(aligned["fwd"])
hit_rate = float((pred == realized).mean())
avg_ret = float(aligned["fwd"].mean())
regime_perf = {
r: float(v)
for r, v in aligned.groupby("regime")["fwd"].mean().to_dict().items()
}
signal_decay = {
"1d": float(aligned["fwd"].mean()),
"5d": float(aligned["fwd"].rolling(5).mean().dropna().mean() if len(aligned) >= 5 else 0.0),
"20d": float(aligned["fwd"].rolling(20).mean().dropna().mean() if len(aligned) >= 20 else 0.0),
}
drift = []
if abs(avg_ret) < 1e-4:
drift.append("low_signal_edge")
if hit_rate < 0.48:
drift.append("hit_rate_deterioration")
conf = aligned.get("confidence", pd.Series(0.5, index=aligned.index))
calibration_score = float(np.clip(1.0 - abs(conf.mean() - hit_rate), 0.0, 1.0))
return EvaluationReport(
generated_at=datetime.now(timezone.utc).isoformat(),
hit_rate=hit_rate,
avg_forward_return=avg_ret,
regime_performance=regime_perf,
signal_decay=signal_decay,
feature_importance=feature_importance,
drift_flags=drift,
calibration_score=calibration_score,
)
+3
View File
@@ -0,0 +1,3 @@
from fx_quant_engine.features.pipeline import FeaturePipeline
__all__ = ["FeaturePipeline"]
+27
View File
@@ -0,0 +1,27 @@
from __future__ import annotations
from collections.abc import Callable
from typing import Any
import pandas as pd
FeatureFn = Callable[[pd.DataFrame, dict[str, Any]], pd.Series]
class FeatureRegistry:
def __init__(self) -> None:
self._features: dict[str, FeatureFn] = {}
def register(self, name: str, fn: FeatureFn) -> None:
if name in self._features:
raise ValueError(f"Feature already registered: {name}")
self._features[name] = fn
def get(self, name: str) -> FeatureFn:
return self._features[name]
def names(self) -> list[str]:
return list(self._features.keys())
registry = FeatureRegistry()
+164
View File
@@ -0,0 +1,164 @@
from __future__ import annotations
import numpy as np
import pandas as pd
from fx_quant_engine.features.base import registry
def _close(df: pd.DataFrame) -> pd.Series:
if "close" in df.columns:
return df["close"]
numeric = df.select_dtypes(include="number")
if numeric.empty:
return pd.Series(0.0, index=df.index)
return numeric.iloc[:, 0]
def returns(df: pd.DataFrame, cfg: dict) -> pd.Series:
return _close(df).pct_change().fillna(0.0)
def moving_averages(df: pd.DataFrame, cfg: dict) -> pd.Series:
w = cfg["windows"]["medium"]
c = _close(df)
return (c / c.rolling(w).mean() - 1.0).fillna(0.0)
def breakouts(df: pd.DataFrame, cfg: dict) -> pd.Series:
w = cfg["windows"]["long"]
c = _close(df)
high = c.rolling(w).max()
low = c.rolling(w).min()
rng = (high - low).replace(0.0, np.nan)
return ((c - low) / rng).fillna(0.5)
def trend_strength(df: pd.DataFrame, cfg: dict) -> pd.Series:
w = cfg["windows"]["medium"]
r = returns(df, cfg)
return (r.rolling(w).mean() / (r.rolling(w).std() + 1e-9)).fillna(0.0)
def momentum_multi_horizon(df: pd.DataFrame, cfg: dict) -> pd.Series:
c = _close(df)
s = cfg["windows"]["short"]
m = cfg["windows"]["medium"]
l = cfg["windows"]["long"]
return ((c.pct_change(s) + c.pct_change(m) + c.pct_change(l)) / 3.0).fillna(0.0)
def momentum_persistence(df: pd.DataFrame, cfg: dict) -> pd.Series:
r = returns(df, cfg)
return r.rolling(cfg["windows"]["short"]).apply(lambda x: float((x > 0).mean()), raw=False).fillna(0.5)
def momentum_acceleration(df: pd.DataFrame, cfg: dict) -> pd.Series:
m = momentum_multi_horizon(df, cfg)
return m.diff().fillna(0.0)
def mean_reversion_zscore(df: pd.DataFrame, cfg: dict) -> pd.Series:
c = _close(df)
w = cfg["windows"]["medium"]
m = c.rolling(w).mean()
s = c.rolling(w).std()
return ((c - m) / (s + 1e-9)).fillna(0.0)
def bollinger_position(df: pd.DataFrame, cfg: dict) -> pd.Series:
z = mean_reversion_zscore(df, cfg)
return (z / 2.0).clip(-1, 1)
def realized_vol(df: pd.DataFrame, cfg: dict) -> pd.Series:
r = returns(df, cfg)
return (r.rolling(cfg["windows"]["medium"]).std() * np.sqrt(252)).fillna(0.0)
def atr_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
c = _close(df)
return c.diff().abs().rolling(cfg["windows"]["short"]).mean().fillna(0.0)
def vol_regime_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
vol = realized_vol(df, cfg)
return (vol / (vol.rolling(cfg["windows"]["long"]).mean() + 1e-9)).fillna(1.0)
def carry_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
short_r = df.get("short_rate", pd.Series(0.05, index=df.index))
usd_r = df.get("usd_rate", pd.Series(0.045, index=df.index))
return (short_r - usd_r).fillna(0.0)
def usd_strength_score(df: pd.DataFrame, cfg: dict) -> pd.Series:
r = returns(df, cfg)
return (-r.rolling(cfg["windows"]["medium"]).mean()).fillna(0.0)
def risk_on_off_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
eq = df.get("equity_proxy", returns(df, cfg)).fillna(0.0)
vix = df.get("vix_proxy", realized_vol(df, cfg)).fillna(0.0)
return (eq - vix).fillna(0.0)
def inr_crude_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
crude = df.get("crude_proxy", pd.Series(0.0, index=df.index))
return crude.pct_change().fillna(0.0)
def em_stress_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
em = df.get("emfx_proxy", returns(df, cfg)).fillna(0.0)
return (-em.rolling(cfg["windows"]["medium"]).mean()).fillna(0.0)
def inr_vol_clustering(df: pd.DataFrame, cfg: dict) -> pd.Series:
r = returns(df, cfg)
return r.abs().rolling(cfg["windows"]["short"]).mean().fillna(0.0)
def rbi_intervention_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
z = mean_reversion_zscore(df, cfg)
return (z.abs() > 2.0).astype(float)
def cross_asset_bond_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
y = df.get("bond_yield_proxy", pd.Series(0.0, index=df.index))
return y.diff().fillna(0.0)
def cross_asset_commodity_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
g = df.get("gold_proxy", pd.Series(0.0, index=df.index))
o = df.get("oil_proxy", pd.Series(0.0, index=df.index))
return (g.pct_change().fillna(0.0) + o.pct_change().fillna(0.0)) / 2.0
def cross_asset_equity_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
e = df.get("equity_proxy", pd.Series(0.0, index=df.index))
return e.pct_change().fillna(0.0)
def register_builtin_features() -> None:
registry.register("returns", returns)
registry.register("moving_averages", moving_averages)
registry.register("breakouts", breakouts)
registry.register("trend_strength", trend_strength)
registry.register("momentum_multi_horizon", momentum_multi_horizon)
registry.register("momentum_persistence", momentum_persistence)
registry.register("momentum_acceleration", momentum_acceleration)
registry.register("mean_reversion_zscore", mean_reversion_zscore)
registry.register("bollinger_position", bollinger_position)
registry.register("realized_vol", realized_vol)
registry.register("atr_proxy", atr_proxy)
registry.register("vol_regime_proxy", vol_regime_proxy)
registry.register("carry_proxy", carry_proxy)
registry.register("usd_strength_score", usd_strength_score)
registry.register("risk_on_off_proxy", risk_on_off_proxy)
registry.register("inr_crude_proxy", inr_crude_proxy)
registry.register("em_stress_proxy", em_stress_proxy)
registry.register("inr_vol_clustering", inr_vol_clustering)
registry.register("rbi_intervention_proxy", rbi_intervention_proxy)
registry.register("cross_asset_bond_proxy", cross_asset_bond_proxy)
registry.register("cross_asset_commodity_proxy", cross_asset_commodity_proxy)
registry.register("cross_asset_equity_proxy", cross_asset_equity_proxy)
+26
View File
@@ -0,0 +1,26 @@
from __future__ import annotations
from typing import Any
import pandas as pd
from fx_quant_engine.features.base import registry
from fx_quant_engine.features.builtins import register_builtin_features
class FeaturePipeline:
def __init__(self, config: dict[str, Any]) -> None:
self.config = config
if not registry.names():
register_builtin_features()
def transform(self, df: pd.DataFrame) -> pd.DataFrame:
feature_cfg = self.config.get("features", {})
out = pd.DataFrame(index=df.index)
for feature_name, enabled in feature_cfg.items():
if not enabled:
continue
if feature_name not in registry.names():
continue
out[feature_name] = registry.get(feature_name)(df, self.config)
return out.fillna(0.0)
+27
View File
@@ -0,0 +1,27 @@
from fx_quant_engine.ingestion.adapters import (
BreezeAdapter,
FreeFXAdapter,
MockAdapter,
NSEAdapter,
RBIAdapter,
ZerodhaAdapter,
)
from fx_quant_engine.ingestion.base import BaseDataAdapter
from fx_quant_engine.ingestion.credentials import APICredentials, load_credentials
from fx_quant_engine.ingestion.http_client import HttpClient, RetryConfig
from fx_quant_engine.ingestion.router import DataSourceRouter
__all__ = [
"BaseDataAdapter",
"BreezeAdapter",
"ZerodhaAdapter",
"NSEAdapter",
"RBIAdapter",
"FreeFXAdapter",
"MockAdapter",
"DataSourceRouter",
"APICredentials",
"load_credentials",
"HttpClient",
"RetryConfig",
]
+264
View File
@@ -0,0 +1,264 @@
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime
import os
import numpy as np
import pandas as pd
from fx_quant_engine.ingestion.base import BaseDataAdapter
from fx_quant_engine.ingestion.credentials import APICredentials, load_credentials
from fx_quant_engine.ingestion.http_client import HttpClient, RetryConfig
from fx_quant_engine.ingestion.parsers import (
BreezeResponseParser,
NSEResponseParser,
RBIResponseParser,
ZerodhaResponseParser,
)
def _mock_price_series(asset: str, start: datetime, end: datetime, seed: int) -> pd.DataFrame:
idx = pd.date_range(start=start, end=end, freq="B")
rng = np.random.default_rng(seed)
steps = rng.normal(loc=0.0, scale=0.002, size=len(idx))
base = 80.0 if asset.endswith("INR") else 1.1
px = base * np.exp(np.cumsum(steps))
return pd.DataFrame({"close": px}, index=idx)
def _mock_macro_series(key: str, start: datetime, end: datetime, seed: int) -> pd.DataFrame:
idx = pd.date_range(start=start, end=end, freq="B")
rng = np.random.default_rng(seed)
values = rng.normal(loc=0.0, scale=1.0, size=len(idx))
return pd.DataFrame({key: values}, index=idx)
def _mock_rate_series(asset: str, start: datetime, end: datetime, seed: int) -> pd.DataFrame:
idx = pd.date_range(start=start, end=end, freq="B")
rng = np.random.default_rng(seed)
rate = 0.05 + rng.normal(loc=0.0, scale=0.002, size=len(idx))
return pd.DataFrame({f"{asset}_rate": rate}, index=idx)
def _retry_config_from_env() -> RetryConfig:
return RetryConfig(
timeout_sec=float(os.getenv("FXE_HTTP_TIMEOUT_SEC", "5.0")),
max_attempts=int(os.getenv("FXE_HTTP_MAX_ATTEMPTS", "3")),
backoff_sec=float(os.getenv("FXE_HTTP_BACKOFF_SEC", "0.5")),
)
def _live_enabled(env_key: str) -> bool:
return os.getenv(env_key, "false").lower() in {"1", "true", "yes", "on"}
@dataclass
class BreezeAdapter(BaseDataAdapter):
use_live: bool = False
base_url: str | None = None
def __post_init__(self) -> None:
self.use_live = self.use_live or _live_enabled("BREEZE_LIVE_ENABLED")
self.base_url = self.base_url or os.getenv("BREEZE_BASE_URL")
self.client = HttpClient(_retry_config_from_env())
self.parser = BreezeResponseParser()
def _creds(self) -> APICredentials:
return load_credentials("BREEZE", required=["API_KEY", "API_SECRET"], optional=["SESSION_TOKEN"])
def fetch_price_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
if not self.base_url:
raise RuntimeError("Breeze live enabled but BREEZE_BASE_URL not configured")
creds = self._creds()
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/api/v1/price",
params={"symbol": asset, "start": start.isoformat(), "end": end.isoformat()},
headers={"X-API-KEY": str(creds.get("api_key") or "")},
)
return self.parser.parse_price(payload, start, end)
return _mock_price_series(asset, start, end, seed=1)
def fetch_macro_data(self, key: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
if not self.base_url:
raise RuntimeError("Breeze live enabled but BREEZE_BASE_URL not configured")
creds = self._creds()
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/api/v1/macro",
params={"key": key, "start": start.isoformat(), "end": end.isoformat()},
headers={"X-API-KEY": str(creds.get("api_key") or "")},
)
return self.parser.parse_macro(payload, key, start, end)
return _mock_macro_series(key, start, end, seed=2)
def fetch_rate_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
if not self.base_url:
raise RuntimeError("Breeze live enabled but BREEZE_BASE_URL not configured")
creds = self._creds()
col = f"{asset}_rate"
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/api/v1/rates",
params={"symbol": asset, "start": start.isoformat(), "end": end.isoformat()},
headers={"X-API-KEY": str(creds.get("api_key") or "")},
)
return self.parser.parse_rate(payload, col, start, end)
return _mock_rate_series(asset, start, end, seed=3)
def health_check(self) -> bool:
if not self.use_live:
return True
return bool(self.base_url)
@dataclass
class ZerodhaAdapter(BaseDataAdapter):
use_live: bool = False
base_url: str | None = None
def __post_init__(self) -> None:
self.use_live = self.use_live or _live_enabled("ZERODHA_LIVE_ENABLED")
self.base_url = self.base_url or os.getenv("ZERODHA_BASE_URL")
self.client = HttpClient(_retry_config_from_env())
self.parser = ZerodhaResponseParser()
def _creds(self) -> APICredentials:
return load_credentials("ZERODHA", required=["API_KEY", "ACCESS_TOKEN"])
def fetch_price_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
if not self.base_url:
raise RuntimeError("Zerodha live enabled but ZERODHA_BASE_URL not configured")
creds = self._creds()
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/instruments/historical",
params={"symbol": asset, "start": start.isoformat(), "end": end.isoformat()},
headers={
"X-Kite-Version": "3",
"Authorization": (
f"token {str(creds.get('api_key') or '')}:{str(creds.get('access_token') or '')}"
),
},
)
return self.parser.parse_price(payload, start, end)
return _mock_price_series(asset, start, end, seed=4)
def fetch_macro_data(self, key: str, start: datetime, end: datetime) -> pd.DataFrame:
raise NotImplementedError("Zerodha macro data not supported")
def fetch_rate_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
raise NotImplementedError("Zerodha rates not supported")
def health_check(self) -> bool:
if not self.use_live:
return True
return bool(self.base_url)
class NSEAdapter(BaseDataAdapter):
def __init__(self, use_live: bool = False, base_url: str | None = None) -> None:
self.use_live = use_live or _live_enabled("NSE_LIVE_ENABLED")
self.base_url = base_url or os.getenv("NSE_BASE_URL", "https://www.nseindia.com")
self.client = HttpClient(_retry_config_from_env())
self.parser = NSEResponseParser()
def fetch_price_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/api/historical/foCPV",
params={"symbol": asset, "from": start.strftime("%d-%m-%Y"), "to": end.strftime("%d-%m-%Y")},
headers={"User-Agent": "fx-quant-engine/0.1"},
)
return self.parser.parse_price(payload, start, end)
return _mock_price_series(asset, start, end, seed=5)
def fetch_macro_data(self, key: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/api/allIndices",
headers={"User-Agent": "fx-quant-engine/0.1"},
)
return self.parser.parse_macro(payload, key, start, end)
return _mock_macro_series(key, start, end, seed=6)
def fetch_rate_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
col = f"{asset}_rate"
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/api/live-analysis-oi-spurts-underlyings",
headers={"User-Agent": "fx-quant-engine/0.1"},
)
return self.parser.parse_rate(payload, col, start, end)
return _mock_rate_series(asset, start, end, seed=7)
def health_check(self) -> bool:
return True
class RBIAdapter(BaseDataAdapter):
def __init__(self, use_live: bool = False, base_url: str | None = None) -> None:
self.use_live = use_live or _live_enabled("RBI_LIVE_ENABLED")
self.base_url = base_url or os.getenv("RBI_BASE_URL", "https://data.rbi.org.in")
self.client = HttpClient(_retry_config_from_env())
self.parser = RBIResponseParser()
def fetch_price_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/api/exchange-rate",
params={"pair": asset, "from": start.isoformat(), "to": end.isoformat()},
)
return self.parser.parse_price(payload, start, end)
return _mock_price_series(asset, start, end, seed=8)
def fetch_macro_data(self, key: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/api/macro",
params={"series": key, "from": start.isoformat(), "to": end.isoformat()},
)
return self.parser.parse_macro(payload, key, start, end)
return _mock_macro_series(key, start, end, seed=9)
def fetch_rate_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
col = f"{asset}_rate"
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/api/policy-rates",
params={"asset": asset, "from": start.isoformat(), "to": end.isoformat()},
)
return self.parser.parse_rate(payload, col, start, end)
return _mock_rate_series(asset, start, end, seed=10)
def health_check(self) -> bool:
return True
class FreeFXAdapter(BaseDataAdapter):
def fetch_price_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
return _mock_price_series(asset, start, end, seed=11)
def fetch_macro_data(self, key: str, start: datetime, end: datetime) -> pd.DataFrame:
return _mock_macro_series(key, start, end, seed=12)
def fetch_rate_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
return _mock_rate_series(asset, start, end, seed=13)
def health_check(self) -> bool:
return True
class MockAdapter(BaseDataAdapter):
def fetch_price_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
return _mock_price_series(asset, start, end, seed=42)
def fetch_macro_data(self, key: str, start: datetime, end: datetime) -> pd.DataFrame:
return _mock_macro_series(key, start, end, seed=43)
def fetch_rate_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
return _mock_rate_series(asset, start, end, seed=44)
def health_check(self) -> bool:
return True
+24
View File
@@ -0,0 +1,24 @@
from __future__ import annotations
from abc import ABC, abstractmethod
from datetime import datetime
import pandas as pd
class BaseDataAdapter(ABC):
@abstractmethod
def fetch_price_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
pass
@abstractmethod
def fetch_macro_data(self, key: str, start: datetime, end: datetime) -> pd.DataFrame:
pass
@abstractmethod
def fetch_rate_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
pass
@abstractmethod
def health_check(self) -> bool:
pass
+36
View File
@@ -0,0 +1,36 @@
from __future__ import annotations
import os
from dataclasses import dataclass
@dataclass(frozen=True)
class APICredentials:
values: dict[str, str]
def get(self, key: str, default: str | None = None) -> str | None:
return self.values.get(key, default)
def load_credentials(prefix: str, required: list[str], optional: list[str] | None = None) -> APICredentials:
optional = optional or []
payload: dict[str, str] = {}
missing: list[str] = []
for key in required:
env_key = f"{prefix}_{key}"
value = os.getenv(env_key)
if not value:
missing.append(env_key)
continue
payload[key.lower()] = value
for key in optional:
env_key = f"{prefix}_{key}"
value = os.getenv(env_key)
if value:
payload[key.lower()] = value
if missing:
raise RuntimeError(f"Missing required credentials: {', '.join(missing)}")
return APICredentials(values=payload)
+43
View File
@@ -0,0 +1,43 @@
from __future__ import annotations
import time
from dataclasses import dataclass
from typing import Any
import requests
@dataclass
class RetryConfig:
timeout_sec: float = 5.0
max_attempts: int = 3
backoff_sec: float = 0.5
class HttpClient:
def __init__(self, config: RetryConfig) -> None:
self.config = config
def get_json(
self,
url: str,
params: dict[str, Any] | None = None,
headers: dict[str, str] | None = None,
) -> dict[str, Any]:
last_err = "unknown"
for attempt in range(1, self.config.max_attempts + 1):
try:
response = requests.get(url, params=params, headers=headers, timeout=self.config.timeout_sec)
response.raise_for_status()
data = response.json()
if not isinstance(data, dict):
raise RuntimeError("Expected JSON object response")
return data
except Exception as exc: # noqa: BLE001
last_err = str(exc)
if attempt < self.config.max_attempts:
time.sleep(self.config.backoff_sec * attempt)
raise RuntimeError(
f"HTTP GET failed after {self.config.max_attempts} attempts for {url}: {last_err}"
)
+94
View File
@@ -0,0 +1,94 @@
from __future__ import annotations
from datetime import datetime
from typing import Any
import pandas as pd
class BaseResponseParser:
def parse_price(self, payload: dict[str, Any], start: datetime, end: datetime) -> pd.DataFrame:
return _extract_price(payload, start, end)
def parse_macro(self, payload: dict[str, Any], key: str, start: datetime, end: datetime) -> pd.DataFrame:
return _extract_series(payload, key, start, end)
def parse_rate(self, payload: dict[str, Any], column: str, start: datetime, end: datetime) -> pd.DataFrame:
return _extract_series(payload, column, start, end)
class BreezeResponseParser(BaseResponseParser):
pass
class ZerodhaResponseParser(BaseResponseParser):
pass
class NSEResponseParser(BaseResponseParser):
pass
class RBIResponseParser(BaseResponseParser):
pass
def _extract_price(payload: dict[str, Any], start: datetime, end: datetime) -> pd.DataFrame:
records = payload.get("data")
if not isinstance(records, list) or not records:
raise RuntimeError("Invalid payload: 'data' must be a non-empty list")
df = pd.DataFrame(records)
if "timestamp" in df.columns:
df["timestamp"] = pd.to_datetime(df["timestamp"], utc=True, errors="coerce")
df = df.dropna(subset=["timestamp"]).set_index("timestamp")
elif "date" in df.columns:
df["date"] = pd.to_datetime(df["date"], utc=True, errors="coerce")
df = df.dropna(subset=["date"]).set_index("date")
else:
raise RuntimeError("Invalid payload: missing timestamp/date column")
if "close" not in df.columns:
if "price" in df.columns:
df = df.rename(columns={"price": "close"})
else:
raise RuntimeError("Invalid payload: missing close/price column")
out = df[["close"]].sort_index()
start_ts = pd.Timestamp(start)
end_ts = pd.Timestamp(end)
start_ts = start_ts.tz_localize("UTC") if start_ts.tzinfo is None else start_ts.tz_convert("UTC")
end_ts = end_ts.tz_localize("UTC") if end_ts.tzinfo is None else end_ts.tz_convert("UTC")
clipped = out[(out.index >= start_ts) & (out.index <= end_ts)]
if clipped.empty:
raise RuntimeError("Invalid payload: no records in requested window")
return clipped
def _extract_series(payload: dict[str, Any], key: str, start: datetime, end: datetime) -> pd.DataFrame:
records = payload.get("data")
if not isinstance(records, list) or not records:
raise RuntimeError("Invalid payload: 'data' must be a non-empty list")
df = pd.DataFrame(records)
if "timestamp" in df.columns:
df["timestamp"] = pd.to_datetime(df["timestamp"], utc=True, errors="coerce")
df = df.dropna(subset=["timestamp"]).set_index("timestamp")
elif "date" in df.columns:
df["date"] = pd.to_datetime(df["date"], utc=True, errors="coerce")
df = df.dropna(subset=["date"]).set_index("date")
else:
raise RuntimeError("Invalid payload: missing timestamp/date column")
if key not in df.columns:
raise RuntimeError(f"Invalid payload: missing required column '{key}'")
out = df[[key]].sort_index()
start_ts = pd.Timestamp(start)
end_ts = pd.Timestamp(end)
start_ts = start_ts.tz_localize("UTC") if start_ts.tzinfo is None else start_ts.tz_convert("UTC")
end_ts = end_ts.tz_localize("UTC") if end_ts.tzinfo is None else end_ts.tz_convert("UTC")
clipped = out[(out.index >= start_ts) & (out.index <= end_ts)]
if clipped.empty:
raise RuntimeError("Invalid payload: no records in requested window")
return clipped
+155
View File
@@ -0,0 +1,155 @@
from __future__ import annotations
from datetime import datetime, timedelta, timezone
from typing import Any
import pandas as pd
from fx_quant_engine.ingestion.base import BaseDataAdapter
from fx_quant_engine.schemas import DataFetchResult, SourceUsageRecord
from fx_quant_engine.utils.logging import get_logger
class DataSourceRouter:
def __init__(
self,
adapters: dict[str, BaseDataAdapter],
config: dict[str, Any],
) -> None:
self.adapters = adapters
self.config = config
self.logger = get_logger(self.__class__.__name__)
self.usage_log: list[SourceUsageRecord] = []
cb_cfg = config.get("circuit_breaker", {})
self.failure_threshold = int(cb_cfg.get("failure_threshold", 3))
self.cooldown_seconds = int(cb_cfg.get("cooldown_seconds", 120))
self.source_state: dict[str, dict[str, Any]] = {
name: {"failures": 0, "open_until": None} for name in adapters
}
def _get_candidate_sources(self, asset: str) -> list[str]:
source_cfg = self.config
mapped = source_cfg.get("asset_source_map", {}).get(asset)
if mapped:
return mapped
g10 = asset in {"EURUSD", "GBPUSD", "USDJPY", "AUDUSD", "USDCAD", "USDCHF"}
key = "g10" if g10 else "default"
return source_cfg.get("source_priority", {}).get(key, ["mock"])
def _is_enabled(self, source: str) -> bool:
return bool(self.config.get("enabled_sources", {}).get(source, False))
def _tag(self, source: str) -> tuple[str, int]:
reliability = self.config.get("reliability_tags", {}).get(source, "low")
latency_ms = int(self.config.get("latency_tags_ms", {}).get(source, 1000))
return reliability, latency_ms
def _source_available(self, source: str, now: datetime) -> bool:
state = self.source_state.get(source, {"open_until": None})
open_until = state.get("open_until")
if open_until is None:
return True
if now >= open_until:
state["open_until"] = None
state["failures"] = 0
self.source_state[source] = state
return True
return False
def _register_success(self, source: str) -> None:
state = self.source_state.get(source, {"failures": 0, "open_until": None})
state["failures"] = 0
state["open_until"] = None
self.source_state[source] = state
def _register_failure(self, source: str, now: datetime) -> None:
state = self.source_state.get(source, {"failures": 0, "open_until": None})
state["failures"] = int(state.get("failures", 0)) + 1
if state["failures"] >= self.failure_threshold:
state["open_until"] = now + timedelta(seconds=self.cooldown_seconds)
self.logger.warning(
"Opening circuit for source=%s until %s after %s failures",
source,
state["open_until"].isoformat(),
state["failures"],
)
self.source_state[source] = state
def fetch_price_data(self, asset: str, start: datetime, end: datetime) -> DataFetchResult:
return self._fetch_with_fallback("price", asset, start, end)
def fetch_macro_data(self, key: str, start: datetime, end: datetime) -> DataFetchResult:
return self._fetch_with_fallback("macro", key, start, end)
def fetch_rate_data(self, asset: str, start: datetime, end: datetime) -> DataFetchResult:
return self._fetch_with_fallback("rate", asset, start, end)
def _fetch_with_fallback(
self,
usage_type: str,
key: str,
start: datetime,
end: datetime,
) -> DataFetchResult:
candidates = self._get_candidate_sources(key)
last_err = "No source attempted"
now = datetime.now(timezone.utc)
for source in candidates:
if not self._is_enabled(source):
continue
if not self._source_available(source, now):
self.logger.info("Skipping source=%s due to cooldown window", source)
continue
adapter = self.adapters.get(source)
if adapter is None:
continue
try:
if usage_type == "price":
data = adapter.fetch_price_data(key, start, end)
elif usage_type == "macro":
data = adapter.fetch_macro_data(key, start, end)
else:
data = adapter.fetch_rate_data(key, start, end)
self._register_success(source)
reliability, latency_ms = self._tag(source)
self.usage_log.append(SourceUsageRecord(asset=key, source=source, usage_type=usage_type))
self.logger.info("Fetched %s data for %s via %s", usage_type, key, source)
return DataFetchResult(
data=data,
source=source,
reliability=reliability,
latency_ms=latency_ms,
)
except Exception as exc: # noqa: BLE001
last_err = str(exc)
self._register_failure(source, now)
self.usage_log.append(
SourceUsageRecord(
asset=key,
source=source,
usage_type=usage_type,
success=False,
error=last_err,
)
)
self.logger.warning(
"Failed %s data fetch for %s via %s: %s",
usage_type,
key,
source,
last_err,
)
return DataFetchResult(
data=pd.DataFrame(),
source="none",
reliability="low",
latency_ms=9999,
success=False,
error=last_err,
)
+3
View File
@@ -0,0 +1,3 @@
from fx_quant_engine.integration.hooks import IntegrationHooks
__all__ = ["IntegrationHooks"]
+26
View File
@@ -0,0 +1,26 @@
from __future__ import annotations
import pandas as pd
from fx_quant_engine.schemas import RegimeState
class IntegrationHooks:
def export_scores(self, features: pd.DataFrame, regime: RegimeState) -> dict[str, float]:
if features.empty:
return {
"usd_strength_score": 0.0,
"inr_stress_score": 0.0,
"carry_attractiveness": 0.0,
"fx_volatility_stress": 0.0,
"risk_on_off_score": 0.0,
}
row = features.iloc[-1]
return {
"usd_strength_score": float(row.get("usd_strength_score", 0.0)),
"inr_stress_score": float(row.get("em_stress_proxy", 0.0) + row.get("inr_vol_clustering", 0.0)),
"carry_attractiveness": float(row.get("carry_proxy", 0.0)),
"fx_volatility_stress": float(row.get("realized_vol", 0.0)),
"risk_on_off_score": float(row.get("risk_on_off_proxy", 0.0)),
"regime_stability": float(regime.stability),
}
+4
View File
@@ -0,0 +1,4 @@
from fx_quant_engine.models.ensemble import EnsembleModel
from fx_quant_engine.models.relative_value import RelativeValueModel
__all__ = ["EnsembleModel", "RelativeValueModel"]
+44
View File
@@ -0,0 +1,44 @@
from __future__ import annotations
from typing import Any
import numpy as np
import pandas as pd
class EnsembleModel:
def __init__(self, config: dict[str, Any]) -> None:
self.config = config
def directional_score(self, row: pd.Series) -> tuple[float, dict[str, float], float]:
weights = self.config.get("directional", {}).get("model_weights", {})
component = {
"trend": float(row.get("trend_strength", 0.0)),
"momentum": float(row.get("momentum_multi_horizon", 0.0)),
"mean_reversion": -float(row.get("mean_reversion_zscore", 0.0)),
"carry": float(row.get("carry_proxy", 0.0)),
"macro": float(row.get("risk_on_off_proxy", 0.0)),
}
wsum = sum(float(weights.get(k, 0.0)) for k in component)
score = 0.0
if wsum > 0:
score = sum(component[k] * float(weights.get(k, 0.0)) for k in component) / wsum
normalized = float(np.tanh(score))
signs = [np.sign(v) for v in component.values() if abs(v) > 1e-12]
agreement = float((abs(sum(signs)) / len(signs))) if signs else 0.0
return normalized, component, agreement
def relative_value_score(self, row_a: pd.Series, row_b: pd.Series) -> tuple[float, dict[str, float], float]:
weights = self.config.get("relative_value", {}).get("model_weights", {})
spread = float(row_a.get("mean_reversion_zscore", 0.0) - row_b.get("mean_reversion_zscore", 0.0))
mom_diff = float(row_a.get("momentum_multi_horizon", 0.0) - row_b.get("momentum_multi_horizon", 0.0))
carry_diff = float(row_a.get("carry_proxy", 0.0) - row_b.get("carry_proxy", 0.0))
component = {"spread": spread, "momentum_diff": mom_diff, "carry_diff": carry_diff}
wsum = sum(float(weights.get(k, 0.0)) for k in component)
score = 0.0
if wsum > 0:
score = sum(component[k] * float(weights.get(k, 0.0)) for k in component) / wsum
normalized = float(np.tanh(score))
signs = [np.sign(v) for v in component.values() if abs(v) > 1e-12]
agreement = float((abs(sum(signs)) / len(signs))) if signs else 0.0
return normalized, component, agreement
+15
View File
@@ -0,0 +1,15 @@
"""Placeholder for future model extensions.
This module is intentionally lightweight in the starter phase. Future additions can
include gradient boosting, probabilistic forecasts, and online learning modules.
"""
from __future__ import annotations
from dataclasses import dataclass
@dataclass
class ModelExtensionConfig:
enabled: bool = False
name: str = "baseline_extension"
+90
View File
@@ -0,0 +1,90 @@
from __future__ import annotations
from dataclasses import dataclass
import numpy as np
import pandas as pd
@dataclass
class RelativeValueResult:
score: float
hedge_ratio: float
spread_zscore: float
expected_volatility: float
drivers: dict[str, float]
long_asset: str
short_asset: str
class RelativeValueModel:
def __init__(self, lookback: int = 60) -> None:
self.lookback = lookback
def generate(
self,
asset_a: str,
asset_b: str,
prices_a: pd.Series,
prices_b: pd.Series,
features_a: pd.DataFrame,
features_b: pd.DataFrame,
) -> RelativeValueResult:
pa, pb = self._align(prices_a, prices_b)
la = pa.astype(float).apply(np.log)
lb = pb.astype(float).apply(np.log)
hedge_ratio = self._hedge_ratio(la.tail(self.lookback), lb.tail(self.lookback))
spread = la - hedge_ratio * lb
window = max(20, self.lookback // 2)
zscore = self._zscore(spread, window=window)
spread_z = float(zscore.iloc[-1]) if len(zscore) else 0.0
row_a = features_a.iloc[-1]
row_b = features_b.iloc[-1]
momentum_diff = float(row_a.get("momentum_multi_horizon", 0.0) - row_b.get("momentum_multi_horizon", 0.0))
carry_diff = float(row_a.get("carry_proxy", 0.0) - row_b.get("carry_proxy", 0.0))
raw = -0.60 * spread_z + 0.25 * momentum_diff + 0.15 * carry_diff
score = float(np.tanh(raw))
if score >= 0:
long_asset, short_asset = asset_a, asset_b
else:
long_asset, short_asset = asset_b, asset_a
expected_vol = float(spread.diff().tail(window).std() * np.sqrt(252.0)) if len(spread) > 2 else 0.0
drivers = {
"spread_zscore": spread_z,
"momentum_diff": momentum_diff,
"carry_diff": carry_diff,
"hedge_ratio": float(hedge_ratio),
}
return RelativeValueResult(
score=abs(score),
hedge_ratio=float(hedge_ratio),
spread_zscore=spread_z,
expected_volatility=max(expected_vol, 0.0),
drivers=drivers,
long_asset=long_asset,
short_asset=short_asset,
)
@staticmethod
def _align(a: pd.Series, b: pd.Series) -> tuple[pd.Series, pd.Series]:
joined = pd.concat([a.rename("a"), b.rename("b")], axis=1).dropna()
return joined["a"], joined["b"]
@staticmethod
def _hedge_ratio(a: pd.Series, b: pd.Series) -> float:
denom = float(np.var(b))
if abs(denom) < 1e-12:
return 1.0
return float(np.cov(a, b)[0, 1] / denom)
@staticmethod
def _zscore(series: pd.Series, window: int) -> pd.Series:
mu = series.rolling(window).mean()
sigma = series.rolling(window).std()
return ((series - mu) / (sigma + 1e-9)).fillna(0.0)
+3
View File
@@ -0,0 +1,3 @@
from fx_quant_engine.outputs.formatter import OutputFormatter
__all__ = ["OutputFormatter"]
+37
View File
@@ -0,0 +1,37 @@
from __future__ import annotations
from typing import Any
from fx_quant_engine.schemas import SignalPayload
class OutputFormatter:
def __init__(self, config: dict[str, Any]) -> None:
self.config = config.get("output", {})
def to_json_payload(self, signal: SignalPayload) -> dict[str, Any]:
p = self.config.get("precision", 4)
data = signal.to_dict()
data["signal_strength"] = round(float(data["signal_strength"]), p)
data["confidence"] = round(float(data["confidence"]), p)
data["expected_volatility"] = round(float(data["expected_volatility"]), p)
data["position_sizing_multiplier"] = round(float(data["position_sizing_multiplier"]), p)
return data
def trader_summary(self, signal: SignalPayload) -> str:
return (
f"{signal.asset} | {signal.signal_type} | {signal.signal_direction} | "
f"strength={signal.signal_strength:.2f} confidence={signal.confidence:.2f} "
f"action={signal.recommended_action} size={signal.position_sizing_multiplier:.2f} "
f"regime={signal.regime.get('combined', 'neutral')}"
)
def dashboard_payload(self, signal: SignalPayload) -> dict[str, Any]:
return {
"asset": signal.asset,
"score": signal.signal_strength,
"confidence": signal.confidence,
"action": signal.recommended_action,
"risk_flags": signal.risk_flags,
"regime": signal.regime,
}
@@ -0,0 +1,3 @@
from fx_quant_engine.preprocessing.cleaning import preprocess_market_data
__all__ = ["preprocess_market_data"]
+12
View File
@@ -0,0 +1,12 @@
from __future__ import annotations
import pandas as pd
def preprocess_market_data(df: pd.DataFrame) -> pd.DataFrame:
if df.empty:
return df
out = df.copy().sort_index()
out = out[~out.index.duplicated(keep="last")]
out = out.ffill().dropna(how="all")
return out
+3
View File
@@ -0,0 +1,3 @@
from fx_quant_engine.regime.engine import RegimeEngine
__all__ = ["RegimeEngine"]
+87
View File
@@ -0,0 +1,87 @@
from __future__ import annotations
from typing import Any
import numpy as np
import pandas as pd
from fx_quant_engine.schemas import RegimeState
class RegimeEngine:
def __init__(self, config: dict[str, Any]) -> None:
self.config = config
def detect(self, features: pd.DataFrame) -> RegimeState:
if features.empty:
return RegimeState("neutral", "normal", "neutral", "risk_on", "neutral", 0.0)
row = features.iloc[-1]
trend = self._trend_regime(row)
vol = self._vol_regime(features)
dollar = self._dollar_regime(row)
risk = self._risk_regime(row, vol)
combined = self._combine(trend, vol, dollar, risk)
stability = self._stability(features)
return RegimeState(trend, vol, dollar, risk, combined, stability)
def _trend_regime(self, row: pd.Series) -> str:
score = float(row.get("trend_strength", 0.0))
th = float(self.config.get("trend", {}).get("slope_threshold", 0.0005))
band = float(self.config.get("trend", {}).get("mr_band", 0.003))
if score > th:
return "trend"
if score < -band:
return "mean_reversion"
return "neutral"
def _vol_regime(self, features: pd.DataFrame) -> str:
vol = features.get("realized_vol", pd.Series(0.0, index=features.index)).fillna(0.0)
cur = float(vol.iloc[-1])
ql = float(vol.quantile(self.config.get("volatility", {}).get("low_quantile", 0.3)))
qh = float(vol.quantile(self.config.get("volatility", {}).get("high_quantile", 0.7)))
qs = float(vol.quantile(self.config.get("volatility", {}).get("stress_quantile", 0.9)))
if cur >= qs:
return "stress"
if cur >= qh:
return "high"
if cur <= ql:
return "low"
return "normal"
def _dollar_regime(self, row: pd.Series) -> str:
usd = float(row.get("usd_strength_score", 0.0))
band = float(self.config.get("dollar", {}).get("neutral_band", 0.2))
if usd > band:
return "strengthening"
if usd < -band:
return "weakening"
return "neutral"
def _risk_regime(self, row: pd.Series, vol_regime: str) -> str:
risk_score = float(row.get("risk_on_off_proxy", 0.0))
threshold = float(self.config.get("risk", {}).get("risk_off_equity_threshold", -0.01))
if vol_regime == "stress":
return "stressed"
if risk_score < threshold:
return "risk_off"
return "risk_on"
def _combine(self, trend: str, vol: str, dollar: str, risk: str) -> str:
if vol == "stress" or risk == "stressed":
return "defensive"
if trend == "trend" and risk == "risk_on":
return "pro_trend"
if trend == "mean_reversion" and risk != "stressed":
return "mean_revert"
if dollar == "strengthening":
return "usd_dominant"
return "neutral"
def _stability(self, features: pd.DataFrame) -> float:
cols = [c for c in ["trend_strength", "realized_vol", "usd_strength_score", "risk_on_off_proxy"] if c in features]
if not cols:
return 0.0
tail = features[cols].tail(10)
vol = float(tail.std().mean())
return float(np.clip(1.0 - vol, 0.0, 1.0))
+3
View File
@@ -0,0 +1,3 @@
from fx_quant_engine.risk.engine import RiskEngine
__all__ = ["RiskEngine"]
+55
View File
@@ -0,0 +1,55 @@
from __future__ import annotations
from datetime import datetime, timezone
from typing import Any
from fx_quant_engine.schemas import RegimeState, SignalPayload
from fx_quant_engine.utils.time import is_within_market_hours
class RiskEngine:
def __init__(self, config: dict[str, Any]) -> None:
self.config = config.get("risk", {})
def apply(self, signal: SignalPayload, regime: RegimeState, reliability: str, liquidity_ok: bool, event_risk: float) -> SignalPayload:
penalties = self.config.get("data_reliability_penalty", {})
rel_mult = float(penalties.get(reliability, 0.6))
size = signal.signal_strength * signal.confidence * rel_mult
if regime.volatility in {"high", "stress"}:
size *= float(self.config.get("stressed_regime_multiplier", 0.5))
signal.risk_flags.append("high_volatility_regime")
if not liquidity_ok:
size *= float(self.config.get("illiquid_market_multiplier", 0.6))
signal.risk_flags.append("liquidity_constraint")
if event_risk > 0.6:
size *= 0.5
signal.risk_flags.append("event_risk_high")
if not self._market_open_now():
signal.risk_flags.append("outside_primary_market_hours")
size *= 0.8
signal.position_sizing_multiplier = max(0.0, min(size, 1.0))
signal.recommended_action = self._action(signal.position_sizing_multiplier)
return signal
def _market_open_now(self) -> bool:
now = datetime.now(timezone.utc)
open_utc = self.config.get("market_hours", {}).get("inr_derivatives_open_utc", "03:45")
close_utc = self.config.get("market_hours", {}).get("inr_derivatives_close_utc", "10:00")
return is_within_market_hours(now, open_utc, close_utc)
@staticmethod
def _action(size: float) -> str:
if size >= 0.75:
return "enter"
if size >= 0.45:
return "enter_small"
if size >= 0.25:
return "hold"
if size > 0.0:
return "reduce"
return "no_trade"
+78
View File
@@ -0,0 +1,78 @@
from __future__ import annotations
from dataclasses import asdict, dataclass, field
from datetime import datetime, timezone
from typing import Any
import pandas as pd
@dataclass
class DataFetchResult:
data: pd.DataFrame
source: str
reliability: str
latency_ms: int
success: bool = True
error: str | None = None
@dataclass
class RegimeState:
trend: str
volatility: str
dollar: str
risk: str
combined: str
stability: float
@dataclass
class SignalPayload:
engine: str
timestamp: str
asset: str
signal_type: str
signal_direction: str
signal_strength: float
confidence: float
regime: dict[str, Any]
expected_volatility: float
risk_flags: list[str]
drivers: dict[str, Any]
recommended_action: str
position_sizing_multiplier: float
def to_dict(self) -> dict[str, Any]:
return asdict(self)
@dataclass
class EvaluationReport:
generated_at: str
hit_rate: float
avg_forward_return: float
regime_performance: dict[str, float]
signal_decay: dict[str, float]
feature_importance: dict[str, float]
drift_flags: list[str]
calibration_score: float
@dataclass
class BacktestResult:
pair_level_returns: dict[str, float]
portfolio_return: float
max_drawdown: float
turnover: float
metrics: dict[str, float]
@dataclass
class SourceUsageRecord:
asset: str
source: str
usage_type: str
timestamp: str = field(default_factory=lambda: datetime.now(timezone.utc).isoformat())
success: bool = True
error: str | None = None
+4
View File
@@ -0,0 +1,4 @@
from fx_quant_engine.signals.confidence import ConfidenceEngine
from fx_quant_engine.signals.engine import SignalEngine
__all__ = ["SignalEngine", "ConfidenceEngine"]
+39
View File
@@ -0,0 +1,39 @@
from __future__ import annotations
from typing import Any
import numpy as np
class ConfidenceEngine:
def __init__(self, config: dict[str, Any]) -> None:
self.config = config
def compute(
self,
feature_agreement: float,
model_agreement: float,
regime_stability: float,
data_quality: float,
volatility: float,
event_risk: float,
) -> float:
c = self.config.get("confidence", {})
agreement_w = float(c.get("agreement_weight", 0.25))
regime_w = float(c.get("regime_stability_weight", 0.20))
data_w = float(c.get("data_quality_weight", 0.20))
vol_w = float(c.get("volatility_penalty_weight", 0.20))
event_w = float(c.get("event_penalty_weight", 0.15))
agreement = 0.5 * (feature_agreement + model_agreement)
vol_penalty = np.clip(volatility, 0.0, 1.0)
event_penalty = np.clip(event_risk, 0.0, 1.0)
raw = (
agreement_w * agreement
+ regime_w * np.clip(regime_stability, 0.0, 1.0)
+ data_w * np.clip(data_quality, 0.0, 1.0)
- vol_w * vol_penalty
- event_w * event_penalty
)
return float(np.clip((raw + 1.0) / 2.0, 0.0, 1.0))
+74
View File
@@ -0,0 +1,74 @@
from __future__ import annotations
from datetime import datetime, timezone
from typing import Any
import numpy as np
import pandas as pd
from fx_quant_engine.schemas import RegimeState, SignalPayload
class SignalEngine:
def __init__(self, engine_name: str = "fx_quant_engine") -> None:
self.engine_name = engine_name
def directional_signal(
self,
asset: str,
feature_row: pd.Series,
regime: RegimeState,
score: float,
confidence: float,
drivers: dict[str, float],
expected_volatility: float,
) -> SignalPayload:
direction = self._direction(asset, score)
return SignalPayload(
engine=self.engine_name,
timestamp=datetime.now(timezone.utc).isoformat(),
asset=asset,
signal_type="directional",
signal_direction=direction,
signal_strength=float(np.clip(abs(score), 0.0, 1.0)),
confidence=float(np.clip(confidence, 0.0, 1.0)),
regime=regime.__dict__,
expected_volatility=float(max(0.0, expected_volatility)),
risk_flags=[],
drivers={k: float(v) for k, v in drivers.items()},
recommended_action="hold",
position_sizing_multiplier=0.0,
)
def relative_value_signal(
self,
long_asset: str,
short_asset: str,
regime: RegimeState,
score: float,
confidence: float,
drivers: dict[str, float],
expected_volatility: float,
) -> SignalPayload:
direction = f"long_{long_asset}_vs_{short_asset}"
asset = f"{long_asset}:{short_asset}"
return SignalPayload(
engine=self.engine_name,
timestamp=datetime.now(timezone.utc).isoformat(),
asset=asset,
signal_type="relative_value",
signal_direction=direction,
signal_strength=float(np.clip(abs(score), 0.0, 1.0)),
confidence=float(np.clip(confidence, 0.0, 1.0)),
regime=regime.__dict__,
expected_volatility=float(max(0.0, expected_volatility)),
risk_flags=[],
drivers={k: float(v) for k, v in drivers.items()},
recommended_action="hold",
position_sizing_multiplier=0.0,
)
def _direction(self, asset: str, score: float) -> str:
if asset.startswith("USD"):
return "long_usd" if score >= 0 else "short_usd"
return "long_base" if score >= 0 else "short_base"
+4
View File
@@ -0,0 +1,4 @@
from fx_quant_engine.utils.config import load_all_configs, load_yaml
from fx_quant_engine.utils.logging import get_logger
__all__ = ["load_yaml", "load_all_configs", "get_logger"]
+27
View File
@@ -0,0 +1,27 @@
from __future__ import annotations
from pathlib import Path
from typing import Any
import yaml
def load_yaml(path: str | Path) -> dict[str, Any]:
with Path(path).open("r", encoding="utf-8") as f:
data = yaml.safe_load(f) or {}
if not isinstance(data, dict):
raise ValueError(f"Config file must contain a mapping: {path}")
return data
def load_all_configs(config_dir: str | Path) -> dict[str, dict[str, Any]]:
cfg_dir = Path(config_dir)
return {
"universe": load_yaml(cfg_dir / "universe.yaml"),
"data_sources": load_yaml(cfg_dir / "data_sources.yaml"),
"features": load_yaml(cfg_dir / "features.yaml"),
"regimes": load_yaml(cfg_dir / "regimes.yaml"),
"models": load_yaml(cfg_dir / "models.yaml"),
"risk": load_yaml(cfg_dir / "risk.yaml"),
"output": load_yaml(cfg_dir / "output.yaml"),
}
+14
View File
@@ -0,0 +1,14 @@
from __future__ import annotations
import logging
def get_logger(name: str) -> logging.Logger:
logger = logging.getLogger(name)
if not logger.handlers:
handler = logging.StreamHandler()
fmt = "%(asctime)s | %(name)s | %(levelname)s | %(message)s"
handler.setFormatter(logging.Formatter(fmt))
logger.addHandler(handler)
logger.setLevel(logging.INFO)
return logger
+16
View File
@@ -0,0 +1,16 @@
from __future__ import annotations
from datetime import datetime, time, timezone
def is_within_market_hours(
now_utc: datetime,
open_hhmm: str,
close_hhmm: str,
) -> bool:
open_h, open_m = [int(x) for x in open_hhmm.split(":")]
close_h, close_m = [int(x) for x in close_hhmm.split(":")]
open_t = time(hour=open_h, minute=open_m)
close_t = time(hour=close_h, minute=close_m)
t = now_utc.astimezone(timezone.utc).time()
return open_t <= t <= close_t
+47
View File
@@ -0,0 +1,47 @@
name: options-quant-engine-ci
on:
push:
paths:
- "options_quant_engine/**"
pull_request:
paths:
- "options_quant_engine/**"
workflow_dispatch:
jobs:
lint-test-build:
runs-on: ubuntu-latest
defaults:
run:
working-directory: options_quant_engine
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -e .[dev]
- name: Lint
run: ruff check .
- name: Test
run: pytest
- name: Generate sample run artifact
run: python scripts/run_engine.py
- name: Upload run artifacts
uses: actions/upload-artifact@v4
with:
name: options-engine-runs
path: options_quant_engine/examples/runs/*.json
if-no-files-found: warn
+193
View File
@@ -0,0 +1,193 @@
# options_quant_engine
A professional, modular, explainable, and production-ready options quant engine with a clean, extensible architecture for standalone deployment.
## What This Engine Does
- Ingests market/macro/rate data from multiple pluggable adapters
- Uses configurable source priority and fallback logic per asset
- Engineers economically meaningful features through a registry-based pipeline
- Detects interpretable market regimes
- Generates directional and relative-value signals
- Separates signal strength from confidence
- Applies risk overlays and outputs action recommendations
- Supports evaluation and realistic backtest scaffolding
- Exports integration hooks for cross-engine multi-asset platforms
## Underlying Universe
Primary INR pairs:
- USDINR
- EURINR
- GBPINR
- JPYINR
Secondary G10 pairs:
- EURUSD
- GBPUSD
- USDJPY
- AUDUSD
- USDCAD
- USDCHF
## Repository Structure
- options_quant_engine/ingestion: adapters + fallback router
- options_quant_engine/preprocessing: data cleaning/alignment
- options_quant_engine/features: registry + modular feature pipeline
- options_quant_engine/regime: explainable regime detection
- options_quant_engine/models: model scoring and extension points
- options_quant_engine/signals: directional/RV signal + confidence engine
- options_quant_engine/risk: risk overlays and action mapping
- options_quant_engine/evaluation: signal quality diagnostics
- options_quant_engine/backtest: no-lookahead pair/portfolio simulation
- options_quant_engine/outputs: machine-readable + trader-readable payloads
- options_quant_engine/integration: cross-engine score exports
- config: YAML-driven behavior
- scripts: runnable examples
- tests: unit tests
- examples: sample payloads and run artifacts
## Data Adapter Architecture
Base interface:
```python
class BaseDataAdapter:
def fetch_price_data(self, asset, start, end):
pass
def fetch_macro_data(self, key, start, end):
pass
def fetch_rate_data(self, asset, start, end):
pass
def health_check(self):
pass
```
Included adapters:
- BreezeAdapter (ICICI Breeze)
- ZerodhaAdapter (optional via config)
- NSEAdapter
- RBIAdapter
- FreeFXAdapter
- MockAdapter
Live API stubs are environment-driven and include strict timeout/retry logic:
- OQE_HTTP_TIMEOUT_SEC
- OQE_HTTP_MAX_ATTEMPTS
- OQE_HTTP_BACKOFF_SEC
- BREEZE_LIVE_ENABLED, BREEZE_BASE_URL, BREEZE_API_KEY, BREEZE_API_SECRET
- ZERODHA_LIVE_ENABLED, ZERODHA_BASE_URL, ZERODHA_API_KEY, ZERODHA_ACCESS_TOKEN
- NSE_LIVE_ENABLED, NSE_BASE_URL
- RBI_LIVE_ENABLED, RBI_BASE_URL
## Config-Driven Files
- config/universe.yaml
- config/data_sources.yaml
- config/features.yaml
- config/regimes.yaml
- config/models.yaml
- config/risk.yaml
- config/output.yaml
## Quickstart
1. Create and activate a Python 3.10+ environment.
1. Install package and dev dependencies:
```bash
pip install -e .[dev]
```
1. Run tests:
```bash
pytest
```
1. Run engine example:
```bash
python scripts/run_engine.py
```
The script saves artifacts to examples/runs.
## Signal Output Schema
Each signal includes:
- direction
- signal strength
- confidence (separate module)
- regime context
- expected volatility
- key drivers
- risk flags
- recommended action
- position sizing multiplier
See examples/sample_signal.json.
## Relative Value Spread Model
Relative-value signal generation uses a true spread framework:
- explicit hedge ratio estimation from aligned log prices
- spread z-score signal component
- pair-level momentum and carry differentials
- expected spread volatility and long/short leg assignment
The output includes `hedge_ratio` and `spread_zscore` in RV payloads.
## CI
GitHub Actions workflow: `.github/workflows/ci.yml`
- ruff lint checks
- pytest execution
- sample run generation
- artifact upload of `examples/runs/*.json`
## Build Phases Coverage
Phase 1:
- repo structure
- config system
- ingestion + preprocessing
- basic features
Phase 2:
- regime detection
- signal engine
- confidence engine
- outputs
Phase 3:
- risk layer
- evaluation framework
- backtesting scaffolding
Phase 4 starter:
- model extension hooks
- integration hooks
- tests
- docs
## Notes
- No credentials are hardcoded.
- Live adapter wiring is stubbed with production-style retries/timeouts and env credentials.
- Mock pathways remain default for deterministic testing and offline development.
@@ -0,0 +1,50 @@
enabled_sources:
breeze: true
zerodha: false
nse: true
rbi: true
freefx: true
mock: true
source_priority:
default: [nse, rbi, freefx, mock]
g10: [freefx, mock]
asset_source_map:
USDINR: [nse, breeze, rbi, mock]
EURINR: [nse, breeze, freefx, mock]
GBPINR: [nse, freefx, mock]
JPYINR: [nse, freefx, mock]
EURUSD: [freefx, mock]
GBPUSD: [freefx, mock]
USDJPY: [freefx, mock]
AUDUSD: [freefx, mock]
USDCAD: [freefx, mock]
USDCHF: [freefx, mock]
reliability_tags:
breeze: medium
zerodha: medium
nse: high
rbi: high
freefx: medium
mock: low
latency_tags_ms:
breeze: 350
zerodha: 250
nse: 500
rbi: 800
freefx: 700
mock: 1
live_api:
breeze_live_enabled: false
zerodha_live_enabled: false
nse_live_enabled: false
rbi_live_enabled: false
http_controls:
timeout_sec: 5
max_attempts: 3
backoff_sec: 0.5
+28
View File
@@ -0,0 +1,28 @@
features:
returns: true
moving_averages: true
breakouts: true
trend_strength: true
momentum_multi_horizon: true
momentum_persistence: true
momentum_acceleration: true
mean_reversion_zscore: true
bollinger_position: true
realized_vol: true
atr_proxy: true
vol_regime_proxy: true
carry_proxy: true
usd_strength_score: true
risk_on_off_proxy: true
inr_crude_proxy: true
em_stress_proxy: true
inr_vol_clustering: true
rbi_intervention_proxy: true
cross_asset_bond_proxy: true
cross_asset_commodity_proxy: true
cross_asset_equity_proxy: true
windows:
short: 5
medium: 20
long: 60
+22
View File
@@ -0,0 +1,22 @@
directional:
model_weights:
trend: 0.35
momentum: 0.25
mean_reversion: 0.15
carry: 0.10
macro: 0.15
relative_value:
lookback: 60
spread_zscore_threshold: 1.0
model_weights:
spread: 0.5
momentum_diff: 0.3
carry_diff: 0.2
confidence:
agreement_weight: 0.25
regime_stability_weight: 0.20
data_quality_weight: 0.20
volatility_penalty_weight: 0.20
event_penalty_weight: 0.15
+5
View File
@@ -0,0 +1,5 @@
output:
include_dashboard_payload: true
include_trader_summary: true
precision: 4
engine_name: options_quant_engine
+15
View File
@@ -0,0 +1,15 @@
trend:
slope_threshold: 0.0005
mr_band: 0.003
volatility:
low_quantile: 0.3
high_quantile: 0.7
stress_quantile: 0.9
dollar:
neutral_band: 0.2
risk:
stress_vol_threshold: 0.75
risk_off_equity_threshold: -0.01
+13
View File
@@ -0,0 +1,13 @@
risk:
target_vol: 0.10
max_gross_exposure: 1.0
max_single_asset_exposure: 0.25
stressed_regime_multiplier: 0.5
illiquid_market_multiplier: 0.6
data_reliability_penalty:
high: 1.0
medium: 0.85
low: 0.6
market_hours:
inr_derivatives_open_utc: "03:45"
inr_derivatives_close_utc: "10:00"
+12
View File
@@ -0,0 +1,12 @@
primary_inr_pairs:
- USDINR
- EURINR
- GBPINR
- JPYINR
secondary_g10_pairs:
- EURUSD
- GBPUSD
- USDJPY
- AUDUSD
- USDCAD
- USDCHF
@@ -0,0 +1,28 @@
{
"engine": "options_quant_engine",
"timestamp": "2026-03-30T10:00:00",
"asset": "USDINR",
"signal_type": "directional",
"signal_direction": "long_usd",
"signal_strength": 0.74,
"confidence": 0.68,
"regime": {
"trend": "trend",
"volatility": "normal",
"dollar": "strengthening",
"risk": "risk_off",
"combined": "usd_dominant",
"stability": 0.71
},
"expected_volatility": 0.089,
"risk_flags": ["outside_primary_market_hours"],
"drivers": {
"trend": 0.51,
"momentum": 0.36,
"mean_reversion": -0.12,
"carry": 0.09,
"macro": -0.07
},
"recommended_action": "enter_small",
"position_sizing_multiplier": 0.65
}
@@ -0,0 +1,5 @@
"""Options Quant Engine package."""
from options_quant_engine.engine import OptionsQuantEngine
__all__ = ["OptionsQuantEngine"]
@@ -0,0 +1,3 @@
from options_quant_engine.backtest.engine import BacktestEngine
__all__ = ["BacktestEngine"]
@@ -0,0 +1,54 @@
from __future__ import annotations
import numpy as np
import pandas as pd
from options_quant_engine.schemas import BacktestResult
class BacktestEngine:
def __init__(self, transaction_cost_bps: float = 1.5, slippage_bps: float = 1.0) -> None:
self.tc = transaction_cost_bps / 10000.0
self.slippage = slippage_bps / 10000.0
def simulate_pair(self, prices: pd.Series, signal_strength: pd.Series) -> pd.Series:
returns = prices.pct_change().fillna(0.0)
position = signal_strength.shift(1).fillna(0.0) # no lookahead
traded = position.diff().abs().fillna(0.0)
net = position * returns - traded * (self.tc + self.slippage)
return net
def simulate_portfolio(self, pair_returns: dict[str, pd.Series]) -> BacktestResult:
if not pair_returns:
return BacktestResult({}, 0.0, 0.0, 0.0, {})
df = pd.DataFrame(pair_returns).fillna(0.0)
port = df.mean(axis=1)
equity = (1.0 + port).cumprod()
dd = equity / equity.cummax() - 1.0
pair_level = {k: float(v.mean() * 252.0) for k, v in pair_returns.items()}
metrics = {
"annualized_return": float(port.mean() * 252.0),
"annualized_vol": float(port.std() * np.sqrt(252.0)),
"sharpe": float((port.mean() / (port.std() + 1e-9)) * np.sqrt(252.0)),
}
return BacktestResult(
pair_level_returns=pair_level,
portfolio_return=float((equity.iloc[-1] - 1.0) if len(equity) else 0.0),
max_drawdown=float(dd.min() if len(dd) else 0.0),
turnover=float(df.diff().abs().sum().sum()),
metrics=metrics,
)
def walk_forward(self, prices: pd.Series, signal_strength: pd.Series, train_window: int = 120, test_window: int = 20) -> pd.Series:
out = []
idx = prices.index
i = train_window
while i < len(idx):
end = min(i + test_window, len(idx))
segment = self.simulate_pair(prices.iloc[:end], signal_strength.iloc[:end])
out.append(segment.iloc[i:end])
i += test_window
return pd.concat(out).sort_index() if out else pd.Series(dtype=float)
@@ -0,0 +1,210 @@
from __future__ import annotations
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
import numpy as np
import pandas as pd
from options_quant_engine.features.pipeline import FeaturePipeline
from options_quant_engine.ingestion.adapters import (
BreezeAdapter,
FreeFXAdapter,
MockAdapter,
NSEAdapter,
RBIAdapter,
ZerodhaAdapter,
)
from options_quant_engine.ingestion.router import DataSourceRouter
from options_quant_engine.integration.hooks import IntegrationHooks
from options_quant_engine.models.ensemble import EnsembleModel
from options_quant_engine.models.relative_value import RelativeValueModel
from options_quant_engine.outputs.formatter import OutputFormatter
from options_quant_engine.preprocessing.cleaning import preprocess_market_data
from options_quant_engine.regime.engine import RegimeEngine
from options_quant_engine.risk.engine import RiskEngine
from options_quant_engine.schemas import SignalPayload
from options_quant_engine.signals.confidence import ConfidenceEngine
from options_quant_engine.signals.engine import SignalEngine
from options_quant_engine.utils.config import load_all_configs
class OptionsQuantEngine:
def __init__(self, config_dir: str | Path = "config") -> None:
self.config = load_all_configs(config_dir)
self.adapters = {
"breeze": BreezeAdapter(),
"zerodha": ZerodhaAdapter(),
"nse": NSEAdapter(),
"rbi": RBIAdapter(),
"freefx": FreeFXAdapter(),
"mock": MockAdapter(),
}
self.router = DataSourceRouter(self.adapters, self.config["data_sources"])
self.features = FeaturePipeline(self.config["features"])
self.regimes = RegimeEngine(self.config["regimes"])
self.models = EnsembleModel(self.config["models"])
self.rv_model = RelativeValueModel(lookback=int(self.config["models"].get("relative_value", {}).get("lookback", 60)))
self.confidence = ConfidenceEngine(self.config["models"])
self.signals = SignalEngine(engine_name=self.config["output"]["output"].get("engine_name", "options_quant_engine"))
self.risk = RiskEngine(self.config["risk"])
self.output = OutputFormatter(self.config["output"])
self.integration = IntegrationHooks()
def _asset_context(self, asset: str, start: datetime, end: datetime) -> dict[str, Any]:
px_res = self.router.fetch_price_data(asset, start, end)
macro_res = self.router.fetch_macro_data("macro_proxy", start, end)
rate_res = self.router.fetch_rate_data(asset, start, end)
if not px_res.success or px_res.data.empty:
raise RuntimeError(f"No price data available for {asset}: {px_res.error}")
df = px_res.data.join(macro_res.data, how="left").join(rate_res.data, how="left")
if f"{asset}_rate" in df.columns:
df["short_rate"] = df[f"{asset}_rate"]
df["usd_rate"] = 0.045
df["equity_proxy"] = df["close"].pct_change().rolling(5).mean().fillna(0.0)
df["crude_proxy"] = (1.0 + df["close"].pct_change().fillna(0.0)).cumprod()
df["emfx_proxy"] = df["close"].pct_change().fillna(0.0)
clean = preprocess_market_data(df)
feat = self.features.transform(clean)
regime = self.regimes.detect(feat)
return {
"price": px_res,
"macro": macro_res,
"rate": rate_res,
"clean": clean,
"features": feat,
"regime": regime,
}
def run_asset(self, asset: str, start: datetime, end: datetime) -> dict[str, Any]:
ctx = self._asset_context(asset, start, end)
px_res = ctx["price"]
feat = ctx["features"]
regime = ctx["regime"]
row = feat.iloc[-1]
score, model_drivers, model_agreement = self.models.directional_score(row)
feature_agreement = float((row > 0).mean())
data_quality = 1.0 if px_res.reliability == "high" else 0.8 if px_res.reliability == "medium" else 0.6
volatility = float(min(1.0, row.get("realized_vol", 0.0)))
event_risk = float(min(1.0, row.get("rbi_intervention_proxy", 0.0)))
conf = self.confidence.compute(
feature_agreement=feature_agreement,
model_agreement=model_agreement,
regime_stability=regime.stability,
data_quality=data_quality,
volatility=volatility,
event_risk=event_risk,
)
sig: SignalPayload = self.signals.directional_signal(
asset=asset,
feature_row=row,
regime=regime,
score=score,
confidence=conf,
drivers=model_drivers,
expected_volatility=float(row.get("realized_vol", 0.0)),
)
sig = self.risk.apply(
signal=sig,
regime=regime,
reliability=px_res.reliability,
liquidity_ok=True,
event_risk=event_risk,
)
payload = self.output.to_json_payload(sig)
summary = self.output.trader_summary(sig)
dashboard = self.output.dashboard_payload(sig)
hooks = self.integration.export_scores(feat, regime)
return {
"signal": payload,
"summary": summary,
"dashboard": dashboard,
"integration_hooks": hooks,
"source_usage": [u.__dict__ for u in self.router.usage_log],
}
def run_relative_value(self, asset_a: str, asset_b: str, start: datetime, end: datetime) -> dict[str, Any]:
ctx_a = self._asset_context(asset_a, start, end)
ctx_b = self._asset_context(asset_b, start, end)
feat_a = ctx_a["features"]
feat_b = ctx_b["features"]
rv = self.rv_model.generate(
asset_a=asset_a,
asset_b=asset_b,
prices_a=ctx_a["clean"]["close"],
prices_b=ctx_b["clean"]["close"],
features_a=feat_a,
features_b=feat_b,
)
regime = ctx_a["regime"] if ctx_a["regime"].stability >= ctx_b["regime"].stability else ctx_b["regime"]
driver_vals = [v for k, v in rv.drivers.items() if k != "hedge_ratio"]
signs = [np.sign(v) for v in driver_vals if abs(v) > 1e-12]
model_agreement = float(abs(sum(signs)) / len(signs)) if signs else 0.0
feature_agreement = float((feat_a.iloc[-1] - feat_b.iloc[-1] > 0).mean())
rel_map = {"high": 1.0, "medium": 0.8, "low": 0.6}
q_a = rel_map.get(ctx_a["price"].reliability, 0.6)
q_b = rel_map.get(ctx_b["price"].reliability, 0.6)
data_quality = float((q_a + q_b) / 2.0)
volatility = float(min(1.0, rv.expected_volatility))
event_risk = float(
min(
1.0,
(
float(feat_a.iloc[-1].get("rbi_intervention_proxy", 0.0))
+ float(feat_b.iloc[-1].get("rbi_intervention_proxy", 0.0))
)
/ 2.0,
)
)
conf = self.confidence.compute(
feature_agreement=feature_agreement,
model_agreement=model_agreement,
regime_stability=regime.stability,
data_quality=data_quality,
volatility=volatility,
event_risk=event_risk,
)
sig = self.signals.relative_value_signal(
long_asset=rv.long_asset,
short_asset=rv.short_asset,
regime=regime,
score=rv.score,
confidence=conf,
drivers=rv.drivers,
expected_volatility=rv.expected_volatility,
)
reliability = "high" if min(q_a, q_b) >= 1.0 else "medium" if min(q_a, q_b) >= 0.8 else "low"
sig = self.risk.apply(
signal=sig,
regime=regime,
reliability=reliability,
liquidity_ok=True,
event_risk=event_risk,
)
return {
"relative_value_pair": f"{asset_a}:{asset_b}",
"long": rv.long_asset,
"short": rv.short_asset,
"spread_score": rv.score,
"hedge_ratio": rv.hedge_ratio,
"spread_zscore": rv.spread_zscore,
"signal": self.output.to_json_payload(sig),
"summary": self.output.trader_summary(sig),
"dashboard": self.output.dashboard_payload(sig),
}
@@ -0,0 +1,3 @@
from options_quant_engine.evaluation.engine import Evaluator
__all__ = ["Evaluator"]
@@ -0,0 +1,67 @@
from __future__ import annotations
from datetime import datetime, timezone
import numpy as np
import pandas as pd
from options_quant_engine.schemas import EvaluationReport
class Evaluator:
def evaluate(
self,
signals: pd.DataFrame,
forward_returns: pd.Series,
regimes: pd.Series,
feature_importance: dict[str, float],
) -> EvaluationReport:
if signals.empty or forward_returns.empty:
return EvaluationReport(
generated_at=datetime.now(timezone.utc).isoformat(),
hit_rate=0.0,
avg_forward_return=0.0,
regime_performance={},
signal_decay={},
feature_importance=feature_importance,
drift_flags=["insufficient_data"],
calibration_score=0.0,
)
aligned = signals.join(forward_returns.rename("fwd"), how="inner")
aligned = aligned.join(regimes.rename("regime"), how="left").fillna("unknown")
pred = np.sign(aligned["signal_strength"] * aligned.get("signal_sign", 1.0))
realized = np.sign(aligned["fwd"])
hit_rate = float((pred == realized).mean())
avg_ret = float(aligned["fwd"].mean())
regime_perf = {
r: float(v)
for r, v in aligned.groupby("regime")["fwd"].mean().to_dict().items()
}
signal_decay = {
"1d": float(aligned["fwd"].mean()),
"5d": float(aligned["fwd"].rolling(5).mean().dropna().mean() if len(aligned) >= 5 else 0.0),
"20d": float(aligned["fwd"].rolling(20).mean().dropna().mean() if len(aligned) >= 20 else 0.0),
}
drift = []
if abs(avg_ret) < 1e-4:
drift.append("low_signal_edge")
if hit_rate < 0.48:
drift.append("hit_rate_deterioration")
conf = aligned.get("confidence", pd.Series(0.5, index=aligned.index))
calibration_score = float(np.clip(1.0 - abs(conf.mean() - hit_rate), 0.0, 1.0))
return EvaluationReport(
generated_at=datetime.now(timezone.utc).isoformat(),
hit_rate=hit_rate,
avg_forward_return=avg_ret,
regime_performance=regime_perf,
signal_decay=signal_decay,
feature_importance=feature_importance,
drift_flags=drift,
calibration_score=calibration_score,
)
@@ -0,0 +1,3 @@
from options_quant_engine.features.pipeline import FeaturePipeline
__all__ = ["FeaturePipeline"]
@@ -0,0 +1,27 @@
from __future__ import annotations
from collections.abc import Callable
from typing import Any
import pandas as pd
FeatureFn = Callable[[pd.DataFrame, dict[str, Any]], pd.Series]
class FeatureRegistry:
def __init__(self) -> None:
self._features: dict[str, FeatureFn] = {}
def register(self, name: str, fn: FeatureFn) -> None:
if name in self._features:
raise ValueError(f"Feature already registered: {name}")
self._features[name] = fn
def get(self, name: str) -> FeatureFn:
return self._features[name]
def names(self) -> list[str]:
return list(self._features.keys())
registry = FeatureRegistry()
@@ -0,0 +1,164 @@
from __future__ import annotations
import numpy as np
import pandas as pd
from options_quant_engine.features.base import registry
def _close(df: pd.DataFrame) -> pd.Series:
if "close" in df.columns:
return df["close"]
numeric = df.select_dtypes(include="number")
if numeric.empty:
return pd.Series(0.0, index=df.index)
return numeric.iloc[:, 0]
def returns(df: pd.DataFrame, cfg: dict) -> pd.Series:
return _close(df).pct_change().fillna(0.0)
def moving_averages(df: pd.DataFrame, cfg: dict) -> pd.Series:
w = cfg["windows"]["medium"]
c = _close(df)
return (c / c.rolling(w).mean() - 1.0).fillna(0.0)
def breakouts(df: pd.DataFrame, cfg: dict) -> pd.Series:
w = cfg["windows"]["long"]
c = _close(df)
high = c.rolling(w).max()
low = c.rolling(w).min()
rng = (high - low).replace(0.0, np.nan)
return ((c - low) / rng).fillna(0.5)
def trend_strength(df: pd.DataFrame, cfg: dict) -> pd.Series:
w = cfg["windows"]["medium"]
r = returns(df, cfg)
return (r.rolling(w).mean() / (r.rolling(w).std() + 1e-9)).fillna(0.0)
def momentum_multi_horizon(df: pd.DataFrame, cfg: dict) -> pd.Series:
c = _close(df)
s = cfg["windows"]["short"]
m = cfg["windows"]["medium"]
l = cfg["windows"]["long"]
return ((c.pct_change(s) + c.pct_change(m) + c.pct_change(l)) / 3.0).fillna(0.0)
def momentum_persistence(df: pd.DataFrame, cfg: dict) -> pd.Series:
r = returns(df, cfg)
return r.rolling(cfg["windows"]["short"]).apply(lambda x: float((x > 0).mean()), raw=False).fillna(0.5)
def momentum_acceleration(df: pd.DataFrame, cfg: dict) -> pd.Series:
m = momentum_multi_horizon(df, cfg)
return m.diff().fillna(0.0)
def mean_reversion_zscore(df: pd.DataFrame, cfg: dict) -> pd.Series:
c = _close(df)
w = cfg["windows"]["medium"]
m = c.rolling(w).mean()
s = c.rolling(w).std()
return ((c - m) / (s + 1e-9)).fillna(0.0)
def bollinger_position(df: pd.DataFrame, cfg: dict) -> pd.Series:
z = mean_reversion_zscore(df, cfg)
return (z / 2.0).clip(-1, 1)
def realized_vol(df: pd.DataFrame, cfg: dict) -> pd.Series:
r = returns(df, cfg)
return (r.rolling(cfg["windows"]["medium"]).std() * np.sqrt(252)).fillna(0.0)
def atr_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
c = _close(df)
return c.diff().abs().rolling(cfg["windows"]["short"]).mean().fillna(0.0)
def vol_regime_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
vol = realized_vol(df, cfg)
return (vol / (vol.rolling(cfg["windows"]["long"]).mean() + 1e-9)).fillna(1.0)
def carry_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
short_r = df.get("short_rate", pd.Series(0.05, index=df.index))
usd_r = df.get("usd_rate", pd.Series(0.045, index=df.index))
return (short_r - usd_r).fillna(0.0)
def usd_strength_score(df: pd.DataFrame, cfg: dict) -> pd.Series:
r = returns(df, cfg)
return (-r.rolling(cfg["windows"]["medium"]).mean()).fillna(0.0)
def risk_on_off_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
eq = df.get("equity_proxy", returns(df, cfg)).fillna(0.0)
vix = df.get("vix_proxy", realized_vol(df, cfg)).fillna(0.0)
return (eq - vix).fillna(0.0)
def inr_crude_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
crude = df.get("crude_proxy", pd.Series(0.0, index=df.index))
return crude.pct_change().fillna(0.0)
def em_stress_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
em = df.get("emfx_proxy", returns(df, cfg)).fillna(0.0)
return (-em.rolling(cfg["windows"]["medium"]).mean()).fillna(0.0)
def inr_vol_clustering(df: pd.DataFrame, cfg: dict) -> pd.Series:
r = returns(df, cfg)
return r.abs().rolling(cfg["windows"]["short"]).mean().fillna(0.0)
def rbi_intervention_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
z = mean_reversion_zscore(df, cfg)
return (z.abs() > 2.0).astype(float)
def cross_asset_bond_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
y = df.get("bond_yield_proxy", pd.Series(0.0, index=df.index))
return y.diff().fillna(0.0)
def cross_asset_commodity_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
g = df.get("gold_proxy", pd.Series(0.0, index=df.index))
o = df.get("oil_proxy", pd.Series(0.0, index=df.index))
return (g.pct_change().fillna(0.0) + o.pct_change().fillna(0.0)) / 2.0
def cross_asset_equity_proxy(df: pd.DataFrame, cfg: dict) -> pd.Series:
e = df.get("equity_proxy", pd.Series(0.0, index=df.index))
return e.pct_change().fillna(0.0)
def register_builtin_features() -> None:
registry.register("returns", returns)
registry.register("moving_averages", moving_averages)
registry.register("breakouts", breakouts)
registry.register("trend_strength", trend_strength)
registry.register("momentum_multi_horizon", momentum_multi_horizon)
registry.register("momentum_persistence", momentum_persistence)
registry.register("momentum_acceleration", momentum_acceleration)
registry.register("mean_reversion_zscore", mean_reversion_zscore)
registry.register("bollinger_position", bollinger_position)
registry.register("realized_vol", realized_vol)
registry.register("atr_proxy", atr_proxy)
registry.register("vol_regime_proxy", vol_regime_proxy)
registry.register("carry_proxy", carry_proxy)
registry.register("usd_strength_score", usd_strength_score)
registry.register("risk_on_off_proxy", risk_on_off_proxy)
registry.register("inr_crude_proxy", inr_crude_proxy)
registry.register("em_stress_proxy", em_stress_proxy)
registry.register("inr_vol_clustering", inr_vol_clustering)
registry.register("rbi_intervention_proxy", rbi_intervention_proxy)
registry.register("cross_asset_bond_proxy", cross_asset_bond_proxy)
registry.register("cross_asset_commodity_proxy", cross_asset_commodity_proxy)
registry.register("cross_asset_equity_proxy", cross_asset_equity_proxy)
@@ -0,0 +1,26 @@
from __future__ import annotations
from typing import Any
import pandas as pd
from options_quant_engine.features.base import registry
from options_quant_engine.features.builtins import register_builtin_features
class FeaturePipeline:
def __init__(self, config: dict[str, Any]) -> None:
self.config = config
if not registry.names():
register_builtin_features()
def transform(self, df: pd.DataFrame) -> pd.DataFrame:
feature_cfg = self.config.get("features", {})
out = pd.DataFrame(index=df.index)
for feature_name, enabled in feature_cfg.items():
if not enabled:
continue
if feature_name not in registry.names():
continue
out[feature_name] = registry.get(feature_name)(df, self.config)
return out.fillna(0.0)
@@ -0,0 +1,21 @@
from options_quant_engine.ingestion.adapters import (
BreezeAdapter,
FreeFXAdapter,
MockAdapter,
NSEAdapter,
RBIAdapter,
ZerodhaAdapter,
)
from options_quant_engine.ingestion.base import BaseDataAdapter
from options_quant_engine.ingestion.router import DataSourceRouter
__all__ = [
"BaseDataAdapter",
"BreezeAdapter",
"ZerodhaAdapter",
"NSEAdapter",
"RBIAdapter",
"FreeFXAdapter",
"MockAdapter",
"DataSourceRouter",
]
@@ -0,0 +1,298 @@
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime
import os
from typing import Any
import numpy as np
import pandas as pd
from options_quant_engine.ingestion.base import BaseDataAdapter
from options_quant_engine.ingestion.credentials import APICredentials, load_credentials
from options_quant_engine.ingestion.http_client import HttpClient, RetryConfig
def _mock_price_series(asset: str, start: datetime, end: datetime, seed: int) -> pd.DataFrame:
idx = pd.date_range(start=start, end=end, freq="B")
rng = np.random.default_rng(seed)
steps = rng.normal(loc=0.0, scale=0.002, size=len(idx))
base = 80.0 if asset.endswith("INR") else 1.1
px = base * np.exp(np.cumsum(steps))
return pd.DataFrame({"close": px}, index=idx)
def _mock_macro_series(key: str, start: datetime, end: datetime, seed: int) -> pd.DataFrame:
idx = pd.date_range(start=start, end=end, freq="B")
rng = np.random.default_rng(seed)
values = rng.normal(loc=0.0, scale=1.0, size=len(idx))
return pd.DataFrame({key: values}, index=idx)
def _mock_rate_series(asset: str, start: datetime, end: datetime, seed: int) -> pd.DataFrame:
idx = pd.date_range(start=start, end=end, freq="B")
rng = np.random.default_rng(seed)
rate = 0.05 + rng.normal(loc=0.0, scale=0.002, size=len(idx))
return pd.DataFrame({f"{asset}_rate": rate}, index=idx)
def _retry_config_from_env() -> RetryConfig:
return RetryConfig(
timeout_sec=float(os.getenv("OQE_HTTP_TIMEOUT_SEC", "5.0")),
max_attempts=int(os.getenv("OQE_HTTP_MAX_ATTEMPTS", "3")),
backoff_sec=float(os.getenv("OQE_HTTP_BACKOFF_SEC", "0.5")),
)
def _to_price_frame(payload: dict[str, Any], start: datetime, end: datetime) -> pd.DataFrame:
records = payload.get("data", [])
if not isinstance(records, list) or not records:
raise RuntimeError("Price payload missing 'data' list")
df = pd.DataFrame(records)
if "timestamp" in df.columns:
df["timestamp"] = pd.to_datetime(df["timestamp"], utc=True, errors="coerce")
df = df.dropna(subset=["timestamp"]).set_index("timestamp")
elif "date" in df.columns:
df["date"] = pd.to_datetime(df["date"], utc=True, errors="coerce")
df = df.dropna(subset=["date"]).set_index("date")
else:
idx = pd.date_range(start=start, end=end, periods=len(df), tz="UTC")
df.index = idx
if "close" not in df.columns:
if "price" in df.columns:
df = df.rename(columns={"price": "close"})
else:
raise RuntimeError("Price payload missing close/price column")
return df[["close"]].sort_index()
def _to_series_frame(payload: dict[str, Any], key: str, start: datetime, end: datetime) -> pd.DataFrame:
records = payload.get("data", [])
if not isinstance(records, list) or not records:
raise RuntimeError("Series payload missing 'data' list")
df = pd.DataFrame(records)
if "timestamp" in df.columns:
df["timestamp"] = pd.to_datetime(df["timestamp"], utc=True, errors="coerce")
df = df.dropna(subset=["timestamp"]).set_index("timestamp")
elif "date" in df.columns:
df["date"] = pd.to_datetime(df["date"], utc=True, errors="coerce")
df = df.dropna(subset=["date"]).set_index("date")
else:
idx = pd.date_range(start=start, end=end, periods=len(df), tz="UTC")
df.index = idx
if key not in df.columns:
# Accept first numeric column as a tolerant API stub fallback.
numeric = df.select_dtypes(include="number")
if numeric.empty:
raise RuntimeError(f"Series payload missing '{key}'")
return numeric.iloc[:, :1].rename(columns={numeric.columns[0]: key})
return df[[key]].sort_index()
def _live_enabled(env_key: str) -> bool:
return os.getenv(env_key, "false").lower() in {"1", "true", "yes", "on"}
@dataclass
class BreezeAdapter(BaseDataAdapter):
use_live: bool = False
base_url: str | None = None
def __post_init__(self) -> None:
self.use_live = self.use_live or _live_enabled("BREEZE_LIVE_ENABLED")
self.base_url = self.base_url or os.getenv("BREEZE_BASE_URL")
self.client = HttpClient(_retry_config_from_env())
def _creds(self) -> APICredentials:
return load_credentials("BREEZE", required=["API_KEY", "API_SECRET"], optional=["SESSION_TOKEN"])
def fetch_price_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
if not self.base_url:
raise RuntimeError("Breeze live enabled but BREEZE_BASE_URL not configured")
creds = self._creds()
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/api/v1/price",
params={"symbol": asset, "start": start.isoformat(), "end": end.isoformat()},
headers={"X-API-KEY": creds.get("api_key", "")},
)
return _to_price_frame(payload, start, end)
return _mock_price_series(asset, start, end, seed=1)
def fetch_macro_data(self, key: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
if not self.base_url:
raise RuntimeError("Breeze live enabled but BREEZE_BASE_URL not configured")
creds = self._creds()
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/api/v1/macro",
params={"key": key, "start": start.isoformat(), "end": end.isoformat()},
headers={"X-API-KEY": creds.get("api_key", "")},
)
return _to_series_frame(payload, key, start, end)
return _mock_macro_series(key, start, end, seed=2)
def fetch_rate_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
if not self.base_url:
raise RuntimeError("Breeze live enabled but BREEZE_BASE_URL not configured")
creds = self._creds()
col = f"{asset}_rate"
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/api/v1/rates",
params={"symbol": asset, "start": start.isoformat(), "end": end.isoformat()},
headers={"X-API-KEY": creds.get("api_key", "")},
)
return _to_series_frame(payload, col, start, end)
return _mock_rate_series(asset, start, end, seed=3)
def health_check(self) -> bool:
if not self.use_live:
return True
return bool(self.base_url)
@dataclass
class ZerodhaAdapter(BaseDataAdapter):
use_live: bool = False
base_url: str | None = None
def __post_init__(self) -> None:
self.use_live = self.use_live or _live_enabled("ZERODHA_LIVE_ENABLED")
self.base_url = self.base_url or os.getenv("ZERODHA_BASE_URL")
self.client = HttpClient(_retry_config_from_env())
def _creds(self) -> APICredentials:
return load_credentials("ZERODHA", required=["API_KEY", "ACCESS_TOKEN"])
def fetch_price_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
if not self.base_url:
raise RuntimeError("Zerodha live enabled but ZERODHA_BASE_URL not configured")
creds = self._creds()
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/instruments/historical",
params={"symbol": asset, "start": start.isoformat(), "end": end.isoformat()},
headers={
"X-Kite-Version": "3",
"Authorization": f"token {creds.get('api_key', '')}:{creds.get('access_token', '')}",
},
)
return _to_price_frame(payload, start, end)
return _mock_price_series(asset, start, end, seed=4)
def fetch_macro_data(self, key: str, start: datetime, end: datetime) -> pd.DataFrame:
raise NotImplementedError("Zerodha macro data not supported")
def fetch_rate_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
raise NotImplementedError("Zerodha rates not supported")
def health_check(self) -> bool:
if not self.use_live:
return True
return bool(self.base_url)
class NSEAdapter(BaseDataAdapter):
def __init__(self, use_live: bool = False, base_url: str | None = None) -> None:
self.use_live = use_live or _live_enabled("NSE_LIVE_ENABLED")
self.base_url = base_url or os.getenv("NSE_BASE_URL", "https://www.nseindia.com")
self.client = HttpClient(_retry_config_from_env())
def fetch_price_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/api/historical/foCPV",
params={"symbol": asset, "from": start.strftime("%d-%m-%Y"), "to": end.strftime("%d-%m-%Y")},
headers={"User-Agent": "options-quant-engine/0.1"},
)
return _to_price_frame(payload, start, end)
return _mock_price_series(asset, start, end, seed=5)
def fetch_macro_data(self, key: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/api/allIndices",
headers={"User-Agent": "options-quant-engine/0.1"},
)
return _to_series_frame(payload, key, start, end)
return _mock_macro_series(key, start, end, seed=6)
def fetch_rate_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
col = f"{asset}_rate"
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/api/live-analysis-oi-spurts-underlyings",
headers={"User-Agent": "options-quant-engine/0.1"},
)
return _to_series_frame(payload, col, start, end)
return _mock_rate_series(asset, start, end, seed=7)
def health_check(self) -> bool:
return True
class RBIAdapter(BaseDataAdapter):
def __init__(self, use_live: bool = False, base_url: str | None = None) -> None:
self.use_live = use_live or _live_enabled("RBI_LIVE_ENABLED")
self.base_url = base_url or os.getenv("RBI_BASE_URL", "https://data.rbi.org.in")
self.client = HttpClient(_retry_config_from_env())
def fetch_price_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/api/exchange-rate",
params={"pair": asset, "from": start.isoformat(), "to": end.isoformat()},
)
return _to_price_frame(payload, start, end)
return _mock_price_series(asset, start, end, seed=8)
def fetch_macro_data(self, key: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/api/macro",
params={"series": key, "from": start.isoformat(), "to": end.isoformat()},
)
return _to_series_frame(payload, key, start, end)
return _mock_macro_series(key, start, end, seed=9)
def fetch_rate_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
if self.use_live:
col = f"{asset}_rate"
payload = self.client.get_json(
f"{self.base_url.rstrip('/')}/api/policy-rates",
params={"asset": asset, "from": start.isoformat(), "to": end.isoformat()},
)
return _to_series_frame(payload, col, start, end)
return _mock_rate_series(asset, start, end, seed=10)
def health_check(self) -> bool:
return True
class FreeFXAdapter(BaseDataAdapter):
def fetch_price_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
return _mock_price_series(asset, start, end, seed=11)
def fetch_macro_data(self, key: str, start: datetime, end: datetime) -> pd.DataFrame:
return _mock_macro_series(key, start, end, seed=12)
def fetch_rate_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
return _mock_rate_series(asset, start, end, seed=13)
def health_check(self) -> bool:
return True
class MockAdapter(BaseDataAdapter):
def fetch_price_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
return _mock_price_series(asset, start, end, seed=42)
def fetch_macro_data(self, key: str, start: datetime, end: datetime) -> pd.DataFrame:
return _mock_macro_series(key, start, end, seed=43)
def fetch_rate_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
return _mock_rate_series(asset, start, end, seed=44)
def health_check(self) -> bool:
return True
@@ -0,0 +1,24 @@
from __future__ import annotations
from abc import ABC, abstractmethod
from datetime import datetime
import pandas as pd
class BaseDataAdapter(ABC):
@abstractmethod
def fetch_price_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
pass
@abstractmethod
def fetch_macro_data(self, key: str, start: datetime, end: datetime) -> pd.DataFrame:
pass
@abstractmethod
def fetch_rate_data(self, asset: str, start: datetime, end: datetime) -> pd.DataFrame:
pass
@abstractmethod
def health_check(self) -> bool:
pass
@@ -0,0 +1,36 @@
from __future__ import annotations
import os
from dataclasses import dataclass
@dataclass(frozen=True)
class APICredentials:
values: dict[str, str]
def get(self, key: str, default: str | None = None) -> str | None:
return self.values.get(key, default)
def load_credentials(prefix: str, required: list[str], optional: list[str] | None = None) -> APICredentials:
optional = optional or []
payload: dict[str, str] = {}
missing: list[str] = []
for key in required:
env_key = f"{prefix}_{key}"
value = os.getenv(env_key)
if not value:
missing.append(env_key)
continue
payload[key.lower()] = value
for key in optional:
env_key = f"{prefix}_{key}"
value = os.getenv(env_key)
if value:
payload[key.lower()] = value
if missing:
raise RuntimeError(f"Missing required credentials: {', '.join(missing)}")
return APICredentials(values=payload)
@@ -0,0 +1,38 @@
from __future__ import annotations
import time
from dataclasses import dataclass
from typing import Any
import requests
@dataclass
class RetryConfig:
timeout_sec: float = 5.0
max_attempts: int = 3
backoff_sec: float = 0.5
class HttpClient:
def __init__(self, config: RetryConfig) -> None:
self.config = config
def get_json(self, url: str, params: dict[str, Any] | None = None, headers: dict[str, str] | None = None) -> dict[str, Any]:
last_err = "unknown"
for attempt in range(1, self.config.max_attempts + 1):
try:
response = requests.get(url, params=params, headers=headers, timeout=self.config.timeout_sec)
response.raise_for_status()
data = response.json()
if not isinstance(data, dict):
raise RuntimeError("Expected JSON object response")
return data
except Exception as exc: # noqa: BLE001
last_err = str(exc)
if attempt < self.config.max_attempts:
time.sleep(self.config.backoff_sec * attempt)
raise RuntimeError(
f"HTTP GET failed after {self.config.max_attempts} attempts for {url}: {last_err}"
)
@@ -0,0 +1,108 @@
from __future__ import annotations
from datetime import datetime
from typing import Any
import pandas as pd
from options_quant_engine.ingestion.base import BaseDataAdapter
from options_quant_engine.schemas import DataFetchResult, SourceUsageRecord
from options_quant_engine.utils.logging import get_logger
class DataSourceRouter:
def __init__(
self,
adapters: dict[str, BaseDataAdapter],
config: dict[str, Any],
) -> None:
self.adapters = adapters
self.config = config
self.logger = get_logger(self.__class__.__name__)
self.usage_log: list[SourceUsageRecord] = []
def _get_candidate_sources(self, asset: str) -> list[str]:
source_cfg = self.config
mapped = source_cfg.get("asset_source_map", {}).get(asset)
if mapped:
return mapped
g10 = asset in {"EURUSD", "GBPUSD", "USDJPY", "AUDUSD", "USDCAD", "USDCHF"}
key = "g10" if g10 else "default"
return source_cfg.get("source_priority", {}).get(key, ["mock"])
def _is_enabled(self, source: str) -> bool:
return bool(self.config.get("enabled_sources", {}).get(source, False))
def _tag(self, source: str) -> tuple[str, int]:
reliability = self.config.get("reliability_tags", {}).get(source, "low")
latency_ms = int(self.config.get("latency_tags_ms", {}).get(source, 1000))
return reliability, latency_ms
def fetch_price_data(self, asset: str, start: datetime, end: datetime) -> DataFetchResult:
return self._fetch_with_fallback("price", asset, start, end)
def fetch_macro_data(self, key: str, start: datetime, end: datetime) -> DataFetchResult:
return self._fetch_with_fallback("macro", key, start, end)
def fetch_rate_data(self, asset: str, start: datetime, end: datetime) -> DataFetchResult:
return self._fetch_with_fallback("rate", asset, start, end)
def _fetch_with_fallback(
self,
usage_type: str,
key: str,
start: datetime,
end: datetime,
) -> DataFetchResult:
candidates = self._get_candidate_sources(key)
last_err = "No source attempted"
for source in candidates:
if not self._is_enabled(source):
continue
adapter = self.adapters.get(source)
if adapter is None:
continue
try:
if usage_type == "price":
data = adapter.fetch_price_data(key, start, end)
elif usage_type == "macro":
data = adapter.fetch_macro_data(key, start, end)
else:
data = adapter.fetch_rate_data(key, start, end)
reliability, latency_ms = self._tag(source)
self.usage_log.append(SourceUsageRecord(asset=key, source=source, usage_type=usage_type))
self.logger.info("Fetched %s data for %s via %s", usage_type, key, source)
return DataFetchResult(
data=data,
source=source,
reliability=reliability,
latency_ms=latency_ms,
)
except Exception as exc: # noqa: BLE001
last_err = str(exc)
self.usage_log.append(
SourceUsageRecord(
asset=key,
source=source,
usage_type=usage_type,
success=False,
error=last_err,
)
)
self.logger.warning(
"Failed %s data fetch for %s via %s: %s",
usage_type,
key,
source,
last_err,
)
return DataFetchResult(
data=pd.DataFrame(),
source="none",
reliability="low",
latency_ms=9999,
success=False,
error=last_err,
)
@@ -0,0 +1,3 @@
from options_quant_engine.integration.hooks import IntegrationHooks
__all__ = ["IntegrationHooks"]
@@ -0,0 +1,26 @@
from __future__ import annotations
import pandas as pd
from options_quant_engine.schemas import RegimeState
class IntegrationHooks:
def export_scores(self, features: pd.DataFrame, regime: RegimeState) -> dict[str, float]:
if features.empty:
return {
"usd_strength_score": 0.0,
"inr_stress_score": 0.0,
"carry_attractiveness": 0.0,
"fx_volatility_stress": 0.0,
"risk_on_off_score": 0.0,
}
row = features.iloc[-1]
return {
"usd_strength_score": float(row.get("usd_strength_score", 0.0)),
"inr_stress_score": float(row.get("em_stress_proxy", 0.0) + row.get("inr_vol_clustering", 0.0)),
"carry_attractiveness": float(row.get("carry_proxy", 0.0)),
"fx_volatility_stress": float(row.get("realized_vol", 0.0)),
"risk_on_off_score": float(row.get("risk_on_off_proxy", 0.0)),
"regime_stability": float(regime.stability),
}
@@ -0,0 +1,4 @@
from options_quant_engine.models.ensemble import EnsembleModel
from options_quant_engine.models.relative_value import RelativeValueModel
__all__ = ["EnsembleModel", "RelativeValueModel"]
@@ -0,0 +1,44 @@
from __future__ import annotations
from typing import Any
import numpy as np
import pandas as pd
class EnsembleModel:
def __init__(self, config: dict[str, Any]) -> None:
self.config = config
def directional_score(self, row: pd.Series) -> tuple[float, dict[str, float], float]:
weights = self.config.get("directional", {}).get("model_weights", {})
component = {
"trend": float(row.get("trend_strength", 0.0)),
"momentum": float(row.get("momentum_multi_horizon", 0.0)),
"mean_reversion": -float(row.get("mean_reversion_zscore", 0.0)),
"carry": float(row.get("carry_proxy", 0.0)),
"macro": float(row.get("risk_on_off_proxy", 0.0)),
}
wsum = sum(float(weights.get(k, 0.0)) for k in component)
score = 0.0
if wsum > 0:
score = sum(component[k] * float(weights.get(k, 0.0)) for k in component) / wsum
normalized = float(np.tanh(score))
signs = [np.sign(v) for v in component.values() if abs(v) > 1e-12]
agreement = float((abs(sum(signs)) / len(signs))) if signs else 0.0
return normalized, component, agreement
def relative_value_score(self, row_a: pd.Series, row_b: pd.Series) -> tuple[float, dict[str, float], float]:
weights = self.config.get("relative_value", {}).get("model_weights", {})
spread = float(row_a.get("mean_reversion_zscore", 0.0) - row_b.get("mean_reversion_zscore", 0.0))
mom_diff = float(row_a.get("momentum_multi_horizon", 0.0) - row_b.get("momentum_multi_horizon", 0.0))
carry_diff = float(row_a.get("carry_proxy", 0.0) - row_b.get("carry_proxy", 0.0))
component = {"spread": spread, "momentum_diff": mom_diff, "carry_diff": carry_diff}
wsum = sum(float(weights.get(k, 0.0)) for k in component)
score = 0.0
if wsum > 0:
score = sum(component[k] * float(weights.get(k, 0.0)) for k in component) / wsum
normalized = float(np.tanh(score))
signs = [np.sign(v) for v in component.values() if abs(v) > 1e-12]
agreement = float((abs(sum(signs)) / len(signs))) if signs else 0.0
return normalized, component, agreement
@@ -0,0 +1,15 @@
"""Placeholder for future model extensions.
This module is intentionally lightweight in the starter phase. Future additions can
include gradient boosting, probabilistic forecasts, and online learning modules.
"""
from __future__ import annotations
from dataclasses import dataclass
@dataclass
class ModelExtensionConfig:
enabled: bool = False
name: str = "baseline_extension"
@@ -0,0 +1,91 @@
from __future__ import annotations
from dataclasses import dataclass
import numpy as np
import pandas as pd
@dataclass
class RelativeValueResult:
score: float
hedge_ratio: float
spread_zscore: float
expected_volatility: float
drivers: dict[str, float]
long_asset: str
short_asset: str
class RelativeValueModel:
def __init__(self, lookback: int = 60) -> None:
self.lookback = lookback
def generate(
self,
asset_a: str,
asset_b: str,
prices_a: pd.Series,
prices_b: pd.Series,
features_a: pd.DataFrame,
features_b: pd.DataFrame,
) -> RelativeValueResult:
pa, pb = self._align(prices_a, prices_b)
la = np.log(pa)
lb = np.log(pb)
hedge_ratio = self._hedge_ratio(la.tail(self.lookback), lb.tail(self.lookback))
spread = la - hedge_ratio * lb
window = max(20, self.lookback // 2)
zscore = self._zscore(spread, window=window)
spread_z = float(zscore.iloc[-1]) if len(zscore) else 0.0
row_a = features_a.iloc[-1]
row_b = features_b.iloc[-1]
momentum_diff = float(row_a.get("momentum_multi_horizon", 0.0) - row_b.get("momentum_multi_horizon", 0.0))
carry_diff = float(row_a.get("carry_proxy", 0.0) - row_b.get("carry_proxy", 0.0))
# Mean-reversion spread score with feature differentials.
raw = -0.60 * spread_z + 0.25 * momentum_diff + 0.15 * carry_diff
score = float(np.tanh(raw))
if score >= 0:
long_asset, short_asset = asset_a, asset_b
else:
long_asset, short_asset = asset_b, asset_a
expected_vol = float(spread.diff().tail(window).std() * np.sqrt(252.0)) if len(spread) > 2 else 0.0
drivers = {
"spread_zscore": spread_z,
"momentum_diff": momentum_diff,
"carry_diff": carry_diff,
"hedge_ratio": float(hedge_ratio),
}
return RelativeValueResult(
score=abs(score),
hedge_ratio=float(hedge_ratio),
spread_zscore=spread_z,
expected_volatility=max(expected_vol, 0.0),
drivers=drivers,
long_asset=long_asset,
short_asset=short_asset,
)
@staticmethod
def _align(a: pd.Series, b: pd.Series) -> tuple[pd.Series, pd.Series]:
joined = pd.concat([a.rename("a"), b.rename("b")], axis=1).dropna()
return joined["a"], joined["b"]
@staticmethod
def _hedge_ratio(a: pd.Series, b: pd.Series) -> float:
denom = float(np.var(b))
if abs(denom) < 1e-12:
return 1.0
return float(np.cov(a, b)[0, 1] / denom)
@staticmethod
def _zscore(series: pd.Series, window: int) -> pd.Series:
mu = series.rolling(window).mean()
sigma = series.rolling(window).std()
return ((series - mu) / (sigma + 1e-9)).fillna(0.0)
@@ -0,0 +1,3 @@
from options_quant_engine.outputs.formatter import OutputFormatter
__all__ = ["OutputFormatter"]
@@ -0,0 +1,37 @@
from __future__ import annotations
from typing import Any
from options_quant_engine.schemas import SignalPayload
class OutputFormatter:
def __init__(self, config: dict[str, Any]) -> None:
self.config = config.get("output", {})
def to_json_payload(self, signal: SignalPayload) -> dict[str, Any]:
p = self.config.get("precision", 4)
data = signal.to_dict()
data["signal_strength"] = round(float(data["signal_strength"]), p)
data["confidence"] = round(float(data["confidence"]), p)
data["expected_volatility"] = round(float(data["expected_volatility"]), p)
data["position_sizing_multiplier"] = round(float(data["position_sizing_multiplier"]), p)
return data
def trader_summary(self, signal: SignalPayload) -> str:
return (
f"{signal.asset} | {signal.signal_type} | {signal.signal_direction} | "
f"strength={signal.signal_strength:.2f} confidence={signal.confidence:.2f} "
f"action={signal.recommended_action} size={signal.position_sizing_multiplier:.2f} "
f"regime={signal.regime.get('combined', 'neutral')}"
)
def dashboard_payload(self, signal: SignalPayload) -> dict[str, Any]:
return {
"asset": signal.asset,
"score": signal.signal_strength,
"confidence": signal.confidence,
"action": signal.recommended_action,
"risk_flags": signal.risk_flags,
"regime": signal.regime,
}
@@ -0,0 +1,3 @@
from options_quant_engine.preprocessing.cleaning import preprocess_market_data
__all__ = ["preprocess_market_data"]
@@ -0,0 +1,12 @@
from __future__ import annotations
import pandas as pd
def preprocess_market_data(df: pd.DataFrame) -> pd.DataFrame:
if df.empty:
return df
out = df.copy().sort_index()
out = out[~out.index.duplicated(keep="last")]
out = out.ffill().dropna(how="all")
return out
@@ -0,0 +1,3 @@
from options_quant_engine.regime.engine import RegimeEngine
__all__ = ["RegimeEngine"]
@@ -0,0 +1,87 @@
from __future__ import annotations
from typing import Any
import numpy as np
import pandas as pd
from options_quant_engine.schemas import RegimeState
class RegimeEngine:
def __init__(self, config: dict[str, Any]) -> None:
self.config = config
def detect(self, features: pd.DataFrame) -> RegimeState:
if features.empty:
return RegimeState("neutral", "normal", "neutral", "risk_on", "neutral", 0.0)
row = features.iloc[-1]
trend = self._trend_regime(row)
vol = self._vol_regime(features)
dollar = self._dollar_regime(row)
risk = self._risk_regime(row, vol)
combined = self._combine(trend, vol, dollar, risk)
stability = self._stability(features)
return RegimeState(trend, vol, dollar, risk, combined, stability)
def _trend_regime(self, row: pd.Series) -> str:
score = float(row.get("trend_strength", 0.0))
th = float(self.config.get("trend", {}).get("slope_threshold", 0.0005))
band = float(self.config.get("trend", {}).get("mr_band", 0.003))
if score > th:
return "trend"
if score < -band:
return "mean_reversion"
return "neutral"
def _vol_regime(self, features: pd.DataFrame) -> str:
vol = features.get("realized_vol", pd.Series(0.0, index=features.index)).fillna(0.0)
cur = float(vol.iloc[-1])
ql = float(vol.quantile(self.config.get("volatility", {}).get("low_quantile", 0.3)))
qh = float(vol.quantile(self.config.get("volatility", {}).get("high_quantile", 0.7)))
qs = float(vol.quantile(self.config.get("volatility", {}).get("stress_quantile", 0.9)))
if cur >= qs:
return "stress"
if cur >= qh:
return "high"
if cur <= ql:
return "low"
return "normal"
def _dollar_regime(self, row: pd.Series) -> str:
usd = float(row.get("usd_strength_score", 0.0))
band = float(self.config.get("dollar", {}).get("neutral_band", 0.2))
if usd > band:
return "strengthening"
if usd < -band:
return "weakening"
return "neutral"
def _risk_regime(self, row: pd.Series, vol_regime: str) -> str:
risk_score = float(row.get("risk_on_off_proxy", 0.0))
threshold = float(self.config.get("risk", {}).get("risk_off_equity_threshold", -0.01))
if vol_regime == "stress":
return "stressed"
if risk_score < threshold:
return "risk_off"
return "risk_on"
def _combine(self, trend: str, vol: str, dollar: str, risk: str) -> str:
if vol == "stress" or risk == "stressed":
return "defensive"
if trend == "trend" and risk == "risk_on":
return "pro_trend"
if trend == "mean_reversion" and risk != "stressed":
return "mean_revert"
if dollar == "strengthening":
return "usd_dominant"
return "neutral"
def _stability(self, features: pd.DataFrame) -> float:
cols = [c for c in ["trend_strength", "realized_vol", "usd_strength_score", "risk_on_off_proxy"] if c in features]
if not cols:
return 0.0
tail = features[cols].tail(10)
vol = float(tail.std().mean())
return float(np.clip(1.0 - vol, 0.0, 1.0))
@@ -0,0 +1,3 @@
from options_quant_engine.risk.engine import RiskEngine
__all__ = ["RiskEngine"]
@@ -0,0 +1,55 @@
from __future__ import annotations
from datetime import datetime, timezone
from typing import Any
from options_quant_engine.schemas import RegimeState, SignalPayload
from options_quant_engine.utils.time import is_within_market_hours
class RiskEngine:
def __init__(self, config: dict[str, Any]) -> None:
self.config = config.get("risk", {})
def apply(self, signal: SignalPayload, regime: RegimeState, reliability: str, liquidity_ok: bool, event_risk: float) -> SignalPayload:
penalties = self.config.get("data_reliability_penalty", {})
rel_mult = float(penalties.get(reliability, 0.6))
size = signal.signal_strength * signal.confidence * rel_mult
if regime.volatility in {"high", "stress"}:
size *= float(self.config.get("stressed_regime_multiplier", 0.5))
signal.risk_flags.append("high_volatility_regime")
if not liquidity_ok:
size *= float(self.config.get("illiquid_market_multiplier", 0.6))
signal.risk_flags.append("liquidity_constraint")
if event_risk > 0.6:
size *= 0.5
signal.risk_flags.append("event_risk_high")
if not self._market_open_now():
signal.risk_flags.append("outside_primary_market_hours")
size *= 0.8
signal.position_sizing_multiplier = max(0.0, min(size, 1.0))
signal.recommended_action = self._action(signal.position_sizing_multiplier)
return signal
def _market_open_now(self) -> bool:
now = datetime.now(timezone.utc)
open_utc = self.config.get("market_hours", {}).get("inr_derivatives_open_utc", "03:45")
close_utc = self.config.get("market_hours", {}).get("inr_derivatives_close_utc", "10:00")
return is_within_market_hours(now, open_utc, close_utc)
@staticmethod
def _action(size: float) -> str:
if size >= 0.75:
return "enter"
if size >= 0.45:
return "enter_small"
if size >= 0.25:
return "hold"
if size > 0.0:
return "reduce"
return "no_trade"
@@ -0,0 +1,78 @@
from __future__ import annotations
from dataclasses import asdict, dataclass, field
from datetime import datetime, timezone
from typing import Any
import pandas as pd
@dataclass
class DataFetchResult:
data: pd.DataFrame
source: str
reliability: str
latency_ms: int
success: bool = True
error: str | None = None
@dataclass
class RegimeState:
trend: str
volatility: str
dollar: str
risk: str
combined: str
stability: float
@dataclass
class SignalPayload:
engine: str
timestamp: str
asset: str
signal_type: str
signal_direction: str
signal_strength: float
confidence: float
regime: dict[str, Any]
expected_volatility: float
risk_flags: list[str]
drivers: dict[str, Any]
recommended_action: str
position_sizing_multiplier: float
def to_dict(self) -> dict[str, Any]:
return asdict(self)
@dataclass
class EvaluationReport:
generated_at: str
hit_rate: float
avg_forward_return: float
regime_performance: dict[str, float]
signal_decay: dict[str, float]
feature_importance: dict[str, float]
drift_flags: list[str]
calibration_score: float
@dataclass
class BacktestResult:
pair_level_returns: dict[str, float]
portfolio_return: float
max_drawdown: float
turnover: float
metrics: dict[str, float]
@dataclass
class SourceUsageRecord:
asset: str
source: str
usage_type: str
timestamp: str = field(default_factory=lambda: datetime.now(timezone.utc).isoformat())
success: bool = True
error: str | None = None
@@ -0,0 +1,4 @@
from options_quant_engine.signals.confidence import ConfidenceEngine
from options_quant_engine.signals.engine import SignalEngine
__all__ = ["SignalEngine", "ConfidenceEngine"]
@@ -0,0 +1,39 @@
from __future__ import annotations
from typing import Any
import numpy as np
class ConfidenceEngine:
def __init__(self, config: dict[str, Any]) -> None:
self.config = config
def compute(
self,
feature_agreement: float,
model_agreement: float,
regime_stability: float,
data_quality: float,
volatility: float,
event_risk: float,
) -> float:
c = self.config.get("confidence", {})
agreement_w = float(c.get("agreement_weight", 0.25))
regime_w = float(c.get("regime_stability_weight", 0.20))
data_w = float(c.get("data_quality_weight", 0.20))
vol_w = float(c.get("volatility_penalty_weight", 0.20))
event_w = float(c.get("event_penalty_weight", 0.15))
agreement = 0.5 * (feature_agreement + model_agreement)
vol_penalty = np.clip(volatility, 0.0, 1.0)
event_penalty = np.clip(event_risk, 0.0, 1.0)
raw = (
agreement_w * agreement
+ regime_w * np.clip(regime_stability, 0.0, 1.0)
+ data_w * np.clip(data_quality, 0.0, 1.0)
- vol_w * vol_penalty
- event_w * event_penalty
)
return float(np.clip((raw + 1.0) / 2.0, 0.0, 1.0))
@@ -0,0 +1,74 @@
from __future__ import annotations
from datetime import datetime, timezone
from typing import Any
import numpy as np
import pandas as pd
from options_quant_engine.schemas import RegimeState, SignalPayload
class SignalEngine:
def __init__(self, engine_name: str = "options_quant_engine") -> None:
self.engine_name = engine_name
def directional_signal(
self,
asset: str,
feature_row: pd.Series,
regime: RegimeState,
score: float,
confidence: float,
drivers: dict[str, float],
expected_volatility: float,
) -> SignalPayload:
direction = self._direction(asset, score)
return SignalPayload(
engine=self.engine_name,
timestamp=datetime.now(timezone.utc).isoformat(),
asset=asset,
signal_type="directional",
signal_direction=direction,
signal_strength=float(np.clip(abs(score), 0.0, 1.0)),
confidence=float(np.clip(confidence, 0.0, 1.0)),
regime=regime.__dict__,
expected_volatility=float(max(0.0, expected_volatility)),
risk_flags=[],
drivers={k: float(v) for k, v in drivers.items()},
recommended_action="hold",
position_sizing_multiplier=0.0,
)
def relative_value_signal(
self,
long_asset: str,
short_asset: str,
regime: RegimeState,
score: float,
confidence: float,
drivers: dict[str, float],
expected_volatility: float,
) -> SignalPayload:
direction = f"long_{long_asset}_vs_{short_asset}"
asset = f"{long_asset}:{short_asset}"
return SignalPayload(
engine=self.engine_name,
timestamp=datetime.now(timezone.utc).isoformat(),
asset=asset,
signal_type="relative_value",
signal_direction=direction,
signal_strength=float(np.clip(abs(score), 0.0, 1.0)),
confidence=float(np.clip(confidence, 0.0, 1.0)),
regime=regime.__dict__,
expected_volatility=float(max(0.0, expected_volatility)),
risk_flags=[],
drivers={k: float(v) for k, v in drivers.items()},
recommended_action="hold",
position_sizing_multiplier=0.0,
)
def _direction(self, asset: str, score: float) -> str:
if asset.startswith("USD"):
return "long_usd" if score >= 0 else "short_usd"
return "long_base" if score >= 0 else "short_base"
@@ -0,0 +1,4 @@
from options_quant_engine.utils.config import load_all_configs, load_yaml
from options_quant_engine.utils.logging import get_logger
__all__ = ["load_yaml", "load_all_configs", "get_logger"]
@@ -0,0 +1,27 @@
from __future__ import annotations
from pathlib import Path
from typing import Any
import yaml
def load_yaml(path: str | Path) -> dict[str, Any]:
with Path(path).open("r", encoding="utf-8") as f:
data = yaml.safe_load(f) or {}
if not isinstance(data, dict):
raise ValueError(f"Config file must contain a mapping: {path}")
return data
def load_all_configs(config_dir: str | Path) -> dict[str, dict[str, Any]]:
cfg_dir = Path(config_dir)
return {
"universe": load_yaml(cfg_dir / "universe.yaml"),
"data_sources": load_yaml(cfg_dir / "data_sources.yaml"),
"features": load_yaml(cfg_dir / "features.yaml"),
"regimes": load_yaml(cfg_dir / "regimes.yaml"),
"models": load_yaml(cfg_dir / "models.yaml"),
"risk": load_yaml(cfg_dir / "risk.yaml"),
"output": load_yaml(cfg_dir / "output.yaml"),
}
@@ -0,0 +1,14 @@
from __future__ import annotations
import logging
def get_logger(name: str) -> logging.Logger:
logger = logging.getLogger(name)
if not logger.handlers:
handler = logging.StreamHandler()
fmt = "%(asctime)s | %(name)s | %(levelname)s | %(message)s"
handler.setFormatter(logging.Formatter(fmt))
logger.addHandler(handler)
logger.setLevel(logging.INFO)
return logger
@@ -0,0 +1,16 @@
from __future__ import annotations
from datetime import datetime, time, timezone
def is_within_market_hours(
now_utc: datetime,
open_hhmm: str,
close_hhmm: str,
) -> bool:
open_h, open_m = [int(x) for x in open_hhmm.split(":")]
close_h, close_m = [int(x) for x in close_hhmm.split(":")]
open_t = time(hour=open_h, minute=open_m)
close_t = time(hour=close_h, minute=close_m)
t = now_utc.astimezone(timezone.utc).time()
return open_t <= t <= close_t
+41
View File
@@ -0,0 +1,41 @@
[build-system]
requires = ["setuptools>=68", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "options-quant-engine"
version = "0.1.0"
description = "Modular, explainable, production-grade options quant engine"
readme = "README.md"
requires-python = ">=3.10"
authors = [{ name = "Options Quant Team" }]
dependencies = [
"numpy>=1.26",
"pandas>=2.2",
"PyYAML>=6.0",
"scipy>=1.12",
"requests>=2.32"
]
[project.optional-dependencies]
dev = [
"pytest>=8.0",
"pytest-cov>=5.0",
"ruff>=0.5"
]
[tool.setuptools.packages.find]
where = ["."]
include = ["options_quant_engine*"]
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-q"
[tool.ruff]
line-length = 100
target-version = "py310"
[tool.ruff.lint]
select = ["E", "F", "I", "B", "UP"]
ignore = ["E501"]

Some files were not shown because too many files have changed in this diff Show More