fx-quant

Algorithmic FX trading system with backtesting, paper/live execution via OANDA, and a Flask web dashboard.

Quick Start (New Machine Setup)

1. Clone & install dependencies

git clone <your-repo-url>
cd fx-quant
python -m pip install -r requirements.txt

2. Configure environment variables

Copy the template and fill in your credentials:

cp config/.env.example config/.env

Required variables in config/.env:

Variable	Description
SUPABASE_URL	Your Supabase project URL
SUPABASE_KEY	Supabase anon/service key
OANDA_API_KEY	OANDA v20 API token
OANDA_ACCOUNT_ID	OANDA account ID
OANDA_ENV	practice or live (default: practice)
DASHBOARD_PASSWORD	Password for web dashboard login
TRADING_ECONOMICS_API_KEY	Trading Economics API key (for economic calendar)

3. Verify connectivity

python src/test_connection.py

4. Load historical data (if Supabase table is empty)

python src/historical_loader.py

5. Run the backtester

python src/backtester.py

Output goes to logs/:

• backtest_trades_<INSTRUMENT>_<GRANULARITY>.csv -- trade log
• backtest_summary_<INSTRUMENT>_<GRANULARITY>.json -- metrics + monthly P&L

6. Visualize trades on a chart

python src/chart_trades.py --start 2025-02-24 --end 2025-02-27
python src/chart_trades.py --granularity M15 --start 2025-04-01 --end 2025-04-15

Saves PNG charts to logs/chart_trades_*.png.

---

Project Structure

fx-quant/
  config/
    system.yaml          # Main configuration (strategy, features, execution)
    .env                 # Secrets (not committed)
  src/
    config_loader.py     # Loads system.yaml + .env
    data_engine.py       # Feature engineering (SMA, EMA, RSI, ATR, VWAP, pivots, engulfing)
    backtester.py        # Backtesting engine (SMA cross + pivot retest strategies)
    order_executor.py    # Paper/live order execution via OANDA
    ai_wrapper.py        # ML ensemble (logistic reg, RF, gradient boosting) signal validation
    dashboard.py         # Flask web dashboard
    chart_trades.py      # Matplotlib/mplfinance trade visualization
    get_candles.py       # Fetch candles from OANDA API
    economic_calendar.py # Economic calendar fetch/upload pipeline
    historical_loader.py # Bulk historical data loader
    supabase_upload.py   # Upload candle data to Supabase
    param_sweep.py       # Strategy parameter optimization
    test_connection.py   # OANDA + Supabase connectivity check
  templates/             # Flask HTML templates (chart, backtest, config, logs)
  logs/                  # Output: trade CSVs, JSON summaries, chart PNGs
  models/                # Saved ML models
  sql/                   # Database schemas
  Dockerfile             # Container (live engine default CMD)
  docker-compose.yml     # live-engine + dashboard services
  railway.toml           # Railway build/deploy config
  requirements.txt       # Python dependencies (portable)
  requirements.docker.txt# Docker-specific deps

Phase 1: Strategy Backtesting Status

Backtested 6 strategy families across 10 currency pairs on 2021-2024 data. Full results in results/phase1/, detailed learnings in results/STRATEGY_LEARNINGS.md.

Working Strategies

S3 — Key Level Momentum Breakout (H1 timeframe) — Best performer

Pair	Trades	Win Rate	PF	PnL (pips)	Max DD
GBP_JPY	155	52.3%	0.99	+408	-13.5%
GBP_USD	179	53.1%	1.02	+97	-10.2%
USD_JPY	138	52.9%	1.00	+66	-9.8%

Breakout of horizontal S/R levels (3+ touch clusters) with volume confirmation, strong candle close, MACD alignment, and ADX > 20. Simple, 4-filter approach on H1. Needs SL/TP tuning to push PF above 1.0 consistently.

Strategies With Potential (Need Tweaks)

S4-F — EMA Ribbon Trend Context (EUR_AUD only): 95 trades, 45.3% WR, PF 1.06, +173 pips. Only profitable on EUR_AUD. Needs pair-specific tuning and SL/TP restructuring.

S6 — EMA Bounce (EUR_AUD/GBP_USD): 59-60% win rate but PF 0.83-0.84. Win rate is strong — needs tighter SL or trailing stop to fix risk/reward.

Strategies Retired

Strategy	Issue	Status
S1 — MA Breakout	35-47% WR, PF 0.40-0.88	No edge
S2 — VWAP Reversal	20-25% WR, 26 consecutive losses	Disabled
S4 — EMA Ribbon (6 variants)	Extensively tested D/E/F/F-v2/G/G-Minimal. Only EUR_AUD S4-F marginal.	Exhausted
S5 — Momentum Exhaustion	High trade count but PF 0.43-0.77	Too much noise

Key Learnings

1. Momentum + mean-reversion filters are contradictory — don't combine in one strategy
2. 3-4 hard filters max — more gates compound multiplicatively and kill trade count
3. Always validate thresholds against data distributions before running backtests
4. Simple strategies beat complex ones — S3 (4 filters) outperforms S4 (7+ filters)
5. H1 timeframe has natural edge — lower timeframes (M5/M15) struggle with noise

Full filter analysis and design principles in results/STRATEGY_LEARNINGS.md.

Next Phase

Moving to Smart Money / institutional flow strategies. Will also revisit S3 (SL/TP tuning, expanded pairs) and S6 (risk/reward restructuring).

---

Phase 2: Live Paper Trading + Extended Analytics

Phase 2 advances 5 strategies to live paper trading and extended backtesting analytics.

Resuming on a New Machine

git clone https://github.com/BrentNeale1/fx-quant.git
cd fx-quant
pip install pandas numpy requests pyyaml

Set your OANDA credentials in config/.env:

OANDA_API_KEY=your-key-here
OANDA_ACCOUNT_ID=your-account-id    # <-- REQUIRED, currently empty
OANDA_ENV=practice

The API key is already populated. You need to add your OANDA_ACCOUNT_ID (find it in the OANDA fxTrade Practice platform under Account Settings).

Running the Live Engine

# Single cycle (test connectivity + signal checks)
python src/live/run.py --once

# Continuous mode (polls every 60s, runs overnight)
python src/live/run.py

The engine runs 5 strategy slots in paper mode:

Slot	Strategy	Pair	Timeframe	Signal Type
1	S7 Tight	GBP_JPY	H1	Liquidity sweep reversal
2	S9	GBP_USD	H1	London session breakout
3	S9 Filtered	GBP_AUD	H1	London session + per-pair filters
4	S4-F	EUR_AUD	M15 + H1	EMA ribbon trend context
5	S3	GBP_JPY	H1	Key level momentum breakout

Signals fire during London/NY sessions (07:00-17:00 UTC). Running outside those hours will show "No signal" which is expected.

Monitoring

• Live state: logs/live_state.json (equity, open positions per slot)
• Trade log: logs/live_trades.csv (closed trades with PnL)
• Paper orders: logs/paper_trades.csv (all order attempts)
• Kill switch: Create STOP_ALL_TRADING file in project root to halt

Running Analytics

# Per-year performance breakdown (2021/2022/2023)
python src/run_regime_analysis.py

# Signal overlap + portfolio metrics
python src/run_correlation_analysis.py

# Kelly criterion + Monte Carlo drawdown simulation
python src/run_kelly_sizing.py

Results output to results/phase2/.

Phase 2 Analytics Summary

Regime Analysis — 4 of 5 strategies show improving PF over time:

Strategy	2021 PF	2022 PF	2023 PF	Trend
S7 Tight / GBP_JPY	0.47	0.89	1.80	UP
S9 / GBP_USD	0.71	0.83	1.38	UP
S9 Filtered / GBP_AUD	0.58	2.57	2.98	UP
S4F / EUR_AUD	1.11	1.79	0.48	DOWN
S3 / GBP_JPY	0.90	1.21	1.07	UP

Correlation — S7+S3 on GBP_JPY: 16.9% overlap (moderate, all same-direction). S9 vs S9_Filtered: 12% temporal overlap (good diversification).

Kelly Sizing — S9_Filtered: half-Kelly 7.3% (p95 DD 4.8%). S4F: 2.4%. S3: 1.6%. S7/S9 base: Kelly<=0 on full dataset.

Railway Cloud Deployment (24/7)

The live engine and dashboard are deployed to Railway for 24/7 operation without a local machine.

Service	Description	URL
live-engine	Runs src/live/run.py, polls every 60s	Internal (no public URL)
dashboard	Flask web dashboard	https://dashboard-production-73e7.up.railway.app

Dashboard login: admin / (password in Railway env vars)

Infrastructure:

• Both services auto-deploy on push to main
• Health checks on /health for both services
• Persistent volume mounted at /app/logs on live-engine (survives redeployments)
• Restart policy: on-failure with max 5 retries

Railway environment variables:

Service	Variables
live-engine	OANDA_API_KEY, OANDA_ACCOUNT_ID, OANDA_ENV=practice
dashboard	DASHBOARD_PASSWORD, OANDA_API_KEY, OANDA_ACCOUNT_ID, OANDA_ENV

Known limitations:

• Kill switch toggle from dashboard won't reach the engine (separate containers)
• /chart route on dashboard needs Supabase credentials

Phase 2 File Structure

src/
  position_manager.py        # Shared Position/TradeRecord + PositionManager
  live/
    __init__.py
    data_feed.py             # OANDA candle polling + indicator computation
    executor.py              # Paper/live order execution
    engine.py                # LiveEngine orchestrator (5 strategy slots)
    run.py                   # Entry point (--once or continuous)
    health.py                # Threaded HTTP health server for Railway
  run_regime_analysis.py     # Per-year performance breakdown
  run_correlation_analysis.py# Signal overlap + portfolio metrics
  run_kelly_sizing.py        # Kelly criterion + Monte Carlo
config/
  system.yaml                # phase2: section with strategy slot config
results/
  phase2/
    regime_analysis.json
    correlation_analysis.json
    kelly_sizing.json

---

Strategies (Legacy Reference)

SMA Cross (original)

Long-only strategy. Goes long when short SMA > long SMA, flat otherwise.

strategy:
  rule: sma_cross
  params:
    short: 50
    long: 100

Pivot Retest + Engulfing

Long/short strategy with dual take-profit and ATR-based stop loss.

Entry conditions (all must be true):

• Price retests a pivot level (broke through, then returned within ATR tolerance)
• SMA 50 aligns with trade direction relative to the pivot level
• Engulfing candle pattern confirmed
• Strong close (in top/bottom 30% of candle range)

Position management:

• SL: 1.5x ATR from entry
• TP1: next pivot level in trade direction (close 50%, move SL to breakeven)
• TP2: pivot level after TP1 (close remaining 50%)

strategy:
  rule: pivot_retest_engulfing
  params:
    sma_period: 50
    lookback_bars: 20
    retest_tolerance_atr: 0.5
    strong_close_pct: 0.30
    sl_atr_multiplier: 1.5

To switch strategies, edit config/system.yaml and change strategy.rule.

Configuration Reference

All settings live in config/system.yaml:

Section	Key settings
brokers[0].instruments	Currency pairs to trade (e.g. EUR_USD)
data.candle_granularities	Timeframes (M5, M15, H1, etc.)
features.*	Indicator windows (SMA, EMA, RSI, ATR, VWAP, volatility)
strategy.*	Active strategy rule + parameters
ai.*	ML ensemble config, confidence threshold, sanity checks
execution.paper_mode	true for paper trading, false for live
execution.interval_seconds	Bot loop interval
execution.max_positions	Max concurrent open trades

Web Dashboard

Flask-based UI for remote management.

# Railway (already deployed)
# https://dashboard-production-73e7.up.railway.app

# Docker (local)
docker-compose up -d

# Or run directly
python src/dashboard.py

Pages: Status (/), Backtest (/backtest), Chart (/chart), Config (/config), Logs (/logs), Health (/health)

Kill Switch

Create STOP_ALL_TRADING in the project root to halt all trading immediately. The bot checks for this file every loop iteration. In live mode, it also closes all open trades. Toggle via the dashboard or manually:

touch STOP_ALL_TRADING   # activate
rm STOP_ALL_TRADING      # deactivate

Economic Calendar

Filters trade entries near high-impact economic events (NFP, CPI, rate decisions, etc.) to reduce slippage and false signals. Uses the Trading Economics API.

Setup:

1. Subscribe to a Trading Economics API plan that includes Calendar data (pricing)
2. Add your API key to config/.env: TRADING_ECONOMICS_API_KEY=your-key-here
3. Create the Supabase table: run sql/create_economic_calendar.sql in the SQL Editor
4. Fetch events: python src/economic_calendar.py (full load) or python src/economic_calendar.py --incremental

Config (config/system.yaml → economic_calendar):

Key	Default	Description
enabled	true	Toggle calendar filter on/off
impact_threshold	High	Minimum impact level to block entries (Low, Medium, High)
event_buffer_minutes	30	Block entries within this many minutes of an event
days_back	400	How far back to fetch on full load

Status: Table created in Supabase. Awaiting Trading Economics API key (subscription request pending). Once the key is obtained, run the loader and the backtester will automatically filter entries near high-impact events.

Running the Bot

# Railway (24/7, already deployed)
# Live engine runs automatically on push to main

# Local: single cycle
python src/live/run.py --once

# Local: continuous loop (60s interval)
python src/live/run.py

# Docker (local)
docker-compose up -d