fx-quant-research/README.md

# tradingbot

A research framework for **5-minute gold / forex machine-learning trading**.

It covers the full pipeline: downloading and cleaning candle data, engineering
technical features, training several model architectures, turning predictions
into risk-managed signals, and backtesting them with walk-forward optimization.

---

## Project structure

```
tradingbot/
├── tradingbot/
│   ├── config.py                 # central config (reads secrets from env)
│   ├── data/
│   │   ├── oanda_connector.py     # OANDA v20 historical candle download
│   │   ├── loader.py              # CSV loading + time-gap analysis
│   │   ├── processor.py           # GoldDataProcessor: cleaning, gap fill, outliers
│   │   └── features.py            # GoldFeatureEngineer + GoldFeaturePipeline
│   ├── models/
│   │   ├── tree_ensemble.py       # TreeEnsemblePredictor (stacked trees + HMM)
│   │   ├── neural_ensemble.py     # NeuralEnsemblePredictor (Transformer/LSTM + Bayesian NN)
│   │   └── model_manager.py       # GoldModelManager (Optuna, Boruta, SHAP, 5-class)
│   ├── signals/
│   │   └── generator.py           # GoldSignalGenerator (signals + stops/targets)
│   ├── backtest/
│   │   ├── walk_forward.py        # WalkForwardOptimizer
│   │   └── metrics.py             # compute_performance_metrics
│   └── viz/
│       └── visualize.py           # visualize_signals + plotting helpers
├── notebooks/                     # exploratory research notebook
├── requirements.txt
└── .gitignore
```

## Model approaches

The framework ships three model architectures that can be used independently:

1. `**TreeEnsemblePredictor**` (`models/tree_ensemble.py`) — a binary up/down
  classifier that stacks XGBoost, LightGBM, CatBoost, RandomForest and
   ExtraTrees, with two meta-models and an HMM market-regime filter.
2. `**NeuralEnsemblePredictor**` (`models/neural_ensemble.py`) — stacks
  gradient-boosted trees with a Transformer + BiLSTM network and a Bayesian
   network (Monte-Carlo dropout) for uncertainty-aware filtering.
3. `**GoldModelManager` pipeline** (`models/model_manager.py` +
  `signals/generator.py` + `backtest/walk_forward.py`) — the most complete
   path: a 5-class classifier (strong/weak up, sideways, weak/strong down) with
   Boruta/RFE/PCA feature selection, Optuna tuning, SHAP analysis, volatility
   regime-aware ensembling, and walk-forward backtesting.

---

## Installation

```bash
pip install -r requirements.txt
```

**TA-Lib** also requires the underlying C library:

- macOS: `brew install ta-lib`
- Ubuntu/Debian: install `ta-lib` (package or build from source)
- Windows: install a prebuilt TA-Lib wheel / binaries

For live data downloads, set your OANDA token (never commit it):

```bash
export OANDA_ACCESS_TOKEN="your-token-here"
```

---

## Usage

### 1. Download data (optional — or bring your own OHLCV CSV)

```python
from tradingbot.data.oanda_connector import download

# Saves e.g. EURUSD_M5_<start>_to_<end>.csv in the current directory.
download(instrument="EUR_USD", timeframe="M5", lookback_days=1825)
```

A CSV is expected with a `timestamp` column plus `open, high, low, close, volume`.

### 2. Quick look at the data

```python
from tradingbot.data.loader import load_ohlcv_csv, summarize, analyze_time_gaps

data = load_ohlcv_csv("EURUSD_M5_20200309_to_20250308.csv")
summarize(data)
analyze_time_gaps(data)
```

### 3a. Tree-ensemble approach

```python
from tradingbot.models.tree_ensemble import run_model

predictor, metrics, results, returns = run_model(
    data_path="XAUUSD_M5_20200222_to_20250220.csv",
    forecast_bars=24,            # 2 hours of 5-minute bars
    confidence_threshold=0.67,
)
```

Visualize the result:

```python
from tradingbot.viz.visualize import visualize_signals

visualize_signals(data, results, returns, metrics, save_path="signals.png")
```

### 3b. Neural-ensemble approach

```python
import pandas as pd
from tradingbot.models.neural_ensemble import NeuralEnsemblePredictor

data = pd.read_csv("XAUUSD_H1_...csv", parse_dates=["timestamp"], index_col="timestamp")
train, test = data[:int(len(data) * 0.9)], data[int(len(data) * 0.9):]

predictor = NeuralEnsemblePredictor(forecast_period=12, confidence_threshold=0.6)
predictor.fit(train)
signals = predictor.predict(test)
```

### 3c. Full pipeline + walk-forward backtest

```python
from tradingbot.data.features import GoldFeaturePipeline
from tradingbot.backtest.walk_forward import WalkForwardOptimizer

# Engineer features and the 5-class target.
processed = GoldFeaturePipeline(forecast_horizon=6).process(
    "XAUUSD_M5_20200222_to_20250220.csv"
)

# Walk-forward optimization (windows are in days).
wf = WalkForwardOptimizer(
    train_window=35,
    step_size=5,
    test_window=5,
    feature_selection_interval=4,
)
results = wf.optimize(processed)
```

Generate signals from a trained model:

```python
from tradingbot.models.model_manager import GoldModelManager
from tradingbot.signals.generator import GoldSignalGenerator

manager = GoldModelManager(model_path="models/step_1", feature_selection_method="importance")
manager.load_models()

gen = GoldSignalGenerator(confidence_threshold=0.7, risk_reward_min=1.5, model_manager=manager)
signals = gen.generate_signals(processed.iloc[-2880:])  # last ~10 days
active = signals[signals["signal"] != 0]
```

---

## Disclaimer

This is research / educational code for exploring ML trading ideas on historical
data. It is **not** financial advice and makes no guarantee of profitability. Markets carry real risk of loss, validate any strategy thoroughly before risking capital.