XauBot/backtests/36_ml_v2_results/COMPARISON_OLD_vs_NEW.md

Perbandingan Model Lama vs Model Baru (ML V2)

Tanggal: 2026-02-08 Tujuan: Jelaskan perbedaan antara model live saat ini dengan model ML V2 yang baru

---

📊 Ringkasan Perbandingan

Aspek	Model Lama (Live)	Model Baru (ML V2 Config D)
File	models/xgboost_model.pkl	backtests/36_ml_v2_results/model_d.pkl
Ukuran File	33 KB	68 KB
Jumlah Features	37 features	76 features (+39 baru)
Test AUC	~0.696 (dari log live)	0.7339
Improvement	—	+5.5% ✅
Target Type	1-bar lookahead	3-bar lookahead
Target Filter	Threshold = 0.0 (no filter)	Threshold = 0.3 * ATR
Model Architecture	XGBoost binary	XGBoost binary (sama)

---

🔍 Perbedaan Detail

1️⃣ Jumlah Features: 37 → 76 (+39 features baru)

Model Lama (37 features):

• Hanya base features dari src/feature_eng.py
• Contoh: RSI, MACD, ATR, BB, EMA, SMA, returns, volume, dll
• Semua dari timeframe M15 saja

Model Baru (76 features):

• 37 base features (sama seperti lama)
• +39 NEW features dari ML V2:
  • 9 H1 multi-timeframe features
  • 10 continuous SMC features
  • 5 regime conditioning features
  • 4 price action features
  • 11 additional features (is_fvg_bull/bear, ob_mitigated, dll)

---

2️⃣ Target Variable: 1-bar → 3-bar dengan ATR filter

Model Lama:

# Prediksi: apakah candle M15 berikutnya naik?
target = (df["close"].shift(-1) > df["close"]).astype(int)
# Threshold: 0.0 (prediksi semua move, termasuk noise)

Masalah: Terlalu noisy — ikut prediksi move kecil ($0.1-$1) yang tidak tradeable

Model Baru:

# Prediksi: apakah ada move signifikan dalam 3 bar ke depan?
max_future = df["close"].shift(-1, -2, -3).max()
min_future = df["close"].shift(-1, -2, -3).min()

# Filter: move harus > 0.3 * ATR (~$3-4 untuk ATR $12)
UP = 1 if (max_future - current) > 0.3 * ATR
DOWN = 0 if (current - min_future) > 0.3 * ATR
HOLD = None (filtered out)  # Move terlalu kecil, tidak diprediksi

Keuntungan: Fokus pada move yang tradeable, filter out noise

---

3️⃣ Performa: Test AUC 0.696 → 0.7339 (+5.5%)

Model Lama:

• Test AUC: ~0.696 (dari live logs)
• Train/Test overfitting: tidak diketahui
• Prediksi banyak noise

Model Baru:

• Test AUC: 0.7339
• Train AUC: 0.7385 (overfitting ratio 1.01 ✅)
• Prediksi lebih akurat, fokus pada tradeable moves

---

📦 39 Features Baru yang Ditambahkan

1. H1 Multi-Timeframe (9 features)

Feature ini menambahkan konteks dari timeframe H1 (1 jam) ke prediksi M15.

Feature	Deskripsi	Kenapa Penting?
h1_ema20	H1 EMA20 value	Higher TF trend
h1_market_structure	H1 BOS-based trend (+1/-1/0)	HTF trend confirmation
h1_ema20_distance	(M15 close - H1 EMA20) / ATR	Overbought/oversold vs HTF
h1_trend_strength	Count H1 BOS in last 10 bars	HTF trend momentum
h1_swing_proximity	Distance to H1 swing / ATR	HTF support/resistance
h1_fvg_active	1 if price inside H1 FVG	HTF imbalance zone
h1_ob_proximity	Distance to H1 OB / ATR	HTF supply/demand zone
h1_atr_ratio	H1 ATR / M15 ATR	Volatility context
h1_rsi	H1 RSI value	HTF momentum

Impact: +0.08 AUC (terbesar!) — menambahkan H1 context adalah game changer

---

2. Continuous SMC Features (10 features)

Model lama hanya punya binary SMC (OB ada/tidak, FVG ada/tidak). Model baru punya continuous SMC values.

Feature	Deskripsi	Kenapa Lebih Baik?
fvg_gap_size_atr	FVG gap size / ATR	Gap besar = more reliable
fvg_age_bars	Bars since last FVG	Fresh FVG = lebih valid
ob_width_atr	OB width / ATR	Wide OB = stronger zone
ob_distance_atr	Distance to OB / ATR	Dekat OB = potential reversal
bos_recency	Bars since last BOS	Fresh BOS = trend just started
confluence_score	Count OB+FVG+BOS in last 10 bars	Multiple SMC signals = stronger
swing_distance_atr	Distance to swing / ATR	Near swing = S/R level
is_fvg_bull / is_fvg_bear	FVG direction	Directional bias
ob_mitigated	OB touched?	OB validity tracking

Impact: +0.004 AUC — incremental improvement

---

3. Regime Conditioning Features (5 features)

Mengadaptasi strategi berdasarkan kondisi market (trending/ranging/volatile).

Feature	Deskripsi	Use Case
regime_confidence	HMM regime probability	High confidence = trust regime
regime_duration_bars	Consecutive bars in regime	Long duration = stable regime
regime_transition_prob	1 / duration	High = regime about to change
volatility_zscore	(ATR - mean) / std	Spike detection
crisis_proximity	ATR / (mean * 2.5)	Extreme volatility warning

Impact: +0.01-0.02 AUC — membantu model tahu kapan harus konservatif

---

4. Price Action Features (4 features)

Candle pattern characteristics.

Feature	Deskripsi	Use Case
wick_ratio	(upper + lower wick) / range	High wick = rejection
body_ratio	body / range	Small body = indecision
gap_from_prev_close	Gap / ATR	Gap up/down detection
consecutive_direction	# candles same direction	Momentum continuation

Impact: +0.01 AUC — pattern recognition

---

🎯 Kenapa Model Baru Lebih Baik?

1. Higher Timeframe Context (H1)

• Model lama cuma lihat M15 → myopic
• Model baru lihat M15 + H1 → big picture + detail
• Analogi: Kayak lihat peta kota (H1) sambil navigate jalan (M15)

2. Continuous SMC Values

• Model lama: "Ada OB atau tidak?" (binary 0/1)
• Model baru: "Seberapa besar OB-nya? Seberapa dekat? Seberapa fresh?" (continuous values)
• Analogi: Bukan cuma tahu "ada hujan", tapi tahu "hujan seberapa deras"

3. Better Target (Less Noise)

• Model lama: prediksi semua move termasuk $0.5 noise
• Model baru: filter move < $3-4, fokus yang tradeable
• Analogi: Bukan tangkap semua ikan, fokus ikan besar aja

4. Regime Awareness

• Model lama: treat semua kondisi market sama
• Model baru: tahu kapan market trending/ranging/volatile
• Analogi: Pakai strategi berbeda untuk cuaca berbeda

---

🚀 Apakah Model Baru Siap Dipakai Live?

✅ Kelebihan:

1. +5.5% AUC improvement (0.696 → 0.7339) ✅
2. Overfitting terkontrol (train/test ratio 1.01) ✅
3. Incremental testing (Baseline → A → B → C → D) semua improve ✅
4. Same architecture (XGBoost, anti-overfitting params sama) ✅

⚠️ Yang Harus Dites Dulu:

1. Backtest dengan trading logic lengkap — AUC tinggi belum tentu profit tinggi
2. Compare WR%, PnL, Sharpe vs model lama di data yang sama
3. Forward test di demo 1 minggu — cek real-time performance
4. Monitor false positives — apakah banyak signal palsu?

📋 Next Steps:

Langkah 1: Backtest Full Trading Logic

# Modifikasi backtest untuk pakai model_d.pkl
# Compare dengan backtest pakai xgboost_model.pkl lama
python backtests/backtest_live_sync.py --model models/xgboost_model.pkl
python backtests/backtest_live_sync.py --model backtests/36_ml_v2_results/model_d.pkl

Langkah 2: Integrate ke Live (Jika Backtest Bagus)

# Modify main_live.py:
# 1. Fetch H1 data
df_h1 = mt5_conn.get_market_data("XAUUSD", "H1", 100)

# 2. Add V2 features
from backtests.ml_v2 import MLV2FeatureEngineer
fe_v2 = MLV2FeatureEngineer()
df_m15 = fe_v2.add_all_v2_features(df_m15, df_h1)

# 3. Load model_d.pkl
model = TradingModelV2.load("models/xgboost_model_v2.pkl")

Langkah 3: Forward Test

• Deploy ke demo account
• Run 1 minggu
• Monitor WR%, PnL, DD

Langkah 4: Deploy ke Live

• Kalau demo success, copy model_d.pkl ke models/
• Deploy production

---

📌 Kesimpulan

Aspek	Model Lama	Model Baru
Features	37 (M15 only)	76 (M15 + H1 + SMC + Regime + PA)
Target	1-bar, no filter	3-bar, ATR filter
Test AUC	0.696	0.7339 (+5.5%)
Status	Live production	Ready for testing
Recommendation	—	✅ Backtest dulu, lalu integrate

Bottom Line: Model baru lebih pintar (76 vs 37 features), lebih akurat (0.7339 vs 0.696 AUC), dan less noisy (ATR filter). Tapi harus dites dengan trading logic lengkap sebelum deploy live.