GifariKemalandClaude Opus 4.6
757b499033
5 major issues reflected in documentation: 1. Confidence calibration (03-SMC, 00-ARSITEKTUR): - Base 55% + 10% each → base 40% + weighted scoring - Structure +15%, BOS/CHoCH +12%, FVG +8%, OB +10%, Trend +10% 2. ATR-based pullback filter (09-Entry, 00-ARSITEKTUR): - Hardcoded $2/$1.5 → bounce 15% ATR, consolidation 10% ATR 3. Smarter time-based exit (10-Exit, 05-Risk, 00-ARSITEKTUR): - 4h: check profit growth, not just profit<$5 - 6h: extend to 8h if profit>$10 and growing + ML agrees 4. Slippage validation (09-Entry, 23-Main, 00-ARSITEKTUR): - Check actual vs expected price, log if >0.15% 5. Partial fill handling (09-Entry, 23-Main, 00-ARSITEKTUR): - Check filled volume, use actual values for tracking Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
95 KiB
95 KiB
Arsitektur Lengkap — Smart AI Trading Bot
Dokumen: Arsitektur keseluruhan sistem dalam 1 file Instrumen: XAUUSD (Gold) M15 Platform: MetaTrader 5 Bahasa: Python 3.11+ (async, Polars, XGBoost, HMM) Database: PostgreSQL + CSV fallback Notifikasi: Telegram Bot API
Daftar Isi
- Gambaran Umum
- Diagram Arsitektur
- 23 Komponen
- Pipeline Data: Dari OHLCV ke Keputusan Trading
- Alur Entry: 11 Filter
- Alur Exit: 10 Kondisi
- Sistem Proteksi Risiko 4 Lapis
- AI/ML Engine
- Smart Money Concepts (SMC)
- Position Lifecycle
- Auto-Retraining & Model Management
- Infrastruktur & Database
- Konfigurasi & Parameter Kritis
- Performa & Timing
- Error Handling & Fault Tolerance
- Daftar File Source Code
1. Gambaran Umum
Apa Ini?
Bot trading otomatis yang menggabungkan 3 otak kecerdasan buatan untuk trading XAUUSD (Emas) di MetaTrader 5:
OTAK 1: Smart Money Concepts (SMC)
→ Membaca pola institusi besar (bank, hedge fund)
→ Menentukan DIMANA entry, SL, dan TP
OTAK 2: XGBoost Machine Learning
→ Memprediksi ARAH harga (naik/turun)
→ Memberikan tingkat keyakinan (confidence)
OTAK 3: Hidden Markov Model (HMM)
→ Membaca KONDISI pasar (tenang/volatile/krisis)
→ Menyesuaikan ukuran posisi dan agresivitas
Filosofi Desain
1. KESELAMATAN MODAL NOMOR 1
→ 4 lapis proteksi stop loss
→ Lot ultra-kecil (0.01-0.02)
→ Circuit breaker otomatis
2. TIDAK PERNAH CRASH
→ Setiap error di-catch, bot terus jalan
→ Database gagal? CSV fallback
→ MT5 putus? Auto-reconnect
3. SELF-IMPROVING
→ Model AI dilatih ulang otomatis setiap hari
→ Rollback otomatis jika model baru lebih buruk
→ Threshold confidence menyesuaikan kondisi pasar
4. TRANSPARAN
→ Setiap keputusan dicatat ke database
→ Notifikasi Telegram real-time
→ Laporan harian, jam-an, dan per-trade
Angka-Angka Kunci
| Parameter | Nilai | Penjelasan |
|---|---|---|
| Modal target | $5,000 | Small account mode |
| Risiko per trade | 1% ($50) | Maksimum kerugian per posisi |
| Lot size | 0.01 - 0.02 | Ultra-konservatif |
| Max daily loss | 3% ($150) | Circuit breaker harian |
| Max total loss | 10% ($500) | Stop total trading |
| Max posisi bersamaan | 2-3 | Menghindari overexposure |
| Cooldown antar trade | 5 menit | Mencegah overtrading |
| Loop speed | ~50ms | Cepat tapi efisien |
| Timeframe | M15 | 15 menit per candle |
2. Diagram Arsitektur
Diagram Keseluruhan Sistem
┌─────────────────────────────────────────────────────────────────────────┐
│ MAIN LIVE (Orchestrator) │
│ main_live.py — TradingBot │
│ Candle-based (M15) + position check ~10 detik │
│ │
│ ┌─────────────── PHASE 1: DATA ──────────────────────────────────┐ │
│ │ │ │
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │
│ │ │ MT5 │ │ Feature │ │ SMC │ │ HMM │ │ │
│ │ │Connector │ ──→│ Engine │ ──→│ Analyzer │ │ Regime │ │ │
│ │ │(broker) │ │(40+ fitur│ │(institusi│ │(3 state) │ │ │
│ │ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │ │
│ │ │ │ │ │ │ │
│ │ │ │ ▼ ▼ │ │
│ │ │ │ ┌──────────┐ ┌──────────┐ │ │
│ │ │ └────────→│ XGBoost │ │ Dynamic │ │ │
│ │ │ │ ML Model │ │Confidence│ │ │
│ │ │ │(prediksi)│ │(threshold│ │ │
│ │ │ └──────────┘ └──────────┘ │ │
│ └───────┼─────────────────────────────┼───────────────┼────────┘ │
│ │ │ │ │
│ ┌───────┼──── PHASE 2: MONITORING ────┼───────────────┼────────┐ │
│ │ │ │ │ │ │
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │
│ │ │ Position │ │ Smart │ │ Risk │ │ │
│ │ │ Manager │ │ Risk │ │ Engine │ │ │
│ │ │(trailing)│ │ Manager │ │ (Kelly) │ │ │
│ │ └──────────┘ └──────────┘ └──────────┘ │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │ │ │ │
│ ┌───────┼──── PHASE 3: ENTRY ─────────┼───────────────┼────────┐ │
│ │ │ │ │ │ │
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │
│ │ │ Session │ │ News │ │ 11-Gate │ │ │
│ │ │ Filter │ │ Agent │ │ Entry │ │ │
│ │ │(waktu) │ │(berita) │ │ Filter │──→ EXECUTE │ │
│ │ └──────────┘ └──────────┘ └──────────┘ │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌───────┼──── PHASE 4: PERIODIK ──────────────────────────────┐ │
│ │ │ │ │
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │
│ │ │ Auto │ │ Telegram │ │ Trade │ │ │
│ │ │ Trainer │ │ Notifier │ │ Logger │ │ │
│ │ │(retrain) │ │(laporan) │ │(DB+CSV) │ │ │
│ │ └──────────┘ └──────────┘ └──────────┘ │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌────────┴────────┐ │
│ │ PostgreSQL │ │
│ │ + CSV Backup │ │
│ └─────────────────┘ │
└─────────────────────────────────────────────────────────────────────────┘
Alur Data (Data Flow)
MT5 Broker (XAUUSD M15)
│
│ 200 bar OHLCV
▼
┌─────────────────┐
│ MT5 Connector │ numpy → Polars (tanpa Pandas)
└────────┬────────┘
│
▼
┌─────────────────┐
│ Feature Engineer │ OHLCV → 40+ fitur teknikal
│ │ RSI, ATR, MACD, BB, EMA, Volume,
│ │ Returns, Volatility, Lags, Trend
└────────┬────────┘
│
┌────┴────┐
│ │
▼ ▼
┌───────┐ ┌───────┐
│ SMC │ │ HMM │
│Analyzer│ │Regime │
│ │ │Detect │
└───┬───┘ └───┬───┘
│ │
│ ┌────┘
│ │
▼ ▼
┌─────────────────┐
│ XGBoost │ 24 fitur → Prediksi BUY/SELL/HOLD
│ ML Predictor │ + Confidence 0-100%
└────────┬────────┘
│
▼
┌─────────────────┐
│ Dynamic │ Sesuaikan threshold berdasarkan
│ Confidence │ sesi, regime, volatilitas, trend
└────────┬────────┘
│
▼
┌─────────────────┐
│ Signal Combiner │ SMC + ML harus setuju
│ (11 Filter) │ + Session + Risk + Cooldown
└────────┬────────┘
│
┌────┴────┐
│ PASS? │
│ │
YES NO → tunggu loop berikutnya
│
▼
┌─────────────────┐
│ Risk Engine │ Kelly Criterion → lot size
│ + Risk Manager │ Validasi order → approve/reject
└────────┬────────┘
│
▼
┌─────────────────┐
│ Execute Order │ Kirim ke MT5 dengan SL & TP
│ via MT5 │ Register ke Position Manager
└────────┬────────┘
│
┌────┴────────────────┐
│ │
▼ ▼
┌───────────┐ ┌────────────────┐
│ Telegram │ │ Trade Logger │
│ Notifier │ │ (DB + CSV) │
└───────────┘ └────────────────┘
3. 23 Komponen
Tabel Komponen Lengkap
| # | Komponen | File | Kategori | Fungsi Utama |
|---|---|---|---|---|
| 1 | HMM Regime Detector | src/regime_detector.py |
AI/ML | Deteksi kondisi pasar (3 regime) |
| 2 | XGBoost Predictor | src/ml_model.py |
AI/ML | Prediksi arah harga + confidence |
| 3 | SMC Analyzer | src/smc_polars.py |
Analisis | Pola institusi: FVG, OB, BOS, CHoCH |
| 4 | Feature Engineering | src/feature_eng.py |
Data | OHLCV → 40+ fitur numerik |
| 5 | Smart Risk Manager | src/smart_risk_manager.py |
Risiko | 4 mode trading, 10 kondisi exit |
| 6 | Session Filter | src/session_filter.py |
Filter | Waktu trading optimal (WIB) |
| 7 | Stop Loss (4 Lapis) | Multi-file | Proteksi | SMC → Software → Emergency → Circuit |
| 8 | Take Profit (6 Layer) | Multi-file | Proteksi | Hard → Momentum → Peak → Probability → Early → Broker |
| 9 | Entry Trade | main_live.py |
Eksekusi | 11 filter berurutan |
| 10 | Exit Trade | main_live.py |
Eksekusi | 10 kondisi exit real-time |
| 11 | News Agent | src/news_agent.py |
Monitor | Monitoring berita (TIDAK memblokir) |
| 12 | Telegram Notifier | src/telegram_notifier.py |
Notifikasi | 11 tipe notifikasi real-time |
| 13 | Auto Trainer | src/auto_trainer.py |
ML Ops | Retraining harian otomatis |
| 14 | Backtest | backtests/backtest_live_sync.py |
Validasi | Simulasi 100% sync dengan live |
| 15 | Dynamic Confidence | src/dynamic_confidence.py |
Adaptif | Threshold ML adaptif (60-85%) |
| 16 | MT5 Connector | src/mt5_connector.py |
Koneksi | Bridge ke broker, auto-reconnect |
| 17 | Configuration | src/config.py |
Config | 6 sub-config, auto-adjust modal |
| 18 | Trade Logger | src/trade_logger.py |
Logging | Dual storage DB + CSV |
| 19 | Position Manager | src/position_manager.py |
Manajemen | Trailing SL, breakeven, market close |
| 20 | Risk Engine | src/risk_engine.py |
Risiko | Kelly Criterion, circuit breaker |
| 21 | Database | src/db/ |
Storage | PostgreSQL, 6 repository |
| 22 | Train Models | train_models.py |
Training | Script training awal |
| 23 | Main Live | main_live.py |
Orchestrator | Koordinasi semua komponen |
Hubungan Antar Komponen
┌─────────────────────────┐
│ CONFIGURATION (17) │
│ Sumber parameter semua │
└────────────┬─────────────┘
│ dikonsumsi oleh semua
▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ MT5 (16) │───→│FeatEng(4)│───→│ SMC (3) │
│ Broker │ │ 40+ fitur│ │ Institusi│
└──────────┘ └────┬─────┘ └────┬─────┘
│ │
▼ ▼
┌──────────┐ ┌──────────┐
│ HMM (1) │ │XGBoost(2)│
│ Regime │ │ Prediksi │
└────┬─────┘ └────┬─────┘
│ │
▼ ▼
┌──────────────────────────┐
│ Dynamic Confidence (15) │
│ Threshold adaptif │
└────────────┬─────────────┘
│
▼
┌──────────┐ ┌──────────────────────────┐ ┌──────────┐
│Session(6)│───→│ ENTRY TRADE (9) │←───│ News(11) │
│ Waktu │ │ 11 Filter Gate │ │ Berita │
└──────────┘ └────────────┬─────────────┘ └──────────┘
│
┌────────┴────────┐
▼ ▼
┌──────────┐ ┌──────────────┐
│RiskEng(20│ │SmartRisk (5) │
│Kelly Lot │ │ 4 Mode │
└────┬─────┘ └──────┬───────┘
│ │
▼ ▼
┌──────────────────────────┐
│ EXECUTE ORDER │
│ via MT5 (16) │
└────────────┬─────────────┘
│
┌────────────┼────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│PosMgr(19)│ │Logger(18)│ │Telegram │
│Trailing │ │ DB+CSV │ │ (12) │
└────┬─────┘ └────┬─────┘ └──────────┘
│ │
▼ ▼
┌──────────┐ ┌──────────┐
│EXIT (10) │ │ DB (21) │
│10 Kondisi│ │PostgreSQL│
└──────────┘ └──────────┘
Periodik:
┌──────────┐ ┌──────────┐ ┌──────────┐
│AutoTrain │ │ Backtest │ │ Train │
│ (13) │ │ (14) │ │Models(22)│
│Harian │ │Validasi │ │Setup awal│
└──────────┘ └──────────┘ └──────────┘
4. Pipeline Data
Dari OHLCV Mentah ke Keputusan Trading
Tahap 1: Data Fetching (MT5 Connector)
MT5 Broker
│
│ mt5.copy_rates_from_pos("XAUUSD", MT5_TIMEFRAME_M15, 0, 200)
│
▼
NumPy Structured Array
│
│ Konversi langsung ke Polars (TANPA Pandas)
│
▼
Polars DataFrame:
┌──────────────────────────────────────────────────────┐
│ time │ open │ high │ low │ close │ tick_volume │ spread │
│ i64 │ f64 │ f64 │ f64 │ f64 │ f64 │ f64 │
├──────┼──────┼──────┼─────┼───────┼─────────────┼────────┤
│ ... │ 2645 │ 2648 │ 2643│ 2647 │ 5234 │ 25 │
└──────────────────────────────────────────────────────┘
Kenapa Polars, bukan Pandas?
- 3-5x lebih cepat untuk operasi vectorized
- Memory-efficient (zero-copy)
- Native lazy evaluation
- Konsisten di seluruh codebase (tidak ada konversi bolak-balik)
Tahap 2: Feature Engineering (40+ Fitur)
Input: Polars DataFrame (200 bar OHLCV)
│
├── Momentum Indicators
│ ├── RSI(14) → 0-100, overbought/oversold
│ ├── MACD(12,26,9) → trend strength & direction
│ └── MACD Histogram → momentum acceleration
│
├── Volatility Indicators
│ ├── ATR(14) → average true range (pips)
│ ├── Bollinger Bands(20,2.0) → upper, lower, width
│ └── Volatility(20) → rolling std of returns
│
├── Trend Indicators
│ ├── EMA(9) / EMA(21) → fast/slow crossover
│ ├── EMA Cross Signal → 1 (bullish) / -1 (bearish)
│ └── SMA(20) → simple moving average
│
├── Price Action
│ ├── Returns(1,5,20) → % perubahan harga
│ ├── Log Returns → untuk distribusi normal
│ ├── Price Position → posisi dalam range BB
│ └── Higher High/Lower Low count → trend structure
│
├── Volume Features
│ ├── Volume SMA(20) → rata-rata volume
│ └── Volume Ratio → current / average
│
├── Lag Features
│ ├── close_lag_1..5 → harga sebelumnya
│ └── returns_lag_1..3 → return sebelumnya
│
└── Time Features
├── Hour, Weekday → waktu candle
└── Session flags → london, ny, overlap
│
▼
Output: DataFrame + 40 kolom baru (semua numerik, siap ML)
Minimum data: 26 bar untuk semua indikator stabil
Tahap 3: SMC Analysis (Pola Institusi)
Input: DataFrame dengan OHLCV
│
├── Swing Points (Fractal)
│ Window: 11 bar (swing_length=5, ±5 dari tengah)
│ Output: swing_high (1/0), swing_low (-1/0), level harga
│
├── Fair Value Gaps (FVG)
│ Bullish: bar[i-2].high < bar[i].low (gap up)
│ Bearish: bar[i-2].low > bar[i].high (gap down)
│ Output: fvg_bull, fvg_bear, fvg_top, fvg_bottom, fvg_mid
│
├── Order Blocks (OB)
│ Lookback: 10 bar
│ Bullish: candle bearish terakhir sebelum move up besar
│ Bearish: candle bullish terakhir sebelum move down besar
│ Output: ob (1/-1), ob_top, ob_bottom, ob_mitigated
│
├── Break of Structure (BOS)
│ BOS: harga break swing high/low → trend continuation
│ Output: bos (1/-1), level yang di-break
│
├── Change of Character (CHoCH)
│ CHoCH: harga break berlawanan arah trend → reversal signal
│ Output: choch (1/-1), level yang di-break
│
└── Liquidity Zones
BSL: Buy Side Liquidity (above swing highs)
SSL: Sell Side Liquidity (below swing lows)
Output: bsl_level, ssl_level
│
▼
Signal Generation:
Syarat: Structure break + (FVG ATAU Order Block)
│
├── Entry: harga saat ini
├── SL: ATR-based, minimum 1.5 × ATR dari entry
├── TP: 2:1 Risk-Reward minimum, cap 4 × ATR
├── Confidence: 40-85% (v5: calibrated weighted scoring) (berdasarkan confluence)
└── Reason: "BOS + Bullish FVG at 2645.50"
Tahap 4: Regime Detection (HMM)
Input: log_returns + normalized_range (volatilitas)
│
│ GaussianHMM(n_components=3, lookback=500)
│
▼
3 Regime:
┌──────────────────────────────────────────────────┐
│ REGIME 0: Low Volatility │
│ → Pasar tenang, range kecil │
│ → Lot multiplier: 1.0x (normal) │
│ → Rekomendasi: TRADE │
│ │
│ REGIME 1: Medium Volatility │
│ → Pasar aktif, trend jelas │
│ → Lot multiplier: 1.0x (normal) │
│ → Rekomendasi: TRADE │
│ │
│ REGIME 2: High Volatility │
│ → Pasar sangat volatile, berbahaya │
│ → Lot multiplier: 0.5x (setengah) │
│ → Rekomendasi: REDUCE │
│ │
│ CRISIS (detected by FlashCrashDetector): │
│ → Move > 2.5% dalam 1 menit │
│ → Lot multiplier: 0.0x (STOP) │
│ → Rekomendasi: EMERGENCY CLOSE ALL │
└──────────────────────────────────────────────────┘
Tahap 5: ML Prediction (XGBoost)
Input: 24 fitur terpilih dari Feature Engineering + SMC + Regime
│
│ XGBoost Binary Classifier
│ Anti-overfitting config:
│ max_depth=3, learning_rate=0.05
│ min_child_weight=10, subsample=0.7
│ colsample_bytree=0.6
│ reg_alpha=1.0 (L1), reg_lambda=5.0 (L2)
│
▼
Output:
┌──────────────────────────────────────────┐
│ prob_up: 0.72 (probabilitas naik) │
│ prob_down: 0.28 (probabilitas turun) │
│ │
│ → Signal: BUY (prob_up > 0.50) │
│ → Confidence: 72% │
│ │
│ Threshold Keputusan: │
│ prob > 0.50 → ada sinyal (minimum) │
│ prob > 0.65 → sinyal kuat │
│ prob > 0.75 → sinyal sangat kuat │
│ prob > 0.80 → lot bisa naik ke 0.02 │
└──────────────────────────────────────────┘
Tahap 6: Dynamic Confidence (Threshold Adaptif)
Scoring (0-100 poin):
Base score: 50
│
├── Session Modifier:
│ Golden Time (20:00-23:59 WIB) → +20
│ London (15:00-23:59) → +15
│ New York (20:00-05:00) → +10
│ Tokyo/Sydney → +0
│ Market Closed → -30
│
├── Regime Modifier:
│ Medium Volatility → +15
│ Low Volatility → +5
│ High Volatility → -5
│ Crisis → -25
│
├── Volatility Modifier:
│ Medium (ideal) → +10
│ Low → +0
│ High → -5
│ Extreme → -10
│
├── Trend Modifier:
│ Trending (jelas) → +10
│ Ranging (sideways) → -5
│
├── SMC Confluence:
│ Ada konfluensi → +10
│ Tidak ada → +0
│
└── ML Confidence:
≥ 70% → +5
≥ 60% → +2
< 60% → +0
│
▼
Quality Level → Threshold:
┌───────────────────────────────────────────────────┐
│ EXCELLENT (≥80 poin) → Threshold: 60% (longgar) │
│ GOOD (65-79) → Threshold: 65% │
│ MODERATE (50-64) → Threshold: 70% │
│ POOR (35-49) → Threshold: 80% (ketat) │
│ AVOID (<35) → Threshold: 85% (SKIP) │
└───────────────────────────────────────────────────┘
Contoh: Golden Time + Medium Vol + Trending + SMC + ML 72%
= 50 + 20 + 15 + 10 + 10 + 10 + 5 = 120 → cap 100
= EXCELLENT → Threshold 60% → ML 72% PASS ✓
5. Alur Entry: 11 Filter
Setiap sinyal harus melewati 11 gerbang berurutan. Satu saja gagal = TIDAK trading.
SINYAL SMC + ML MASUK
│
▼
┌─ FILTER 1: Session Filter ──────────────────────────────┐
│ Apakah sekarang jam trading yang dibolehkan? │
│ ✗ 00:00-06:00 WIB (dead zone) → BLOCK │
│ ✗ Jumat ≥ 23:00 WIB (weekend risk) → BLOCK │
│ ✓ London/NY/Golden Time → PASS │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 2: Risk Mode ───────────────────────────────────┐
│ Apakah mode trading bukan STOPPED? │
│ ✗ STOPPED (daily/total limit hit) → BLOCK │
│ ✓ NORMAL / RECOVERY / PROTECTED → PASS │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 3: SMC Signal ──────────────────────────────────┐
│ Apakah ada setup SMC yang valid? │
│ ✗ Tidak ada FVG/OB + BOS/CHoCH → BLOCK │
│ ✓ Ada sinyal BUY/SELL dengan SL & TP → PASS │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 4: ML Confidence ───────────────────────────────┐
│ Apakah ML confidence ≥ dynamic threshold? │
│ ✗ ML confidence < threshold → BLOCK │
│ ✓ ML confidence ≥ threshold → PASS │
│ (threshold 60-85% tergantung market quality) │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 5: ML Agreement ────────────────────────────────┐
│ Apakah ML TIDAK strongly disagree dengan SMC? │
│ ✗ ML > 65% berlawanan arah SMC → BLOCK (conflict) │
│ ✓ ML setuju atau netral → PASS │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 6: Market Quality ──────────────────────────────┐
│ Apakah Dynamic Confidence bukan AVOID? │
│ ✗ Quality == AVOID (score < 35) → BLOCK │
│ ✓ EXCELLENT/GOOD/MODERATE/POOR → PASS │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 7: Signal Confirmation ─────────────────────────┐
│ Apakah sinyal konsisten 2x berturut-turut? │
│ ✗ Sinyal baru muncul 1x → BLOCK (tunggu konfirmasi) │
│ ✓ Sinyal sudah 2x berturut → PASS │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 8: Pullback Filter (v5: ATR-based) ────────────┐
│ Apakah momentum selaras dengan arah sinyal? │
│ ✗ BUY tapi RSI > 80 (overbought) → BLOCK │
│ ✗ SELL tapi RSI < 20 (oversold) → BLOCK │
│ ✗ MACD histogram berlawanan → BLOCK │
│ ✗ Harga bounce > 15% ATR dalam 3 candle → BLOCK │
│ ✓ Movement < 10% ATR (consolidation) → PASS │
│ ✓ Momentum selaras → PASS │
│ (v5: threshold dinamis berdasarkan ATR, bukan fixed $2) │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 9: Trade Cooldown ──────────────────────────────┐
│ Apakah sudah ≥ 5 menit sejak trade terakhir? │
│ ✗ < 300 detik sejak trade terakhir → BLOCK │
│ ✓ ≥ 300 detik → PASS │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 10: Position Limit ─────────────────────────────┐
│ Apakah jumlah posisi terbuka < limit? │
│ ✗ Sudah 2+ posisi terbuka → BLOCK │
│ ✓ < 2 posisi → PASS │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 11: Lot Size ───────────────────────────────────┐
│ Apakah lot size yang dihitung > 0? │
│ ✗ Lot = 0 (regime crisis / risk terlalu tinggi) → BLOCK│
│ ✓ Lot ≥ 0.01 → PASS │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
╔═══════════════════════╗
║ EXECUTE TRADE ║
║ BUY atau SELL ║
║ via MT5 Connector ║
╚═══════════════════════╝
│
├── Register ke SmartRiskManager
├── Log ke TradeLogger (DB + CSV)
└── Kirim notifikasi Telegram
6. Alur Exit: 10 Kondisi
Setiap posisi terbuka dievaluasi setiap ~10 detik (di antara candle) atau setiap candle baru (full analysis) terhadap 10 kondisi exit:
POSISI TERBUKA (dicek setiap ~10 detik)
│
│ Update: profit, momentum, peak, durasi
│
▼
┌─ KONDISI 1: Smart Take Profit ──────────────────────────┐
│ (a) Profit ≥ $40 → TUTUP (hard TP) │
│ (b) Profit ≥ $25 + momentum < -30 → TUTUP │
│ (c) Peak > $30, sekarang < 60% peak → TUTUP │
│ (d) Profit ≥ $20 + TP probability < 25% → TUTUP │
└──────────────────────────────────────────────────────────┘
│ tidak trigger
▼
┌─ KONDISI 2: Early Exit (Profit Kecil) ──────────────────┐
│ Profit $5-$15 + ML reversal ≥ 65% + momentum < -50 │
│ → TUTUP (amankan profit kecil sebelum hilang) │
└──────────────────────────────────────────────────────────┘
│ tidak trigger
▼
┌─ KONDISI 3: Early Cut (v4 — Smart Hold DIHAPUS) ────────┐
│ Loss >= 30% max ($15) DAN momentum < -30? │
│ → TUTUP CEPAT (early cut, jangan tunggu recovery) │
│ │
│ v4: "Smart Hold" dihapus — tidak ada lagi hold losers │
│ menunggu golden time atau sesi London. │
└──────────────────────────────────────────────────────────┘
│ tidak trigger
▼
┌─ KONDISI 4: ML Trend Reversal ──────────────────────────┐
│ ML confidence ≥ 65% BERLAWANAN ARAH posisi │
│ + 3x warning berturut-turut │
│ → TUTUP (AI mendeteksi pembalikan trend) │
└──────────────────────────────────────────────────────────┘
│ tidak trigger
▼
┌─ KONDISI 5: Maximum Loss ───────────────────────────────┐
│ Loss ≥ 50% dari max_loss_per_trade ($25 dari $50) │
│ → TUTUP (kerugian terlalu besar) │
└──────────────────────────────────────────────────────────┘
│ tidak trigger
▼
┌─ KONDISI 6: Stall Detection ────────────────────────────┐
│ Posisi sudah 10+ bar tanpa profit signifikan │
│ → TUTUP (pasar tidak bergerak sesuai ekspektasi) │
└──────────────────────────────────────────────────────────┘
│ tidak trigger
▼
┌─ KONDISI 7: Daily Limit ────────────────────────────────┐
│ Total daily loss mendekati limit │
│ → TUTUP semua posisi (proteksi sisa modal hari ini) │
└──────────────────────────────────────────────────────────┘
│ tidak trigger
▼
┌─ KONDISI 8: Weekend Close ──────────────────────────────┐
│ Mendekati market close weekend + ada posisi profit │
│ → TUTUP (hindari gap risk Senin) │
└──────────────────────────────────────────────────────────┘
│ tidak trigger
▼
┌─ KONDISI 9: Smart Time-Based (v5: Don't Cut Winners) ──┐
│ (a) > 4 jam + no growth → TUTUP (stuck) │
│ (b) > 4 jam + growing + ML agrees → HOLD (extend) │
│ (c) > 6 jam + profit < $10 → TUTUP │
│ (d) > 6 jam + profit > $10 + growing → extend ke 8 jam │
│ (e) > 8 jam → TUTUP (final max time) │
└──────────────────────────────────────────────────────────┘
│ tidak trigger
▼
┌─ KONDISI 10: Default HOLD ──────────────────────────────┐
│ Tidak ada kondisi terpenuhi │
│ → HOLD (biarkan posisi berjalan) │
└──────────────────────────────────────────────────────────┘
Position Manager (Tambahan per Posisi)
Selain 10 kondisi di atas, Position Manager juga menjalankan:
┌─ Market Close Handler (Prioritas Tertinggi) ────────────┐
│ Dekat close harian/weekend? │
│ ├── Profit ≥ $10 + dekat close → TUTUP (amankan) │
│ ├── Loss + weekend + SL > 50% → TUTUP (gap risk) │
│ └── Loss kecil + weekend → HOLD (bisa recovery) │
└──────────────────────────────────────────────────────────┘
┌─ Breakeven Protection ──────────────────────────────────┐
│ Profit ≥ 15 pips → Pindah SL ke entry + 2 buffer │
│ (posisi tidak bisa rugi lagi) │
└──────────────────────────────────────────────────────────┘
┌─ Trailing Stop ─────────────────────────────────────────┐
│ Profit ≥ 25 pips → SL mengikuti harga, jarak 10 pips │
│ (kunci profit sambil biarkan berjalan) │
└──────────────────────────────────────────────────────────┘
7. Sistem Proteksi Risiko 4 Lapis
╔══════════════════════════════════════════════════════════════════╗
║ LAPIS 1: BROKER STOP LOSS ║
║ (Otomatis oleh MT5) ║
║ ║
║ SL = Entry ± (1.5 × ATR) minimum 10 pips ║
║ Dikirim bersama order ke broker ║
║ Aktif 24/7, bahkan jika bot mati ║
║ Max loss: ~$50-80 per trade ║
╠══════════════════════════════════════════════════════════════════╣
║ LAPIS 2: SOFTWARE SMART EXIT ║
║ (Bot mengevaluasi setiap ~10 detik) ║
║ ║
║ 10 kondisi exit (lihat bagian 6) ║
║ Biasanya menutup SEBELUM broker SL kena ║
║ Target close: loss ≤ $25 (lebih ketat dari broker) ║
║ Termasuk: momentum, ML reversal, stall, time limit ║
╠══════════════════════════════════════════════════════════════════╣
║ LAPIS 3: EMERGENCY STOP LOSS ║
║ (Backup jika software gagal) ║
║ ║
║ Max loss per trade: 2% modal ($100 untuk $5K) ║
║ Diset sebagai broker SL terpisah ║
║ Aktif jika software error/hang ║
╠══════════════════════════════════════════════════════════════════╣
║ LAPIS 4: CIRCUIT BREAKER ║
║ (Hentikan semua trading) ║
║ ║
║ Trigger 1: Daily loss ≥ 3% ($150) → Stop hari ini ║
║ Trigger 2: Total loss ≥ 10% ($500) → Stop total ║
║ Trigger 3: Flash crash > 2.5% / 1 menit → CLOSE ALL ║
║ Reset: Otomatis di hari baru (daily), manual (total) ║
╚══════════════════════════════════════════════════════════════════╝
4 Mode Trading (Smart Risk Manager)
┌──────────────────────────────────────────────────────────┐
│ MODE: NORMAL │
│ Kondisi: Semua aman, tidak ada masalah │
│ Lot: 0.01 - 0.02 (berdasarkan confidence) │
│ Max posisi: 2-3 │
│ │
│ │ 3x loss berturut-turut │
│ ▼ │
│ MODE: RECOVERY │
│ Kondisi: Setelah kerugian beruntun │
│ Lot: 0.01 (minimum saja) │
│ Max posisi: 1 │
│ │
│ │ mendekati 80% daily limit │
│ ▼ │
│ MODE: PROTECTED │
│ Kondisi: Hampir kena daily limit │
│ Lot: 0.01 (minimum saja) │
│ Max posisi: 1 │
│ │
│ │ daily/total limit tercapai │
│ ▼ │
│ MODE: STOPPED │
│ Kondisi: Batas kerugian tercapai │
│ Lot: 0 (TIDAK BOLEH trading) │
│ Max posisi: 0 (tutup semua) │
│ Reset: Otomatis hari baru │
└──────────────────────────────────────────────────────────┘
Lot Sizing: Risk-Constrained Half-Kelly
Langkah 1: Hitung Kelly Fraction
f* = (win_rate × avg_rr - (1 - win_rate)) / avg_rr
Contoh: win_rate=55%, avg_rr=2.0
f* = (0.55 × 2.0 - 0.45) / 2.0 = 0.325 (32.5%)
Langkah 2: Cap Kelly (max 25%)
f* = min(0.325, 0.25) = 0.25
Langkah 3: Half-Kelly (safety)
f* = 0.25 × 0.5 = 0.125 (12.5%)
Langkah 4: Apply regime multiplier
High volatility: × 0.5 = 0.0625
Normal: × 1.0 = 0.125
Langkah 5: Cap di config limit
config risk_per_trade = 1%
actual_risk = min(0.125, 0.01) = 0.01 (1%)
Langkah 6: Hitung lot
risk_amount = $5000 × 1% = $50
SL distance = 50 pips → pip_value ~$1/pip/0.01lot
lot = $50 / (50 × $1) = 0.01 lot
Langkah 7: ML Confidence boost
ML ≥ 80% → lot × 2 = 0.02 lot (maximum)
ML < 65% → lot = 0.01 (minimum)
Langkah 8: Session multiplier
Golden Time: × 1.2
Sydney: × 0.5
Final lot: 0.01 - 0.02 (ultra-konservatif)
8. AI/ML Engine
Hidden Markov Model (HMM) — Otak Regime
┌──────────────────────────────────────────────────────────┐
│ HIDDEN MARKOV MODEL │
│ │
│ Library: hmmlearn.GaussianHMM │
│ Input: log_returns + rolling_volatility (2 fitur) │
│ States: 3 (Low, Medium, High Volatility) │
│ Lookback: 500 bar untuk training │
│ Retrain: setiap 20 bar (auto-update) │
│ │
│ Transition Matrix (contoh): │
│ To Low To Med To High │
│ Fr Low [ 0.85 0.12 0.03 ] │
│ Fr Med [ 0.10 0.80 0.10 ] │
│ Fr High [ 0.05 0.15 0.80 ] │
│ │
│ Distribusi Emisi (per state): │
│ Low: μ_return ≈ 0, σ_return = kecil │
│ Med: μ_return ≈ 0, σ_return = sedang │
│ High: μ_return ≈ 0, σ_return = besar │
│ │
│ Output: │
│ ├── regime: 0/1/2 (low/medium/high) │
│ ├── confidence: 0.0 - 1.0 │
│ ├── lot_multiplier: 1.0 / 0.5 / 0.0 │
│ └── recommendation: TRADE / REDUCE / SLEEP │
└──────────────────────────────────────────────────────────┘
XGBoost — Otak Prediksi
┌──────────────────────────────────────────────────────────┐
│ XGBOOST BINARY CLASSIFIER │
│ │
│ Library: xgboost │
│ Objective: binary:logistic │
│ Target: UP (1) / DOWN (0) pada bar berikutnya │
│ │
│ Anti-Overfitting Config: │
│ ├── max_depth: 3 (shallow trees) │
│ ├── learning_rate: 0.05 (slow learning) │
│ ├── min_child_weight: 10 (min samples per leaf) │
│ ├── subsample: 0.7 (70% data per tree) │
│ ├── colsample_bytree: 0.6 (60% features per tree) │
│ ├── reg_alpha: 1.0 (L1 regularization) │
│ ├── reg_lambda: 5.0 (L2 regularization) │
│ ├── gamma: 1.0 (min loss reduction) │
│ └── num_boost_round: 50 (few rounds) │
│ │
│ 24 Fitur Input (Top 10): │
│ 1. RSI(14) 6. price_position │
│ 2. MACD_histogram 7. volatility_20 │
│ 3. ATR(14) 8. returns_5 │
│ 4. bb_width 9. ema_cross │
│ 5. returns_1 10. regime │
│ │
│ Output: │
│ ├── signal: BUY / SELL / HOLD │
│ ├── probability: 0.0 - 1.0 (prob of UP) │
│ └── confidence: 0.0 - 1.0 (prob of winning side) │
│ │
│ Validation: │
│ ├── Train/Test: 70% / 30% (50-bar gap, anti leakage) │
│ ├── Walk-forward: 500 train / 50 test / 50 step │
│ ├── Target AUC: > 0.65 │
│ ├── Rollback AUC: < 0.60 (v4: dinaikkan dari 0.52) │
│ └── Overfitting ratio: train_AUC/test_AUC < 1.15 │
└──────────────────────────────────────────────────────────┘
Kombinasi Sinyal (SMC + ML)
SMC Signal: "BUY at 2645, SL 2635, TP 2665, conf 75%"
ML Signal: "BUY, confidence 72%"
│
▼
┌─ KOMBINASI ──────────────────────────────────────────────┐
│ │
│ CASE 1: SMC BUY + ML BUY (≥50%) │
│ → Combined confidence = (75% + 72%) / 2 = 73.5% │
│ → ENTRY (jika pass 11 filter lainnya) │
│ │
│ CASE 2: SMC BUY + ML SELL (≥65%) │
│ → ML strongly disagrees → BLOCK (filter #5) │
│ → TIDAK entry │
│ │
│ CASE 3: SMC BUY + ML uncertain (<50%) │
│ → ML tidak yakin → BLOCK (filter #4) │
│ → TIDAK entry │
│ │
│ CASE 4: Tidak ada SMC signal │
│ → Tidak ada entry point → SKIP │
│ → SMC adalah sinyal PRIMER (wajib ada) │
│ │
│ PRINSIP: │
│ SMC = sinyal UTAMA (menentukan entry/SL/TP) │
│ ML = KONFIRMASI (bisa memblokir, tidak bisa inisiasi)│
│ HMM = PENYESUAI (mengatur agresivitas) │
└──────────────────────────────────────────────────────────┘
9. Smart Money Concepts (SMC)
6 Konsep yang Dianalisis
┌──────────────────────────────────────────────────────────┐
│ 1. SWING POINTS (Fractal High/Low) │
│ │
│ Swing High: titik tertinggi dalam window 11 bar │
│ ↑ │
│ ____/\____ ← 5 bar kiri lebih rendah │
│ \____ ← 5 bar kanan lebih rendah │
│ │
│ Swing Low: titik terendah dalam window 11 bar │
│ ____ ____ ← 5 bar kiri lebih tinggi │
│ \____/ ← 5 bar kanan lebih tinggi │
│ ↑ │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ 2. FAIR VALUE GAP (FVG) — Ketidakseimbangan Harga │
│ │
│ Bullish FVG (gap up): │
│ Bar[i-2].high < Bar[i].low (ada gap) │
│ │
│ │ │ │
│ │ │ ← gap (FVG zone) │
│ │ ├────┐ │
│ ├───┘ │ │
│ │ │ │
│ │
│ Harga cenderung kembali mengisi FVG → entry zone │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ 3. ORDER BLOCK (OB) — Zona Institusi │
│ │
│ Bullish OB: candle bearish terakhir sebelum rally │
│ (zona dimana institusi menempatkan buy order besar) │
│ │
│ /───\ │
│ / \ │
│ ────\ / │
│ OB \_/ ← entry zone │
│ │
│ Lookback: 10 bar untuk deteksi │
│ Mitigated: true jika harga sudah revisit │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ 4. BREAK OF STRUCTURE (BOS) — Kelanjutan Trend │
│ │
│ Uptrend BOS: │
│ Harga break di ATAS swing high sebelumnya │
│ → Trend bullish berlanjut │
│ │
│ /\ /\ │
│ / \ / \ /\ ← BOS (break above prev high) │
│ / \/ \/ │
│ / │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ 5. CHANGE OF CHARACTER (CHoCH) — Perubahan Trend │
│ │
│ Uptrend → Downtrend: │
│ Harga break di BAWAH swing low terakhir │
│ → Trend berubah dari bullish ke bearish │
│ │
│ /\ /\ │
│ / \ / \ │
│ / \/ \ │
│ \____ ← CHoCH (break below prev low) │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ 6. LIQUIDITY ZONES — Target Likuiditas │
│ │
│ BSL (Buy Side Liquidity): di atas swing highs │
│ SSL (Sell Side Liquidity): di bawah swing lows │
│ │
│ ---- BSL ---- (stop loss para seller berkumpul) │
│ /\ /\ │
│ / \ / \ │
│ / \/ \ │
│ ---- SSL ---- (stop loss para buyer berkumpul) │
│ │
│ Institusi sering "hunt" liquidity zone ini │
└──────────────────────────────────────────────────────────┘
Signal Generation
SYARAT SINYAL SMC:
Structure Break (BOS atau CHoCH)
+
Zone (FVG atau Order Block)
=
VALID SIGNAL
BUY Signal:
├── BOS bullish ATAU CHoCH bearish→bullish
├── + Bullish FVG ATAU Bullish OB di bawah harga
├── Entry: harga saat ini
├── SL: di bawah zone, minimum 1.5 × ATR
├── TP: 2:1 R:R, maximum 4 × ATR
└── Confidence: 40-85% (v5: calibrated weighted scoring) (lebih banyak confluence = lebih tinggi)
SELL Signal:
├── BOS bearish ATAU CHoCH bullish→bearish
├── + Bearish FVG ATAU Bearish OB di atas harga
├── Entry: harga saat ini
├── SL: di atas zone, minimum 1.5 × ATR
├── TP: 2:1 R:R, maximum 4 × ATR
└── Confidence: 40-85% (v5: calibrated weighted scoring)
10. Position Lifecycle
Dari Lahir Sampai Mati (Siklus Hidup Posisi)
╔═══════════════════════════════════════════════════════════════╗
║ TAHAP 1: SINYAL TERDETEKSI ║
║ ║
║ SMC menemukan setup + ML konfirmasi + 11 filter PASS ║
║ → Keputusan: BUKA POSISI ║
╚═══════════════════════════════════════════════════════════════╝
│
▼
╔═══════════════════════════════════════════════════════════════╗
║ TAHAP 2: LOT SIZE CALCULATION ║
║ ║
║ Risk Engine (Kelly): ║
║ Balance $5000 × 1% risk = $50 max loss ║
║ SL distance 50 pips → lot = 0.01 ║
║ ║
║ ML Confidence boost: ║
║ ML ≥ 80% → 0.02 lot (double) ║
║ ML < 65% → 0.01 lot (minimum) ║
║ ║
║ Session multiplier: ║
║ Golden: × 1.2, Sydney: × 0.5 ║
║ ║
║ Regime multiplier: ║
║ Normal: × 1.0, High Vol: × 0.5, Crisis: × 0.0 ║
╚═══════════════════════════════════════════════════════════════╝
│
▼
╔═══════════════════════════════════════════════════════════════╗
║ TAHAP 3: ORDER VALIDATION ║
║ ║
║ Risk Engine memvalidasi: ║
║ ✓ SL di sisi yang benar (BUY: SL < entry) ║
║ ✓ TP di sisi yang benar (BUY: TP > entry) ║
║ ✓ Lot dalam range (0.01 - 0.05) ║
║ ✓ Entry dekat harga saat ini (< 0.1%) ║
║ ✓ Risk% ≤ 1.5× config limit ║
║ ✓ Circuit breaker TIDAK aktif ║
╚═══════════════════════════════════════════════════════════════╝
│
▼
╔═══════════════════════════════════════════════════════════════╗
║ TAHAP 4: ORDER EXECUTION ║
║ ║
║ MT5 Connector mengirim order: ║
║ → Symbol: XAUUSD ║
║ → Type: BUY/SELL ║
║ → Lot: 0.01-0.02 ║
║ → SL: ATR-based (broker level) ║
║ → TP: 2:1 R:R (broker level) ║
║ → Deviation: 20 points (slippage tolerance) ║
║ → Retry: max 3 attempts jika gagal ║
╚═══════════════════════════════════════════════════════════════╝
│
▼
╔═══════════════════════════════════════════════════════════════╗
║ TAHAP 4b: POST-EXECUTION VALIDATION (v5 BARU) ║
║ ║
║ Slippage Validation: ║
║ → Bandingkan harga aktual vs expected ║
║ → Max acceptable: 0.15% dari harga (~$4 untuk XAUUSD) ║
║ → Log WARNING jika melebihi batas ║
║ → Gunakan harga AKTUAL untuk position tracking ║
║ ║
║ Partial Fill Handling: ║
║ → Cek apakah volume terisi = volume diminta ║
║ → Jika partial: log fill ratio, update lot_size aktual ║
║ → Risk calculation tetap akurat dengan volume sebenarnya ║
╚═══════════════════════════════════════════════════════════════╝
│
▼
╔═══════════════════════════════════════════════════════════════╗
║ TAHAP 5: POSITION REGISTERED ║
║ ║
║ Smart Risk Manager: ║
║ → Catat entry price AKTUAL, direction, lot AKTUAL, timestamp║
║ → v5: Menggunakan harga & volume dari broker (bukan planned)║
║ → Inisialisasi peak_profit = 0 ║
║ → Mulai tracking momentum ║
║ ║
║ Trade Logger: ║
║ → Insert ke PostgreSQL (30+ field) ║
║ → Backup ke CSV ║
║ ║
║ Telegram: ║
║ → Kirim notifikasi trade open ║
║ → Detail: entry, SL, TP, R:R, confidence, regime ║
╚═══════════════════════════════════════════════════════════════╝
│
▼
╔═══════════════════════════════════════════════════════════════╗
║ TAHAP 6: ACTIVE MONITORING (setiap ~10 detik + candle baru) ║
║ ║
║ ┌── Update profit/loss real-time ║
║ ├── Update peak profit (tertinggi yang pernah dicapai) ║
║ ├── Hitung momentum (kecepatan perubahan profit) ║
║ ├── Hitung TP probability ║
║ ├── Cek 10 kondisi exit (lihat bagian 6) ║
║ ├── Cek Position Manager (trailing, breakeven) ║
║ └── Cek Market Close Handler (dekat close?) ║
║ ║
║ Setiap ~10 detik (atau candle baru), posisi dievaluasi: ║
║ → HOLD (lanjut) ║
║ → CLOSE (tutup dengan alasan spesifik) ║
╚═══════════════════════════════════════════════════════════════╝
│
│ trigger close
▼
╔═══════════════════════════════════════════════════════════════╗
║ TAHAP 7: POSITION CLOSED ║
║ ║
║ MT5 Connector: ║
║ → Close position via market order ║
║ ║
║ Smart Risk Manager: ║
║ → Record profit/loss ║
║ → Update daily/total loss counters ║
║ → Update win/loss streak ║
║ → Check mode transition (NORMAL→RECOVERY→PROTECTED→STOPPED)║
║ ║
║ Trade Logger: ║
║ → Update trade record: exit price, profit, duration, reason ║
║ → Update PostgreSQL + CSV ║
║ ║
║ Telegram: ║
║ → Kirim notifikasi trade close ║
║ → Detail: profit, duration, exit reason, balance ║
╚═══════════════════════════════════════════════════════════════╝
11. Auto-Retraining & Model Management
Lifecycle Model AI
┌──────────────────────────────────────────────────────────────┐
│ INITIAL TRAINING (train_models.py) │
│ Dijalankan 1x saat setup │
│ │
│ 1. Fetch 10,000 bar M15 dari MT5 (~104 hari) │
│ 2. Feature Engineering → 40+ fitur │
│ 3. SMC Analysis → struktur pasar │
│ 4. Create target → UP/DOWN (lookahead=1) │
│ 4b. Split 70/30 dengan 50-bar gap (anti temporal leakage) │
│ 5. Train HMM (3 regime, lookback=500) │
│ 6. Train XGBoost (50 rounds, early_stop=5) │
│ 7. Walk-forward validation (500 train/50 test/50 step) │
│ 8. Save → models/hmm_regime.pkl + xgboost_model.pkl │
└──────────────────────────────┬───────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────┐
│ DAILY AUTO-RETRAINING (Auto Trainer) │
│ Otomatis setiap hari 05:00 WIB │
│ │
│ Schedule: │
│ ├── Harian (05:00 WIB): 8,000 bar, 50 rounds │
│ ├── Weekend (05:00 Sabtu): 15,000 bar, 80 rounds (deep) │
│ └── Emergency: jika AUC < 0.65 (kualitas turun) │
│ │
│ Proses: │
│ 1. Backup model saat ini → models/backup/ │
│ 2. Fetch data baru dari MT5 │
│ 3. Feature Engineering + SMC │
│ 4. Train HMM baru + XGBoost baru │
│ 5. Validasi: test AUC ≥ 0.60? (v4: dinaikkan dari 0.52) │
│ ├── Ya → Save model baru, reload di memory │
│ └── Tidak → ROLLBACK ke model sebelumnya │
│ 6. Log hasil ke PostgreSQL │
│ 7. Kirim laporan via Telegram │
│ │
│ Safety: │
│ ├── Max 5 backup disimpan (rotasi) │
│ ├── Min 20 jam antar retrain (cooldown) │
│ ├── Auto-rollback jika AUC < 0.60 (v4 threshold) │
│ └── Model lama selalu tersedia untuk rollback │
└──────────────────────────────────────────────────────────────┘
Perbandingan Initial vs Auto Training
| Aspek | train_models.py | Auto Trainer |
|---|---|---|
| Kapan | Manual, 1x setup | Otomatis, harian |
| Data | 10,000 bar | 8K (harian) / 15K (weekend) |
| Boost rounds | 50 | 50 (harian) / 80 (weekend) |
| Walk-forward | Ya | Tidak |
| Backup | Tidak | Ya (5 terakhir) |
| Rollback | Tidak | Ya (AUC < 0.60) |
| Database | Tidak | Ya (PostgreSQL) |
| Tujuan | Setup awal | Maintenance rutin |
12. Infrastruktur & Database
PostgreSQL Schema
trading_db
├── trades (Semua trade: open, close, profit, SMC, ML, features)
├── training_runs (Log setiap training: AUC, akurasi, durasi, rollback)
├── signals (Setiap sinyal yang dihasilkan: executed atau tidak)
├── market_snapshots (Snapshot periodik: harga, regime, volatilitas)
├── bot_status (Status bot: uptime, loop count, balance, risk mode)
└── daily_summaries (Ringkasan harian: win rate, profit factor, per sesi)
Tabel trades (Detail)
-- Identifikasi
ticket, symbol, direction (BUY/SELL)
-- Harga
entry_price, exit_price, stop_loss, take_profit
-- Hasil
lot_size, profit_usd, profit_pips
opened_at, closed_at, duration_seconds
-- Konteks Entry
entry_regime, entry_volatility, entry_session
smc_signal, smc_confidence, smc_reason
smc_fvg_detected, smc_ob_detected, smc_bos_detected, smc_choch_detected
ml_signal, ml_confidence
market_quality, market_score, dynamic_threshold
-- Konteks Exit
exit_reason, exit_regime, exit_ml_signal
-- Keuangan
balance_before, balance_after, equity_at_entry
-- Data Lengkap
features_entry (JSON), features_exit (JSON)
bot_version, trade_mode
Connection Architecture
Bot Components
├── TradeLogger → TradeRepository, SignalRepository, MarketSnapshotRepository
├── AutoTrainer → TrainingRepository
├── main_live.py → BotStatusRepository, DailySummaryRepository
└── Dashboard → Semua repository (READ)
│
▼
DatabaseConnection (Singleton)
│
▼
ThreadedConnectionPool (1-10 koneksi)
│
▼
PostgreSQL Server
Graceful Degradation
PostgreSQL tersedia?
├── Ya → Gunakan DB + CSV backup (dual write)
└── Tidak → CSV saja (bot tetap berjalan 100%)
Bot TIDAK PERNAH crash karena database.
Semua operasi DB dibungkus try-except.
13. Konfigurasi & Parameter Kritis
Configuration System
.env file
│
▼
TradingConfig.from_env()
│
├── RiskConfig
│ ├── risk_per_trade: 1.0% (SMALL) / 0.5% (MEDIUM)
│ ├── max_daily_loss: 3.0% (SMALL) / 2.0% (MEDIUM)
│ ├── max_total_loss: 10.0%
│ ├── max_positions: 3 (SMALL) / 5 (MEDIUM)
│ ├── min_lot: 0.01
│ ├── max_lot: 0.05 (SMALL) / 2.0 (MEDIUM)
│ └── max_leverage: 1:100 (SMALL) / 1:30 (MEDIUM)
│
├── SMCConfig
│ ├── swing_length: 5
│ ├── fvg_min_gap_pips: 2.0
│ ├── ob_lookback: 10
│ └── bos_close_break: true
│
├── MLConfig
│ ├── confidence_threshold: 0.65
│ ├── entry_confidence: 0.70
│ ├── high_confidence: 0.75
│ ├── very_high_confidence: 0.80
│ └── retrain_frequency_days: 7
│
├── ThresholdsConfig
│ ├── ml_min_confidence: 0.65
│ ├── ml_high_confidence: 0.75
│ ├── trade_cooldown_seconds: 300
│ ├── min_profit_to_secure: $15
│ ├── good_profit: $25
│ ├── great_profit: $40
│ ├── flash_crash_threshold: 2.5%
│ └── sydney_lot_multiplier: 0.5
│
└── RegimeConfig
├── n_regimes: 3
├── lookback: 500
└── retrain_frequency: 20
Capital Mode Auto-Detection
Balance ≤ $10,000 → SMALL MODE
├── Risk: 1% per trade ($50 pada $5K)
├── Daily limit: 3% ($150)
├── Lot: 0.01-0.05
├── Leverage: 1:100
├── Timeframe: M15
└── Max posisi: 3
Balance > $10,000 → MEDIUM MODE
├── Risk: 0.5% per trade
├── Daily limit: 2%
├── Lot: 0.01-2.0
├── Leverage: 1:30
├── Timeframe: H1
└── Max posisi: 5
Session Schedule (WIB = GMT+7)
┌────────────────────────────────────────────────────────────┐
│ WAKTU (WIB) │ SESI │ LOT MULT │ KETERANGAN │
├──────────────┼───────────────┼──────────┼─────────────────┤
│ 00:00-04:00 │ DEAD ZONE │ BLOCKED │ Likuiditas rendah│
│ 04:00-06:00 │ ROLLOVER │ BLOCKED │ Spread melebar │
│ 06:00-07:00 │ Sydney │ 0.5x │ Pasar baru buka │
│ 07:00-13:00 │ Tokyo+Sydney │ 0.7x │ Asia aktif │
│ 13:00-15:00 │ Tokyo akhir │ 0.7x │ Transisi │
│ 15:00-20:00 │ London │ 1.0x │ Volatilitas naik │
│ 20:00-23:59 │ ★ GOLDEN TIME │ 1.2x │ London+NY overlap│
│ Jumat ≥23:00 │ WEEKEND RISK │ BLOCKED │ Gap risk │
└────────────────────────────────────────────────────────────┘
★ Golden Time (20:00-23:59 WIB) = waktu paling optimal
→ Spread ketat, likuiditas maksimal, volatilitas ideal
→ Lot multiplier 1.2x (bonus)
→ v4: Smart Hold dihapus — tidak ada lagi hold losers menunggu sesi tertentu
14. Performa & Timing
Main Loop Breakdown
Target: < 50ms per iterasi analisis
FULL ANALYSIS (saat candle baru M15):
┌────────────────────────────────────────────────────────────┐
│ Komponen │ Waktu │ Keterangan │
├────────────────────────┼──────────┼────────────────────────┤
│ MT5 data fetch │ ~10ms │ 200 bar M15 via API │
│ Feature engineering │ ~5ms │ 40+ fitur, Polars │
│ SMC analysis │ ~5ms │ 6 konsep, Polars native│
│ HMM predict │ ~2ms │ 2 fitur → 1 regime │
│ XGBoost predict │ ~3ms │ 24 fitur → 1 signal │
│ Position monitoring │ ~5ms │ Per posisi terbuka │
│ Entry logic │ ~5ms │ 11 filter check │
│ Overhead │ ~15ms │ Logging, state update │
├────────────────────────┼──────────┼────────────────────────┤
│ TOTAL │ ~50ms │ │
└────────────────────────────────────────────────────────────┘
POSITION CHECK ONLY (di antara candle, setiap ~10 detik):
┌────────────────────────────────────────────────────────────┐
│ Komponen │ Waktu │ Keterangan │
├────────────────────────┼──────────┼────────────────────────┤
│ MT5 data fetch │ ~5ms │ 50 bar saja │
│ Feature engineering │ ~3ms │ Minimal fitur │
│ ML prediction │ ~3ms │ Untuk exit evaluation │
│ Position evaluation │ ~5ms │ 10 kondisi exit │
│ Overhead │ ~5ms │ Logging │
├────────────────────────┼──────────┼────────────────────────┤
│ TOTAL │ ~21ms │ │
└────────────────────────────────────────────────────────────┘
Timer Periodik
┌──────────────────────────────────────────────────────────┐
│ Event │ Interval │ Cara Trigger │
├────────────────────────┼────────────────┼─────────────────┤
│ Full analysis + entry │ Candle baru M15│ Deteksi candle │
│ Position monitoring │ ~10 detik │ Di antara candle│
│ Performance logging │ 4 candle (~1j) │ candle_count % 4│
│ Auto-retrain check │ 20 candle (~5j)│ candle_count %20│
│ Market update Telegram │ 30 menit │ Timer │
│ Hourly analysis │ 1 jam │ Timer │
│ Daily summary + reset │ Ganti hari │ Date check │
└──────────────────────────────────────────────────────────┘
15. Error Handling & Fault Tolerance
Prinsip: Bot TIDAK PERNAH Crash
┌──────────────────────────────────────────────────────────┐
│ LEVEL 1: Per-Loop Error Handling │
│ │
│ try: │
│ # Fetch data, analyze, trade │
│ except ConnectionError: │
│ # MT5 disconnected → reconnect() │
│ except Exception as e: │
│ # Log error → lanjut loop berikutnya │
│ # Bot TIDAK crash dari error tunggal │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ LEVEL 2: MT5 Auto-Reconnect │
│ │
│ MT5 putus? │
│ ├── Attempt 1: reconnect (tunggu 2 detik) │
│ ├── Attempt 2: reconnect (tunggu 4 detik) │
│ ├── Attempt 3: reconnect (tunggu 8 detik) │
│ ├── Cooldown 60 detik │
│ └── Retry cycle (max 5 per cooldown) │
│ │
│ Selama disconnected: │
│ → Position monitoring PAUSE │
│ → Entry baru DITUNDA │
│ → Posisi terbuka dilindungi broker SL (lapis 1) │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ LEVEL 3: Database Graceful Degradation │
│ │
│ PostgreSQL down? │
│ ├── Switch ke CSV-only mode │
│ ├── Semua data tetap dicatat │
│ ├── Trading tetap berjalan normal │
│ └── Retry DB connection periodik │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ LEVEL 4: Telegram Failure │
│ │
│ Telegram API error? │
│ ├── Log error secara silent │
│ ├── Trading tetap jalan 100% │
│ └── Retry di notifikasi berikutnya │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ LEVEL 5: Model File Missing │
│ │
│ .pkl file tidak ditemukan? │
│ ├── Log warning │
│ ├── Skip prediksi (ML/HMM) │
│ ├── Trading bisa jalan tanpa ML (SMC only) │
│ └── Trigger: jalankan train_models.py │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ LEVEL 6: Flash Crash Protection │
│ │
│ Harga bergerak > 2.5% dalam 1 menit? │
│ ├── EMERGENCY: Close ALL positions │
│ ├── Circuit breaker AKTIF │
│ ├── Kirim alert KRITIS via Telegram │
│ └── Bot masuk mode STOPPED │
└──────────────────────────────────────────────────────────┘
Startup & Shutdown
STARTUP SEQUENCE:
1. Load konfigurasi dari .env
2. Connect ke MT5 (max 3 retry)
3. Load model HMM dari models/hmm_regime.pkl
4. Load model XGBoost dari models/xgboost_model.pkl
5. Initialize SmartRiskManager (set balance, limits)
6. Initialize SessionFilter (WIB timezone)
7. Initialize TelegramNotifier
8. Initialize TradeLogger (connect DB)
9. Initialize AutoTrainer
10. Send Telegram: "BOT STARTED" (config, balance, risk settings)
11. Mulai main loop
SHUTDOWN SEQUENCE (SIGINT/SIGTERM):
1. Signal diterima
2. Hentikan loop utama
3. Kirim Telegram: "BOT STOPPED" (balance, trades, uptime)
4. Disconnect MT5
5. Close database connections
6. Exit
16. Daftar File Source Code
Smart Automatic Trading BOT + AI/
│
├── main_live.py # Orchestrator utama (TradingBot)
├── train_models.py # Script training awal
├── .env # Environment variables (credentials)
│
├── src/
│ ├── config.py # Konfigurasi terpusat (6 sub-config)
│ ├── mt5_connector.py # Bridge ke MetaTrader 5
│ ├── feature_eng.py # Feature Engineering (40+ fitur)
│ ├── regime_detector.py # HMM Regime Detection (3 state)
│ ├── ml_model.py # XGBoost Signal Predictor
│ ├── smc_polars.py # Smart Money Concepts (6 konsep)
│ ├── smart_risk_manager.py # 4-Mode Risk Manager
│ ├── risk_engine.py # Kelly Criterion + Circuit Breaker
│ ├── session_filter.py # Session Time Filter (WIB)
│ ├── dynamic_confidence.py # Dynamic Threshold Manager
│ ├── news_agent.py # News Event Monitor
│ ├── telegram_notifier.py # Telegram Push Notifications
│ ├── auto_trainer.py # Daily Auto-Retraining
│ ├── trade_logger.py # Dual Storage Logger (DB+CSV)
│ ├── position_manager.py # Position Manager + Market Close
│ │
│ └── db/
│ ├── __init__.py # DB exports
│ ├── connection.py # PostgreSQL Singleton + Pool
│ └── repository.py # 6 Repository classes
│
├── models/
│ ├── xgboost_model.pkl # Trained XGBoost model
│ ├── hmm_regime.pkl # Trained HMM model
│ └── backup/ # Auto-backup (5 terakhir)
│
├── data/
│ ├── training_data.parquet # Data training terakhir
│ └── trade_logs/ # CSV backup (per bulan)
│ ├── trades_2025_01.csv
│ ├── trades_2025_02.csv
│ └── ...
│
├── backtests/
│ └── backtest_live_sync.py # Backtest 100% sync live
│
├── logs/
│ └── training_YYYY-MM-DD.log # Log training detail
│
└── docs/
└── arsitektur-ai/
├── 00-ARSITEKTUR-LENGKAP.md # Dokumen ini
├── README.md # Index komponen
└── 01-23 (per komponen) # Detail per modul
Ringkasan Eksekutif
Smart AI Trading Bot adalah sistem trading otomatis yang menggabungkan:
-
Smart Money Concepts (SMC) sebagai sinyal UTAMA — mendeteksi zona institusi (FVG, Order Block, BOS, CHoCH) untuk menentukan entry, SL, dan TP yang presisi.
-
XGBoost Machine Learning sebagai KONFIRMASI — memprediksi arah harga dengan 24 fitur teknikal, memblokir trade jika tidak setuju dengan SMC.
-
Hidden Markov Model (HMM) sebagai PENYESUAI — mendeteksi kondisi pasar (tenang/volatile/krisis) untuk menyesuaikan agresivitas.
-
4-Lapis Proteksi Risiko — dari broker SL, software smart exit, emergency stop, hingga circuit breaker. Lot ultra-kecil (0.01-0.02) memastikan kerugian per trade maximum $50 (1%).
-
Self-Improving — model AI dilatih ulang otomatis setiap hari dengan auto-rollback jika model baru lebih buruk.
-
Fault-Tolerant — bot tidak pernah crash. MT5 putus? Auto-reconnect. Database mati? CSV fallback. Error? Log dan lanjut.
Semua ini dikoordinasikan oleh Main Live Orchestrator yang menjalankan loop candle-based — analisis penuh hanya saat candle M15 baru terbentuk (~50ms per iterasi), dengan pengecekan posisi setiap ~10 detik di antara candle (~21ms). Mengevaluasi 11 filter entry dan 10 kondisi exit secara real-time, dengan notifikasi Telegram untuk setiap kejadian penting.
TARGET: Trading XAUUSD M15 yang KONSISTEN dan AMAN
dengan kerugian terkontrol dan profit teroptimasi.