# Arsitektur Lengkap — Smart AI Trading Bot > **Dokumen:** Arsitektur keseluruhan sistem dalam 1 file > **Instrumen:** XAUUSD (Gold) M15 > **Platform:** MetaTrader 5 > **Bahasa:** Python 3.11+ (async, Polars, XGBoost, HMM) > **Database:** PostgreSQL + CSV fallback > **Notifikasi:** Telegram Bot API --- ## Daftar Isi 1. [Gambaran Umum](#1-gambaran-umum) 2. [Diagram Arsitektur](#2-diagram-arsitektur) 3. [23 Komponen](#3-23-komponen) 4. [Pipeline Data: Dari OHLCV ke Keputusan Trading](#4-pipeline-data) 5. [Alur Entry: 11 Filter](#5-alur-entry-11-filter) 6. [Alur Exit: 10 Kondisi](#6-alur-exit-10-kondisi) 7. [Sistem Proteksi Risiko 4 Lapis](#7-sistem-proteksi-risiko-4-lapis) 8. [AI/ML Engine](#8-aiml-engine) 9. [Smart Money Concepts (SMC)](#9-smart-money-concepts) 10. [Position Lifecycle](#10-position-lifecycle) 11. [Auto-Retraining & Model Management](#11-auto-retraining) 12. [Infrastruktur & Database](#12-infrastruktur--database) 13. [Konfigurasi & Parameter Kritis](#13-konfigurasi--parameter-kritis) 14. [Performa & Timing](#14-performa--timing) 15. [Error Handling & Fault Tolerance](#15-error-handling--fault-tolerance) 16. [Daftar File Source Code](#16-daftar-file-source-code) --- ## 1. Gambaran Umum ### Apa Ini? Bot trading otomatis yang menggabungkan **3 otak kecerdasan buatan** untuk trading XAUUSD (Emas) di MetaTrader 5: ``` OTAK 1: Smart Money Concepts (SMC) → Membaca pola institusi besar (bank, hedge fund) → Menentukan DIMANA entry, SL, dan TP OTAK 2: XGBoost Machine Learning → Memprediksi ARAH harga (naik/turun) → Memberikan tingkat keyakinan (confidence) OTAK 3: Hidden Markov Model (HMM) → Membaca KONDISI pasar (tenang/volatile/krisis) → Menyesuaikan ukuran posisi dan agresivitas ``` ### Filosofi Desain ``` 1. KESELAMATAN MODAL NOMOR 1 → 4 lapis proteksi stop loss → Lot ultra-kecil (0.01-0.02) → Circuit breaker otomatis 2. TIDAK PERNAH CRASH → Setiap error di-catch, bot terus jalan → Database gagal? CSV fallback → MT5 putus? Auto-reconnect 3. SELF-IMPROVING → Model AI dilatih ulang otomatis setiap hari → Rollback otomatis jika model baru lebih buruk → Threshold confidence menyesuaikan kondisi pasar 4. TRANSPARAN → Setiap keputusan dicatat ke database → Notifikasi Telegram real-time → Laporan harian, jam-an, dan per-trade ``` ### Angka-Angka Kunci | Parameter | Nilai | Penjelasan | |-----------|-------|------------| | Modal target | $5,000 | Small account mode | | Risiko per trade | 1% ($50) | Maksimum kerugian per posisi | | Lot size | 0.01 - 0.02 | Ultra-konservatif | | Max daily loss | 3% ($150) | Circuit breaker harian | | Max total loss | 10% ($500) | Stop total trading | | Max posisi bersamaan | 2-3 | Menghindari overexposure | | Cooldown antar trade | 5 menit | Mencegah overtrading | | Loop speed | ~50ms | Cepat tapi efisien | | Timeframe | M15 | 15 menit per candle | --- ## 2. Diagram Arsitektur ### Diagram Keseluruhan Sistem ``` ┌─────────────────────────────────────────────────────────────────────────┐ │ MAIN LIVE (Orchestrator) │ │ main_live.py — TradingBot │ │ Candle-based (M15) + position check ~10 detik │ │ │ │ ┌─────────────── PHASE 1: DATA ──────────────────────────────────┐ │ │ │ │ │ │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │ │ │ │ MT5 │ │ Feature │ │ SMC │ │ HMM │ │ │ │ │ │Connector │ ──→│ Engine │ ──→│ Analyzer │ │ Regime │ │ │ │ │ │(broker) │ │(40+ fitur│ │(institusi│ │(3 state) │ │ │ │ │ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │ │ │ │ │ │ │ │ │ │ │ │ │ │ ▼ ▼ │ │ │ │ │ │ ┌──────────┐ ┌──────────┐ │ │ │ │ │ └────────→│ XGBoost │ │ Dynamic │ │ │ │ │ │ │ ML Model │ │Confidence│ │ │ │ │ │ │(prediksi)│ │(threshold│ │ │ │ │ │ └──────────┘ └──────────┘ │ │ │ └───────┼─────────────────────────────┼───────────────┼────────┘ │ │ │ │ │ │ │ ┌───────┼──── PHASE 2: MONITORING ────┼───────────────┼────────┐ │ │ │ │ │ │ │ │ │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │ │ │ │ Position │ │ Smart │ │ Risk │ │ │ │ │ │ Manager │ │ Risk │ │ Engine │ │ │ │ │ │(trailing)│ │ Manager │ │ (Kelly) │ │ │ │ │ └──────────┘ └──────────┘ └──────────┘ │ │ │ └────────────────────────────────────────────────────────────┘ │ │ │ │ │ │ │ ┌───────┼──── PHASE 3: ENTRY ─────────┼───────────────┼────────┐ │ │ │ │ │ │ │ │ │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │ │ │ │ Session │ │ News │ │ 11-Gate │ │ │ │ │ │ Filter │ │ Agent │ │ Entry │ │ │ │ │ │(waktu) │ │(berita) │ │ Filter │──→ EXECUTE │ │ │ │ └──────────┘ └──────────┘ └──────────┘ │ │ │ └────────────────────────────────────────────────────────────┘ │ │ │ │ │ ┌───────┼──── PHASE 4: PERIODIK ──────────────────────────────┐ │ │ │ │ │ │ │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │ │ │ │ Auto │ │ Telegram │ │ Trade │ │ │ │ │ │ Trainer │ │ Notifier │ │ Logger │ │ │ │ │ │(retrain) │ │(laporan) │ │(DB+CSV) │ │ │ │ │ └──────────┘ └──────────┘ └──────────┘ │ │ │ └─────────────────────────────────────────────────────────────┘ │ │ │ │ │ ┌────────┴────────┐ │ │ │ PostgreSQL │ │ │ │ + CSV Backup │ │ │ └─────────────────┘ │ └─────────────────────────────────────────────────────────────────────────┘ ``` ### Alur Data (Data Flow) ``` MT5 Broker (XAUUSD M15) │ │ 200 bar OHLCV ▼ ┌─────────────────┐ │ MT5 Connector │ numpy → Polars (tanpa Pandas) └────────┬────────┘ │ ▼ ┌─────────────────┐ │ Feature Engineer │ OHLCV → 40+ fitur teknikal │ │ RSI, ATR, MACD, BB, EMA, Volume, │ │ Returns, Volatility, Lags, Trend └────────┬────────┘ │ ┌────┴────┐ │ │ ▼ ▼ ┌───────┐ ┌───────┐ │ SMC │ │ HMM │ │Analyzer│ │Regime │ │ │ │Detect │ └───┬───┘ └───┬───┘ │ │ │ ┌────┘ │ │ ▼ ▼ ┌─────────────────┐ │ XGBoost │ 24 fitur → Prediksi BUY/SELL/HOLD │ ML Predictor │ + Confidence 0-100% └────────┬────────┘ │ ▼ ┌─────────────────┐ │ Dynamic │ Sesuaikan threshold berdasarkan │ Confidence │ sesi, regime, volatilitas, trend └────────┬────────┘ │ ▼ ┌─────────────────┐ │ Signal Combiner │ SMC + ML harus setuju │ (11 Filter) │ + Session + Risk + Cooldown └────────┬────────┘ │ ┌────┴────┐ │ PASS? │ │ │ YES NO → tunggu loop berikutnya │ ▼ ┌─────────────────┐ │ Risk Engine │ Kelly Criterion → lot size │ + Risk Manager │ Validasi order → approve/reject └────────┬────────┘ │ ▼ ┌─────────────────┐ │ Execute Order │ Kirim ke MT5 dengan SL & TP │ via MT5 │ Register ke Position Manager └────────┬────────┘ │ ┌────┴────────────────┐ │ │ ▼ ▼ ┌───────────┐ ┌────────────────┐ │ Telegram │ │ Trade Logger │ │ Notifier │ │ (DB + CSV) │ └───────────┘ └────────────────┘ ``` --- ## 3. 23 Komponen ### Tabel Komponen Lengkap | # | Komponen | File | Kategori | Fungsi Utama | |---|----------|------|----------|-------------| | 1 | HMM Regime Detector | `src/regime_detector.py` | AI/ML | Deteksi kondisi pasar (3 regime) | | 2 | XGBoost Predictor | `src/ml_model.py` | AI/ML | Prediksi arah harga + confidence | | 3 | SMC Analyzer | `src/smc_polars.py` | Analisis | Pola institusi: FVG, OB, BOS, CHoCH | | 4 | Feature Engineering | `src/feature_eng.py` | Data | OHLCV → 40+ fitur numerik | | 5 | Smart Risk Manager | `src/smart_risk_manager.py` | Risiko | 4 mode trading, 10 kondisi exit | | 6 | Session Filter | `src/session_filter.py` | Filter | Waktu trading optimal (WIB) | | 7 | Stop Loss (4 Lapis) | Multi-file | Proteksi | SMC → Software → Emergency → Circuit | | 8 | Take Profit (6 Layer) | Multi-file | Proteksi | Hard → Momentum → Peak → Probability → Early → Broker | | 9 | Entry Trade | `main_live.py` | Eksekusi | 11 filter berurutan | | 10 | Exit Trade | `main_live.py` | Eksekusi | 10 kondisi exit real-time | | 11 | News Agent | `src/news_agent.py` | Monitor | Monitoring berita (TIDAK memblokir) | | 12 | Telegram Notifier | `src/telegram_notifier.py` | Notifikasi | 11 tipe notifikasi real-time | | 13 | Auto Trainer | `src/auto_trainer.py` | ML Ops | Retraining harian otomatis | | 14 | Backtest | `backtests/backtest_live_sync.py` | Validasi | Simulasi 100% sync dengan live | | 15 | Dynamic Confidence | `src/dynamic_confidence.py` | Adaptif | Threshold ML adaptif (60-85%) | | 16 | MT5 Connector | `src/mt5_connector.py` | Koneksi | Bridge ke broker, auto-reconnect | | 17 | Configuration | `src/config.py` | Config | 6 sub-config, auto-adjust modal | | 18 | Trade Logger | `src/trade_logger.py` | Logging | Dual storage DB + CSV | | 19 | Position Manager | `src/position_manager.py` | Manajemen | Trailing SL, breakeven, market close | | 20 | Risk Engine | `src/risk_engine.py` | Risiko | Kelly Criterion, circuit breaker | | 21 | Database | `src/db/` | Storage | PostgreSQL, 6 repository | | 22 | Train Models | `train_models.py` | Training | Script training awal | | 23 | Main Live | `main_live.py` | Orchestrator | Koordinasi semua komponen | ### Hubungan Antar Komponen ``` ┌─────────────────────────┐ │ CONFIGURATION (17) │ │ Sumber parameter semua │ └────────────┬─────────────┘ │ dikonsumsi oleh semua ▼ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ MT5 (16) │───→│FeatEng(4)│───→│ SMC (3) │ │ Broker │ │ 40+ fitur│ │ Institusi│ └──────────┘ └────┬─────┘ └────┬─────┘ │ │ ▼ ▼ ┌──────────┐ ┌──────────┐ │ HMM (1) │ │XGBoost(2)│ │ Regime │ │ Prediksi │ └────┬─────┘ └────┬─────┘ │ │ ▼ ▼ ┌──────────────────────────┐ │ Dynamic Confidence (15) │ │ Threshold adaptif │ └────────────┬─────────────┘ │ ▼ ┌──────────┐ ┌──────────────────────────┐ ┌──────────┐ │Session(6)│───→│ ENTRY TRADE (9) │←───│ News(11) │ │ Waktu │ │ 11 Filter Gate │ │ Berita │ └──────────┘ └────────────┬─────────────┘ └──────────┘ │ ┌────────┴────────┐ ▼ ▼ ┌──────────┐ ┌──────────────┐ │RiskEng(20│ │SmartRisk (5) │ │Kelly Lot │ │ 4 Mode │ └────┬─────┘ └──────┬───────┘ │ │ ▼ ▼ ┌──────────────────────────┐ │ EXECUTE ORDER │ │ via MT5 (16) │ └────────────┬─────────────┘ │ ┌────────────┼────────────┐ ▼ ▼ ▼ ┌──────────┐ ┌──────────┐ ┌──────────┐ │PosMgr(19)│ │Logger(18)│ │Telegram │ │Trailing │ │ DB+CSV │ │ (12) │ └────┬─────┘ └────┬─────┘ └──────────┘ │ │ ▼ ▼ ┌──────────┐ ┌──────────┐ │EXIT (10) │ │ DB (21) │ │10 Kondisi│ │PostgreSQL│ └──────────┘ └──────────┘ Periodik: ┌──────────┐ ┌──────────┐ ┌──────────┐ │AutoTrain │ │ Backtest │ │ Train │ │ (13) │ │ (14) │ │Models(22)│ │Harian │ │Validasi │ │Setup awal│ └──────────┘ └──────────┘ └──────────┘ ``` --- ## 4. Pipeline Data ### Dari OHLCV Mentah ke Keputusan Trading #### Tahap 1: Data Fetching (MT5 Connector) ``` MT5 Broker │ │ mt5.copy_rates_from_pos("XAUUSD", MT5_TIMEFRAME_M15, 0, 200) │ ▼ NumPy Structured Array │ │ Konversi langsung ke Polars (TANPA Pandas) │ ▼ Polars DataFrame: ┌──────────────────────────────────────────────────────┐ │ time │ open │ high │ low │ close │ tick_volume │ spread │ │ i64 │ f64 │ f64 │ f64 │ f64 │ f64 │ f64 │ ├──────┼──────┼──────┼─────┼───────┼─────────────┼────────┤ │ ... │ 2645 │ 2648 │ 2643│ 2647 │ 5234 │ 25 │ └──────────────────────────────────────────────────────┘ ``` **Kenapa Polars, bukan Pandas?** - 3-5x lebih cepat untuk operasi vectorized - Memory-efficient (zero-copy) - Native lazy evaluation - Konsisten di seluruh codebase (tidak ada konversi bolak-balik) #### Tahap 2: Feature Engineering (40+ Fitur) ``` Input: Polars DataFrame (200 bar OHLCV) │ ├── Momentum Indicators │ ├── RSI(14) → 0-100, overbought/oversold │ ├── MACD(12,26,9) → trend strength & direction │ └── MACD Histogram → momentum acceleration │ ├── Volatility Indicators │ ├── ATR(14) → average true range (pips) │ ├── Bollinger Bands(20,2.0) → upper, lower, width │ └── Volatility(20) → rolling std of returns │ ├── Trend Indicators │ ├── EMA(9) / EMA(21) → fast/slow crossover │ ├── EMA Cross Signal → 1 (bullish) / -1 (bearish) │ └── SMA(20) → simple moving average │ ├── Price Action │ ├── Returns(1,5,20) → % perubahan harga │ ├── Log Returns → untuk distribusi normal │ ├── Price Position → posisi dalam range BB │ └── Higher High/Lower Low count → trend structure │ ├── Volume Features │ ├── Volume SMA(20) → rata-rata volume │ └── Volume Ratio → current / average │ ├── Lag Features │ ├── close_lag_1..5 → harga sebelumnya │ └── returns_lag_1..3 → return sebelumnya │ └── Time Features ├── Hour, Weekday → waktu candle └── Session flags → london, ny, overlap │ ▼ Output: DataFrame + 40 kolom baru (semua numerik, siap ML) ``` **Minimum data:** 26 bar untuk semua indikator stabil #### Tahap 3: SMC Analysis (Pola Institusi) ``` Input: DataFrame dengan OHLCV │ ├── Swing Points (Fractal) │ Window: 11 bar (swing_length=5, ±5 dari tengah) │ Output: swing_high (1/0), swing_low (-1/0), level harga │ ├── Fair Value Gaps (FVG) │ Bullish: bar[i-2].high < bar[i].low (gap up) │ Bearish: bar[i-2].low > bar[i].high (gap down) │ Output: fvg_bull, fvg_bear, fvg_top, fvg_bottom, fvg_mid │ ├── Order Blocks (OB) │ Lookback: 10 bar │ Bullish: candle bearish terakhir sebelum move up besar │ Bearish: candle bullish terakhir sebelum move down besar │ Output: ob (1/-1), ob_top, ob_bottom, ob_mitigated │ ├── Break of Structure (BOS) │ BOS: harga break swing high/low → trend continuation │ Output: bos (1/-1), level yang di-break │ ├── Change of Character (CHoCH) │ CHoCH: harga break berlawanan arah trend → reversal signal │ Output: choch (1/-1), level yang di-break │ └── Liquidity Zones BSL: Buy Side Liquidity (above swing highs) SSL: Sell Side Liquidity (below swing lows) Output: bsl_level, ssl_level │ ▼ Signal Generation: Syarat: Structure break + (FVG ATAU Order Block) │ ├── Entry: harga saat ini ├── SL: ATR-based, minimum 1.5 × ATR dari entry ├── TP: 2:1 Risk-Reward minimum, cap 4 × ATR ├── Confidence: 40-85% (v5: calibrated weighted scoring) (berdasarkan confluence) └── Reason: "BOS + Bullish FVG at 2645.50" ``` #### Tahap 4: Regime Detection (HMM) ``` Input: log_returns + normalized_range (volatilitas) │ │ GaussianHMM(n_components=3, lookback=500) │ ▼ 3 Regime: ┌──────────────────────────────────────────────────┐ │ REGIME 0: Low Volatility │ │ → Pasar tenang, range kecil │ │ → Lot multiplier: 1.0x (normal) │ │ → Rekomendasi: TRADE │ │ │ │ REGIME 1: Medium Volatility │ │ → Pasar aktif, trend jelas │ │ → Lot multiplier: 1.0x (normal) │ │ → Rekomendasi: TRADE │ │ │ │ REGIME 2: High Volatility │ │ → Pasar sangat volatile, berbahaya │ │ → Lot multiplier: 0.5x (setengah) │ │ → Rekomendasi: REDUCE │ │ │ │ CRISIS (detected by FlashCrashDetector): │ │ → Move > 2.5% dalam 1 menit │ │ → Lot multiplier: 0.0x (STOP) │ │ → Rekomendasi: EMERGENCY CLOSE ALL │ └──────────────────────────────────────────────────┘ ``` #### Tahap 5: ML Prediction (XGBoost) ``` Input: 24 fitur terpilih dari Feature Engineering + SMC + Regime │ │ XGBoost Binary Classifier │ Anti-overfitting config: │ max_depth=3, learning_rate=0.05 │ min_child_weight=10, subsample=0.7 │ colsample_bytree=0.6 │ reg_alpha=1.0 (L1), reg_lambda=5.0 (L2) │ ▼ Output: ┌──────────────────────────────────────────┐ │ prob_up: 0.72 (probabilitas naik) │ │ prob_down: 0.28 (probabilitas turun) │ │ │ │ → Signal: BUY (prob_up > 0.50) │ │ → Confidence: 72% │ │ │ │ Threshold Keputusan: │ │ prob > 0.50 → ada sinyal (minimum) │ │ prob > 0.65 → sinyal kuat │ │ prob > 0.75 → sinyal sangat kuat │ │ prob > 0.80 → lot bisa naik ke 0.02 │ └──────────────────────────────────────────┘ ``` #### Tahap 6: Dynamic Confidence (Threshold Adaptif) ``` Scoring (0-100 poin): Base score: 50 │ ├── Session Modifier: │ Golden Time (20:00-23:59 WIB) → +20 │ London (15:00-23:59) → +15 │ New York (20:00-05:00) → +10 │ Tokyo/Sydney → +0 │ Market Closed → -30 │ ├── Regime Modifier: │ Medium Volatility → +15 │ Low Volatility → +5 │ High Volatility → -5 │ Crisis → -25 │ ├── Volatility Modifier: │ Medium (ideal) → +10 │ Low → +0 │ High → -5 │ Extreme → -10 │ ├── Trend Modifier: │ Trending (jelas) → +10 │ Ranging (sideways) → -5 │ ├── SMC Confluence: │ Ada konfluensi → +10 │ Tidak ada → +0 │ └── ML Confidence: ≥ 70% → +5 ≥ 60% → +2 < 60% → +0 │ ▼ Quality Level → Threshold: ┌───────────────────────────────────────────────────┐ │ EXCELLENT (≥80 poin) → Threshold: 60% (longgar) │ │ GOOD (65-79) → Threshold: 65% │ │ MODERATE (50-64) → Threshold: 70% │ │ POOR (35-49) → Threshold: 80% (ketat) │ │ AVOID (<35) → Threshold: 85% (SKIP) │ └───────────────────────────────────────────────────┘ Contoh: Golden Time + Medium Vol + Trending + SMC + ML 72% = 50 + 20 + 15 + 10 + 10 + 10 + 5 = 120 → cap 100 = EXCELLENT → Threshold 60% → ML 72% PASS ✓ ``` --- ## 5. Alur Entry: 11 Filter Setiap sinyal harus melewati **11 gerbang berurutan**. Satu saja gagal = TIDAK trading. ``` SINYAL SMC + ML MASUK │ ▼ ┌─ FILTER 1: Session Filter ──────────────────────────────┐ │ Apakah sekarang jam trading yang dibolehkan? │ │ ✗ 00:00-06:00 WIB (dead zone) → BLOCK │ │ ✗ Jumat ≥ 23:00 WIB (weekend risk) → BLOCK │ │ ✓ London/NY/Golden Time → PASS │ └──────────────────────────────────────────────────────────┘ │ PASS ▼ ┌─ FILTER 2: Risk Mode ───────────────────────────────────┐ │ Apakah mode trading bukan STOPPED? │ │ ✗ STOPPED (daily/total limit hit) → BLOCK │ │ ✓ NORMAL / RECOVERY / PROTECTED → PASS │ └──────────────────────────────────────────────────────────┘ │ PASS ▼ ┌─ FILTER 3: SMC Signal ──────────────────────────────────┐ │ Apakah ada setup SMC yang valid? │ │ ✗ Tidak ada FVG/OB + BOS/CHoCH → BLOCK │ │ ✓ Ada sinyal BUY/SELL dengan SL & TP → PASS │ └──────────────────────────────────────────────────────────┘ │ PASS ▼ ┌─ FILTER 4: ML Confidence ───────────────────────────────┐ │ Apakah ML confidence ≥ dynamic threshold? │ │ ✗ ML confidence < threshold → BLOCK │ │ ✓ ML confidence ≥ threshold → PASS │ │ (threshold 60-85% tergantung market quality) │ └──────────────────────────────────────────────────────────┘ │ PASS ▼ ┌─ FILTER 5: ML Agreement ────────────────────────────────┐ │ Apakah ML TIDAK strongly disagree dengan SMC? │ │ ✗ ML > 65% berlawanan arah SMC → BLOCK (conflict) │ │ ✓ ML setuju atau netral → PASS │ └──────────────────────────────────────────────────────────┘ │ PASS ▼ ┌─ FILTER 6: Market Quality ──────────────────────────────┐ │ Apakah Dynamic Confidence bukan AVOID? │ │ ✗ Quality == AVOID (score < 35) → BLOCK │ │ ✓ EXCELLENT/GOOD/MODERATE/POOR → PASS │ └──────────────────────────────────────────────────────────┘ │ PASS ▼ ┌─ FILTER 7: Signal Confirmation ─────────────────────────┐ │ Apakah sinyal konsisten 2x berturut-turut? │ │ ✗ Sinyal baru muncul 1x → BLOCK (tunggu konfirmasi) │ │ ✓ Sinyal sudah 2x berturut → PASS │ └──────────────────────────────────────────────────────────┘ │ PASS ▼ ┌─ FILTER 8: Pullback Filter (v5: ATR-based) ────────────┐ │ Apakah momentum selaras dengan arah sinyal? │ │ ✗ BUY tapi RSI > 80 (overbought) → BLOCK │ │ ✗ SELL tapi RSI < 20 (oversold) → BLOCK │ │ ✗ MACD histogram berlawanan → BLOCK │ │ ✗ Harga bounce > 15% ATR dalam 3 candle → BLOCK │ │ ✓ Movement < 10% ATR (consolidation) → PASS │ │ ✓ Momentum selaras → PASS │ │ (v5: threshold dinamis berdasarkan ATR, bukan fixed $2) │ └──────────────────────────────────────────────────────────┘ │ PASS ▼ ┌─ FILTER 9: Trade Cooldown ──────────────────────────────┐ │ Apakah sudah ≥ 5 menit sejak trade terakhir? │ │ ✗ < 300 detik sejak trade terakhir → BLOCK │ │ ✓ ≥ 300 detik → PASS │ └──────────────────────────────────────────────────────────┘ │ PASS ▼ ┌─ FILTER 10: Position Limit ─────────────────────────────┐ │ Apakah jumlah posisi terbuka < limit? │ │ ✗ Sudah 2+ posisi terbuka → BLOCK │ │ ✓ < 2 posisi → PASS │ └──────────────────────────────────────────────────────────┘ │ PASS ▼ ┌─ FILTER 11: Lot Size ───────────────────────────────────┐ │ Apakah lot size yang dihitung > 0? │ │ ✗ Lot = 0 (regime crisis / risk terlalu tinggi) → BLOCK│ │ ✓ Lot ≥ 0.01 → PASS │ └──────────────────────────────────────────────────────────┘ │ PASS ▼ ╔═══════════════════════╗ ║ EXECUTE TRADE ║ ║ BUY atau SELL ║ ║ via MT5 Connector ║ ╚═══════════════════════╝ │ ├── Register ke SmartRiskManager ├── Log ke TradeLogger (DB + CSV) └── Kirim notifikasi Telegram ``` --- ## 6. Alur Exit: 10 Kondisi Setiap posisi terbuka dievaluasi **setiap ~10 detik** (di antara candle) atau **setiap candle baru** (full analysis) terhadap 10 kondisi exit: ``` POSISI TERBUKA (dicek setiap ~10 detik) │ │ Update: profit, momentum, peak, durasi │ ▼ ┌─ KONDISI 1: Smart Take Profit ──────────────────────────┐ │ (a) Profit ≥ $40 → TUTUP (hard TP) │ │ (b) Profit ≥ $25 + momentum < -30 → TUTUP │ │ (c) Peak > $30, sekarang < 60% peak → TUTUP │ │ (d) Profit ≥ $20 + TP probability < 25% → TUTUP │ └──────────────────────────────────────────────────────────┘ │ tidak trigger ▼ ┌─ KONDISI 2: Early Exit (Profit Kecil) ──────────────────┐ │ Profit $5-$15 + ML reversal ≥ 65% + momentum < -50 │ │ → TUTUP (amankan profit kecil sebelum hilang) │ └──────────────────────────────────────────────────────────┘ │ tidak trigger ▼ ┌─ KONDISI 3: Early Cut (v4 — Smart Hold DIHAPUS) ────────┐ │ Loss >= 30% max ($15) DAN momentum < -30? │ │ → TUTUP CEPAT (early cut, jangan tunggu recovery) │ │ │ │ v4: "Smart Hold" dihapus — tidak ada lagi hold losers │ │ menunggu golden time atau sesi London. │ └──────────────────────────────────────────────────────────┘ │ tidak trigger ▼ ┌─ KONDISI 4: ML Trend Reversal ──────────────────────────┐ │ ML confidence ≥ 65% BERLAWANAN ARAH posisi │ │ + 3x warning berturut-turut │ │ → TUTUP (AI mendeteksi pembalikan trend) │ └──────────────────────────────────────────────────────────┘ │ tidak trigger ▼ ┌─ KONDISI 5: Maximum Loss ───────────────────────────────┐ │ Loss ≥ 50% dari max_loss_per_trade ($25 dari $50) │ │ → TUTUP (kerugian terlalu besar) │ └──────────────────────────────────────────────────────────┘ │ tidak trigger ▼ ┌─ KONDISI 6: Stall Detection ────────────────────────────┐ │ Posisi sudah 10+ bar tanpa profit signifikan │ │ → TUTUP (pasar tidak bergerak sesuai ekspektasi) │ └──────────────────────────────────────────────────────────┘ │ tidak trigger ▼ ┌─ KONDISI 7: Daily Limit ────────────────────────────────┐ │ Total daily loss mendekati limit │ │ → TUTUP semua posisi (proteksi sisa modal hari ini) │ └──────────────────────────────────────────────────────────┘ │ tidak trigger ▼ ┌─ KONDISI 8: Weekend Close ──────────────────────────────┐ │ Mendekati market close weekend + ada posisi profit │ │ → TUTUP (hindari gap risk Senin) │ └──────────────────────────────────────────────────────────┘ │ tidak trigger ▼ ┌─ KONDISI 9: Smart Time-Based (v5: Don't Cut Winners) ──┐ │ (a) > 4 jam + no growth → TUTUP (stuck) │ │ (b) > 4 jam + growing + ML agrees → HOLD (extend) │ │ (c) > 6 jam + profit < $10 → TUTUP │ │ (d) > 6 jam + profit > $10 + growing → extend ke 8 jam │ │ (e) > 8 jam → TUTUP (final max time) │ └──────────────────────────────────────────────────────────┘ │ tidak trigger ▼ ┌─ KONDISI 10: Default HOLD ──────────────────────────────┐ │ Tidak ada kondisi terpenuhi │ │ → HOLD (biarkan posisi berjalan) │ └──────────────────────────────────────────────────────────┘ ``` ### Position Manager (Tambahan per Posisi) Selain 10 kondisi di atas, Position Manager juga menjalankan: ``` ┌─ Market Close Handler (Prioritas Tertinggi) ────────────┐ │ Dekat close harian/weekend? │ │ ├── Profit ≥ $10 + dekat close → TUTUP (amankan) │ │ ├── Loss + weekend + SL > 50% → TUTUP (gap risk) │ │ └── Loss kecil + weekend → HOLD (bisa recovery) │ └──────────────────────────────────────────────────────────┘ ┌─ Breakeven Protection ──────────────────────────────────┐ │ Profit ≥ 15 pips → Pindah SL ke entry + 2 buffer │ │ (posisi tidak bisa rugi lagi) │ └──────────────────────────────────────────────────────────┘ ┌─ Trailing Stop ─────────────────────────────────────────┐ │ Profit ≥ 25 pips → SL mengikuti harga, jarak 10 pips │ │ (kunci profit sambil biarkan berjalan) │ └──────────────────────────────────────────────────────────┘ ``` --- ## 7. Sistem Proteksi Risiko 4 Lapis ``` ╔══════════════════════════════════════════════════════════════════╗ ║ LAPIS 1: BROKER STOP LOSS ║ ║ (Otomatis oleh MT5) ║ ║ ║ ║ SL = Entry ± (1.5 × ATR) minimum 10 pips ║ ║ Dikirim bersama order ke broker ║ ║ Aktif 24/7, bahkan jika bot mati ║ ║ Max loss: ~$50-80 per trade ║ ╠══════════════════════════════════════════════════════════════════╣ ║ LAPIS 2: SOFTWARE SMART EXIT ║ ║ (Bot mengevaluasi setiap ~10 detik) ║ ║ ║ ║ 10 kondisi exit (lihat bagian 6) ║ ║ Biasanya menutup SEBELUM broker SL kena ║ ║ Target close: loss ≤ $25 (lebih ketat dari broker) ║ ║ Termasuk: momentum, ML reversal, stall, time limit ║ ╠══════════════════════════════════════════════════════════════════╣ ║ LAPIS 3: EMERGENCY STOP LOSS ║ ║ (Backup jika software gagal) ║ ║ ║ ║ Max loss per trade: 2% modal ($100 untuk $5K) ║ ║ Diset sebagai broker SL terpisah ║ ║ Aktif jika software error/hang ║ ╠══════════════════════════════════════════════════════════════════╣ ║ LAPIS 4: CIRCUIT BREAKER ║ ║ (Hentikan semua trading) ║ ║ ║ ║ Trigger 1: Daily loss ≥ 3% ($150) → Stop hari ini ║ ║ Trigger 2: Total loss ≥ 10% ($500) → Stop total ║ ║ Trigger 3: Flash crash > 2.5% / 1 menit → CLOSE ALL ║ ║ Reset: Otomatis di hari baru (daily), manual (total) ║ ╚══════════════════════════════════════════════════════════════════╝ ``` ### 4 Mode Trading (Smart Risk Manager) ``` ┌──────────────────────────────────────────────────────────┐ │ MODE: NORMAL │ │ Kondisi: Semua aman, tidak ada masalah │ │ Lot: 0.01 - 0.02 (berdasarkan confidence) │ │ Max posisi: 2-3 │ │ │ │ │ 3x loss berturut-turut │ │ ▼ │ │ MODE: RECOVERY │ │ Kondisi: Setelah kerugian beruntun │ │ Lot: 0.01 (minimum saja) │ │ Max posisi: 1 │ │ │ │ │ mendekati 80% daily limit │ │ ▼ │ │ MODE: PROTECTED │ │ Kondisi: Hampir kena daily limit │ │ Lot: 0.01 (minimum saja) │ │ Max posisi: 1 │ │ │ │ │ daily/total limit tercapai │ │ ▼ │ │ MODE: STOPPED │ │ Kondisi: Batas kerugian tercapai │ │ Lot: 0 (TIDAK BOLEH trading) │ │ Max posisi: 0 (tutup semua) │ │ Reset: Otomatis hari baru │ └──────────────────────────────────────────────────────────┘ ``` ### Lot Sizing: Risk-Constrained Half-Kelly ``` Langkah 1: Hitung Kelly Fraction f* = (win_rate × avg_rr - (1 - win_rate)) / avg_rr Contoh: win_rate=55%, avg_rr=2.0 f* = (0.55 × 2.0 - 0.45) / 2.0 = 0.325 (32.5%) Langkah 2: Cap Kelly (max 25%) f* = min(0.325, 0.25) = 0.25 Langkah 3: Half-Kelly (safety) f* = 0.25 × 0.5 = 0.125 (12.5%) Langkah 4: Apply regime multiplier High volatility: × 0.5 = 0.0625 Normal: × 1.0 = 0.125 Langkah 5: Cap di config limit config risk_per_trade = 1% actual_risk = min(0.125, 0.01) = 0.01 (1%) Langkah 6: Hitung lot risk_amount = $5000 × 1% = $50 SL distance = 50 pips → pip_value ~$1/pip/0.01lot lot = $50 / (50 × $1) = 0.01 lot Langkah 7: ML Confidence boost ML ≥ 80% → lot × 2 = 0.02 lot (maximum) ML < 65% → lot = 0.01 (minimum) Langkah 8: Session multiplier Golden Time: × 1.2 Sydney: × 0.5 Final lot: 0.01 - 0.02 (ultra-konservatif) ``` --- ## 8. AI/ML Engine ### Hidden Markov Model (HMM) — Otak Regime ``` ┌──────────────────────────────────────────────────────────┐ │ HIDDEN MARKOV MODEL │ │ │ │ Library: hmmlearn.GaussianHMM │ │ Input: log_returns + rolling_volatility (2 fitur) │ │ States: 3 (Low, Medium, High Volatility) │ │ Lookback: 500 bar untuk training │ │ Retrain: setiap 20 bar (auto-update) │ │ │ │ Transition Matrix (contoh): │ │ To Low To Med To High │ │ Fr Low [ 0.85 0.12 0.03 ] │ │ Fr Med [ 0.10 0.80 0.10 ] │ │ Fr High [ 0.05 0.15 0.80 ] │ │ │ │ Distribusi Emisi (per state): │ │ Low: μ_return ≈ 0, σ_return = kecil │ │ Med: μ_return ≈ 0, σ_return = sedang │ │ High: μ_return ≈ 0, σ_return = besar │ │ │ │ Output: │ │ ├── regime: 0/1/2 (low/medium/high) │ │ ├── confidence: 0.0 - 1.0 │ │ ├── lot_multiplier: 1.0 / 0.5 / 0.0 │ │ └── recommendation: TRADE / REDUCE / SLEEP │ └──────────────────────────────────────────────────────────┘ ``` ### XGBoost — Otak Prediksi ``` ┌──────────────────────────────────────────────────────────┐ │ XGBOOST BINARY CLASSIFIER │ │ │ │ Library: xgboost │ │ Objective: binary:logistic │ │ Target: UP (1) / DOWN (0) pada bar berikutnya │ │ │ │ Anti-Overfitting Config: │ │ ├── max_depth: 3 (shallow trees) │ │ ├── learning_rate: 0.05 (slow learning) │ │ ├── min_child_weight: 10 (min samples per leaf) │ │ ├── subsample: 0.7 (70% data per tree) │ │ ├── colsample_bytree: 0.6 (60% features per tree) │ │ ├── reg_alpha: 1.0 (L1 regularization) │ │ ├── reg_lambda: 5.0 (L2 regularization) │ │ ├── gamma: 1.0 (min loss reduction) │ │ └── num_boost_round: 50 (few rounds) │ │ │ │ 24 Fitur Input (Top 10): │ │ 1. RSI(14) 6. price_position │ │ 2. MACD_histogram 7. volatility_20 │ │ 3. ATR(14) 8. returns_5 │ │ 4. bb_width 9. ema_cross │ │ 5. returns_1 10. regime │ │ │ │ Output: │ │ ├── signal: BUY / SELL / HOLD │ │ ├── probability: 0.0 - 1.0 (prob of UP) │ │ └── confidence: 0.0 - 1.0 (prob of winning side) │ │ │ │ Validation: │ │ ├── Train/Test: 70% / 30% (50-bar gap, anti leakage) │ │ ├── Walk-forward: 500 train / 50 test / 50 step │ │ ├── Target AUC: > 0.65 │ │ ├── Rollback AUC: < 0.60 (v4: dinaikkan dari 0.52) │ │ └── Overfitting ratio: train_AUC/test_AUC < 1.15 │ └──────────────────────────────────────────────────────────┘ ``` ### Kombinasi Sinyal (SMC + ML) ``` SMC Signal: "BUY at 2645, SL 2635, TP 2665, conf 75%" ML Signal: "BUY, confidence 72%" │ ▼ ┌─ KOMBINASI ──────────────────────────────────────────────┐ │ │ │ CASE 1: SMC BUY + ML BUY (≥50%) │ │ → Combined confidence = (75% + 72%) / 2 = 73.5% │ │ → ENTRY (jika pass 11 filter lainnya) │ │ │ │ CASE 2: SMC BUY + ML SELL (≥65%) │ │ → ML strongly disagrees → BLOCK (filter #5) │ │ → TIDAK entry │ │ │ │ CASE 3: SMC BUY + ML uncertain (<50%) │ │ → ML tidak yakin → BLOCK (filter #4) │ │ → TIDAK entry │ │ │ │ CASE 4: Tidak ada SMC signal │ │ → Tidak ada entry point → SKIP │ │ → SMC adalah sinyal PRIMER (wajib ada) │ │ │ │ PRINSIP: │ │ SMC = sinyal UTAMA (menentukan entry/SL/TP) │ │ ML = KONFIRMASI (bisa memblokir, tidak bisa inisiasi)│ │ HMM = PENYESUAI (mengatur agresivitas) │ └──────────────────────────────────────────────────────────┘ ``` --- ## 9. Smart Money Concepts (SMC) ### 6 Konsep yang Dianalisis ``` ┌──────────────────────────────────────────────────────────┐ │ 1. SWING POINTS (Fractal High/Low) │ │ │ │ Swing High: titik tertinggi dalam window 11 bar │ │ ↑ │ │ ____/\____ ← 5 bar kiri lebih rendah │ │ \____ ← 5 bar kanan lebih rendah │ │ │ │ Swing Low: titik terendah dalam window 11 bar │ │ ____ ____ ← 5 bar kiri lebih tinggi │ │ \____/ ← 5 bar kanan lebih tinggi │ │ ↑ │ └──────────────────────────────────────────────────────────┘ ┌──────────────────────────────────────────────────────────┐ │ 2. FAIR VALUE GAP (FVG) — Ketidakseimbangan Harga │ │ │ │ Bullish FVG (gap up): │ │ Bar[i-2].high < Bar[i].low (ada gap) │ │ │ │ │ │ │ │ │ │ ← gap (FVG zone) │ │ │ ├────┐ │ │ ├───┘ │ │ │ │ │ │ │ │ │ Harga cenderung kembali mengisi FVG → entry zone │ └──────────────────────────────────────────────────────────┘ ┌──────────────────────────────────────────────────────────┐ │ 3. ORDER BLOCK (OB) — Zona Institusi │ │ │ │ Bullish OB: candle bearish terakhir sebelum rally │ │ (zona dimana institusi menempatkan buy order besar) │ │ │ │ /───\ │ │ / \ │ │ ────\ / │ │ OB \_/ ← entry zone │ │ │ │ Lookback: 10 bar untuk deteksi │ │ Mitigated: true jika harga sudah revisit │ └──────────────────────────────────────────────────────────┘ ┌──────────────────────────────────────────────────────────┐ │ 4. BREAK OF STRUCTURE (BOS) — Kelanjutan Trend │ │ │ │ Uptrend BOS: │ │ Harga break di ATAS swing high sebelumnya │ │ → Trend bullish berlanjut │ │ │ │ /\ /\ │ │ / \ / \ /\ ← BOS (break above prev high) │ │ / \/ \/ │ │ / │ └──────────────────────────────────────────────────────────┘ ┌──────────────────────────────────────────────────────────┐ │ 5. CHANGE OF CHARACTER (CHoCH) — Perubahan Trend │ │ │ │ Uptrend → Downtrend: │ │ Harga break di BAWAH swing low terakhir │ │ → Trend berubah dari bullish ke bearish │ │ │ │ /\ /\ │ │ / \ / \ │ │ / \/ \ │ │ \____ ← CHoCH (break below prev low) │ └──────────────────────────────────────────────────────────┘ ┌──────────────────────────────────────────────────────────┐ │ 6. LIQUIDITY ZONES — Target Likuiditas │ │ │ │ BSL (Buy Side Liquidity): di atas swing highs │ │ SSL (Sell Side Liquidity): di bawah swing lows │ │ │ │ ---- BSL ---- (stop loss para seller berkumpul) │ │ /\ /\ │ │ / \ / \ │ │ / \/ \ │ │ ---- SSL ---- (stop loss para buyer berkumpul) │ │ │ │ Institusi sering "hunt" liquidity zone ini │ └──────────────────────────────────────────────────────────┘ ``` ### Signal Generation ``` SYARAT SINYAL SMC: Structure Break (BOS atau CHoCH) + Zone (FVG atau Order Block) = VALID SIGNAL BUY Signal: ├── BOS bullish ATAU CHoCH bearish→bullish ├── + Bullish FVG ATAU Bullish OB di bawah harga ├── Entry: harga saat ini ├── SL: di bawah zone, minimum 1.5 × ATR ├── TP: 2:1 R:R, maximum 4 × ATR └── Confidence: 40-85% (v5: calibrated weighted scoring) (lebih banyak confluence = lebih tinggi) SELL Signal: ├── BOS bearish ATAU CHoCH bullish→bearish ├── + Bearish FVG ATAU Bearish OB di atas harga ├── Entry: harga saat ini ├── SL: di atas zone, minimum 1.5 × ATR ├── TP: 2:1 R:R, maximum 4 × ATR └── Confidence: 40-85% (v5: calibrated weighted scoring) ``` --- ## 10. Position Lifecycle ### Dari Lahir Sampai Mati (Siklus Hidup Posisi) ``` ╔═══════════════════════════════════════════════════════════════╗ ║ TAHAP 1: SINYAL TERDETEKSI ║ ║ ║ ║ SMC menemukan setup + ML konfirmasi + 11 filter PASS ║ ║ → Keputusan: BUKA POSISI ║ ╚═══════════════════════════════════════════════════════════════╝ │ ▼ ╔═══════════════════════════════════════════════════════════════╗ ║ TAHAP 2: LOT SIZE CALCULATION ║ ║ ║ ║ Risk Engine (Kelly): ║ ║ Balance $5000 × 1% risk = $50 max loss ║ ║ SL distance 50 pips → lot = 0.01 ║ ║ ║ ║ ML Confidence boost: ║ ║ ML ≥ 80% → 0.02 lot (double) ║ ║ ML < 65% → 0.01 lot (minimum) ║ ║ ║ ║ Session multiplier: ║ ║ Golden: × 1.2, Sydney: × 0.5 ║ ║ ║ ║ Regime multiplier: ║ ║ Normal: × 1.0, High Vol: × 0.5, Crisis: × 0.0 ║ ╚═══════════════════════════════════════════════════════════════╝ │ ▼ ╔═══════════════════════════════════════════════════════════════╗ ║ TAHAP 3: ORDER VALIDATION ║ ║ ║ ║ Risk Engine memvalidasi: ║ ║ ✓ SL di sisi yang benar (BUY: SL < entry) ║ ║ ✓ TP di sisi yang benar (BUY: TP > entry) ║ ║ ✓ Lot dalam range (0.01 - 0.05) ║ ║ ✓ Entry dekat harga saat ini (< 0.1%) ║ ║ ✓ Risk% ≤ 1.5× config limit ║ ║ ✓ Circuit breaker TIDAK aktif ║ ╚═══════════════════════════════════════════════════════════════╝ │ ▼ ╔═══════════════════════════════════════════════════════════════╗ ║ TAHAP 4: ORDER EXECUTION ║ ║ ║ ║ MT5 Connector mengirim order: ║ ║ → Symbol: XAUUSD ║ ║ → Type: BUY/SELL ║ ║ → Lot: 0.01-0.02 ║ ║ → SL: ATR-based (broker level) ║ ║ → TP: 2:1 R:R (broker level) ║ ║ → Deviation: 20 points (slippage tolerance) ║ ║ → Retry: max 3 attempts jika gagal ║ ╚═══════════════════════════════════════════════════════════════╝ │ ▼ ╔═══════════════════════════════════════════════════════════════╗ ║ TAHAP 4b: POST-EXECUTION VALIDATION (v5 BARU) ║ ║ ║ ║ Slippage Validation: ║ ║ → Bandingkan harga aktual vs expected ║ ║ → Max acceptable: 0.15% dari harga (~$4 untuk XAUUSD) ║ ║ → Log WARNING jika melebihi batas ║ ║ → Gunakan harga AKTUAL untuk position tracking ║ ║ ║ ║ Partial Fill Handling: ║ ║ → Cek apakah volume terisi = volume diminta ║ ║ → Jika partial: log fill ratio, update lot_size aktual ║ ║ → Risk calculation tetap akurat dengan volume sebenarnya ║ ╚═══════════════════════════════════════════════════════════════╝ │ ▼ ╔═══════════════════════════════════════════════════════════════╗ ║ TAHAP 5: POSITION REGISTERED ║ ║ ║ ║ Smart Risk Manager: ║ ║ → Catat entry price AKTUAL, direction, lot AKTUAL, timestamp║ ║ → v5: Menggunakan harga & volume dari broker (bukan planned)║ ║ → Inisialisasi peak_profit = 0 ║ ║ → Mulai tracking momentum ║ ║ ║ ║ Trade Logger: ║ ║ → Insert ke PostgreSQL (30+ field) ║ ║ → Backup ke CSV ║ ║ ║ ║ Telegram: ║ ║ → Kirim notifikasi trade open ║ ║ → Detail: entry, SL, TP, R:R, confidence, regime ║ ╚═══════════════════════════════════════════════════════════════╝ │ ▼ ╔═══════════════════════════════════════════════════════════════╗ ║ TAHAP 6: ACTIVE MONITORING (setiap ~10 detik + candle baru) ║ ║ ║ ║ ┌── Update profit/loss real-time ║ ║ ├── Update peak profit (tertinggi yang pernah dicapai) ║ ║ ├── Hitung momentum (kecepatan perubahan profit) ║ ║ ├── Hitung TP probability ║ ║ ├── Cek 10 kondisi exit (lihat bagian 6) ║ ║ ├── Cek Position Manager (trailing, breakeven) ║ ║ └── Cek Market Close Handler (dekat close?) ║ ║ ║ ║ Setiap ~10 detik (atau candle baru), posisi dievaluasi: ║ ║ → HOLD (lanjut) ║ ║ → CLOSE (tutup dengan alasan spesifik) ║ ╚═══════════════════════════════════════════════════════════════╝ │ │ trigger close ▼ ╔═══════════════════════════════════════════════════════════════╗ ║ TAHAP 7: POSITION CLOSED ║ ║ ║ ║ MT5 Connector: ║ ║ → Close position via market order ║ ║ ║ ║ Smart Risk Manager: ║ ║ → Record profit/loss ║ ║ → Update daily/total loss counters ║ ║ → Update win/loss streak ║ ║ → Check mode transition (NORMAL→RECOVERY→PROTECTED→STOPPED)║ ║ ║ ║ Trade Logger: ║ ║ → Update trade record: exit price, profit, duration, reason ║ ║ → Update PostgreSQL + CSV ║ ║ ║ ║ Telegram: ║ ║ → Kirim notifikasi trade close ║ ║ → Detail: profit, duration, exit reason, balance ║ ╚═══════════════════════════════════════════════════════════════╝ ``` --- ## 11. Auto-Retraining & Model Management ### Lifecycle Model AI ``` ┌──────────────────────────────────────────────────────────────┐ │ INITIAL TRAINING (train_models.py) │ │ Dijalankan 1x saat setup │ │ │ │ 1. Fetch 10,000 bar M15 dari MT5 (~104 hari) │ │ 2. Feature Engineering → 40+ fitur │ │ 3. SMC Analysis → struktur pasar │ │ 4. Create target → UP/DOWN (lookahead=1) │ │ 4b. Split 70/30 dengan 50-bar gap (anti temporal leakage) │ │ 5. Train HMM (3 regime, lookback=500) │ │ 6. Train XGBoost (50 rounds, early_stop=5) │ │ 7. Walk-forward validation (500 train/50 test/50 step) │ │ 8. Save → models/hmm_regime.pkl + xgboost_model.pkl │ └──────────────────────────────┬───────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────────┐ │ DAILY AUTO-RETRAINING (Auto Trainer) │ │ Otomatis setiap hari 05:00 WIB │ │ │ │ Schedule: │ │ ├── Harian (05:00 WIB): 8,000 bar, 50 rounds │ │ ├── Weekend (05:00 Sabtu): 15,000 bar, 80 rounds (deep) │ │ └── Emergency: jika AUC < 0.65 (kualitas turun) │ │ │ │ Proses: │ │ 1. Backup model saat ini → models/backup/ │ │ 2. Fetch data baru dari MT5 │ │ 3. Feature Engineering + SMC │ │ 4. Train HMM baru + XGBoost baru │ │ 5. Validasi: test AUC ≥ 0.60? (v4: dinaikkan dari 0.52) │ │ ├── Ya → Save model baru, reload di memory │ │ └── Tidak → ROLLBACK ke model sebelumnya │ │ 6. Log hasil ke PostgreSQL │ │ 7. Kirim laporan via Telegram │ │ │ │ Safety: │ │ ├── Max 5 backup disimpan (rotasi) │ │ ├── Min 20 jam antar retrain (cooldown) │ │ ├── Auto-rollback jika AUC < 0.60 (v4 threshold) │ │ └── Model lama selalu tersedia untuk rollback │ └──────────────────────────────────────────────────────────────┘ ``` ### Perbandingan Initial vs Auto Training | Aspek | train_models.py | Auto Trainer | |-------|-----------------|-------------| | Kapan | Manual, 1x setup | Otomatis, harian | | Data | 10,000 bar | 8K (harian) / 15K (weekend) | | Boost rounds | 50 | 50 (harian) / 80 (weekend) | | Walk-forward | Ya | Tidak | | Backup | Tidak | Ya (5 terakhir) | | Rollback | Tidak | Ya (AUC < 0.60) | | Database | Tidak | Ya (PostgreSQL) | | Tujuan | Setup awal | Maintenance rutin | --- ## 12. Infrastruktur & Database ### PostgreSQL Schema ``` trading_db ├── trades (Semua trade: open, close, profit, SMC, ML, features) ├── training_runs (Log setiap training: AUC, akurasi, durasi, rollback) ├── signals (Setiap sinyal yang dihasilkan: executed atau tidak) ├── market_snapshots (Snapshot periodik: harga, regime, volatilitas) ├── bot_status (Status bot: uptime, loop count, balance, risk mode) └── daily_summaries (Ringkasan harian: win rate, profit factor, per sesi) ``` ### Tabel `trades` (Detail) ```sql -- Identifikasi ticket, symbol, direction (BUY/SELL) -- Harga entry_price, exit_price, stop_loss, take_profit -- Hasil lot_size, profit_usd, profit_pips opened_at, closed_at, duration_seconds -- Konteks Entry entry_regime, entry_volatility, entry_session smc_signal, smc_confidence, smc_reason smc_fvg_detected, smc_ob_detected, smc_bos_detected, smc_choch_detected ml_signal, ml_confidence market_quality, market_score, dynamic_threshold -- Konteks Exit exit_reason, exit_regime, exit_ml_signal -- Keuangan balance_before, balance_after, equity_at_entry -- Data Lengkap features_entry (JSON), features_exit (JSON) bot_version, trade_mode ``` ### Connection Architecture ``` Bot Components ├── TradeLogger → TradeRepository, SignalRepository, MarketSnapshotRepository ├── AutoTrainer → TrainingRepository ├── main_live.py → BotStatusRepository, DailySummaryRepository └── Dashboard → Semua repository (READ) │ ▼ DatabaseConnection (Singleton) │ ▼ ThreadedConnectionPool (1-10 koneksi) │ ▼ PostgreSQL Server ``` ### Graceful Degradation ``` PostgreSQL tersedia? ├── Ya → Gunakan DB + CSV backup (dual write) └── Tidak → CSV saja (bot tetap berjalan 100%) Bot TIDAK PERNAH crash karena database. Semua operasi DB dibungkus try-except. ``` --- ## 13. Konfigurasi & Parameter Kritis ### Configuration System ``` .env file │ ▼ TradingConfig.from_env() │ ├── RiskConfig │ ├── risk_per_trade: 1.0% (SMALL) / 0.5% (MEDIUM) │ ├── max_daily_loss: 3.0% (SMALL) / 2.0% (MEDIUM) │ ├── max_total_loss: 10.0% │ ├── max_positions: 3 (SMALL) / 5 (MEDIUM) │ ├── min_lot: 0.01 │ ├── max_lot: 0.05 (SMALL) / 2.0 (MEDIUM) │ └── max_leverage: 1:100 (SMALL) / 1:30 (MEDIUM) │ ├── SMCConfig │ ├── swing_length: 5 │ ├── fvg_min_gap_pips: 2.0 │ ├── ob_lookback: 10 │ └── bos_close_break: true │ ├── MLConfig │ ├── confidence_threshold: 0.65 │ ├── entry_confidence: 0.70 │ ├── high_confidence: 0.75 │ ├── very_high_confidence: 0.80 │ └── retrain_frequency_days: 7 │ ├── ThresholdsConfig │ ├── ml_min_confidence: 0.65 │ ├── ml_high_confidence: 0.75 │ ├── trade_cooldown_seconds: 300 │ ├── min_profit_to_secure: $15 │ ├── good_profit: $25 │ ├── great_profit: $40 │ ├── flash_crash_threshold: 2.5% │ └── sydney_lot_multiplier: 0.5 │ └── RegimeConfig ├── n_regimes: 3 ├── lookback: 500 └── retrain_frequency: 20 ``` ### Capital Mode Auto-Detection ``` Balance ≤ $10,000 → SMALL MODE ├── Risk: 1% per trade ($50 pada $5K) ├── Daily limit: 3% ($150) ├── Lot: 0.01-0.05 ├── Leverage: 1:100 ├── Timeframe: M15 └── Max posisi: 3 Balance > $10,000 → MEDIUM MODE ├── Risk: 0.5% per trade ├── Daily limit: 2% ├── Lot: 0.01-2.0 ├── Leverage: 1:30 ├── Timeframe: H1 └── Max posisi: 5 ``` ### Session Schedule (WIB = GMT+7) ``` ┌────────────────────────────────────────────────────────────┐ │ WAKTU (WIB) │ SESI │ LOT MULT │ KETERANGAN │ ├──────────────┼───────────────┼──────────┼─────────────────┤ │ 00:00-04:00 │ DEAD ZONE │ BLOCKED │ Likuiditas rendah│ │ 04:00-06:00 │ ROLLOVER │ BLOCKED │ Spread melebar │ │ 06:00-07:00 │ Sydney │ 0.5x │ Pasar baru buka │ │ 07:00-13:00 │ Tokyo+Sydney │ 0.7x │ Asia aktif │ │ 13:00-15:00 │ Tokyo akhir │ 0.7x │ Transisi │ │ 15:00-20:00 │ London │ 1.0x │ Volatilitas naik │ │ 20:00-23:59 │ ★ GOLDEN TIME │ 1.2x │ London+NY overlap│ │ Jumat ≥23:00 │ WEEKEND RISK │ BLOCKED │ Gap risk │ └────────────────────────────────────────────────────────────┘ ★ Golden Time (20:00-23:59 WIB) = waktu paling optimal → Spread ketat, likuiditas maksimal, volatilitas ideal → Lot multiplier 1.2x (bonus) → v4: Smart Hold dihapus — tidak ada lagi hold losers menunggu sesi tertentu ``` --- ## 14. Performa & Timing ### Main Loop Breakdown ``` Target: < 50ms per iterasi analisis FULL ANALYSIS (saat candle baru M15): ┌────────────────────────────────────────────────────────────┐ │ Komponen │ Waktu │ Keterangan │ ├────────────────────────┼──────────┼────────────────────────┤ │ MT5 data fetch │ ~10ms │ 200 bar M15 via API │ │ Feature engineering │ ~5ms │ 40+ fitur, Polars │ │ SMC analysis │ ~5ms │ 6 konsep, Polars native│ │ HMM predict │ ~2ms │ 2 fitur → 1 regime │ │ XGBoost predict │ ~3ms │ 24 fitur → 1 signal │ │ Position monitoring │ ~5ms │ Per posisi terbuka │ │ Entry logic │ ~5ms │ 11 filter check │ │ Overhead │ ~15ms │ Logging, state update │ ├────────────────────────┼──────────┼────────────────────────┤ │ TOTAL │ ~50ms │ │ └────────────────────────────────────────────────────────────┘ POSITION CHECK ONLY (di antara candle, setiap ~10 detik): ┌────────────────────────────────────────────────────────────┐ │ Komponen │ Waktu │ Keterangan │ ├────────────────────────┼──────────┼────────────────────────┤ │ MT5 data fetch │ ~5ms │ 50 bar saja │ │ Feature engineering │ ~3ms │ Minimal fitur │ │ ML prediction │ ~3ms │ Untuk exit evaluation │ │ Position evaluation │ ~5ms │ 10 kondisi exit │ │ Overhead │ ~5ms │ Logging │ ├────────────────────────┼──────────┼────────────────────────┤ │ TOTAL │ ~21ms │ │ └────────────────────────────────────────────────────────────┘ ``` ### Timer Periodik ``` ┌──────────────────────────────────────────────────────────┐ │ Event │ Interval │ Cara Trigger │ ├────────────────────────┼────────────────┼─────────────────┤ │ Full analysis + entry │ Candle baru M15│ Deteksi candle │ │ Position monitoring │ ~10 detik │ Di antara candle│ │ Performance logging │ 4 candle (~1j) │ candle_count % 4│ │ Auto-retrain check │ 20 candle (~5j)│ candle_count %20│ │ Market update Telegram │ 30 menit │ Timer │ │ Hourly analysis │ 1 jam │ Timer │ │ Daily summary + reset │ Ganti hari │ Date check │ └──────────────────────────────────────────────────────────┘ ``` --- ## 15. Error Handling & Fault Tolerance ### Prinsip: Bot TIDAK PERNAH Crash ``` ┌──────────────────────────────────────────────────────────┐ │ LEVEL 1: Per-Loop Error Handling │ │ │ │ try: │ │ # Fetch data, analyze, trade │ │ except ConnectionError: │ │ # MT5 disconnected → reconnect() │ │ except Exception as e: │ │ # Log error → lanjut loop berikutnya │ │ # Bot TIDAK crash dari error tunggal │ └──────────────────────────────────────────────────────────┘ ┌──────────────────────────────────────────────────────────┐ │ LEVEL 2: MT5 Auto-Reconnect │ │ │ │ MT5 putus? │ │ ├── Attempt 1: reconnect (tunggu 2 detik) │ │ ├── Attempt 2: reconnect (tunggu 4 detik) │ │ ├── Attempt 3: reconnect (tunggu 8 detik) │ │ ├── Cooldown 60 detik │ │ └── Retry cycle (max 5 per cooldown) │ │ │ │ Selama disconnected: │ │ → Position monitoring PAUSE │ │ → Entry baru DITUNDA │ │ → Posisi terbuka dilindungi broker SL (lapis 1) │ └──────────────────────────────────────────────────────────┘ ┌──────────────────────────────────────────────────────────┐ │ LEVEL 3: Database Graceful Degradation │ │ │ │ PostgreSQL down? │ │ ├── Switch ke CSV-only mode │ │ ├── Semua data tetap dicatat │ │ ├── Trading tetap berjalan normal │ │ └── Retry DB connection periodik │ └──────────────────────────────────────────────────────────┘ ┌──────────────────────────────────────────────────────────┐ │ LEVEL 4: Telegram Failure │ │ │ │ Telegram API error? │ │ ├── Log error secara silent │ │ ├── Trading tetap jalan 100% │ │ └── Retry di notifikasi berikutnya │ └──────────────────────────────────────────────────────────┘ ┌──────────────────────────────────────────────────────────┐ │ LEVEL 5: Model File Missing │ │ │ │ .pkl file tidak ditemukan? │ │ ├── Log warning │ │ ├── Skip prediksi (ML/HMM) │ │ ├── Trading bisa jalan tanpa ML (SMC only) │ │ └── Trigger: jalankan train_models.py │ └──────────────────────────────────────────────────────────┘ ┌──────────────────────────────────────────────────────────┐ │ LEVEL 6: Flash Crash Protection │ │ │ │ Harga bergerak > 2.5% dalam 1 menit? │ │ ├── EMERGENCY: Close ALL positions │ │ ├── Circuit breaker AKTIF │ │ ├── Kirim alert KRITIS via Telegram │ │ └── Bot masuk mode STOPPED │ └──────────────────────────────────────────────────────────┘ ``` ### Startup & Shutdown ``` STARTUP SEQUENCE: 1. Load konfigurasi dari .env 2. Connect ke MT5 (max 3 retry) 3. Load model HMM dari models/hmm_regime.pkl 4. Load model XGBoost dari models/xgboost_model.pkl 5. Initialize SmartRiskManager (set balance, limits) 6. Initialize SessionFilter (WIB timezone) 7. Initialize TelegramNotifier 8. Initialize TradeLogger (connect DB) 9. Initialize AutoTrainer 10. Send Telegram: "BOT STARTED" (config, balance, risk settings) 11. Mulai main loop SHUTDOWN SEQUENCE (SIGINT/SIGTERM): 1. Signal diterima 2. Hentikan loop utama 3. Kirim Telegram: "BOT STOPPED" (balance, trades, uptime) 4. Disconnect MT5 5. Close database connections 6. Exit ``` --- ## 16. Daftar File Source Code ``` Smart Automatic Trading BOT + AI/ │ ├── main_live.py # Orchestrator utama (TradingBot) ├── train_models.py # Script training awal ├── .env # Environment variables (credentials) │ ├── src/ │ ├── config.py # Konfigurasi terpusat (6 sub-config) │ ├── mt5_connector.py # Bridge ke MetaTrader 5 │ ├── feature_eng.py # Feature Engineering (40+ fitur) │ ├── regime_detector.py # HMM Regime Detection (3 state) │ ├── ml_model.py # XGBoost Signal Predictor │ ├── smc_polars.py # Smart Money Concepts (6 konsep) │ ├── smart_risk_manager.py # 4-Mode Risk Manager │ ├── risk_engine.py # Kelly Criterion + Circuit Breaker │ ├── session_filter.py # Session Time Filter (WIB) │ ├── dynamic_confidence.py # Dynamic Threshold Manager │ ├── news_agent.py # News Event Monitor │ ├── telegram_notifier.py # Telegram Push Notifications │ ├── auto_trainer.py # Daily Auto-Retraining │ ├── trade_logger.py # Dual Storage Logger (DB+CSV) │ ├── position_manager.py # Position Manager + Market Close │ │ │ └── db/ │ ├── __init__.py # DB exports │ ├── connection.py # PostgreSQL Singleton + Pool │ └── repository.py # 6 Repository classes │ ├── models/ │ ├── xgboost_model.pkl # Trained XGBoost model │ ├── hmm_regime.pkl # Trained HMM model │ └── backup/ # Auto-backup (5 terakhir) │ ├── data/ │ ├── training_data.parquet # Data training terakhir │ └── trade_logs/ # CSV backup (per bulan) │ ├── trades_2025_01.csv │ ├── trades_2025_02.csv │ └── ... │ ├── backtests/ │ └── backtest_live_sync.py # Backtest 100% sync live │ ├── logs/ │ └── training_YYYY-MM-DD.log # Log training detail │ └── docs/ └── arsitektur-ai/ ├── 00-ARSITEKTUR-LENGKAP.md # Dokumen ini ├── README.md # Index komponen └── 01-23 (per komponen) # Detail per modul ``` --- ## Ringkasan Eksekutif **Smart AI Trading Bot** adalah sistem trading otomatis yang menggabungkan: 1. **Smart Money Concepts (SMC)** sebagai sinyal UTAMA — mendeteksi zona institusi (FVG, Order Block, BOS, CHoCH) untuk menentukan entry, SL, dan TP yang presisi. 2. **XGBoost Machine Learning** sebagai KONFIRMASI — memprediksi arah harga dengan 24 fitur teknikal, memblokir trade jika tidak setuju dengan SMC. 3. **Hidden Markov Model (HMM)** sebagai PENYESUAI — mendeteksi kondisi pasar (tenang/volatile/krisis) untuk menyesuaikan agresivitas. 4. **4-Lapis Proteksi Risiko** — dari broker SL, software smart exit, emergency stop, hingga circuit breaker. Lot ultra-kecil (0.01-0.02) memastikan kerugian per trade maximum $50 (1%). 5. **Self-Improving** — model AI dilatih ulang otomatis setiap hari dengan auto-rollback jika model baru lebih buruk. 6. **Fault-Tolerant** — bot tidak pernah crash. MT5 putus? Auto-reconnect. Database mati? CSV fallback. Error? Log dan lanjut. Semua ini dikoordinasikan oleh **Main Live Orchestrator** yang menjalankan loop **candle-based** — analisis penuh hanya saat candle M15 baru terbentuk (~50ms per iterasi), dengan pengecekan posisi setiap ~10 detik di antara candle (~21ms). Mengevaluasi 11 filter entry dan 10 kondisi exit secara real-time, dengan notifikasi Telegram untuk setiap kejadian penting. ``` TARGET: Trading XAUUSD M15 yang KONSISTEN dan AMAN dengan kerugian terkontrol dan profit teroptimasi. ```