XauBot/docs/arsitektur-ai/00-ARSITEKTUR-LENGKAP.md

Arsitektur Lengkap — Smart AI Trading Bot

Dokumen: Arsitektur keseluruhan sistem dalam 1 file Instrumen: XAUUSD (Gold) M15 Platform: MetaTrader 5 Bahasa: Python 3.11+ (async, Polars, XGBoost, HMM) Database: PostgreSQL + CSV fallback Notifikasi: Telegram Bot API

---

Daftar Isi

1. Gambaran Umum
2. Diagram Arsitektur
3. 23 Komponen
4. Pipeline Data: Dari OHLCV ke Keputusan Trading
5. Alur Entry: 14 Filter
6. Alur Exit: 12 Kondisi
7. Sistem Proteksi Risiko 4 Lapis
8. AI/ML Engine
9. Smart Money Concepts (SMC)
10. Position Lifecycle
11. Auto-Retraining & Model Management
12. Infrastruktur & Database
13. Konfigurasi & Parameter Kritis
14. Performa & Timing
15. Error Handling & Fault Tolerance
16. Daftar File Source Code

---

1. Gambaran Umum

Apa Ini?

Bot trading otomatis yang menggabungkan 3 otak kecerdasan buatan untuk trading XAUUSD (Emas) di MetaTrader 5:

OTAK 1: Smart Money Concepts (SMC)
        → Membaca pola institusi besar (bank, hedge fund)
        → Menentukan DIMANA entry, SL, dan TP

OTAK 2: XGBoost Machine Learning
        → Memprediksi ARAH harga (naik/turun)
        → Memberikan tingkat keyakinan (confidence)

OTAK 3: Hidden Markov Model (HMM)
        → Membaca KONDISI pasar (tenang/volatile/krisis)
        → Menyesuaikan ukuran posisi dan agresivitas

Filosofi Desain

1. KESELAMATAN MODAL NOMOR 1
   → 4 lapis proteksi stop loss
   → Lot ultra-kecil (0.01-0.02)
   → Circuit breaker otomatis

2. TIDAK PERNAH CRASH
   → Setiap error di-catch, bot terus jalan
   → Database gagal? CSV fallback
   → MT5 putus? Auto-reconnect

3. SELF-IMPROVING
   → Model AI dilatih ulang otomatis setiap hari
   → Rollback otomatis jika model baru lebih buruk
   → Threshold confidence menyesuaikan kondisi pasar

4. TRANSPARAN
   → Setiap keputusan dicatat ke database
   → Notifikasi Telegram real-time
   → Laporan harian, jam-an, dan per-trade

Angka-Angka Kunci

Parameter	Nilai	Penjelasan
Modal target	$5,000	Small account mode
Risiko per trade	1% ($50)	Maksimum kerugian per posisi
Lot size	0.01 - 0.02	Ultra-konservatif
Max daily loss	3% ($150)	Circuit breaker harian
Max total loss	10% ($500)	Stop total trading
Max posisi bersamaan	2-3	Menghindari overexposure
Cooldown antar trade	5 menit	Mencegah overtrading
Loop speed	~50ms	Cepat tapi efisien
Timeframe	M15	15 menit per candle

---

2. Diagram Arsitektur

Diagram Keseluruhan Sistem

graph TD
    subgraph MAIN["MAIN LIVE — Orchestrator<br/>main_live.py — TradingBot<br/>Candle-based M15 + position check ~10 detik"]
        direction TB

        subgraph P1["PHASE 1: DATA"]
            MT5C["MT5 Connector<br/>(broker)"] --> FEng["Feature Engine<br/>(40+ fitur)"]
            FEng --> SMCA["SMC Analyzer<br/>(institusi)"]
            MT5C ~~~ HMMD["HMM Regime<br/>(3 state)"]
            FEng --> XGB["XGBoost ML Model<br/>(prediksi)"]
            SMCA --> XGB
            HMMD --> DC["Dynamic Confidence<br/>(threshold)"]
        end

        subgraph P2["PHASE 2: MONITORING"]
            PM["Position Manager<br/>(trailing)"]
            SRM["Smart Risk Manager"]
            RiskE["Risk Engine<br/>(Kelly)"]
        end

        subgraph P3["PHASE 3: ENTRY"]
            SF["Session Filter<br/>(waktu)"]
            NA["News Agent<br/>(berita)"]
            EF["14-Gate Entry Filter"] --> EXEC["EXECUTE"]
        end

        subgraph P4["PHASE 4: PERIODIK"]
            AT["Auto Trainer<br/>(retrain)"]
            TN["Telegram Notifier<br/>(laporan)"]
            TL["Trade Logger<br/>(DB+CSV)"]
        end

        P1 --> P2
        P2 --> P3
        P3 --> P4
        P4 --> DB["PostgreSQL + CSV Backup"]
    end

Alur Data (Data Flow)

flowchart TD
    MT5B["MT5 Broker<br/>(XAUUSD M15)"] -->|"200 bar OHLCV"| MT5C["MT5 Connector<br/>numpy → Polars (tanpa Pandas)"]
    MT5C --> FE["Feature Engineer<br/>OHLCV → 40+ fitur teknikal<br/>RSI, ATR, MACD, BB, EMA, Volume"]
    FE --> SMC["SMC Analyzer"]
    FE --> HMM["HMM Regime Detect"]
    SMC --> XGB["XGBoost ML Predictor<br/>24 fitur → BUY/SELL/HOLD<br/>+ Confidence 0-100%"]
    HMM --> XGB
    XGB --> DCM["Dynamic Confidence<br/>Threshold berdasarkan sesi,<br/>regime, volatilitas, trend"]
    DCM --> SC["Signal Combiner (14 Filter)<br/>SMC + ML harus setuju<br/>+ Session + Risk + Cooldown"]
    SC --> PASS{"PASS?"}
    PASS -->|"NO"| WAIT["Tunggu loop berikutnya"]
    PASS -->|"YES"| RE["Risk Engine + Risk Manager<br/>Kelly Criterion → lot size<br/>Validasi order → approve/reject"]
    RE --> EXEC["Execute Order via MT5<br/>Kirim ke MT5 dengan SL dan TP<br/>Register ke Position Manager"]
    EXEC --> TN["Telegram Notifier"]
    EXEC --> TL["Trade Logger<br/>(DB + CSV)"]

---

3. 23 Komponen

Tabel Komponen Lengkap

#	Komponen	File	Kategori	Fungsi Utama
1	HMM Regime Detector	src/regime_detector.py	AI/ML	Deteksi kondisi pasar (3 regime)
2	XGBoost Predictor	src/ml_model.py	AI/ML	Prediksi arah harga + confidence
3	SMC Analyzer	src/smc_polars.py	Analisis	Pola institusi: FVG, OB, BOS, CHoCH
4	Feature Engineering	src/feature_eng.py	Data	OHLCV → 40+ fitur numerik
5	Smart Risk Manager	src/smart_risk_manager.py	Risiko	4 mode trading, 12 kondisi exit
6	Session Filter	src/session_filter.py	Filter	Waktu trading optimal (WIB)
7	Stop Loss (4 Lapis)	Multi-file	Proteksi	SMC → Software → Emergency → Circuit
8	Take Profit (6 Layer)	Multi-file	Proteksi	Hard → Momentum → Peak → Probability → Early → Broker
9	Entry Trade	main_live.py	Eksekusi	14 filter berurutan
10	Exit Trade	main_live.py	Eksekusi	12 kondisi exit real-time
11	News Agent	src/news_agent.py	Monitor	NONAKTIF — dikomentari di kode
12	Telegram Notifier	src/telegram_notifier.py	Notifikasi	11 tipe notifikasi real-time
13	Auto Trainer	src/auto_trainer.py	ML Ops	Retraining harian otomatis
14	Backtest	backtests/backtest_live_sync.py	Validasi	Simulasi 100% sync dengan live
15	Dynamic Confidence	src/dynamic_confidence.py	Adaptif	Threshold ML adaptif (60-85%)
16	MT5 Connector	src/mt5_connector.py	Koneksi	Bridge ke broker, auto-reconnect
17	Configuration	src/config.py	Config	6 sub-config, auto-adjust modal
18	Trade Logger	src/trade_logger.py	Logging	Dual storage DB + CSV
19	Position Manager	src/position_manager.py	Manajemen	Trailing SL, breakeven, market close
20	Risk Engine	src/risk_engine.py	Risiko	Kelly Criterion, circuit breaker
21	Database	src/db/	Storage	PostgreSQL, 6 repository
22	Train Models	train_models.py	Training	Script training awal
23	Main Live	main_live.py	Orchestrator	Koordinasi semua komponen

Hubungan Antar Komponen

flowchart TD
    CONFIG["CONFIGURATION (17)<br/>Sumber parameter semua"] -->|"dikonsumsi oleh semua"| MT5_16
    CONFIG --> FE4
    CONFIG --> SMC3

    MT5_16["MT5 (16)<br/>Broker"] --> FE4["FeatEng (4)<br/>40+ fitur"]
    FE4 --> SMC3["SMC (3)<br/>Institusi"]
    FE4 --> HMM1["HMM (1)<br/>Regime"]
    FE4 --> XGB2["XGBoost (2)<br/>Prediksi"]
    SMC3 --> XGB2

    HMM1 --> DC15["Dynamic Confidence (15)<br/>Threshold adaptif"]
    XGB2 --> DC15

    SESSION6["Session (6)<br/>Waktu"] --> ENTRY9["ENTRY TRADE (9)<br/>14 Filter Gate"]
    DC15 --> ENTRY9
    NEWS11["News (11)<br/>Berita"] --> ENTRY9

    ENTRY9 --> RISK20["RiskEng (20)<br/>Kelly Lot"]
    ENTRY9 --> SRISK5["SmartRisk (5)<br/>4 Mode"]

    RISK20 --> EXEC["EXECUTE ORDER<br/>via MT5 (16)"]
    SRISK5 --> EXEC

    EXEC --> PM19["PosMgr (19)<br/>Trailing"]
    EXEC --> LOG18["Logger (18)<br/>DB+CSV"]
    EXEC --> TG12["Telegram (12)"]

    PM19 --> EXIT10["EXIT (10)<br/>12 Kondisi"]
    LOG18 --> DB21["DB (21)<br/>PostgreSQL"]

    subgraph PERIODIK["Periodik"]
        AT13["AutoTrain (13)<br/>Harian"]
        BT14["Backtest (14)<br/>Validasi"]
        TM22["Train Models (22)<br/>Setup awal"]
    end

---

4. Pipeline Data

Dari OHLCV Mentah ke Keputusan Trading

Tahap 1: Data Fetching (MT5 Connector)

flowchart TD
    MT5B["MT5 Broker"] -->|"mt5.copy_rates_from_pos<br/>(XAUUSD, M15, 0, 200)"| NPA["NumPy Structured Array"]
    NPA -->|"Konversi langsung ke Polars<br/>(TANPA Pandas)"| PDF["Polars DataFrame:<br/>time (i64), open (f64), high (f64),<br/>low (f64), close (f64),<br/>tick_volume (f64), spread (f64)"]

Kenapa Polars, bukan Pandas?

• 3-5x lebih cepat untuk operasi vectorized
• Memory-efficient (zero-copy)
• Native lazy evaluation
• Konsisten di seluruh codebase (tidak ada konversi bolak-balik)

Tahap 2: Feature Engineering (40+ Fitur)

flowchart TD
    INPUT["Input: Polars DataFrame<br/>(200 bar OHLCV)"] --> MOM["Momentum Indicators"]
    INPUT --> VOL["Volatility Indicators"]
    INPUT --> TREND["Trend Indicators"]
    INPUT --> PA["Price Action"]
    INPUT --> VOLF["Volume Features"]
    INPUT --> LAG["Lag Features"]
    INPUT --> TIME["Time Features"]

    MOM --> M1["RSI(14) - 0-100, overbought/oversold"]
    MOM --> M2["MACD(12,26,9) - trend strength and direction"]
    MOM --> M3["MACD Histogram - momentum acceleration"]

    VOL --> V1["ATR(14) - average true range (pips)"]
    VOL --> V2["Bollinger Bands(20,2.0) - upper, lower, width"]
    VOL --> V3["Volatility(20) - rolling std of returns"]

    TREND --> T1["EMA(9) / EMA(21) - fast/slow crossover"]
    TREND --> T2["EMA Cross Signal - 1 bullish / -1 bearish"]
    TREND --> T3["SMA(20) - simple moving average"]

    PA --> P1["Returns(1,5,20) - % perubahan harga"]
    PA --> P2["Log Returns - untuk distribusi normal"]
    PA --> P3["Price Position - posisi dalam range BB"]
    PA --> P4["Higher High/Lower Low count - trend structure"]

    VOLF --> VF1["Volume SMA(20) - rata-rata volume"]
    VOLF --> VF2["Volume Ratio - current / average"]

    LAG --> L1["close_lag_1..5 - harga sebelumnya"]
    LAG --> L2["returns_lag_1..3 - return sebelumnya"]

    TIME --> TI1["Hour, Weekday - waktu candle"]
    TIME --> TI2["Session flags - london, ny, overlap"]

    M1 & M2 & M3 & V1 & V2 & V3 & T1 & T2 & T3 & P1 & P2 & P3 & P4 & VF1 & VF2 & L1 & L2 & TI1 & TI2 --> OUTPUT["Output: DataFrame + 40 kolom baru<br/>(semua numerik, siap ML)"]

Minimum data: 26 bar untuk semua indikator stabil

Tahap 3: SMC Analysis (Pola Institusi)

flowchart TD
    INPUT["Input: DataFrame dengan OHLCV"] --> SP["Swing Points (Fractal)<br/>Window: 11 bar (swing_length=5, +/-5 dari tengah)<br/>Output: swing_high (1/0), swing_low (-1/0), level harga"]
    INPUT --> FVG["Fair Value Gaps (FVG)<br/>Bullish: bar i-2 high < bar i low (gap up)<br/>Bearish: bar i-2 low > bar i high (gap down)<br/>Output: fvg_bull, fvg_bear, fvg_top, fvg_bottom, fvg_mid"]
    INPUT --> OB["Order Blocks (OB)<br/>Lookback: 10 bar<br/>Bullish: candle bearish terakhir sebelum move up besar<br/>Bearish: candle bullish terakhir sebelum move down besar<br/>Output: ob (1/-1), ob_top, ob_bottom, ob_mitigated"]
    INPUT --> BOS["Break of Structure (BOS)<br/>Harga break swing high/low = trend continuation<br/>Output: bos (1/-1), level yang di-break"]
    INPUT --> CHOCH["Change of Character (CHoCH)<br/>Harga break berlawanan arah trend = reversal signal<br/>Output: choch (1/-1), level yang di-break"]
    INPUT --> LIQ["Liquidity Zones<br/>BSL: Buy Side Liquidity (above swing highs)<br/>SSL: Sell Side Liquidity (below swing lows)<br/>Output: bsl_level, ssl_level"]

    SP & FVG & OB & BOS & CHOCH & LIQ --> SIG["Signal Generation<br/>Syarat: Structure break + (FVG ATAU Order Block)"]

    SIG --> ENTRY["Entry: harga saat ini"]
    SIG --> SL["SL: ATR-based, minimum 1.5 x ATR dari entry"]
    SIG --> TP["TP: 2:1 Risk-Reward minimum, cap 4 x ATR"]
    SIG --> CONF["Confidence: 40-85%<br/>(v5: calibrated weighted scoring)"]
    SIG --> REASON["Reason: BOS + Bullish FVG at 2645.50"]

Tahap 4: Regime Detection (HMM)

flowchart TD
    INPUT["Input: log_returns + normalized_range (volatilitas)"] -->|"GaussianHMM(n_components=3, lookback=500)"| REGIME["3 Regime Output"]

    REGIME --> R0["REGIME 0: Low Volatility<br/>Pasar tenang, range kecil<br/>Lot multiplier: 1.0x (normal)<br/>Rekomendasi: TRADE"]
    REGIME --> R1["REGIME 1: Medium Volatility<br/>Pasar aktif, trend jelas<br/>Lot multiplier: 1.0x (normal)<br/>Rekomendasi: TRADE"]
    REGIME --> R2["REGIME 2: High Volatility<br/>Pasar sangat volatile, berbahaya<br/>Lot multiplier: 0.5x (setengah)<br/>Rekomendasi: REDUCE"]
    REGIME --> RC["CRISIS (FlashCrashDetector)<br/>Move > 2.5% dalam 1 menit<br/>Lot multiplier: 0.0x (STOP)<br/>Rekomendasi: EMERGENCY CLOSE ALL"]

    style R0 fill:#4CAF50,color:#fff
    style R1 fill:#2196F3,color:#fff
    style R2 fill:#FF9800,color:#fff
    style RC fill:#F44336,color:#fff

Tahap 5: ML Prediction (XGBoost)

flowchart TD
    INPUT["Input: 24 fitur terpilih dari<br/>Feature Engineering + SMC + Regime"] -->|"XGBoost Binary Classifier<br/>max_depth=3, learning_rate=0.05<br/>min_child_weight=10, subsample=0.7<br/>colsample_bytree=0.6<br/>reg_alpha=1.0 (L1), reg_lambda=5.0 (L2)"| OUTPUT["Output:<br/>prob_up: 0.72 (probabilitas naik)<br/>prob_down: 0.28 (probabilitas turun)"]

    OUTPUT --> SIG["Signal: BUY (prob_up > 0.50)<br/>Confidence: 72%"]
    SIG --> TH1["prob > 0.50 - ada sinyal (minimum)"]
    SIG --> TH2["prob > 0.65 - sinyal kuat"]
    SIG --> TH3["prob > 0.75 - sinyal sangat kuat"]
    SIG --> TH4["prob > 0.80 - lot bisa naik ke 0.02"]

Tahap 6: Dynamic Confidence (Threshold Adaptif)

flowchart TD
    BASE["Base Score: 50"] --> SESS["Session Modifier<br/>Golden Time (20:00-23:59 WIB): +20<br/>London (15:00-23:59): +15<br/>New York (20:00-05:00): +10<br/>Tokyo/Sydney: +0<br/>Market Closed: -30"]
    BASE --> REG["Regime Modifier<br/>Medium Volatility: +15<br/>Low Volatility: +5<br/>High Volatility: -5<br/>Crisis: -25"]
    BASE --> VOLM["Volatility Modifier<br/>Medium (ideal): +10<br/>Low: +0<br/>High: -5<br/>Extreme: -10"]
    BASE --> TREN["Trend Modifier<br/>Trending (jelas): +10<br/>Ranging (sideways): -5"]
    BASE --> SMCC["SMC Confluence<br/>Ada konfluensi: +10<br/>Tidak ada: +0"]
    BASE --> MLC["ML Confidence<br/>ge 70%: +5<br/>ge 60%: +2<br/>lt 60%: +0"]

    SESS & REG & VOLM & TREN & SMCC & MLC --> TOTAL["Total Score (0-100)"]

    TOTAL --> EXC["EXCELLENT (ge 80 poin)<br/>Threshold: 60% (longgar)"]
    TOTAL --> GOOD["GOOD (65-79)<br/>Threshold: 65%"]
    TOTAL --> MOD["MODERATE (50-64)<br/>Threshold: 70%"]
    TOTAL --> POOR["POOR (35-49)<br/>Threshold: 80% (ketat)"]
    TOTAL --> AVOID["AVOID (lt 35)<br/>Threshold: 85% (SKIP)"]

    style EXC fill:#4CAF50,color:#fff
    style GOOD fill:#8BC34A,color:#fff
    style MOD fill:#FF9800,color:#fff
    style POOR fill:#FF5722,color:#fff
    style AVOID fill:#F44336,color:#fff

Contoh: Golden Time + Medium Vol + Trending + SMC + ML 72% = 50 + 20 + 15 + 10 + 10 + 10 + 5 = 120 (cap 100) = EXCELLENT -> Threshold 60% -> ML 72% PASS

---

5. Alur Entry: 14 Filter

Setiap sinyal harus melewati 14 gerbang berurutan. Satu saja gagal = TIDAK trading.

flowchart TD
    S["SINYAL SMC + ML MASUK"] --> F1{"Filter 1: Session<br/>Jam trading dibolehkan?<br/>BLOCK: 00:00-06:00 WIB (dead zone)<br/>BLOCK: Jumat ge 23:00 WIB"}
    F1 -->|"BLOCK"| SKIP1["SKIP"]
    F1 -->|"PASS: London/NY/Golden Time"| F2{"Filter 2: Risk Mode<br/>Bukan STOPPED?<br/>BLOCK: daily/total limit hit"}
    F2 -->|"BLOCK"| SKIP2["SKIP"]
    F2 -->|"PASS: NORMAL/RECOVERY/PROTECTED"| F3{"Filter 3: SMC Signal<br/>Ada setup SMC valid?<br/>BLOCK: Tidak ada FVG/OB + BOS/CHoCH"}
    F3 -->|"BLOCK"| SKIP3["SKIP"]
    F3 -->|"PASS: Ada sinyal BUY/SELL + SL + TP"| F4{"Filter 4: ML Confidence<br/>ML confidence ge dynamic threshold?<br/>Threshold 60-85% tergantung quality"}
    F4 -->|"BLOCK"| SKIP4["SKIP"]
    F4 -->|"PASS: confidence ge threshold"| F5{"Filter 5: ML Agreement<br/>ML TIDAK strongly disagree?<br/>BLOCK: ML > 65% berlawanan arah SMC"}
    F5 -->|"BLOCK"| SKIP5["SKIP"]
    F5 -->|"PASS: ML setuju atau netral"| F6{"Filter 6: Market Quality<br/>Dynamic Confidence bukan AVOID?<br/>BLOCK: Quality == AVOID (score lt 35)"}
    F6 -->|"BLOCK"| SKIP6["SKIP"]
    F6 -->|"PASS: EXCELLENT/GOOD/MODERATE/POOR"| F7{"Filter 7: Signal Confirmation<br/>Sinyal konsisten 2x berturut?<br/>BLOCK: Baru muncul 1x"}
    F7 -->|"BLOCK"| SKIP7["SKIP"]
    F7 -->|"PASS: 2x berturut"| F8{"Filter 8: Pullback (v5: ATR-based)<br/>Momentum selaras?<br/>BLOCK: RSI overbought/oversold<br/>BLOCK: MACD berlawanan<br/>BLOCK: Bounce > 15% ATR (3 candle)"}
    F8 -->|"BLOCK"| SKIP8["SKIP"]
    F8 -->|"PASS: Momentum selaras"| F9{"Filter 9: Trade Cooldown<br/>Sudah ge 5 menit sejak trade terakhir?<br/>BLOCK: lt 300 detik"}
    F9 -->|"BLOCK"| SKIP9["SKIP"]
    F9 -->|"PASS: ge 300 detik"| F10{"Filter 10: Position Limit<br/>Posisi terbuka lt limit?<br/>BLOCK: Sudah 2+ posisi terbuka"}
    F10 -->|"BLOCK"| SKIP10["SKIP"]
    F10 -->|"PASS: lt 2 posisi"| F11{"Filter 11: Lot Size<br/>Lot size > 0?<br/>BLOCK: Lot = 0 (crisis/risk tinggi)"}
    F11 -->|"BLOCK"| SKIP11["SKIP"]
    F11 -->|"PASS: Lot ge 0.01"| F12{"Filter 12: Flash Crash Guard<br/>TIDAK ada flash crash?<br/>BLOCK: Move > 2.5% dalam 1 menit"}
    F12 -->|"BLOCK + CLOSE ALL"| SKIP12["SKIP"]
    F12 -->|"PASS: Normal"| F13{"Filter 13: H1 Bias (#31B)<br/>EMA20 H1 selaras arah sinyal?<br/>BLOCK: BUY tapi harga lt EMA20 H1<br/>BLOCK: SELL tapi harga > EMA20 H1"}
    F13 -->|"BLOCK"| SKIP13["SKIP"]
    F13 -->|"PASS: Selaras"| F14{"Filter 14: Time Filter (#34A)<br/>Bukan jam transisi?<br/>BLOCK: Jam 9 atau 21 WIB"}
    F14 -->|"BLOCK"| SKIP14["SKIP"]
    F14 -->|"PASS"| EXEC["EXECUTE TRADE<br/>BUY atau SELL<br/>via MT5 Connector"]

    EXEC --> REG["Register ke SmartRiskManager"]
    EXEC --> LOG["Log ke TradeLogger (DB + CSV)"]
    EXEC --> TG["Kirim notifikasi Telegram"]

    style EXEC fill:#4CAF50,color:#fff,stroke:#388E3C,stroke-width:3px

---

6. Alur Exit: 12 Kondisi

Setiap posisi terbuka dievaluasi setiap ~10 detik (di antara candle) atau setiap candle baru (full analysis) terhadap 12 kondisi exit:

flowchart TD
    POS["POSISI TERBUKA<br/>(dicek setiap ~10 detik)<br/>Update: profit, momentum, peak, durasi"] --> K1{"Kondisi 1: Smart Take Profit<br/>(a) Profit ge $40 = hard TP<br/>(b) Profit ge $25 + momentum lt -30<br/>(c) Peak > $30, sekarang lt 60% peak<br/>(d) Profit ge $20 + TP prob lt 25%"}
    K1 -->|"TRIGGER"| CLOSE1["TUTUP: Smart TP"]
    K1 -->|"tidak trigger"| K2{"Kondisi 2: Early Exit (Profit Kecil)<br/>Profit $5-$15 + ML reversal ge 65%<br/>+ momentum lt -50"}
    K2 -->|"TRIGGER"| CLOSE2["TUTUP: Amankan profit kecil"]
    K2 -->|"tidak trigger"| K3{"Kondisi 3: Early Cut<br/>(v4: Smart Hold DIHAPUS)<br/>Loss ge 30% max ($15)<br/>DAN momentum lt -30"}
    K3 -->|"TRIGGER"| CLOSE3["TUTUP CEPAT: Early cut"]
    K3 -->|"tidak trigger"| K4{"Kondisi 4: ML Trend Reversal<br/>ML confidence ge 65%<br/>BERLAWANAN ARAH posisi<br/>+ 3x warning berturut-turut"}
    K4 -->|"TRIGGER"| CLOSE4["TUTUP: AI deteksi pembalikan"]
    K4 -->|"tidak trigger"| K5{"Kondisi 5: Maximum Loss<br/>Loss ge 50% dari max_loss_per_trade<br/>($25 dari $50)"}
    K5 -->|"TRIGGER"| CLOSE5["TUTUP: Kerugian terlalu besar"]
    K5 -->|"tidak trigger"| K6{"Kondisi 6: Stall Detection<br/>Posisi 10+ bar<br/>tanpa profit signifikan"}
    K6 -->|"TRIGGER"| CLOSE6["TUTUP: Pasar tidak bergerak"]
    K6 -->|"tidak trigger"| K7{"Kondisi 7: Daily Limit<br/>Total daily loss<br/>mendekati limit"}
    K7 -->|"TRIGGER"| CLOSE7["TUTUP SEMUA: Proteksi modal"]
    K7 -->|"tidak trigger"| K8{"Kondisi 8: Weekend Close<br/>Dekat market close weekend<br/>+ ada posisi profit"}
    K8 -->|"TRIGGER"| CLOSE8["TUTUP: Hindari gap risk"]
    K8 -->|"tidak trigger"| K9{"Kondisi 9: Smart Time-Based<br/>(v5: Dont Cut Winners)<br/>(a) > 4 jam + no growth = TUTUP<br/>(b) > 4 jam + growing + ML = HOLD<br/>(c) > 6 jam + profit lt $10 = TUTUP<br/>(d) > 6 jam + profit > $10 = extend 8j<br/>(e) > 8 jam = TUTUP final"}
    K9 -->|"TRIGGER"| CLOSE9["TUTUP: Time-based"]
    K9 -->|"tidak trigger"| K10{"Kondisi 10: Trailing Stop Loss<br/>Profit ge 25 pips?<br/>SL ikuti harga jarak 10 pips"}
    K10 -->|"TRIGGER"| CLOSE10["TRAILING: Kunci profit"]
    K10 -->|"tidak trigger"| K11{"Kondisi 11: Breakeven Protection<br/>Profit ge 15 pips?<br/>SL ke entry + 2 poin buffer"}
    K11 -->|"TRIGGER"| CLOSE11["BREAKEVEN: Posisi aman"]
    K11 -->|"tidak trigger"| K12["Kondisi 12: Default HOLD<br/>Tidak ada kondisi terpenuhi<br/>Biarkan posisi berjalan"]

    style CLOSE1 fill:#F44336,color:#fff
    style CLOSE2 fill:#F44336,color:#fff
    style CLOSE3 fill:#F44336,color:#fff
    style CLOSE4 fill:#F44336,color:#fff
    style CLOSE5 fill:#F44336,color:#fff
    style CLOSE6 fill:#F44336,color:#fff
    style CLOSE7 fill:#F44336,color:#fff
    style CLOSE8 fill:#F44336,color:#fff
    style CLOSE9 fill:#F44336,color:#fff
    style CLOSE10 fill:#FF9800,color:#fff
    style CLOSE11 fill:#FF9800,color:#fff
    style K12 fill:#4CAF50,color:#fff

Position Manager (Tambahan per Posisi)

Selain 12 kondisi di atas, Position Manager juga menjalankan:

flowchart TD
    PM["Position Manager (per posisi)"] --> MCH{"Market Close Handler<br/>(Prioritas Tertinggi)<br/>Dekat close harian/weekend?"}
    MCH -->|"Profit ge $10 + dekat close"| CLOSE_MC["TUTUP (amankan)"]
    MCH -->|"Loss + weekend + SL > 50%"| CLOSE_GAP["TUTUP (gap risk)"]
    MCH -->|"Loss kecil + weekend"| HOLD_MC["HOLD (bisa recovery)"]
    MCH -->|"Tidak dekat close"| BE{"Breakeven Protection<br/>Profit ge 15 pips?"}
    BE -->|"Ya"| BE_ACT["Pindah SL ke entry + 2 buffer<br/>(posisi tidak bisa rugi lagi)"]
    BE -->|"Tidak"| TS{"Trailing Stop<br/>Profit ge 25 pips?"}
    TS -->|"Ya"| TS_ACT["SL mengikuti harga, jarak 10 pips<br/>(kunci profit sambil biarkan berjalan)"]
    TS -->|"Tidak"| CONT["Lanjut monitoring"]

---

7. Sistem Proteksi Risiko 4 Lapis

flowchart TD
    subgraph L1["LAPIS 1: BROKER STOP LOSS (Otomatis oleh MT5)"]
        L1D["SL = Entry +/- (1.5 x ATR), minimum 10 pips<br/>Dikirim bersama order ke broker<br/>Aktif 24/7, bahkan jika bot mati<br/>Max loss: ~$50-80 per trade"]
    end
    subgraph L2["LAPIS 2: SOFTWARE SMART EXIT (Bot evaluasi setiap ~10 detik)"]
        L2D["12 kondisi exit (lihat bagian 6)<br/>Biasanya menutup SEBELUM broker SL kena<br/>Target close: loss le $25 (lebih ketat dari broker)<br/>Termasuk: momentum, ML reversal, stall, time limit"]
    end
    subgraph L3["LAPIS 3: EMERGENCY STOP LOSS (Backup jika software gagal)"]
        L3D["Max loss per trade: 2% modal ($100 untuk $5K)<br/>Diset sebagai broker SL terpisah<br/>Aktif jika software error/hang"]
    end
    subgraph L4["LAPIS 4: CIRCUIT BREAKER (Hentikan semua trading)"]
        L4D["Trigger 1: Daily loss ge 3% ($150) = Stop hari ini<br/>Trigger 2: Total loss ge 10% ($500) = Stop total<br/>Trigger 3: Flash crash > 2.5% / 1 menit = CLOSE ALL<br/>Reset: Otomatis hari baru (daily), manual (total)"]
    end

    L1 --> L2 --> L3 --> L4

    style L1 fill:#4CAF50,color:#fff
    style L2 fill:#2196F3,color:#fff
    style L3 fill:#FF9800,color:#fff
    style L4 fill:#F44336,color:#fff

4 Mode Trading (Smart Risk Manager)

flowchart TD
    NORMAL["MODE: NORMAL<br/>Kondisi: Semua aman, tidak ada masalah<br/>Lot: 0.01 - 0.02 (berdasarkan confidence)<br/>Max posisi: 2-3"]
    NORMAL -->|"3x loss berturut-turut"| RECOVERY["MODE: RECOVERY<br/>Kondisi: Setelah kerugian beruntun<br/>Lot: 0.01 (minimum saja)<br/>Max posisi: 1"]
    RECOVERY -->|"mendekati 80% daily limit"| PROTECTED["MODE: PROTECTED<br/>Kondisi: Hampir kena daily limit<br/>Lot: 0.01 (minimum saja)<br/>Max posisi: 1"]
    PROTECTED -->|"daily/total limit tercapai"| STOPPED["MODE: STOPPED<br/>Kondisi: Batas kerugian tercapai<br/>Lot: 0 (TIDAK BOLEH trading)<br/>Max posisi: 0 (tutup semua)<br/>Reset: Otomatis hari baru"]

    style NORMAL fill:#4CAF50,color:#fff
    style RECOVERY fill:#FF9800,color:#fff
    style PROTECTED fill:#FF5722,color:#fff
    style STOPPED fill:#F44336,color:#fff

Lot Sizing: Risk-Constrained Half-Kelly

Langkah 1: Hitung Kelly Fraction
    f* = (win_rate × avg_rr - (1 - win_rate)) / avg_rr

    Contoh: win_rate=55%, avg_rr=2.0
    f* = (0.55 × 2.0 - 0.45) / 2.0 = 0.325 (32.5%)

Langkah 2: Cap Kelly (max 25%)
    f* = min(0.325, 0.25) = 0.25

Langkah 3: Half-Kelly (safety)
    f* = 0.25 × 0.5 = 0.125 (12.5%)

Langkah 4: Apply regime multiplier
    High volatility: × 0.5 = 0.0625
    Normal: × 1.0 = 0.125

Langkah 5: Cap di config limit
    config risk_per_trade = 1%
    actual_risk = min(0.125, 0.01) = 0.01 (1%)

Langkah 6: Hitung lot
    risk_amount = $5000 × 1% = $50
    SL distance = 50 pips → pip_value ~$1/pip/0.01lot
    lot = $50 / (50 × $1) = 0.01 lot

Langkah 7: ML Confidence boost
    ML ≥ 80% → lot × 2 = 0.02 lot (maximum)
    ML < 65% → lot = 0.01 (minimum)

Langkah 8: Session multiplier
    Golden Time: × 1.2
    Sydney: × 0.5

    Final lot: 0.01 - 0.02 (ultra-konservatif)

---

8. AI/ML Engine

Hidden Markov Model (HMM) — Otak Regime

flowchart TD
    subgraph HMM["HIDDEN MARKOV MODEL"]
        direction TB
        INFO["Library: hmmlearn.GaussianHMM<br/>Input: log_returns + rolling_volatility (2 fitur)<br/>States: 3 (Low, Medium, High Volatility)<br/>Lookback: 500 bar untuk training<br/>Retrain: setiap 20 bar (auto-update)"]
        TRANS["Transition Matrix (contoh):<br/>Fr Low: To Low 0.85, To Med 0.12, To High 0.03<br/>Fr Med: To Low 0.10, To Med 0.80, To High 0.10<br/>Fr High: To Low 0.05, To Med 0.15, To High 0.80"]
        EMISI["Distribusi Emisi (per state):<br/>Low: mu_return ~ 0, sigma = kecil<br/>Med: mu_return ~ 0, sigma = sedang<br/>High: mu_return ~ 0, sigma = besar"]
        OUTPUT["Output:<br/>regime: 0/1/2 (low/medium/high)<br/>confidence: 0.0 - 1.0<br/>lot_multiplier: 1.0 / 0.5 / 0.0<br/>recommendation: TRADE / REDUCE / SLEEP"]
        INFO --> TRANS --> EMISI --> OUTPUT
    end

XGBoost — Otak Prediksi

flowchart TD
    subgraph XGB["XGBOOST BINARY CLASSIFIER"]
        direction TB
        INFO2["Library: xgboost<br/>Objective: binary:logistic<br/>Target: UP (1) / DOWN (0) pada bar berikutnya"]
        CONFIG2["Anti-Overfitting Config:<br/>max_depth: 3 (shallow trees)<br/>learning_rate: 0.05 (slow learning)<br/>min_child_weight: 10 (min samples/leaf)<br/>subsample: 0.7 (70% data/tree)<br/>colsample_bytree: 0.6 (60% features/tree)<br/>reg_alpha: 1.0 (L1), reg_lambda: 5.0 (L2)<br/>gamma: 1.0 (min loss reduction)<br/>num_boost_round: 50 (few rounds)"]
        FITUR2["24 Fitur Input (Top 10):<br/>1. RSI(14), 2. MACD_histogram<br/>3. ATR(14), 4. bb_width<br/>5. returns_1, 6. price_position<br/>7. volatility_20, 8. returns_5<br/>9. ema_cross, 10. regime"]
        OUT2["Output:<br/>signal: BUY / SELL / HOLD<br/>probability: 0.0-1.0 (prob of UP)<br/>confidence: 0.0-1.0 (prob of winning side)"]
        VAL2["Validation:<br/>Train/Test: 70%/30% (50-bar gap, anti leakage)<br/>Walk-forward: 500 train / 50 test / 50 step<br/>Target AUC: > 0.65<br/>Rollback AUC: lt 0.60 (v4: dinaikkan dari 0.52)<br/>Overfitting ratio: train_AUC/test_AUC lt 1.15"]
        INFO2 --> CONFIG2 --> FITUR2 --> OUT2 --> VAL2
    end

Kombinasi Sinyal (SMC + ML)

flowchart TD
    SMC_SIG["SMC Signal: BUY at 2645, SL 2635, TP 2665, conf 75%"]
    ML_SIG["ML Signal: BUY, confidence 72%"]

    SMC_SIG & ML_SIG --> COMBINE{"KOMBINASI SMC + ML"}

    COMBINE --> C1["CASE 1: SMC BUY + ML BUY (ge 50%)<br/>Combined confidence = (75%+72%)/2 = 73.5%<br/>= ENTRY (jika pass 14 filter)"]
    COMBINE --> C2["CASE 2: SMC BUY + ML SELL (ge 65%)<br/>ML strongly disagrees = BLOCK (filter #5)<br/>= TIDAK entry"]
    COMBINE --> C3["CASE 3: SMC BUY + ML uncertain (lt 50%)<br/>ML tidak yakin = BLOCK (filter #4)<br/>= TIDAK entry"]
    COMBINE --> C4["CASE 4: Tidak ada SMC signal<br/>Tidak ada entry point = SKIP<br/>SMC adalah sinyal PRIMER (wajib ada)"]

    C1 & C2 & C3 & C4 --> PRINSIP["PRINSIP:<br/>SMC = sinyal UTAMA (entry/SL/TP)<br/>ML = KONFIRMASI (bisa blokir, tidak bisa inisiasi)<br/>HMM = PENYESUAI (mengatur agresivitas)"]

    style C1 fill:#4CAF50,color:#fff
    style C2 fill:#F44336,color:#fff
    style C3 fill:#F44336,color:#fff
    style C4 fill:#9E9E9E,color:#fff

---

9. Smart Money Concepts (SMC)

6 Konsep yang Dianalisis

flowchart TD
    subgraph SP["1. SWING POINTS (Fractal High/Low)"]
        SP_D["Swing High: titik tertinggi dalam window 11 bar<br/>5 bar kiri lebih rendah, 5 bar kanan lebih rendah<br/><br/>Swing Low: titik terendah dalam window 11 bar<br/>5 bar kiri lebih tinggi, 5 bar kanan lebih tinggi"]
    end
    subgraph FVG2["2. FAIR VALUE GAP (FVG) - Ketidakseimbangan Harga"]
        FVG_D["Bullish FVG (gap up): Bar i-2 high lt Bar i low (ada gap)<br/>Bearish FVG (gap down): Bar i-2 low > Bar i high<br/>Harga cenderung kembali mengisi FVG = entry zone"]
    end
    subgraph OB2["3. ORDER BLOCK (OB) - Zona Institusi"]
        OB_D["Bullish OB: candle bearish terakhir sebelum rally<br/>(zona dimana institusi menempatkan buy order besar)<br/>Lookback: 10 bar untuk deteksi<br/>Mitigated: true jika harga sudah revisit"]
    end
    subgraph BOS2["4. BREAK OF STRUCTURE (BOS) - Kelanjutan Trend"]
        BOS_D["Uptrend BOS: harga break di ATAS swing high sebelumnya<br/>= Trend bullish berlanjut<br/>Downtrend BOS: harga break di BAWAH swing low<br/>= Trend bearish berlanjut"]
    end
    subgraph CHOCH2["5. CHANGE OF CHARACTER (CHoCH) - Perubahan Trend"]
        CHOCH_D["Uptrend ke Downtrend:<br/>Harga break di BAWAH swing low terakhir<br/>= Trend berubah dari bullish ke bearish<br/>(sinyal reversal)"]
    end
    subgraph LIQ2["6. LIQUIDITY ZONES - Target Likuiditas"]
        LIQ_D["BSL (Buy Side Liquidity): di atas swing highs<br/>(stop loss para seller berkumpul)<br/>SSL (Sell Side Liquidity): di bawah swing lows<br/>(stop loss para buyer berkumpul)<br/>Institusi sering hunt liquidity zone ini"]
    end

Signal Generation

flowchart TD
    SB["Structure Break<br/>(BOS atau CHoCH)"] --> VALID["VALID SIGNAL"]
    ZN["Zone<br/>(FVG atau Order Block)"] --> VALID

    VALID --> BUY_SIG["BUY Signal:<br/>BOS bullish ATAU CHoCH bearish-to-bullish<br/>+ Bullish FVG ATAU Bullish OB di bawah harga<br/>Entry: harga saat ini<br/>SL: di bawah zone, minimum 1.5 x ATR<br/>TP: 2:1 R:R, maximum 4 x ATR<br/>Confidence: 40-85% (v5: calibrated weighted scoring)"]
    VALID --> SELL_SIG["SELL Signal:<br/>BOS bearish ATAU CHoCH bullish-to-bearish<br/>+ Bearish FVG ATAU Bearish OB di atas harga<br/>Entry: harga saat ini<br/>SL: di atas zone, minimum 1.5 x ATR<br/>TP: 2:1 R:R, maximum 4 x ATR<br/>Confidence: 40-85% (v5: calibrated weighted scoring)"]

    style BUY_SIG fill:#4CAF50,color:#fff
    style SELL_SIG fill:#F44336,color:#fff

---

10. Position Lifecycle

Dari Lahir Sampai Mati (Siklus Hidup Posisi)

flowchart TD
    T1["TAHAP 1: SINYAL TERDETEKSI<br/>SMC menemukan setup + ML konfirmasi + 14 filter PASS<br/>Keputusan: BUKA POSISI"]
    T1 --> T2["TAHAP 2: LOT SIZE CALCULATION<br/>Risk Engine (Kelly): Balance $5000 x 1% = $50 max loss<br/>SL distance 50 pips = lot 0.01<br/>ML ge 80% = 0.02 lot, ML lt 65% = 0.01 lot<br/>Session: Golden x1.2, Sydney x0.5<br/>Regime: Normal x1.0, High Vol x0.5, Crisis x0.0"]
    T2 --> T3["TAHAP 3: ORDER VALIDATION<br/>SL di sisi benar (BUY: SL lt entry)<br/>TP di sisi benar (BUY: TP > entry)<br/>Lot dalam range (0.01 - 0.05)<br/>Entry dekat harga saat ini (lt 0.1%)<br/>Risk% le 1.5x config limit<br/>Circuit breaker TIDAK aktif"]
    T3 --> T4["TAHAP 4: ORDER EXECUTION<br/>MT5 Connector mengirim order:<br/>Symbol: XAUUSD, Type: BUY/SELL<br/>Lot: 0.01-0.02, SL: ATR-based<br/>TP: 2:1 R:R, Deviation: 20 points<br/>Retry: max 3 attempts"]
    T4 --> T4B["TAHAP 4b: POST-EXECUTION VALIDATION (v5)<br/>Slippage Validation: max 0.15% (~$4 XAUUSD)<br/>Log WARNING jika melebihi batas<br/>Gunakan harga AKTUAL untuk tracking<br/>Partial Fill: cek volume, update lot aktual"]
    T4B --> T5["TAHAP 5: POSITION REGISTERED<br/>Smart Risk Manager: catat entry AKTUAL, peak=0<br/>Trade Logger: insert PostgreSQL (30+ field) + CSV<br/>Telegram: notifikasi trade open<br/>(entry, SL, TP, R:R, confidence, regime)"]
    T5 --> T6["TAHAP 6: ACTIVE MONITORING<br/>(setiap ~10 detik + candle baru)<br/>Update profit/loss, peak profit, momentum<br/>Hitung TP probability<br/>Cek 12 kondisi exit<br/>Cek Position Manager (trailing, breakeven)<br/>Cek Market Close Handler"]
    T6 -->|"trigger close"| T7["TAHAP 7: POSITION CLOSED<br/>MT5: Close via market order<br/>Risk Manager: record P/L, update counters,<br/>check mode transition<br/>Logger: update exit price, profit, duration, reason<br/>Telegram: notifikasi trade close"]
    T6 -->|"HOLD"| T6

    style T1 fill:#1565C0,color:#fff
    style T7 fill:#C62828,color:#fff

---

11. Auto-Retraining & Model Management

Lifecycle Model AI

flowchart TD
    INIT["INITIAL TRAINING (train_models.py)<br/>Dijalankan 1x saat setup"]
    INIT --> I1["1. Fetch 10,000 bar M15 dari MT5 (~104 hari)"]
    I1 --> I2["2. Feature Engineering = 40+ fitur"]
    I2 --> I3["3. SMC Analysis = struktur pasar"]
    I3 --> I4["4. Create target UP/DOWN (lookahead=1)<br/>4b. Split 70/30, 50-bar gap (anti leakage)"]
    I4 --> I5["5. Train HMM (3 regime, lookback=500)"]
    I5 --> I6["6. Train XGBoost (50 rounds, early_stop=5)"]
    I6 --> I7["7. Walk-forward validation (500/50/50)"]
    I7 --> I8["8. Save: hmm_regime.pkl + xgboost_model.pkl"]

    I8 --> DAILY["DAILY AUTO-RETRAINING (Auto Trainer)<br/>Otomatis setiap hari 05:00 WIB"]

    DAILY --> SCHED["Schedule:<br/>Harian (05:00 WIB): 8,000 bar, 50 rounds<br/>Weekend (Sabtu): 15,000 bar, 80 rounds (deep)<br/>Emergency: jika AUC lt 0.65"]

    SCHED --> D1["1. Backup model saat ini"]
    D1 --> D2["2. Fetch data baru dari MT5"]
    D2 --> D3["3. Feature Engineering + SMC"]
    D3 --> D4["4. Train HMM baru + XGBoost baru"]
    D4 --> D5{"5. Validasi: test AUC ge 0.60?"}
    D5 -->|"Ya"| SAVE["Save model baru, reload di memory"]
    D5 -->|"Tidak"| ROLLBACK["ROLLBACK ke model sebelumnya"]
    SAVE --> D6["6. Log hasil ke PostgreSQL"]
    ROLLBACK --> D6
    D6 --> D7["7. Kirim laporan via Telegram"]

    D7 --> SAFETY["Safety:<br/>Max 5 backup (rotasi)<br/>Min 20 jam antar retrain (cooldown)<br/>Auto-rollback jika AUC lt 0.60<br/>Model lama selalu tersedia"]

Perbandingan Initial vs Auto Training

Aspek	train_models.py	Auto Trainer
Kapan	Manual, 1x setup	Otomatis, harian
Data	10,000 bar	8K (harian) / 15K (weekend)
Boost rounds	50	50 (harian) / 80 (weekend)
Walk-forward	Ya	Tidak
Backup	Tidak	Ya (5 terakhir)
Rollback	Tidak	Ya (AUC < 0.60)
Database	Tidak	Ya (PostgreSQL)
Tujuan	Setup awal	Maintenance rutin

---

12. Infrastruktur & Database

PostgreSQL Schema

flowchart TD
    DB["trading_db"] --> T["trades<br/>Semua trade: open, close, profit, SMC, ML, features"]
    DB --> TR["training_runs<br/>Log setiap training: AUC, akurasi, durasi, rollback"]
    DB --> SG["signals<br/>Setiap sinyal yang dihasilkan: executed atau tidak"]
    DB --> MS["market_snapshots<br/>Snapshot periodik: harga, regime, volatilitas"]
    DB --> BS["bot_status<br/>Status bot: uptime, loop count, balance, risk mode"]
    DB --> DS["daily_summaries<br/>Ringkasan harian: win rate, profit factor, per sesi"]

Tabel trades (Detail)

-- Identifikasi
ticket, symbol, direction (BUY/SELL)

-- Harga
entry_price, exit_price, stop_loss, take_profit

-- Hasil
lot_size, profit_usd, profit_pips
opened_at, closed_at, duration_seconds

-- Konteks Entry
entry_regime, entry_volatility, entry_session
smc_signal, smc_confidence, smc_reason
smc_fvg_detected, smc_ob_detected, smc_bos_detected, smc_choch_detected
ml_signal, ml_confidence
market_quality, market_score, dynamic_threshold

-- Konteks Exit
exit_reason, exit_regime, exit_ml_signal

-- Keuangan
balance_before, balance_after, equity_at_entry

-- Data Lengkap
features_entry (JSON), features_exit (JSON)
bot_version, trade_mode

Connection Architecture

flowchart TD
    TL["TradeLogger"] -->|"TradeRepository, SignalRepository,<br/>MarketSnapshotRepository"| DBC["DatabaseConnection (Singleton)"]
    AT2["AutoTrainer"] -->|"TrainingRepository"| DBC
    ML["main_live.py"] -->|"BotStatusRepository,<br/>DailySummaryRepository"| DBC
    DASH["Dashboard"] -->|"Semua repository (READ)"| DBC

    DBC --> POOL["ThreadedConnectionPool (1-10 koneksi)"]
    POOL --> PG["PostgreSQL Server"]

Graceful Degradation

flowchart TD
    CHECK{"PostgreSQL tersedia?"}
    CHECK -->|"Ya"| DUAL["Gunakan DB + CSV backup (dual write)"]
    CHECK -->|"Tidak"| CSV["CSV saja (bot tetap berjalan 100%)"]
    DUAL --> NOTE["Bot TIDAK PERNAH crash karena database.<br/>Semua operasi DB dibungkus try-except."]
    CSV --> NOTE

---

13. Konfigurasi & Parameter Kritis

Configuration System

flowchart TD
    ENV[".env file"] --> TC["TradingConfig.from_env()"]

    TC --> RC["RiskConfig<br/>risk_per_trade: 1.0% (SMALL) / 0.5% (MEDIUM)<br/>max_daily_loss: 3.0% (SMALL) / 2.0% (MEDIUM)<br/>max_total_loss: 10.0%<br/>max_positions: 3 (SMALL) / 5 (MEDIUM)<br/>min_lot: 0.01<br/>max_lot: 0.05 (SMALL) / 2.0 (MEDIUM)<br/>max_leverage: 1:100 (SMALL) / 1:30 (MEDIUM)"]

    TC --> SC["SMCConfig<br/>swing_length: 5<br/>fvg_min_gap_pips: 2.0<br/>ob_lookback: 10<br/>bos_close_break: true"]

    TC --> MLC2["MLConfig<br/>confidence_threshold: 0.65<br/>entry_confidence: 0.70<br/>high_confidence: 0.75<br/>very_high_confidence: 0.80<br/>retrain_frequency_days: 7"]

    TC --> THC["ThresholdsConfig<br/>ml_min_confidence: 0.65<br/>ml_high_confidence: 0.75<br/>trade_cooldown_seconds: 300<br/>min_profit_to_secure: $15<br/>good_profit: $25, great_profit: $40<br/>flash_crash_threshold: 2.5%<br/>sydney_lot_multiplier: 0.5"]

    TC --> RGC["RegimeConfig<br/>n_regimes: 3<br/>lookback: 500<br/>retrain_frequency: 20"]

Capital Mode Auto-Detection

flowchart TD
    BAL{"Balance?"}
    BAL -->|"le $10,000"| SMALL["SMALL MODE<br/>Risk: 1% per trade ($50 pada $5K)<br/>Daily limit: 3% ($150)<br/>Lot: 0.01-0.05<br/>Leverage: 1:100<br/>Timeframe: M15<br/>Max posisi: 3"]
    BAL -->|"> $10,000"| MEDIUM["MEDIUM MODE<br/>Risk: 0.5% per trade<br/>Daily limit: 2%<br/>Lot: 0.01-2.0<br/>Leverage: 1:30<br/>Timeframe: H1<br/>Max posisi: 5"]

Session Schedule (WIB = GMT+7)

flowchart LR
    subgraph SESSION["Session Schedule (WIB = GMT+7)"]
        direction TB
        DZ["00:00-04:00 DEAD ZONE<br/>BLOCKED - Likuiditas rendah"]
        RO["04:00-06:00 ROLLOVER<br/>BLOCKED - Spread melebar"]
        SY["06:00-07:00 Sydney<br/>0.5x - Pasar baru buka"]
        TK["07:00-13:00 Tokyo+Sydney<br/>0.7x - Asia aktif"]
        TA["13:00-15:00 Tokyo akhir<br/>0.7x - Transisi"]
        LD["15:00-20:00 London<br/>1.0x - Volatilitas naik"]
        GT["20:00-23:59 GOLDEN TIME<br/>1.2x - London+NY overlap"]
        WR["Jumat ge 23:00 WEEKEND RISK<br/>BLOCKED - Gap risk"]
    end

    style DZ fill:#F44336,color:#fff
    style RO fill:#F44336,color:#fff
    style SY fill:#FF9800,color:#fff
    style TK fill:#FFC107,color:#000
    style TA fill:#FFC107,color:#000
    style LD fill:#2196F3,color:#fff
    style GT fill:#4CAF50,color:#fff
    style WR fill:#F44336,color:#fff

Golden Time (20:00-23:59 WIB) = waktu paling optimal

• Spread ketat, likuiditas maksimal, volatilitas ideal
• Lot multiplier 1.2x (bonus)
• v4: Smart Hold dihapus -- tidak ada lagi hold losers menunggu sesi tertentu

---

14. Performa & Timing

Main Loop Breakdown

Target: < 50ms per iterasi analisis

FULL ANALYSIS (saat candle baru M15):

Komponen	Waktu	Keterangan
MT5 data fetch	~10ms	200 bar M15 via API
Feature engineering	~5ms	40+ fitur, Polars
SMC analysis	~5ms	6 konsep, Polars native
HMM predict	~2ms	2 fitur -> 1 regime
XGBoost predict	~3ms	24 fitur -> 1 signal
Position monitoring	~5ms	Per posisi terbuka
Entry logic	~5ms	14 filter check
Overhead	~15ms	Logging, state update
TOTAL	~50ms

POSITION CHECK ONLY (di antara candle, setiap ~10 detik):

Komponen	Waktu	Keterangan
MT5 data fetch	~5ms	50 bar saja
Feature engineering	~3ms	Minimal fitur
ML prediction	~3ms	Untuk exit evaluation
Position evaluation	~5ms	12 kondisi exit
Overhead	~5ms	Logging
TOTAL	~21ms

Timer Periodik

Event	Interval	Cara Trigger
Full analysis + entry	Candle baru M15	Deteksi candle
Position monitoring	~10 detik	Di antara candle
Performance logging	4 candle (~1j)	candle_count % 4
Auto-retrain check	20 candle (~5j)	candle_count % 20
Market update Telegram	30 menit	Timer
Hourly analysis	1 jam	Timer
Daily summary + reset	Ganti hari	Date check

---

15. Error Handling & Fault Tolerance

Prinsip: Bot TIDAK PERNAH Crash

flowchart TD
    subgraph LV1["LEVEL 1: Per-Loop Error Handling"]
        LV1D["try: Fetch data, analyze, trade<br/>except ConnectionError: reconnect()<br/>except Exception: Log error, lanjut loop berikutnya<br/>Bot TIDAK crash dari error tunggal"]
    end
    subgraph LV2["LEVEL 2: MT5 Auto-Reconnect"]
        LV2D["MT5 putus?<br/>Attempt 1: reconnect (tunggu 2 detik)<br/>Attempt 2: reconnect (tunggu 4 detik)<br/>Attempt 3: reconnect (tunggu 8 detik)<br/>Cooldown 60 detik, Retry cycle (max 5/cooldown)<br/>Selama disconnected: monitoring PAUSE,<br/>entry DITUNDA, posisi dilindungi broker SL"]
    end
    subgraph LV3["LEVEL 3: Database Graceful Degradation"]
        LV3D["PostgreSQL down?<br/>Switch ke CSV-only mode<br/>Semua data tetap dicatat<br/>Trading tetap berjalan normal<br/>Retry DB connection periodik"]
    end
    subgraph LV4["LEVEL 4: Telegram Failure"]
        LV4D["Telegram API error?<br/>Log error secara silent<br/>Trading tetap jalan 100%<br/>Retry di notifikasi berikutnya"]
    end
    subgraph LV5["LEVEL 5: Model File Missing"]
        LV5D[".pkl file tidak ditemukan?<br/>Log warning<br/>Skip prediksi (ML/HMM)<br/>Trading bisa jalan tanpa ML (SMC only)<br/>Trigger: jalankan train_models.py"]
    end
    subgraph LV6["LEVEL 6: Flash Crash Protection"]
        LV6D["Harga bergerak > 2.5% dalam 1 menit?<br/>EMERGENCY: Close ALL positions<br/>Circuit breaker AKTIF<br/>Kirim alert KRITIS via Telegram<br/>Bot masuk mode STOPPED"]
    end

    LV1 --> LV2 --> LV3 --> LV4 --> LV5 --> LV6

Startup & Shutdown

flowchart TD
    subgraph STARTUP["STARTUP SEQUENCE"]
        direction TB
        S1["1. Load konfigurasi dari .env"] --> S2["2. Connect ke MT5 (max 3 retry)"]
        S2 --> S3["3. Load model HMM dari models/hmm_regime.pkl"]
        S3 --> S4["4. Load model XGBoost dari models/xgboost_model.pkl"]
        S4 --> S5["5. Initialize SmartRiskManager (set balance, limits)"]
        S5 --> S6["6. Initialize SessionFilter (WIB timezone)"]
        S6 --> S7["7. Initialize TelegramNotifier"]
        S7 --> S8["8. Initialize TradeLogger (connect DB)"]
        S8 --> S9["9. Initialize AutoTrainer"]
        S9 --> S10["10. Send Telegram: BOT STARTED"]
        S10 --> S11["11. Mulai main loop"]
    end

    subgraph SHUTDOWN["SHUTDOWN SEQUENCE (SIGINT/SIGTERM)"]
        direction TB
        D1["1. Signal diterima"] --> D2["2. Hentikan loop utama"]
        D2 --> D3["3. Kirim Telegram: BOT STOPPED"]
        D3 --> D4["4. Disconnect MT5"]
        D4 --> D5["5. Close database connections"]
        D5 --> D6["6. Exit"]
    end

---

16. Daftar File Source Code

flowchart TD
    ROOT["Smart Automatic Trading BOT + AI/"]
    ROOT --> MAIN["main_live.py - Orchestrator utama"]
    ROOT --> TRAIN["train_models.py - Script training awal"]
    ROOT --> ENVF[".env - Environment variables"]

    ROOT --> SRC["src/"]
    SRC --> CFG["config.py - Konfigurasi terpusat (6 sub-config)"]
    SRC --> MT5F["mt5_connector.py - Bridge ke MetaTrader 5"]
    SRC --> FEF["feature_eng.py - Feature Engineering (40+ fitur)"]
    SRC --> RDF["regime_detector.py - HMM Regime Detection"]
    SRC --> MLF["ml_model.py - XGBoost Signal Predictor"]
    SRC --> SMCF["smc_polars.py - Smart Money Concepts (6 konsep)"]
    SRC --> SRMF["smart_risk_manager.py - 4-Mode Risk Manager"]
    SRC --> REF["risk_engine.py - Kelly Criterion + Circuit Breaker"]
    SRC --> SFF["session_filter.py - Session Time Filter (WIB)"]
    SRC --> DCF["dynamic_confidence.py - Dynamic Threshold Manager"]
    SRC --> NAF["news_agent.py - News Event Monitor"]
    SRC --> TNF["telegram_notifier.py - Telegram Push Notifications"]
    SRC --> ATF["auto_trainer.py - Daily Auto-Retraining"]
    SRC --> TLF["trade_logger.py - Dual Storage Logger (DB+CSV)"]
    SRC --> PMF["position_manager.py - Position Manager + Market Close"]
    SRC --> DBD["db/"]
    DBD --> DBINIT["__init__.py - DB exports"]
    DBD --> DBCONN["connection.py - PostgreSQL Singleton + Pool"]
    DBD --> DBREPO["repository.py - 6 Repository classes"]

    ROOT --> MODELS["models/"]
    MODELS --> XGBM["xgboost_model.pkl - Trained XGBoost model"]
    MODELS --> HMMM["hmm_regime.pkl - Trained HMM model"]
    MODELS --> BKUP["backup/ - Auto-backup (5 terakhir)"]

    ROOT --> DATA["data/"]
    DATA --> TDATA["training_data.parquet"]
    DATA --> TLOGS["trade_logs/ - CSV backup (per bulan)"]

    ROOT --> BTESTS["backtests/"]
    BTESTS --> BTSYNC["backtest_live_sync.py - 100% sync live"]

    ROOT --> LOGS["logs/"]
    LOGS --> LOGF["training_YYYY-MM-DD.log"]

    ROOT --> DOCS["docs/arsitektur-ai/"]
    DOCS --> DOC0["00-ARSITEKTUR-LENGKAP.md - Dokumen ini"]
    DOCS --> DOCR["README.md - Index komponen"]
    DOCS --> DOC1["01-23 (per komponen) - Detail per modul"]

---

Ringkasan Eksekutif

Smart AI Trading Bot adalah sistem trading otomatis yang menggabungkan:

1. Smart Money Concepts (SMC) sebagai sinyal UTAMA — mendeteksi zona institusi (FVG, Order Block, BOS, CHoCH) untuk menentukan entry, SL, dan TP yang presisi.

2. XGBoost Machine Learning sebagai KONFIRMASI — memprediksi arah harga dengan 24 fitur teknikal, memblokir trade jika tidak setuju dengan SMC.

3. Hidden Markov Model (HMM) sebagai PENYESUAI — mendeteksi kondisi pasar (tenang/volatile/krisis) untuk menyesuaikan agresivitas.

4. 4-Lapis Proteksi Risiko — dari broker SL, software smart exit, emergency stop, hingga circuit breaker. Lot ultra-kecil (0.01-0.02) memastikan kerugian per trade maximum $50 (1%).

5. Self-Improving — model AI dilatih ulang otomatis setiap hari dengan auto-rollback jika model baru lebih buruk.

6. Fault-Tolerant — bot tidak pernah crash. MT5 putus? Auto-reconnect. Database mati? CSV fallback. Error? Log dan lanjut.

Semua ini dikoordinasikan oleh Main Live Orchestrator yang menjalankan loop candle-based — analisis penuh hanya saat candle M15 baru terbentuk (~50ms per iterasi), dengan pengecekan posisi setiap ~10 detik di antara candle (~21ms). Mengevaluasi 14 filter entry dan 12 kondisi exit secara real-time, dengan notifikasi Telegram untuk setiap kejadian penting.

TARGET: Trading XAUUSD M15 yang KONSISTEN dan AMAN
        dengan kerugian terkontrol dan profit teroptimasi.