gavindiaz/XauBot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
d99df49dfca7df5fcaef6be3c6e5f7e173a9d84f
XauBot/docs/arsitektur-ai/00-ARSITEKTUR-LENGKAP.md
T
GifariKemalandClaude Opus 4.6 757b499033 docs: sync architecture docs with v5 major issues fix 
5 major issues reflected in documentation:

1. Confidence calibration (03-SMC, 00-ARSITEKTUR):
   - Base 55% + 10% each → base 40% + weighted scoring
   - Structure +15%, BOS/CHoCH +12%, FVG +8%, OB +10%, Trend +10%

2. ATR-based pullback filter (09-Entry, 00-ARSITEKTUR):
   - Hardcoded $2/$1.5 → bounce 15% ATR, consolidation 10% ATR

3. Smarter time-based exit (10-Exit, 05-Risk, 00-ARSITEKTUR):
   - 4h: check profit growth, not just profit<$5
   - 6h: extend to 8h if profit>$10 and growing + ML agrees

4. Slippage validation (09-Entry, 23-Main, 00-ARSITEKTUR):
   - Check actual vs expected price, log if >0.15%

5. Partial fill handling (09-Entry, 23-Main, 00-ARSITEKTUR):
   - Check filled volume, use actual values for tracking

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 10:06:43 +07:00

1717 lines
95 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Arsitektur Lengkap — Smart AI Trading Bot
> **Dokumen:** Arsitektur keseluruhan sistem dalam 1 file
> **Instrumen:** XAUUSD (Gold) M15
> **Platform:** MetaTrader 5
> **Bahasa:** Python 3.11+ (async, Polars, XGBoost, HMM)
> **Database:** PostgreSQL + CSV fallback
> **Notifikasi:** Telegram Bot API
---
## Daftar Isi
1. [Gambaran Umum](#1-gambaran-umum)
2. [Diagram Arsitektur](#2-diagram-arsitektur)
3. [23 Komponen](#3-23-komponen)
4. [Pipeline Data: Dari OHLCV ke Keputusan Trading](#4-pipeline-data)
5. [Alur Entry: 11 Filter](#5-alur-entry-11-filter)
6. [Alur Exit: 10 Kondisi](#6-alur-exit-10-kondisi)
7. [Sistem Proteksi Risiko 4 Lapis](#7-sistem-proteksi-risiko-4-lapis)
8. [AI/ML Engine](#8-aiml-engine)
9. [Smart Money Concepts (SMC)](#9-smart-money-concepts)
10. [Position Lifecycle](#10-position-lifecycle)
11. [Auto-Retraining & Model Management](#11-auto-retraining)
12. [Infrastruktur & Database](#12-infrastruktur--database)
13. [Konfigurasi & Parameter Kritis](#13-konfigurasi--parameter-kritis)
14. [Performa & Timing](#14-performa--timing)
15. [Error Handling & Fault Tolerance](#15-error-handling--fault-tolerance)
16. [Daftar File Source Code](#16-daftar-file-source-code)
---
## 1. Gambaran Umum
### Apa Ini?
Bot trading otomatis yang menggabungkan **3 otak kecerdasan buatan** untuk trading XAUUSD (Emas) di MetaTrader 5:
```
OTAK 1: Smart Money Concepts (SMC)
→ Membaca pola institusi besar (bank, hedge fund)
→ Menentukan DIMANA entry, SL, dan TP
OTAK 2: XGBoost Machine Learning
→ Memprediksi ARAH harga (naik/turun)
→ Memberikan tingkat keyakinan (confidence)
OTAK 3: Hidden Markov Model (HMM)
→ Membaca KONDISI pasar (tenang/volatile/krisis)
→ Menyesuaikan ukuran posisi dan agresivitas
```
### Filosofi Desain
```
1. KESELAMATAN MODAL NOMOR 1
→ 4 lapis proteksi stop loss
→ Lot ultra-kecil (0.01-0.02)
→ Circuit breaker otomatis
2. TIDAK PERNAH CRASH
→ Setiap error di-catch, bot terus jalan
→ Database gagal? CSV fallback
→ MT5 putus? Auto-reconnect
3. SELF-IMPROVING
→ Model AI dilatih ulang otomatis setiap hari
→ Rollback otomatis jika model baru lebih buruk
→ Threshold confidence menyesuaikan kondisi pasar
4. TRANSPARAN
→ Setiap keputusan dicatat ke database
→ Notifikasi Telegram real-time
→ Laporan harian, jam-an, dan per-trade
```
### Angka-Angka Kunci
| Parameter | Nilai | Penjelasan |
|-----------|-------|------------|
| Modal target | $5,000 | Small account mode |
| Risiko per trade | 1% ($50) | Maksimum kerugian per posisi |
| Lot size | 0.01 - 0.02 | Ultra-konservatif |
| Max daily loss | 3% ($150) | Circuit breaker harian |
| Max total loss | 10% ($500) | Stop total trading |
| Max posisi bersamaan | 2-3 | Menghindari overexposure |
| Cooldown antar trade | 5 menit | Mencegah overtrading |
| Loop speed | ~50ms | Cepat tapi efisien |
| Timeframe | M15 | 15 menit per candle |
---
## 2. Diagram Arsitektur
### Diagram Keseluruhan Sistem
```
┌─────────────────────────────────────────────────────────────────────────┐
│ MAIN LIVE (Orchestrator) │
│ main_live.py — TradingBot │
│ Candle-based (M15) + position check ~10 detik │
│ │
│ ┌─────────────── PHASE 1: DATA ──────────────────────────────────┐ │
│ │ │ │
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │
│ │ │ MT5 │ │ Feature │ │ SMC │ │ HMM │ │ │
│ │ │Connector │ ──→│ Engine │ ──→│ Analyzer │ │ Regime │ │ │
│ │ │(broker) │ │(40+ fitur│ │(institusi│ │(3 state) │ │ │
│ │ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │ │
│ │ │ │ │ │ │ │
│ │ │ │ ▼ ▼ │ │
│ │ │ │ ┌──────────┐ ┌──────────┐ │ │
│ │ │ └────────→│ XGBoost │ │ Dynamic │ │ │
│ │ │ │ ML Model │ │Confidence│ │ │
│ │ │ │(prediksi)│ │(threshold│ │ │
│ │ │ └──────────┘ └──────────┘ │ │
│ └───────┼─────────────────────────────┼───────────────┼────────┘ │
│ │ │ │ │
│ ┌───────┼──── PHASE 2: MONITORING ────┼───────────────┼────────┐ │
│ │ │ │ │ │ │
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │
│ │ │ Position │ │ Smart │ │ Risk │ │ │
│ │ │ Manager │ │ Risk │ │ Engine │ │ │
│ │ │(trailing)│ │ Manager │ │ (Kelly) │ │ │
│ │ └──────────┘ └──────────┘ └──────────┘ │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │ │ │ │
│ ┌───────┼──── PHASE 3: ENTRY ─────────┼───────────────┼────────┐ │
│ │ │ │ │ │ │
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │
│ │ │ Session │ │ News │ │ 11-Gate │ │ │
│ │ │ Filter │ │ Agent │ │ Entry │ │ │
│ │ │(waktu) │ │(berita) │ │ Filter │──→ EXECUTE │ │
│ │ └──────────┘ └──────────┘ └──────────┘ │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌───────┼──── PHASE 4: PERIODIK ──────────────────────────────┐ │
│ │ │ │ │
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │
│ │ │ Auto │ │ Telegram │ │ Trade │ │ │
│ │ │ Trainer │ │ Notifier │ │ Logger │ │ │
│ │ │(retrain) │ │(laporan) │ │(DB+CSV) │ │ │
│ │ └──────────┘ └──────────┘ └──────────┘ │ │
│ └─────────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌────────┴────────┐ │
│ │ PostgreSQL │ │
│ │ + CSV Backup │ │
│ └─────────────────┘ │
└─────────────────────────────────────────────────────────────────────────┘
```
### Alur Data (Data Flow)
```
MT5 Broker (XAUUSD M15)
│
│ 200 bar OHLCV
▼
┌─────────────────┐
│ MT5 Connector │ numpy → Polars (tanpa Pandas)
└────────┬────────┘
│
▼
┌─────────────────┐
│ Feature Engineer │ OHLCV → 40+ fitur teknikal
│ │ RSI, ATR, MACD, BB, EMA, Volume,
│ │ Returns, Volatility, Lags, Trend
└────────┬────────┘
│
┌────┴────┐
│ │
▼ ▼
┌───────┐ ┌───────┐
│ SMC │ │ HMM │
│Analyzer│ │Regime │
│ │ │Detect │
└───┬───┘ └───┬───┘
│ │
│ ┌────┘
│ │
▼ ▼
┌─────────────────┐
│ XGBoost │ 24 fitur → Prediksi BUY/SELL/HOLD
│ ML Predictor │ + Confidence 0-100%
└────────┬────────┘
│
▼
┌─────────────────┐
│ Dynamic │ Sesuaikan threshold berdasarkan
│ Confidence │ sesi, regime, volatilitas, trend
└────────┬────────┘
│
▼
┌─────────────────┐
│ Signal Combiner │ SMC + ML harus setuju
│ (11 Filter) │ + Session + Risk + Cooldown
└────────┬────────┘
│
┌────┴────┐
│ PASS? │
│ │
YES NO → tunggu loop berikutnya
│
▼
┌─────────────────┐
│ Risk Engine │ Kelly Criterion → lot size
│ + Risk Manager │ Validasi order → approve/reject
└────────┬────────┘
│
▼
┌─────────────────┐
│ Execute Order │ Kirim ke MT5 dengan SL & TP
│ via MT5 │ Register ke Position Manager
└────────┬────────┘
│
┌────┴────────────────┐
│ │
▼ ▼
┌───────────┐ ┌────────────────┐
│ Telegram │ │ Trade Logger │
│ Notifier │ │ (DB + CSV) │
└───────────┘ └────────────────┘
```
---
## 3. 23 Komponen
### Tabel Komponen Lengkap
| # | Komponen | File | Kategori | Fungsi Utama |
|---|----------|------|----------|-------------|
| 1 | HMM Regime Detector | `src/regime_detector.py` | AI/ML | Deteksi kondisi pasar (3 regime) |
| 2 | XGBoost Predictor | `src/ml_model.py` | AI/ML | Prediksi arah harga + confidence |
| 3 | SMC Analyzer | `src/smc_polars.py` | Analisis | Pola institusi: FVG, OB, BOS, CHoCH |
| 4 | Feature Engineering | `src/feature_eng.py` | Data | OHLCV → 40+ fitur numerik |
| 5 | Smart Risk Manager | `src/smart_risk_manager.py` | Risiko | 4 mode trading, 10 kondisi exit |
| 6 | Session Filter | `src/session_filter.py` | Filter | Waktu trading optimal (WIB) |
| 7 | Stop Loss (4 Lapis) | Multi-file | Proteksi | SMC → Software → Emergency → Circuit |
| 8 | Take Profit (6 Layer) | Multi-file | Proteksi | Hard → Momentum → Peak → Probability → Early → Broker |
| 9 | Entry Trade | `main_live.py` | Eksekusi | 11 filter berurutan |
| 10 | Exit Trade | `main_live.py` | Eksekusi | 10 kondisi exit real-time |
| 11 | News Agent | `src/news_agent.py` | Monitor | Monitoring berita (TIDAK memblokir) |
| 12 | Telegram Notifier | `src/telegram_notifier.py` | Notifikasi | 11 tipe notifikasi real-time |
| 13 | Auto Trainer | `src/auto_trainer.py` | ML Ops | Retraining harian otomatis |
| 14 | Backtest | `backtests/backtest_live_sync.py` | Validasi | Simulasi 100% sync dengan live |
| 15 | Dynamic Confidence | `src/dynamic_confidence.py` | Adaptif | Threshold ML adaptif (60-85%) |
| 16 | MT5 Connector | `src/mt5_connector.py` | Koneksi | Bridge ke broker, auto-reconnect |
| 17 | Configuration | `src/config.py` | Config | 6 sub-config, auto-adjust modal |
| 18 | Trade Logger | `src/trade_logger.py` | Logging | Dual storage DB + CSV |
| 19 | Position Manager | `src/position_manager.py` | Manajemen | Trailing SL, breakeven, market close |
| 20 | Risk Engine | `src/risk_engine.py` | Risiko | Kelly Criterion, circuit breaker |
| 21 | Database | `src/db/` | Storage | PostgreSQL, 6 repository |
| 22 | Train Models | `train_models.py` | Training | Script training awal |
| 23 | Main Live | `main_live.py` | Orchestrator | Koordinasi semua komponen |
### Hubungan Antar Komponen
```
┌─────────────────────────┐
│ CONFIGURATION (17) │
│ Sumber parameter semua │
└────────────┬─────────────┘
│ dikonsumsi oleh semua
▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ MT5 (16) │───→│FeatEng(4)│───→│ SMC (3) │
│ Broker │ │ 40+ fitur│ │ Institusi│
└──────────┘ └────┬─────┘ └────┬─────┘
│ │
▼ ▼
┌──────────┐ ┌──────────┐
│ HMM (1) │ │XGBoost(2)│
│ Regime │ │ Prediksi │
└────┬─────┘ └────┬─────┘
│ │
▼ ▼
┌──────────────────────────┐
│ Dynamic Confidence (15) │
│ Threshold adaptif │
└────────────┬─────────────┘
│
▼
┌──────────┐ ┌──────────────────────────┐ ┌──────────┐
│Session(6)│───→│ ENTRY TRADE (9) │←───│ News(11) │
│ Waktu │ │ 11 Filter Gate │ │ Berita │
└──────────┘ └────────────┬─────────────┘ └──────────┘
│
┌────────┴────────┐
▼ ▼
┌──────────┐ ┌──────────────┐
│RiskEng(20│ │SmartRisk (5) │
│Kelly Lot │ │ 4 Mode │
└────┬─────┘ └──────┬───────┘
│ │
▼ ▼
┌──────────────────────────┐
│ EXECUTE ORDER │
│ via MT5 (16) │
└────────────┬─────────────┘
│
┌────────────┼────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│PosMgr(19)│ │Logger(18)│ │Telegram │
│Trailing │ │ DB+CSV │ │ (12) │
└────┬─────┘ └────┬─────┘ └──────────┘
│ │
▼ ▼
┌──────────┐ ┌──────────┐
│EXIT (10) │ │ DB (21) │
│10 Kondisi│ │PostgreSQL│
└──────────┘ └──────────┘
Periodik:
┌──────────┐ ┌──────────┐ ┌──────────┐
│AutoTrain │ │ Backtest │ │ Train │
│ (13) │ │ (14) │ │Models(22)│
│Harian │ │Validasi │ │Setup awal│
└──────────┘ └──────────┘ └──────────┘
```
---
## 4. Pipeline Data
### Dari OHLCV Mentah ke Keputusan Trading
#### Tahap 1: Data Fetching (MT5 Connector)
```
MT5 Broker
│
│ mt5.copy_rates_from_pos("XAUUSD", MT5_TIMEFRAME_M15, 0, 200)
│
▼
NumPy Structured Array
│
│ Konversi langsung ke Polars (TANPA Pandas)
│
▼
Polars DataFrame:
┌──────────────────────────────────────────────────────┐
│ time │ open │ high │ low │ close │ tick_volume │ spread │
│ i64 │ f64 │ f64 │ f64 │ f64 │ f64 │ f64 │
├──────┼──────┼──────┼─────┼───────┼─────────────┼────────┤
│ ... │ 2645 │ 2648 │ 2643│ 2647 │ 5234 │ 25 │
└──────────────────────────────────────────────────────┘
```
**Kenapa Polars, bukan Pandas?**
- 3-5x lebih cepat untuk operasi vectorized
- Memory-efficient (zero-copy)
- Native lazy evaluation
- Konsisten di seluruh codebase (tidak ada konversi bolak-balik)
#### Tahap 2: Feature Engineering (40+ Fitur)
```
Input: Polars DataFrame (200 bar OHLCV)
│
├── Momentum Indicators
│ ├── RSI(14) → 0-100, overbought/oversold
│ ├── MACD(12,26,9) → trend strength & direction
│ └── MACD Histogram → momentum acceleration
│
├── Volatility Indicators
│ ├── ATR(14) → average true range (pips)
│ ├── Bollinger Bands(20,2.0) → upper, lower, width
│ └── Volatility(20) → rolling std of returns
│
├── Trend Indicators
│ ├── EMA(9) / EMA(21) → fast/slow crossover
│ ├── EMA Cross Signal → 1 (bullish) / -1 (bearish)
│ └── SMA(20) → simple moving average
│
├── Price Action
│ ├── Returns(1,5,20) → % perubahan harga
│ ├── Log Returns → untuk distribusi normal
│ ├── Price Position → posisi dalam range BB
│ └── Higher High/Lower Low count → trend structure
│
├── Volume Features
│ ├── Volume SMA(20) → rata-rata volume
│ └── Volume Ratio → current / average
│
├── Lag Features
│ ├── close_lag_1..5 → harga sebelumnya
│ └── returns_lag_1..3 → return sebelumnya
│
└── Time Features
├── Hour, Weekday → waktu candle
└── Session flags → london, ny, overlap
│
▼
Output: DataFrame + 40 kolom baru (semua numerik, siap ML)
```
**Minimum data:** 26 bar untuk semua indikator stabil
#### Tahap 3: SMC Analysis (Pola Institusi)
```
Input: DataFrame dengan OHLCV
│
├── Swing Points (Fractal)
│ Window: 11 bar (swing_length=5, ±5 dari tengah)
│ Output: swing_high (1/0), swing_low (-1/0), level harga
│
├── Fair Value Gaps (FVG)
│ Bullish: bar[i-2].high < bar[i].low (gap up)
│ Bearish: bar[i-2].low > bar[i].high (gap down)
│ Output: fvg_bull, fvg_bear, fvg_top, fvg_bottom, fvg_mid
│
├── Order Blocks (OB)
│ Lookback: 10 bar
│ Bullish: candle bearish terakhir sebelum move up besar
│ Bearish: candle bullish terakhir sebelum move down besar
│ Output: ob (1/-1), ob_top, ob_bottom, ob_mitigated
│
├── Break of Structure (BOS)
│ BOS: harga break swing high/low → trend continuation
│ Output: bos (1/-1), level yang di-break
│
├── Change of Character (CHoCH)
│ CHoCH: harga break berlawanan arah trend → reversal signal
│ Output: choch (1/-1), level yang di-break
│
└── Liquidity Zones
BSL: Buy Side Liquidity (above swing highs)
SSL: Sell Side Liquidity (below swing lows)
Output: bsl_level, ssl_level
│
▼
Signal Generation:
Syarat: Structure break + (FVG ATAU Order Block)
│
├── Entry: harga saat ini
├── SL: ATR-based, minimum 1.5 × ATR dari entry
├── TP: 2:1 Risk-Reward minimum, cap 4 × ATR
├── Confidence: 40-85% (v5: calibrated weighted scoring) (berdasarkan confluence)
└── Reason: "BOS + Bullish FVG at 2645.50"
```
#### Tahap 4: Regime Detection (HMM)
```
Input: log_returns + normalized_range (volatilitas)
│
│ GaussianHMM(n_components=3, lookback=500)
│
▼
3 Regime:
┌──────────────────────────────────────────────────┐
│ REGIME 0: Low Volatility │
│ → Pasar tenang, range kecil │
│ → Lot multiplier: 1.0x (normal) │
│ → Rekomendasi: TRADE │
│ │
│ REGIME 1: Medium Volatility │
│ → Pasar aktif, trend jelas │
│ → Lot multiplier: 1.0x (normal) │
│ → Rekomendasi: TRADE │
│ │
│ REGIME 2: High Volatility │
│ → Pasar sangat volatile, berbahaya │
│ → Lot multiplier: 0.5x (setengah) │
│ → Rekomendasi: REDUCE │
│ │
│ CRISIS (detected by FlashCrashDetector): │
│ → Move > 2.5% dalam 1 menit │
│ → Lot multiplier: 0.0x (STOP) │
│ → Rekomendasi: EMERGENCY CLOSE ALL │
└──────────────────────────────────────────────────┘
```
#### Tahap 5: ML Prediction (XGBoost)
```
Input: 24 fitur terpilih dari Feature Engineering + SMC + Regime
│
│ XGBoost Binary Classifier
│ Anti-overfitting config:
│ max_depth=3, learning_rate=0.05
│ min_child_weight=10, subsample=0.7
│ colsample_bytree=0.6
│ reg_alpha=1.0 (L1), reg_lambda=5.0 (L2)
│
▼
Output:
┌──────────────────────────────────────────┐
│ prob_up: 0.72 (probabilitas naik) │
│ prob_down: 0.28 (probabilitas turun) │
│ │
│ → Signal: BUY (prob_up > 0.50) │
│ → Confidence: 72% │
│ │
│ Threshold Keputusan: │
│ prob > 0.50 → ada sinyal (minimum) │
│ prob > 0.65 → sinyal kuat │
│ prob > 0.75 → sinyal sangat kuat │
│ prob > 0.80 → lot bisa naik ke 0.02 │
└──────────────────────────────────────────┘
```
#### Tahap 6: Dynamic Confidence (Threshold Adaptif)
```
Scoring (0-100 poin):
Base score: 50
│
├── Session Modifier:
│ Golden Time (20:00-23:59 WIB) → +20
│ London (15:00-23:59) → +15
│ New York (20:00-05:00) → +10
│ Tokyo/Sydney → +0
│ Market Closed → -30
│
├── Regime Modifier:
│ Medium Volatility → +15
│ Low Volatility → +5
│ High Volatility → -5
│ Crisis → -25
│
├── Volatility Modifier:
│ Medium (ideal) → +10
│ Low → +0
│ High → -5
│ Extreme → -10
│
├── Trend Modifier:
│ Trending (jelas) → +10
│ Ranging (sideways) → -5
│
├── SMC Confluence:
│ Ada konfluensi → +10
│ Tidak ada → +0
│
└── ML Confidence:
≥ 70% → +5
≥ 60% → +2
< 60% → +0
│
▼
Quality Level → Threshold:
┌───────────────────────────────────────────────────┐
│ EXCELLENT (≥80 poin) → Threshold: 60% (longgar) │
│ GOOD (65-79) → Threshold: 65% │
│ MODERATE (50-64) → Threshold: 70% │
│ POOR (35-49) → Threshold: 80% (ketat) │
│ AVOID (<35) → Threshold: 85% (SKIP) │
└───────────────────────────────────────────────────┘
Contoh: Golden Time + Medium Vol + Trending + SMC + ML 72%
= 50 + 20 + 15 + 10 + 10 + 10 + 5 = 120 → cap 100
= EXCELLENT → Threshold 60% → ML 72% PASS ✓
```
---
## 5. Alur Entry: 11 Filter
Setiap sinyal harus melewati **11 gerbang berurutan**. Satu saja gagal = TIDAK trading.
```
SINYAL SMC + ML MASUK
│
▼
┌─ FILTER 1: Session Filter ──────────────────────────────┐
│ Apakah sekarang jam trading yang dibolehkan? │
│ ✗ 00:00-06:00 WIB (dead zone) → BLOCK │
│ ✗ Jumat ≥ 23:00 WIB (weekend risk) → BLOCK │
│ ✓ London/NY/Golden Time → PASS │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 2: Risk Mode ───────────────────────────────────┐
│ Apakah mode trading bukan STOPPED? │
│ ✗ STOPPED (daily/total limit hit) → BLOCK │
│ ✓ NORMAL / RECOVERY / PROTECTED → PASS │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 3: SMC Signal ──────────────────────────────────┐
│ Apakah ada setup SMC yang valid? │
│ ✗ Tidak ada FVG/OB + BOS/CHoCH → BLOCK │
│ ✓ Ada sinyal BUY/SELL dengan SL & TP → PASS │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 4: ML Confidence ───────────────────────────────┐
│ Apakah ML confidence ≥ dynamic threshold? │
│ ✗ ML confidence < threshold → BLOCK │
│ ✓ ML confidence ≥ threshold → PASS │
│ (threshold 60-85% tergantung market quality) │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 5: ML Agreement ────────────────────────────────┐
│ Apakah ML TIDAK strongly disagree dengan SMC? │
│ ✗ ML > 65% berlawanan arah SMC → BLOCK (conflict) │
│ ✓ ML setuju atau netral → PASS │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 6: Market Quality ──────────────────────────────┐
│ Apakah Dynamic Confidence bukan AVOID? │
│ ✗ Quality == AVOID (score < 35) → BLOCK │
│ ✓ EXCELLENT/GOOD/MODERATE/POOR → PASS │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 7: Signal Confirmation ─────────────────────────┐
│ Apakah sinyal konsisten 2x berturut-turut? │
│ ✗ Sinyal baru muncul 1x → BLOCK (tunggu konfirmasi) │
│ ✓ Sinyal sudah 2x berturut → PASS │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 8: Pullback Filter (v5: ATR-based) ────────────┐
│ Apakah momentum selaras dengan arah sinyal? │
│ ✗ BUY tapi RSI > 80 (overbought) → BLOCK │
│ ✗ SELL tapi RSI < 20 (oversold) → BLOCK │
│ ✗ MACD histogram berlawanan → BLOCK │
│ ✗ Harga bounce > 15% ATR dalam 3 candle → BLOCK │
│ ✓ Movement < 10% ATR (consolidation) → PASS │
│ ✓ Momentum selaras → PASS │
│ (v5: threshold dinamis berdasarkan ATR, bukan fixed $2) │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 9: Trade Cooldown ──────────────────────────────┐
│ Apakah sudah ≥ 5 menit sejak trade terakhir? │
│ ✗ < 300 detik sejak trade terakhir → BLOCK │
│ ✓ ≥ 300 detik → PASS │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 10: Position Limit ─────────────────────────────┐
│ Apakah jumlah posisi terbuka < limit? │
│ ✗ Sudah 2+ posisi terbuka → BLOCK │
│ ✓ < 2 posisi → PASS │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
┌─ FILTER 11: Lot Size ───────────────────────────────────┐
│ Apakah lot size yang dihitung > 0? │
│ ✗ Lot = 0 (regime crisis / risk terlalu tinggi) → BLOCK│
│ ✓ Lot ≥ 0.01 → PASS │
└──────────────────────────────────────────────────────────┘
│ PASS
▼
╔═══════════════════════╗
║ EXECUTE TRADE ║
║ BUY atau SELL ║
║ via MT5 Connector ║
╚═══════════════════════╝
│
├── Register ke SmartRiskManager
├── Log ke TradeLogger (DB + CSV)
└── Kirim notifikasi Telegram
```
---
## 6. Alur Exit: 10 Kondisi
Setiap posisi terbuka dievaluasi **setiap ~10 detik** (di antara candle) atau **setiap candle baru** (full analysis) terhadap 10 kondisi exit:
```
POSISI TERBUKA (dicek setiap ~10 detik)
│
│ Update: profit, momentum, peak, durasi
│
▼
┌─ KONDISI 1: Smart Take Profit ──────────────────────────┐
│ (a) Profit ≥ $40 → TUTUP (hard TP) │
│ (b) Profit ≥ $25 + momentum < -30 → TUTUP │
│ (c) Peak > $30, sekarang < 60% peak → TUTUP │
│ (d) Profit ≥ $20 + TP probability < 25% → TUTUP │
└──────────────────────────────────────────────────────────┘
│ tidak trigger
▼
┌─ KONDISI 2: Early Exit (Profit Kecil) ──────────────────┐
│ Profit $5-$15 + ML reversal ≥ 65% + momentum < -50 │
│ → TUTUP (amankan profit kecil sebelum hilang) │
└──────────────────────────────────────────────────────────┘
│ tidak trigger
▼
┌─ KONDISI 3: Early Cut (v4 — Smart Hold DIHAPUS) ────────┐
│ Loss >= 30% max ($15) DAN momentum < -30? │
│ → TUTUP CEPAT (early cut, jangan tunggu recovery) │
│ │
│ v4: "Smart Hold" dihapus — tidak ada lagi hold losers │
│ menunggu golden time atau sesi London. │
└──────────────────────────────────────────────────────────┘
│ tidak trigger
▼
┌─ KONDISI 4: ML Trend Reversal ──────────────────────────┐
│ ML confidence ≥ 65% BERLAWANAN ARAH posisi │
│ + 3x warning berturut-turut │
│ → TUTUP (AI mendeteksi pembalikan trend) │
└──────────────────────────────────────────────────────────┘
│ tidak trigger
▼
┌─ KONDISI 5: Maximum Loss ───────────────────────────────┐
│ Loss ≥ 50% dari max_loss_per_trade ($25 dari $50) │
│ → TUTUP (kerugian terlalu besar) │
└──────────────────────────────────────────────────────────┘
│ tidak trigger
▼
┌─ KONDISI 6: Stall Detection ────────────────────────────┐
│ Posisi sudah 10+ bar tanpa profit signifikan │
│ → TUTUP (pasar tidak bergerak sesuai ekspektasi) │
└──────────────────────────────────────────────────────────┘
│ tidak trigger
▼
┌─ KONDISI 7: Daily Limit ────────────────────────────────┐
│ Total daily loss mendekati limit │
│ → TUTUP semua posisi (proteksi sisa modal hari ini) │
└──────────────────────────────────────────────────────────┘
│ tidak trigger
▼
┌─ KONDISI 8: Weekend Close ──────────────────────────────┐
│ Mendekati market close weekend + ada posisi profit │
│ → TUTUP (hindari gap risk Senin) │
└──────────────────────────────────────────────────────────┘
│ tidak trigger
▼
┌─ KONDISI 9: Smart Time-Based (v5: Don't Cut Winners) ──┐
│ (a) > 4 jam + no growth → TUTUP (stuck) │
│ (b) > 4 jam + growing + ML agrees → HOLD (extend) │
│ (c) > 6 jam + profit < $10 → TUTUP │
│ (d) > 6 jam + profit > $10 + growing → extend ke 8 jam │
│ (e) > 8 jam → TUTUP (final max time) │
└──────────────────────────────────────────────────────────┘
│ tidak trigger
▼
┌─ KONDISI 10: Default HOLD ──────────────────────────────┐
│ Tidak ada kondisi terpenuhi │
│ → HOLD (biarkan posisi berjalan) │
└──────────────────────────────────────────────────────────┘
```
### Position Manager (Tambahan per Posisi)
Selain 10 kondisi di atas, Position Manager juga menjalankan:
```
┌─ Market Close Handler (Prioritas Tertinggi) ────────────┐
│ Dekat close harian/weekend? │
│ ├── Profit ≥ $10 + dekat close → TUTUP (amankan) │
│ ├── Loss + weekend + SL > 50% → TUTUP (gap risk) │
│ └── Loss kecil + weekend → HOLD (bisa recovery) │
└──────────────────────────────────────────────────────────┘
┌─ Breakeven Protection ──────────────────────────────────┐
│ Profit ≥ 15 pips → Pindah SL ke entry + 2 buffer │
│ (posisi tidak bisa rugi lagi) │
└──────────────────────────────────────────────────────────┘
┌─ Trailing Stop ─────────────────────────────────────────┐
│ Profit ≥ 25 pips → SL mengikuti harga, jarak 10 pips │
│ (kunci profit sambil biarkan berjalan) │
└──────────────────────────────────────────────────────────┘
```
---
## 7. Sistem Proteksi Risiko 4 Lapis
```
╔══════════════════════════════════════════════════════════════════╗
║ LAPIS 1: BROKER STOP LOSS ║
║ (Otomatis oleh MT5) ║
║ ║
║ SL = Entry ± (1.5 × ATR) minimum 10 pips ║
║ Dikirim bersama order ke broker ║
║ Aktif 24/7, bahkan jika bot mati ║
║ Max loss: ~$50-80 per trade ║
╠══════════════════════════════════════════════════════════════════╣
║ LAPIS 2: SOFTWARE SMART EXIT ║
║ (Bot mengevaluasi setiap ~10 detik) ║
║ ║
║ 10 kondisi exit (lihat bagian 6) ║
║ Biasanya menutup SEBELUM broker SL kena ║
║ Target close: loss ≤ $25 (lebih ketat dari broker) ║
║ Termasuk: momentum, ML reversal, stall, time limit ║
╠══════════════════════════════════════════════════════════════════╣
║ LAPIS 3: EMERGENCY STOP LOSS ║
║ (Backup jika software gagal) ║
║ ║
║ Max loss per trade: 2% modal ($100 untuk $5K) ║
║ Diset sebagai broker SL terpisah ║
║ Aktif jika software error/hang ║
╠══════════════════════════════════════════════════════════════════╣
║ LAPIS 4: CIRCUIT BREAKER ║
║ (Hentikan semua trading) ║
║ ║
║ Trigger 1: Daily loss ≥ 3% ($150) → Stop hari ini ║
║ Trigger 2: Total loss ≥ 10% ($500) → Stop total ║
║ Trigger 3: Flash crash > 2.5% / 1 menit → CLOSE ALL ║
║ Reset: Otomatis di hari baru (daily), manual (total) ║
╚══════════════════════════════════════════════════════════════════╝
```
### 4 Mode Trading (Smart Risk Manager)
```
┌──────────────────────────────────────────────────────────┐
│ MODE: NORMAL │
│ Kondisi: Semua aman, tidak ada masalah │
│ Lot: 0.01 - 0.02 (berdasarkan confidence) │
│ Max posisi: 2-3 │
│ │
│ │ 3x loss berturut-turut │
│ ▼ │
│ MODE: RECOVERY │
│ Kondisi: Setelah kerugian beruntun │
│ Lot: 0.01 (minimum saja) │
│ Max posisi: 1 │
│ │
│ │ mendekati 80% daily limit │
│ ▼ │
│ MODE: PROTECTED │
│ Kondisi: Hampir kena daily limit │
│ Lot: 0.01 (minimum saja) │
│ Max posisi: 1 │
│ │
│ │ daily/total limit tercapai │
│ ▼ │
│ MODE: STOPPED │
│ Kondisi: Batas kerugian tercapai │
│ Lot: 0 (TIDAK BOLEH trading) │
│ Max posisi: 0 (tutup semua) │
│ Reset: Otomatis hari baru │
└──────────────────────────────────────────────────────────┘
```
### Lot Sizing: Risk-Constrained Half-Kelly
```
Langkah 1: Hitung Kelly Fraction
f* = (win_rate × avg_rr - (1 - win_rate)) / avg_rr
Contoh: win_rate=55%, avg_rr=2.0
f* = (0.55 × 2.0 - 0.45) / 2.0 = 0.325 (32.5%)
Langkah 2: Cap Kelly (max 25%)
f* = min(0.325, 0.25) = 0.25
Langkah 3: Half-Kelly (safety)
f* = 0.25 × 0.5 = 0.125 (12.5%)
Langkah 4: Apply regime multiplier
High volatility: × 0.5 = 0.0625
Normal: × 1.0 = 0.125
Langkah 5: Cap di config limit
config risk_per_trade = 1%
actual_risk = min(0.125, 0.01) = 0.01 (1%)
Langkah 6: Hitung lot
risk_amount = $5000 × 1% = $50
SL distance = 50 pips → pip_value ~$1/pip/0.01lot
lot = $50 / (50 × $1) = 0.01 lot
Langkah 7: ML Confidence boost
ML ≥ 80% → lot × 2 = 0.02 lot (maximum)
ML < 65% → lot = 0.01 (minimum)
Langkah 8: Session multiplier
Golden Time: × 1.2
Sydney: × 0.5
Final lot: 0.01 - 0.02 (ultra-konservatif)
```
---
## 8. AI/ML Engine
### Hidden Markov Model (HMM) — Otak Regime
```
┌──────────────────────────────────────────────────────────┐
│ HIDDEN MARKOV MODEL │
│ │
│ Library: hmmlearn.GaussianHMM │
│ Input: log_returns + rolling_volatility (2 fitur) │
│ States: 3 (Low, Medium, High Volatility) │
│ Lookback: 500 bar untuk training │
│ Retrain: setiap 20 bar (auto-update) │
│ │
│ Transition Matrix (contoh): │
│ To Low To Med To High │
│ Fr Low [ 0.85 0.12 0.03 ] │
│ Fr Med [ 0.10 0.80 0.10 ] │
│ Fr High [ 0.05 0.15 0.80 ] │
│ │
│ Distribusi Emisi (per state): │
│ Low: μ_return ≈ 0, σ_return = kecil │
│ Med: μ_return ≈ 0, σ_return = sedang │
│ High: μ_return ≈ 0, σ_return = besar │
│ │
│ Output: │
│ ├── regime: 0/1/2 (low/medium/high) │
│ ├── confidence: 0.0 - 1.0 │
│ ├── lot_multiplier: 1.0 / 0.5 / 0.0 │
│ └── recommendation: TRADE / REDUCE / SLEEP │
└──────────────────────────────────────────────────────────┘
```
### XGBoost — Otak Prediksi
```
┌──────────────────────────────────────────────────────────┐
│ XGBOOST BINARY CLASSIFIER │
│ │
│ Library: xgboost │
│ Objective: binary:logistic │
│ Target: UP (1) / DOWN (0) pada bar berikutnya │
│ │
│ Anti-Overfitting Config: │
│ ├── max_depth: 3 (shallow trees) │
│ ├── learning_rate: 0.05 (slow learning) │
│ ├── min_child_weight: 10 (min samples per leaf) │
│ ├── subsample: 0.7 (70% data per tree) │
│ ├── colsample_bytree: 0.6 (60% features per tree) │
│ ├── reg_alpha: 1.0 (L1 regularization) │
│ ├── reg_lambda: 5.0 (L2 regularization) │
│ ├── gamma: 1.0 (min loss reduction) │
│ └── num_boost_round: 50 (few rounds) │
│ │
│ 24 Fitur Input (Top 10): │
│ 1. RSI(14) 6. price_position │
│ 2. MACD_histogram 7. volatility_20 │
│ 3. ATR(14) 8. returns_5 │
│ 4. bb_width 9. ema_cross │
│ 5. returns_1 10. regime │
│ │
│ Output: │
│ ├── signal: BUY / SELL / HOLD │
│ ├── probability: 0.0 - 1.0 (prob of UP) │
│ └── confidence: 0.0 - 1.0 (prob of winning side) │
│ │
│ Validation: │
│ ├── Train/Test: 70% / 30% (50-bar gap, anti leakage) │
│ ├── Walk-forward: 500 train / 50 test / 50 step │
│ ├── Target AUC: > 0.65 │
│ ├── Rollback AUC: < 0.60 (v4: dinaikkan dari 0.52) │
│ └── Overfitting ratio: train_AUC/test_AUC < 1.15 │
└──────────────────────────────────────────────────────────┘
```
### Kombinasi Sinyal (SMC + ML)
```
SMC Signal: "BUY at 2645, SL 2635, TP 2665, conf 75%"
ML Signal: "BUY, confidence 72%"
│
▼
┌─ KOMBINASI ──────────────────────────────────────────────┐
│ │
│ CASE 1: SMC BUY + ML BUY (≥50%) │
│ → Combined confidence = (75% + 72%) / 2 = 73.5% │
│ → ENTRY (jika pass 11 filter lainnya) │
│ │
│ CASE 2: SMC BUY + ML SELL (≥65%) │
│ → ML strongly disagrees → BLOCK (filter #5) │
│ → TIDAK entry │
│ │
│ CASE 3: SMC BUY + ML uncertain (<50%) │
│ → ML tidak yakin → BLOCK (filter #4) │
│ → TIDAK entry │
│ │
│ CASE 4: Tidak ada SMC signal │
│ → Tidak ada entry point → SKIP │
│ → SMC adalah sinyal PRIMER (wajib ada) │
│ │
│ PRINSIP: │
│ SMC = sinyal UTAMA (menentukan entry/SL/TP) │
│ ML = KONFIRMASI (bisa memblokir, tidak bisa inisiasi)│
│ HMM = PENYESUAI (mengatur agresivitas) │
└──────────────────────────────────────────────────────────┘
```
---
## 9. Smart Money Concepts (SMC)
### 6 Konsep yang Dianalisis
```
┌──────────────────────────────────────────────────────────┐
│ 1. SWING POINTS (Fractal High/Low) │
│ │
│ Swing High: titik tertinggi dalam window 11 bar │
│ ↑ │
│ ____/\____ ← 5 bar kiri lebih rendah │
│ \____ ← 5 bar kanan lebih rendah │
│ │
│ Swing Low: titik terendah dalam window 11 bar │
│ ____ ____ ← 5 bar kiri lebih tinggi │
│ \____/ ← 5 bar kanan lebih tinggi │
│ ↑ │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ 2. FAIR VALUE GAP (FVG) — Ketidakseimbangan Harga │
│ │
│ Bullish FVG (gap up): │
│ Bar[i-2].high < Bar[i].low (ada gap) │
│ │
│ │ │ │
│ │ │ ← gap (FVG zone) │
│ │ ├────┐ │
│ ├───┘ │ │
│ │ │ │
│ │
│ Harga cenderung kembali mengisi FVG → entry zone │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ 3. ORDER BLOCK (OB) — Zona Institusi │
│ │
│ Bullish OB: candle bearish terakhir sebelum rally │
│ (zona dimana institusi menempatkan buy order besar) │
│ │
│ /───\ │
│ / \ │
│ ────\ / │
│ OB \_/ ← entry zone │
│ │
│ Lookback: 10 bar untuk deteksi │
│ Mitigated: true jika harga sudah revisit │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ 4. BREAK OF STRUCTURE (BOS) — Kelanjutan Trend │
│ │
│ Uptrend BOS: │
│ Harga break di ATAS swing high sebelumnya │
│ → Trend bullish berlanjut │
│ │
│ /\ /\ │
│ / \ / \ /\ ← BOS (break above prev high) │
│ / \/ \/ │
│ / │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ 5. CHANGE OF CHARACTER (CHoCH) — Perubahan Trend │
│ │
│ Uptrend → Downtrend: │
│ Harga break di BAWAH swing low terakhir │
│ → Trend berubah dari bullish ke bearish │
│ │
│ /\ /\ │
│ / \ / \ │
│ / \/ \ │
│ \____ ← CHoCH (break below prev low) │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ 6. LIQUIDITY ZONES — Target Likuiditas │
│ │
│ BSL (Buy Side Liquidity): di atas swing highs │
│ SSL (Sell Side Liquidity): di bawah swing lows │
│ │
│ ---- BSL ---- (stop loss para seller berkumpul) │
│ /\ /\ │
│ / \ / \ │
│ / \/ \ │
│ ---- SSL ---- (stop loss para buyer berkumpul) │
│ │
│ Institusi sering "hunt" liquidity zone ini │
└──────────────────────────────────────────────────────────┘
```
### Signal Generation
```
SYARAT SINYAL SMC:
Structure Break (BOS atau CHoCH)
+
Zone (FVG atau Order Block)
=
VALID SIGNAL
BUY Signal:
├── BOS bullish ATAU CHoCH bearish→bullish
├── + Bullish FVG ATAU Bullish OB di bawah harga
├── Entry: harga saat ini
├── SL: di bawah zone, minimum 1.5 × ATR
├── TP: 2:1 R:R, maximum 4 × ATR
└── Confidence: 40-85% (v5: calibrated weighted scoring) (lebih banyak confluence = lebih tinggi)
SELL Signal:
├── BOS bearish ATAU CHoCH bullish→bearish
├── + Bearish FVG ATAU Bearish OB di atas harga
├── Entry: harga saat ini
├── SL: di atas zone, minimum 1.5 × ATR
├── TP: 2:1 R:R, maximum 4 × ATR
└── Confidence: 40-85% (v5: calibrated weighted scoring)
```
---
## 10. Position Lifecycle
### Dari Lahir Sampai Mati (Siklus Hidup Posisi)
```
╔═══════════════════════════════════════════════════════════════╗
║ TAHAP 1: SINYAL TERDETEKSI ║
║ ║
║ SMC menemukan setup + ML konfirmasi + 11 filter PASS ║
║ → Keputusan: BUKA POSISI ║
╚═══════════════════════════════════════════════════════════════╝
│
▼
╔═══════════════════════════════════════════════════════════════╗
║ TAHAP 2: LOT SIZE CALCULATION ║
║ ║
║ Risk Engine (Kelly): ║
║ Balance $5000 × 1% risk = $50 max loss ║
║ SL distance 50 pips → lot = 0.01 ║
║ ║
║ ML Confidence boost: ║
║ ML ≥ 80% → 0.02 lot (double) ║
║ ML < 65% → 0.01 lot (minimum) ║
║ ║
║ Session multiplier: ║
║ Golden: × 1.2, Sydney: × 0.5 ║
║ ║
║ Regime multiplier: ║
║ Normal: × 1.0, High Vol: × 0.5, Crisis: × 0.0 ║
╚═══════════════════════════════════════════════════════════════╝
│
▼
╔═══════════════════════════════════════════════════════════════╗
║ TAHAP 3: ORDER VALIDATION ║
║ ║
║ Risk Engine memvalidasi: ║
║ ✓ SL di sisi yang benar (BUY: SL < entry) ║
║ ✓ TP di sisi yang benar (BUY: TP > entry) ║
║ ✓ Lot dalam range (0.01 - 0.05) ║
║ ✓ Entry dekat harga saat ini (< 0.1%) ║
║ ✓ Risk% ≤ 1.5× config limit ║
║ ✓ Circuit breaker TIDAK aktif ║
╚═══════════════════════════════════════════════════════════════╝
│
▼
╔═══════════════════════════════════════════════════════════════╗
║ TAHAP 4: ORDER EXECUTION ║
║ ║
║ MT5 Connector mengirim order: ║
║ → Symbol: XAUUSD ║
║ → Type: BUY/SELL ║
║ → Lot: 0.01-0.02 ║
║ → SL: ATR-based (broker level) ║
║ → TP: 2:1 R:R (broker level) ║
║ → Deviation: 20 points (slippage tolerance) ║
║ → Retry: max 3 attempts jika gagal ║
╚═══════════════════════════════════════════════════════════════╝
│
▼
╔═══════════════════════════════════════════════════════════════╗
║ TAHAP 4b: POST-EXECUTION VALIDATION (v5 BARU) ║
║ ║
║ Slippage Validation: ║
║ → Bandingkan harga aktual vs expected ║
║ → Max acceptable: 0.15% dari harga (~$4 untuk XAUUSD) ║
║ → Log WARNING jika melebihi batas ║
║ → Gunakan harga AKTUAL untuk position tracking ║
║ ║
║ Partial Fill Handling: ║
║ → Cek apakah volume terisi = volume diminta ║
║ → Jika partial: log fill ratio, update lot_size aktual ║
║ → Risk calculation tetap akurat dengan volume sebenarnya ║
╚═══════════════════════════════════════════════════════════════╝
│
▼
╔═══════════════════════════════════════════════════════════════╗
║ TAHAP 5: POSITION REGISTERED ║
║ ║
║ Smart Risk Manager: ║
║ → Catat entry price AKTUAL, direction, lot AKTUAL, timestamp║
║ → v5: Menggunakan harga & volume dari broker (bukan planned)║
║ → Inisialisasi peak_profit = 0 ║
║ → Mulai tracking momentum ║
║ ║
║ Trade Logger: ║
║ → Insert ke PostgreSQL (30+ field) ║
║ → Backup ke CSV ║
║ ║
║ Telegram: ║
║ → Kirim notifikasi trade open ║
║ → Detail: entry, SL, TP, R:R, confidence, regime ║
╚═══════════════════════════════════════════════════════════════╝
│
▼
╔═══════════════════════════════════════════════════════════════╗
║ TAHAP 6: ACTIVE MONITORING (setiap ~10 detik + candle baru) ║
║ ║
║ ┌── Update profit/loss real-time ║
║ ├── Update peak profit (tertinggi yang pernah dicapai) ║
║ ├── Hitung momentum (kecepatan perubahan profit) ║
║ ├── Hitung TP probability ║
║ ├── Cek 10 kondisi exit (lihat bagian 6) ║
║ ├── Cek Position Manager (trailing, breakeven) ║
║ └── Cek Market Close Handler (dekat close?) ║
║ ║
║ Setiap ~10 detik (atau candle baru), posisi dievaluasi: ║
║ → HOLD (lanjut) ║
║ → CLOSE (tutup dengan alasan spesifik) ║
╚═══════════════════════════════════════════════════════════════╝
│
│ trigger close
▼
╔═══════════════════════════════════════════════════════════════╗
║ TAHAP 7: POSITION CLOSED ║
║ ║
║ MT5 Connector: ║
║ → Close position via market order ║
║ ║
║ Smart Risk Manager: ║
║ → Record profit/loss ║
║ → Update daily/total loss counters ║
║ → Update win/loss streak ║
║ → Check mode transition (NORMAL→RECOVERY→PROTECTED→STOPPED)║
║ ║
║ Trade Logger: ║
║ → Update trade record: exit price, profit, duration, reason ║
║ → Update PostgreSQL + CSV ║
║ ║
║ Telegram: ║
║ → Kirim notifikasi trade close ║
║ → Detail: profit, duration, exit reason, balance ║
╚═══════════════════════════════════════════════════════════════╝
```
---
## 11. Auto-Retraining & Model Management
### Lifecycle Model AI
```
┌──────────────────────────────────────────────────────────────┐
│ INITIAL TRAINING (train_models.py) │
│ Dijalankan 1x saat setup │
│ │
│ 1. Fetch 10,000 bar M15 dari MT5 (~104 hari) │
│ 2. Feature Engineering → 40+ fitur │
│ 3. SMC Analysis → struktur pasar │
│ 4. Create target → UP/DOWN (lookahead=1) │
│ 4b. Split 70/30 dengan 50-bar gap (anti temporal leakage) │
│ 5. Train HMM (3 regime, lookback=500) │
│ 6. Train XGBoost (50 rounds, early_stop=5) │
│ 7. Walk-forward validation (500 train/50 test/50 step) │
│ 8. Save → models/hmm_regime.pkl + xgboost_model.pkl │
└──────────────────────────────┬───────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────┐
│ DAILY AUTO-RETRAINING (Auto Trainer) │
│ Otomatis setiap hari 05:00 WIB │
│ │
│ Schedule: │
│ ├── Harian (05:00 WIB): 8,000 bar, 50 rounds │
│ ├── Weekend (05:00 Sabtu): 15,000 bar, 80 rounds (deep) │
│ └── Emergency: jika AUC < 0.65 (kualitas turun) │
│ │
│ Proses: │
│ 1. Backup model saat ini → models/backup/ │
│ 2. Fetch data baru dari MT5 │
│ 3. Feature Engineering + SMC │
│ 4. Train HMM baru + XGBoost baru │
│ 5. Validasi: test AUC ≥ 0.60? (v4: dinaikkan dari 0.52) │
│ ├── Ya → Save model baru, reload di memory │
│ └── Tidak → ROLLBACK ke model sebelumnya │
│ 6. Log hasil ke PostgreSQL │
│ 7. Kirim laporan via Telegram │
│ │
│ Safety: │
│ ├── Max 5 backup disimpan (rotasi) │
│ ├── Min 20 jam antar retrain (cooldown) │
│ ├── Auto-rollback jika AUC < 0.60 (v4 threshold) │
│ └── Model lama selalu tersedia untuk rollback │
└──────────────────────────────────────────────────────────────┘
```
### Perbandingan Initial vs Auto Training
| Aspek | train_models.py | Auto Trainer |
|-------|-----------------|-------------|
| Kapan | Manual, 1x setup | Otomatis, harian |
| Data | 10,000 bar | 8K (harian) / 15K (weekend) |
| Boost rounds | 50 | 50 (harian) / 80 (weekend) |
| Walk-forward | Ya | Tidak |
| Backup | Tidak | Ya (5 terakhir) |
| Rollback | Tidak | Ya (AUC < 0.60) |
| Database | Tidak | Ya (PostgreSQL) |
| Tujuan | Setup awal | Maintenance rutin |
---
## 12. Infrastruktur & Database
### PostgreSQL Schema
```
trading_db
├── trades (Semua trade: open, close, profit, SMC, ML, features)
├── training_runs (Log setiap training: AUC, akurasi, durasi, rollback)
├── signals (Setiap sinyal yang dihasilkan: executed atau tidak)
├── market_snapshots (Snapshot periodik: harga, regime, volatilitas)
├── bot_status (Status bot: uptime, loop count, balance, risk mode)
└── daily_summaries (Ringkasan harian: win rate, profit factor, per sesi)
```
### Tabel `trades` (Detail)
```sql
-- Identifikasi
ticket, symbol, direction (BUY/SELL)
-- Harga
entry_price, exit_price, stop_loss, take_profit
-- Hasil
lot_size, profit_usd, profit_pips
opened_at, closed_at, duration_seconds
-- Konteks Entry
entry_regime, entry_volatility, entry_session
smc_signal, smc_confidence, smc_reason
smc_fvg_detected, smc_ob_detected, smc_bos_detected, smc_choch_detected
ml_signal, ml_confidence
market_quality, market_score, dynamic_threshold
-- Konteks Exit
exit_reason, exit_regime, exit_ml_signal
-- Keuangan
balance_before, balance_after, equity_at_entry
-- Data Lengkap
features_entry (JSON), features_exit (JSON)
bot_version, trade_mode
```
### Connection Architecture
```
Bot Components
├── TradeLogger → TradeRepository, SignalRepository, MarketSnapshotRepository
├── AutoTrainer → TrainingRepository
├── main_live.py → BotStatusRepository, DailySummaryRepository
└── Dashboard → Semua repository (READ)
│
▼
DatabaseConnection (Singleton)
│
▼
ThreadedConnectionPool (1-10 koneksi)
│
▼
PostgreSQL Server
```
### Graceful Degradation
```
PostgreSQL tersedia?
├── Ya → Gunakan DB + CSV backup (dual write)
└── Tidak → CSV saja (bot tetap berjalan 100%)
Bot TIDAK PERNAH crash karena database.
Semua operasi DB dibungkus try-except.
```
---
## 13. Konfigurasi & Parameter Kritis
### Configuration System
```
.env file
│
▼
TradingConfig.from_env()
│
├── RiskConfig
│ ├── risk_per_trade: 1.0% (SMALL) / 0.5% (MEDIUM)
│ ├── max_daily_loss: 3.0% (SMALL) / 2.0% (MEDIUM)
│ ├── max_total_loss: 10.0%
│ ├── max_positions: 3 (SMALL) / 5 (MEDIUM)
│ ├── min_lot: 0.01
│ ├── max_lot: 0.05 (SMALL) / 2.0 (MEDIUM)
│ └── max_leverage: 1:100 (SMALL) / 1:30 (MEDIUM)
│
├── SMCConfig
│ ├── swing_length: 5
│ ├── fvg_min_gap_pips: 2.0
│ ├── ob_lookback: 10
│ └── bos_close_break: true
│
├── MLConfig
│ ├── confidence_threshold: 0.65
│ ├── entry_confidence: 0.70
│ ├── high_confidence: 0.75
│ ├── very_high_confidence: 0.80
│ └── retrain_frequency_days: 7
│
├── ThresholdsConfig
│ ├── ml_min_confidence: 0.65
│ ├── ml_high_confidence: 0.75
│ ├── trade_cooldown_seconds: 300
│ ├── min_profit_to_secure: $15
│ ├── good_profit: $25
│ ├── great_profit: $40
│ ├── flash_crash_threshold: 2.5%
│ └── sydney_lot_multiplier: 0.5
│
└── RegimeConfig
├── n_regimes: 3
├── lookback: 500
└── retrain_frequency: 20
```
### Capital Mode Auto-Detection
```
Balance ≤ $10,000 → SMALL MODE
├── Risk: 1% per trade ($50 pada $5K)
├── Daily limit: 3% ($150)
├── Lot: 0.01-0.05
├── Leverage: 1:100
├── Timeframe: M15
└── Max posisi: 3
Balance > $10,000 → MEDIUM MODE
├── Risk: 0.5% per trade
├── Daily limit: 2%
├── Lot: 0.01-2.0
├── Leverage: 1:30
├── Timeframe: H1
└── Max posisi: 5
```
### Session Schedule (WIB = GMT+7)
```
┌────────────────────────────────────────────────────────────┐
│ WAKTU (WIB) │ SESI │ LOT MULT │ KETERANGAN │
├──────────────┼───────────────┼──────────┼─────────────────┤
│ 00:00-04:00 │ DEAD ZONE │ BLOCKED │ Likuiditas rendah│
│ 04:00-06:00 │ ROLLOVER │ BLOCKED │ Spread melebar │
│ 06:00-07:00 │ Sydney │ 0.5x │ Pasar baru buka │
│ 07:00-13:00 │ Tokyo+Sydney │ 0.7x │ Asia aktif │
│ 13:00-15:00 │ Tokyo akhir │ 0.7x │ Transisi │
│ 15:00-20:00 │ London │ 1.0x │ Volatilitas naik │
│ 20:00-23:59 │ ★ GOLDEN TIME │ 1.2x │ London+NY overlap│
│ Jumat ≥23:00 │ WEEKEND RISK │ BLOCKED │ Gap risk │
└────────────────────────────────────────────────────────────┘
★ Golden Time (20:00-23:59 WIB) = waktu paling optimal
→ Spread ketat, likuiditas maksimal, volatilitas ideal
→ Lot multiplier 1.2x (bonus)
→ v4: Smart Hold dihapus — tidak ada lagi hold losers menunggu sesi tertentu
```
---
## 14. Performa & Timing
### Main Loop Breakdown
```
Target: < 50ms per iterasi analisis
FULL ANALYSIS (saat candle baru M15):
┌────────────────────────────────────────────────────────────┐
│ Komponen │ Waktu │ Keterangan │
├────────────────────────┼──────────┼────────────────────────┤
│ MT5 data fetch │ ~10ms │ 200 bar M15 via API │
│ Feature engineering │ ~5ms │ 40+ fitur, Polars │
│ SMC analysis │ ~5ms │ 6 konsep, Polars native│
│ HMM predict │ ~2ms │ 2 fitur → 1 regime │
│ XGBoost predict │ ~3ms │ 24 fitur → 1 signal │
│ Position monitoring │ ~5ms │ Per posisi terbuka │
│ Entry logic │ ~5ms │ 11 filter check │
│ Overhead │ ~15ms │ Logging, state update │
├────────────────────────┼──────────┼────────────────────────┤
│ TOTAL │ ~50ms │ │
└────────────────────────────────────────────────────────────┘
POSITION CHECK ONLY (di antara candle, setiap ~10 detik):
┌────────────────────────────────────────────────────────────┐
│ Komponen │ Waktu │ Keterangan │
├────────────────────────┼──────────┼────────────────────────┤
│ MT5 data fetch │ ~5ms │ 50 bar saja │
│ Feature engineering │ ~3ms │ Minimal fitur │
│ ML prediction │ ~3ms │ Untuk exit evaluation │
│ Position evaluation │ ~5ms │ 10 kondisi exit │
│ Overhead │ ~5ms │ Logging │
├────────────────────────┼──────────┼────────────────────────┤
│ TOTAL │ ~21ms │ │
└────────────────────────────────────────────────────────────┘
```
### Timer Periodik
```
┌──────────────────────────────────────────────────────────┐
│ Event │ Interval │ Cara Trigger │
├────────────────────────┼────────────────┼─────────────────┤
│ Full analysis + entry │ Candle baru M15│ Deteksi candle │
│ Position monitoring │ ~10 detik │ Di antara candle│
│ Performance logging │ 4 candle (~1j) │ candle_count % 4│
│ Auto-retrain check │ 20 candle (~5j)│ candle_count %20│
│ Market update Telegram │ 30 menit │ Timer │
│ Hourly analysis │ 1 jam │ Timer │
│ Daily summary + reset │ Ganti hari │ Date check │
└──────────────────────────────────────────────────────────┘
```
---
## 15. Error Handling & Fault Tolerance
### Prinsip: Bot TIDAK PERNAH Crash
```
┌──────────────────────────────────────────────────────────┐
│ LEVEL 1: Per-Loop Error Handling │
│ │
│ try: │
│ # Fetch data, analyze, trade │
│ except ConnectionError: │
│ # MT5 disconnected → reconnect() │
│ except Exception as e: │
│ # Log error → lanjut loop berikutnya │
│ # Bot TIDAK crash dari error tunggal │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ LEVEL 2: MT5 Auto-Reconnect │
│ │
│ MT5 putus? │
│ ├── Attempt 1: reconnect (tunggu 2 detik) │
│ ├── Attempt 2: reconnect (tunggu 4 detik) │
│ ├── Attempt 3: reconnect (tunggu 8 detik) │
│ ├── Cooldown 60 detik │
│ └── Retry cycle (max 5 per cooldown) │
│ │
│ Selama disconnected: │
│ → Position monitoring PAUSE │
│ → Entry baru DITUNDA │
│ → Posisi terbuka dilindungi broker SL (lapis 1) │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ LEVEL 3: Database Graceful Degradation │
│ │
│ PostgreSQL down? │
│ ├── Switch ke CSV-only mode │
│ ├── Semua data tetap dicatat │
│ ├── Trading tetap berjalan normal │
│ └── Retry DB connection periodik │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ LEVEL 4: Telegram Failure │
│ │
│ Telegram API error? │
│ ├── Log error secara silent │
│ ├── Trading tetap jalan 100% │
│ └── Retry di notifikasi berikutnya │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ LEVEL 5: Model File Missing │
│ │
│ .pkl file tidak ditemukan? │
│ ├── Log warning │
│ ├── Skip prediksi (ML/HMM) │
│ ├── Trading bisa jalan tanpa ML (SMC only) │
│ └── Trigger: jalankan train_models.py │
└──────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────┐
│ LEVEL 6: Flash Crash Protection │
│ │
│ Harga bergerak > 2.5% dalam 1 menit? │
│ ├── EMERGENCY: Close ALL positions │
│ ├── Circuit breaker AKTIF │
│ ├── Kirim alert KRITIS via Telegram │
│ └── Bot masuk mode STOPPED │
└──────────────────────────────────────────────────────────┘
```
### Startup & Shutdown
```
STARTUP SEQUENCE:
1. Load konfigurasi dari .env
2. Connect ke MT5 (max 3 retry)
3. Load model HMM dari models/hmm_regime.pkl
4. Load model XGBoost dari models/xgboost_model.pkl
5. Initialize SmartRiskManager (set balance, limits)
6. Initialize SessionFilter (WIB timezone)
7. Initialize TelegramNotifier
8. Initialize TradeLogger (connect DB)
9. Initialize AutoTrainer
10. Send Telegram: "BOT STARTED" (config, balance, risk settings)
11. Mulai main loop
SHUTDOWN SEQUENCE (SIGINT/SIGTERM):
1. Signal diterima
2. Hentikan loop utama
3. Kirim Telegram: "BOT STOPPED" (balance, trades, uptime)
4. Disconnect MT5
5. Close database connections
6. Exit
```
---
## 16. Daftar File Source Code
```
Smart Automatic Trading BOT + AI/
│
├── main_live.py # Orchestrator utama (TradingBot)
├── train_models.py # Script training awal
├── .env # Environment variables (credentials)
│
├── src/
│ ├── config.py # Konfigurasi terpusat (6 sub-config)
│ ├── mt5_connector.py # Bridge ke MetaTrader 5
│ ├── feature_eng.py # Feature Engineering (40+ fitur)
│ ├── regime_detector.py # HMM Regime Detection (3 state)
│ ├── ml_model.py # XGBoost Signal Predictor
│ ├── smc_polars.py # Smart Money Concepts (6 konsep)
│ ├── smart_risk_manager.py # 4-Mode Risk Manager
│ ├── risk_engine.py # Kelly Criterion + Circuit Breaker
│ ├── session_filter.py # Session Time Filter (WIB)
│ ├── dynamic_confidence.py # Dynamic Threshold Manager
│ ├── news_agent.py # News Event Monitor
│ ├── telegram_notifier.py # Telegram Push Notifications
│ ├── auto_trainer.py # Daily Auto-Retraining
│ ├── trade_logger.py # Dual Storage Logger (DB+CSV)
│ ├── position_manager.py # Position Manager + Market Close
│ │
│ └── db/
│ ├── __init__.py # DB exports
│ ├── connection.py # PostgreSQL Singleton + Pool
│ └── repository.py # 6 Repository classes
│
├── models/
│ ├── xgboost_model.pkl # Trained XGBoost model
│ ├── hmm_regime.pkl # Trained HMM model
│ └── backup/ # Auto-backup (5 terakhir)
│
├── data/
│ ├── training_data.parquet # Data training terakhir
│ └── trade_logs/ # CSV backup (per bulan)
│ ├── trades_2025_01.csv
│ ├── trades_2025_02.csv
│ └── ...
│
├── backtests/
│ └── backtest_live_sync.py # Backtest 100% sync live
│
├── logs/
│ └── training_YYYY-MM-DD.log # Log training detail
│
└── docs/
└── arsitektur-ai/
├── 00-ARSITEKTUR-LENGKAP.md # Dokumen ini
├── README.md # Index komponen
└── 01-23 (per komponen) # Detail per modul
```
---
## Ringkasan Eksekutif
**Smart AI Trading Bot** adalah sistem trading otomatis yang menggabungkan:
1. **Smart Money Concepts (SMC)** sebagai sinyal UTAMA — mendeteksi zona institusi (FVG, Order Block, BOS, CHoCH) untuk menentukan entry, SL, dan TP yang presisi.
2. **XGBoost Machine Learning** sebagai KONFIRMASI — memprediksi arah harga dengan 24 fitur teknikal, memblokir trade jika tidak setuju dengan SMC.
3. **Hidden Markov Model (HMM)** sebagai PENYESUAI — mendeteksi kondisi pasar (tenang/volatile/krisis) untuk menyesuaikan agresivitas.
4. **4-Lapis Proteksi Risiko** — dari broker SL, software smart exit, emergency stop, hingga circuit breaker. Lot ultra-kecil (0.01-0.02) memastikan kerugian per trade maximum $50 (1%).
5. **Self-Improving** — model AI dilatih ulang otomatis setiap hari dengan auto-rollback jika model baru lebih buruk.
6. **Fault-Tolerant** — bot tidak pernah crash. MT5 putus? Auto-reconnect. Database mati? CSV fallback. Error? Log dan lanjut.
Semua ini dikoordinasikan oleh **Main Live Orchestrator** yang menjalankan loop **candle-based** — analisis penuh hanya saat candle M15 baru terbentuk (~50ms per iterasi), dengan pengecekan posisi setiap ~10 detik di antara candle (~21ms). Mengevaluasi 11 filter entry dan 10 kondisi exit secara real-time, dengan notifikasi Telegram untuk setiap kejadian penting.
```
TARGET: Trading XAUUSD M15 yang KONSISTEN dan AMAN
dengan kerugian terkontrol dan profit teroptimasi.
```
Reference in New Issue View Git Blame Copy Permalink