docs: add architecture documentation for remaining 8 components (16-23)
New documentation files: - 16-MT5-Connector: Broker bridge with auto-reconnect & Polars native - 17-Configuration: 6 sub-configs with capital mode auto-adjustment - 18-Trade-Logger: Dual storage (PostgreSQL + CSV), thread-safe - 19-Position-Manager: 7 action conditions, trailing SL, market close handler - 20-Risk-Engine: Kelly Criterion sizing, circuit breaker, order validation - 21-Database: PostgreSQL integration with 6 repositories - 22-Train-Models: Initial training script (HMM + XGBoost) - 23-Main-Live-Orchestrator: Main loop coordinating 15+ components Updated README.md with complete index of all 23 components. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
GifariKemal
@@ -0,0 +1,205 @@
|
||||
# MT5 Connector — Jembatan ke MetaTrader 5
|
||||
|
||||
> **File:** `src/mt5_connector.py`
|
||||
> **Class:** `MT5Connector`, `MT5SimulationConnector`
|
||||
> **Library:** MetaTrader5 (Python API)
|
||||
|
||||
---
|
||||
|
||||
## Apa Itu MT5 Connector?
|
||||
|
||||
MT5 Connector adalah **jembatan komunikasi** antara bot AI dan terminal MetaTrader 5. Semua interaksi dengan broker — ambil data harga, kirim order, cek posisi — dilakukan melalui modul ini.
|
||||
|
||||
**Analogi:** MT5 Connector seperti **penerjemah di bandara** — menerjemahkan perintah bot (Python) ke bahasa yang dipahami broker (MT5 API), dan sebaliknya.
|
||||
|
||||
---
|
||||
|
||||
## Fungsi Utama
|
||||
|
||||
| Method | Fungsi | Return |
|
||||
|--------|--------|--------|
|
||||
| `connect()` | Koneksi ke MT5 terminal | `bool` |
|
||||
| `disconnect()` | Putus koneksi | - |
|
||||
| `reconnect()` | Reconnect otomatis | `bool` |
|
||||
| `ensure_connected()` | Cek & auto-reconnect | `bool` |
|
||||
| `get_market_data()` | Ambil data OHLCV | `pl.DataFrame` |
|
||||
| `get_tick()` | Ambil harga real-time | `TickData` |
|
||||
| `send_order()` | Kirim order BUY/SELL | `OrderResult` |
|
||||
| `close_position()` | Tutup posisi | `OrderResult` |
|
||||
| `get_open_positions()` | Cek posisi terbuka | `pl.DataFrame` |
|
||||
| `get_symbol_info()` | Info simbol (spread, dll) | `Dict` |
|
||||
|
||||
---
|
||||
|
||||
## Koneksi & Auto-Reconnect
|
||||
|
||||
```
|
||||
connect(max_retries=3)
|
||||
|
|
||||
v
|
||||
Shutdown koneksi lama (jika ada)
|
||||
|
|
||||
v
|
||||
mt5.initialize(login, password, server)
|
||||
|
|
||||
v
|
||||
Tunggu 2 detik (stabilisasi terminal)
|
||||
|
|
||||
v
|
||||
Verifikasi: terminal_info() != None?
|
||||
|
|
||||
├── Ya → Cek terminal.connected?
|
||||
│ ├── Ya → ✅ Connected!
|
||||
│ └── Tidak → Tunggu 3 detik → Retry
|
||||
│
|
||||
└── Tidak → Exponential backoff (2s, 4s, 8s) → Retry
|
||||
```
|
||||
|
||||
### Auto-Reconnect
|
||||
|
||||
```python
|
||||
ensure_connected():
|
||||
"""
|
||||
Dipanggil sebelum setiap operasi penting.
|
||||
|
||||
1. Cek flag _connected
|
||||
2. Coba mt5.account_info()
|
||||
3. Gagal? → reconnect()
|
||||
4. Max 5 attempts, lalu cooldown 60 detik
|
||||
"""
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data Fetching (Polars Native)
|
||||
|
||||
```python
|
||||
get_market_data(symbol="XAUUSD", timeframe="M15", count=200)
|
||||
```
|
||||
|
||||
**Proses:**
|
||||
|
||||
```
|
||||
MT5 Terminal
|
||||
|
|
||||
v
|
||||
mt5.copy_rates_from_pos() → numpy structured array
|
||||
|
|
||||
v
|
||||
LANGSUNG ke Polars DataFrame (TANPA Pandas)
|
||||
|
|
||||
v
|
||||
Cast types:
|
||||
├── time: Unix timestamp → Datetime
|
||||
├── open/high/low/close: Float64
|
||||
├── tick_volume → volume (Int64)
|
||||
└── spread, real_volume: Int64
|
||||
|
|
||||
v
|
||||
Return pl.DataFrame
|
||||
```
|
||||
|
||||
**Kolom output:**
|
||||
|
||||
| Kolom | Tipe | Keterangan |
|
||||
|-------|------|------------|
|
||||
| `time` | Datetime | Waktu candle |
|
||||
| `open` | Float64 | Harga buka |
|
||||
| `high` | Float64 | Harga tertinggi |
|
||||
| `low` | Float64 | Harga terendah |
|
||||
| `close` | Float64 | Harga tutup |
|
||||
| `volume` | Int64 | Tick volume |
|
||||
| `spread` | Int64 | Spread |
|
||||
| `real_volume` | Int64 | Real volume |
|
||||
|
||||
---
|
||||
|
||||
## Order Execution
|
||||
|
||||
```python
|
||||
send_order(
|
||||
symbol="XAUUSD",
|
||||
order_type="BUY", # atau "SELL"
|
||||
volume=0.01, # Lot size
|
||||
sl=4937.00, # Stop Loss
|
||||
tp=4976.00, # Take Profit
|
||||
deviation=20, # Max slippage (points)
|
||||
magic=123456, # Bot ID
|
||||
comment="AI Bot",
|
||||
max_retries=3,
|
||||
)
|
||||
```
|
||||
|
||||
**Retry Logic:**
|
||||
|
||||
```
|
||||
Kirim order
|
||||
|
|
||||
├── RETCODE 10009 (DONE) → ✅ Success
|
||||
|
|
||||
├── RETCODE 10013-10016 (INVALID) → ❌ Non-retryable
|
||||
|
|
||||
├── RETCODE 10027 (TRADE DISABLED) → ❌ Raise error
|
||||
|
|
||||
└── RETCODE lain (requote/reject) → 🔄 Retry (max 3x)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Timeframe Mapping
|
||||
|
||||
| String | MT5 Constant | Penggunaan |
|
||||
|--------|-------------|------------|
|
||||
| `M1` | TIMEFRAME_M1 | 1 menit |
|
||||
| `M5` | TIMEFRAME_M5 | 5 menit |
|
||||
| `M15` | TIMEFRAME_M15 | **Utama** (execution) |
|
||||
| `M30` | TIMEFRAME_M30 | 30 menit |
|
||||
| `H1` | TIMEFRAME_H1 | 1 jam |
|
||||
| `H4` | TIMEFRAME_H4 | Trend analysis |
|
||||
| `D1` | TIMEFRAME_D1 | 1 hari |
|
||||
|
||||
---
|
||||
|
||||
## Error Codes
|
||||
|
||||
| Code | Nama | Aksi |
|
||||
|------|------|------|
|
||||
| 10009 | DONE | Order berhasil |
|
||||
| 10004 | REQUOTE | Retry |
|
||||
| 10006 | REJECT | Retry |
|
||||
| 10013 | INVALID | Stop, order salah |
|
||||
| 10014 | INVALID_VOLUME | Stop, lot salah |
|
||||
| 10015 | INVALID_PRICE | Stop, harga salah |
|
||||
| 10016 | INVALID_STOPS | Stop, SL/TP salah |
|
||||
| 10027 | TRADE_DISABLED | AutoTrading off |
|
||||
| -10003 | NO_CONNECTION | Reconnect |
|
||||
| -10004 | NO_IPC | Reconnect |
|
||||
|
||||
---
|
||||
|
||||
## Simulation Mode
|
||||
|
||||
```python
|
||||
class MT5SimulationConnector(MT5Connector):
|
||||
"""
|
||||
Untuk testing tanpa MT5 terminal.
|
||||
|
||||
- connect() selalu berhasil
|
||||
- get_market_data() generate data sintetis (random walk)
|
||||
- Base price XAUUSD: $2000
|
||||
"""
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Konfigurasi Koneksi
|
||||
|
||||
```python
|
||||
MT5Connector(
|
||||
login=12345678, # Dari .env MT5_LOGIN
|
||||
password="password123", # Dari .env MT5_PASSWORD
|
||||
server="BrokerServer-Live", # Dari .env MT5_SERVER
|
||||
path="C:/Program Files/MT5/...", # Dari .env MT5_PATH (opsional)
|
||||
timeout=60000, # 60 detik timeout
|
||||
)
|
||||
```
|
||||
@@ -0,0 +1,164 @@
|
||||
# Configuration — Pusat Pengaturan Bot
|
||||
|
||||
> **File:** `src/config.py`
|
||||
> **Class:** `TradingConfig`, `RiskConfig`, `SMCConfig`, `MLConfig`, `ThresholdsConfig`, `RegimeConfig`
|
||||
> **Sumber:** Environment variables (`.env`)
|
||||
|
||||
---
|
||||
|
||||
## Apa Itu Configuration?
|
||||
|
||||
Configuration adalah **pusat pengaturan** seluruh parameter bot — dari kredensial MT5 hingga threshold AI. Semua pengaturan otomatis menyesuaikan berdasarkan ukuran modal (small/medium).
|
||||
|
||||
**Analogi:** Configuration seperti **kokpit pesawat** — semua tombol dan dial pengaturan ada di satu tempat, dan bisa diubah sebelum "terbang" (trading).
|
||||
|
||||
---
|
||||
|
||||
## Capital Mode (Otomatis)
|
||||
|
||||
| Mode | Modal | Risk/Trade | Max Daily Loss | Leverage | Max Lot | Max Posisi | Timeframe |
|
||||
|------|-------|-----------|----------------|----------|---------|-----------|-----------|
|
||||
| **SMALL** | ≤ $10K | 1% | 3% | 1:100 | 0.05 | 3 | M15 |
|
||||
| **MEDIUM** | > $10K | 0.5% | 2% | 1:30 | 2.0 | 5 | H1 |
|
||||
|
||||
```python
|
||||
# Otomatis berdasarkan capital
|
||||
if capital <= 10000:
|
||||
mode = SMALL # Growth mode
|
||||
else:
|
||||
mode = MEDIUM # Preservation mode
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6 Sub-Konfigurasi
|
||||
|
||||
### 1. RiskConfig
|
||||
|
||||
```python
|
||||
RiskConfig(
|
||||
risk_per_trade=1.0, # 1% per trade ($50 dari $5K)
|
||||
max_daily_loss=3.0, # 3% max daily loss ($150)
|
||||
max_leverage=100, # 1:100
|
||||
max_positions=3, # Max 3 posisi bersamaan
|
||||
max_lot_size=0.05, # Max 0.05 lot
|
||||
min_lot_size=0.01, # Min 0.01 lot
|
||||
lot_step=0.01, # Increment 0.01
|
||||
)
|
||||
```
|
||||
|
||||
### 2. SMCConfig
|
||||
|
||||
```python
|
||||
SMCConfig(
|
||||
swing_length=5, # 5 bar untuk swing detection
|
||||
fvg_min_gap_pips=2.0, # Min gap FVG: 2 pips
|
||||
ob_lookback=10, # Order block lookback: 10 bar
|
||||
bos_close_break=True, # Butuh close break untuk BOS
|
||||
)
|
||||
```
|
||||
|
||||
### 3. MLConfig
|
||||
|
||||
```python
|
||||
MLConfig(
|
||||
model_path="models/xgboost_model.json",
|
||||
confidence_threshold=0.65, # Min confidence untuk entry
|
||||
retrain_frequency_days=7, # Retrain setiap 7 hari
|
||||
lookback_periods=1000, # Data lookback
|
||||
)
|
||||
```
|
||||
|
||||
### 4. ThresholdsConfig
|
||||
|
||||
```python
|
||||
ThresholdsConfig(
|
||||
# ML Confidence
|
||||
ml_min_confidence=0.65, # Minimum confidence
|
||||
ml_entry_confidence=0.70, # Default entry
|
||||
ml_high_confidence=0.75, # High confidence
|
||||
ml_very_high_confidence=0.80, # Lot multiplier trigger
|
||||
|
||||
# Risk
|
||||
trend_reversal_confidence=0.75, # Trigger reversal close
|
||||
protected_mode_threshold=0.80, # Enter protected mode
|
||||
|
||||
# Profit/Loss (USD)
|
||||
min_profit_to_secure=15.0, # Min profit to consider secure
|
||||
good_profit_level=25.0, # Good profit
|
||||
great_profit_level=40.0, # Take it!
|
||||
|
||||
# Timing
|
||||
trade_cooldown_seconds=300, # 5 menit antar trade
|
||||
loop_interval_seconds=30.0, # Main loop interval
|
||||
|
||||
# Session
|
||||
sydney_lot_multiplier=0.5, # Sydney lot reduction
|
||||
)
|
||||
```
|
||||
|
||||
### 5. RegimeConfig
|
||||
|
||||
```python
|
||||
RegimeConfig(
|
||||
n_regimes=3, # 3 HMM states
|
||||
lookback_periods=500, # HMM training lookback
|
||||
retrain_frequency=20, # Retrain setiap 20 bar
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Environment Variables (.env)
|
||||
|
||||
| Variable | Contoh | Wajib | Keterangan |
|
||||
|----------|--------|-------|------------|
|
||||
| `MT5_LOGIN` | `12345678` | Ya | Akun MT5 |
|
||||
| `MT5_PASSWORD` | `p@ssw0rd` | Ya | Password MT5 |
|
||||
| `MT5_SERVER` | `BrokerName-Live` | Ya | Server broker |
|
||||
| `MT5_PATH` | `C:\...\terminal64.exe` | Tidak | Path MT5 |
|
||||
| `CAPITAL` | `5000` | Tidak | Modal ($5000 default) |
|
||||
| `SYMBOL` | `XAUUSD` | Tidak | Simbol trading |
|
||||
| `RISK_PER_TRADE` | `1.0` | Tidak | Override risk % |
|
||||
| `MAX_DAILY_LOSS_PERCENT` | `3.0` | Tidak | Override daily loss |
|
||||
| `AI_CONFIDENCE_THRESHOLD` | `0.65` | Tidak | Override ML threshold |
|
||||
| `TELEGRAM_BOT_TOKEN` | `123:ABC...` | Tidak | Token Telegram |
|
||||
| `TELEGRAM_CHAT_ID` | `-1001234...` | Tidak | Chat ID Telegram |
|
||||
| `DB_HOST` | `localhost` | Tidak | PostgreSQL host |
|
||||
| `DB_NAME` | `trading_db` | Tidak | Database name |
|
||||
|
||||
---
|
||||
|
||||
## Position Sizing (Kelly Criterion)
|
||||
|
||||
```python
|
||||
def calculate_position_size(entry_price, stop_loss_price, balance):
|
||||
"""
|
||||
Risk-Constrained Kelly Criterion:
|
||||
|
||||
risk_amount = balance × risk% ($5000 × 1% = $50)
|
||||
sl_pips = |entry - SL| / 0.1
|
||||
lot = risk_amount / (sl_pips × pip_value)
|
||||
lot × 0.5 (Half-Kelly untuk safety)
|
||||
|
||||
Clamp: min_lot ≤ lot ≤ max_lot
|
||||
"""
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Validasi Otomatis
|
||||
|
||||
```
|
||||
Saat TradingConfig dibuat:
|
||||
|
|
||||
v
|
||||
_validate_required_settings():
|
||||
├── MT5_LOGIN != 0?
|
||||
├── MT5_PASSWORD tidak kosong?
|
||||
├── MT5_SERVER tidak kosong?
|
||||
└── Capital > 0?
|
||||
|
|
||||
├── Ada yang gagal → ValueError
|
||||
└── Semua OK → _configure_by_capital()
|
||||
```
|
||||
@@ -0,0 +1,155 @@
|
||||
# Trade Logger — Pencatat Trade Otomatis
|
||||
|
||||
> **File:** `src/trade_logger.py`
|
||||
> **Class:** `TradeLogger`
|
||||
> **Storage:** PostgreSQL (primary) + CSV (fallback)
|
||||
|
||||
---
|
||||
|
||||
## Apa Itu Trade Logger?
|
||||
|
||||
Trade Logger mencatat **setiap trade, sinyal, dan kondisi pasar** secara otomatis ke database dan file CSV. Data ini digunakan untuk analisis performa, retraining ML model, dan debugging.
|
||||
|
||||
**Analogi:** Trade Logger seperti **black box di pesawat** — merekam semua yang terjadi untuk analisis setelah penerbangan (trading).
|
||||
|
||||
---
|
||||
|
||||
## 3 Tipe Data yang Dicatat
|
||||
|
||||
### 1. Trade Record (Per Trade)
|
||||
|
||||
Setiap trade dibuka/ditutup dicatat lengkap:
|
||||
|
||||
| Kategori | Field |
|
||||
|----------|-------|
|
||||
| **Identitas** | ticket, symbol |
|
||||
| **Trade** | direction, lot_size, entry_price, exit_price, SL, TP |
|
||||
| **Hasil** | profit_usd, profit_pips, duration_seconds |
|
||||
| **Waktu** | open_time, close_time |
|
||||
| **Market** | regime, volatility, session, spread, ATR |
|
||||
| **SMC** | signal, confidence, reason, FVG/OB/BOS/CHoCH flags |
|
||||
| **ML** | signal, confidence |
|
||||
| **Dynamic** | market_quality, market_score, threshold |
|
||||
| **Exit** | exit_reason, exit_regime, exit_ml_signal |
|
||||
| **Balance** | balance_before, balance_after, equity_at_entry |
|
||||
| **Features** | JSON snapshot fitur saat entry & exit |
|
||||
|
||||
### 2. Signal Record (Per Sinyal)
|
||||
|
||||
Setiap sinyal yang dihasilkan (termasuk yang **tidak** dieksekusi):
|
||||
|
||||
```
|
||||
timestamp, symbol, price
|
||||
signal_type, signal_source, confidence
|
||||
smc_*, ml_*
|
||||
regime, session, volatility, market_score
|
||||
trade_executed (bool)
|
||||
execution_reason ("executed" / "below_threshold" / "max_positions" / ...)
|
||||
```
|
||||
|
||||
### 3. Market Snapshot (Periodik)
|
||||
|
||||
Snapshot kondisi pasar secara berkala:
|
||||
|
||||
```
|
||||
timestamp, symbol, price, OHLC
|
||||
regime, volatility, session, ATR, spread
|
||||
ml_signal, ml_confidence
|
||||
smc_signal, smc_confidence
|
||||
open_positions, floating_pnl
|
||||
features (JSON)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Dual Storage
|
||||
|
||||
```
|
||||
Event Terjadi (trade/signal/snapshot)
|
||||
|
|
||||
v
|
||||
┌─────────────────────┐ ┌──────────────────┐
|
||||
│ PostgreSQL (Primary) │ │ CSV (Fallback) │
|
||||
│ │ │ │
|
||||
│ ├── trades table │ │ data/trade_logs/│
|
||||
│ ├── signals table │ │ ├── trades/ │
|
||||
│ ├── market_snapshots │ │ │ └── trades_2025_02.csv
|
||||
│ └── bot_status │ │ ├── signals/ │
|
||||
│ │ │ │ └── signals_2025_02.csv
|
||||
│ Cepat, queryable, │ │ └── snapshots/ │
|
||||
│ thread-safe pooling │ │ └── snapshots_2025_02.csv
|
||||
└─────────────────────┘ │ │
|
||||
│ Selalu ditulis │
|
||||
│ (backup) │
|
||||
└──────────────────┘
|
||||
```
|
||||
|
||||
- **DB tidak tersedia?** → CSV saja (graceful degradation)
|
||||
- **DB tersedia?** → Tulis ke DB **DAN** CSV (double safety)
|
||||
|
||||
---
|
||||
|
||||
## Proses Log Trade
|
||||
|
||||
```
|
||||
Trade Dibuka:
|
||||
|
|
||||
v
|
||||
log_trade_open(ticket, entry_price, regime, smc_*, ml_*, ...)
|
||||
|
|
||||
├── Simpan ke _pending_trades[ticket] (di memory)
|
||||
└── INSERT ke database (trades table)
|
||||
|
||||
... trading berjalan ...
|
||||
|
||||
Trade Ditutup:
|
||||
|
|
||||
v
|
||||
log_trade_close(ticket, exit_price, profit, exit_reason, ...)
|
||||
|
|
||||
├── Ambil data pending dari memory
|
||||
├── Hitung durasi: close_time - open_time
|
||||
├── UPDATE database (exit_price, profit, duration, ...)
|
||||
└── APPEND ke CSV (trades_YYYY_MM.csv)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Analisis Helper
|
||||
|
||||
| Method | Fungsi |
|
||||
|--------|--------|
|
||||
| `get_recent_trades(10)` | 10 trade terakhir |
|
||||
| `get_win_rate(30)` | Win rate 30 hari |
|
||||
| `get_smc_performance(30)` | Performa per pattern SMC |
|
||||
| `get_trades_for_training(30)` | Data untuk ML retraining |
|
||||
| `get_stats()` | Statistik logger |
|
||||
|
||||
---
|
||||
|
||||
## Thread Safety
|
||||
|
||||
```python
|
||||
self._lock = threading.Lock()
|
||||
|
||||
# Setiap operasi CSV dilindungi lock
|
||||
with self._lock:
|
||||
# Write to CSV
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## File CSV (Terorganisir per Bulan)
|
||||
|
||||
```
|
||||
data/trade_logs/
|
||||
├── trades/
|
||||
│ ├── trades_2025_01.csv
|
||||
│ └── trades_2025_02.csv
|
||||
├── signals/
|
||||
│ ├── signals_2025_01.csv
|
||||
│ └── signals_2025_02.csv
|
||||
└── snapshots/
|
||||
├── snapshots_2025_01.csv
|
||||
└── snapshots_2025_02.csv
|
||||
```
|
||||
@@ -0,0 +1,165 @@
|
||||
# Position Manager — Manajemen Posisi Cerdas
|
||||
|
||||
> **File:** `src/position_manager.py`
|
||||
> **Class:** `SmartPositionManager`, `SmartMarketCloseHandler`
|
||||
> **Fitur:** Trailing SL, Profit Protection, Market Close Handler
|
||||
|
||||
---
|
||||
|
||||
## Apa Itu Position Manager?
|
||||
|
||||
Position Manager mengelola posisi terbuka secara **aktif dan cerdas** — trailing stop loss, proteksi profit, dan keputusan otomatis saat market mendekati penutupan.
|
||||
|
||||
**Analogi:** Position Manager seperti **co-pilot yang mengawasi perjalanan** — mengamankan keuntungan saat angin baik, dan mengambil tindakan darurat saat cuaca memburuk.
|
||||
|
||||
---
|
||||
|
||||
## 2 Komponen Utama
|
||||
|
||||
### A. SmartPositionManager
|
||||
|
||||
Mengelola posisi aktif: trailing SL, breakeven, profit protection.
|
||||
|
||||
### B. SmartMarketCloseHandler
|
||||
|
||||
Keputusan cerdas saat market mendekati penutupan (harian/weekend).
|
||||
|
||||
---
|
||||
|
||||
## SmartPositionManager — 7 Kondisi Aksi
|
||||
|
||||
Untuk setiap posisi terbuka, dicek berurutan:
|
||||
|
||||
### 0. Market Close Check (Prioritas Tertinggi)
|
||||
|
||||
```
|
||||
Dekat market close?
|
||||
├── Profit >= $10 + dekat close → CLOSE (amankan profit)
|
||||
├── Loss + dekat weekend + SL >50% hit → CLOSE (gap risk)
|
||||
├── Loss + dekat weekend + loss > $100 → CLOSE (gap risk)
|
||||
└── Loss kecil + dekat weekend → HOLD (bisa recovery Senin)
|
||||
```
|
||||
|
||||
### 1. Regime Danger
|
||||
|
||||
```
|
||||
Regime CRISIS atau HIGH_VOLATILITY + profit > $50:
|
||||
→ CLOSE (amankan profit dari volatilitas)
|
||||
```
|
||||
|
||||
### 2. Opposite Signal
|
||||
|
||||
```
|
||||
Posisi BUY + sinyal bearish kuat + profit > $25:
|
||||
→ CLOSE (amankan sebelum reversal)
|
||||
|
||||
Posisi SELL + sinyal bullish kuat + profit > $25:
|
||||
→ CLOSE (amankan sebelum reversal)
|
||||
```
|
||||
|
||||
### 3. Drawdown from Peak
|
||||
|
||||
```
|
||||
Peak profit > $50 DAN drawdown > 30% dari peak:
|
||||
→ CLOSE (profit sudah turun terlalu banyak)
|
||||
|
||||
Contoh: Peak $80, sekarang $50 → drawdown 37.5% → CLOSE
|
||||
```
|
||||
|
||||
### 4. High Urgency
|
||||
|
||||
```
|
||||
Urgency score >= 7 (dari 10) DAN profit > 0:
|
||||
→ CLOSE (banyak sinyal bahaya bersamaan)
|
||||
```
|
||||
|
||||
### 5. Breakeven Protection
|
||||
|
||||
```
|
||||
Profit >= 15 pips:
|
||||
→ Pindah SL ke breakeven + 2 poin buffer
|
||||
(tidak bisa rugi lagi)
|
||||
```
|
||||
|
||||
### 6. Trailing Stop
|
||||
|
||||
```
|
||||
Profit >= 25 pips:
|
||||
→ SL mengikuti harga dengan jarak 10 pips
|
||||
(kunci profit sambil biarkan profit berjalan)
|
||||
```
|
||||
|
||||
### 7. Default: HOLD
|
||||
|
||||
```
|
||||
Tidak ada kondisi terpenuhi → HOLD posisi
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## SmartMarketCloseHandler
|
||||
|
||||
### Market Hours (XAUUSD)
|
||||
|
||||
```
|
||||
Minggu 17:00 EST (Senin 05:00 WIB) → Jumat 17:00 EST (Sabtu 05:00 WIB)
|
||||
24 jam, 5 hari seminggu
|
||||
|
||||
Daily close: 05:00 WIB (= 17:00 EST hari sebelumnya)
|
||||
Weekend close: Sabtu 05:00 WIB (= Jumat 17:00 EST)
|
||||
```
|
||||
|
||||
### Konfigurasi
|
||||
|
||||
```python
|
||||
SmartMarketCloseHandler(
|
||||
daily_close_hour_wib=5, # 05:00 WIB
|
||||
hours_before_close=2.0, # "Dekat close" = 2 jam sebelumnya
|
||||
min_profit_to_take=10.0, # Ambil profit >= $10 sebelum close
|
||||
max_loss_to_hold=100.0, # Hold loss sampai $100
|
||||
weekend_loss_cut_percent=50.0, # Cut jika SL >50% hit sebelum weekend
|
||||
)
|
||||
```
|
||||
|
||||
### 4 Rekomendasi
|
||||
|
||||
| Rekomendasi | Kondisi | Aksi |
|
||||
|-------------|---------|------|
|
||||
| **CLOSE_PROFIT** | Profit + dekat close | Tutup, amankan profit |
|
||||
| **CUT_LOSS_WEEKEND** | Loss besar + dekat weekend | Tutup, hindari gap |
|
||||
| **HOLD_LOSS** | Loss kecil + dekat close | Hold, bisa recovery |
|
||||
| **NORMAL** | Belum dekat close | Lanjut normal |
|
||||
|
||||
---
|
||||
|
||||
## Market Analysis (Urgency Score)
|
||||
|
||||
```
|
||||
Score dimulai dari 0, lalu ditambah:
|
||||
|
||||
Regime crisis/high_vol: +3
|
||||
ML opposite >75% confidence: +2
|
||||
RSI >75 (overbought): +2
|
||||
RSI <25 (oversold): +2
|
||||
Trend + momentum berlawanan: +3
|
||||
|
||||
Total max: ~10
|
||||
Score >= 7 = HIGH URGENCY → tutup jika ada profit
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Konfigurasi SmartPositionManager
|
||||
|
||||
```python
|
||||
SmartPositionManager(
|
||||
breakeven_pips=15.0, # Breakeven setelah 15 pips profit
|
||||
trail_start_pips=25.0, # Mulai trailing setelah 25 pips
|
||||
trail_step_pips=10.0, # Trail distance: 10 pips
|
||||
min_profit_to_protect=50.0, # Min $50 untuk proteksi profit
|
||||
max_drawdown_from_peak=30.0, # Max 30% drawdown dari peak
|
||||
enable_market_close_handler=True, # Aktifkan market close handler
|
||||
min_profit_before_close=10.0, # Take profit $10+ sebelum close
|
||||
max_loss_to_hold=100.0, # Hold loss sampai $100
|
||||
)
|
||||
```
|
||||
@@ -0,0 +1,176 @@
|
||||
# Risk Engine — Mesin Risiko & Circuit Breaker
|
||||
|
||||
> **File:** `src/risk_engine.py`
|
||||
> **Class:** `RiskEngine`
|
||||
> **Digunakan oleh:** `main_live.py`
|
||||
|
||||
---
|
||||
|
||||
## Apa Itu Risk Engine?
|
||||
|
||||
Risk Engine adalah **lapisan proteksi fundamental** yang menghitung ukuran posisi, memvalidasi order, dan mengaktifkan circuit breaker saat batas risiko terlampaui.
|
||||
|
||||
**Analogi:** Risk Engine seperti **sistem rem ABS di mobil** — menghitung kecepatan aman, memvalidasi manuver, dan menghentikan paksa jika ada bahaya.
|
||||
|
||||
---
|
||||
|
||||
## 4 Fungsi Utama
|
||||
|
||||
### 1. Position Sizing (Kelly Criterion)
|
||||
|
||||
```
|
||||
Risk-Constrained Half-Kelly:
|
||||
|
||||
1. Hitung Kelly fraction:
|
||||
f* = (p × b - q) / b
|
||||
dimana:
|
||||
p = win rate (misal 0.55)
|
||||
q = 1 - p (0.45)
|
||||
b = avg win/loss ratio (misal 2.0)
|
||||
|
||||
2. Cap Kelly: max 25%
|
||||
|
||||
3. Half-Kelly: f* × 0.5 (safety)
|
||||
|
||||
4. Apply regime multiplier (0.5x - 1.0x)
|
||||
|
||||
5. Cap di config limit: max risk_per_trade%
|
||||
|
||||
6. Hitung lot:
|
||||
risk_amount = balance × actual_risk%
|
||||
lot = risk_amount / (SL_pips × pip_value)
|
||||
|
||||
7. Round ke lot_step, clamp ke min/max
|
||||
```
|
||||
|
||||
**Contoh:**
|
||||
|
||||
```
|
||||
Balance: $5,000
|
||||
Win rate: 55%
|
||||
Win/Loss ratio: 2.0
|
||||
Kelly: (0.55 × 2.0 - 0.45) / 2.0 = 0.325 (32.5%)
|
||||
Half-Kelly: 16.25%
|
||||
Cap: min(16.25%, 1.0%) = 1.0%
|
||||
Risk amount: $50
|
||||
SL distance: 50 pips ($5 per pip per 0.01 lot)
|
||||
Lot: $50 / (50 × $1) = 0.01 lot (menambahkan regime multiplier)
|
||||
```
|
||||
|
||||
### 2. Risk Check (Real-time)
|
||||
|
||||
```python
|
||||
check_risk(balance, equity, open_positions, current_price)
|
||||
|
|
||||
v
|
||||
Hitung daily P/L: equity - starting_balance
|
||||
|
|
||||
v
|
||||
Cek circuit breaker aktif? → can_trade = False
|
||||
|
|
||||
v
|
||||
Daily loss >= max_daily_loss%? → CIRCUIT BREAKER
|
||||
|
|
||||
v
|
||||
Posisi >= max_positions? → can_trade = False
|
||||
|
|
||||
v
|
||||
Return RiskMetrics(daily_pnl, drawdown, can_trade, reason)
|
||||
```
|
||||
|
||||
### 3. Order Validation
|
||||
|
||||
```python
|
||||
validate_order(type, entry, sl, tp, lot, price, balance)
|
||||
|
|
||||
├── Circuit breaker aktif? → REJECT
|
||||
├── BUY: SL >= entry? → REJECT ("SL harus di bawah entry")
|
||||
├── BUY: TP <= entry? → REJECT ("TP harus di atas entry")
|
||||
├── Lot < minimum? → REJECT
|
||||
├── Lot > maximum? → REJECT
|
||||
├── Entry terlalu jauh dari current price (>0.1%)? → REJECT
|
||||
├── Risk% > 1.5× config limit? → REJECT
|
||||
└── Semua OK → APPROVED
|
||||
```
|
||||
|
||||
### 4. Circuit Breaker
|
||||
|
||||
```
|
||||
TRIGGER:
|
||||
Daily loss >= max_daily_loss% (3% untuk $5K account)
|
||||
|
||||
EFEK:
|
||||
→ can_trade = False
|
||||
→ Semua entry baru DITOLAK
|
||||
→ TIDAK menutup posisi yang ada
|
||||
|
||||
RESET:
|
||||
→ Otomatis pada hari baru
|
||||
→ Manual via reset_circuit_breaker()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Daily Stats Tracking
|
||||
|
||||
```python
|
||||
# Auto-initialize setiap hari baru
|
||||
_daily_stats[today] = {
|
||||
"starting_balance": equity, # Basis untuk % hitung
|
||||
"trades": 0, # Total trade hari ini
|
||||
"wins": 0, # Trade profit
|
||||
"losses": 0, # Trade loss
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Return Types
|
||||
|
||||
### RiskMetrics
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class RiskMetrics:
|
||||
daily_pnl: float # P/L hari ini ($)
|
||||
daily_pnl_percent: float # P/L hari ini (%)
|
||||
open_exposure: float # Total exposure ($)
|
||||
max_drawdown: float # Drawdown dari peak (%)
|
||||
position_count: int # Jumlah posisi terbuka
|
||||
can_trade: bool # Boleh buka posisi baru?
|
||||
reason: str # Alasan
|
||||
```
|
||||
|
||||
### PositionSizeResult
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class PositionSizeResult:
|
||||
lot_size: float # Ukuran lot yang dihitung
|
||||
risk_amount: float # Risk dalam USD
|
||||
risk_percent: float # Risk dalam %
|
||||
stop_distance: float # Jarak SL (harga)
|
||||
take_profit_distance: float # Jarak TP (harga)
|
||||
approved: bool # Disetujui?
|
||||
rejection_reason: str # Alasan penolakan
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Hubungan dengan Smart Risk Manager
|
||||
|
||||
```
|
||||
RiskEngine (modul ini)
|
||||
├── Kelly Criterion position sizing
|
||||
├── Circuit breaker (daily loss limit)
|
||||
├── Order validation
|
||||
└── Foundational risk checks
|
||||
|
||||
SmartRiskManager (05-Risk-Management.md)
|
||||
├── 4 trading modes (NORMAL/RECOVERY/PROTECTED/STOPPED)
|
||||
├── Smart exit logic (10 kondisi)
|
||||
├── Position monitoring per-detik
|
||||
└── Higher-level risk decisions
|
||||
```
|
||||
|
||||
**RiskEngine** adalah mesin kalkulasi dasar, **SmartRiskManager** adalah manajer tingkat tinggi yang menggunakannya.
|
||||
@@ -0,0 +1,225 @@
|
||||
# Database Module — PostgreSQL Integration
|
||||
|
||||
> **File:** `src/db/connection.py`, `src/db/repository.py`
|
||||
> **Database:** PostgreSQL
|
||||
> **Library:** psycopg2 (connection pooling)
|
||||
|
||||
---
|
||||
|
||||
## Apa Itu Database Module?
|
||||
|
||||
Database Module menyediakan **penyimpanan persisten** untuk semua data trading — trade history, training log, sinyal, snapshot pasar, dan status bot. Menggunakan PostgreSQL dengan connection pooling untuk performa tinggi.
|
||||
|
||||
**Analogi:** Database Module seperti **arsip perpustakaan** — menyimpan semua catatan trading secara terorganisir, bisa dicari kapan saja, dan tidak hilang meski bot di-restart.
|
||||
|
||||
---
|
||||
|
||||
## Arsitektur
|
||||
|
||||
```
|
||||
Bot Components
|
||||
├── TradeLogger → TradeRepository, SignalRepository, MarketSnapshotRepository
|
||||
├── AutoTrainer → TrainingRepository
|
||||
├── main_live.py → BotStatusRepository, DailySummaryRepository
|
||||
└── Dashboard → Semua repository (READ)
|
||||
|
|
||||
v
|
||||
DatabaseConnection (Singleton)
|
||||
|
|
||||
v
|
||||
ThreadedConnectionPool (1-10 koneksi)
|
||||
|
|
||||
v
|
||||
PostgreSQL Server
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Connection (Singleton + Pooling)
|
||||
|
||||
```python
|
||||
class DatabaseConnection:
|
||||
"""
|
||||
Thread-safe singleton dengan connection pooling.
|
||||
|
||||
- Hanya 1 instance (singleton pattern)
|
||||
- Pool: 1-10 koneksi (ThreadedConnectionPool)
|
||||
- Auto-reconnect jika putus
|
||||
- Context manager support
|
||||
"""
|
||||
```
|
||||
|
||||
### Konfigurasi
|
||||
|
||||
```
|
||||
DB_HOST=localhost
|
||||
DB_PORT=5432
|
||||
DB_NAME=trading_db
|
||||
DB_USER=trading_bot
|
||||
DB_PASSWORD=trading_bot_2026
|
||||
```
|
||||
|
||||
### Penggunaan
|
||||
|
||||
```python
|
||||
from src.db import get_db, init_db
|
||||
|
||||
# Initialize
|
||||
if init_db():
|
||||
db = get_db()
|
||||
|
||||
# Query
|
||||
with db.get_cursor() as cur:
|
||||
cur.execute("SELECT * FROM trades WHERE profit_usd > 0")
|
||||
rows = cur.fetchall()
|
||||
|
||||
# Simple execute
|
||||
result = db.execute("SELECT count(*) FROM trades", fetch=True)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6 Repository
|
||||
|
||||
### 1. TradeRepository
|
||||
|
||||
| Method | Fungsi |
|
||||
|--------|--------|
|
||||
| `insert_trade()` | Insert trade baru (saat open) |
|
||||
| `update_trade_close()` | Update exit data (saat close) |
|
||||
| `get_trade_by_ticket()` | Cari trade per ticket |
|
||||
| `get_open_trades()` | Trade yang belum ditutup |
|
||||
| `get_recent_trades(100)` | 100 trade terakhir |
|
||||
| `get_trades_for_training(30)` | Trade 30 hari untuk ML |
|
||||
| `get_daily_stats(date)` | Statistik per hari |
|
||||
| `get_session_stats("London", 30)` | Statistik per sesi |
|
||||
| `get_smc_pattern_stats(30)` | Performa per pola SMC |
|
||||
|
||||
### 2. TrainingRepository
|
||||
|
||||
| Method | Fungsi |
|
||||
|--------|--------|
|
||||
| `insert_training_run()` | Catat mulai training |
|
||||
| `update_training_complete()` | Update hasil training |
|
||||
| `mark_rollback()` | Tandai model di-rollback |
|
||||
| `get_latest_successful()` | Training sukses terakhir |
|
||||
| `get_training_history(20)` | 20 training terakhir |
|
||||
|
||||
### 3. SignalRepository
|
||||
|
||||
| Method | Fungsi |
|
||||
|--------|--------|
|
||||
| `insert_signal()` | Catat sinyal yang dihasilkan |
|
||||
| `mark_executed()` | Tandai sinyal yang dieksekusi |
|
||||
| `get_recent_signals(100)` | 100 sinyal terakhir |
|
||||
| `get_signal_stats(24)` | Statistik 24 jam |
|
||||
|
||||
### 4. MarketSnapshotRepository
|
||||
|
||||
| Method | Fungsi |
|
||||
|--------|--------|
|
||||
| `insert_snapshot()` | Simpan snapshot pasar |
|
||||
| `get_recent_snapshots(60)` | Snapshot 60 menit terakhir |
|
||||
|
||||
### 5. BotStatusRepository
|
||||
|
||||
| Method | Fungsi |
|
||||
|--------|--------|
|
||||
| `insert_status()` | Catat status bot |
|
||||
| `get_latest_status()` | Status terbaru |
|
||||
|
||||
### 6. DailySummaryRepository
|
||||
|
||||
| Method | Fungsi |
|
||||
|--------|--------|
|
||||
| `upsert_summary()` | Insert/update ringkasan harian |
|
||||
| `get_summary(date)` | Ringkasan per tanggal |
|
||||
| `get_recent_summaries(30)` | 30 hari terakhir |
|
||||
|
||||
---
|
||||
|
||||
## Tabel Database
|
||||
|
||||
### trades
|
||||
|
||||
```sql
|
||||
├── ticket, symbol, direction
|
||||
├── entry_price, exit_price, stop_loss, take_profit
|
||||
├── lot_size, profit_usd, profit_pips
|
||||
├── opened_at, closed_at, duration_seconds
|
||||
├── entry_regime, entry_volatility, entry_session
|
||||
├── smc_signal, smc_confidence, smc_reason
|
||||
├── smc_fvg_detected, smc_ob_detected, smc_bos_detected, smc_choch_detected
|
||||
├── ml_signal, ml_confidence
|
||||
├── market_quality, market_score, dynamic_threshold
|
||||
├── exit_reason, exit_regime, exit_ml_signal
|
||||
├── balance_before, balance_after, equity_at_entry
|
||||
├── features_entry (JSON), features_exit (JSON)
|
||||
└── bot_version, trade_mode
|
||||
```
|
||||
|
||||
### training_runs
|
||||
|
||||
```sql
|
||||
├── training_type, bars_used, num_boost_rounds
|
||||
├── hmm_trained, hmm_n_regimes
|
||||
├── xgb_trained, train_auc, test_auc
|
||||
├── train_accuracy, test_accuracy
|
||||
├── model_path, backup_path
|
||||
├── success, error_message
|
||||
├── started_at, completed_at, duration_seconds
|
||||
└── rolled_back, rollback_reason, rollback_at
|
||||
```
|
||||
|
||||
### signals
|
||||
|
||||
```sql
|
||||
├── signal_time, symbol, price
|
||||
├── signal_type, signal_source, combined_confidence
|
||||
├── smc_*, ml_*
|
||||
├── regime, session, volatility, market_score
|
||||
└── executed, execution_reason, trade_ticket
|
||||
```
|
||||
|
||||
### market_snapshots
|
||||
|
||||
```sql
|
||||
├── snapshot_time, symbol, price, OHLC
|
||||
├── regime, volatility, session, ATR, spread
|
||||
├── ml_signal, ml_confidence, smc_signal, smc_confidence
|
||||
└── open_positions, floating_pnl, features (JSON)
|
||||
```
|
||||
|
||||
### bot_status
|
||||
|
||||
```sql
|
||||
├── status_time, is_running, status
|
||||
├── loop_count, avg_execution_ms, uptime_seconds
|
||||
├── balance, equity, margin_used
|
||||
├── open_positions, floating_pnl, daily_pnl
|
||||
└── risk_mode, current_session, is_golden_time
|
||||
```
|
||||
|
||||
### daily_summaries
|
||||
|
||||
```sql
|
||||
├── summary_date
|
||||
├── total/winning/losing/breakeven_trades
|
||||
├── gross_profit, gross_loss, net_profit
|
||||
├── start_balance, end_balance
|
||||
├── win_rate, profit_factor, avg win/loss
|
||||
├── trades per session (sydney/tokyo/london/ny/golden)
|
||||
└── SMC pattern stats (fvg/ob trades & wins)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Graceful Degradation
|
||||
|
||||
```
|
||||
PostgreSQL tersedia?
|
||||
├── Ya → Gunakan DB + CSV backup
|
||||
└── Tidak → CSV saja (semua tetap berjalan)
|
||||
|
||||
Bot TIDAK pernah crash karena database.
|
||||
```
|
||||
@@ -0,0 +1,164 @@
|
||||
# Train Models — Script Training Awal
|
||||
|
||||
> **File:** `train_models.py`
|
||||
> **Tipe:** Script CLI (bukan modul)
|
||||
> **Output:** `models/xgboost_model.pkl`, `models/hmm_regime.pkl`
|
||||
|
||||
---
|
||||
|
||||
## Apa Itu Train Models?
|
||||
|
||||
Train Models adalah script **pelatihan awal** yang dijalankan sekali sebelum bot mulai trading. Mengambil data historis dari MT5, melatih HMM dan XGBoost, lalu menyimpan model ke file `.pkl`.
|
||||
|
||||
**Analogi:** Train Models seperti **sekolah penerbangan** — melatih pilot (model AI) sebelum terbang pertama kali. Setelah itu, pelatihan rutin dilakukan oleh Auto Trainer (13).
|
||||
|
||||
---
|
||||
|
||||
## Cara Penggunaan
|
||||
|
||||
```bash
|
||||
python train_models.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Pipeline Training
|
||||
|
||||
```
|
||||
1. LOAD CONFIG
|
||||
├── get_config() dari .env
|
||||
└── Symbol, capital, mode
|
||||
|
||||
2. CONNECT MT5
|
||||
├── Login, password, server
|
||||
└── Verifikasi: balance, equity
|
||||
|
||||
3. FETCH DATA
|
||||
├── 10.000 bar XAUUSD M15
|
||||
└── ~104 hari data historis
|
||||
|
||||
4. FEATURE ENGINEERING
|
||||
├── FeatureEngineer.calculate_all() → 40+ fitur teknikal
|
||||
├── SMCAnalyzer.calculate_all() → Struktur pasar
|
||||
└── create_target(lookahead=1) → Label UP/DOWN
|
||||
|
||||
5. SAVE DATA
|
||||
└── data/training_data.parquet
|
||||
|
||||
6. TRAIN HMM
|
||||
├── MarketRegimeDetector(n_regimes=3, lookback=500)
|
||||
├── fit(df)
|
||||
├── Log: distribusi regime, transition matrix
|
||||
└── Save → models/hmm_regime.pkl
|
||||
|
||||
7. TRAIN XGBOOST
|
||||
├── TradingModel(confidence_threshold=0.60)
|
||||
├── fit(train_ratio=0.7, boost_rounds=50, early_stop=5)
|
||||
├── Log: top 10 feature importance
|
||||
├── Walk-forward validation (train=500, test=50, step=50)
|
||||
├── Log: avg train/test AUC, overfitting ratio
|
||||
└── Save → models/xgboost_model.pkl
|
||||
|
||||
8. DISCONNECT
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Parameter Training
|
||||
|
||||
| Parameter | Nilai | Keterangan |
|
||||
|-----------|-------|------------|
|
||||
| Data | 10.000 bar M15 | ~104 hari |
|
||||
| Train/Test Split | 70% / 30% | Lebih banyak test data |
|
||||
| XGBoost Rounds | 50 | Anti-overfitting |
|
||||
| Early Stopping | 5 rounds | Stop lebih awal |
|
||||
| HMM Regimes | 3 | Low/Medium/High volatility |
|
||||
| HMM Lookback | 500 bar | Window training |
|
||||
| Walk-forward Window | 500 train / 50 test | Validasi robustness |
|
||||
|
||||
---
|
||||
|
||||
## Output
|
||||
|
||||
```
|
||||
models/
|
||||
├── xgboost_model.pkl # Model XGBoost (binary classifier)
|
||||
└── hmm_regime.pkl # Model HMM (regime detector)
|
||||
|
||||
data/
|
||||
└── training_data.parquet # Data training (untuk referensi)
|
||||
|
||||
logs/
|
||||
└── training_YYYY-MM-DD.log # Log training detail
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Contoh Output Log
|
||||
|
||||
```
|
||||
[08:00] ============================================================
|
||||
[08:00] SMART TRADING BOT - MODEL TRAINING
|
||||
[08:00] ============================================================
|
||||
[08:00] Symbol: XAUUSD
|
||||
[08:00] Capital: $5,000.00
|
||||
[08:00] Mode: small
|
||||
[08:00] Connecting to MT5...
|
||||
[08:00] MT5 connected successfully!
|
||||
[08:00] Account Balance: $5,094.68
|
||||
[08:00] Fetching 10000 bars of XAUUSD M15 data...
|
||||
[08:01] Received 10000 bars
|
||||
[08:01] Date range: 2024-10-25 to 2025-02-06
|
||||
[08:01] Applying feature engineering...
|
||||
[08:01] Total features created: 52
|
||||
[08:01] ============================================================
|
||||
[08:01] Training HMM Regime Model
|
||||
[08:01] ============================================================
|
||||
[08:01] Regime Distribution:
|
||||
[08:01] low_volatility: 3200 bars
|
||||
[08:01] medium_volatility: 4500 bars
|
||||
[08:01] high_volatility: 2300 bars
|
||||
[08:02] ============================================================
|
||||
[08:02] Training XGBoost Model (Anti-Overfit Config)
|
||||
[08:02] ============================================================
|
||||
[08:02] Available features: 37/40
|
||||
[08:02] Top 10 Feature Importance:
|
||||
[08:02] rsi: 0.0842
|
||||
[08:02] macd_histogram: 0.0756
|
||||
[08:02] atr: 0.0689
|
||||
[08:02] ...
|
||||
[08:03] Walk-forward Results:
|
||||
[08:03] Avg Train AUC: 0.7234
|
||||
[08:03] Avg Test AUC: 0.6891
|
||||
[08:03] Overfitting ratio: 1.05
|
||||
[08:03] ============================================================
|
||||
[08:03] TRAINING COMPLETE
|
||||
[08:03] ============================================================
|
||||
[08:03] HMM Model: SAVED
|
||||
[08:03] XGBoost Model: SAVED
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Kapan Dijalankan?
|
||||
|
||||
| Situasi | Script |
|
||||
|---------|--------|
|
||||
| **Pertama kali setup** | `train_models.py` (wajib) |
|
||||
| **Setelah update kode** | `train_models.py` (opsional) |
|
||||
| **Rutin harian** | Auto Trainer (otomatis) |
|
||||
| **Model buruk** | `train_models.py` (manual retrain) |
|
||||
|
||||
---
|
||||
|
||||
## Perbedaan dengan Auto Trainer
|
||||
|
||||
| Aspek | train_models.py | Auto Trainer |
|
||||
|-------|-----------------|-------------|
|
||||
| **Kapan** | Manual, 1x | Otomatis, harian |
|
||||
| **Data** | 10K bar | 8K (daily) / 15K (weekend) |
|
||||
| **Backup** | Tidak | Ya (5 terakhir) |
|
||||
| **Rollback** | Tidak | Ya (AUC < 0.52) |
|
||||
| **Database** | Tidak | Ya (PostgreSQL) |
|
||||
| **Walk-forward** | Ya | Tidak |
|
||||
| **Tujuan** | Setup awal | Maintenance rutin |
|
||||
@@ -0,0 +1,239 @@
|
||||
# Main Live — Orchestrator Utama
|
||||
|
||||
> **File:** `main_live.py`
|
||||
> **Class:** `TradingBot`
|
||||
> **Runtime:** Async event loop (asyncio)
|
||||
> **Target:** < 0.05 detik per loop
|
||||
|
||||
---
|
||||
|
||||
## Apa Itu Main Live?
|
||||
|
||||
Main Live adalah **otak pusat** yang mengorkestrasi semua komponen bot. Menjalankan loop utama setiap ~1 detik, mengkoordinasikan 15+ komponen dari data fetching hingga order execution.
|
||||
|
||||
**Analogi:** Main Live seperti **konduktor orkestra** — tidak memainkan alat musik sendiri, tapi mengarahkan semua pemain (komponen) agar bermain harmonis pada waktu yang tepat.
|
||||
|
||||
---
|
||||
|
||||
## Komponen yang Dimuat
|
||||
|
||||
```python
|
||||
class TradingBot:
|
||||
def __init__(self):
|
||||
# Koneksi
|
||||
self.mt5 = MT5Connector(...) # Jembatan ke broker
|
||||
self.telegram = TelegramNotifier(...) # Notifikasi
|
||||
|
||||
# AI Models
|
||||
self.ml_model = TradingModel(...) # XGBoost predictor
|
||||
self.regime_detector = MarketRegimeDetector(...) # HMM regime
|
||||
self.smc = SMCAnalyzer(...) # Smart Money Concepts
|
||||
|
||||
# Analisis
|
||||
self.features = FeatureEngineer() # 40+ fitur
|
||||
self.dynamic_confidence = DynamicConfidenceManager(...) # Threshold
|
||||
self.session_filter = SessionFilter(...) # Waktu trading
|
||||
self.news_agent = NewsAgent(...) # Monitor berita
|
||||
|
||||
# Risiko
|
||||
self.smart_risk = SmartRiskManager(...) # Risk management
|
||||
self.risk_engine = RiskEngine(...) # Kelly criterion
|
||||
self.position_manager = SmartPositionManager(...) # Position mgmt
|
||||
|
||||
# Logging & Training
|
||||
self.trade_logger = TradeLogger(...) # Pencatat trade
|
||||
self.auto_trainer = AutoTrainer(...) # Retraining otomatis
|
||||
|
||||
# State
|
||||
self.flash_crash_detector = FlashCrashDetector(...) # Proteksi
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Main Loop (Setiap ~1 Detik)
|
||||
|
||||
```
|
||||
STARTUP:
|
||||
Load models → Connect MT5 → Send Telegram startup
|
||||
|
|
||||
v
|
||||
LOOP UTAMA (setiap ~1 detik):
|
||||
|
|
||||
|===[PHASE 1: DATA]=================================
|
||||
|
|
||||
├── Fetch 200 bar M15 XAUUSD dari MT5
|
||||
├── Feature Engineering (40+ fitur)
|
||||
├── SMC Analysis (Swing, FVG, OB, BOS, CHoCH)
|
||||
├── HMM Regime Detection
|
||||
└── XGBoost Prediction
|
||||
|
|
||||
|===[PHASE 2: MONITORING]===========================
|
||||
|
|
||||
├── Cek posisi terbuka (setiap 1 detik)
|
||||
│ └── Untuk setiap posisi:
|
||||
│ ├── Update profit & momentum
|
||||
│ ├── 10 kondisi exit (smart_risk.evaluate_position)
|
||||
│ └── Jika should_close → tutup → log → Telegram
|
||||
|
|
||||
├── Position Manager (trailing SL, breakeven)
|
||||
│ └── Smart Market Close Handler
|
||||
|
|
||||
|===[PHASE 3: ENTRY]=================================
|
||||
|
|
||||
├── [1] Session Filter → boleh trading?
|
||||
├── [2] Risk Mode → bukan STOPPED?
|
||||
├── [3] SMC Signal → ada setup?
|
||||
├── [4] ML Confidence → >= threshold?
|
||||
├── [5] ML Agreement → tidak strongly disagree?
|
||||
├── [6] Dynamic Quality → bukan AVOID?
|
||||
├── [7] Confirmation → 2x berturut?
|
||||
├── [8] Pullback Filter → momentum selaras?
|
||||
├── [9] Cooldown → 5 menit sejak trade terakhir?
|
||||
├── [10] Position Limit → < 2 posisi?
|
||||
├── [11] Lot Size → > 0?
|
||||
└── SEMUA PASS → Execute trade → Log → Telegram
|
||||
|
|
||||
|===[PHASE 4: PERIODIK]==============================
|
||||
|
|
||||
├── Setiap 5 menit: Cek auto-retrain
|
||||
├── Setiap 30 menit: Market update (Telegram)
|
||||
├── Setiap 1 jam: Hourly analysis (Telegram)
|
||||
├── Pergantian hari: Daily summary + reset
|
||||
└── News Agent: Monitor (non-blocking)
|
||||
|
|
||||
v
|
||||
Tunggu ~1 detik → Loop lagi
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Startup Sequence
|
||||
|
||||
```
|
||||
1. Load konfigurasi dari .env
|
||||
2. Connect ke MT5 (max 3 retry)
|
||||
3. Load model HMM dari models/hmm_regime.pkl
|
||||
4. Load model XGBoost dari models/xgboost_model.pkl
|
||||
5. Initialize SmartRiskManager (set balance, limits)
|
||||
6. Initialize SessionFilter (WIB timezone)
|
||||
7. Initialize TelegramNotifier
|
||||
8. Initialize TradeLogger
|
||||
9. Initialize AutoTrainer
|
||||
10. Send Telegram: "BOT STARTED" (config, balance, risk settings)
|
||||
11. Mulai main loop
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Shutdown Sequence
|
||||
|
||||
```
|
||||
1. Signal SIGINT/SIGTERM diterima
|
||||
2. Hentikan loop utama
|
||||
3. Kirim Telegram: "BOT STOPPED" (balance, trades, uptime)
|
||||
4. Disconnect MT5
|
||||
5. Close database connections
|
||||
6. Exit
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
```
|
||||
Setiap iterasi loop dibungkus try-except:
|
||||
|
||||
try:
|
||||
# Fetch data, analyze, trade
|
||||
except ConnectionError:
|
||||
# MT5 disconnected → reconnect()
|
||||
except Exception as e:
|
||||
# Log error → lanjut loop berikutnya
|
||||
# Bot TIDAK crash dari error tunggal
|
||||
|
||||
Prinsip: NEVER STOP TRADING karena error non-kritis
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Timer Periodik
|
||||
|
||||
| Event | Interval | Aksi |
|
||||
|-------|----------|------|
|
||||
| Data fetch + analysis | ~1 detik | Setiap loop |
|
||||
| Position monitoring | ~1 detik | Setiap loop |
|
||||
| News monitoring log | 5 menit | `loop_count % 300` |
|
||||
| Auto-retrain check | 5 menit | `loop_count % 300` |
|
||||
| Market update Telegram | 30 menit | Timer |
|
||||
| Hourly analysis Telegram | 1 jam | Timer |
|
||||
| Daily summary | Pergantian hari | Date check |
|
||||
|
||||
---
|
||||
|
||||
## Performa Target
|
||||
|
||||
```
|
||||
Target: < 0.05 detik per loop (50ms)
|
||||
|
||||
Breakdown:
|
||||
├── MT5 data fetch: ~10ms
|
||||
├── Feature engineering: ~5ms (Polars, vectorized)
|
||||
├── SMC analysis: ~5ms (Polars native)
|
||||
├── HMM predict: ~2ms
|
||||
├── XGBoost predict: ~3ms
|
||||
├── Position monitoring: ~5ms
|
||||
├── Entry logic: ~5ms
|
||||
└── Overhead: ~15ms
|
||||
------
|
||||
~50ms total
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Hubungan Semua Komponen
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ main_live.py │
|
||||
│ (TradingBot) │
|
||||
│ │
|
||||
│ ┌─────────┐ ┌─────────┐ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ MT5 │ │ Feature │ │ SMC │ │ HMM │ │
|
||||
│ │Connector│→ │ Eng │→ │ Analyzer │→ │ Detector │ │
|
||||
│ └─────────┘ └─────────┘ └──────────┘ └──────────┘ │
|
||||
│ ↑ ↓ ↓ │
|
||||
│ │ ┌──────────────────────┐ │
|
||||
│ │ │ Dynamic Confidence │ │
|
||||
│ │ └──────────────────────┘ │
|
||||
│ │ ↓ │
|
||||
│ │ ┌──────────────────┐ │
|
||||
│ │ │ XGBoost Model │ │
|
||||
│ │ └──────────────────┘ │
|
||||
│ │ ↓ │
|
||||
│ │ ┌─────────────────────────────────┐ │
|
||||
│ │ │ Entry Logic (11 Filters) │ │
|
||||
│ │ │ Session, Risk, SMC, ML, ... │ │
|
||||
│ │ └─────────────────────────────────┘ │
|
||||
│ │ ↓ │
|
||||
│ │ ┌────────────┐ ┌───────────────┐ │
|
||||
│ ├────│ Risk Engine│ │Smart Risk Mgr │ │
|
||||
│ │ └────────────┘ └───────────────┘ │
|
||||
│ │ ↓ │
|
||||
│ │←── Execute Order (BUY/SELL) │
|
||||
│ │ ↓ │
|
||||
│ │ ┌────────────┐ ┌───────────────┐ │
|
||||
│ │ │ Position │ │ Trade Logger │ │
|
||||
│ │ │ Manager │ │ (DB + CSV) │ │
|
||||
│ │ └────────────┘ └───────────────┘ │
|
||||
│ │ ↓ │
|
||||
│ │ ┌────────────┐ ┌───────────────┐ │
|
||||
│ │ │ Telegram │ │ Auto Trainer │ │
|
||||
│ │ │ Notifier │ │ (retraining) │ │
|
||||
│ │ └────────────┘ └───────────────┘ │
|
||||
│ │ │
|
||||
│ │ ┌────────────┐ ┌───────────────┐ │
|
||||
│ │ │News Agent │ │Session Filter │ │
|
||||
│ │ │(monitor) │ │(waktu trading)│ │
|
||||
│ │ └────────────┘ └───────────────┘ │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
@@ -31,12 +31,23 @@
|
||||
| 9 | [Entry Trade](09-Entry-Trade.md) | `main_live.py` | Proses masuk posisi (11 filter) |
|
||||
| 10 | [Exit Trade](10-Exit-Trade.md) | `main_live.py` | Proses keluar posisi (10 kondisi) |
|
||||
|
||||
### Koneksi & Konfigurasi
|
||||
|
||||
| # | Komponen | File Source | Fungsi |
|
||||
|---|----------|------------|--------|
|
||||
| 16 | [MT5 Connector](16-MT5-Connector.md) | `src/mt5_connector.py` | Jembatan ke broker MT5 (auto-reconnect) |
|
||||
| 17 | [Configuration](17-Configuration.md) | `src/config.py` | Konfigurasi terpusat (6 sub-config) |
|
||||
|
||||
### Pendukung
|
||||
|
||||
| # | Komponen | File Source | Fungsi |
|
||||
|---|----------|------------|--------|
|
||||
| 11 | [News Agent](11-News-Agent.md) | `src/news_agent.py` | Monitoring berita ekonomi |
|
||||
| 12 | [Telegram Notifications](12-Telegram-Notifications.md) | `src/telegram_notifier.py` | Notifikasi real-time ke Telegram |
|
||||
| 18 | [Trade Logger](18-Trade-Logger.md) | `src/trade_logger.py` | Pencatatan trade dual-storage (DB + CSV) |
|
||||
| 19 | [Position Manager](19-Position-Manager.md) | `src/position_manager.py` | Manajemen posisi aktif (trailing, breakeven) |
|
||||
| 20 | [Risk Engine](20-Risk-Engine.md) | `src/risk_engine.py` | Mesin risiko & circuit breaker (Kelly Criterion) |
|
||||
| 21 | [Database](21-Database.md) | `src/db/` | PostgreSQL integration (6 repository) |
|
||||
|
||||
### Training & Validasi
|
||||
|
||||
@@ -45,6 +56,13 @@
|
||||
| 13 | [Auto Trainer](13-Auto-Trainer.md) | `src/auto_trainer.py` | Retraining model otomatis (pelatih malam) |
|
||||
| 14 | [Backtest](14-Backtest.md) | `backtests/backtest_live_sync.py` | Simulasi trading 100% sync dengan live |
|
||||
| 15 | [Dynamic Confidence](15-Dynamic-Confidence.md) | `src/dynamic_confidence.py` | Penyesuaian threshold otomatis (termometer) |
|
||||
| 22 | [Train Models](22-Train-Models.md) | `train_models.py` | Script training awal (HMM + XGBoost) |
|
||||
|
||||
### Orchestrator
|
||||
|
||||
| # | Komponen | File Source | Fungsi |
|
||||
|---|----------|------------|--------|
|
||||
| 23 | [Main Live Orchestrator](23-Main-Live-Orchestrator.md) | `main_live.py` | Otak pusat bot, koordinasi semua komponen |
|
||||
|
||||
---
|
||||
|
||||
@@ -122,3 +140,11 @@ Regime Signal
|
||||
| Auto Trainer | "Apakah model AI masih akurat? Perlu dilatih ulang?" |
|
||||
| Backtest | "Apakah strategi ini profitable di data historis?" |
|
||||
| Dynamic Confidence | "Seberapa selektif bot harus trading saat ini?" |
|
||||
| MT5 Connector | "Bagaimana bot terhubung ke broker dan mengirim order?" |
|
||||
| Configuration | "Bagaimana semua parameter dikonfigurasi?" |
|
||||
| Trade Logger | "Dimana semua data trade disimpan?" |
|
||||
| Position Manager | "Bagaimana posisi terbuka dikelola secara aktif?" |
|
||||
| Risk Engine | "Berapa ukuran lot yang aman? Sudah lewat batas harian?" |
|
||||
| Database | "Bagaimana data persisten disimpan dan di-query?" |
|
||||
| Train Models | "Bagaimana model AI dilatih pertama kali?" |
|
||||
| Main Live | "Siapa yang mengorkestrasi semua komponen?" |
|
||||
|
||||
Reference in New Issue
Block a user