gavindiaz/XauBot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add 5 dashboard features — dark mode, trade history, backtests, model insights, alerts

Browse Source 
- Dark mode: class-based theme toggle with localStorage persistence and flash prevention
- Trade History (/trades): paginated table, stats cards, equity curve chart with DB API endpoints
- Backtest Viewer (/backtests): log parser for 35 backtest results, sidebar + detail + comparison tabs
- Model Insights: dashboard card + dialog showing feature importance, regime distribution, training history
- Alert/Signal Log (/alerts): signal stats, filterable table with execution tracking
- API: 8 new endpoints with psycopg2 DB connection pool
- Dark mode sweep across books page, about dialog, and all dashboard components
- Architecture docs rewritten with Mermaid diagrams (23 docs)
- README and FEATURES.md rewritten bilingual (Indonesian + English)
- main_live.py: write model_metrics.json on startup and retrain

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
GifariKemal
2026-02-09 05:46:54 +07:00
parent b2dc2dacd7
commit e8355b3f62
230 changed files with 69573 additions and 5673 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/FEATURES.md
+234 -259
View File
@@ -1,346 +1,321 @@
# XAUBot AI — Feature Reference
# XAUBot AI — Referensi Fitur
## Overview
## Gambaran Umum
XAUBot AI is an automated XAUUSD (Gold) trading bot that combines **XGBoost Machine Learning**, **Smart Money Concepts (SMC)**, and **Hidden Markov Model (HMM)** regime detection. It operates on MetaTrader 5 via an asynchronous Python loop, executing trades on the M15 (15-minute) timeframe.
XAUBot AI adalah bot *trading* XAUUSD (Emas) otomatis yang menggabungkan **XGBoost *Machine Learning***, **Smart Money Concepts (SMC)**, dan **Hidden Markov Model (HMM)** untuk deteksi *regime*. Bot ini beroperasi di MetaTrader 5 melalui *loop* Python asinkron, mengeksekusi *trade* pada *timeframe* M15 (15 menit).
The bot follows a strict pipeline: data is fetched, features are engineered, market structure is analyzed, regime is classified, ML predictions are generated, and a series of 14 sequential filters determine whether a trade is executed. Once in a position, 12 exit conditions are monitored every 5-10 seconds.
Bot mengikuti *pipeline* yang ketat: data diambil, fitur direkayasa, struktur pasar dianalisis, *regime* diklasifikasikan, prediksi ML dihasilkan, dan serangkaian 14 *filter* berurutan menentukan apakah *trade* dieksekusi. Setelah posisi terbuka, 12 kondisi *exit* dipantau setiap 5-10 detik.
---
## Entry Filter Pipeline
## *Pipeline* 14 *Entry Filter*
There are **14 filters** that run in order during `_trading_iteration()`. A signal must pass **ALL** of them to execute a trade.
Terdapat **14 *filter*** yang berjalan secara berurutan selama `_trading_iteration()`. Sebuah sinyal harus melewati **SEMUA** *filter* untuk mengeksekusi *trade*.
### 1. Data Fetch
- Pulls **200 M15 bars** from MetaTrader 5.
- Data is converted to a **Polars DataFrame** (not Pandas).
### 1. Pengambilan Data
- Mengambil **200 *bar* M15** dari MetaTrader 5.
- Data dikonversi ke **Polars *DataFrame*** (bukan Pandas).
### 2. Feature Engineering
- Calculates **37 technical features** from the OHLCV data.
- Includes: RSI, ATR, MACD, Bollinger Bands, EMA (multiple periods), Stochastic, volume-based indicators, and more.
- All computations use Polars for performance.
### 2. Rekayasa Fitur (*Feature Engineering*)
- Menghitung **37 fitur teknikal** dari data OHLCV.
- Meliputi: *RSI*, *ATR*, *MACD*, *Bollinger Bands*, *EMA* (berbagai periode), *Stochastic*, indikator berbasis volume, dan lainnya.
- Semua komputasi menggunakan Polars untuk performa.
### 3. SMC Analysis
- Detects institutional **Smart Money Concepts** structures:
- **Order Blocks (OB)** — supply/demand zones from institutional activity.
- **Fair Value Gaps (FVG)** — imbalances in price action.
- **Break of Structure (BOS)** — continuation signals.
- **Change of Character (CHoCH)** — reversal signals.
### 3. Analisis *SMC*
- Mendeteksi struktur institusional *Smart Money Concepts*:
- ***Order Block* (OB)** — zona *supply/demand* dari aktivitas institusional.
- ***Fair Value Gap* (FVG)** — ketidakseimbangan dalam *price action*.
- ***Break of Structure* (BOS)** — sinyal kelanjutan tren.
- ***Change of Character* (CHoCH)** — sinyal pembalikan arah.
### 4. Regime Detection
- **HMM (Hidden Markov Model)** classifies the current market state:
- `TRENDING` — directional movement, favorable for entries.
- `RANGING` — sideways consolidation, reduced sizing.
- `HIGH_VOLATILITY` — erratic movement, caution required.
- `CRISIS` — extreme conditions, trading blocked.
### 4. Deteksi *Regime*
- ***HMM* (*Hidden Markov Model*)** mengklasifikasikan kondisi pasar saat ini:
- `TRENDING` — pergerakan searah, kondusif untuk *entry*.
- `RANGING` — konsolidasi menyamping, ukuran posisi dikurangi.
- `HIGH_VOLATILITY` — pergerakan tidak menentu, butuh kehati-hatian.
- `CRISIS` — kondisi ekstrem, *trading* diblokir.
### 5. Flash Crash Guard
- Emergency protection: if price move exceeds a threshold percentage, **all positions are immediately closed**.
- Prevents catastrophic loss during sudden market dislocations.
### 5. Pelindung *Flash Crash*
- Proteksi darurat: jika pergerakan harga melebihi ambang persentase tertentu, **semua posisi langsung ditutup**.
- Mencegah kerugian katastropik saat dislokasi pasar mendadak.
### 6. Regime Filter
- Blocks trading entirely if the regime recommendation is `SLEEP`.
- Prevents entries during unfavorable market conditions identified by the HMM.
### 6. *Filter Regime*
- Memblokir *trading* sepenuhnya jika rekomendasi *regime* adalah `SLEEP`.
- Mencegah *entry* saat kondisi pasar tidak menguntungkan yang teridentifikasi oleh *HMM*.
### 7. Risk Check
- Blocks trading if:
- **Daily loss limit** has been reached (5% of capital).
- **Equity** is too low relative to required margin.
- **Total loss limit** has been breached (10% of capital).
### 7. Pemeriksaan Risiko
- Memblokir *trading* jika:
- **Batas kerugian harian** telah tercapai (5% dari kapital).
- ***Equity*** terlalu rendah relatif terhadap *margin* yang dibutuhkan.
- **Batas kerugian total** telah dilanggar (10% dari kapital).
### 8. Session Filter
- Filters based on **WIB (Western Indonesian Time)** trading sessions.
- Each session applies a **lot size multiplier** to control exposure:
- **Sydney** (06:00-13:00 WIB) — 0.5x multiplier (low volatility).
- **Tokyo** (07:00-16:00 WIB) — 0.7x multiplier (medium volatility).
- **London** (15:00-24:00 WIB) — 1.0x multiplier (high volatility).
- **New York** (20:00-24:00 WIB) — 1.0x multiplier (extreme volatility).
- **Off-Hours** (00:00-06:00 WIB) — **blocked entirely**.
### 8. *Filter* Sesi
- Memfilter berdasarkan sesi *trading* **WIB (Waktu Indonesia Barat)**.
- Setiap sesi menerapkan ***lot size multiplier*** untuk mengontrol eksposur:
- **Sydney** (06:00-13:00 WIB) — *multiplier* 0.5x (volatilitas rendah).
- **Tokyo** (07:00-16:00 WIB) — *multiplier* 0.7x (volatilitas sedang).
- **London** (15:00-24:00 WIB) — *multiplier* 1.0x (volatilitas tinggi).
- **New York** (20:00-24:00 WIB) — *multiplier* 1.0x (volatilitas ekstrem).
- ***Off-Hours*** (00:00-06:00 WIB) — **diblokir sepenuhnya**.
### 9. H1 Bias Filter (#31B)
- Multi-timeframe confirmation using **EMA20 on the H1 chart**.
- Price position relative to H1 EMA20 determines directional bias:
- **BULLISH** (price above EMA20) — only BUY signals allowed.
- **BEARISH** (price below EMA20) — only SELL signals allowed.
- **NEUTRAL** (price near EMA20) — **all signals blocked**.
- Backtest result: **+$343 improvement, 81.8% win rate, Sharpe 3.97**.
### 9. *Filter Bias* H1 (#31B)
- Konfirmasi *multi-timeframe* menggunakan ***EMA20* pada *chart* H1**.
- Posisi harga relatif terhadap *EMA20* H1 menentukan bias arah:
- **BULLISH** (harga di atas *EMA20*) — hanya sinyal *BUY* yang diizinkan.
- **BEARISH** (harga di bawah *EMA20*) — hanya sinyal *SELL* yang diizinkan.
- **NEUTRAL** (harga dekat *EMA20*) — **semua sinyal diblokir**.
- Hasil *backtest*: **+$343 peningkatan, *win rate* 81.8%, *Sharpe* 3.97**.
### 10. SMC Signal Generation
- Generates a **BUY or SELL signal** based on SMC structure analysis.
- Each signal includes a **confidence score** derived from the quality of the detected structures (OB proximity, FVG alignment, BOS/CHoCH context).
### 10. Generasi Sinyal *SMC*
- Menghasilkan sinyal ***BUY* atau *SELL*** berdasarkan analisis struktur *SMC*.
- Setiap sinyal memiliki ***confidence score*** yang berasal dari kualitas struktur yang terdeteksi (kedekatan *OB*, keselarasan *FVG*, konteks *BOS*/*CHoCH*).
### 11. Signal Combination
- Combines **SMC signal + ML (XGBoost) prediction**.
- Applies a **dynamic confidence threshold** that adapts based on:
- Current trading session.
- Market regime.
- Recent volatility.
- Both signals must agree on direction; combined confidence must exceed the threshold.
### 11. Kombinasi Sinyal
- Menggabungkan **sinyal *SMC* + prediksi *ML* (*XGBoost*)**.
- Menerapkan ***dynamic confidence threshold*** yang beradaptasi berdasarkan:
- Sesi *trading* saat ini.
- *Regime* pasar.
- Volatilitas terkini.
- Kedua sinyal harus sepakat arah; *confidence* gabungan harus melampaui *threshold*.
### 12. Time Filter (#34A)
- Skips specific WIB hours known for poor conditions:
- **Hour 9 WIB** — end of New York session, low liquidity.
- **Hour 21 WIB** — London-New York transition, prone to whipsaw.
- Backtest result: **+$356 improvement**.
### 12. *Filter* Waktu (#34A)
- Melewatkan jam WIB tertentu yang dikenal berkondisi buruk:
- **Jam 9 WIB** — akhir sesi *New York*, likuiditas rendah.
- **Jam 21 WIB** — transisi *London*-*New York*, rawan *whipsaw*.
- Hasil *backtest*: **+$356 peningkatan**.
### 13. Trade Cooldown
- Enforces a minimum **150 seconds (2.5 minutes)** between consecutive trades.
- Prevents overtrading and rapid-fire entries from noisy signals.
### 13. *Cooldown Trade*
- Memberlakukan jeda minimum **150 detik (2.5 menit)** antara *trade* berturut-turut.
- Mencegah *overtrading* dan *entry* bertubi-tubi dari sinyal yang noisy.
### 14. Smart Risk Gate
- Final gate before execution. Checks:
- **Trading mode**: `NORMAL`, `RECOVERY`, `PROTECTED`, or `STOPPED`.
- **Lot size calculation**: Based on ATR, capital mode, and session multiplier.
- **Position limit**: Maximum **2 concurrent positions** allowed.
- If mode is `STOPPED`, no trade is executed regardless of signal quality.
### 14. Gerbang Risiko Cerdas (*Smart Risk Gate*)
- Gerbang terakhir sebelum eksekusi. Memeriksa:
- **Mode *trading***: `NORMAL`, `RECOVERY`, `PROTECTED`, atau `STOPPED`.
- **Perhitungan *lot size***: Berdasarkan *ATR*, mode kapital, dan *multiplier* sesi.
- **Batas posisi**: Maksimal **2 posisi bersamaan** diizinkan.
- Jika mode `STOPPED`, tidak ada *trade* yang dieksekusi terlepas dari kualitas sinyal.
---
## Exit Conditions
## 12 Kondisi *Exit*
**12 exit conditions** are checked every **5-10 seconds** while a position is open.
**12 kondisi *exit*** diperiksa setiap **5-10 detik** selama posisi terbuka.
### 1. Take Profit (Broker-Level TP)
- TP is set at the broker level at entry time.
- Calculated using ATR-based risk-reward ratios.
### 1. *Take Profit* (TP Level Broker)
- *TP* dipasang di level broker saat *entry*.
- Dihitung menggunakan rasio *risk-reward* berbasis *ATR*.
### 2. Trailing Stop (#24B)
- **ATR-adaptive trailing stop**:
- Activation distance: **ATR x 4.0**.
- Step size: **ATR x 3.0**.
- Locks in profits as price moves favorably.
### 2. *Trailing Stop* (#24B)
- ***Trailing stop* adaptif berbasis *ATR***:
- Jarak aktivasi: ***ATR* x 4.0**.
- Ukuran langkah: ***ATR* x 3.0**.
- Mengunci keuntungan seiring harga bergerak menguntungkan.
### 3. Breakeven Move (#24B)
- Moves stop loss to **entry price** (breakeven) when unrealized profit exceeds **ATR x 2.0**.
- Eliminates risk on the trade after a favorable move.
### 3. Perpindahan *Breakeven* (#24B)
- Memindahkan *stop loss* ke **harga *entry*** (*breakeven*) saat keuntungan belum direalisasi melampaui ***ATR* x 2.0**.
- Menghilangkan risiko pada *trade* setelah pergerakan menguntungkan.
### 4. ML Reversal Exit
- Closes the position if the ML model's confidence **flips direction** with confidence exceeding **75%**.
- Responds to changing market conditions detected by XGBoost.
### 4. *Exit* Pembalikan *ML*
- Menutup posisi jika *confidence* model *ML* **berbalik arah** dengan *confidence* melebihi **75%**.
- Merespons perubahan kondisi pasar yang terdeteksi oleh *XGBoost*.
### 5. Max Loss Per Trade
- **Software-level stop loss** at **1% of capital**.
- Acts as a safety net in addition to broker SL.
### 5. Kerugian Maksimal Per *Trade*
- ***Stop loss* level perangkat lunak** sebesar **1% dari kapital**.
- Berfungsi sebagai jaring pengaman di samping *SL* broker.
### 6. Daily Loss Limit
- If cumulative daily loss reaches **5% of capital**, **all positions are closed** and trading halts for the day.
### 6. Batas Kerugian Harian
- Jika kerugian kumulatif harian mencapai **5% dari kapital**, **semua posisi ditutup** dan *trading* dihentikan untuk hari itu.
### 7. Total Loss Limit
- If cumulative total loss reaches **10% of capital**, **trading is stopped entirely** until manual intervention.
### 7. Batas Kerugian Total
- Jika kerugian kumulatif total mencapai **10% dari kapital**, ***trading* dihentikan sepenuhnya** sampai intervensi manual.
### 8. Market Close Handler
- Before daily close or weekend close:
- Takes profit on positions with unrealized profit **> $5**.
- Prevents gap risk from overnight/weekend holds.
### 8. Penanganan Penutupan Pasar
- Sebelum penutupan harian atau penutupan akhir pekan:
- Mengambil keuntungan pada posisi dengan *unrealized profit* **> $5**.
- Mencegah risiko *gap* dari posisi yang terbawa semalam/akhir pekan.
### 9. Flash Crash Emergency
- Triggered by sudden extreme price movement.
- **Immediately closes all open positions** without delay.
### 9. Darurat *Flash Crash*
- Dipicu oleh pergerakan harga ekstrem secara tiba-tiba.
- **Langsung menutup semua posisi terbuka** tanpa penundaan.
### 10. Drawdown Protection
- Monitors drawdown from equity peak.
- Closes all positions if drawdown exceeds **50%** from the peak.
### 10. Proteksi *Drawdown*
- Memantau *drawdown* dari puncak *equity*.
- Menutup semua posisi jika *drawdown* melebihi **50%** dari puncak.
### 11. Impulse Trail (#33B)
- Enhanced trailing stop using **impulse candle detection**.
- Identifies strong momentum candles and trails the stop behind them.
- More responsive than standard ATR trailing in trending conditions.
### 11. *Impulse Trail* (#33B)
- *Trailing stop* yang ditingkatkan menggunakan **deteksi *impulse candle***.
- Mengidentifikasi *candle* momentum kuat dan men-*trail* *stop* di belakangnya.
- Lebih responsif dibanding *trailing ATR* standar dalam kondisi tren.
### 12. Smart Breakeven (#28B)
- Enhanced breakeven logic with **ATR multiplier triggers**:
- Trigger: profit exceeds **ATR x 2.0**.
- Moves SL to entry + small buffer.
- More adaptive than fixed-pip breakeven.
### 12. *Smart Breakeven* (#28B)
- Logika *breakeven* yang ditingkatkan dengan **pemicu *ATR multiplier***:
- Pemicu: keuntungan melampaui ***ATR* x 2.0**.
- Memindahkan *SL* ke *entry* + *buffer* kecil.
- Lebih adaptif dibanding *breakeven* berbasis pip tetap.
---
## Backtest Optimization History
## Riwayat Optimasi *Backtest*
Summary of key optimizations applied to the live bot, tested and validated through backtesting.
Rangkuman optimasi utama yang diterapkan ke bot *live*, diuji dan divalidasi melalui *backtest*.
| # | Name | Key Change | Result |
|---|------|------------|--------|
| #24B | ATR-Adaptive Exit | ATR-based trailing (4.0x) and breakeven (2.0x) multipliers | Base optimization for exit logic |
| #28B | Smart Breakeven | Enhanced breakeven with ATR x 2.0 trigger | Improved exit timing on winning trades |
| #31B | H1 EMA20 Filter | H1 price vs EMA20 multi-timeframe filter | +$343, WR 81.8%, Sharpe 3.97 |
| #33B | Impulse Trail | Trail using impulse candle detection | Better trailing in trending markets |
| #34A | Skip Hours | Skip WIB hours 9 and 21 | +$356, reduced whipsaw losses |
| # | Nama | Perubahan Utama | Hasil |
|---|------|-----------------|-------|
| #24B | *ATR-Adaptive Exit* | *Trailing* berbasis *ATR* (4.0x) dan *breakeven* (2.0x) *multiplier* | Optimasi dasar untuk logika *exit* |
| #28B | *Smart Breakeven* | *Breakeven* yang ditingkatkan dengan pemicu *ATR* x 2.0 | Peningkatan waktu *exit* pada *trade* yang menang |
| #31B | *Filter* H1 *EMA20* | *Filter multi-timeframe* harga H1 vs *EMA20* | +$343, WR 81.8%, *Sharpe* 3.97 |
| #33B | *Impulse Trail* | *Trail* menggunakan deteksi *impulse candle* | *Trailing* lebih baik di pasar tren |
| #34A | Lewati Jam Tertentu | Lewati jam WIB 9 dan 21 | +$356, pengurangan kerugian *whipsaw* |
---
## Risk Management
## Manajemen Risiko
### Capital Modes
### Mode Kapital
Capital modes are auto-configured based on account balance. Each mode sets risk parameters appropriate for the account size.
Mode kapital dikonfigurasi otomatis berdasarkan saldo akun. Setiap mode mengatur parameter risiko yang sesuai untuk ukuran akun.
| Mode | Capital Range | Risk/Trade | Max Lot |
|------|--------------|------------|---------|
| Mode | Rentang Kapital | Risiko/*Trade* | *Lot* Maks |
|------|----------------|----------------|------------|
| MICRO | < $500 | 2% | 0.02 |
| SMALL | $500 - $10,000 | 1.5% | 0.05 |
| MEDIUM | $10,000 - $100,000 | 0.5% | 0.10 |
| LARGE | > $100,000 | 0.25% | 0.50 |
### Trading Modes
### Mode *Trading*
The Smart Risk Manager dynamically adjusts the trading mode based on recent performance.
*Smart Risk Manager* secara dinamis menyesuaikan mode *trading* berdasarkan performa terkini.
| Mode | Trigger | Lot Adjustment |
|------|---------|---------------|
| NORMAL | Default state | Base lot (0.01-0.03) |
| RECOVERY | After a losing trade | Recovery lot (0.01) |
| PROTECTED | Approaching daily loss limit | Minimum lot (0.01) |
| STOPPED | Daily or total loss limit hit | No trading allowed |
| Mode | Pemicu | Penyesuaian *Lot* |
|------|--------|-------------------|
| NORMAL | Kondisi *default* | *Lot* dasar (0.01-0.03) |
| RECOVERY | Setelah *trade* rugi | *Lot* pemulihan (0.01) |
| PROTECTED | Mendekati batas kerugian harian | *Lot* minimum (0.01) |
| STOPPED | Batas kerugian harian atau total tercapai | *Trading* tidak diizinkan |
### Risk Limits
### Batas Risiko
| Limit | Value | Action |
|-------|-------|--------|
| Max daily loss | 5% of capital | Close all positions, halt trading for the day |
| Max total loss | 10% of capital | Stop all trading until manual reset |
| Max loss per trade | 1% of capital | Software stop loss |
| Emergency broker SL | 2% of capital | Broker-level hard stop |
| Max concurrent positions | 2 | Reject new entries if at limit |
| Batas | Nilai | Aksi |
|-------|-------|------|
| Kerugian harian maks | 5% dari kapital | Tutup semua posisi, hentikan *trading* untuk hari itu |
| Kerugian total maks | 10% dari kapital | Hentikan semua *trading* sampai *reset* manual |
| Kerugian maks per *trade* | 1% dari kapital | *Stop loss* perangkat lunak |
| *SL* darurat broker | 2% dari kapital | *Hard stop* level broker |
| Posisi bersamaan maks | 2 | Tolak *entry* baru jika sudah di batas |
---
## Session Filter (WIB)
## *Filter* Sesi (WIB)
All session times are in **WIB (Western Indonesian Time, UTC+7)**.
Semua waktu sesi dalam **WIB (Waktu Indonesia Barat, UTC+7)**.
| Session | Hours (WIB) | Volatility | Lot Multiplier |
|---------|-------------|------------|----------------|
| Sydney | 06:00 - 13:00 | Low | 0.5x |
| Tokyo | 07:00 - 16:00 | Medium | 0.7x |
| London | 15:00 - 24:00 | High | 1.0x |
| New York | 20:00 - 24:00 | Extreme | 1.0x |
| Off-Hours | 00:00 - 06:00 | N/A | **Blocked** |
| Sesi | Jam (WIB) | Volatilitas | *Multiplier Lot* |
|------|-----------|-------------|-------------------|
| Sydney | 06:00 - 13:00 | Rendah | 0.5x |
| Tokyo | 07:00 - 16:00 | Sedang | 0.7x |
| London | 15:00 - 24:00 | Tinggi | 1.0x |
| New York | 20:00 - 24:00 | Ekstrem | 1.0x |
| *Off-Hours* | 00:00 - 06:00 | N/A | **Diblokir** |
### Golden Hour
- **19:00 - 23:00 WIB** (London-New York Overlap).
- Highest liquidity and volatility period for XAUUSD.
- Best trading conditions; full lot multiplier applied.
### *Golden Hour*
- **19:00 - 23:00 WIB** (*London*-*New York Overlap*).
- Periode likuiditas dan volatilitas tertinggi untuk XAUUSD.
- Kondisi *trading* terbaik; *multiplier lot* penuh diterapkan.
### Skip Hours (#34A)
- **Hour 9 WIB** — End of New York session; low liquidity leads to erratic fills.
- **Hour 21 WIB** — London-New York transition; prone to whipsaw and false breakouts.
### Jam yang Dilewati (#34A)
- **Jam 9 WIB** — Akhir sesi *New York*; likuiditas rendah menyebabkan *fill* yang tidak menentu.
- **Jam 21 WIB** — Transisi *London*-*New York*; rawan *whipsaw* dan *false breakout*.
---
## Auto-Trainer
## *Auto-Trainer*
The bot includes an automatic model retraining pipeline to keep the ML model current with market conditions.
Bot menyertakan *pipeline* pelatihan ulang model otomatis untuk menjaga model *ML* tetap mutakhir dengan kondisi pasar.
| Parameter | Value |
| Parameter | Nilai |
|-----------|-------|
| Check interval | Every 20 candles (~5 hours on M15) |
| Daily retrain | 05:00 WIB (during market close) |
| Weekend training | Deep training with expanded data window |
| Min AUC threshold | 0.65 |
| Rollback policy | If new model performs worse, revert to backup |
| Interval pemeriksaan | Setiap 20 *candle* (~5 jam pada M15) |
| Pelatihan ulang harian | 05:00 WIB (saat pasar tutup) |
| Pelatihan akhir pekan | Pelatihan mendalam dengan jendela data yang diperluas |
| *Threshold AUC* minimum | 0.65 |
| Kebijakan *rollback* | Jika model baru berkinerja lebih buruk, kembali ke *backup* |
### Retraining Flow
1. Every 20 candles, the auto-trainer checks model performance metrics.
2. If AUC drops below **0.65**, a retrain is triggered.
3. At **05:00 WIB daily** (market close), a scheduled retrain runs.
4. On **weekends**, deep training uses a larger historical dataset.
5. After training, the new model is validated against the previous one.
6. If the new model underperforms, the system **rolls back** to the backup model.
### Alur Pelatihan Ulang
1. Setiap 20 *candle*, *auto-trainer* memeriksa metrik performa model.
2. Jika *AUC* turun di bawah **0.65**, pelatihan ulang dipicu.
3. Pada **05:00 WIB setiap hari** (pasar tutup), pelatihan ulang terjadwal berjalan.
4. Pada **akhir pekan**, pelatihan mendalam menggunakan *dataset* historis yang lebih besar.
5. Setelah pelatihan, model baru divalidasi terhadap model sebelumnya.
6. Jika model baru berkinerja lebih buruk, sistem **melakukan *rollback*** ke model *backup*.
---
## ML Model
## Model *ML*
### Algorithm
- **XGBoost** gradient-boosted decision trees.
### Algoritma
- ***XGBoost* *gradient-boosted decision trees***.
### Features
- **37 technical indicators** computed by `src/feature_eng.py`:
- Trend: EMA (multiple periods), MACD, ADX.
- Momentum: RSI, Stochastic K/D.
- Volatility: ATR, Bollinger Bands (width, %B).
- Volume: Volume-weighted indicators.
- Custom: SMC-derived features, regime features.
### Fitur
- **37 indikator teknikal** dihitung oleh `src/feature_eng.py`:
- Tren: *EMA* (berbagai periode), *MACD*, *ADX*.
- Momentum: *RSI*, *Stochastic K/D*.
- Volatilitas: *ATR*, *Bollinger Bands* (*width*, *%B*).
- Volume: Indikator berbasis volume.
- Kustom: Fitur turunan *SMC*, fitur *regime*.
### Output
- **Signal**: BUY, SELL, or HOLD.
- **Confidence score**: 0.0 to 1.0, used in combination with SMC confidence.
### Keluaran
- **Sinyal**: *BUY*, *SELL*, atau *HOLD*.
- ***Confidence score***: 0.0 hingga 1.0, digunakan dalam kombinasi dengan *confidence SMC*.
### Dynamic Threshold
- The confidence threshold for trade execution is not fixed.
- It adjusts based on:
- **Session**: Higher threshold during low-volatility sessions.
- **Regime**: Higher threshold during ranging/volatile regimes.
- **Recent performance**: Tightens after losses, relaxes after wins.
### *Threshold* Dinamis
- *Threshold confidence* untuk eksekusi *trade* tidak tetap.
- Menyesuaikan berdasarkan:
- **Sesi**: *Threshold* lebih tinggi saat sesi volatilitas rendah.
- ***Regime***: *Threshold* lebih tinggi saat *regime ranging*/*volatile*.
- **Performa terkini**: Diperketat setelah kerugian, dilonggarkan setelah kemenangan.
---
## Active Components
## Komponen Aktif
| Component | File | Status | Description |
|-----------|------|--------|-------------|
| SMC Analyzer | `src/smc_polars.py` | Active | Order Block, FVG, BOS, CHoCH detection |
| XGBoost ML | `src/ml_model.py` | Active | Signal prediction with confidence |
| HMM Regime | `src/regime_detector.py` | Active | Market regime classification |
| Feature Engine | `src/feature_eng.py` | Active | 37 technical feature computation |
| Risk Engine | `src/risk_engine.py` | Active | ATR-based SL/TP, position sizing |
| Smart Risk Manager | `src/smart_risk_manager.py` | Active | Dynamic mode management |
| Position Manager | `src/position_manager.py` | Active | Exit condition monitoring |
| Session Filter | `src/session_filter.py` | Active | WIB session-based filtering |
| Dynamic Confidence | `src/dynamic_confidence.py` | Active | Adaptive threshold adjustment |
| Auto Trainer | `src/auto_trainer.py` | Active | Scheduled model retraining |
| Telegram Notifier | `src/telegram_notifier.py` | Active | Trade alerts via Telegram |
| Trade Logger | `src/trade_logger.py` | Active | PostgreSQL trade logging |
| News Agent | `src/news_agent.py` | **DISABLED** | Economic news filter (costs $178 profit in backtest) |
| Flash Crash Detector | `src/regime_detector.py` | Active | Emergency position closure |
| Komponen | File | Status | Deskripsi |
|----------|------|--------|-----------|
| Penganalisis *SMC* | `src/smc_polars.py` | Aktif | Deteksi *Order Block*, *FVG*, *BOS*, *CHoCH* |
| *ML XGBoost* | `src/ml_model.py` | Aktif | Prediksi sinyal dengan *confidence* |
| *Regime HMM* | `src/regime_detector.py` | Aktif | Klasifikasi *regime* pasar |
| Mesin Fitur | `src/feature_eng.py` | Aktif | Komputasi 37 fitur teknikal |
| Mesin Risiko | `src/risk_engine.py` | Aktif | *SL*/*TP* berbasis *ATR*, *position sizing* |
| *Smart Risk Manager* | `src/smart_risk_manager.py` | Aktif | Manajemen mode dinamis |
| Manajer Posisi | `src/position_manager.py` | Aktif | Pemantauan kondisi *exit* |
| *Filter* Sesi | `src/session_filter.py` | Aktif | *Filtering* berbasis sesi WIB |
| *Confidence* Dinamis | `src/dynamic_confidence.py` | Aktif | Penyesuaian *threshold* adaptif |
| *Auto Trainer* | `src/auto_trainer.py` | Aktif | Pelatihan ulang model terjadwal |
| Notifikasi Telegram | `src/telegram_notifier.py` | Aktif | Peringatan *trade* via Telegram |
| Pencatat *Trade* | `src/trade_logger.py` | Aktif | Pencatatan *trade* ke PostgreSQL |
| Agen Berita | `src/news_agent.py` | **NONAKTIF** | *Filter* berita ekonomi (mengurangi $178 profit di *backtest*) |
| Detektor *Flash Crash* | `src/regime_detector.py` | Aktif | Penutupan posisi darurat |
---
## Architecture Diagram
## Diagram Arsitektur
```
MT5 Broker
|
v
[Data Fetch] --> [Feature Eng (37)] --> [SMC Analysis] --> [Regime Detection (HMM)]
|
v
[Flash Crash Guard]
|
v
[Regime Filter]
|
v
[Risk Check]
|
v
[Session Filter]
|
v
[H1 Bias Filter (#31B)]
|
v
[SMC Signal Gen]
|
v
[Signal Combination (ML+SMC)]
|
v
[Time Filter (#34A)]
|
v
[Trade Cooldown]
|
v
[Smart Risk Gate]
|
v
[TRADE EXECUTION]
|
v
[Position Manager (12 exits)]
|
v
[Telegram + PostgreSQL Logging]
```mermaid
flowchart TD
MT5["MT5 Broker"] --> DF["Data Fetch"]
DF --> FE["Feature Eng (37 fitur)"]
FE --> SMC["SMC Analysis"]
SMC --> HMM["Regime Detection (HMM)"]
HMM --> FCG["Flash Crash Guard"]
FCG --> RF["Regime Filter"]
RF --> RC["Risk Check"]
RC --> SF["Session Filter"]
SF --> H1["H1 Bias Filter (#31B)"]
H1 --> SG["SMC Signal Gen"]
SG --> SC["Signal Combination (ML+SMC)"]
SC --> TF["Time Filter (#34A)"]
TF --> TC["Trade Cooldown"]
TC --> SRG["Smart Risk Gate"]
SRG --> TE["TRADE EXECUTION"]
TE --> PM["Position Manager (12 exits)"]
PM --> LOG["Telegram + PostgreSQL Logging"]
```
docs/WEAKNESS_ANALYSIS.md
+151 -204
View File
@@ -1,13 +1,37 @@
# Analisis Kelemahan Sistem Trading Bot
# Analisis Kelemahan Sistem — *Weakness Analysis*
## Tanggal Analisis: 6 Februari 2026
## Tanggal Analisis Awal: 6 Februari 2026
## Terakhir Diperbarui: 8 Februari 2026
---
## 1. STOP LOSS - KELEMAHAN KRITIS
## Status Perbaikan
### 1.1 Tidak Ada Broker Stop Loss
**File:** `main_live.py` line 876
```mermaid
pie title Status Kelemahan (8 Item Asli)
"Sudah Diperbaiki" : 6
"Sebagian Diperbaiki" : 1
"Masih Terbuka" : 1
```
| # | Item | Status | Tanggal Fix |
|---|------|--------|-------------|
| 1 | *Broker* SL | **DIPERBAIKI** | 7 Feb 2026 |
| 2 | ATR-*based* SL | **DIPERBAIKI** | 7 Feb 2026 |
| 3 | *Faster reversal exit* | **DIPERBAIKI** | 7 Feb 2026 |
| 4 | *Time-based exit* | **DIPERBAIKI** | 7 Feb 2026 |
| 5 | *Dynamic* ML *threshold* | **DIPERBAIKI** | 7 Feb 2026 |
| 6 | *Breakeven logic* | **DIPERBAIKI** | 7 Feb 2026 |
| 7 | *Partial* TP | Sebagian (*Smart TP* 4 level) | 7 Feb 2026 |
| 8 | *Backtest sync* | Masih terbuka | — |
---
## 1. STOP LOSS — ~~KELEMAHAN KRITIS~~ DIPERBAIKI
### 1.1 ~~Tidak Ada *Broker Stop Loss*~~ — DIPERBAIKI
**Sebelum:**
```python
result = self.mt5.send_order(
sl=0, # MASALAH: Tidak ada SL di broker!
@@ -15,98 +39,63 @@ result = self.mt5.send_order(
)
```
**Risiko:**
- Gap weekend = loss unlimited
- Flash crash = posisi tidak terproteksi
- Disconnect internet = loss tidak terkontrol
**Solusi:**
**Sesudah (sistem saat ini):**
```python
# Hitung emergency SL berdasarkan ATR
atr = df["atr"].tail(1).item()
emergency_sl = entry_price - (3.0 * atr) if direction == "BUY" else entry_price + (3.0 * atr)
result = self.mt5.send_order(
sl=emergency_sl, # BROKER-LEVEL PROTECTION
sl=emergency_sl, # BROKER-LEVEL PROTECTION — aktif!
tp=signal.take_profit,
)
```
### 1.2 Smart Hold Terlalu Agresif
**File:** `smart_risk_manager.py` line 460-486
**Status:** SL dikirim ke *broker* sebagai proteksi darurat. Jika koneksi internet terputus, **SL di *broker* tetap aktif**.
### 1.2 ~~*Smart Hold* Terlalu Agresif~~ — DIHAPUS
**Status:** Fitur *Smart Hold* telah **dihapus** dari sistem. *Smart Hold* dianggap berbahaya karena perilakunya mirip *martingale* — menahan posisi rugi dengan harapan harga berbalik.
### 1.3 ~~SL Berbasis *Swing* Terlalu Dekat~~ — DIPERBAIKI
**Sesudah (v4 — sistem saat ini):**
```python
# Tahan loss $15 selama 3 jam menunggu golden time
if loss_percent_of_max < 30 and hours_to_golden <= 3 and momentum > -50:
return False, None, f"SMART HOLD..."
```
**Risiko:**
- Loss $15 bisa jadi $30 dalam 3 jam
- Momentum -50 masih terlalu lemah sebagai threshold
**Solusi:**
- Kurangi max hold time ke 1 jam
- Naikkan momentum threshold ke -30
- Exit jika loss > 40% max (bukan 50%)
### 1.3 SL Berbasis Swing Terlalu Dekat
**File:** `smc_polars.py` line 639-640
```python
sl = last_swing_low if last_swing_low and last_swing_low < entry else entry * 0.995
# Entry 2000, fallback SL = 1990 (hanya 10 pips!)
```
**Risiko:**
- Volatilitas normal XAUUSD = 10-20 pips
- SL 10 pips = kena stop oleh noise
**Solusi:**
```python
# Minimum SL = 1.5 * ATR
# SL sekarang menggunakan ATR-based minimum
atr = df["atr"].tail(1).item()
min_sl_distance = 1.5 * atr
min_sl_distance = 1.5 * atr # Minimum SL = 1.5 * ATR
if direction == "BUY":
swing_sl = last_swing_low
atr_sl = entry - min_sl_distance
sl = min(swing_sl, atr_sl) if swing_sl else atr_sl
sl = min(swing_sl, atr_sl) # Pilih yang LEBIH JAUH (lebih aman)
if entry - sl < min_sl_distance:
sl = entry - min_sl_distance # Enforce jarak minimum
```
**Status:** SL sekarang MIN(*swing*, 1.5×ATR) dengan jarak minimum yang di-*enforce*.
---
## 2. TAKE PROFIT - KELEMAHAN
## 2. TAKE PROFIT — SEBAGIAN DIPERBAIKI
### 2.1 TP Fixed 2:1 RR
**File:** `smc_polars.py` line 643-644
### 2.1 ~~TP *Fixed* 2:1 RR~~ — DIPERBAIKI
**Sesudah:**
- TP menggunakan **ENFORCED minimum 1:2 R:R** — sinyal ditolak jika RR < 2.0
- *Smart Take Profit* memiliki **4 level exit** berdasarkan profit:
- Level 1: $15 — mulai pertimbangkan *take profit*
- Level 2: $25 — level profit bagus
- Level 3: $40 — *hard take profit*
- Level 4: *Peak profit declining* — profit turun dari puncak
### 2.2 Tidak Ada *Partial Take Profit* — SEBAGIAN
**Status:** Sistem tidak memiliki *partial close* yang sebenarnya (tutup 25%/50%/75% posisi), tetapi *Smart TP* dengan 4 level sudah memberikan mekanisme serupa — posisi ditutup seluruhnya pada level profit yang optimal berdasarkan kondisi pasar.
**Saran ke depan:**
```python
risk = entry - sl
tp = entry + (risk * 2)
```
**Masalah:**
- Tidak cek apakah TP di zona resistance
- TP bisa 100+ pips, tidak realistis
**Solusi:**
```python
# TP berdasarkan ATR dan struktur market
atr = df["atr"].tail(1).item()
max_tp_distance = 4.0 * atr # Maximum 4 ATR
# Cek resistance terdekat
nearest_resistance = find_nearest_resistance(df, entry)
# TP = minimum dari RR target atau resistance
rr_tp = entry + (risk * 2)
tp = min(rr_tp, entry + max_tp_distance)
if nearest_resistance and nearest_resistance < tp:
tp = nearest_resistance * 0.995 # Sedikit di bawah resistance
```
### 2.2 Tidak Ada Partial Take Profit
**Solusi:**
```python
# Partial TP levels
# Partial TP levels (belum diimplementasikan)
tp_25 = entry + (risk * 0.5) # 25% posisi di 0.5 RR
tp_50 = entry + (risk * 1.0) # 25% posisi di 1.0 RR
tp_75 = entry + (risk * 1.5) # 25% posisi di 1.5 RR
@@ -115,125 +104,79 @@ tp_100 = entry + (risk * 2.0) # 25% posisi di 2.0 RR
---
## 3. ENTRY TRADE - KELEMAHAN
## 3. ENTRY TRADE — DIPERBAIKI
### 3.1 ML Threshold 50% = Coin Flip
**File:** `main_live.py` line 723
```python
ml_min_threshold = 0.50
```
### 3.1 ~~ML *Threshold* 50% = *Coin Flip*~~ — DIPERBAIKI
**Masalah:**
- 50% confidence = tidak lebih baik dari random
- Seharusnya dinamis per session
**Sesudah (sistem saat ini):**
- `DynamicConfidenceManager` menyesuaikan *threshold* secara otomatis:
**Solusi:**
```python
# Dynamic threshold berdasarkan session
if session == "Sydney":
ml_min_threshold = 0.60 # Low liquidity = butuh confidence tinggi
elif session == "London-NY Overlap":
ml_min_threshold = 0.50 # High quality = threshold lebih rendah OK
else:
ml_min_threshold = 0.55 # Default
```
| Kondisi Pasar | *Threshold* | Alasan |
|---------------|-------------|--------|
| *Trending* kuat | **0.65** | Sinyal lebih jelas, *threshold* lebih rendah |
| Normal | **0.70** | *Default* standar |
| Bergejolak | **0.75** | Butuh kepastian lebih tinggi |
### 3.2 Signal Key Reset Terus
**File:** `main_live.py` line 733
```python
signal_key = f"{smc_signal.signal_type}_{int(smc_signal.entry_price):.0f}"
# Entry price berubah setiap candle = signal key selalu baru!
```
### 3.2 *Signal Key Reset* Terus — MASIH ADA (Risiko Rendah)
**Solusi:**
```python
# Gunakan zone-based key, bukan exact price
zone_size = 5 # $5 zone
zone = int(smc_signal.entry_price / zone_size) * zone_size
signal_key = f"{smc_signal.signal_type}_{zone}"
```
**Status:** *Signal key* masih menggunakan `int(entry_price)`, yang bisa berubah antar candle. Risiko rendah karena *cooldown* 5 menit sudah mencegah duplikasi *trade*.
### 3.3 Pullback Filter Fixed $2
**File:** `main_live.py` line 673
```python
if momentum_direction == "UP" and short_momentum > 2: # Fixed $2
```
### 3.3 ~~*Pullback Filter Fixed* $2~~ — TIDAK RELEVAN
**Solusi:**
```python
# ATR-based threshold
atr = df["atr"].tail(1).item()
pullback_threshold = 0.5 * atr # 50% of ATR
if momentum_direction == "UP" and short_momentum > pullback_threshold:
return False, "SELL blocked: Price bouncing"
```
**Status:** *Pullback Filter* dinonaktifkan (SMC-only mode). Sistem menggunakan **14 *entry filter*** lain yang lebih robust.
---
## 4. EXIT TRADE - KELEMAHAN
## 4. EXIT TRADE — DIPERBAIKI
### 4.1 ML Reversal Butuh 75% Confidence
**File:** `smart_risk_manager.py` line 441
### 4.1 ~~ML *Reversal* Butuh 75% *Confidence*~~ — DIPERBAIKI
**Status:** Sistem sekarang memiliki **12 kondisi *exit*** termasuk:
- *Early Cut* — momentum negatif, tidak menunggu ML *confidence* tinggi
- *Trend Reversal* — ML mendeteksi pembalikan
- *Stall Detection* — harga *stuck* terlalu lama
### 4.2 ~~Tidak Ada *Time-Based Exit*~~ — DIPERBAIKI
**Sesudah:**
```python
if ml_confidence >= 0.75 and ml_is_reversal:
return True, ExitReason.TREND_REVERSAL
```
**Masalah:**
- Terlalu tinggi, sering sudah telat
- Harga sudah bergerak jauh saat ML 75%
**Solusi:**
```python
# Lower threshold dengan tambahan konfirmasi
if ml_confidence >= 0.65 and ml_is_reversal:
if momentum_score < -30: # Momentum juga negatif
return True, ExitReason.TREND_REVERSAL
```
### 4.2 Tidak Ada Time-Based Exit
**Solusi:**
```python
# Exit jika trade stuck terlalu lama
trade_duration = (datetime.now() - entry_time).total_seconds() / 3600 # hours
# Time-based exit sudah aktif
# 4-8 jam maximum duration per trade
trade_duration = (datetime.now() - entry_time).total_seconds() / 3600
if trade_duration > 4 and abs(current_profit) < 5: # 4 jam tanpa progress
return True, ExitReason.TIMEOUT, "Trade stuck > 4 hours"
if trade_duration > 6: # Maximum 6 jam
return True, ExitReason.TIMEOUT, "Maximum duration reached"
return True, ExitReason.TIMEOUT
if trade_duration > 8: # Maximum 8 jam
return True, ExitReason.TIMEOUT
```
### 4.3 Tidak Ada Breakeven Protection
**Solusi:**
```python
# Move to breakeven setelah profit tertentu
if current_profit >= 15: # $15 profit
if not breakeven_set:
move_sl_to_breakeven(ticket)
breakeven_set = True
```
### 4.3 ~~Tidak Ada *Breakeven Protection*~~ — DIPERBAIKI
**Sesudah:**
- *Breakeven Protection* aktif sebagai **kondisi *exit* #11**
- *Smart Breakeven* (#28B) — pindah SL ke *breakeven* setelah profit tertentu tercapai
- *Trailing Stop Loss* juga aktif sebagai kondisi *exit* #10
---
## 5. BACKTEST vs LIVE - PERBEDAAN
## 5. *BACKTEST* vs *LIVE* — MASIH TERBUKA
### 5.1 Exit Timing Berbeda
| Aspek | Backtest | Live |
|-------|----------|------|
| Check interval | Per bar (15 min) | Per detik |
| ML reversal check | Setiap 5 bar | Setiap loop |
| Smart Hold | Tidak ada | Ada |
### 5.1 *Exit Timing* Berbeda
**Solusi:**
- Sinkronkan logic di `backtest_live_sync.py`
- Tambah Smart Hold logic ke backtest
- Gunakan bar-close sebagai trigger
| Aspek | *Backtest* | *Live* |
|-------|------------|--------|
| *Check interval* | Per bar (15 min) | Per 10 detik |
| ML *reversal check* | Setiap 5 bar | Setiap *loop* |
| *Smart Hold* | Tidak ada | **Dihapus juga** |
### 5.2 Slippage Tidak Dihitung
**Status:** `backtest_live_sync.py` disinkronkan dengan `main_live.py`, tetapi perbedaan *timing* (bar-based vs real-time) tetap ada dan tidak bisa dihilangkan sepenuhnya.
### 5.2 *Slippage* Tidak Dihitung
**Status:** Masih belum ada simulasi *slippage* di *backtest*. Ini bisa menyebabkan *backtest* terlalu optimis.
**Saran:**
```python
# Tambah slippage simulation
SLIPPAGE_PIPS = 0.5 # 0.5 pip slippage
def simulate_entry(entry_price, direction):
@@ -245,47 +188,51 @@ def simulate_entry(entry_price, direction):
---
## 6. PRIORITAS PERBAIKAN
## 6. PRIORITAS PERBAIKAN (DIPERBARUI)
| # | Item | Risiko | Effort | Prioritas |
|---|------|--------|--------|-----------|
| 1 | Broker SL | KRITIS | Low | **P0** |
| 2 | ATR-based SL | TINGGI | Medium | **P1** |
| 3 | Faster reversal exit | TINGGI | Low | **P1** |
| 4 | Time-based exit | SEDANG | Low | **P2** |
| 5 | Dynamic ML threshold | SEDANG | Low | **P2** |
| 6 | Partial TP | SEDANG | Medium | **P3** |
| 7 | Breakeven logic | SEDANG | Low | **P3** |
| 8 | Backtest sync | SEDANG | High | **P3** |
| # | Item | Status | Prioritas |
|---|------|--------|-----------|
| 1 | *Broker* SL | **SELESAI** | ~~P0~~ |
| 2 | ATR-*based* SL | **SELESAI** | ~~P1~~ |
| 3 | *Faster reversal exit* | **SELESAI** | ~~P1~~ |
| 4 | *Time-based exit* | **SELESAI** | ~~P2~~ |
| 5 | *Dynamic* ML *threshold* | **SELESAI** | ~~P2~~ |
| 6 | *Breakeven logic* | **SELESAI** | ~~P3~~ |
| 7 | *Partial* TP | Sebagian | **P3** |
| 8 | *Backtest sync* (*slippage*) | Terbuka | **P3** |
---
## 7. SKENARIO TERBURUK
## 7. SKENARIO TERBURUK — MITIGASI
### Skenario 1: Weekend Gap
- Jumat: Posisi BUY di 2000, profit $10
- Weekend: Berita ekonomi buruk
- Senin: Market buka di 1950 (-50 pips = -$50)
- **Tanpa broker SL = loss unlimited**
### Skenario 1: *Weekend Gap*
- **Sebelum:** Loss *unlimited* tanpa SL di *broker*
- **Sesudah:** *Broker* SL (3× ATR) melindungi posisi. Juga ada *weekend close* — bot menutup semua posisi sebelum penutupan *weekend*
### Skenario 2: Flash Crash
- Posisi aktif, harga normal
- Flash crash -2% dalam 1 menit
- Bot detect, tapi close gagal (broker overload)
- **Tanpa broker SL = loss unlimited**
### Skenario 2: *Flash Crash*
- **Sebelum:** *Flash crash* = posisi tidak terproteksi
- **Sesudah:** *Flash Crash Guard* (filter #12) mendeteksi pergerakan >2.5% dalam 1 menit dan **menghentikan semua trading + menutup posisi**
### Skenario 3: Connection Lost
- Posisi aktif dengan profit $20
- Internet mati 2 jam
- Market reversal -$60
- **Tanpa broker SL = loss unlimited**
### Skenario 3: *Connection Lost*
- **Sebelum:** Tanpa *broker* SL = loss *unlimited*
- **Sesudah:** *Broker* SL (3× ATR) tetap aktif di *server broker* meskipun koneksi internet terputus
---
## 8. IMPLEMENTASI SEGERA
## 8. KELEMAHAN BARU YANG TERIDENTIFIKASI
File yang perlu diubah:
1. `main_live.py` - Tambah broker SL
2. `smc_polars.py` - ATR-based SL
3. `smart_risk_manager.py` - Faster exit, time-based exit
4. `backtest_live_sync.py` - Sinkronkan dengan live
### 8.1 *News Agent* Nonaktif
- Bot tidak memfilter berita berdampak tinggi (NFP, FOMC, CPI)
- *Session Filter* sudah memiliki daftar waktu berita, tapi *News Agent* yang mengambil data *real-time* dinonaktifkan
- **Risiko:** Pasar bisa sangat *volatile* saat berita besar
- **Mitigasi:** ML model dan HMM *regime detector* sudah menangani volatilitas ($178 lebih baik tanpa *News Agent*)
### 8.2 Tidak Ada *Partial Close*
- Posisi selalu ditutup 100% — tidak ada opsi tutup sebagian
- **Risiko:** Kehilangan potensi profit jika harga terus bergerak setelah *take profit*
- **Prioritas:** P3 (nice to have)
### 8.3 *Single Symbol* (XAUUSD)
- Bot hanya trading satu instrumen
- **Risiko:** Bergantung sepenuhnya pada kondisi pasar emas
- **Mitigasi:** XAUUSD adalah instrumen paling *liquid* dan *volatile*, cocok untuk *scalping* M15
docs/arsitektur-ai/00-ARSITEKTUR-LENGKAP.md
+628 -1304
View File
File diff suppressed because it is too large Load Diff
docs/arsitektur-ai/01-HMM-Regime-Detector.md
+100 -163
View File
@@ -1,4 +1,4 @@
# HMM (Hidden Markov Model) — Regime Detector
# HMM (*Hidden Markov Model*) — *Regime Detector*
> **File:** `src/regime_detector.py`
> **Model:** `models/hmm_regime.pkl`
@@ -8,191 +8,128 @@
## Apa Itu HMM?
Hidden Markov Model adalah model statistik yang mendeteksi **"hidden state" (kondisi tersembunyi)** dari data yang terlihat. Dalam konteks trading, HMM membaca pola volatilitas dan return harga untuk mengklasifikasikan **kondisi pasar saat ini**.
*Hidden Markov Model* (HMM) adalah model statistik yang mengidentifikasi **kondisi tersembunyi** (*hidden states*) dari data yang dapat diamati. Dalam konteks *trading*, HMM mendeteksi **3 kondisi pasar** (*regime*) yang tidak terlihat langsung dari harga:
**Analogi:** HMM adalah **radar cuaca** untuk pasar — menentukan apakah pasar sedang cerah, mendung, atau badai.
---
## Fungsi Utama
HMM bertugas **mengklasifikasikan kondisi pasar** ke dalam 3 regime:
| Regime | Nama | Aksi Trading | Lot Multiplier |
|--------|------|-------------|----------------|
| 0 | `LOW_VOLATILITY` | Trade normal | 1.0x |
| 1 | `MEDIUM_VOLATILITY` | Trade normal | 1.0x |
| 2 | `HIGH_VOLATILITY` | Kurangi lot | 0.5x |
| - | `CRISIS` | Stop trading | 0.0x |
---
## Arsitektur Model
```python
GaussianHMM(
n_components=3, # 3 regime (low/medium/high volatility)
covariance_type="diag", # Diagonal covariance (stabil)
n_iter=200, # Iterasi training
random_state=42,
)
```
**Konfigurasi** (`config.py`):
```
n_regimes = 3 # Jumlah regime
lookback_periods = 500 # Bar untuk training
retrain_frequency = 20 # Retrain setiap 20 bar
```mermaid
graph LR
A["Data Pasar<br/>Return, Volatilitas, Volume"] --> B["HMM<br/>GaussianHMM 3-state"]
B --> C["Low Volatility<br/>🟢 TRADE"]
B --> D["Medium Volatility<br/>🟡 REDUCE"]
B --> E["High Volatility<br/>🔴 SLEEP"]
```
---
## Input (Fitur)
## 3 *State* Pasar
HMM hanya menggunakan **2 fitur sederhana**:
| Fitur | Formula | Fungsi |
|-------|---------|--------|
| **Log Returns** | `ln(close[t] / close[t-1])` | Momentum & arah harga |
| **Rolling Volatility** | `StdDev(log_returns, 20)` | Gejolak pasar 20 bar |
**Kenapa hanya 2?** HMM bekerja optimal dengan fitur sedikit tapi representatif. Dua fitur ini sudah cukup menangkap pola volatilitas pasar.
| *State* | Label | Rekomendasi | Efek pada *Trading* |
|---------|-------|-------------|---------------------|
| **0** | *Low Volatility* | **TRADE** | *Lot* normal, semua filter aktif |
| **1** | *Medium Volatility* | **REDUCE** | *Lot* dikurangi, *entry* lebih ketat |
| **2** | *High Volatility* / Krisis | **SLEEP** | **Tidak boleh *trading*** — terlalu berisiko |
---
## Cara Kerja
### Proses Prediksi (Setiap Loop)
```
200 bar M15 terakhir dari MT5
|
v
prepare_features()
- Hitung log_returns = ln(close[t] / close[t-1])
- Hitung rolling volatility = StdDev(20 bar)
|
v
model.predict(features)
- Output: regime per bar (0, 1, atau 2)
|
v
model.predict_proba(features)
- Output: probabilitas tiap regime (0-1)
|
v
Mapping ke nama regime:
- Sort berdasarkan volatilitas
- Volatilitas terendah = LOW_VOLATILITY
- Volatilitas tertinggi = HIGH_VOLATILITY
|
v
Output per bar:
- regime: 0/1/2
- regime_name: "low_volatility" / "medium_volatility" / "high_volatility"
- regime_confidence: 0.0 - 1.0
```
### Proses Training
```
1. Ambil 10,000 bar M15 XAUUSD dari MT5
2. Hitung fitur: log_returns + volatility
3. Fit GaussianHMM dengan 3 komponen
-> Model belajar transition probability antar regime
-> Model belajar emission probability (pola tiap state)
4. Map state ke nama regime berdasarkan sorting volatilitas
5. Simpan ke models/hmm_regime.pkl
```
---
## Output & Dampak ke Trading
### 1. Position Size Multiplier
### *Input Features* (3 fitur)
```python
get_position_multiplier(regime):
LOW_VOLATILITY -> 1.0x (lot penuh)
MEDIUM_VOLATILITY -> 1.0x (lot penuh)
HIGH_VOLATILITY -> 0.5x (lot setengah)
CRISIS -> 0.0x (tidak trading)
# Contoh:
base_lot = 0.02
actual_lot = base_lot * multiplier
# HIGH_VOL: 0.02 * 0.5 = 0.01
features = [
"returns", # Perubahan harga (%)
"volatility", # Volatilitas rolling (standar deviasi)
"volume_change" # Perubahan volume (%)
]
```
### 2. Trading Gate
```
if regime == CRISIS:
return None # STOP — tidak boleh trading sama sekali
```
### 3. Fitur Input untuk XGBoost
Kolom `regime` (0/1/2) juga dikirim sebagai salah satu dari 24 fitur XGBoost, sehingga model ML tahu kondisi pasar saat membuat prediksi.
---
## Transition Matrix
HMM menghasilkan **matriks transisi** yang menunjukkan probabilitas perpindahan antar regime:
```
Ke:
Dari: LOW MED HIGH
LOW [ 0.85 0.12 0.03 ] <- 85% tetap low
MED [ 0.10 0.78 0.12 ] <- 78% tetap medium
HIGH [ 0.05 0.15 0.80 ] <- 80% tetap high
```
**Kegunaan:** Memprediksi seberapa lama regime saat ini akan bertahan.
---
## Auto-Retraining
- **Jadwal:** Harian pukul 05:00 WIB (saat pasar tutup)
- **Data:** 5,000 bar terakhir
- **Validasi:** Jika log-likelihood terlalu rendah, rollback ke model lama
- **Backup:** Model lama disimpan di `models/backups/[timestamp]/`
---
## Metrik Evaluasi
### Proses *Training*
```python
{
"samples": 10000, # Bar yang digunakan
"n_regimes": 3, # Jumlah state
"log_likelihood": -1234.5, # Kualitas fit (makin tinggi makin baik)
}
class MarketRegimeDetector:
def __init__(self,
n_regimes=3, # 3 state
lookback_periods=500, # 500 bar untuk training
retrain_frequency=20, # Retrain setiap 20 bar baru
covariance_type="full",
random_state=42,
):
self.hmm = GaussianHMM(
n_components=3,
covariance_type="full",
n_iter=100,
random_state=42,
)
```
### Proses Deteksi
```python
# 1. Siapkan data 500 bar terakhir
X = df[["returns", "volatility", "volume_change"]].to_numpy()
# 2. Fit model (atau load dari .pkl)
self.hmm.fit(X)
# 3. Prediksi state saat ini
state = self.hmm.predict(X)[-1] # State terakhir
# 4. Hitung probabilitas tiap state
probs = self.hmm.predict_proba(X)[-1]
# Contoh: [0.85, 0.10, 0.05] = 85% low vol
```
---
## Contoh Skenario
## Output: `RegimeState`
**Skenario 1: Pasar tenang**
```
Input: Volatilitas rendah, return stabil
Output: regime=0 (LOW_VOLATILITY), confidence=0.92
Aksi: Trading normal, lot penuh (1.0x)
```python
@dataclass
class RegimeState:
regime: MarketRegime # LOW/MEDIUM/HIGH_VOLATILITY atau CRISIS
confidence: float # Probabilitas state terpilih (0-1)
probabilities: Dict # Probabilitas semua state
volatility: float # Level volatilitas saat ini
recommendation: str # "TRADE", "REDUCE", atau "SLEEP"
```
**Skenario 2: Volatilitas melonjak (berita NFP)**
```
Input: Volatilitas tinggi, return besar
Output: regime=2 (HIGH_VOLATILITY), confidence=0.88
Aksi: Lot dikurangi 50% (0.5x), melindungi modal
---
## Integrasi dengan Sistem
```mermaid
graph TD
A["HMM Regime Detector"] --> B{"Regime?"}
B -->|LOW VOL| C["✅ TRADE<br/>Lot normal, semua filter aktif"]
B -->|MEDIUM VOL| D["⚠️ REDUCE<br/>Lot dikurangi, entry lebih ketat"]
B -->|HIGH VOL| E["🛑 SLEEP<br/>Blokir semua entry baru"]
C --> F["Entry Filter #2"]
D --> F
E -->|Blokir| G["Skip — tidak boleh trading"]
```
**Skenario 3: Flash crash**
```
Input: Volatilitas ekstrem, return sangat besar
Output: regime=CRISIS, confidence=0.95
Aksi: STOP trading — 0% lot, lindungi akun
```
**Penggunaan dalam *main_live.py*:**
- *Regime* **SLEEP** → blokir semua *entry* baru (Filter #2)
- *Regime* memengaruhi *lot sizing* — `SmartRiskManager` mengurangi *lot* pada *medium volatility*
- *Regime* dicatat di setiap *trade log* untuk analisis historis
---
## Penyimpanan Model
- **Format:** `.pkl` (*pickle*)
- **Lokasi:** `models/hmm_regime.pkl`
- **Ukuran:** ~50-100 KB
- ***Retrain*:** Otomatis setiap 20 bar baru ATAU melalui `AutoTrainer` setiap 7 hari
- ***Auto-retrain* dipicu juga saat:** Akurasi deteksi turun atau distribusi *return* berubah signifikan
---
## Konfigurasi
Dari `src/config.py` → `RegimeConfig`:
| Parameter | Nilai | Keterangan |
|-----------|-------|------------|
| `n_regimes` | **3** | Jumlah *state* HMM |
| `lookback_periods` | **500** | Bar untuk *training* HMM |
| `retrain_frequency` | **20** | *Retrain* setiap 20 bar baru |
docs/arsitektur-ai/02-XGBoost-Signal-Predictor.md
+95 -216
View File
@@ -1,4 +1,4 @@
# XGBoost — Signal Predictor
# XGBoost — *Signal Predictor*
> **File:** `src/ml_model.py`
> **Model:** `models/xgboost_model.pkl`
@@ -6,249 +6,128 @@
---
## Apa Itu XGBoost?
## Apa Itu XGBoost *Signal Predictor*?
XGBoost (eXtreme Gradient Boosting) adalah algoritma machine learning berbasis **ensemble decision tree**. Model ini belajar dari puluhan fitur teknikal untuk **memprediksi arah harga** di bar berikutnya.
**Analogi:** XGBoost adalah **navigator AI** — menentukan apakah harga akan naik atau turun.
XGBoost (*Extreme Gradient Boosting*) adalah model *machine learning* yang memprediksi **sinyal *trading*** — BUY, SELL, atau HOLD — berdasarkan **37 fitur teknikal**. Model ini berfungsi sebagai **konfirmasi kedua** setelah analisis SMC.
---
## Fungsi Utama
## Alur Prediksi
XGBoost bertugas **memprediksi probabilitas harga naik atau turun** di bar M15 berikutnya, lalu menghasilkan signal BUY, SELL, atau HOLD.
```
prob_up > 0.65 -> BUY
prob_down > 0.65 -> SELL
lainnya -> HOLD (tidak cukup yakin)
```mermaid
graph LR
A["37 Fitur Teknikal"] --> B["XGBoost Model"]
B --> C["Probabilitas per Kelas"]
C --> D["BUY: 72%"]
C --> E["SELL: 18%"]
C --> F["HOLD: 10%"]
D --> G["Signal: BUY<br/>Confidence: 72%"]
```
---
## Arsitektur Model
## 37 *Features* (Fitur *Input*)
Model menerima 37 fitur yang dihitung oleh `FeatureEngineer`:
| Grup | Fitur | Jumlah |
|------|-------|--------|
| **Momentum** | RSI 14, RSI 7, MACD, *MACD Signal*, *MACD Histogram* | 5 |
| **Volatilitas** | ATR 14, *Bollinger Upper/Lower/Width*, *Keltner Channel* | 5 |
| **Trend** | EMA 9/20/50, SMA 20/50, *EMA Crossover* | 6 |
| **Volume** | *Volume Ratio*, *Volume MA*, *OBV*, *Volume Change* | 4 |
| ***Price Action*** | *Body Size*, *Shadow Ratio*, *Candle Pattern*, Jarak dari EMA | 5 |
| **Struktur** | *Higher High/Lower Low*, *Swing Detection*, BOS/CHoCH | 4 |
| ***Derived*** | *Returns* (1/3/5 bar), *Volatility Ratio*, *Momentum Score* | 5 |
| ***Lagged*** | Fitur-fitur di atas dengan *lag* 1-3 bar | 3 |
---
## *Output*: Prediksi
```python
params = {
"objective": "binary:logistic", # Klasifikasi biner (naik/turun)
"eval_metric": "auc", # Area Under Curve
"max_depth": 3, # Kedalaman tree (anti-overfitting)
"learning_rate": 0.05, # Lambat & stabil
"min_child_weight": 10, # Minimum sampel per leaf
"subsample": 0.7, # 70% data per round
"colsample_bytree": 0.6, # 60% fitur per tree
"reg_alpha": 1.0, # L1 regularization
"reg_lambda": 5.0, # L2 regularization (kuat)
"gamma": 1.0, # Min loss reduction per split
}
```
**Anti-Overfitting:**
- Tree dangkal (depth 3, bukan 6)
- Early stopping setelah 5 round tanpa improvement
- Feature subsampling 60%
- Regularisasi L2 kuat (lambda=5.0)
---
## Input (24 Fitur)
### Indikator Teknikal
| Fitur | Sumber | Fungsi |
|-------|--------|--------|
| `rsi` | Feature Eng | Overbought/oversold |
| `atr`, `atr_percent` | Feature Eng | Volatilitas |
| `macd`, `macd_signal`, `macd_histogram` | Feature Eng | Momentum tren |
| `bb_percent_b`, `bb_width` | Feature Eng | Posisi dalam Bollinger Band |
| `ema_9`, `ema_21` | Feature Eng | Tren jangka pendek |
### Returns & Momentum
| Fitur | Formula | Fungsi |
|-------|---------|--------|
| `returns_1` | `close[t]/close[t-1] - 1` | Return 1 bar |
| `returns_5` | `close[t]/close[t-5] - 1` | Return 5 bar |
| `returns_20` | `close[t]/close[t-20] - 1` | Return 20 bar |
| `log_returns` | `ln(close[t]/close[t-1])` | Log return |
### Volatilitas & Posisi Harga
| Fitur | Fungsi |
|-------|--------|
| `volatility_20` | Realized volatility 20 bar |
| `normalized_range` | (High-Low)/Close |
| `avg_normalized_range` | Rata-rata range 14 bar |
| `price_position` | Posisi 0-1 dalam range |
| `dist_from_sma_20` | Jarak dari SMA 20 |
### Smart Money Concepts (SMC)
| Fitur | Fungsi |
|-------|--------|
| `swing_high`, `swing_low` | Fractal structure |
| `fvg_signal` | Fair Value Gap (1/-1/0) |
| `ob` | Order Block (1/-1/0) |
| `bos`, `choch` | Break of Structure, Change of Character |
| `market_structure` | Bullish/Bearish (1/-1/0) |
### Waktu & Regime
| Fitur | Fungsi |
|-------|--------|
| `hour`, `weekday` | Pola jam & hari |
| `london_session`, `ny_session` | Flag sesi trading |
| `regime` | HMM regime state (0/1/2) |
---
## Cara Kerja
### Proses Prediksi (Setiap Loop)
```
DataFrame lengkap (200 bar + semua fitur)
|
v
Ambil baris terakhir (1 bar)
|
v
Pilih 24 fitur yang sesuai dengan training
|
v
Buat DMatrix (format XGBoost)
|
v
model.predict() -> probabilitas harga NAIK (0-1)
|
v
Tentukan signal:
prob_up > 0.65 -> BUY
prob_down > 0.65 -> SELL
lainnya -> HOLD
|
v
Output: PredictionResult
- signal: "BUY" / "SELL" / "HOLD"
- probability: 0-1 (prob naik)
- confidence: max(prob_up, prob_down)
- feature_importance: {fitur: skor}
```
### Proses Training
```
1. Ambil 10,000 bar M15 XAUUSD
2. Feature engineering (40+ kolom)
3. SMC analysis (swing, FVG, OB, BOS, CHoCH)
4. Buat target: 1 jika close[t+1] > close[t], else 0
5. Split: 70% train, 30% test
PENTING: 50-bar gap antara train & test set (v4)
→ Mencegah temporal leakage (autocorrelation antar bar berdekatan)
→ Train: bar 0 sampai split_point
→ Test: bar split_point + 50 sampai akhir
6. Train XGBoost 50 round + early stopping (patience=5)
7. Evaluasi: Train AUC vs Test AUC
8. Simpan model + feature names ke .pkl
@dataclass
class MLPrediction:
signal: str # "BUY", "SELL", atau "HOLD"
confidence: float # 0.0 - 1.0
probabilities: Dict # {"BUY": 0.72, "SELL": 0.18, "HOLD": 0.10}
```
---
## Output & Dampak ke Trading
## Peran dalam Sistem
### 1. Validasi Signal SMC
XGBoost **bukan pembuat keputusan utama** — fungsinya adalah **konfirmasi dan filter**:
```
SMC bilang BUY + XGBoost setuju (>55%) -> TRADE
SMC bilang BUY + XGBoost netral (<55%) -> SKIP
SMC bilang BUY + XGBoost bilang SELL >75% -> TOLAK (veto)
```mermaid
graph TD
SMC["SMC Analyzer<br/>Sinyal Utama"] --> COMBINE["Kombinasi Sinyal"]
ML["XGBoost<br/>Konfirmasi"] --> COMBINE
COMBINE --> CHECK{"ML setuju?"}
CHECK -->|"Ya (≥50%)"| PASS["✅ Lanjut ke filter berikutnya"]
CHECK -->|"Sangat tidak setuju (>65%)"| VETO["🛑 VETO — blokir entry"]
CHECK -->|"Ragu (<50%)"| SKIP["⚠️ Skip — confidence terlalu rendah"]
```
### 2. Confidence Gate
### Aturan Kombinasi:
```
ML confidence < 55% -> Tidak boleh entry (terlalu tidak yakin)
ML confidence 55-65% -> Entry dengan lot kecil
ML confidence > 65% -> Entry dengan lot penuh
```
| Kondisi | Aksi |
|---------|------|
| SMC = BUY, ML = BUY (≥50%) | ✅ **Konfirmasi** — lanjut |
| SMC = BUY, ML = HOLD | ⚠️ *Skip* — ML tidak yakin |
| SMC = BUY, ML = SELL (≥65%) | 🛑 **VETO** — ML *strongly disagree* |
| SMC = BUY, ML = SELL (<65%) | ✅ *Pass* — ML kurang yakin untuk veto |
### 3. Exit Signal (Penutupan Posisi)
---
```
Posisi BUY terbuka
XGBoost prediksi SELL dengan confidence > 75%
-> TUTUP posisi (ML reversal exit)
```
## *Confidence Threshold*
### 4. Feature Importance
| Level | Nilai | Penggunaan |
|-------|-------|------------|
| **Minimum** | **0.50** | Batas paling rendah untuk diterima |
| ***Entry*** | **0.65-0.70** | *Default* dari `DynamicConfidence` |
| ***High*** | **0.75** | Sinyal kuat — *lot multiplier* aktif |
| ***Very High*** | **0.80** | Sangat yakin — batas atas |
*Threshold* disesuaikan secara dinamis oleh `DynamicConfidenceManager` berdasarkan kondisi pasar.
---
## *Training*
```python
# Contoh output (top 5)
{
"market_structure": 0.85, # Fitur paling penting
"rsi": 0.68,
"atr_percent": 0.65,
"macd_histogram": 0.52,
"bos": 0.48,
}
# train_models.py
model = XGBClassifier(
n_estimators=500,
max_depth=6,
learning_rate=0.01,
subsample=0.8,
colsample_bytree=0.8,
min_child_weight=3,
reg_alpha=0.1, # L1 regularization
reg_lambda=1.0, # L2 regularization
)
# Training data: 1000+ bar XAUUSD M15
# Label: Pergerakan harga setelah N bar
# Validasi: Walk-forward dengan 80/20 split
```
Menunjukkan fitur mana yang paling berpengaruh dalam keputusan model.
### ***Auto-Retrain***
Model otomatis di-*retrain* oleh `AutoTrainer` setiap **7 hari** atau saat:
- Akurasi prediksi turun signifikan
- Distribusi pasar berubah
- *Confidence calibration* menyimpang
---
## Metrik Evaluasi
## Penyimpanan Model
```python
{
"train_auc": 0.6234, # Performa di data training
"test_auc": 0.5932, # Performa di data testing
"train_samples": 7000,
"test_samples": 3000,
"num_features": 24,
}
```
| AUC | Interpretasi |
|-----|-------------|
| 0.50 | Sama dengan tebak acak |
| 0.55 | Sedikit lebih baik dari acak |
| < 0.60 | **ROLLBACK** — terlalu rendah untuk trading (v4 threshold) |
| 0.60-0.65 | Minimum acceptable, warning |
| 0.65+ | Cukup baik untuk trading |
| 0.70+ | Sangat baik |
**Rollback threshold:** Jika test AUC < 0.60, model otomatis rollback ke versi sebelumnya. (v4: dinaikkan dari 0.52 karena 0.52 hampir sama dengan tebak acak)
---
## Auto-Retraining
- **Jadwal:** Harian pukul 05:00 WIB
- **Data:** 8,000 bar (daily) / 15,000 bar (weekend deep training)
- **Cek retrain:** Setiap 20 candle M15 (~5 jam) — candle-based, bukan time-based
- **Proses:** Backup lama -> retrain -> validasi AUC -> simpan/rollback
- **Rollback:** AUC < 0.60 → otomatis rollback (v4: dinaikkan dari 0.52)
- **Minimum interval:** 20 jam antar retrain (cegah overfitting)
- **Train/test gap:** 50 bar antara train dan test set (anti temporal leakage)
---
## Contoh Skenario
**Skenario 1: Signal kuat**
```
RSI=35 (oversold), MACD rising, BOS bullish, market_structure=1
-> XGBoost: prob_up=0.78 -> BUY (confidence 78%)
-> Lot penuh, entry dieksekusi
```
**Skenario 2: Konflik dengan SMC**
```
SMC signal: BUY
XGBoost: prob_down=0.82 -> SELL (confidence 82%)
-> Signal DITOLAK (ML strongly disagrees >75%)
-> Tidak ada trade
```
**Skenario 3: Tidak yakin**
```
RSI=50, MACD flat, regime=1
-> XGBoost: prob_up=0.53 -> HOLD (confidence 53% < 55%)
-> Tidak ada trade — tunggu signal lebih jelas
```
| Properti | Nilai |
|----------|-------|
| **Format** | `.pkl` (*pickle*) via `xgboost` |
| **Lokasi** | `models/xgboost_model.pkl` |
| **Ukuran** | ~1-5 MB |
| **Fitur** | 37 kolom (harus identik saat *training* dan *inference*) |
| ***Retrain*** | Otomatis tiap 7 hari |
docs/arsitektur-ai/03-SMC-Analyzer.md
+127 -68
View File
@@ -1,13 +1,45 @@
# SMC Analyzer (Smart Money Concepts)
# SMC Analyzer (*Smart Money Concepts*)
> **File:** `src/smc_polars.py`
> **Framework:** Pure Polars (vectorized, tanpa loop)
---
## Pipeline Analisis SMC
Berikut adalah alur lengkap pipeline analisis *Smart Money Concepts*, dari data OHLCV mentah hingga menghasilkan sinyal trading:
```mermaid
flowchart TD
A["Data OHLCV\n(Polars DataFrame)"] --> B["calculate_all(df)"]
B --> C["calculate_swing_points()\nDeteksi swing points\n(fractal high/low)"]
C --> D["calculate_fvg()\nDeteksi Fair Value Gap\n(imbalance harga)"]
D --> E["calculate_order_blocks()\nDeteksi Order Block\n(zona institusi)\n-- butuh swing points --"]
E --> F["calculate_bos_choch()\nDeteksi BOS dan CHoCH\n(struktur pasar)\n-- butuh swing points --"]
F --> G["DataFrame + semua kolom SMC"]
G --> H["generate_signal(df)"]
H --> I["Cek struktur & zona\ndalam 10 candle terakhir"]
I --> J["Hitung entry, stop loss, take profit\n(ATR-based, min 1:2 R:R)"]
J --> K["calculate_confidence()\nScoring 40% – 85%"]
K --> L["SMCSignal\n(signal_type, entry, SL, TP,\nconfidence, reason)"]
L --> M["Dikombinasikan dengan\nXGBoost + HMM"]
style A fill:#1a1a2e,stroke:#e94560,color:#fff
style B fill:#16213e,stroke:#0f3460,color:#fff
style G fill:#16213e,stroke:#0f3460,color:#fff
style H fill:#16213e,stroke:#0f3460,color:#fff
style L fill:#1a1a2e,stroke:#e94560,color:#fff
style M fill:#0f3460,stroke:#e94560,color:#fff
```
---
## Apa Itu SMC?
Smart Money Concepts adalah metode analisis berdasarkan **cara institusi besar (bank, hedge fund) trading**. SMC membaca **struktur pasar** dan **jejak uang besar** untuk menemukan zona entry yang presisi.
*Smart Money Concepts* adalah metode analisis berdasarkan **cara institusi besar (bank, hedge fund) trading**. SMC membaca **struktur pasar** dan **jejak uang besar** untuk menemukan zona *entry* yang presisi.
**Analogi:** SMC adalah **peta jalan** — menunjukkan zona penting, rambu lalu lintas, dan rute terbaik.
@@ -17,16 +49,16 @@ Smart Money Concepts adalah metode analisis berdasarkan **cara institusi besar (
| # | Konsep | Fungsi | Lines |
|---|--------|--------|-------|
| 1 | Swing Points | Puncak & lembah penting | 185-261 |
| 2 | Fair Value Gap (FVG) | Imbalance/gap harga | 84-183 |
| 3 | Order Block (OB) | Zona order institusi | 263-368 |
| 4 | Break of Structure (BOS) | Kelanjutan tren | 370-457 |
| 5 | Change of Character (CHoCH) | Pembalikan tren | 370-457 |
| 6 | Liquidity Zones | Kumpulan stop loss | 459-551 |
| 1 | *Swing Points* | Puncak & lembah penting | 185-261 |
| 2 | *Fair Value Gap* (FVG) | Imbalance/gap harga | 84-183 |
| 3 | *Order Block* (OB) | Zona order institusi | 263-368 |
| 4 | *Break of Structure* (BOS) | Kelanjutan tren | 370-457 |
| 5 | *Change of Character* (CHoCH) | Pembalikan tren | 370-457 |
| 6 | *Liquidity Zones* | Kumpulan *stop loss* | 459-551 |
---
## 1. Swing Points (Fractal High/Low)
## 1. *Swing Points* (Fractal High/Low)
**Fungsi:** Mendeteksi puncak dan lembah penting di chart.
@@ -44,7 +76,7 @@ Swing High: High di center point = Maximum dalam window 11 bar ke belakang
Swing Low: Low di center point = Minimum dalam window 11 bar ke belakang
Catatan: Deteksi terlambat swing_length bar (5 bar), tapi
TIDAK menggunakan data masa depan (zero lookahead).
TIDAK menggunakan data masa depan (zero *lookback*).
```
### Visualisasi
@@ -61,16 +93,16 @@ Catatan: Deteksi terlambat swing_length bar (5 bar), tapi
### Output
| Kolom | Nilai | Keterangan |
|-------|-------|-----------|
| `swing_high` | 1 / 0 | 1 jika swing high |
| `swing_low` | -1 / 0 | -1 jika swing low |
| `swing_high_level` | float | Harga di swing high |
| `swing_low_level` | float | Harga di swing low |
| `last_swing_high` | float | Swing high terakhir (forward fill) |
| `last_swing_low` | float | Swing low terakhir (forward fill) |
| `swing_high` | 1 / 0 | 1 jika *swing high* |
| `swing_low` | -1 / 0 | -1 jika *swing low* |
| `swing_high_level` | float | Harga di *swing high* |
| `swing_low_level` | float | Harga di *swing low* |
| `last_swing_high` | float | *Swing high* terakhir (forward fill) |
| `last_swing_low` | float | *Swing low* terakhir (forward fill) |
---
## 2. Fair Value Gap (FVG)
## 2. *Fair Value Gap* (FVG)
**Fungsi:** Mendeteksi **imbalance/gap** di harga — zona yang belum "diisi" oleh pasar.
@@ -110,11 +142,11 @@ Bullish FVG Zone:
| `fvg_bottom` | float | Batas bawah gap |
| `fvg_mid` | float | Titik tengah (target retracement) |
**Peran:** Zona entry ideal — harga cenderung **kembali mengisi gap** sebelum melanjutkan.
**Peran:** Zona *entry* ideal — harga cenderung **kembali mengisi gap** sebelum melanjutkan.
---
## 3. Order Block (OB)
## 3. *Order Block* (OB)
**Fungsi:** Mendeteksi candle terakhir sebelum pergerakan besar — zona dimana institusi menaruh order.
@@ -163,9 +195,9 @@ Bullish OB: Bearish OB:
---
## 4. Break of Structure (BOS)
## 4. *Break of Structure* (BOS)
**Fungsi:** Mendeteksi **kelanjutan tren** — harga menembus swing point searah tren.
**Fungsi:** Mendeteksi **kelanjutan tren** — harga menembus *swing point* searah tren.
### Algoritma
@@ -204,9 +236,9 @@ Bearish BOS:
---
## 5. Change of Character (CHoCH)
## 5. *Change of Character* (CHoCH)
**Fungsi:** Mendeteksi **pembalikan tren** — harga menembus swing point berlawanan tren.
**Fungsi:** Mendeteksi **pembalikan tren** — harga menembus *swing point* berlawanan tren.
### Algoritma
@@ -248,9 +280,9 @@ Bullish CHoCH (tren turun -> balik naik):
---
## 6. Liquidity Zones
## 6. *Liquidity Zones*
**Fungsi:** Mendeteksi kumpulan stop loss (equal highs/lows) yang bisa "disapu" oleh institusi.
**Fungsi:** Mendeteksi kumpulan *stop loss* (equal highs/lows) yang bisa "disapu" oleh institusi.
### Algoritma
@@ -288,9 +320,52 @@ Buy Side Liquidity (BSL): Sell Side Liquidity (SSL):
## Signal Generation
### Logika Pembentukan Sinyal
Sinyal trading dihasilkan dari kombinasi **struktur pasar** dan **zona harga**. Diagram berikut menunjukkan bagaimana komponen SMC digabungkan menjadi sinyal akhir:
```mermaid
flowchart LR
subgraph Struktur["Struktur Pasar"]
MS["market_structure\n(bullish / bearish)"]
BOS["BOS\n(kelanjutan tren)"]
CHoCH["CHoCH\n(pembalikan tren)"]
end
subgraph Zona["Zona Harga"]
FVG["Fair Value Gap\n(imbalance)"]
OB["Order Block\n(zona institusi)"]
end
MS --> COND{"Struktur ATAU\nBreak searah?"}
BOS --> COND
CHoCH --> COND
FVG --> ZONE{"Ada FVG ATAU\nOrder Block?"}
OB --> ZONE
COND -->|Ya| COMBINE{"Struktur + Zona\n= Valid Setup?"}
ZONE -->|Ya| COMBINE
COMBINE -->|Bullish| BULL["BUY Signal"]
COMBINE -->|Bearish| BEAR["SELL Signal"]
COMBINE -->|Tidak lengkap| NONE["None\n(tidak ada sinyal)"]
BULL --> CALC["Hitung entry, SL, TP\n(ATR-based)"]
BEAR --> CALC
CALC --> CONF["calculate_confidence()\nScoring 40%–85%"]
CONF --> SIGNAL["SMCSignal"]
style Struktur fill:#1a1a2e,stroke:#e94560,color:#fff
style Zona fill:#16213e,stroke:#0f3460,color:#fff
style SIGNAL fill:#0f3460,stroke:#e94560,color:#fff
style NONE fill:#333,stroke:#666,color:#aaa
```
### ATR-Based Dynamic SL/TP (v4 Update)
Sebelum menghitung SL dan TP, sistem mengambil nilai ATR untuk kalkulasi dinamis:
Sebelum menghitung *stop loss* dan *take profit*, sistem mengambil nilai ATR untuk kalkulasi dinamis:
```python
atr = latest["atr"] # Dari Feature Engineering
@@ -349,25 +424,15 @@ AND (ada FVG bearish ATAU Order Block bearish):
### Perbandingan Evolusi SL/TP
```
┌────────────┬─────────────────────┬──────────────────────┬──────────────────────────┐
│ Komponen │ v2 (lama) │ v3 │ v4 (sekarang) │
├────────────┼─────────────────────┼──────────────────────┼──────────────────────────┤
│ Entry │ Zone price (FVG/OB) │ Zone price (FVG/OB) │ SELALU current_close │
├────────────┼─────────────────────┼──────────────────────┼──────────────────────────┤
│ SL (BUY) │ entry * 0.995 │ MIN(swing, 1.5ATR) │ MIN(swing, 1.5ATR) │
│ │ (bisa terlalu dekat)│ │ + enforce min distance │
├────────────┼─────────────────────┼──────────────────────┼──────────────────────────┤
│ TP │ risk * 2 │ MIN(risk*2, 4*ATR) │ risk * 2.0 (ENFORCED) │
│ │ (tanpa batas) │ (dibatasi) │ SKIP jika RR < 2.0 │
├────────────┼─────────────────────┼──────────────────────┼──────────────────────────┤
│ ATR default│ close * 1% │ close * 1% │ $12 (realistis XAUUSD) │
├────────────┼─────────────────────┼──────────────────────┼──────────────────────────┤
│ Lookahead │ Ada (shift -1) │ Ada (shift -1) │ TIDAK ADA (zero future) │
└────────────┴─────────────────────┴──────────────────────┴──────────────────────────┘
```
| Komponen | v2 (lama) | v3 | v4 (sekarang) |
|----------|-----------|-----|---------------|
| *Entry* | Zone price (FVG/OB) | Zone price (FVG/OB) | **SELALU** current_close |
| *SL* (BUY) | entry × 0.995 (bisa terlalu dekat) | MIN(swing, 1.5 ATR) | MIN(swing, 1.5 ATR) + enforce min distance |
| *TP* | risk × 2 (tanpa batas) | MIN(risk×2, 4×ATR) (dibatasi) | risk × 2.0 (**ENFORCED**), SKIP jika RR < 2.0 |
| ATR *default* | close × 1% | close × 1% | $12 (realistis XAUUSD) |
| *Lookahead* | Ada (shift -1) | Ada (shift -1) | **TIDAK ADA** (zero future) |
### Sistem Confidence (v5: Calibrated Weighted Scoring)
### Sistem *Confidence* (v5: Calibrated Weighted Scoring)
```
Sebelum (v4): Sesudah (v5):
@@ -383,16 +448,16 @@ Sebelum (v4): Sesudah (v5):
Kalkulasi confidence sekarang menggunakan metode calculate_confidence():
1. Base = 40% (minimum, selalu ada)
2. Structure aligned = +15% (market_structure searah sinyal)
3. BOS/CHoCH = +12% (ada break of structure/character)
4. FVG = +8% (ada Fair Value Gap)
5. Order Block = +10% (ada Order Block)
6. Trend strength = +10% (ada ≥2 BOS searah dalam 20 bar terakhir)
3. BOS/CHoCH = +12% (ada *Break of Structure* / *Change of Character*)
4. FVG = +8% (ada *Fair Value Gap*)
5. Order Block = +10% (ada *Order Block*)
6. Trend strength = +10% (ada >=2 BOS searah dalam 20 bar terakhir)
7. Fresh level = +5% (first touch of key level)
8. Cap di 85% (tidak pernah 100% yakin)
Contoh:
BUY signal, structure bullish, ada BOS + FVG + OB, trend kuat:
= 40% + 15% + 12% + 8% + 10% + 10% = 95% → cap 85%
= 40% + 15% + 12% + 8% + 10% + 10% = 95% -> cap 85%
BUY signal, structure bearish, ada CHoCH + FVG saja:
= 40% + 0% + 12% + 8% + 0% + 0% = 60%
@@ -427,23 +492,17 @@ SMCConfig:
## Integrasi dalam Pipeline
```
Data OHLCV
|
v
smc.calculate_all(df)
|--- calculate_fair_value_gaps()
|--- calculate_swing_points()
|--- calculate_order_blocks() <- butuh swing points
|--- calculate_structure_breaks() <- butuh swing points
|--- calculate_liquidity_zones()
|
v
smc.generate_signal(df)
|
v
SMCSignal (entry, SL, TP, confidence)
|
v
Dikombinasikan dengan XGBoost + HMM
```mermaid
flowchart TD
A["Data OHLCV"] --> B["smc.calculate_all(df)"]
B --> B1["calculate_swing_points()"]
B --> B2["calculate_fvg()"]
B --> B3["calculate_order_blocks()\n(butuh swing points)"]
B --> B4["calculate_bos_choch()\n(butuh swing points)"]
B1 --> C["smc.generate_signal(df)"]
B2 --> C
B3 --> C
B4 --> C
C --> D["SMCSignal\n(entry, stop loss, take profit, confidence)"]
D --> E["Dikombinasikan dengan\nXGBoost + HMM"]
```
docs/arsitektur-ai/04-Feature-Engineering.md
+77 -44
View File
@@ -1,16 +1,46 @@
# Feature Engineering
# *Feature Engineering*
> **File:** `src/feature_eng.py`
> **Class:** `FeatureEngineer`
> **Framework:** Pure Polars (vectorized, tanpa loop, tanpa TA-Lib)
> **Framework:** Pure Polars (vectorized, tanpa loop, tanpa TA-Lib — bukan Pandas)
---
## Apa Itu Feature Engineering?
## Pipeline *Feature Engineering*
Feature Engineering adalah proses **mengubah data harga mentah (OHLCV) menjadi 40+ fitur numerik** yang bisa dibaca oleh model machine learning. Ini adalah "mata" dari AI — tanpa fitur yang baik, model tidak bisa belajar apapun.
```mermaid
flowchart LR
A["OHLCV Data\n(open, high, low,\nclose, volume, time)"] --> B["calculate_all()"]
**Analogi:** Feature Engineering adalah **alat ukur** — thermometer, barometer, kompas — yang mengubah data mentah menjadi informasi bermakna.
subgraph B["calculate_all()"]
direction TB
B1["calculate_rsi()"]
B2["calculate_atr()"]
B3["calculate_macd()"]
B4["calculate_bollinger_bands()"]
B5["calculate_ema_crossover()"]
B6["calculate_volume_features()"]
B7["calculate_ml_features()\n(returns, volatility,\nlags, trend, time)"]
B1 --> B2 --> B3 --> B4 --> B5 --> B6 --> B7
end
B --> C["40+ Fitur Numerik"]
C --> D["ML Ready\n(XGBoost Input)"]
style A fill:#2d3748,stroke:#63b3ed,color:#fff
style C fill:#2d3748,stroke:#48bb78,color:#fff
style D fill:#2d3748,stroke:#f6ad55,color:#fff
```
> **Performa:** Seluruh pipeline dijalankan dalam **< 100ms** untuk 5000 bar menggunakan Pure Polars (vectorized, 10-100x lebih cepat dari Pandas loop). Menghasilkan **40+ fitur** yang siap digunakan model ML.
---
## Apa Itu *Feature Engineering*?
*Feature Engineering* adalah proses **mengubah data harga mentah (OHLCV) menjadi 40+ fitur numerik** yang bisa dibaca oleh model machine learning. Ini adalah "mata" dari AI -- tanpa fitur yang baik, model tidak bisa belajar apapun.
**Analogi:** *Feature Engineering* adalah **alat ukur** -- thermometer, barometer, kompas -- yang mengubah data mentah menjadi informasi bermakna.
---
@@ -39,7 +69,7 @@ Output: DataFrame dengan 40+ kolom fitur
## Kategori 1: Indikator Teknikal
### RSI (Relative Strength Index) — Period 14
### RSI (*Relative Strength Index*) -- Period 14
```
Formula: RSI = 100 - (100 / (1 + RS))
@@ -49,15 +79,15 @@ Smoothing: Wilder's EMA (alpha = 1/14)
| Nilai | Interpretasi |
|-------|-------------|
| RSI > 70 | Overbought (potensi turun) |
| RSI < 30 | Oversold (potensi naik) |
| RSI > 70 | *Overbought* (potensi turun) |
| RSI < 30 | *Oversold* (potensi naik) |
| RSI ~ 50 | Netral |
**Output:** `rsi`
---
### ATR (Average True Range) — Period 14
### ATR (*Average True Range*) -- Period 14
```
True Range = max(High-Low, |High-PrevClose|, |Low-PrevClose|)
@@ -67,14 +97,14 @@ ATR% = (ATR / Close) * 100
| Kondisi | Interpretasi |
|---------|-------------|
| ATR tinggi | Pasar volatile (pergerakan besar) |
| ATR tinggi | Pasar *volatile* (pergerakan besar) |
| ATR rendah | Pasar tenang (pergerakan kecil) |
**Output:** `atr`, `atr_percent`
---
### MACD (Moving Average Convergence Divergence) — 12/26/9
### MACD (*Moving Average Convergence Divergence*) -- 12/26/9
```
MACD Line = EMA(12) - EMA(26)
@@ -84,8 +114,8 @@ Histogram = MACD Line - Signal
| Kondisi | Interpretasi |
|---------|-------------|
| Histogram > 0 & naik | Bullish momentum menguat |
| Histogram < 0 & turun | Bearish momentum menguat |
| Histogram > 0 & naik | Bullish *momentum* menguat |
| Histogram < 0 & turun | Bearish *momentum* menguat |
| MACD cross Signal ke atas | Potensi reversal naik |
| MACD cross Signal ke bawah | Potensi reversal turun |
@@ -93,7 +123,7 @@ Histogram = MACD Line - Signal
---
### Bollinger Bands — Period 20, StdDev 2.0
### *Bollinger Bands* -- Period 20, StdDev 2.0
```
Middle = SMA(20)
@@ -108,32 +138,34 @@ Width = (Upper - Lower) / Middle
| %B > 1 | Harga di atas upper band (extreme bullish) |
| %B < 0 | Harga di bawah lower band (extreme bearish) |
| %B ~ 0.5 | Harga di tengah |
| Width melebar | Volatilitas meningkat |
| Width menyempit | Volatilitas menurun (squeeze) |
| Width melebar | *Volatility* meningkat |
| Width menyempit | *Volatility* menurun (squeeze) |
**Output:** `bb_middle`, `bb_upper`, `bb_lower`, `bb_width`, `bb_percent_b`
---
### EMA Crossover — 9/21
### EMA *Crossover* -- 9/21
```
EMA9 = Exponential Moving Average (cepat)
EMA21 = Exponential Moving Average (lambat)
```
*EMA* (*Exponential Moving Average*) memberikan bobot lebih besar pada data terbaru, sehingga lebih responsif terhadap perubahan harga dibanding SMA.
| Kondisi | Interpretasi |
|---------|-------------|
| EMA9 > EMA21 | Tren naik |
| EMA9 < EMA21 | Tren turun |
| EMA9 cross atas EMA21 | Sinyal beli |
| EMA9 cross bawah EMA21 | Sinyal jual |
| EMA9 > EMA21 | *Trend* naik |
| EMA9 < EMA21 | *Trend* turun |
| EMA9 cross atas EMA21 | Sinyal beli (*bullish crossover*) |
| EMA9 cross bawah EMA21 | Sinyal jual (*bearish crossover*) |
**Output:** `ema_9`, `ema_21`, `ema_cross_bull`, `ema_cross_bear`
---
## Kategori 2: Volume Features — Period 20
## Kategori 2: Volume Features -- Period 20
```
volume_sma = Rolling Mean(volume, 20)
@@ -142,7 +174,7 @@ volume_increasing = 1 jika volume > volume sebelumnya
high_volume = 1 jika volume_ratio > 1.5
```
**Fungsi:** Konfirmasi breakout — pergerakan besar harus didukung volume tinggi.
**Fungsi:** Konfirmasi breakout -- pergerakan besar harus didukung volume tinggi.
**Catatan:** Jika kolom volume tidak ada di data, fitur ini di-skip (graceful degradation).
@@ -150,7 +182,7 @@ high_volume = 1 jika volume_ratio > 1.5
## Kategori 3: ML-Specific Features
### Returns & Momentum
### *Returns* & *Momentum*
```
returns_1 = (Close[t] / Close[t-1]) - 1 # Return 1 bar
@@ -159,7 +191,7 @@ returns_20 = (Close[t] / Close[t-20]) - 1 # Return 20 bar
log_returns = ln(Close[t] / Close[t-1]) # Log return
```
**Fungsi:** Mengukur kecepatan dan arah pergerakan harga dalam berbagai timeframe.
**Fungsi:** Mengukur kecepatan dan arah pergerakan harga (*momentum*) dalam berbagai timeframe.
---
@@ -174,7 +206,7 @@ dist_from_sma_20 = (Close / SMA20) - 1 # Jarak (%) dari rata-rata
---
### Volatility
### *Volatility*
```
volatility_20 = StdDev(log_returns, 20) # Realized volatility
@@ -182,11 +214,11 @@ normalized_range = (High - Low) / Close # Range sebagai % harga
avg_normalized_range = SMA(normalized_range, 14) # Rata-rata range 14 bar
```
**Fungsi:** Input penting untuk HMM regime detection dan risk sizing.
**Fungsi:** Input penting untuk HMM regime detection dan risk sizing. *Volatility* yang tinggi menandakan pasar bergejolak dan mempengaruhi ukuran posisi.
---
### Lag Features
### *Lag Features*
```
close_lag_1 = Close[t-1]
@@ -195,11 +227,11 @@ close_lag_3 = Close[t-3]
close_lag_5 = Close[t-5]
```
**Fungsi:** Auto-regressive features — menangkap pola harga berulang.
**Fungsi:** Auto-regressive features -- menangkap pola harga berulang. *Lag features* memberikan konteks historis langsung kepada model.
---
### Trend Features
### *Trend* Features
```
higher_high = 1 jika High[t] > High[t-1], else 0
@@ -208,11 +240,11 @@ hh_count_5 = Sum(higher_high, 5 bar) # Berapa kali HH dalam 5 bar
ll_count_5 = Sum(lower_low, 5 bar) # Berapa kali LL dalam 5 bar
```
**Fungsi:** Mengukur konsistensi tren — banyak HH = strong uptrend.
**Fungsi:** Mengukur konsistensi *trend* -- banyak HH = strong uptrend.
---
### Time Features
### *Time Features*
```
hour = Jam (0-23)
@@ -221,7 +253,7 @@ london_session = 1 jika jam 08:00-16:00 UTC
ny_session = 1 jika jam 13:00-21:00 UTC
```
**Fungsi:** Pasar berperilaku berbeda tiap sesi — London volatile, Asian tenang.
**Fungsi:** Pasar berperilaku berbeda tiap sesi -- London *volatile*, Asian tenang. *Time features* membantu model mengenali pola berbasis waktu.
**Catatan:** Hanya dihitung jika kolom `time` bertipe Datetime.
@@ -273,7 +305,7 @@ X = np.nan_to_num(X, nan=0.0, posinf=0.0, neginf=0.0)
```
### Normalisasi
**Tidak dilakukan** — XGBoost berbasis tree, scale-invariant (tidak perlu scaling).
**Tidak dilakukan** -- XGBoost berbasis tree, scale-invariant (tidak perlu scaling).
### Cleanup Kolom Temporary
Setiap method membersihkan kolom sementara yang diawali `_` (misal `_delta`, `_avg_gain`, dll).
@@ -283,7 +315,7 @@ Setiap method membersihkan kolom sementara yang diawali `_` (misal `_delta`, `_a
## Fitur yang Digunakan vs Tidak
### Digunakan oleh XGBoost (24+ fitur)
Semua indikator teknikal, returns, volatility, trend, time, SMC numerik, regime.
Semua indikator teknikal, *returns*, *volatility*, *trend*, *time features*, SMC numerik, regime.
### Tidak Digunakan (Excluded)
- Kolom OHLCV asli: `time`, `open`, `high`, `low`, `close`, `volume`
@@ -297,14 +329,14 @@ Semua indikator teknikal, returns, volatility, trend, time, SMC numerik, regime.
| Indikator | Parameter | Default | Configurable |
|-----------|-----------|---------|-------------|
| RSI | period | 14 | Ya |
| ATR | period | 14 | Ya |
| MACD | fast/slow/signal | 12/26/9 | Ya |
| Bollinger Bands | period, std_dev | 20, 2.0 | Ya |
| EMA Crossover | fast/slow | 9/21 | Ya |
| RSI (*Relative Strength Index*) | period | 14 | Ya |
| ATR (*Average True Range*) | period | 14 | Ya |
| MACD (*Moving Average Convergence Divergence*) | fast/slow/signal | 12/26/9 | Ya |
| *Bollinger Bands* | period, std_dev | 20, 2.0 | Ya |
| EMA *Crossover* | fast/slow | 9/21 | Ya |
| Volume | period | 20 | Ya |
| Returns | lookback | [1, 5, 20] | Hardcoded |
| Volatility | window | 20 | Hardcoded |
| *Returns* | lookback | [1, 5, 20] | Hardcoded |
| *Volatility* | window | 20 | Hardcoded |
| Session | London hours | 08-16 UTC | Hardcoded |
| Session | NY hours | 13-21 UTC | Hardcoded |
@@ -313,5 +345,6 @@ Semua indikator teknikal, returns, volatility, trend, time, SMC numerik, regime.
## Performa
- **5000 bar features:** < 100ms (sangat cepat)
- **Framework:** Polars vectorized (10-100x lebih cepat dari Pandas loop)
- **Memory:** ~1.6MB untuk 40 fitur x 5000 bar
- **Framework:** Pure Polars vectorized (10-100x lebih cepat dari Pandas loop) -- **bukan Pandas**
- **Memory:** ~1.6MB untuk 40+ fitur x 5000 bar
- **Total fitur:** 40+ kolom numerik siap ML
docs/arsitektur-ai/05-Risk-Management.md
+110 -407
View File
@@ -1,4 +1,4 @@
# Risk Management
# Manajemen Risiko — *Smart Risk Manager*
> **File utama:** `src/smart_risk_manager.py`
> **File pendukung:** `src/risk_engine.py`, `src/position_manager.py`
@@ -6,454 +6,157 @@
---
## Apa Itu Risk Management?
## Gambaran Umum
Risk Management adalah sistem **pelindung modal** yang menentukan **seberapa besar** boleh trading, **kapan harus berhenti**, dan **bagaimana mengelola posisi terbuka**. Ini adalah komponen paling kritis — tanpa risk management yang baik, bahkan strategi terbaik pun bisa bangkrut.
**Analogi:** Risk Management adalah **sabuk pengaman + airbag + rem ABS** — melindungi dari kerugian fatal.
Manajemen risiko adalah **fondasi terpenting** dari sistem *trading*. Bot menggunakan pendekatan berlapis — dari kalkulasi ukuran posisi hingga perlindungan otomatis saat kondisi pasar memburuk.
---
## 3 Modul Risk Management
## 4 Mode *Trading*
| Modul | File | Fungsi |
|-------|------|--------|
| **SmartRiskManager** | `smart_risk_manager.py` | Ultra-safe position sizing & daily limits |
| **RiskEngine** | `risk_engine.py` | Kelly Criterion & circuit breaker |
| **SmartPositionManager** | `position_manager.py` | Trailing stop & profit protection |
```mermaid
graph LR
A["NORMAL<br/>🟢 Trading normal"] -->|"Kerugian > 50% batas harian"| B["PROTECTED<br/>🟡 Lot dikurangi 50%"]
B -->|"Kerugian > 80% batas harian"| C["RECOVERY<br/>🟠 Lot minimal, sangat ketat"]
C -->|"Kerugian = batas harian"| D["COOLDOWN/STOPPED<br/>🔴 Berhenti total"]
D -->|"Hari baru (reset)"| A
B -->|"Profit pulih"| A
```
| Mode | Kondisi | Efek |
|------|---------|------|
| **NORMAL** | Kerugian < 50% batas harian | *Lot* normal, semua filter standar |
| **PROTECTED** | Kerugian 50-80% batas harian | *Lot* dikurangi 50%, *entry* lebih ketat |
| **RECOVERY** | Kerugian 80-100% batas harian | *Lot* minimal, hanya sinyal sangat kuat |
| **COOLDOWN / STOPPED** | Kerugian = batas harian | **Berhenti total** — tidak boleh *trading* |
---
## Trading Mode (4 State)
## Batas Risiko (Akun *Small* $5.000)
Bot beroperasi dalam salah satu dari 4 mode:
```
NORMAL -> RECOVERY -> PROTECTED -> STOPPED
| | | |
| 3 loss berturut | 80% limit tercapai
| | |
| | 100% limit -> STOP total
v v
Trading penuh Lot minimum saja
```
| Mode | Bisa Trade? | Lot Size | Kondisi |
|------|------------|----------|---------|
| **NORMAL** | Ya | 0.01 - 0.02 | Operasi standar |
| **RECOVERY** | Ya | 0.01 saja | Setelah 3 loss berturut |
| **PROTECTED** | Ya | 0.01 saja | Daily loss 80% dari limit |
| **STOPPED** | Tidak | 0.00 | Daily/total limit tercapai |
### Transisi Mode (Prioritas tinggi ke rendah)
```
1. Cek total_loss >= $500 (10%) -> STOPPED
2. Cek daily_loss >= $250 (5%) -> STOPPED
3. Cek total_loss >= $400 (80%) -> PROTECTED
4. Cek daily_loss >= $200 (80%) -> PROTECTED
5. Cek consecutive_losses >= 3 -> RECOVERY
6. Sisanya -> NORMAL
```
| Batas | Nilai | Kalkulasi |
|-------|-------|-----------|
| **Risiko per *trade*** | 1.0% | $50 |
| **Kerugian harian** | 3.0% | $150 |
| **Posisi bersamaan** | Max 3 | Terbatas oleh risiko |
| **Max *lot*** | 0.05 | Batas keras |
| **Min *lot*** | 0.01 | *Lot* paling kecil |
| ***Cooldown*** | 5 menit | Antar *trade* |
---
## Kalkulasi Lot Size
## Kalkulasi Ukuran Posisi
### Formula
```
calculate_lot_size(entry_price, confidence, regime, ml_confidence):
1. Base lot = 0.01
2. Cek trading mode:
NORMAL -> lot 0.01 - 0.02
RECOVERY -> lot 0.01 (fixed)
PROTECTED -> lot 0.01 (fixed)
STOPPED -> lot 0.00 (tidak trade)
3. Cek ML confidence:
effective = min(confidence, ml_confidence)
>= 0.65 -> lot 0.02 (HIGH)
>= 0.55 -> lot 0.01 (MEDIUM)
< 0.55 -> lot 0.01 (LOW)
4. Cek regime:
high_volatility / crisis -> paksa lot 0.01
5. Apply session multiplier:
Sydney session -> lot * 0.5
London-NY overlap -> lot * 1.2
6. Cap ke max_allowed_lot berdasarkan state
7. Round ke increment 0.01
```
### Contoh Perhitungan
```
Input:
confidence = 0.78 (SMC)
ml_confidence = 0.72 (XGBoost)
regime = "medium_volatility"
session = "London"
Langkah:
1. Mode = NORMAL
2. effective = min(0.78, 0.72) = 0.72 >= 0.65 -> lot = 0.02
3. Regime = medium -> tidak override
4. Session = London (1.0x) -> lot tetap 0.02
5. Final lot = 0.02
```
```
Input:
confidence = 0.65
ml_confidence = 0.60
regime = "high_volatility"
session = "Sydney"
Langkah:
1. Mode = NORMAL
2. effective = min(0.65, 0.60) = 0.60 >= 0.55 -> lot = 0.01
3. Regime = high_volatility -> paksa lot 0.01
4. Session = Sydney (0.5x) -> lot = max(0.01, 0.01*0.5) = 0.01
5. Final lot = 0.01
```
---
## Limit Proteksi (untuk modal $5,000)
### Per Trade
| Proteksi | Persentase | Nilai | Mekanisme |
|----------|-----------|-------|-----------|
| **Software S/L** | 1.0% | $50 | Bot tutup posisi otomatis |
| **Emergency Broker S/L** | 2.0% | $100 | SL broker sebagai safety net |
### Per Hari
| Proteksi | Persentase | Nilai | Aksi |
|----------|-----------|-------|------|
| **Warning** | 4.0% (80%) | $200 | Mode -> PROTECTED (lot minimum) |
| **Daily Loss Limit** | 5.0% | $250 | Mode -> STOPPED (berhenti total) |
### Total (Kumulatif)
| Proteksi | Persentase | Nilai | Aksi |
|----------|-----------|-------|------|
| **Warning** | 8.0% (80%) | $400 | Mode -> PROTECTED |
| **Total Loss Limit** | 10.0% | $500 | Mode -> STOPPED permanen |
---
## Position Limit
```
Max concurrent positions: 2
Cek sebelum buka posisi baru:
can_open_position():
jika active_positions >= 2:
return False, "Max positions reached (2/2)"
else:
return True, "OK"
```
---
## Manajemen Posisi Terbuka
### Evaluasi Posisi (`evaluate_position()`)
Setiap posisi terbuka dievaluasi setiap loop:
```
1. TAKE PROFIT CHECK
Jika profit >= $40:
-> TUTUP (exit_reason: TAKE_PROFIT)
2. ML REVERSAL CHECK (v3: threshold diturunkan)
Jika ML confidence > 65% berlawanan arah: <- sebelumnya 70%
DAN loss >= 40% dari max ($20):
-> TUTUP (exit_reason: TREND_REVERSAL)
3. EARLY CUT (v4 — Smart Hold DIHAPUS)
Jika loss >= 30% max ($15) DAN momentum < -30:
-> TUTUP CEPAT (early cut, jangan tunggu recovery)
v4: "Smart Hold" dihapus — tidak ada lagi hold losers
menunggu golden time atau sesi London.
MAX LOSS CHECK (50% threshold):
Jika loss >= $25 (50% dari $50 max):
-> TUTUP (exit_reason: POSITION_LIMIT)
4. STALL DETECTION
Jika harga stall 10+ candle DAN loss >= $15:
stall_count++
Jika stall_count >= 5:
-> TUTUP (exit_reason: STALL)
5. PROFIT PROTECTION (Peak Tracking)
Jika peak_profit > $30 DAN current < 60% dari peak:
-> TUTUP (lindungi profit)
6. SMART TIME-BASED EXIT (v5: Don't Cut Winners)
Jika posisi terbuka >= 4 jam:
- profit < $5 DAN tidak growing -> TUTUP
- profit >= $5 DAN growing + ML agrees -> HOLD (extend)
Jika posisi terbuka >= 6 jam:
- profit < $10 ATAU tidak growing -> TUTUP
- profit >= $10 DAN growing -> extend ke 8 jam
Jika posisi terbuka >= 8 jam:
-> TUTUP (take profit atau max time)
```
### Smart Time-Based Exit Detail (v5 Update)
```
Jam 0 Jam 4 Jam 6 Jam 8
|------------|------------------|------------------|-----> waktu
| | |
| stuck? | profitable? | FINAL EXIT
| (no growth) | (growing?) |
| -> TUTUP | -> extend! |
| | not growing? |
| growing? | -> TUTUP |
| -> HOLD | |
v5 Perubahan dari v3:
- 4h: Cek profit GROWTH (momentum), bukan hanya profit < $5
- 6h: Bukan force close lagi — extend ke 8h jika profit > $10 + growing
- ML agreement diperhitungkan sebelum timeout
- Prinsip: jangan potong pemenang yang masih berjalan
```
### Broker Stop Loss (v3: ATR-Based Protection)
**Perubahan utama v3:** Bot sekarang mengirim **SL ke broker** (bukan SL=0 seperti sebelumnya).
Bot menggunakan **metode *Half-Kelly Criterion***:
```python
# v2 (lama): Tidak ada proteksi broker
result = mt5.send_order(sl=0, ...) # Bergantung 100% pada software
# 1. Hitung jumlah risiko
risk_amount = balance * risk_per_trade / 100
# $5.000 * 1% = $50
# v3 (baru): ATR-based broker protection
broker_sl = signal.stop_loss # SL dari SMC (ATR-based, min 1.5 ATR)
# 2. Hitung jarak SL
sl_distance = abs(entry - stop_loss)
sl_pips = sl_distance / 0.1 # XAUUSD
# Validasi jarak minimum (10 pips untuk XAUUSD)
min_sl_distance = 1.0 # $1 = 10 pips
if direction == "BUY" and current_price - broker_sl < min_sl_distance:
broker_sl = current_price - (min_sl_distance * 2) # Paksa lebih lebar
if direction == "SELL" and broker_sl - current_price < min_sl_distance:
broker_sl = current_price + (min_sl_distance * 2) # Paksa lebih lebar
# 3. Hitung lot
lot = risk_amount / (sl_pips * pip_value)
result = mt5.send_order(sl=broker_sl, ...) # SL AKTIF di broker
```
# 4. Half-Kelly (keamanan)
lot *= 0.5
**Fallback jika broker reject SL:**
```python
# Error code 10016 = SL/TP rejected
if not result.success and result.retcode == 10016:
# Fallback ke software SL (tanpa broker protection)
result = mt5.send_order(sl=0, ...) # Software tetap mengelola
```
# 5. Apply multiplier
lot *= session_multiplier # 0.5x - 1.2x
lot *= regime_multiplier # Dikurangi saat volatile
### Emergency Stop Loss (Safety Net Terakhir)
```python
calculate_emergency_sl(entry_price, lot_size, direction):
pip_value = lot_size * 10 # XAUUSD
emergency_pips = emergency_sl_usd / pip_value # $100 / pip_value
price_distance = emergency_pips * 0.01
if direction == "BUY":
sl = entry_price - price_distance
else:
sl = entry_price + price_distance
```
### Perbandingan Proteksi Lama vs Baru
```
┌─────────────────┬───────────────────────┬──────────────────────────┐
│ Skenario │ Sebelum (v2) │ Sesudah (v3) │
├─────────────────┼───────────────────────┼──────────────────────────┤
│ Weekend Gap │ Loss unlimited │ Broker SL aktif │
├─────────────────┼───────────────────────┼──────────────────────────┤
│ Flash Crash │ Bergantung software │ Broker SL aktif │
├─────────────────┼───────────────────────┼──────────────────────────┤
│ Connection Lost │ Loss unlimited │ Broker SL aktif │
├─────────────────┼───────────────────────┼──────────────────────────┤
│ Trade Stuck │ Ditahan selamanya │ Exit max 6 jam │
├─────────────────┼───────────────────────┼──────────────────────────┤
│ Reversal Lambat │ Tunggu 70% confidence │ Exit di 65% (lebih cepat)│
└─────────────────┴───────────────────────┴──────────────────────────┘
# 6. Batasi
lot = max(0.01, min(lot, 0.05))
```
---
## Circuit Breaker (RiskEngine)
## Proteksi Berlapis
### Lapis 1: *Entry Filter* (Sebelum *Trade*)
14 filter *entry* harus lolos — termasuk *session filter*, *regime check*, dan *smart risk gate*.
### Lapis 2: *Position Monitoring* (Saat *Trade* Aktif)
12 kondisi *exit* diperiksa setiap ~10 detik:
- *Smart Take Profit* (4 sub-kondisi)
- *Early Cut* (momentum negatif)
- *Trend Reversal* (ML sinyal balik)
- *Max Loss* per *trade*
- *Stall Detection*
- Batas harian
- *Weekend close*
- *Time-based exit* (4-8 jam)
- *Trailing SL* + *Breakeven*
### Lapis 3: *Broker-Level SL*
```python
# Automatic halt jika kondisi darurat
if daily_pnl_percent <= -max_daily_loss:
activate_circuit_breaker("Daily loss limit breached")
can_trade = False
# Flash crash protection
if price_move > flash_crash_threshold (2.5%):
activate_circuit_breaker("Flash crash detected")
can_trade = False
# SL dikirim ke broker sebagai proteksi darurat
result = mt5.send_order(
sl=emergency_sl, # Berbasis ATR — proteksi server-side
tp=signal.take_profit,
)
```
---
Jika koneksi internet terputus, **SL di *broker* tetap aktif**.
## Drawdown Tracking
### Daily Drawdown
```python
# Saat loss:
daily_loss += abs(profit)
total_loss += abs(profit)
consecutive_losses += 1
# Saat profit:
total_loss = max(0, total_loss - profit) # Recovery
consecutive_losses = 0 # Reset
```
### Peak Equity Drawdown
```python
# Track peak equity
if equity > peak_equity:
peak_equity = equity
# Hitung drawdown
drawdown = ((peak_equity - equity) / peak_equity) * 100
```
### Per-Position Peak Tracking
```python
# Track peak profit per posisi
peak_profits[ticket] = max(peak_profits[ticket], current_profit)
# Profit protection: tutup jika profit turun 40% dari peak
if current_profit < peak_profit * 0.6:
close_position() # Lindungi profit
```
---
## Daily Reset
### Lapis 4: *Circuit Breaker*
```python
check_new_day():
if date.today() != current_date:
# Reset semua counter harian
daily_loss = 0
daily_trades = 0
consecutive_losses = 0
mode = NORMAL (jika total_loss OK)
current_date = today
# Flash crash detection — hentikan semua trading
flash_crash_threshold = 2.5 # Pergerakan 2.5% dalam 1 menit
if move_percent > threshold:
HALT_ALL_TRADING
```
---
## Integrasi dalam Main Loop
## 7 Alasan *Exit* (*ExitReason*)
```
Main Trading Loop (candle-based + position check setiap ~10 detik)
|
v
1. check_new_day() <- Reset harian
|
v
2. get_trading_recommendation()
|-- can_trade? -> Jika False, skip
|-- mode? -> Tentukan lot limit
|
v
3. calculate_lot_size() <- Hitung lot aman
|-- Input: confidence, regime, ml_confidence
|-- Output: lot 0.01-0.02
|
v
4. Apply session_multiplier <- Sydney 0.5x, Golden 1.2x
|
v
5. can_open_position() <- Cek limit posisi (max 2)
|
v
6. execute_trade() <- Kirim order ke MT5 (v3: DENGAN broker SL)
|-- broker_sl = signal.stop_loss (ATR-based)
|-- Fallback sl=0 jika broker reject
|-- register_position() <- Track posisi baru + entry_time
|
v
7. evaluate_position() <- Monitor posisi terbuka
|-- Cek TP, ML reversal (65%), max loss, stall
|-- Cek time-based exit (4 jam / 6 jam) <- v3 BARU
|
v
8. record_trade_result() <- Catat profit/loss
|-- Update daily_loss, total_loss
|-- Cek apakah limit tercapai
```
| Kode | Deskripsi |
|------|-----------|
| `TAKE_PROFIT` | Target profit tercapai atau profit diamankan |
| `TREND_REVERSAL` | ML mendeteksi pembalikan *trend* |
| `DAILY_LIMIT` | Batas kerugian harian tercapai |
| `POSITION_LIMIT` | Batas kerugian per posisi (S/L) |
| `TOTAL_LIMIT` | Batas kerugian total tercapai |
| `WEEKEND_CLOSE` | Mendekati penutupan *weekend* |
| `MANUAL` | Penutupan manual oleh pengguna |
---
## Semua Parameter Konfigurasi
## *State* Risiko Harian
| Parameter | Nilai | Fungsi |
|-----------|-------|--------|
| `capital` | $5,000 | Modal awal |
| `max_daily_loss_percent` | 5.0% | Limit harian ($250) |
| `max_total_loss_percent` | 10.0% | Limit kumulatif ($500) |
| `max_loss_per_trade_percent` | 1.0% | Software SL ($50) |
| `emergency_sl_percent` | 2.0% | Broker SL ($100) |
| `base_lot_size` | 0.01 | Lot minimum |
| `max_lot_size` | 0.02 | Lot maximum |
| `recovery_lot_size` | 0.01 | Lot saat recovery |
| `trend_reversal_threshold` | **0.65** | ML confidence untuk tutup (v3: diturunkan dari 0.70) |
| `max_concurrent_positions` | 2 | Posisi terbuka max |
| `flash_crash_threshold` | 2.5% | Deteksi crash |
| `breakeven_pips` | 15.0 | Pindah SL ke breakeven |
| `trail_start_pips` | 25.0 | Mulai trailing stop |
| `trail_step_pips` | 10.0 | Jarak trailing |
```python
@dataclass
class RiskState:
mode: TradingMode # NORMAL/PROTECTED/RECOVERY/COOLDOWN
daily_profit: float # Total profit hari ini ($)
daily_loss: float # Total kerugian hari ini ($)
daily_trades: int # Jumlah trade hari ini
consecutive_losses: int # Kerugian berturut-turut
last_loss_amount: float # Kerugian terakhir ($)
can_trade: bool # Boleh trading atau tidak
```
*State* **di-reset setiap hari baru** (00:00 WIB) — hari baru, kesempatan baru.
---
## Sinkronisasi Backtest (backtest_live_sync.py)
## *Position Guard* (Per Posisi)
Backtest menggunakan **logika exit yang identik** dengan live trading:
Setiap posisi yang terbuka memiliki *guard* sendiri yang melacak:
```
Exit reversal: 0.65 (65% ML confidence) <- synced dengan live
Smart time-based exit (v5):
16 bars (4 jam M15) + no growth -> exit (stuck)
16 bars + growing + ML agrees -> hold (extend)
24 bars (6 jam M15) + profit<$10 -> exit
24 bars + profit>$10 + growing -> extend ke 32 bars (8 jam)
32 bars (8 jam M15) -> final exit
Perhitungan bar:
bars_since_entry = current_bar_index - entry_bar_index
16 bars * 15 menit = 4 jam
24 bars * 15 menit = 6 jam
32 bars * 15 menit = 8 jam (v5: max extended time)
```
**Kenapa penting disinkronkan?** Agar hasil backtest akurat mewakili performa live trading.
---
## Filosofi Kunci
1. **Dual-Layer SL** — ATR-based broker SL + software-managed exit (v3 update)
2. **Ultra-Conservative** — Lot 0.01-0.02 saja, tidak pernah agresif
3. **Multi-Layer Protection** — Per-trade, per-day, total limit, circuit breaker
4. **Recovery First** — Setelah loss, otomatis masuk mode defensif
5. **Profit Protection** — Jika profit sudah besar, lindungi dari drawback
6. **Smart Time-Bounded** — Tidak ada posisi "zombie", max 6-8 jam, tapi jangan potong pemenang (v5 update)
7. **Faster Reversal** — Exit lebih cepat di 65% ML confidence (v3 update)
| Properti | Keterangan |
|----------|------------|
| `entry_price` | Harga masuk |
| `peak_profit` | Profit tertinggi yang pernah dicapai |
| `profit_history` | Riwayat profit (untuk *stall detection*) |
| `reversal_warnings` | Jumlah peringatan *reversal* dari ML |
| `stall_count` | Berapa kali harga *stuck* |
| `entry_time` | Waktu masuk (untuk *time-based exit*) |
docs/arsitektur-ai/06-Session-Filter.md
+94 -255
View File
@@ -1,299 +1,138 @@
# Session Filter
# *Session Filter* — Filter Sesi Perdagangan
> **File:** `src/session_filter.py`
> **Class:** `SessionFilter`
> **Timezone:** WIB (Waktu Indonesia Barat / GMT+7)
> **Zona Waktu:** WIB (Waktu Indonesia Barat / GMT+7)
---
## Apa Itu Session Filter?
## Apa Itu *Session Filter*?
Session Filter menentukan **kapan bot boleh trading** berdasarkan sesi pasar global. Setiap sesi memiliki karakteristik berbeda — volatilitas, likuiditas, dan spread. Bot menyesuaikan perilaku berdasarkan sesi yang sedang aktif.
**Analogi:** Session Filter adalah **jadwal kerja** — bot tahu kapan harus bekerja keras, kapan santai, dan kapan istirahat.
*Session Filter* mengontrol **kapan bot boleh *trading*** berdasarkan sesi pasar global. Setiap sesi memiliki karakteristik volatilitas yang berbeda — bot memilih **sesi terbaik** untuk memaksimalkan peluang.
---
## 7 Sesi yang Didefinisikan
## Peta Sesi *Trading* (WIB)
| Sesi | Enum | Waktu (WIB) | Volatilitas | Multiplier |
|------|------|-------------|-------------|------------|
| **Sydney** | `SYDNEY` | 06:00 - 13:00 | Low | 0.5x |
| **Tokyo** | `TOKYO` | 07:00 - 16:00 | Medium | 0.7x |
| **London** | `LONDON` | 15:00 - 23:59 | High | 1.0x |
| **New York** | `NEW_YORK` | 20:00 - 23:59 | Extreme | 1.0x |
| **Tokyo-London Overlap** | `OVERLAP_TOKYO_LONDON` | 15:00 - 16:00 | High | 1.0x |
| **London-NY Overlap** | `OVERLAP_LONDON_NY` | 20:00 - 23:59 | Extreme | **1.2x** |
| **Off Hours** | `OFF_HOURS` | Diluar sesi | - | 0.0x |
```mermaid
gantt
title Sesi Trading dalam WIB (GMT+7)
dateFormat HH:mm
axisFormat %H:%M
---
section Sesi
Sydney (Low Vol) :06:00, 13:00
Tokyo (Medium Vol) :07:00, 16:00
London (High Vol) :15:00, 23:59
New York (Extreme Vol) :20:00, 23:59
## Visualisasi Timeline (WIB)
section Overlap
Tokyo-London DIBLOKIR :crit, 15:00, 16:00
London-NY GOLDEN :active, 20:00, 23:59
```
JAM WIB: 00 02 04 06 08 10 12 14 16 18 20 22 24
|---|---|---|---|---|---|---|---|---|---|---|---|---|
DANGER: [=========] <- Dead Zone (00-04)
DANGER: [===] <- Rollover (04-06)
SYDNEY: [===========] 0.5x
TOKYO: [=============] 0.7x
OVERLAP T-L: [=] 1.0x
LONDON: [===================] 1.0x
NEW YORK: [=======] 1.0x
GOLDEN: [=======] 1.2x ★
|---|---|---|---|---|---|---|---|---|---|---|---|---|
00 02 04 06 08 10 12 14 16 18 20 22 24
```
**★ GOLDEN TIME (20:00-23:59 WIB):** Waktu terbaik — likuiditas tertinggi, London & NY overlap.
---
## Zona Bahaya (Danger Zones)
| Zona | Waktu (WIB) | Alasan | Aksi |
|------|-------------|--------|------|
| **Dead Zone** | 00:00 - 04:00 | Likuiditas rendah, spread tinggi | Block trading |
| **Rollover** | 04:00 - 06:00 | Spread melebar saat rollover broker | Block trading |
---
## Logika `can_trade()` — Keputusan Utama
```
can_trade() -> (bool, str, float)
bisa? alasan multiplier
Langkah pengecekan (urut prioritas):
1. Weekend?
|-- Sabtu / Minggu -> (False, "Market tutup", 0.0)
2. Jumat >= 23:00?
|-- Ya -> (False, "Hindari gap weekend", 0.0)
3. Danger Zone?
|-- 00:00-04:00 -> (False, "Likuiditas rendah", 0.0)
|-- 04:00-06:00 -> (False, "Spread melebar", 0.0)
4. Sesi saat ini?
|-- Cek overlap dulu (prioritas tertinggi)
|-- Lalu cek sesi utama
5. allow_trading flag?
|-- False -> (False, "Tidak diizinkan", 0.0)
6. Aggressive Mode?
|-- Sydney -> (True, "SAFE MODE 0.5x", 0.5)
|-- Low volatility -> (False, "Tunggu sesi volatile", mult)
|-- High/Extreme -> (True, "Trading OK", mult)
7. Default
|-- (True, "Trading OK - {sesi}", multiplier)
section Bahaya
Dead Zone :crit, 00:00, 04:00
Rollover :crit, 04:00, 06:00
```
---
## Prioritas Deteksi Sesi
## Konfigurasi Sesi (Terkini)
| Sesi | Jam WIB | Volatilitas | *Trading* | *Lot Multiplier* | Catatan |
|------|---------|-------------|-----------|-------------------|---------|
| **Sydney** | 06:00 - 13:00 | *Low* | **Ya** | 0.5x | *Backtest* membuktikan profit $5.934 |
| **Tokyo** | 07:00 - 16:00 | *Medium* | **Ya** | 0.7x | Volume cukup |
| **Tokyo-London *Overlap*** | 15:00 - 16:00 | *High* | **Tidak** | 0.0x | #24B: *Backtest* +$345 tanpa sesi ini |
| **London** | 15:00 - 23:59 | *High* | **Ya** | 1.0x | Sesi utama Eropa |
| **London-NY *Overlap*** | 20:00 - 23:59 | *Extreme* | **Ya** | **1.2x** | **Waktu emas** — volatilitas tertinggi |
| **New York** | 20:00 - 23:59 | *Extreme* | **Ya** | 1.0x | Sesi utama AS |
| ***Off Hours*** | Lainnya | *Low* | **Tidak** | 0.0x | Di luar semua sesi |
---
## Zona Bahaya
| Zona | Jam WIB | Alasan | Aksi |
|------|---------|--------|------|
| ***Dead Zone*** | 00:00 - 04:00 | Likuiditas rendah, *spread* tinggi | **Blokir *trading*** |
| ***Rollover*** | 04:00 - 06:00 | *Spread* sangat lebar saat *rollover* | **Blokir *trading*** |
| **Jumat *Close*** | Sabtu 04:30+ | Mendekati tutup *weekend* | **Blokir *entry* baru** |
| ***Weekend*** | Sabtu - Minggu | Pasar tutup | **Blokir sepenuhnya** |
---
## Jam *Skip* Khusus (#34A)
Selain filter sesi, ada filter waktu tambahan dari optimasi *backtest*:
```python
# Overlap dicek PERTAMA (prioritas tertinggi)
1. London-NY Overlap (20:00-23:59) -> GOLDEN TIME 1.2x
2. Tokyo-London Overlap (15:00-16:00)
# Lalu sesi utama
3. London (15:00-23:59)
4. New York (20:00-23:59)
5. Tokyo (07:00-16:00)
6. Sydney (06:00-13:00)
# Terakhir
7. Off Hours (default)
# main_live.py — Filter #34A
# Skip jam 9 dan 21 WIB — backtest menambah +$356 profit
wib_hour = datetime.now(ZoneInfo("Asia/Jakarta")).hour
if wib_hour in (9, 21):
return # Jam transisi — volatilitas tidak optimal
```
---
## Dampak ke Position Sizing
Session multiplier diterapkan **setelah** kalkulasi lot dari SmartRiskManager:
```
Lot dasar dari Risk Manager: 0.02
|
v
Session multiplier:
Sydney (0.5x): 0.02 * 0.5 = 0.01
Tokyo (0.7x): 0.02 * 0.7 = 0.014 -> 0.01 (rounded)
London (1.0x): 0.02 * 1.0 = 0.02
Golden (1.2x): 0.02 * 1.2 = 0.024 -> 0.02 (capped)
|
v
Final lot (min 0.01, max 0.02)
```
---
## Weekend & Friday Handling
### Weekend
```
Sabtu (weekday=5): Market tutup -> tidak trading
Minggu (weekday=6): Market tutup -> tidak trading
```
### Friday Close
```
Jumat >= 23:00 WIB:
-> Block semua trade baru
-> Alasan: Hindari gap weekend (harga bisa gap besar saat buka Senin)
```
---
## News Event Monitoring
### Event yang Dipantau
| Event | Waktu (WIB) | Buffer Sebelum | Buffer Sesudah |
|-------|-------------|---------------|----------------|
| **NFP** (Non-Farm Payroll) | 19:30 | 15 menit | 30 menit |
| **FOMC** (Fed Decision) | 01:00 | 15 menit | 45 menit |
| **CPI** (Inflation) | 19:30 | 15 menit | 30 menit |
### Kebijakan News: MONITORING ONLY (Tidak Blocking)
```
Backtest menunjukkan:
- Win rate saat news: 62.1%
- Win rate normal: 64.9%
- Selisih kecil, tapi BLOCKING news KEHILANGAN $178 profit
Keputusan: ML model sudah cukup menangani volatilitas news.
News hanya di-LOG, TIDAK memblokir trading.
```
---
## Aggressive Mode
Bot default menggunakan `aggressive_mode=True`:
## Cara Kerja
```python
create_wib_session_filter(aggressive=True)
```
def can_trade(self) -> Tuple[bool, str, float]:
"""
Returns:
can_trade: Boleh trading atau tidak
reason: Alasan dalam bahasa Indonesia
multiplier: Pengali lot size (0.0 - 1.2)
"""
# 1. Cek weekend
if self.is_weekend():
return False, "Market tutup (weekend)", 0.0
### Efek Aggressive Mode
# 2. Cek Friday close
if self.is_friday_close():
return False, "Mendekati penutupan Jumat", 0.0
| Sesi | Tanpa Aggressive | Dengan Aggressive |
|------|-----------------|-------------------|
| Sydney | Block | **Allow** (0.5x, proven profitable) |
| Tokyo | Allow | Block (volatilitas kurang) |
| London | Allow | Allow |
| New York | Allow | Allow |
| Golden | Allow | Allow (boost 1.2x) |
# 3. Cek zona bahaya
if self.is_danger_zone():
return False, "Zona bahaya: spread melebar", 0.0
**Alasan Sydney diizinkan:** Backtest menunjukkan win rate 62% dan profit $5,934 di sesi Sydney.
# 4. Cek sesi saat ini
session, config = self.get_current_session()
if not config.allow_trading:
return False, f"Trading tidak diizinkan saat {config.name}", 0.0
---
## Golden Time (London-NY Overlap)
```
Waktu: 20:00 - 23:59 WIB
Multiplier: 1.2x (BOOSTED)
Volatilitas: Extreme
Kenapa spesial?
- London dan New York sama-sama aktif
- Likuiditas TERTINGGI sepanjang hari
- Pergerakan harga paling signifikan
- Volume trading terbesar
Aturan tambahan di main_live.py:
- Require ML + SMC alignment (keduanya harus setuju)
- Lot boleh lebih besar (1.2x multiplier)
return True, f"Trading OK - {config.name}", config.position_size_multiplier
```
---
## Integrasi dalam Main Loop
## Pengaruh pada *Position Sizing*
*Session multiplier* langsung mengubah ukuran *lot*:
```python
# 1. Inisialisasi
self.session_filter = create_wib_session_filter(aggressive=True)
# 2. Cek setiap loop
session_ok, session_reason, session_multiplier = self.session_filter.can_trade()
if not session_ok:
# Log setiap 5 menit
logger.info(f"Session: {session_reason}")
next = self.session_filter.get_next_trading_window()
logger.info(f"Next: {next['session']} in {next['hours_until']} hours")
return # Skip, tidak trading
# 3. Simpan multiplier untuk lot sizing
self._current_session_multiplier = session_multiplier
# 4. Apply ke lot size (setelah risk calculation)
# main_live.py
safe_lot = smart_risk.calculate_lot_size(...)
safe_lot = max(0.01, safe_lot * session_multiplier)
# Contoh (akun $5.000):
# London-NY Overlap: 0.02 * 1.2 = 0.024 → 0.02 lot (setelah rounding)
# Sydney: 0.02 * 0.5 = 0.010 → 0.01 lot (half size)
# Tokyo: 0.02 * 0.7 = 0.014 → 0.01 lot
```
---
## Status Report
## Waktu Berita Dampak Tinggi
```python
get_status_report() -> {
"current_time_wib": "2026-02-06 20:15:00",
"current_session": "London-NY Overlap",
"volatility": "extreme",
"can_trade": True,
"reason": "Trading OK - GOLDEN TIME (1.2x)",
"position_multiplier": 1.2,
"is_weekend": False,
"is_friday_close": False,
"is_danger_zone": False,
}
```
Bot juga memiliki daftar waktu berita ekonomi penting:
---
| Berita | Jam WIB | *Buffer* Sebelum | *Buffer* Setelah |
|--------|---------|------------------|------------------|
| NFP (*Non-Farm Payrolls*) | 19:30 | 15 menit | 30 menit |
| FOMC (*Federal Reserve*) | 01:00 | 15 menit | 45 menit |
| CPI (*Consumer Price Index*) | 19:30 | 15 menit | 30 menit |
## Contoh Skenario
**Skenario 1: Golden Time**
```
Waktu: 21:30 WIB (Rabu)
Sesi: London-NY Overlap
-> can_trade = True
-> multiplier = 1.2x
-> Lot 0.02 * 1.2 = 0.024 -> cap 0.02
-> Trading optimal!
```
**Skenario 2: Sydney pagi**
```
Waktu: 08:00 WIB (Selasa)
Sesi: Sydney
-> can_trade = True (aggressive mode)
-> multiplier = 0.5x
-> Lot 0.02 * 0.5 = 0.01
-> SAFE MODE: lot minimum
```
**Skenario 3: Dead zone**
```
Waktu: 02:30 WIB (Kamis)
Sesi: Off Hours (Danger Zone)
-> can_trade = False
-> Alasan: "Likuiditas rendah, spread tinggi"
-> Bot istirahat, tunggu sesi berikutnya
```
**Skenario 4: Jumat malam**
```
Waktu: 23:15 WIB (Jumat)
-> can_trade = False
-> Alasan: "Hindari gap weekend"
-> Tidak buka posisi baru
```
> **Catatan:** *News Agent* saat ini **nonaktif** (`main_live.py` baris 64). Filter berita direncanakan untuk diaktifkan kembali di versi mendatang.
docs/arsitektur-ai/07-Stop-Loss.md
+45 -21
View File
@@ -1,18 +1,42 @@
# Stop Loss (S/L) — Sistem Proteksi Berlapis
# *Stop Loss* (S/L) — Sistem Proteksi Berlapis
> **File terkait:** `src/smc_polars.py`, `main_live.py`, `src/smart_risk_manager.py`
---
## Apa Itu Stop Loss di Bot Ini?
```mermaid
block-beta
columns 1
block:layer1["Layer 1 — SMC ATR-Based Stop Loss"]:1
A["Dikirim ke broker sebagai SL aktif\n1.5 ATR dari entry (~$15-$30)"]
end
block:layer2["Layer 2 — Software Smart Exit"]:1
B["Bot monitor posisi & tutup otomatis\nDinamis berdasarkan konteks ($25-$50)"]
end
block:layer3["Layer 3 — Emergency Broker SL"]:1
C["Safety net 2% modal\nAktif jika software gagal ($100)"]
end
block:layer4["Layer 4 — Circuit Breaker"]:1
D["Halt total semua trading\nFlash crash 2.5% / daily limit -5%"]
end
Stop Loss bukan hanya satu angka — ini adalah **sistem proteksi 4 lapis** yang bekerja bersamaan. Jika satu layer gagal, layer berikutnya siap melindungi.
**Analogi:** SL di bot ini seperti sistem keamanan gedung — ada CCTV (software monitoring), security (broker SL), alarm kebakaran (emergency SL), dan sprinkler otomatis (circuit breaker).
style layer1 fill:#2d7d46,color:#fff
style layer2 fill:#2d6a9f,color:#fff
style layer3 fill:#b8860b,color:#fff
style layer4 fill:#a82020,color:#fff
```
---
## 4 Layer Stop Loss
## Apa Itu *Stop Loss* di Bot Ini?
*Stop Loss* bukan hanya satu angka — ini adalah **sistem proteksi 4 lapis** yang bekerja bersamaan. Jika satu layer gagal, layer berikutnya siap melindungi.
**Analogi:** SL di bot ini seperti sistem keamanan gedung — ada CCTV (software monitoring), security (broker SL), alarm kebakaran (*Emergency* SL), dan sprinkler otomatis (*Circuit Breaker*).
---
## 4 Layer *Stop Loss*
```
Layer 1: SMC ATR-Based SL <- Dikirim ke broker sebagai SL aktif
@@ -38,7 +62,7 @@ Semakin jauh = semakin jarang tercapai (backup)
---
## Layer 1: SMC ATR-Based Stop Loss
## Layer 1: SMC *ATR*-Based *Stop Loss*
**Sumber:** `smc_polars.py` (Lines 631-652, 694-702)
**Dikirim ke:** Broker MT5 sebagai SL order aktif
@@ -65,7 +89,7 @@ SL = MAX(swing_sl, atr_sl) # $4965.00 (pilih yang LEBIH JAUH)
```
Sebelum (v2): SL = swing_low ATAU entry * 0.995
-> Bisa sangat dekat, gampang kena whipsaw
-> Bisa sangat dekat, gampang kena *whipsaw*
Sesudah (v3): SL = MIN(swing_low, entry - 1.5*ATR)
-> Selalu minimal 1.5 ATR dari entry
@@ -74,7 +98,7 @@ Sesudah (v3): SL = MIN(swing_low, entry - 1.5*ATR)
---
## Layer 2: Software Smart Exit
## Layer 2: Software *Smart Exit*
**Sumber:** `smart_risk_manager.py` (Lines 559-724)
**Mekanisme:** Bot monitor posisi setiap detik dan tutup otomatis
@@ -106,7 +130,7 @@ Trigger exit jika:
```
Hard SL (broker):
- Kaku, tidak bisa diubah
- Bisa kena whipsaw lalu harga balik
- Bisa kena *whipsaw* lalu harga balik
- Tidak bisa mempertimbangkan konteks
Software SL (bot):
@@ -118,7 +142,7 @@ Software SL (bot):
---
## Layer 3: Emergency Broker Stop Loss
## Layer 3: *Emergency* Broker *Stop Loss*
**Sumber:** `smart_risk_manager.py` (Lines 305-346)
**Fungsi:** Jaring pengaman TERAKHIR jika software gagal (disconnect, crash, dll)
@@ -140,17 +164,17 @@ BUY: SL = entry - $10.00 = $4940.00
SELL: SL = entry + $10.00 = $4960.00
```
### Kapan Emergency SL Tercapai?
### Kapan *Emergency* SL Tercapai?
Seharusnya **tidak pernah** — software SL ($50) akan menutup jauh sebelum emergency SL ($100). Emergency SL hanya tercapai jika:
Seharusnya **tidak pernah** — software SL ($50) akan menutup jauh sebelum *Emergency* SL ($100). *Emergency* SL hanya tercapai jika:
- Bot crash / disconnect
- Server bermasalah
- Internet putus
- Harga gap melewati semua level
- Harga *gap* melewati semua level
---
## Layer 4: Circuit Breaker
## Layer 4: *Circuit Breaker*
**Sumber:** `risk_engine.py` (Lines 143-151)
**Fungsi:** Halt trading total saat kondisi darurat
@@ -212,10 +236,10 @@ if not result.success and retcode == 10016:
| Layer | Sumber | Jarak dari Entry | Max Loss | Kondisi Trigger |
|-------|--------|-----------------|----------|-----------------|
| **1. SMC ATR** | Broker SL aktif | 1.5 ATR (~$12-15) | ~$15-30 | Harga hit SL level |
| **1. SMC *ATR*** | Broker SL aktif | 1.5 *ATR* (~$12-15) | ~$15-30 | Harga hit SL level |
| **2. Software** | Bot monitoring | Dinamis | $25-50 | Loss threshold + konteks |
| **3. Emergency** | Broker safety net | 2% modal ($10) | $100 | Software gagal |
| **4. Circuit** | Halt total | Semua posisi | Unlimited cap | Flash crash / daily limit |
| **3. *Emergency*** | Broker safety net | 2% modal ($10) | $100 | Software gagal |
| **4. *Circuit Breaker*** | Halt total | Semua posisi | Unlimited cap | *Flash crash* / daily limit |
---
@@ -240,16 +264,16 @@ Entry BUY @ $4950, SL broker @ $4937
-> Loss: ~$13 (bukan unlimited!)
```
### Skenario 3: Weekend Gap
### Skenario 3: Weekend *Gap*
```
Jumat: Entry BUY @ $4950, SL broker @ $4937
-> Senin buka gap di $4910 (melewati SL)
-> Senin buka *gap* di $4910 (melewati SL)
-> Broker eksekusi SL di harga terbaik ~$4910
-> Loss: ~$40 (lebih dari SL tapi terproteksi)
```
### Skenario 4: Flash Crash (Tanpa Broker SL Fallback)
### Skenario 4: *Flash Crash* (Tanpa Broker SL Fallback)
```
Entry BUY @ $4950, sl=0 (broker reject)
docs/arsitektur-ai/08-Take-Profit.md
+48 -22
View File
@@ -1,18 +1,44 @@
# Take Profit (T/P) — Sistem Pengambilan Profit Cerdas
# *Take Profit* (T/P) — Sistem Pengambilan Profit Cerdas
> **File terkait:** `src/smc_polars.py`, `main_live.py`, `src/smart_risk_manager.py`
---
## Apa Itu Take Profit di Bot Ini?
## Flowchart Prioritas *Take Profit*
Take Profit bukan hanya satu target harga — ini adalah **sistem multi-layer** yang secara cerdas memutuskan kapan mengambil profit berdasarkan momentum, probabilitas, dan peak tracking.
```mermaid
flowchart TD
A["Evaluasi Posisi Terbuka"] --> B{"profit >= $40?"}
B -- Ya --> B1["Layer 1: Hard TP\nTutup langsung"]
B -- Tidak --> C{"profit >= $25\nDAN momentum < -30?"}
C -- Ya --> C1["Layer 2: Momentum TP\nAmankan profit"]
C -- Tidak --> D{"peak > $30\nDAN current < 60% peak?"}
D -- Ya --> D1["Layer 3: Peak Protection\nKunci sisa profit"]
D -- Tidak --> E{"profit >= $20\nDAN TP probability < 25%?"}
E -- Ya --> E1["Layer 4: Probability TP\nAmbil sekarang"]
E -- Tidak --> F{"profit $5-15\nDAN ML reversal\nDAN momentum < -50?"}
F -- Ya --> F1["Layer 5: Early Exit\nProfit kecil > loss"]
F -- Tidak --> G["Layer 6: Broker TP\nHarga hit level otomatis"]
**Analogi:** TP di bot ini seperti **pemanen buah pintar** — tahu kapan buah sudah matang (hard TP), kapan cuaca akan buruk (momentum drop), dan kapan panen sebelum busuk (peak protection).
style B1 fill:#16a34a,color:#fff
style C1 fill:#2563eb,color:#fff
style D1 fill:#7c3aed,color:#fff
style E1 fill:#d97706,color:#fff
style F1 fill:#dc2626,color:#fff
style G fill:#64748b,color:#fff
```
---
## Layer Take Profit
## Apa Itu *Take Profit* di Bot Ini?
*Take Profit* bukan hanya satu target harga — ini adalah **sistem multi-layer** yang secara cerdas memutuskan kapan mengambil profit berdasarkan *momentum*, *probability*, dan *peak protection*.
**Analogi:** TP di bot ini seperti **pemanen buah pintar** — tahu kapan buah sudah matang (*Hard TP*), kapan cuaca akan buruk (*momentum* drop), dan kapan panen sebelum busuk (*Peak Protection*).
---
## Layer *Take Profit*
```
Layer 1: Broker TP <- Target harga dikirim ke broker (SMC-generated)
@@ -77,7 +103,7 @@ Jika harga mencapai TP level, broker otomatis menutup posisi — tidak perlu bot
---
## Layer 2: Hard Take Profit ($40)
## Layer 2: *Hard Take Profit* ($40)
**Sumber:** `smart_risk_manager.py` (Lines 595-599)
@@ -92,7 +118,7 @@ if current_profit >= 40:
---
## Layer 3: Momentum-Based TP ($25+)
## Layer 3: *Momentum*-Based TP ($25+)
**Sumber:** `smart_risk_manager.py` (Lines 601-603)
@@ -103,7 +129,7 @@ if current_profit >= 25 and momentum < -30:
"[SECURE] Securing $25.00 (momentum dropping)"
```
### Bagaimana Momentum Dihitung
### Bagaimana *Momentum* Dihitung
```python
# PositionGuard.calculate_momentum() (Lines 113-131)
@@ -136,7 +162,7 @@ Profit ($)
---
## Layer 4: Peak Protection ($30+ peak)
## Layer 4: *Peak Protection* ($30+ peak)
**Sumber:** `smart_risk_manager.py` (Lines 605-607)
@@ -174,7 +200,7 @@ Profit ($)
---
## Layer 5: Probability-Based TP ($20+)
## Layer 5: *Probability*-Based TP ($20+)
**Sumber:** `smart_risk_manager.py` (Lines 609-611)
@@ -185,7 +211,7 @@ if tp_probability < 25 and current_profit >= 20:
"[PROB] Taking profit $20 (TP prob: 15%)"
```
### Cara Hitung TP Probability
### Cara Hitung TP *Probability*
```python
# PositionGuard.get_tp_probability() (Lines 133-168)
@@ -196,10 +222,10 @@ Factor 1: Progress ke TP (0-40 poin)
-> Makin dekat ke TP = skor tinggi
Factor 2: Momentum (0-30 poin)
-> Momentum positif = skor tinggi
-> *Momentum* positif = skor tinggi
Factor 3: ML Confidence Trend (0-20 poin)
-> ML confidence naik = skor tinggi
Factor 3: ML *Confidence* Trend (0-20 poin)
-> ML *confidence* naik = skor tinggi
Factor 4: Time Penalty (0-10 poin DIKURANGI)
-> 2 poin per jam (makin lama = makin rendah)
@@ -209,7 +235,7 @@ probability = factor1 + factor2 + factor3 - time_penalty
---
## Layer 6: Early Exit (Profit Kecil + Reversal)
## Layer 6: *Early Exit* (Profit Kecil + Reversal)
**Sumber:** `smart_risk_manager.py` (Lines 617-627)
@@ -261,7 +287,7 @@ Entry BUY @ $4950, TP broker @ $4990
-> Tutup sebelum broker TP level
```
### Skenario 3: Momentum Drop
### Skenario 3: *Momentum* Drop
```
Entry BUY @ $4950
@@ -271,7 +297,7 @@ Entry BUY @ $4950
-> MOMENTUM TP: amankan $25
```
### Skenario 4: Peak Protection
### Skenario 4: *Peak Protection*
```
Entry BUY @ $4950
@@ -289,8 +315,8 @@ Entry BUY @ $4950
| Layer | Trigger | Profit Min | Kondisi Tambahan |
|-------|---------|-----------|------------------|
| **1. Broker TP** | Harga hit level | - | Otomatis, independen |
| **2. Hard TP** | profit >= $40 | $40 | Tidak ada |
| **3. Momentum TP** | profit >= $25 | $25 | momentum < -30 |
| **4. Peak Protection** | peak > $30 | ~$18+ | current < 60% peak |
| **5. Probability TP** | profit >= $20 | $20 | TP probability < 25% |
| **6. Early Exit** | profit $5-15 | $5 | ML reversal + momentum < -50 |
| **2. *Hard TP*** | profit >= $40 | $40 | Tidak ada |
| **3. *Momentum* TP** | profit >= $25 | $25 | *momentum* < -30 |
| **4. *Peak Protection*** | peak > $30 | ~$18+ | current < 60% peak |
| **5. *Probability* TP** | profit >= $20 | $20 | TP *probability* < 25% |
| **6. *Early Exit*** | profit $5-15 | $5 | ML reversal + *momentum* < -50 |
docs/arsitektur-ai/09-Entry-Trade.md
+179 -280
View File
@@ -1,276 +1,210 @@
# Entry Trade — Proses Masuk Posisi
# *Entry Trade* — Proses Masuk Posisi
> **File utama:** `main_live.py`
> **File pendukung:** `src/smc_polars.py`, `src/ml_model.py`, `src/smart_risk_manager.py`, `src/session_filter.py`
---
## Apa Itu Entry Trade?
## Apa Itu *Entry Trade*?
Entry Trade adalah keseluruhan proses dari **mendeteksi peluang** hingga **mengirim order ke broker**. Bot menggunakan **10+ filter** yang harus SEMUA lolos sebelum satu trade dieksekusi.
*Entry Trade* adalah keseluruhan proses dari **mendeteksi peluang** hingga **mengirim *order* ke *broker***. Bot menggunakan **14 filter** yang harus **SEMUA lolos** sebelum satu *trade* dieksekusi.
**Analogi:** Entry Trade seperti **proses boarding pesawat** — harus punya tiket (signal), passport valid (confirmation), lulus security check (risk), tepat waktu (session), dan gate terbuka (position limit).
**Analogi:** *Entry Trade* seperti **proses *boarding* pesawat** — harus punya tiket (*signal*), *passport* valid (*confirmation*), lulus *security check* (risiko), tepat waktu (sesi), dan *gate* terbuka (*position limit*).
---
## Checklist Entry (Semua Harus PASS)
## Daftar *Checklist Entry* (Semua Harus PASS)
```
1. [SESSION] Session filter izinkan trading?
2. [RISK MODE] Trading mode bukan STOPPED?
3. [SMC SIGNAL] Ada signal dari SMC Analyzer?
4. [ML CONFIRM] XGBoost confidence >= 50%?
5. [ML AGREE] ML tidak strongly disagree (>65% berlawanan)?
6. [QUALITY] Market quality bukan AVOID/CRISIS?
7. [CONFIRM] Signal konsisten 2 bar berturut?
8. [PULLBACK] Bukan sedang pullback/retrace?
9. [COOLDOWN] Sudah 5 menit sejak trade terakhir?
10. [POS LIMIT] Posisi terbuka < 2?
11. [LOT SIZE] Lot > 0 setelah semua adjustment?
| # | Filter | Keterangan | Status |
|---|--------|------------|--------|
| 1 | ***Flash Crash Guard*** | Apakah ada pergerakan harga ekstrem? | **Aktif** |
| 2 | ***Regime Filter*** | Apakah *regime* HMM bukan SLEEP? | **Aktif** |
| 3 | ***Risk Check*** | Apakah `risk_metrics.can_trade` = `true`? | **Aktif** |
| 4 | ***Session Filter*** | Apakah sesi perdagangan mengizinkan *trading*? | **Aktif** |
| 5 | ***SMC Signal*** | Apakah ada sinyal valid dari SMC *Analyzer*? | **Aktif** |
| 6 | ***Signal Combination*** | Apakah kombinasi SMC + ML menghasilkan sinyal akhir? | **Aktif** |
| 7 | **H1 *Bias* (#31B)** | Apakah *bias* H1 EMA20 sejalan dengan sinyal? | **Aktif** |
| 8 | **Filter Waktu (#34A)** | Apakah bukan jam 9 atau 21 WIB? | **Aktif** |
| 9 | ***Trade Cooldown*** | Sudah 5 menit sejak *trade* terakhir? | **Aktif** |
| 10 | ***Pullback Filter*** | Apakah bukan sedang *pullback/retrace*? | **Nonaktif** |
| 11 | ***Smart Risk Gate*** | Mode *trading* bukan STOPPED/COOLDOWN? | **Aktif** |
| 12 | **Kalkulasi *Lot*** | Apakah *lot size* > 0 setelah semua *adjustment*? | **Aktif** |
| 13 | ***Spread* Validasi** | Apakah *spread* tidak terlalu lebar? | **Aktif** |
| 14 | **Batas Posisi** | Posisi terbuka < 2? | **Aktif** |
SEMUA PASS -> Execute Trade
SATU GAGAL -> Skip, tunggu loop berikutnya
```
> **Semua PASS** → Eksekusi *Trade*
> **Satu GAGAL** → *Skip*, tunggu *loop* berikutnya
---
## Step-by-Step Flow
## *Step-by-Step Flow*
### Step 1: Session Filter
### Filter 1: *Flash Crash Guard*
```python
# main_live.py
is_flash, move_pct = self.flash_crash.detect(df.tail(5))
if is_flash:
return # Pergerakan harga ekstrem terdeteksi
```
**Bisa *block*:** Pergerakan harga > 2.5% dalam 1 menit (*flash crash threshold* dari `config.py`).
---
### Filter 2: *Regime Filter*
```python
regime_sleep = regime_state and regime_state.recommendation == "SLEEP"
if regime_sleep:
return # HMM mendeteksi kondisi krisis
```
**Bisa *block*:** *Regime* HIGH_VOLATILITY / CRISIS — pasar terlalu bergejolak.
---
### Filter 3: *Risk Check*
```python
if not risk_metrics.can_trade:
return # Risiko di luar batas
```
---
### Filter 4: *Session Filter*
```python
# main_live.py Lines 472-483
session_ok, session_reason, session_multiplier = self.session_filter.can_trade()
if not session_ok:
return # Skip — bukan waktu trading
# Simpan multiplier untuk lot sizing nanti
self._current_session_multiplier = session_multiplier
return # Bukan waktu trading
```
**Bisa block:** Weekend, Friday >23:00, danger zone (00:00-06:00), low volatility session.
**Bisa *block*:** *Weekend*, Jumat > 23:00, zona bahaya (00:00-06:00), sesi *low volatility*.
**Tokyo-London *overlap*** (15:00-16:00 WIB) **diblokir** — hasil optimasi *backtest* #24B.
---
### Step 2: Risk Mode Check
### Filter 5: *SMC Signal*
```python
# main_live.py Lines 537-542
risk_rec = self.smart_risk.get_trading_recommendation()
if not risk_rec["can_trade"]:
return # STOPPED mode — daily/total limit tercapai
```
**Bisa block:** Mode STOPPED (daily loss >= $250, total loss >= $500).
---
### Step 3: SMC Signal Generation
```python
# main_live.py Lines 498-499
smc_signal = self.smc.generate_signal(df)
if smc_signal is None:
return # Tidak ada setup SMC yang valid
```
**SMC membutuhkan:**
- Market structure (bullish/bearish) ATAU BOS/CHoCH
- DAN (FVG ATAU Order Block)
- Minimum 2:1 risk/reward
- Struktur pasar (*bullish/bearish*) ATAU BOS/CHoCH
- DAN (FVG ATAU *Order Block*)
- Minimum 2:1 *risk/reward*
**Output:** Entry price, SL, TP, confidence (55-85%), reason.
**Output:** *Entry price*, SL, TP, *confidence* (55-85%), alasan.
---
### Step 4: ML Confidence Check
### Filter 6: *Signal Combination*
```python
# main_live.py Lines 419-425
ml_prediction = self.ml_model.predict(df, feature_cols)
# Lines 664-669
if ml_prediction.confidence < 0.50:
return # ML terlalu tidak yakin
final_signal = self._combine_signals(smc_signal, ml_prediction, regime_state)
if final_signal is None:
return # Sinyal terfilter
```
Menggabungkan **SMC + ML + *Regime*** menjadi satu sinyal akhir. ML harus *agree* atau minimal tidak *strongly disagree* (> 65% *confidence* berlawanan).
---
### Step 5: ML Agreement Check
### Filter 7: H1 *Bias* (#31B)
```python
# main_live.py Lines 676-684
# Jika SMC bilang BUY tapi ML bilang SELL dengan confidence > 65%:
if smc_signal.signal_type == "BUY":
if ml_prediction.signal == "SELL" and ml_prediction.confidence > 0.65:
return # ML strongly disagrees — VETO
if smc_signal.signal_type == "SELL":
if ml_prediction.signal == "BUY" and ml_prediction.confidence > 0.65:
return # ML strongly disagrees — VETO
# Backtest #31B: H1 EMA20 filter menambah +$345 profit
if h1_bias == "BULLISH" and final_signal.signal_type == "SELL":
return # BUY signal vs H1 bullish = blokir
if h1_bias == "BEARISH" and final_signal.signal_type == "BUY":
return # SELL signal vs H1 bearish = blokir
if h1_bias == "NEUTRAL":
return # Tidak ada bias jelas = blokir
```
**Tujuan:** Hanya masuk posisi yang sejalan dengan *trend* H1.
---
### Step 6: Dynamic Market Quality
### Filter 8: Filter Waktu (#34A)
```python
# main_live.py Lines 618-657
# Analisis kualitas pasar berdasarkan:
# - Session (London/NY = tinggi, Sydney = rendah)
# - Regime (low vol = bagus, crisis = block)
# - Volatility (medium = ideal)
# - Trend strength
# - SMC confluence
# - ML signal alignment
quality_score = analyze_market_quality(...)
# EXCELLENT (80+), GOOD (60+), MODERATE (40+), POOR (20+), AVOID (<20), CRISIS
if quality == "AVOID" or quality == "CRISIS":
return # Pasar tidak layak untuk trading
# Backtest #34A: skip jam 9 dan 21 WIB menambah +$356 profit
wib_hour = datetime.now(ZoneInfo("Asia/Jakarta")).hour
if wib_hour in (9, 21):
return # Jam transisi — volatilitas tidak optimal
```
**Tujuan:** Menghindari jam transisi sesi yang berpotensi *whipsaw*.
---
### Step 7: Signal Confirmation (2 Bar Berturut)
### Filter 9: *Trade Cooldown*
```python
# main_live.py Lines 686-709
signal_key = f"{smc_signal.signal_type}_{smc_signal.entry_price:.0f}"
if signal_key in self._signal_persistence:
self._signal_persistence[signal_key] += 1
else:
self._signal_persistence[signal_key] = 1
if self._signal_persistence[signal_key] < 2:
return # Belum dikonfirmasi — tunggu 1 loop lagi
# Signal sudah muncul 2x berturut -> CONFIRMED
```
**Tujuan:** Mencegah whipsaw — signal yang hanya muncul 1 detik kemungkinan noise.
---
### Step 8: Pullback Filter
```python
# main_live.py Lines 742-871
can_enter, pullback_reason = self._check_pullback_filter(df, signal.signal_type)
if not can_enter:
return # Sedang pullback, tunggu momentum selaras
```
**v5: Threshold sekarang ATR-based (bukan hardcoded)**
```
ATR diambil dari data (default $12 untuk XAUUSD)
bounce_threshold = ATR × 0.15 # ~$1.80 (sebelumnya: $2.00 fixed)
consolidation_threshold = ATR × 0.10 # ~$1.20 (sebelumnya: $1.50 fixed)
Kenapa ATR-based?
- Threshold menyesuaikan volatilitas pasar saat ini
- Saat volatilitas tinggi (ATR=$20): bounce=$3, consolidation=$2
- Saat volatilitas rendah (ATR=$8): bounce=$1.2, consolidation=$0.8
- Lebih akurat daripada threshold tetap
```
**Untuk signal BUY, block jika:**
- Harga turun > bounce_threshold (15% ATR) dalam 3 candle terakhir
- MACD bearish + harga turun
- Harga jauh di bawah EMA9 + terus turun
**Untuk signal SELL, block jika:**
- Harga naik > bounce_threshold (15% ATR) dalam 3 candle terakhir
- MACD bullish + harga naik
- Harga jauh di atas EMA9 + terus naik
**Komponen yang dicek:**
```
1. Short-term Momentum (3 candle terakhir)
-> Arah pergerakan harga terkini
-> Block jika bounce > 15% ATR (v5: dinamis)
2. MACD Histogram
-> Rising = bullish momentum
-> Falling = bearish momentum
3. Harga vs EMA9
-> Di atas = bullish bias
-> Di bawah = bearish bias
4. RSI Extreme
-> RSI > 80 = overbought (block BUY)
-> RSI < 20 = oversold (block SELL)
5. Consolidation Check
-> Jika movement < 10% ATR = consolidation → ALLOW
-> v5: dinamis, bukan fixed $1.5
```
---
### Step 9: Trade Cooldown
```python
# main_live.py Lines 520-524
trade_cooldown = 300 # 5 menit
if last_trade_time:
elapsed = (now - last_trade_time).total_seconds()
if elapsed < trade_cooldown:
return # Tunggu cooldown selesai
if last_trade_time and (now - last_trade_time).total_seconds() < 300:
return # Tunggu cooldown selesai
```
**Tujuan:** Mencegah overtrading — minimal 5 menit antar trade.
**Tujuan:** Mencegah *overtrading* — minimal 5 menit antar *trade*.
---
### Step 10: Position Limit
### Filter 10: *Pullback Filter* (NONAKTIF)
```python
# main_live.py Lines 588-592
can_open, limit_reason = self.smart_risk.can_open_position()
if not can_open:
return # Sudah 2 posisi terbuka (max)
# DISABLED — mode SMC-only
# Struktur SMC sudah memvalidasi zona entry
```
> Filter ini dinonaktifkan karena analisis SMC sudah mencakup validasi *pullback* dalam logika *Order Block* dan FVG.
---
### Step 11: Lot Size Calculation
### Filter 11: *Smart Risk Gate*
```python
# main_live.py Lines 544-560
safe_lot = self.smart_risk.calculate_lot_size(
entry_price=signal.entry_price,
confidence=signal.confidence, # SMC confidence
regime=regime_name, # HMM regime
ml_confidence=ml_prediction.confidence, # ML confidence
)
risk_rec = self.smart_risk.get_trading_recommendation()
if not risk_rec["can_trade"]:
return # Mode STOPPED/COOLDOWN
```
# Apply session multiplier
**4 mode** *Smart Risk*: NORMAL → PROTECTED → RECOVERY → COOLDOWN/STOPPED.
---
### Filter 12-14: *Lot*, *Spread*, dan Batas Posisi
```python
# Kalkulasi lot
safe_lot = self.smart_risk.calculate_lot_size(...)
safe_lot = max(0.01, safe_lot * session_multiplier)
if safe_lot <= 0:
return # Lot 0 = tidak boleh trade
# Validasi spread
if spread > max_allowed:
return # Spread terlalu lebar
# Batas posisi (max 2)
can_open, limit_reason = self.smart_risk.can_open_position()
if not can_open:
return # Sudah 2 posisi terbuka
```
---
## Eksekusi Order
## Eksekusi *Order*
Setelah semua 11 filter lolos:
Setelah semua **14 filter** lolos:
```python
# main_live.py Lines 985-1008
# Step A: Ambil harga real-time
tick = mt5.get_tick(symbol)
current_price = tick.ask if BUY else tick.bid
@@ -284,128 +218,93 @@ if jarak_terlalu_dekat:
result = mt5.send_order(
symbol="XAUUSD",
order_type="BUY" / "SELL",
volume=0.01 - 0.02, # Lot dari risk calculation
sl=broker_sl, # ATR-based SL (v3)
tp=signal.take_profit, # SMC TP (ATR-capped)
magic=123456, # ID bot
volume=0.01 - 0.05,
sl=broker_sl, # SL berbasis ATR
tp=signal.take_profit, # TP dari SMC (ATR-capped)
magic=123456,
comment="AI Safe v3",
)
# Step D: Fallback jika broker reject SL
if gagal dan error 10016:
result = mt5.send_order(sl=0, ...) # Tanpa broker SL
result = mt5.send_order(sl=0, ...)
# Step E: Slippage Validation (v5 BARU)
# Step E: Validasi slippage
if result.success:
actual_price = result.price
slippage = abs(actual_price - signal.entry_price)
max_slippage = signal.entry_price * 0.0015 # 0.15% dari harga
slippage = abs(result.price - signal.entry_price)
max_slippage = signal.entry_price * 0.0015 # 0.15%
if slippage > max_slippage:
log WARNING "HIGH SLIPPAGE" # Catat slippage tinggi
# Gunakan harga AKTUAL untuk tracking, bukan harga expected
log WARNING "HIGH SLIPPAGE"
# Step F: Partial Fill Check (v5 BARU)
filled_volume = result.volume
if filled_volume < requested_volume:
log WARNING "PARTIAL FILL"
# Update lot_size ke volume yang sebenarnya terisi
position.lot_size = filled_volume
# Step G: Register posisi (gunakan nilai AKTUAL)
# Step F: Register posisi (gunakan nilai AKTUAL)
smart_risk.register_position(
ticket=result.order_id,
entry_price=actual_price, # v5: harga aktual (bukan expected)
lot_size=filled_volume, # v5: volume aktual (bukan requested)
entry_price=result.price, # Harga aktual
lot_size=result.volume, # Volume aktual
direction=signal.signal_type,
)
```
### Slippage & Partial Fill (v5 Detail)
```
SLIPPAGE VALIDATION:
expected_price = signal.entry_price
actual_price = result.price (dari broker)
slippage = |actual - expected|
max_acceptable = 0.15% dari harga (~$4 untuk XAUUSD @$2650)
Jika slippage > max_acceptable:
-> LOG WARNING (untuk monitoring & analisis)
-> Tetap pakai harga aktual untuk position tracking
PARTIAL FILL HANDLING:
requested_volume = lot dari risk calculation
filled_volume = result.volume (dari broker)
Jika filled < requested:
-> LOG WARNING dengan fill ratio (%)
-> Update position.lot_size ke filled_volume
-> Risk calculation tetap akurat (berdasarkan volume sebenarnya)
```
---
## Post-Entry
## *Post-Entry*
```python
# Step H: Log trade detail
trade_logger.log_trade_open(
signal, ml_prediction, regime, market_quality, ...
)
# Log trade detail ke PostgreSQL
trade_logger.log_trade_open(signal, ml_prediction, regime, market_quality, ...)
# Step G: Kirim notifikasi Telegram
# Kirim notifikasi Telegram
await telegram.send_trade_open(trade_info)
# Step H: Update cooldown timer
# Update cooldown timer
last_trade_time = now
```
---
## Diagram Flow Lengkap
## Diagram *Flow* Lengkap
```
Loop setiap 1 detik
|
v
Fetch 200 bar M15 -> Feature Eng -> SMC -> HMM -> XGBoost
|
v
[1] Session OK? ----NO----> Skip
|YES
[2] Risk OK? -------NO----> Skip (STOPPED)
|YES
[3] SMC Signal? ----NO----> Skip (tidak ada setup)
|YES
[4] ML >= 50%? -----NO----> Skip (terlalu uncertain)
|YES
[5] ML Agree? ------NO----> Skip (ML veto)
|YES
[6] Quality OK? ----NO----> Skip (AVOID/CRISIS)
|YES
[7] Confirmed 2x? --NO----> Skip (tunggu konfirmasi)
|YES
[8] No Pullback? ---NO----> Skip (retrace)
|YES
[9] Cooldown OK? ---NO----> Skip (< 5 menit)
|YES
[10] Pos < 2? ------NO----> Skip (full)
|YES
[11] Lot > 0? ------NO----> Skip
|YES
v
EXECUTE TRADE -> Register -> Log -> Telegram
```mermaid
graph TD
A["Loop Setiap ~30 Detik"] --> B["Fetch 200 Bar M15"]
B --> C["Feature Eng + SMC + HMM + XGBoost"]
C --> F1{"1. Flash Crash?"}
F1 -->|Ya| SKIP["Skip ↩"]
F1 -->|Tidak| F2{"2. Regime SLEEP?"}
F2 -->|Ya| SKIP
F2 -->|Tidak| F3{"3. Risk OK?"}
F3 -->|Tidak| SKIP
F3 -->|Ya| F4{"4. Session OK?"}
F4 -->|Tidak| SKIP
F4 -->|Ya| F5{"5. SMC Signal?"}
F5 -->|Tidak| SKIP
F5 -->|Ya| F6{"6. Signal Combo?"}
F6 -->|Tidak| SKIP
F6 -->|Ya| F7{"7. H1 Bias OK?"}
F7 -->|Tidak| SKIP
F7 -->|Ya| F8{"8. Jam OK?"}
F8 -->|Tidak| SKIP
F8 -->|Ya| F9{"9. Cooldown OK?"}
F9 -->|Tidak| SKIP
F9 -->|Ya| F11{"10. Risk Gate?"}
F11 -->|Tidak| SKIP
F11 -->|Ya| F12{"11-14. Lot/Spread/Pos?"}
F12 -->|Tidak| SKIP
F12 -->|Ya| EXEC["EKSEKUSI TRADE"]
EXEC --> POST["Register + Log + Telegram"]
```
---
## Statistik Filter
Dalam kondisi normal, dari ratusan loop per jam:
- **~95%** diblokir oleh "tidak ada SMC signal" (pasar sideways)
- **~3%** diblokir oleh ML disagreement atau low confidence
- **~1%** diblokir oleh pullback filter atau session
- **<1%** lolos semua filter dan menghasilkan trade
Dalam kondisi normal, dari ratusan *loop* per jam:
**Rata-rata:** 3-8 trade per hari (sangat selektif).
| Sumber *Block* | Persentase | Keterangan |
|-----------------|-----------|------------|
| Tidak ada sinyal SMC | **~95%** | Pasar *sideways*, tidak ada *setup* |
| ML *disagreement* / *low confidence* | **~3%** | ML tidak yakin atau berlawanan |
| *Pullback*, sesi, H1 *bias* | **~1%** | Filter waktu dan arah |
| **Lolos semua → *Trade*** | **< 1%** | Sangat selektif |
**Rata-rata:** 3-8 *trade* per hari.
docs/arsitektur-ai/10-Exit-Trade.md
+174 -311
View File
@@ -1,348 +1,211 @@
# Exit Trade — Proses Keluar Posisi
# *Exit Trade* — Proses Keluar Posisi
> **File utama:** `main_live.py`, `src/smart_risk_manager.py`
> **File pendukung:** `src/position_manager.py`
---
## Apa Itu Exit Trade?
## Apa Itu *Exit Trade*?
Exit Trade adalah keseluruhan proses **monitoring posisi terbuka** dan **memutuskan kapan menutup**. Bot memeriksa setiap posisi terbuka **setiap ~10 detik** (di antara candle) atau **setiap candle baru** (full analysis) dengan 10 kondisi exit berbeda.
*Exit Trade* adalah keseluruhan proses **monitoring posisi terbuka** dan **memutuskan kapan menutup**. Bot memeriksa setiap posisi terbuka **setiap ~10 detik** (di antara *candle*) atau **setiap *candle* baru** (*full analysis*) dengan **12 kondisi *exit*** berbeda.
**Analogi:** Exit Trade seperti **pilot otomatis di pesawat** — terus monitor ketinggian (profit), cuaca (momentum), bahan bakar (waktu), dan bisa landing darurat kapan saja.
**Prinsip utama:** **Jangan biarkan *winner* menjadi *loser***, tapi juga **jangan potong *winner* terlalu cepat**.
---
## 2 Jalur Exit
## 12 Kondisi *Exit*
```
Jalur 1: BROKER EXIT (otomatis, independen)
-> Harga hit TP level -> tutup otomatis
-> Harga hit SL level -> tutup otomatis
-> Tidak perlu bot online
Jalur 2: SOFTWARE EXIT (cerdas, kontekstual)
-> Bot evaluasi setiap 1 detik
-> Mempertimbangkan momentum, ML, waktu, dll
-> 10 kondisi exit berbeda
```mermaid
graph TD
POS["Posisi Terbuka"] --> C1{"1. Smart TP<br/>Profit ≥ $15?"}
C1 -->|Ya & kondisi| CLOSE["TUTUP POSISI"]
C1 -->|Tidak| C2{"2. Early Exit<br/>Profit $5-15?"}
C2 -->|Ya & reversal| CLOSE
C2 -->|Tidak| C3{"3. Early Cut<br/>Loss + momentum?"}
C3 -->|Ya| CLOSE
C3 -->|Tidak| C4{"4. Trend Reversal<br/>ML sinyal balik?"}
C4 -->|Ya| CLOSE
C4 -->|Tidak| C5{"5. Max Loss<br/>Loss > 50% max?"}
C5 -->|Ya| CLOSE
C5 -->|Tidak| C6{"6. Stall<br/>Stuck + rugi?"}
C6 -->|Ya| CLOSE
C6 -->|Tidak| C7{"7. Daily Limit<br/>Batas harian?"}
C7 -->|Ya| CLOSE
C7 -->|Tidak| C8{"8. Weekend Close"}
C8 -->|Ya| CLOSE
C8 -->|Tidak| C9{"9. Time-Based<br/>4-8 jam?"}
C9 -->|Ya & kondisi| CLOSE
C9 -->|Tidak| C10{"10-12. Trailing SL<br/>Breakeven, dll"}
C10 -->|Ya| CLOSE
C10 -->|Tidak| HOLD["TAHAN POSISI ↩"]
```
---
## Monitoring Loop
### CHECK 1: *Smart Take Profit* (Profit ≥ $15)
| Kondisi | Aksi | Keterangan |
|---------|------|------------|
| Profit ≥ **$40** | **Tutup** langsung | Target tercapai — *hard TP* |
| Profit ≥ **$25** dan momentum < -30 | **Tutup** | Profit bagus tapi momentum turun |
| *Peak profit* > $30, profit turun ke < 60% *peak* | **Tutup** | Lindungi profit dari *peak* |
| Probabilitas TP < 25%, profit ≥ **$20** | **Tutup** | Kemungkinan TP rendah |
| Momentum ≥ 0 | **Tahan** | Masih bagus, biarkan berjalan |
---
### CHECK 2: *Smart Early Exit* (Profit $5-15)
```python
# main_live.py Lines 1117-1215
# Setiap 1 detik, untuk SETIAP posisi terbuka:
for position in open_positions:
# Update data posisi
current_price = mt5.get_tick(symbol)
current_profit = position.profit
# Update history untuk analisis momentum
guard.update_history(current_price, current_profit, ml_confidence)
# Evaluasi: haruskah ditutup?
should_close, reason, message = smart_risk.evaluate_position(
ticket=ticket,
current_price=current_price,
current_profit=profit,
ml_signal=ml_prediction.signal,
ml_confidence=ml_prediction.confidence,
regime=regime_state,
)
if should_close:
# Tutup posisi
close_position(ticket, reason)
if 5 <= current_profit < 15:
if momentum < -50 and ml_confidence >= 0.65:
if ml_signal berlawanan dengan arah posisi:
TUTUP # Sinyal reversal kuat + profit kecil
```
**Tujuan:** Ambil profit kecil jika momentum sangat negatif DAN ML yakin *trend* berbalik.
---
## 10 Kondisi Exit (Urutan Pengecekan)
### CHECK 1: Smart Take Profit (profit >= $15)
```
Ketika profit sudah cukup besar, evaluasi apakah harus diamankan:
a) Hard TP: profit >= $40
-> TUTUP langsung, target tercapai
b) Momentum TP: profit >= $25 DAN momentum < -30
-> TUTUP, profit sedang turun cepat
c) Peak Protection: peak > $30 DAN current < 60% peak
-> TUTUP, lindungi dari drawback lebih dalam
d) Probability TP: TP_prob < 25% DAN profit >= $20
-> TUTUP, kemungkinan capai TP sudah rendah
e) Strong Momentum: momentum >= 0
-> HOLD, biarkan profit berjalan (let it run)
```
---
### CHECK 2: Early Exit Small Profit ($5-$15)
```
Profit masih kecil tapi ada tanda bahaya:
IF profit $5-$15
AND momentum < -50 (turun sangat cepat)
AND ML confidence >= 65% berlawanan arah:
-> TUTUP, ambil profit kecil sebelum hilang
```
---
### CHECK 3: Early Cut (v4 — Smart Hold DIHAPUS)
```
Posisi sedang rugi — potong cepat jika sinyal buruk:
IF profit < 0:
Loss >= 30% max ($15 dari $50) DAN momentum < -30
-> TUTUP CEPAT (early cut, jangan tunggu recovery)
v4 PERUBAHAN: "Smart Hold" DIHAPUS
- SEBELUMNYA: Hold posisi rugi menunggu golden time (20:00-23:59)
- SEBELUMNYA: Hold posisi rugi kecil di sesi London (15:00-19:00)
- SEKARANG: Tidak ada lagi "hold losers hoping for recovery"
- ALASAN: Menahan posisi rugi menunggu sesi tertentu = perilaku
berbahaya (martingale mentality). Proper risk management:
ikuti aturan SL, jangan berharap recovery.
```
---
### CHECK 4: Trend Reversal Detection
```
ML mendeteksi perubahan tren:
IF ML confidence >= 65% berlawanan dengan posisi:
a) Loss > 40% max DAN profit < -$8
-> TUTUP (reversal + loss signifikan)
b) Akumulasi 3x reversal warning DAN loss < -$10
-> TUTUP (multiple warnings = konfirmasi reversal)
c) Belum memenuhi threshold
-> reversal_warnings += 1 (catat warning)
```
---
### CHECK 5: Maximum Loss Per Trade
```
Loss mencapai batas toleransi:
IF loss >= 50% dari max_loss ($25 dari $50):
Exception: golden_time <= 1 jam DAN momentum > -40
-> HOLD (kesempatan terakhir recovery)
Selain itu:
-> TUTUP [S/L] Position loss limit
```
---
### CHECK 6: Stall Detection
```
Harga tidak bergerak kemana-mana:
IF 10 candle terakhir range profit < $3
AND current_profit < -$15:
stall_count += 1
IF stall_count >= 5:
-> TUTUP [STALL] Harga stuck, buang waktu & margin
```
---
### CHECK 7: Daily Loss Limit
```
Mencegah daily loss limit terlampaui:
potential_daily_loss = daily_loss + abs(min(0, current_profit))
IF potential_daily_loss >= max_daily_loss ($250):
-> TUTUP [LIMIT] Akan melampaui batas harian
```
---
### CHECK 8: Weekend Close
```
Proteksi dari gap weekend:
IF hari Jumat setelah 04:00 WIB:
a) profit > 0
-> TUTUP [WEEKEND] Amankan profit
b) profit > -$10
-> TUTUP [WEEKEND] Loss kecil, hindari gap
c) profit <= -$10
-> HOLD (loss terlalu besar untuk cut, evaluasi manual)
```
---
### CHECK 9: Smart Time-Based Exit (v5: Smarter — Don't Cut Winners)
```
Mencegah posisi "zombie" TAPI jangan potong posisi profit yang masih tumbuh.
trade_duration = (sekarang - entry_time) dalam jam
profit_growing = momentum > 0
ml_agrees = ML signal searah dengan posisi
IF 4+ jam:
a) profit < $5 DAN NOT profit_growing:
- profit >= $0 -> TUTUP [TIMEOUT] Breakeven + no growth
- profit > -$15 -> TUTUP [TIMEOUT] Small loss + no growth
b) profit >= $5 DAN profit_growing DAN ml_agrees:
-> HOLD (extend time — profit masih tumbuh!)
-> Log: "extending time"
IF 6+ jam:
a) profit < $10 ATAU NOT profit_growing:
-> TUTUP [MAX TIME] (sebelumnya: force close apapun kondisi)
b) profit >= $10 DAN profit_growing:
-> EXTEND ke max 8 jam (v5 BARU — biarkan profit berjalan)
-> Di jam 8+ -> TUTUP [MAX TIME] Take profit
v5 PERUBAHAN:
- 4h: Sekarang cek profit growth, bukan hanya profit < $5
- 6h: BUKAN force close lagi — extend ke 8h jika profitable + growing
- ML agreement dipertimbangkan sebelum timeout
- Prinsip: jangan potong pemenang yang masih berjalan
```
**Visualisasi:**
```
Jam: 0 1 2 3 4 5 6 7 8
|-----|-----|-----|-----|-----|-----|-----|-----|
entry | | |
| | |
4h check: 6h check: 8h FINAL EXIT
stuck? profitable?
no growth? growing?
-> exit -> extend!
not growing?
-> exit
```
---
### CHECK 10: Default — HOLD
```
Tidak ada kondisi exit terpenuhi:
-> HOLD posisi
-> Log status: momentum, TP probability, ML signal
-> Evaluasi ulang di cek berikutnya (~10 detik atau candle baru)
```
---
## Exit Reason Enum
| Reason | Kode | Deskripsi |
|--------|------|-----------|
| `TAKE_PROFIT` | take_profit | Target profit tercapai |
| `TREND_REVERSAL` | trend_reversal | ML deteksi reversal |
| `DAILY_LIMIT` | daily_limit | Batas harian tercapai |
| `POSITION_LIMIT` | position_limit | Max loss per trade |
| `TOTAL_LIMIT` | total_limit | Batas total tercapai |
| `WEEKEND_CLOSE` | weekend_close | Penutupan Jumat |
| `TIMEOUT` | timeout | Time-based exit (4h/6h) |
| `STALL` | stall | Harga stuck |
| `MANUAL` | manual | Penutupan manual |
---
## Post-Exit Flow
### CHECK 3: *Early Cut* (Loss + Momentum Negatif)
```python
# Setelah posisi ditutup:
if current_profit < 0:
loss_pct = abs(current_profit) / max_loss_per_trade * 100
if momentum < -50 and loss_pct >= 30:
TUTUP # Potong kerugian sebelum makin besar
```
# 1. Record hasil trade
risk_result = smart_risk.record_trade_result(profit)
# Update: daily_loss, total_loss, consecutive_losses, mode
**Perubahan dari v1:** *Smart Hold* (menahan posisi rugi menunggu *golden time*) **sudah dihapus** — dianggap berbahaya dan melawan prinsip manajemen risiko yang benar.
# 2. Unregister dari monitoring
smart_risk.unregister_position(ticket)
---
# 3. Log trade
trade_logger.log_trade_close(
ticket, entry_price, exit_price, profit, pips,
duration, exit_reason, ml_signal, regime, ...
)
### CHECK 4: *Trend Reversal* (Sinyal ML Berbalik)
# 4. Kirim notifikasi Telegram
await telegram.send_trade_close(trade_info)
# Format: WIN/LOSS/BE, P/L, pips, duration, balance
| Kondisi | Aksi |
|---------|------|
| ML sinyal **berbalik** + *confidence* ≥ 75% + loss > 40% max + loss > $8 | **Tutup** |
| **3x** peringatan *reversal* berturut-turut + loss > $10 | **Tutup** |
# 5. Cek limit violations
if risk_result["daily_limit_hit"]:
await send_critical_alert("DAILY LOSS LIMIT")
# Mode -> STOPPED, tidak ada trade lagi hari ini
**Perubahan:** *Threshold* diturunkan dari 5x ke **3x** peringatan, dan batas loss dari 60% ke **40%** — lebih responsif.
if risk_result["total_limit_hit"]:
await send_critical_alert("TOTAL LOSS LIMIT")
# Mode -> STOPPED permanen
---
### CHECK 5: *Maximum Loss Per Trade*
```python
if current_profit <= -(max_loss_per_trade * 0.50):
TUTUP # 50% dari batas max — tanpa pengecualian
```
Untuk akun *small* ($5.000): max loss per *trade* = ~$50, *trigger* di $25.
---
### CHECK 6: *Stall Detection*
```python
if len(profit_history) >= 10:
recent_range = max(last_10) - min(last_10)
if recent_range < $3 and current_profit < -$15:
stall_count += 1
if stall_count >= 5:
TUTUP # Harga stuck, posisi rugi
```
**Tujuan:** Deteksi posisi yang "terjebak" — harga tidak bergerak tapi posisi rugi.
---
### CHECK 7: Batas Kerugian Harian
```python
potensi_loss = daily_loss + abs(current_profit)
if potensi_loss >= max_daily_loss:
TUTUP # Akan melebihi batas kerugian harian
```
---
## Diagram Exit Flow
### CHECK 8: *Weekend Close*
```python
# Sabtu 04:30+ WIB (30 menit sebelum market tutup)
if near_weekend_close:
if profit > 0:
TUTUP # Amankan profit
elif loss > -$10:
TUTUP # Loss kecil — hindari gap weekend
```
Setiap ~10 detik (atau candle baru), per posisi terbuka:
|
v
Update profit & momentum
|
v
[1] Profit >= $15? ----YES---> Smart TP evaluation
|NO (hard/$40, momentum, peak, prob)
v
[2] Profit $5-$15? ----YES---> Reversal + momentum drop?
|NO -> Early exit
v
[3] Profit < 0? -------YES---> Loss>30% + momentum<-30?
|NO -> EARLY CUT
v
[4] ML Reversal 65%+? -YES---> Loss > 40%? -> TUTUP
|NO Else warning++
v
[5] Loss >= 50% max? --YES---> TUTUP (kecuali golden time)
|NO
v
[6] Stall 10+ candle? -YES---> stall++ -> 5x? TUTUP
|NO
v
[7] Daily limit? ------YES---> TUTUP
|NO
v
[8] Friday close? -----YES---> TUTUP (profit>0 atau loss>-$10)
|NO
v
[9] Time >= 4h? -------YES---> profit<$5? TUTUP
| Time >= 6h? ---YES---> FORCE EXIT
|NO
v
[10] HOLD -> evaluasi ulang 1 detik kemudian
**Tujuan:** Hindari risiko *gap weekend* — posisi tanpa proteksi selama 2 hari.
---
### CHECK 9: *Smart Time-Based Exit*
| Durasi | Kondisi | Aksi |
|--------|---------|------|
| **4+ jam** | Profit < $5, momentum tidak tumbuh | **Tutup** — posisi *stuck* |
| **4+ jam** | Profit ≥ $5, momentum positif, ML sejalan | **Tahan** — perpanjang waktu |
| **6+ jam** | Profit < $10 ATAU momentum negatif | **Tutup** — terlalu lama |
| **8+ jam** | Apapun kondisinya | **Tutup** — batas waktu absolut |
**Perubahan:** Tidak lagi memotong *winner* secara paksa — posisi yang masih tumbuh bisa diperpanjang.
---
### CHECK 10-12: *Position Manager* (Tambahan)
Selain 9 kondisi di atas dari `SmartRiskManager`, `SmartPositionManager` juga menjalankan:
| # | Kondisi | Keterangan |
|---|---------|------------|
| 10 | ***Trailing Stop Loss*** | SL mengikuti harga naik (berbasis ATR) — `atr_trail_start_mult=4.0`, `atr_trail_step_mult=3.0` |
| 11 | ***Breakeven Protection*** | Pindahkan SL ke titik *entry* setelah profit ≥ 30 *pips* |
| 12 | **Proteksi *Drawdown*** | Tutup jika *drawdown* dari *peak* terlalu besar |
---
## Alasan *Exit* (*ExitReason Enum*)
| Kode | Deskripsi |
|------|-----------|
| `TAKE_PROFIT` | Target profit tercapai atau profit diamankan |
| `TREND_REVERSAL` | ML mendeteksi pembalikan *trend* |
| `DAILY_LIMIT` | Batas kerugian harian tercapai |
| `POSITION_LIMIT` | Batas kerugian per posisi tercapai (S/L) |
| `TOTAL_LIMIT` | Batas kerugian total tercapai |
| `WEEKEND_CLOSE` | Mendekati penutupan *weekend* |
| `MANUAL` | Penutupan manual |
---
## Diagram *Flow* Evaluasi Posisi
```mermaid
graph LR
A["Setiap ~10 detik"] --> B["Hitung Profit/Loss<br/>Momentum, TP Prob"]
B --> C["SmartRiskManager<br/>evaluate_position()"]
C --> D{"Harus<br/>Tutup?"}
D -->|Ya| E["Tutup via MT5<br/>+ Log + Telegram"]
D -->|Tidak| F["SmartPositionManager<br/>Trailing SL, Breakeven"]
F --> G{"SL Perlu<br/>Digeser?"}
G -->|Ya| H["Modify SL<br/>via MT5"]
G -->|Tidak| I["Tahan Posisi ↩"]
```
---
## Statistik *Exit*
Berdasarkan data *backtest* (Jan 2025 - Feb 2026):
| Alasan *Exit* | Persentase | Rata-rata P/L |
|----------------|-----------|----------------|
| *Take Profit* (semua jenis) | **~40%** | **+$18.50** |
| *Trend Reversal* / *Early Cut* | **~25%** | **-$12.30** |
| *Time-Based Exit* | **~15%** | **+$3.20** |
| *Trailing SL* hit | **~10%** | **+$8.70** |
| Max Loss / *Daily Limit* | **~8%** | **-$22.50** |
| *Weekend Close* | **~2%** | **+$5.10** |
docs/arsitektur-ai/11-News-Agent.md
+46 -185
View File
@@ -1,212 +1,73 @@
# News Agent — Monitoring Berita Ekonomi
# *News Agent* — Monitoring Berita Ekonomi
> **File:** `src/news_agent.py`
> **Class:** `NewsAgent`
> **Status:** Aktif tapi **TIDAK MEMBLOKIR** trading (monitoring only)
> **Status: NONAKTIF** — dikomentari di `main_live.py` baris 64
---
## Apa Itu News Agent?
## Status Saat Ini
News Agent memonitor **berita ekonomi high-impact** (NFP, FOMC, CPI) yang bisa menyebabkan volatilitas ekstrem di pasar gold. Awalnya dirancang untuk memblokir trading saat news, tapi setelah backtest menunjukkan bahwa blocking justru **kehilangan $178 profit**, sekarang hanya berfungsi sebagai **monitor dan logger**.
**Analogi:** News Agent seperti **stasiun cuaca** — melaporkan badai yang datang, tapi pilot (bot) tetap terbang karena pesawat (ML model) sudah cukup tangguh menangani turbulensi.
> **PENTING:** Modul *News Agent* saat ini **tidak aktif** dalam sistem *live*. *Import* dikomentari di `main_live.py`:
>
> ```python
> # from src.news_agent import NewsAgent, create_news_agent, MarketCondition # DISABLED
> ```
>
> Modul ini tersedia dalam *codebase* untuk aktivasi di masa mendatang.
---
## Kenapa Tidak Blocking?
## Apa Itu *News Agent*?
```
Hasil Backtest (29 trades):
- Win rate tanpa filter: 64.9%
- Win rate saat news: 62.1% (selisih hanya 2.8%)
- Profit yang hilang jika filter aktif: $178.15
*News Agent* adalah modul yang **memonitor berita ekonomi** berdampak tinggi dan menilai potensi dampaknya terhadap perdagangan XAUUSD. Ketika aktif, modul ini dapat:
Kesimpulan:
-> ML model sudah cukup menangani volatilitas news
-> Blocking justru kehilangan peluang profit
-> Monitoring cukup, tidak perlu blocking
1. **Mengambil kalender ekonomi** dari sumber eksternal
2. **Menilai dampak berita** terhadap pasar emas
3. **Memberi peringatan** atau **memblokir *trading*** saat berita berdampak tinggi
---
## *Design* Modul
```mermaid
graph TD
A["Sumber Berita<br/>API Kalender Ekonomi"] --> B["NewsAgent"]
B --> C{"Dampak?"}
C -->|"Rendah"| D["✅ Trading normal"]
C -->|"Sedang"| E["⚠️ Kurangi lot"]
C -->|"Tinggi"| F["🛑 Blokir trading"]
F --> G["NFP, FOMC, CPI<br/>Buffer 15-45 menit"]
```
---
## Event yang Dipantau
## Berita Berdampak Tinggi (Tersimpan di `session_filter.py`)
### 3 Event High-Impact
Meskipun *News Agent* nonaktif, daftar waktu berita sudah tersimpan di *Session Filter*:
| Event | Waktu (WIB) | Hari | Dampak ke Gold |
|-------|-------------|------|---------------|
| **NFP** (Non-Farm Payroll) | 20:30 | Jumat pertama bulan | Sangat tinggi |
| **FOMC** (Fed Decision) | 02:00 | ~8x per tahun | Sangat tinggi |
| **CPI** (Inflation) | 20:30 | Tgl 10-15 (Sel/Rab/Kam) | Tinggi |
### Deteksi Event
```python
# NFP: Jumat pertama bulan
if weekday == 4 and day <= 7: # Friday, day 1-7
if 19 <= hour <= 21: # 19:00-21:00 WIB
return "NFP (Non-Farm Payroll) - HIGH IMPACT"
# FOMC: Tanggal spesifik (hardcoded schedule)
fomc_dates = [
(1,29), (3,19), (5,7), (6,18), (7,30), # 2025
(9,17), (11,5), (12,17),
(1,29), (3,18), (5,6), (6,17), (7,29), # 2026
]
if (month, day) in fomc_dates:
if 1 <= hour <= 3: # 01:00-03:00 WIB
return "FOMC Decision - HIGH IMPACT"
# CPI: Sekitar tanggal 10-15, hari kerja
if 10 <= day <= 15 and 19 <= hour <= 21:
if weekday in [1, 2, 3]: # Selasa-Kamis
return "CPI (Inflation) - HIGH IMPACT"
```
| Berita | Jam WIB | *Buffer* Sebelum | *Buffer* Setelah |
|--------|---------|------------------|------------------|
| **NFP** (*Non-Farm Payrolls*) | 19:30 | 15 menit | 30 menit |
| **FOMC** (*Federal Reserve*) | 01:00 | 15 menit | 45 menit |
| **CPI** (*Consumer Price Index*) | 19:30 | 15 menit | 30 menit |
---
## Buffer Times
## Rencana Aktivasi
| Parameter | Default | Aktif di Production |
|-----------|---------|-------------------|
| `news_buffer_minutes` | 30 menit | **0** (disabled) |
| `high_impact_buffer_minutes` | 60 menit | **0** (disabled) |
Langkah-langkah untuk mengaktifkan kembali *News Agent*:
```python
# Inisialisasi di main_live.py
self.news_agent = create_news_agent(
news_buffer_minutes=0, # No blocking
high_impact_buffer_minutes=0, # No blocking
)
```
1. Hapus komentar di `main_live.py` baris 64
2. Konfigurasi sumber data berita di `.env`
3. Integrasikan pengecekan berita ke *entry filter pipeline*
4. Uji coba dengan mode *monitoring only* (tidak memblokir, hanya *log*)
5. Aktifkan *blocking* setelah validasi
---
## Market Condition States
## Mengapa Dinonaktifkan?
| Kondisi | Bisa Trade? | Lot Multiplier | Trigger |
|---------|------------|---------------|---------|
| `SAFE` | Ya | 1.0x | Tidak ada news |
| `CAUTION` | Ya | 0.5x | News medium-impact |
| `DANGER_NEWS` | Tidak* | 0.0x | High-impact news |
| `DANGER_SENTIMENT` | Tidak* | 0.5x | Sentimen sangat bearish |
*\*Di production, DANGER tetap diizinkan trading (monitoring only)*
---
## Analisis Sentimen
News Agent juga bisa menganalisis headline berita berdasarkan keyword:
### Keyword Bullish (untuk Gold)
```
Geopolitical: war, conflict, invasion, crisis, escalation
Economic: rate cut, dovish, easing, recession, stimulus
Market: safe haven, gold surge, gold rally, buy gold
```
### Keyword Bearish (untuk Gold)
```
Geopolitical: peace deal, ceasefire, de-escalation
Economic: rate hike, hawkish, tightening, strong dollar
Market: risk on, stocks rally, sell gold, gold crash
```
### Keyword Volatile
```
breaking, urgent, flash, sudden, unexpected, shock, crash, spike
```
### Scoring
```
Setiap keyword match:
Bullish: +0.3
Bearish: -0.3
Volatile: -0.1 (penalty)
Score range: -1.0 (sangat bearish) sampai +1.0 (sangat bullish)
Confidence berdasarkan jumlah keyword yang match
```
---
## Method `should_trade()`
```python
def should_trade(headlines=None) -> (bool, str, float):
"""
Returns:
can_trade: bool <- Apakah aman trading
reason: str <- Alasan
lot_multiplier: float <- Pengali lot (0.0-1.0)
"""
# 1. Cek economic calendar (MT5 + hardcoded events)
# 2. Analisis sentimen (jika ada headlines)
# 3. Tentukan kondisi pasar
# 4. Return rekomendasi
```
---
## Integrasi di Main Loop
```python
# main_live.py Lines 485-496
# NEWS AGENT MONITORING (NO BLOCKING)
can_trade_news, news_reason, news_lot_mult = self.news_agent.should_trade()
# Hanya LOG, TIDAK block
if not can_trade_news and loop_count % 300 == 0: # Setiap 5 menit
logger.info(f"News Agent: HIGH IMPACT NEWS - {news_reason} (trading allowed)")
# Catatan:
# - news_lot_mult dihitung tapi TIDAK diterapkan
# - Trading tetap berjalan normal
# - Informasi digunakan untuk logging dan analisis
```
---
## Sumber Data
| Sumber | Status | Keterangan |
|--------|--------|-----------|
| **MT5 Calendar** | Aktif | Cek economic calendar dari terminal |
| **Hardcoded Events** | Aktif (Fallback) | NFP, FOMC, CPI schedule |
| **NewsAPI** | Tersedia, tidak digunakan | External API (butuh API key) |
| **ForexFactory** | Tersedia, tidak diimplementasi | Placeholder untuk scraping |
---
## Konfigurasi
```python
NewsAgent(
news_buffer_minutes=30, # Buffer news biasa (disabled: 0)
high_impact_buffer_minutes=60, # Buffer high-impact (disabled: 0)
enable_mt5_calendar=True, # Cek MT5 calendar
enable_sentiment=True, # Analisis sentimen
)
# Cache
_cache_duration = 15 menit # Cache hasil calendar check
```
---
## Contoh Output Log
```
[14:30] News Agent: HIGH IMPACT NEWS - NFP (Non-Farm Payroll) (trading allowed)
[14:35] News Agent: Market condition SAFE - no upcoming events
[20:25] News Agent: HIGH IMPACT NEWS - CPI (Inflation) (trading allowed)
```
**Catatan:** Meskipun terdeteksi "HIGH IMPACT NEWS", bot tetap trading. Log ini berguna untuk analisis post-trade — apakah trade yang terjadi saat news perform baik atau buruk.
- **Ketergantungan API eksternal** — memerlukan *API key* dan koneksi internet stabil
- **Latensi tambahan** — setiap pengecekan berita menambah waktu *loop*
- **Hasil *backtest* tanpa *News Agent* sudah baik** — sistem sudah terproteksi oleh *session filter* dan *regime detector*
docs/arsitektur-ai/12-Telegram-Notifications.md
+132 -48
View File
@@ -1,16 +1,60 @@
# Telegram Notifications — Sistem Notifikasi Real-Time
# *Telegram Notifications* — Sistem Notifikasi *Real-Time*
> **File:** `src/telegram_notifier.py`
> **Class:** `TelegramNotifier`
> **API:** Telegram Bot API (async via aiohttp)
> **API:** Telegram Bot API (*async* via aiohttp)
---
## Apa Itu Telegram Notifications?
## Arsitektur Notifikasi
Telegram Notifications mengirimkan **laporan real-time** ke grup Telegram setiap kali terjadi event penting — trade dibuka/ditutup, laporan harian, alert darurat, dan status sistem.
```mermaid
flowchart LR
A["Event\n(Trade / Alert / Timer)"] --> B["TelegramNotifier\n(async aiohttp)"]
B --> C["Telegram Bot API\n(/sendMessage\n/sendPhoto\n/sendDocument)"]
C --> D["User / Grup Telegram"]
**Analogi:** Telegram Notifications seperti **dashboard pilot di cockpit** — menampilkan semua informasi penting secara real-time tanpa harus melihat layar trading.
style A fill:#2d2d2d,stroke:#f5a623,color:#fff
style B fill:#2d2d2d,stroke:#4a9eff,color:#fff
style C fill:#2d2d2d,stroke:#50c878,color:#fff
style D fill:#2d2d2d,stroke:#ff6b6b,color:#fff
```
```mermaid
flowchart TD
LOOP["Main Loop\n(setiap 1 detik)"] --> NEW_DAY{"New day?"}
NEW_DAY -- Ya --> DAILY["Daily Summary + Reset"]
NEW_DAY -- Tidak --> HOURLY{"Hourly timer?"}
HOURLY -- Ya --> HOUR_MSG["Hourly Analysis"]
HOURLY -- Tidak --> HALF{"30-min timer?"}
HALF -- Ya --> MARKET["Market Update"]
HALF -- Tidak --> TRADE{"Trade executed?"}
TRADE -- Ya --> OPEN["Trade Open Notification"]
TRADE -- Tidak --> CLOSE{"Position closed?"}
CLOSE -- Ya --> CLOSE_MSG["Trade Close Notification"]
CLOSE -- Tidak --> LIMIT{"Limit hit?"}
LIMIT -- Ya --> CRIT["Critical Limit Alert"]
LIMIT -- Tidak --> FLASH{"Flash crash?"}
FLASH -- Ya --> EMERG["Emergency Close Alert"]
FLASH -- Tidak --> LOOP
style LOOP fill:#1a1a2e,stroke:#4a9eff,color:#fff
style DAILY fill:#1a1a2e,stroke:#50c878,color:#fff
style HOUR_MSG fill:#1a1a2e,stroke:#50c878,color:#fff
style MARKET fill:#1a1a2e,stroke:#50c878,color:#fff
style OPEN fill:#1a1a2e,stroke:#f5a623,color:#fff
style CLOSE_MSG fill:#1a1a2e,stroke:#f5a623,color:#fff
style CRIT fill:#1a1a2e,stroke:#ff6b6b,color:#fff
style EMERG fill:#1a1a2e,stroke:#ff6b6b,color:#fff
```
---
## Apa Itu *Telegram Notifications*?
*Telegram Notifications* mengirimkan **laporan *real-time*** ke grup Telegram setiap kali terjadi event penting — trade dibuka/ditutup, laporan harian, alert darurat, dan status sistem.
**Analogi:** *Telegram Notifications* seperti **dashboard pilot di cockpit** — menampilkan semua informasi penting secara *real-time* tanpa harus melihat layar trading.
---
@@ -40,23 +84,23 @@ enabled = bool(bot_token and chat_id) # Auto-disable jika tidak dikonfigurasi
| # | Tipe | Trigger | Frekuensi |
|---|------|---------|-----------|
| 1 | Trade Open | Order berhasil dieksekusi | Per trade |
| 2 | Trade Close | Posisi ditutup | Per trade |
| 3 | Market Update | Timer 30 menit | Setiap 30 menit |
| 4 | Hourly Analysis | Timer 1 jam | Setiap 1 jam |
| 5 | Daily Summary | Pergantian hari | 1x per hari |
| 6 | Startup | Bot dinyalakan | 1x per sesi |
| 7 | Shutdown | Bot dimatikan | 1x per sesi |
| 8 | News Alert | Event ekonomi terdeteksi | Per event |
| 9 | Critical Limit | Daily/total loss limit | Per event |
| 10 | Emergency Close | Flash crash / darurat | Per event |
| 11 | System Status | Status berkala | Per request |
| 1 | *Trade Open* | Order berhasil dieksekusi | Per trade |
| 2 | *Trade Close* | Posisi ditutup | Per trade |
| 3 | *Market Update* | Timer 30 menit | Setiap 30 menit |
| 4 | *Hourly Analysis* | Timer 1 jam | Setiap 1 jam |
| 5 | *Daily Summary* | Pergantian hari | 1x per hari |
| 6 | *Startup* | Bot dinyalakan | 1x per sesi |
| 7 | *Shutdown* | Bot dimatikan | 1x per sesi |
| 8 | *News Alert* | Event ekonomi terdeteksi | Per event |
| 9 | *Critical Limit* | Daily/total loss limit | Per event |
| 10 | *Emergency Close* | *Flash crash* / darurat | Per event |
| 11 | *System Status* | Status berkala | Per request |
---
## Format Pesan
### 1. Trade Open
### 1. *Trade Open*
```
🟢 LONG #123456
@@ -76,13 +120,13 @@ enabled = bool(bot_token and chat_id) # Auto-disable jika tidak dikonfigurasi
| 🟢/🔴 | BUY (hijau) / SELL (merah) |
| LONG/SHORT | Arah posisi |
| #123456 | Ticket ID dari broker |
| R:R | Risk to Reward ratio |
| AI: 75% | ML confidence |
| medium_volatility | HMM regime |
| R:R | *Risk to Reward ratio* |
| AI: 75% | ML *confidence* |
| medium_volatility | HMM *regime* |
---
### 2. Trade Close
### 2. *Trade Close*
```
✅ WIN #123456
@@ -106,7 +150,7 @@ enabled = bool(bot_token and chat_id) # Auto-disable jika tidak dikonfigurasi
---
### 3. Market Update (Setiap 30 Menit)
### 3. *Market Update* (Setiap 30 Menit)
```
📊 XAUUSD $4965.00
@@ -120,7 +164,7 @@ enabled = bool(bot_token and chat_id) # Auto-disable jika tidak dikonfigurasi
---
### 4. Hourly Analysis (Setiap 1 Jam)
### 4. *Hourly Analysis* (Setiap 1 Jam)
```
📊 HOURLY 14:00 WIB
@@ -152,7 +196,7 @@ Risk NORMAL
---
### 5. Daily Summary
### 5. *Daily Summary*
```
🎉 DAILY REPORT 2025-02-06
@@ -187,7 +231,7 @@ Recent Trades
---
### 6. Startup
### 6. *Startup*
```
🚀 BOT STARTED
@@ -211,7 +255,7 @@ Risk Settings
---
### 7. Shutdown
### 7. *Shutdown*
```
🔴 BOT STOPPED
@@ -227,7 +271,7 @@ Session Summary
---
### 8. News Alert
### 8. *News Alert*
```
🚨 NEWS DANGER_NEWS
@@ -245,7 +289,7 @@ Session Summary
---
### 9. Critical Limit Alert
### 9. *Critical Limit Alert*
```
🚨 DAILY LOSS LIMIT REACHED 🚨
@@ -259,7 +303,7 @@ Will resume tomorrow automatically.
---
### 10. Emergency Close
### 10. *Emergency Close*
```
🚨 EMERGENCY CLOSE COMPLETE
@@ -274,26 +318,62 @@ Total P/L: -$45.00
| Alert Type | Emoji | Contoh |
|-----------|-------|--------|
| flash_crash | 🚨 | "Flash crash detected on XAUUSD" |
| high_volatility | ⚡ | "Volatility spike detected" |
| connection_error | 📡 | "MT5 connection lost" |
| model_retrain | 🔄 | "ML model retrained successfully" |
| market_close | 🔔 | "Market closing in 30 minutes" |
| low_balance | 💰 | "Account balance below threshold" |
| *flash_crash* | 🚨 | "Flash crash detected on XAUUSD" |
| *high_volatility* | ⚡ | "Volatility spike detected" |
| *connection_error* | 📡 | "MT5 connection lost" |
| *model_retrain* | 🔄 | "ML model retrained successfully" |
| *market_close* | 🔔 | "Market closing in 30 minutes" |
| *low_balance* | 💰 | "Account balance below threshold" |
---
## 3 Metode Pengiriman
```mermaid
flowchart LR
N["TelegramNotifier"] --> SM["send_message()\n/sendMessage\nTeks biasa"]
N --> SP["send_photo()\n/sendPhoto\nChart / grafik"]
N --> SD["send_document()\n/sendDocument\nFile PDF"]
style N fill:#2d2d2d,stroke:#4a9eff,color:#fff
style SM fill:#2d2d2d,stroke:#50c878,color:#fff
style SP fill:#2d2d2d,stroke:#f5a623,color:#fff
style SD fill:#2d2d2d,stroke:#ff6b6b,color:#fff
```
| Metode | Endpoint | Kegunaan |
|--------|----------|---------|
| `send_message()` | `/sendMessage` | Teks biasa (semua notifikasi) |
| `send_photo()` | `/sendPhoto` | Chart/grafik (daily report) |
| `send_photo()` | `/sendPhoto` | Chart/grafik (*daily report*) |
| `send_document()` | `/sendDocument` | File PDF (laporan detail) |
---
## Error Handling
## *Error Handling*
```mermaid
flowchart TD
SEND["send_message() / send_photo()"] --> TRY{"Try-Except"}
TRY -- Berhasil --> CHECK{"HTTP Status\n== 200?"}
CHECK -- Ya --> OK["Return True\n(Terkirim)"]
CHECK -- Tidak --> LOG_ERR["Log Error\nReturn False"]
TRY -- Exception --> LOG_WARN["Log Warning\nLanjut Trading"]
DISABLED{"Token / ChatID\nkosong?"} --> AUTO["Auto-disable\nReturn True"]
EMERG["Emergency Close"] --> CLOSE_POS["Tutup Posisi Dulu"]
CLOSE_POS --> TRY_NOTIF{"Kirim Notifikasi"}
TRY_NOTIF -- Gagal --> IGNORE["pass\n(Trading > Notifikasi)"]
TRY_NOTIF -- Berhasil --> OK2["Notifikasi Terkirim"]
style SEND fill:#1a1a2e,stroke:#4a9eff,color:#fff
style OK fill:#1a1a2e,stroke:#50c878,color:#fff
style OK2 fill:#1a1a2e,stroke:#50c878,color:#fff
style LOG_ERR fill:#1a1a2e,stroke:#ff6b6b,color:#fff
style LOG_WARN fill:#1a1a2e,stroke:#f5a623,color:#fff
style IGNORE fill:#1a1a2e,stroke:#f5a623,color:#fff
style AUTO fill:#1a1a2e,stroke:#888,color:#fff
```
```
Strategi: GRACEFUL DEGRADATION
@@ -313,6 +393,8 @@ Strategi: GRACEFUL DEGRADATION
-> Bot tetap berjalan tanpa notifikasi
```
Strategi ini menerapkan pola *graceful degradation* — kegagalan notifikasi **tidak pernah** menghentikan proses trading. Sistem *emergency close* akan tetap menutup posisi meskipun Telegram tidak responsif, menerapkan prinsip *circuit breaker* di mana komponen non-kritis diisolasi dari jalur kritis.
```python
# Contoh: Emergency close TIDAK boleh gagal karena Telegram
try:
@@ -323,20 +405,22 @@ except:
---
## Rate Limiting
## *Rate Limiting*
| Notifikasi | Interval |
|-----------|----------|
| Trade Open/Close | Langsung (per event) |
| Market Update | 30 menit |
| Hourly Analysis | 1 jam |
| Daily Summary | 1x per hari |
| Startup/Shutdown | 1x per sesi |
| Min message interval | 1 detik (variable) |
| *Trade Open/Close* | Langsung (per event) |
| *Market Update* | 30 menit |
| *Hourly Analysis* | 1 jam |
| *Daily Summary* | 1x per hari |
| *Startup* / *Shutdown* | 1x per sesi |
| Min *message interval* | 1 detik (variable) |
*Rate limiting* mencegah flooding ke Telegram Bot API yang memiliki batas ~30 pesan/detik per grup. Interval minimum 1 detik antar pesan menjaga bot tetap dalam batas aman.
---
## Kapan Notifikasi Dikirim di Main Loop
## Kapan Notifikasi Dikirim di *Main Loop*
```
Main Loop (setiap 1 detik)
@@ -362,9 +446,9 @@ Main Loop (setiap 1 detik)
---
## Formatting HTML
## *Formatting* HTML
Semua pesan menggunakan HTML parse mode:
Semua pesan menggunakan HTML *parse mode*:
```html
<b>Bold</b> -> Label penting
@@ -372,7 +456,7 @@ Semua pesan menggunakan HTML parse mode:
<i>Italic</i> -> Info tambahan, alasan signal
```
Tree structure menggunakan box-drawing characters:
Tree structure menggunakan *box-drawing characters*:
```
├ -> Item tengah
docs/arsitektur-ai/13-Auto-Trainer.md
+141 -95
View File
@@ -1,4 +1,4 @@
# Auto Trainer — Sistem Retraining Otomatis
# *Auto Trainer* --- Sistem *Retraining* Otomatis
> **File:** `src/auto_trainer.py`
> **Class:** `AutoTrainer`
@@ -6,21 +6,21 @@
---
## Apa Itu Auto Trainer?
## Apa Itu *Auto Trainer*?
Auto Trainer adalah sistem yang **melatih ulang model AI secara otomatis** agar tetap up-to-date dengan kondisi pasar terbaru. Retraining dilakukan saat market tutup (05:00 WIB) untuk menghindari gangguan saat trading aktif.
*Auto Trainer* adalah sistem yang **melatih ulang model AI secara otomatis** agar tetap up-to-date dengan kondisi pasar terbaru. *Retraining* dilakukan saat market tutup (05:00 WIB) untuk menghindari gangguan saat trading aktif.
**Analogi:** Auto Trainer seperti **pelatih yang membuat atlet berlatih setiap malam** — setelah pertandingan selesai, atlet (model AI) dilatih dengan data terbaru agar siap menghadapi tantangan esok hari.
**Analogi:** *Auto Trainer* seperti **pelatih yang membuat atlet berlatih setiap malam** --- setelah pertandingan selesai, atlet (model AI) dilatih dengan data terbaru agar siap menghadapi tantangan esok hari.
---
## Jadwal Retraining
## Jadwal *Retraining*
| Tipe | Waktu | Data | Boost Rounds | Kondisi |
|------|-------|------|-------------|---------|
| **Daily** | 05:00 WIB (market close) | 8.000 bar | 50 | Senin–Jumat |
| **Weekend** | 05:00 WIB Sabtu/Minggu | 15.000 bar | 80 | Deep training |
| **Emergency** | Kapan saja | 8.000 bar | 50 | AUC < 0.65 |
| **Daily** | 05:00 WIB (market close) | 8.000 bar | 50 | Senin--Jumat |
| **Weekend** | 05:00 WIB Sabtu/Minggu | 15.000 bar | 80 | *Deep training* |
| **Emergency** | Kapan saja | 8.000 bar | 50 | *AUC* < 0.65 |
| **Initial** | Pertama kali | 8.000 bar | 50 | Belum pernah training |
```
@@ -53,91 +53,137 @@ AutoTrainer(
---
## Proses Retraining (Step-by-Step)
## Proses *Retraining* (Step-by-Step)
### Flowchart Keputusan *Retraining*
```mermaid
flowchart TD
A[Cek should_retrain] --> B{Sudah >= 20 jam\nsejak retrain terakhir?}
B -- Tidak --> Z[Skip retrain]
B -- Ya --> C{Jam 05:00 WIB\natau AUC < 0.65?}
C -- Tidak --> Z
C -- Ya --> D{Weekend?}
D -- Ya --> E[Deep training:\n15K bar, 80 rounds]
D -- Tidak --> F[Daily training:\n8K bar, 50 rounds]
E --> G[Backup model lama]
F --> G
G --> H[Fetch data dari MT5]
H --> I[Feature Engineering\n+ SMC Analysis]
I --> J[Train HMM + XGBoost]
J --> K[Validasi AUC]
K --> L{Test AUC >= 0.60?}
L -- Ya --> M[Simpan model baru]
L -- Tidak --> N[Rollback ke model lama]
M --> O[Record hasil ke DB]
N --> O
O --> P[Selesai]
```
### Detail Langkah-Langkah
```
1. SHOULD RETRAIN CHECK
├ Sudah >= 20 jam sejak retrain terakhir?
├ Sekarang jam 05:00 WIB (±30 menit)?
├ Weekend? → Deep training (15K bar)
└ AUC < 0.65? → Emergency retrain
+-- Sudah >= 20 jam sejak retrain terakhir?
+-- Sekarang jam 05:00 WIB (+-30 menit)?
+-- Weekend? -> Deep training (15K bar)
+-- AUC < 0.65? -> Emergency retrain
2. BACKUP MODEL LAMA
├ Copy xgboost_model.pkl → backups/YYYYMMDD_HHMMSS/
├ Copy hmm_regime.pkl → backups/YYYYMMDD_HHMMSS/
└ Bersihkan backup lama (simpan 5 terakhir)
+-- Copy xgboost_model.pkl -> backups/YYYYMMDD_HHMMSS/
+-- Copy hmm_regime.pkl -> backups/YYYYMMDD_HHMMSS/
+-- Bersihkan backup lama (simpan 5 terakhir)
3. FETCH DATA TERBARU
├ Ambil 8K bar (daily) atau 15K bar (weekend) dari MT5
├ Symbol: XAUUSD, Timeframe: M15
└ Validasi: minimal 1000 bar
+-- Ambil 8K bar (daily) atau 15K bar (weekend) dari MT5
+-- Symbol: XAUUSD, Timeframe: M15
+-- Validasi: minimal 1000 bar
4. FEATURE ENGINEERING
├ FeatureEngineer.calculate_all() → 40+ fitur
├ SMCAnalyzer.calculate_all() → struktur pasar
└ create_target(lookahead=1) → label UP/DOWN
+-- FeatureEngineer.calculate_all() -> 40+ fitur
+-- SMCAnalyzer.calculate_all() -> struktur pasar
+-- create_target(lookahead=1) -> label UP/DOWN
5. TRAINING HMM
├ MarketRegimeDetector(n_regimes=3, lookback=500)
├ hmm.fit(df)
└ Save → models/hmm_regime.pkl
+-- MarketRegimeDetector(n_regimes=3, lookback=500)
+-- hmm.fit(df)
+-- Save -> models/hmm_regime.pkl
6. TRAINING XGBOOST
├ TradingModel(confidence_threshold=0.60)
├ xgb.fit(train_ratio=0.7, num_boost_round=50/80)
├ Early stopping: 5 rounds
└ Save → models/xgboost_model.pkl
+-- TradingModel(confidence_threshold=0.60)
+-- xgb.fit(train_ratio=0.7, num_boost_round=50/80)
+-- Early stopping: 5 rounds
+-- Save -> models/xgboost_model.pkl
7. VALIDASI
├ Cek Train AUC & Test AUC
├ Test AUC < 0.60? → ROLLBACK ke model lama (v4: dinaikkan dari 0.52)
├ Test AUC < 0.65? → WARNING (alert)
└ Test AUC >= 0.65? → SUCCESS
+-- Cek Train AUC & Test AUC
+-- Test AUC < 0.60? -> ROLLBACK ke model lama (v4: dinaikkan dari 0.52)
+-- Test AUC < 0.65? -> WARNING (alert)
+-- Test AUC >= 0.65? -> SUCCESS
8. RECORD HASIL
├ Simpan ke PostgreSQL (training_runs table)
├ Backup ke file (retrain_history.txt)
└ Log: durasi, AUC, accuracy, status
+-- Simpan ke PostgreSQL (training_runs table)
+-- Backup ke file (retrain_history.txt)
+-- Log: durasi, AUC, accuracy, status
```
---
## Backup & Rollback
## *Backup* & *Rollback*
### Sistem Backup
### Sistem *Backup*
```
models/
├── xgboost_model.pkl # Model aktif
├── hmm_regime.pkl # Model aktif
└── backups/
├── 20250206_050015/ # Backup terbaru
│ ├── xgboost_model.pkl
│ └── hmm_regime.pkl
├── 20250205_050012/ # Backup kemarin
│ ├── xgboost_model.pkl
│ └── hmm_regime.pkl
└── ... (max 5 backup)
+-- xgboost_model.pkl # Model aktif
+-- hmm_regime.pkl # Model aktif
+-- backups/
+-- 20250206_050015/ # Backup terbaru
| +-- xgboost_model.pkl
| +-- hmm_regime.pkl
+-- 20250205_050012/ # Backup kemarin
| +-- xgboost_model.pkl
| +-- hmm_regime.pkl
+-- ... (max 5 backup)
```
### Kapan Rollback?
### Kapan *Rollback*?
```
Model baru di-training
|
v
Cek Test AUC
|
├── AUC >= 0.65 ──> KEEP model baru ✅
|
├── AUC 0.60-0.65 ──> KEEP tapi WARNING ⚠️
| (akan trigger emergency retrain nanti)
|
└── AUC < 0.60 ──> ROLLBACK ke model lama 🔄
(v4: dinaikkan dari 0.52, karena 0.52 hampir = acak)
#### Diagram Validasi *AUC*
```mermaid
flowchart TD
A[Model baru selesai di-training] --> B[Hitung Test AUC]
B --> C{Test AUC >= 0.65?}
C -- Ya --> D[KEEP model baru]
D --> D1[Status: SUCCESS]
C -- Tidak --> E{Test AUC >= 0.60?}
E -- Ya --> F[KEEP model baru\ndengan WARNING]
F --> F1[Status: WARNING]
F1 --> F2[Akan trigger\nemergency retrain nanti]
E -- Tidak --> G[ROLLBACK ke model lama]
G --> G1[Status: ROLLBACK]
G1 --> G2[v4: threshold dinaikkan\ndari 0.52 ke 0.60]
style D fill:#22c55e,color:#fff
style D1 fill:#22c55e,color:#fff
style F fill:#eab308,color:#000
style F1 fill:#eab308,color:#000
style F2 fill:#eab308,color:#000
style G fill:#ef4444,color:#fff
style G1 fill:#ef4444,color:#fff
style G2 fill:#ef4444,color:#fff
```
### Method Rollback
#### Ringkasan Keputusan
| Kondisi | Aksi | Status |
|---------|------|--------|
| *AUC* >= 0.65 | KEEP model baru | SUCCESS |
| *AUC* 0.60--0.65 | KEEP tapi WARNING (akan trigger *emergency* retrain nanti) | WARNING |
| *AUC* < 0.60 | *ROLLBACK* ke model lama (v4: dinaikkan dari 0.52, karena 0.52 hampir = acak) | ROLLBACK |
### Method *Rollback*
```python
def rollback_models(reason="Manual rollback"):
@@ -151,28 +197,28 @@ def rollback_models(reason="Manual rollback"):
---
## AUC Monitoring
## *AUC* Monitoring
### Apa Itu AUC?
### Apa Itu *AUC*?
AUC (Area Under Curve) mengukur **seberapa baik model membedakan sinyal BUY vs SELL**:
*AUC* (*Area Under Curve*) mengukur **seberapa baik model membedakan sinyal BUY vs SELL**:
| AUC | Arti | Aksi |
| *AUC* | Arti | Aksi |
|-----|------|------|
| 0.80+ | Sangat bagus | Model dalam kondisi prima |
| 0.65-0.80 | Bagus | Normal, lanjut trading |
| 0.60-0.65 | Minimum | Warning, pertimbangkan retrain |
| < 0.60 | Buruk | **ROLLBACK** + retrain segera (v4 threshold) |
| 0.60-0.65 | Minimum | Warning, pertimbangkan *retraining* |
| < 0.60 | Buruk | **ROLLBACK** + *retraining* segera (v4 threshold) |
| 0.50 | Sama dengan tebak koin | Model tidak berguna |
### Auto-Retrain on Low AUC
### Auto-Retrain on Low *AUC*
```python
def should_retrain_due_to_low_auc():
"""
Cek AUC saat ini:
AUC < 0.65? → Perlu retrain
Tapi: sudah retrain < 4 jam lalu? → Tunggu
AUC < 0.65? -> Perlu retrain
Tapi: sudah retrain < 4 jam lalu? -> Tunggu
(mencegah retrain loop)
"""
```
@@ -185,23 +231,23 @@ def should_retrain_due_to_low_auc():
```
Table: training_runs
├── id # Auto-increment
├── training_type # "daily" / "weekend"
├── bars_used # 8000 / 15000
├── num_boost_rounds # 50 / 80
├── started_at # Timestamp mulai
├── completed_at # Timestamp selesai
├── duration_seconds # Durasi training
├── hmm_trained # Boolean
├── xgb_trained # Boolean
├── train_auc # AUC di data training
├── test_auc # AUC di data test
├── train_accuracy # Akurasi training
├── test_accuracy # Akurasi test
├── model_path # Path model disimpan
├── backup_path # Path backup model lama
├── success # Boolean
└── error_message # Pesan error (jika gagal)
+-- id # Auto-increment
+-- training_type # "daily" / "weekend"
+-- bars_used # 8000 / 15000
+-- num_boost_rounds # 50 / 80
+-- started_at # Timestamp mulai
+-- completed_at # Timestamp selesai
+-- duration_seconds # Durasi training
+-- hmm_trained # Boolean
+-- xgb_trained # Boolean
+-- train_auc # AUC di data training
+-- test_auc # AUC di data test
+-- train_accuracy # Akurasi training
+-- test_accuracy # Akurasi test
+-- model_path # Path model disimpan
+-- backup_path # Path backup model lama
+-- success # Boolean
+-- error_message # Pesan error (jika gagal)
```
### File Fallback
@@ -209,9 +255,9 @@ Table: training_runs
Jika PostgreSQL tidak tersedia:
```
data/retrain_history.txt
├── 2025-02-06T05:00:15+07:00
├── 2025-02-05T05:00:12+07:00
└── ... (append per retrain)
+-- 2025-02-06T05:00:15+07:00
+-- 2025-02-05T05:00:12+07:00
+-- ... (append per retrain)
```
---
@@ -256,18 +302,18 @@ if candle_count % 20 == 0: # Setiap 20 candle baru
| Data | 8.000 bar M15 (~83 hari) |
| Train/Test Split | 70% / 30% |
| XGBoost Rounds | 50 |
| Early Stopping | 5 rounds |
| *Early Stopping* | 5 rounds |
| HMM Regimes | 3 |
| HMM Lookback | 500 bar |
### Weekend Deep Training (Sabtu-Minggu)
### Weekend *Deep Training* (Sabtu-Minggu)
| Parameter | Nilai |
|-----------|-------|
| Data | 15.000 bar M15 (~156 hari) |
| Train/Test Split | 70% / 30% |
| XGBoost Rounds | 80 |
| Early Stopping | 5 rounds |
| *Early Stopping* | 5 rounds |
| HMM Regimes | 3 |
| HMM Lookback | 500 bar |
docs/arsitektur-ai/14-Backtest.md
+37 -49
View File
@@ -51,32 +51,29 @@ self.dynamic_confidence = create_dynamic_confidence()
Semua filter entry di-replikasi:
```
Untuk setiap bar dalam data historis:
|
v
[1] COOLDOWN: Jarak >= 20 bar dari trade terakhir? (~5 menit M15)
|YES
[2] SESSION: Bukan Off Hours (04:00-06:00 WIB)?
|YES
[3] GOLDEN TIME: (opsional) Hanya 19:00-23:00 WIB?
|YES
[4] REGIME: Bukan CRISIS?
|YES
[5] SMC SIGNAL: Ada signal dari SMCAnalyzer?
|YES
[6] DYNAMIC CONFIDENCE: Market quality bukan AVOID?
|YES
[7] ML THRESHOLD: Confidence >= threshold (50%-65%)?
|YES
[8] ML AGREEMENT: ML tidak strongly disagree (>65% berlawanan)?
|YES
[9] SIGNAL CONFIRMATION: Signal muncul 2x berturut?
|YES
[10] PULLBACK FILTER: Momentum tidak berlawanan?
|YES
v
EXECUTE SIMULATED TRADE
```mermaid
flowchart TD
START["Untuk setiap bar dalam data historis"] --> F1{"1. COOLDOWN\n>= 20 bar dari trade terakhir?"}
F1 -->|YES| F2{"2. SESSION\nBukan Off Hours 04:00-06:00?"}
F1 -->|NO| SKIP["SKIP"]
F2 -->|YES| F3{"3. GOLDEN TIME\nHanya 19:00-23:00? (opsional)"}
F2 -->|NO| SKIP
F3 -->|YES| F4{"4. REGIME\nBukan CRISIS?"}
F3 -->|NO| SKIP
F4 -->|YES| F5{"5. SMC SIGNAL\nAda signal?"}
F4 -->|NO| SKIP
F5 -->|YES| F6{"6. DYNAMIC CONFIDENCE\nBukan AVOID?"}
F5 -->|NO| SKIP
F6 -->|YES| F7{"7. ML THRESHOLD\nConfidence >= 50-65%?"}
F6 -->|NO| SKIP
F7 -->|YES| F8{"8. ML AGREEMENT\nTidak strongly disagree?"}
F7 -->|NO| SKIP
F8 -->|YES| F9{"9. SIGNAL CONFIRMATION\n2x berturut?"}
F8 -->|NO| SKIP
F9 -->|YES| F10{"10. PULLBACK FILTER\nMomentum tidak berlawanan?"}
F9 -->|NO| SKIP
F10 -->|YES| EXEC["EXECUTE SIMULATED TRADE"]
F10 -->|NO| SKIP
```
---
@@ -341,27 +338,18 @@ backtests/results/
## Data Flow
```
MT5 Connected
|
v
Fetch 50.000 bar M15 XAUUSD
|
v
FeatureEngineer.calculate_all() → 40+ fitur
SMCAnalyzer.calculate_all() → Struktur pasar
RegimeDetector.predict() → Regime label
|
v
Filter: Jan 2025 - Now
|
v
Loop setiap bar:
├── Entry check (10 filter)
├── Simulate exit (5 kondisi)
├── Record trade result
└── Update statistics
|
v
Print laporan + Save CSV
```mermaid
flowchart TD
A["MT5 Connected"] --> B["Fetch 50.000 bar M15 XAUUSD"]
B --> C["FeatureEngineer.calculate_all() → 40+ fitur\nSMCAnalyzer.calculate_all() → Struktur pasar\nRegimeDetector.predict() → Regime label"]
C --> D["Filter: Jan 2025 - Now"]
D --> E["Loop setiap bar"]
E --> E1["Entry check (14 filter)"]
E --> E2["Simulate exit (5 kondisi)"]
E --> E3["Record trade result"]
E --> E4["Update statistics"]
E1 --> F["Print laporan + Save CSV"]
E2 --> F
E3 --> F
E4 --> F
```
docs/arsitektur-ai/15-Dynamic-Confidence.md
+124 -72
View File
@@ -1,4 +1,4 @@
# Dynamic Confidence — Penyesuaian Threshold Otomatis
# *Dynamic Confidence* --- Penyesuaian *Threshold* Otomatis
> **File:** `src/dynamic_confidence.py`
> **Class:** `DynamicConfidenceManager`
@@ -6,21 +6,73 @@
---
## Apa Itu Dynamic Confidence?
## Apa Itu *Dynamic Confidence*?
Dynamic Confidence adalah sistem yang **menyesuaikan confidence threshold ML secara otomatis** berdasarkan kondisi pasar saat ini. Saat kondisi ideal, threshold diturunkan agar lebih banyak peluang. Saat kondisi buruk, threshold dinaikkan untuk lebih selektif.
*Dynamic Confidence* adalah sistem yang **menyesuaikan confidence *threshold* ML secara otomatis** berdasarkan kondisi pasar saat ini. Saat kondisi ideal, *threshold* diturunkan agar lebih banyak peluang. Saat kondisi buruk, *threshold* dinaikkan untuk lebih selektif.
**Analogi:** Dynamic Confidence seperti **termometer yang mengatur AC otomatis** — saat cuaca panas (pasar bagus), AC diset dingin (threshold rendah, lebih banyak trade). Saat cuaca dingin (pasar buruk), AC dimatikan (threshold tinggi, kurangi trade).
**Analogi:** *Dynamic Confidence* seperti **termometer yang mengatur AC otomatis** --- saat cuaca panas (pasar bagus), AC diset dingin (*threshold* rendah, lebih banyak trade). Saat cuaca dingin (pasar buruk), AC dimatikan (*threshold* tinggi, kurangi trade).
---
## Flowchart
```mermaid
flowchart TD
A["Kondisi Market Saat Ini"] --> B["6 Faktor Dianalisis"]
B --> F1["1. Session<br/>+/- 20 poin"]
B --> F2["2. Regime<br/>+/- 15 poin"]
B --> F3["3. Volatility<br/>+/- 10 poin"]
B --> F4["4. Trend Clarity<br/>+/- 10 poin"]
B --> F5["5. SMC Confluence<br/>+/- 10 poin"]
B --> F6["6. ML Alignment<br/>+/- 5 poin"]
F1 --> S["Score (0 - 100)"]
F2 --> S
F3 --> S
F4 --> S
F5 --> S
F6 --> S
S --> Q{"Quality Level?"}
Q -->|"Score >= 80"| E["EXCELLENT<br/>Threshold: 60%"]
Q -->|"Score 65-79"| G["GOOD<br/>Threshold: 65%"]
Q -->|"Score 50-64"| M["MODERATE<br/>Threshold: 70%"]
Q -->|"Score 35-49"| P["POOR<br/>Threshold: 80%"]
Q -->|"Score < 35"| AV["AVOID<br/>Threshold: 85%"]
E --> D{"ML Confidence<br/>>= Threshold?"}
G --> D
M --> D
P --> D
AV --> SKIP["SKIP --- Jangan Trade"]
D -->|"YES"| ENTRY["ENTRY Diizinkan"]
D -->|"NO"| WAIT["TUNGGU --- Confidence Kurang"]
style A fill:#4a90d9,color:#fff
style S fill:#f5a623,color:#fff
style Q fill:#7b68ee,color:#fff
style E fill:#27ae60,color:#fff
style G fill:#2ecc71,color:#fff
style M fill:#f39c12,color:#fff
style P fill:#e67e22,color:#fff
style AV fill:#e74c3c,color:#fff
style ENTRY fill:#27ae60,color:#fff
style WAIT fill:#e67e22,color:#fff
style SKIP fill:#e74c3c,color:#fff
```
---
## Prinsip Dasar
```
Market BAGUS (trending, session bagus) → Threshold RENDAH (60%) → Lebih banyak trade
Market BIASA (normal) → Threshold SEDANG (70%) → Trade normal
Market JELEK (choppy, low liquidity) → Threshold TINGGI (80%) → Sangat selektif
Market BERBAHAYA (crisis, weekend) → Threshold MAXIMUM (85%) → Hindari trading
Market BAGUS (trending, session bagus) --> Threshold RENDAH (60%) --> Lebih banyak trade
Market BIASA (normal) --> Threshold SEDANG (70%) --> Trade normal
Market JELEK (choppy, low liquidity) --> Threshold TINGGI (80%) --> Sangat selektif
Market BERBAHAYA (crisis, weekend) --> Threshold MAXIMUM (85%) --> Hindari trading
```
---
@@ -41,50 +93,50 @@ DynamicConfidenceManager(
Score dimulai dari **50** (tengah), lalu disesuaikan oleh 6 faktor:
### Faktor 1: Session (±20 poin)
### Faktor 1: *Session* (+/- 20 poin)
| Session | Poin | Alasan |
| *Session* | Poin | Alasan |
|---------|------|--------|
| London-NY Overlap / Golden | **+20** | Likuiditas tertinggi, spread rendah |
| London | **+15** | Volume tinggi |
| New York | **+10** | Volume tinggi |
| Asia/Tokyo | **+0** | Volatilitas rendah |
| Asia/Tokyo | **+0** | *Volatility* rendah |
| Market Closed/Weekend | **-30** | Tidak ada likuiditas |
| Lainnya | **+5** | Default |
### Faktor 2: Regime (±15 poin)
### Faktor 2: *Regime* (+/- 15 poin)
| Regime | Poin | Alasan |
| *Regime* | Poin | Alasan |
|--------|------|--------|
| Medium Volatility | **+15** | Kondisi ideal untuk trading |
| Low Volatility | **+5** | Hati-hati ranging |
| High Volatility | **-5** | Perlu lot kecil |
| Medium *Volatility* | **+15** | Kondisi ideal untuk trading |
| Low *Volatility* | **+5** | Hati-hati *ranging* |
| High *Volatility* | **-5** | Perlu lot kecil |
| Crisis | **-25** | Hindari trading |
### Faktor 3: Volatility (±10 poin)
### Faktor 3: *Volatility* (+/- 10 poin)
| Volatility | Poin | Alasan |
| *Volatility* | Poin | Alasan |
|-----------|------|--------|
| Medium | **+10** | Pergerakan cukup, bisa diprediksi |
| Low | **+0** | Pergerakan terlalu kecil |
| High | **-5** | Sulit diprediksi |
| Extreme | **-10** | Sangat berbahaya |
### Faktor 4: Trend Clarity (±10 poin)
### Faktor 4: Trend Clarity (+/- 10 poin)
| Trend | Poin | Alasan |
|-------|------|--------|
| Uptrend / Downtrend | **+10** | Arah jelas, sinyal lebih akurat |
| Neutral / Ranging | **-5** | Sinyal sering whipsaw |
| Uptrend / Downtrend (*trending*) | **+10** | Arah jelas, sinyal lebih akurat |
| Neutral / *Ranging* | **-5** | Sinyal sering whipsaw |
### Faktor 5: SMC Confluence (±10 poin)
### Faktor 5: SMC *Confluence* (+/- 10 poin)
| Kondisi | Poin | Alasan |
|---------|------|--------|
| Ada sinyal SMC (OB/FVG/BOS) | **+10** | Konfirmasi tambahan |
| Tidak ada sinyal | **+0** | Tanpa konfirmasi |
### Faktor 6: ML Alignment (±5 poin)
### Faktor 6: ML Alignment (+/- 5 poin)
| ML Confidence | Poin | Alasan |
|--------------|------|--------|
@@ -94,9 +146,9 @@ Score dimulai dari **50** (tengah), lalu disesuaikan oleh 6 faktor:
---
## Pemetaan Score ke Quality
## Pemetaan Score ke *Market Quality*
Score dihitung (0–100), lalu dipetakan ke **5 level kualitas**:
Score dihitung (0--100), lalu dipetakan ke **5 level kualitas**:
```
Score: 0 10 20 30 35 50 65 80 100
@@ -106,7 +158,7 @@ Score: 0 10 20 30 35 50 65 80 100
| thresh: 85% |80% | 70% |65% | 60%
```
| Score | Quality | Threshold | Aksi |
| Score | Quality | *Threshold* | Aksi |
|-------|---------|-----------|------|
| **80+** | EXCELLENT | 60% | Trade dengan percaya diri |
| **65-79** | GOOD | 65% | Trade normal |
@@ -123,48 +175,48 @@ Score: 0 10 20 30 35 50 65 80 100
```
Base score: 50
[+20] Session: London-NY Overlap → 70
[+15] Regime: Medium Volatility → 85
[+10] Volatility: Medium → 95
[+10] Trend: UPTREND → 105 → cap 100
[+10] SMC: Ada FVG + BOS → 100
[+5] ML: 72% confidence → 100
[+20] Session: London-NY Overlap --> 70
[+15] Regime: Medium Volatility --> 85
[+10] Volatility: Medium --> 95
[+10] Trend: UPTREND --> 105 --> cap 100
[+10] SMC: Ada FVG + BOS --> 100
[+5] ML: 72% confidence --> 100
Score: 100 → EXCELLENT → Threshold: 60%
Score: 100 --> EXCELLENT --> Threshold: 60%
```
**Artinya:** ML cukup confidence 60% saja untuk entry. Lebih banyak trade opportunity.
### Contoh 2: Kondisi Jelek (Score: 40)
### Contoh 2: Kondisi Jelek (Score: 50)
```
Base score: 50
[+0] Session: Asia → 50
[+5] Regime: Low Volatility → 55
[+0] Volatility: Low → 55
[-5] Trend: RANGING → 50
[+0] SMC: Tidak ada signal → 50
[+0] ML: 58% confidence → 50
[+0] Session: Asia --> 50
[+5] Regime: Low Volatility --> 55
[+0] Volatility: Low --> 55
[-5] Trend: RANGING --> 50
[+0] SMC: Tidak ada signal --> 50
[+0] ML: 58% confidence --> 50
Score: 50 → MODERATE → Threshold: 70%
Score: 50 --> MODERATE --> Threshold: 70%
```
**Artinya:** ML harus confidence 70% untuk entry. Lebih selektif.
### Contoh 3: Kondisi Berbahaya (Score: 15)
### Contoh 3: Kondisi Berbahaya (Score: 0)
```
Base score: 50
[-30] Session: Weekend → 20
[-25] Regime: Crisis → -5 → cap 0
[-10] Volatility: Extreme → 0
[-5] Trend: Ranging → 0
[+0] SMC: Tidak ada → 0
[+0] ML: 55% → 0
[-30] Session: Weekend --> 20
[-25] Regime: Crisis --> -5 --> cap 0
[-10] Volatility: Extreme --> 0
[-5] Trend: Ranging --> 0
[+0] SMC: Tidak ada --> 0
[+0] ML: 55% --> 0
Score: 0 → AVOID → Threshold: 85% (praktis tidak trade)
Score: 0 --> AVOID --> Threshold: 85% (praktis tidak trade)
```
---
@@ -172,7 +224,7 @@ Score: 0 → AVOID → Threshold: 85% (praktis tidak trade)
## Integrasi di Entry Flow
```python
# main_live.py — Step 6 dari 11 filter entry
# main_live.py --- Step 6 dari 11 filter entry
# 1. Analisis kondisi market
market_analysis = dynamic_confidence.analyze_market(
@@ -187,7 +239,7 @@ market_analysis = dynamic_confidence.analyze_market(
# 2. Cek quality
if market_analysis.quality == MarketQuality.AVOID:
return # SKIP — market tidak layak
return # SKIP --- market tidak layak
# 3. Cek apakah ML confidence memenuhi threshold dinamis
can_entry, reason = dynamic_confidence.get_entry_decision(
@@ -204,7 +256,7 @@ can_entry, reason = dynamic_confidence.get_entry_decision(
## Integrasi di Backtest
```python
# backtest_live_sync.py — identik dengan live
# backtest_live_sync.py --- identik dengan live
market_analysis = self.dynamic_confidence.analyze_market(
session=session_name,
@@ -229,9 +281,9 @@ def get_entry_decision(ml_confidence, analysis) -> (bool, str):
"""
Keputusan final entry berdasarkan analisis.
1. Quality == AVOID? → False (jangan trade)
2. ML confidence >= threshold? → True (entry OK)
3. ML confidence < threshold? → False (tunggu)
1. Quality == AVOID? --> False (jangan trade)
2. ML confidence >= threshold? --> True (entry OK)
3. ML confidence < threshold? --> False (tunggu)
"""
# Contoh output:
@@ -247,18 +299,18 @@ def get_entry_decision(ml_confidence, analysis) -> (bool, str):
```python
def get_threshold_summary(analysis) -> str:
"""
Output: "Market: EXCELLENT (score=95) → Threshold: 60%"
Output: "Market: EXCELLENT (score=95) --> Threshold: 60%"
"""
```
Contoh log di main_live.py:
```
[14:30] Market: EXCELLENT (score=95) → Threshold: 60%
[14:30] Market: EXCELLENT (score=95) --> Threshold: 60%
[14:35] Entry OK: ML 68% >= threshold 60% (score=95)
[15:00] Market: MODERATE (score=55) → Threshold: 70%
[15:00] Market: MODERATE (score=55) --> Threshold: 70%
[15:05] Wait: ML 62% < threshold 70% (need +8%)
[04:00] Market: AVOID (score=15) → Threshold: 85%
[04:00] Market: AVOID (score=15) --> Threshold: 85%
```
---
@@ -270,26 +322,26 @@ Kondisi Market Saat Ini
|
v
6 Faktor Dianalisis:
├── Session ±20 poin
├── Regime ±15 poin
├── Volatility ±10 poin
├── Trend ±10 poin
├── SMC ±10 poin
└── ML ±5 poin
+-- Session +/- 20 poin
+-- Regime +/- 15 poin
+-- Volatility +/- 10 poin
+-- Trend +/- 10 poin
+-- SMC +/- 10 poin
+-- ML +/- 5 poin
|
v
Score (0-100)
|
v
Quality Level:
├── EXCELLENT (80+) → Threshold 60%
├── GOOD (65-79) → Threshold 65%
├── MODERATE (50-64)→ Threshold 70%
├── POOR (35-49) → Threshold 80%
└── AVOID (<35) → Threshold 85% / SKIP
+-- EXCELLENT (80+) --> Threshold 60%
+-- GOOD (65-79) --> Threshold 65%
+-- MODERATE (50-64) --> Threshold 70%
+-- POOR (35-49) --> Threshold 80%
+-- AVOID (<35) --> Threshold 85% / SKIP
|
v
ML Confidence >= Threshold?
├── YES → ENTRY diizinkan
└── NO → TUNGGU
+-- YES --> ENTRY diizinkan
+-- NO --> TUNGGU
```
docs/arsitektur-ai/16-MT5-Connector.md
+139 -43
View File
@@ -1,4 +1,4 @@
# MT5 Connector — Jembatan ke MetaTrader 5
# *MT5 Connector* — Jembatan ke *MetaTrader* 5
> **File:** `src/mt5_connector.py`
> **Class:** `MT5Connector`, `MT5SimulationConnector`
@@ -6,11 +6,11 @@
---
## Apa Itu MT5 Connector?
## Apa Itu *MT5 Connector*?
MT5 Connector adalah **jembatan komunikasi** antara bot AI dan terminal MetaTrader 5. Semua interaksi dengan broker — ambil data harga, kirim order, cek posisi — dilakukan melalui modul ini.
*MT5 Connector* adalah **jembatan komunikasi** antara bot AI dan terminal *MetaTrader* 5. Semua interaksi dengan broker — ambil data harga, kirim order, cek posisi — dilakukan melalui modul ini.
**Analogi:** MT5 Connector seperti **penerjemah di bandara** — menerjemahkan perintah bot (Python) ke bahasa yang dipahami broker (MT5 API), dan sebaliknya.
**Analogi:** *MT5 Connector* seperti **penerjemah di bandara** — menerjemahkan perintah bot (Python) ke bahasa yang dipahami broker (MT5 API), dan sebaliknya.
---
@@ -21,41 +21,76 @@ MT5 Connector adalah **jembatan komunikasi** antara bot AI dan terminal MetaTrad
| `connect()` | Koneksi ke MT5 terminal | `bool` |
| `disconnect()` | Putus koneksi | - |
| `reconnect()` | Reconnect otomatis | `bool` |
| `ensure_connected()` | Cek & auto-reconnect | `bool` |
| `ensure_connected()` | Cek & *auto-reconnect* | `bool` |
| `get_market_data()` | Ambil data OHLCV | `pl.DataFrame` |
| `get_tick()` | Ambil harga real-time | `TickData` |
| `send_order()` | Kirim order BUY/SELL | `OrderResult` |
| `close_position()` | Tutup posisi | `OrderResult` |
| `get_open_positions()` | Cek posisi terbuka | `pl.DataFrame` |
| `get_symbol_info()` | Info simbol (spread, dll) | `Dict` |
| `get_multi_timeframe_data()` | Ambil data multi-timeframe | `Dict[str, pl.DataFrame]` |
---
## Koneksi & Auto-Reconnect
## Koneksi & *Auto-Reconnect*
```
connect(max_retries=3)
|
v
Shutdown koneksi lama (jika ada)
|
v
mt5.initialize(login, password, server)
|
v
Tunggu 2 detik (stabilisasi terminal)
|
v
Verifikasi: terminal_info() != None?
|
├── Ya → Cek terminal.connected?
│ ├── Ya → ✅ Connected!
│ └── Tidak → Tunggu 3 detik → Retry
│
└── Tidak → Exponential backoff (2s, 4s, 8s) → Retry
### Connection Flow
```mermaid
flowchart TD
A([Start connect]) --> B[Shutdown koneksi lama]
B --> C[mt5.initialize\nlogin, password, server]
C --> D{Initialize\nberhasil?}
D -- Ya --> E[Tunggu 2 detik\nstabilisasi terminal]
D -- Tidak --> K{Attempt\n< max_retries?}
E --> F{terminal_info\n!= None?}
F -- Ya --> G{terminal\n.connected?}
F -- Tidak --> J[Shutdown & retry]
G -- Ya --> H[Set _connected = True\nAmbil account_info\nSelect symbol XAUUSD]
G -- Tidak --> I[Tunggu 3 detik\nCek ulang terminal]
I --> G2{Masih belum\nconnected?}
G2 -- Ya --> J
G2 -- Tidak --> H
H --> Z([Connected!])
J --> K
K -- Ya --> L[Exponential backoff\n2s, 4s, 8s]
L --> B
K -- Tidak --> M([ConnectionError\nRaise exception])
style A fill:#4CAF50,color:#fff
style Z fill:#4CAF50,color:#fff
style M fill:#f44336,color:#fff
style H fill:#2196F3,color:#fff
```
### Auto-Reconnect
### Mekanisme *Auto-Reconnect*
```mermaid
flowchart TD
A([ensure_connected\ndipanggil]) --> B{Flag\n_connected?}
B -- False --> C[reconnect]
B -- True --> D[Cek mt5.account_info]
D --> E{Info\nvalid?}
E -- Ya --> F([Tetap connected\nReset attempt counter])
E -- Tidak --> G[Set _connected = False\nIncrement attempt]
G --> H{Attempt >\nmax 5?}
H -- Ya --> I{Sudah lewat\n60 detik cooldown?}
I -- Ya --> J[Reset attempt = 0]
I -- Tidak --> K([Return False\nMasih dalam cooldown])
J --> C
H -- Tidak --> C
C --> L{reconnect\nberhasil?}
L -- Ya --> M([Reconnected!\nReset attempt counter])
L -- Tidak --> N([Return False])
style A fill:#FF9800,color:#fff
style F fill:#4CAF50,color:#fff
style M fill:#4CAF50,color:#fff
style K fill:#f44336,color:#fff
style N fill:#f44336,color:#fff
```
**Detail *Auto-Reconnect*:**
```python
ensure_connected():
@@ -69,12 +104,14 @@ ensure_connected():
"""
```
Mekanisme *exponential backoff* memastikan bot tidak membombardir server broker dengan request berulang. Setiap kali koneksi gagal, waktu tunggu berlipat ganda (2s, 4s, 8s). Setelah 5 kali gagal berturut-turut, bot masuk fase *cooldown* selama 60 detik sebelum mencoba lagi.
---
## Data Fetching (Polars Native)
```python
get_market_data(symbol="XAUUSD", timeframe="M15", count=200)
get_market_data(symbol="XAUUSD", timeframe="M15", count=1000, max_retries=3)
```
**Proses:**
@@ -83,6 +120,12 @@ get_market_data(symbol="XAUUSD", timeframe="M15", count=200)
MT5 Terminal
|
v
ensure_connected() → auto-reconnect jika putus
|
v
mt5.symbol_select() → pastikan simbol aktif di Market Watch
|
v
mt5.copy_rates_from_pos() → numpy structured array
|
v
@@ -99,6 +142,8 @@ Cast types:
Return pl.DataFrame
```
> **Catatan penting:** Data dikonversi langsung dari NumPy structured array ke Polars DataFrame. Tidak ada konversi perantara via Pandas. Ini adalah optimisasi kritis untuk performa — menjaga target **< 50ms per loop**.
**Kolom output:**
| Kolom | Tipe | Keterangan |
@@ -130,19 +175,40 @@ send_order(
)
```
**Retry Logic:**
### Order Execution Flow dengan *Retry* Logic
```mermaid
flowchart TD
A([send_order\ndipanggil]) --> B[Ambil tick data\nmt5.symbol_info_tick]
B --> C{Tick\nvalid?}
C -- Tidak --> D([Return: Failed\nNo tick data])
C -- Ya --> E[Tentukan harga\nBUY → ask / SELL → bid]
E --> F[Build request:\naction, symbol, volume,\ntype, price, SL, TP,\ndeviation, magic]
F --> G[mt5.order_send]
G --> H{Result\n== None?}
H -- Ya --> I[Log error]
I --> P
H -- Tidak --> J{RETCODE?}
J -- 10009 DONE --> K([Order berhasil!\nReturn OrderResult])
J -- "10013-10016\nINVALID" --> L([Return: Failed\nNon-retryable error])
J -- 10027\nTRADE_DISABLED --> M([Raise RuntimeError\nAutoTrading off])
J -- "10004 REQUOTE\n10006 REJECT\nlainnya" --> N[Log warning\nTunggu 0.5 detik]
N --> P{Attempt\n< max_retries?}
P -- Ya --> Q[Refresh harga\nUlangi order]
Q --> G
P -- Tidak --> R([Return: Failed\nMax retries exceeded])
style A fill:#FF9800,color:#fff
style K fill:#4CAF50,color:#fff
style D fill:#f44336,color:#fff
style L fill:#f44336,color:#fff
style M fill:#f44336,color:#fff
style R fill:#f44336,color:#fff
```
Kirim order
|
├── RETCODE 10009 (DONE) → ✅ Success
|
├── RETCODE 10013-10016 (INVALID) → ❌ Non-retryable
|
├── RETCODE 10027 (TRADE DISABLED) → ❌ Raise error
|
└── RETCODE lain (requote/reject) → 🔄 Retry (max 3x)
```
Parameter `deviation` mengontrol toleransi *slippage* maksimum dalam poin. Jika harga bergeser melebihi batas ini saat eksekusi, broker akan menolak order (REQUOTE) dan bot akan melakukan *retry* otomatis dengan harga terbaru.
**Close Position** juga menggunakan logika *retry* yang sama — setiap attempt mengambil ulang harga terbaru untuk memastikan akurasi.
---
@@ -154,9 +220,10 @@ Kirim order
| `M5` | TIMEFRAME_M5 | 5 menit |
| `M15` | TIMEFRAME_M15 | **Utama** (execution) |
| `M30` | TIMEFRAME_M30 | 30 menit |
| `H1` | TIMEFRAME_H1 | 1 jam |
| `H1` | TIMEFRAME_H1 | 1 jam (EMA20 filter) |
| `H4` | TIMEFRAME_H4 | Trend analysis |
| `D1` | TIMEFRAME_D1 | 1 hari |
| `W1` | TIMEFRAME_W1 | 1 minggu |
---
@@ -165,19 +232,24 @@ Kirim order
| Code | Nama | Aksi |
|------|------|------|
| 10009 | DONE | Order berhasil |
| 10004 | REQUOTE | Retry |
| 10006 | REJECT | Retry |
| 10004 | REQUOTE | *Retry* — harga berubah |
| 10006 | REJECT | *Retry* — ditolak server |
| 10013 | INVALID | Stop, order salah |
| 10014 | INVALID_VOLUME | Stop, lot salah |
| 10015 | INVALID_PRICE | Stop, harga salah |
| 10016 | INVALID_STOPS | Stop, SL/TP salah |
| 10027 | TRADE_DISABLED | AutoTrading off |
| -10001 | COMMON_ERROR | Reconnect |
| -10002 | INVALID_PARAMS | Reconnect |
| -10003 | NO_CONNECTION | Reconnect |
| -10004 | NO_IPC | Reconnect |
| -1 | TERMINAL_CALL_FAILED | Reconnect |
Error code -10003, -10004, -10001, -10002, dan -1 termasuk dalam `CONNECTION_ERRORS` dan secara otomatis memicu mekanisme *auto-reconnect*.
---
## Simulation Mode
## *Simulation Mode*
```python
class MT5SimulationConnector(MT5Connector):
@@ -187,9 +259,14 @@ class MT5SimulationConnector(MT5Connector):
- connect() selalu berhasil
- get_market_data() generate data sintetis (random walk)
- Base price XAUUSD: $2000
- Berguna untuk development & unit testing
"""
```
*Simulation mode* memungkinkan pengembangan dan testing tanpa perlu koneksi ke terminal *MetaTrader* yang sebenarnya. Connector ini menghasilkan data OHLCV sintetis menggunakan random walk dari harga dasar $2000.
> **Catatan:** *Simulation mode* secara otomatis aktif jika library MetaTrader5 tidak terinstal di environment.
---
## Konfigurasi Koneksi
@@ -203,3 +280,22 @@ MT5Connector(
timeout=60000, # 60 detik timeout
)
```
### Context Manager Support
*MT5 Connector* mendukung penggunaan sebagai context manager:
```python
with MT5Connector(login, password, server) as mt5_conn:
data = mt5_conn.get_market_data("XAUUSD", "M15")
# Otomatis disconnect saat keluar blok
```
---
## Catatan Teknis
- **Polars, bukan Pandas:** Semua konversi data dari MT5 menggunakan Polars secara langsung. Tidak ada *connection pooling* atau konversi via Pandas.
- **Thread Safety:** *MT5 Connector* berjalan di satu thread utama. Library MT5 Python API tidak thread-safe, jadi semua operasi dilakukan secara sekuensial.
- **Password Security:** Password disimpan dengan prefix `_` (`self._password`) sebagai konvensi private attribute.
- **Symbol Pre-selection:** Setelah koneksi berhasil, simbol XAUUSD otomatis di-select di Market Watch untuk memastikan data siap diambil.
docs/arsitektur-ai/17-Configuration.md
+120 -124
View File
@@ -1,164 +1,160 @@
# Configuration — Pusat Pengaturan Bot
# Konfigurasi — Pusat Pengaturan Bot
> **File:** `src/config.py`
> **Class:** `TradingConfig`, `RiskConfig`, `SMCConfig`, `MLConfig`, `ThresholdsConfig`, `RegimeConfig`
> **Sumber:** Environment variables (`.env`)
> **Sumber:** *Environment variables* (`.env`)
---
## Apa Itu Configuration?
## Struktur Konfigurasi
Configuration adalah **pusat pengaturan** seluruh parameter bot — dari kredensial MT5 hingga threshold AI. Semua pengaturan otomatis menyesuaikan berdasarkan ukuran modal (small/medium).
**Analogi:** Configuration seperti **kokpit pesawat** — semua tombol dan dial pengaturan ada di satu tempat, dan bisa diubah sebelum "terbang" (trading).
---
## Capital Mode (Otomatis)
| Mode | Modal | Risk/Trade | Max Daily Loss | Leverage | Max Lot | Max Posisi | Timeframe |
|------|-------|-----------|----------------|----------|---------|-----------|-----------|
| **SMALL** | ≤ $10K | 1% | 3% | 1:100 | 0.05 | 3 | M15 |
| **MEDIUM** | > $10K | 0.5% | 2% | 1:30 | 2.0 | 5 | H1 |
```python
# Otomatis berdasarkan capital
if capital <= 10000:
mode = SMALL # Growth mode
else:
mode = MEDIUM # Preservation mode
```mermaid
graph TD
A[".env File"] --> B["TradingConfig.from_env()"]
B --> C["CapitalMode<br/>SMALL / MEDIUM"]
C -->|"≤ $10.000"| D["SMALL<br/>Risk 1%, Max Lot 0.05"]
C -->|"> $10.000"| E["MEDIUM<br/>Risk 0.5%, Max Lot 2.0"]
B --> F["RiskConfig"]
B --> G["SMCConfig"]
B --> H["MLConfig"]
B --> I["ThresholdsConfig"]
B --> J["RegimeConfig"]
```
---
## 6 Sub-Konfigurasi
## 2 Mode Kapital
### 1. RiskConfig
### Mode SMALL (≤ $10.000) — *Growth Mode*
```python
RiskConfig(
risk_per_trade=1.0, # 1% per trade ($50 dari $5K)
max_daily_loss=3.0, # 3% max daily loss ($150)
max_leverage=100, # 1:100
max_positions=3, # Max 3 posisi bersamaan
max_lot_size=0.05, # Max 0.05 lot
min_lot_size=0.01, # Min 0.01 lot
lot_step=0.01, # Increment 0.01
)
```
| Parameter | Nilai | Keterangan |
|-----------|-------|------------|
| `risk_per_trade` | **1.0%** | $50 risiko per *trade* (akun $5.000) |
| `max_daily_loss` | **3.0%** | Batas kerugian harian $150 |
| `max_leverage` | **1:100** | *Leverage* tinggi untuk akun kecil |
| `max_positions` | **3** | Maksimal 3 posisi bersamaan |
| `max_lot_size` | **0.05** | Batas atas *lot* per *trade* |
| `min_lot_size` | **0.01** | *Lot* minimum |
| `execution_timeframe` | **M15** | *Scalping* / *day trading* |
### 2. SMCConfig
### Mode MEDIUM (> $10.000) — *Preservation Mode*
```python
SMCConfig(
swing_length=5, # 5 bar untuk swing detection
fvg_min_gap_pips=2.0, # Min gap FVG: 2 pips
ob_lookback=10, # Order block lookback: 10 bar
bos_close_break=True, # Butuh close break untuk BOS
)
```
| Parameter | Nilai | Keterangan |
|-----------|-------|------------|
| `risk_per_trade` | **0.5%** | $250 risiko per *trade* (akun $50.000) |
| `max_daily_loss` | **2.0%** | Batas kerugian harian $1.000 |
| `max_leverage` | **1:30** | *Leverage* konservatif |
| `max_positions` | **5** | Lebih banyak diversifikasi |
| `max_lot_size` | **2.0** | Batas atas *lot* |
| `execution_timeframe` | **H1** | *Swing trading* |
| `trend_timeframe` | **H4** | Analisis *trend* jangka menengah |
### 3. MLConfig
---
```python
MLConfig(
model_path="models/xgboost_model.json",
confidence_threshold=0.65, # Min confidence untuk entry
retrain_frequency_days=7, # Retrain setiap 7 hari
lookback_periods=1000, # Data lookback
)
```
## *Thresholds* (Ambang Batas)
### 4. ThresholdsConfig
### *ML Confidence*
```python
ThresholdsConfig(
# ML Confidence
ml_min_confidence=0.65, # Minimum confidence
ml_entry_confidence=0.70, # Default entry
ml_high_confidence=0.75, # High confidence
ml_very_high_confidence=0.80, # Lot multiplier trigger
| Parameter | Nilai | Fungsi |
|-----------|-------|--------|
| `ml_min_confidence` | **0.65** | Minimum *confidence* untuk pertimbangkan sinyal |
| `ml_entry_confidence` | **0.70** | *Default confidence* untuk *entry* |
| `ml_high_confidence` | **0.75** | *High confidence* — sinyal kuat |
| `ml_very_high_confidence` | **0.80** | Sangat yakin — *lot multiplier* aktif |
# Risk
trend_reversal_confidence=0.75, # Trigger reversal close
protected_mode_threshold=0.80, # Enter protected mode
### *Dynamic Threshold*
# Profit/Loss (USD)
min_profit_to_secure=15.0, # Min profit to consider secure
good_profit_level=25.0, # Good profit
great_profit_level=40.0, # Take it!
| Parameter | Nilai | Kondisi |
|-----------|-------|---------|
| `dynamic_threshold_aggressive` | **0.65** | Pasar *trending* kuat |
| `dynamic_threshold_moderate` | **0.70** | Kondisi normal |
| `dynamic_threshold_conservative` | **0.75** | Pasar bergejolak |
# Timing
trade_cooldown_seconds=300, # 5 menit antar trade
loop_interval_seconds=30.0, # Main loop interval
### *Profit/Loss* ($)
# Session
sydney_lot_multiplier=0.5, # Sydney lot reduction
)
```
| Parameter | Nilai | Keterangan |
|-----------|-------|------------|
| `min_profit_to_secure` | **$15** | Mulai pertimbangkan *take profit* |
| `good_profit_level` | **$25** | Level profit bagus |
| `great_profit_level` | **$40** | *Hard take profit* — ambil profit |
### 5. RegimeConfig
### *Trading Timing*
```python
RegimeConfig(
n_regimes=3, # 3 HMM states
lookback_periods=500, # HMM training lookback
retrain_frequency=20, # Retrain setiap 20 bar
)
| Parameter | Nilai | Keterangan |
|-----------|-------|------------|
| `trade_cooldown_seconds` | **300** | 5 menit antar *trade* |
| `loop_interval_seconds` | **30** | Interval *main loop* |
| `sydney_lot_multiplier` | **0.5** | *Lot* dikurangi 50% saat Sydney |
---
## *Environment Variables* (`.env`)
```env
# MetaTrader 5 — WAJIB
MT5_LOGIN=12345678
MT5_PASSWORD=your_password
MT5_SERVER=YourBroker-Server
MT5_PATH=C:/Program Files/MetaTrader 5/terminal64.exe
# Telegram — OPSIONAL
TELEGRAM_BOT_TOKEN=bot123:ABC-DEF
TELEGRAM_CHAT_ID=123456789
# Trading
CAPITAL=5000 # Menentukan mode (SMALL/MEDIUM)
SYMBOL=XAUUSD # Pair yang diperdagangkan
# Override (opsional)
RISK_PER_TRADE=1.0 # Override risk per trade (%)
MAX_DAILY_LOSS_PERCENT=3.0
MAX_POSITION_SIZE=0.05
AI_CONFIDENCE_THRESHOLD=0.65
FLASH_CRASH_THRESHOLD=2.5
```
---
## Environment Variables (.env)
## Kalkulasi *Position Sizing*
| Variable | Contoh | Wajib | Keterangan |
|----------|--------|-------|------------|
| `MT5_LOGIN` | `12345678` | Ya | Akun MT5 |
| `MT5_PASSWORD` | `p@ssw0rd` | Ya | Password MT5 |
| `MT5_SERVER` | `BrokerName-Live` | Ya | Server broker |
| `MT5_PATH` | `C:\...\terminal64.exe` | Tidak | Path MT5 |
| `CAPITAL` | `5000` | Tidak | Modal ($5000 default) |
| `SYMBOL` | `XAUUSD` | Tidak | Simbol trading |
| `RISK_PER_TRADE` | `1.0` | Tidak | Override risk % |
| `MAX_DAILY_LOSS_PERCENT` | `3.0` | Tidak | Override daily loss |
| `AI_CONFIDENCE_THRESHOLD` | `0.65` | Tidak | Override ML threshold |
| `TELEGRAM_BOT_TOKEN` | `123:ABC...` | Tidak | Token Telegram |
| `TELEGRAM_CHAT_ID` | `-1001234...` | Tidak | Chat ID Telegram |
| `DB_HOST` | `localhost` | Tidak | PostgreSQL host |
| `DB_NAME` | `trading_db` | Tidak | Database name |
---
## Position Sizing (Kelly Criterion)
Bot menggunakan **metode *Half-Kelly Criterion*** untuk menghitung ukuran *lot*:
```python
def calculate_position_size(entry_price, stop_loss_price, balance):
"""
Risk-Constrained Kelly Criterion:
def calculate_position_size(self, entry_price, stop_loss_price, account_balance=None):
# 1. Hitung jumlah risiko dalam $
risk_amount = balance * (risk_per_trade / 100)
risk_amount = balance × risk% ($5000 × 1% = $50)
sl_pips = |entry - SL| / 0.1
lot = risk_amount / (sl_pips × pip_value)
lot × 0.5 (Half-Kelly untuk safety)
# 2. Hitung jarak SL dalam pips
sl_distance = abs(entry_price - stop_loss_price)
sl_pips = sl_distance / 0.1 # XAUUSD: 1 pip = $0.1
Clamp: min_lot ≤ lot ≤ max_lot
"""
# 3. Hitung lot size
lot_size = risk_amount / (sl_pips * pip_value_per_lot)
# 4. Apply Half-Kelly (keamanan)
lot_size *= 0.5
# 5. Round dan batasi
lot_size = round(lot_size / lot_step) * lot_step
lot_size = max(min_lot, min(lot_size, max_lot))
```
**Contoh:** Akun $5.000, SL 50 *pips*, risiko 1%:
- Risiko = $50
- *Lot* = $50 / (50 × $1) = 0.01 *lot* (setelah *Half-Kelly*)
---
## Validasi Otomatis
## Validasi Konfigurasi
```python
def _validate_required_settings(self):
"""Validasi environment variables wajib saat startup."""
# MT5_LOGIN harus ada dan > 0
# MT5_PASSWORD harus ada dan tidak kosong
# MT5_SERVER harus ada dan tidak kosong
# CAPITAL harus > 0
# Jika ada yang hilang → ValueError dengan pesan jelas
```
Saat TradingConfig dibuat:
|
v
_validate_required_settings():
├── MT5_LOGIN != 0?
├── MT5_PASSWORD tidak kosong?
├── MT5_SERVER tidak kosong?
└── Capital > 0?
|
├── Ada yang gagal → ValueError
└── Semua OK → _configure_by_capital()
```
Bot **tidak bisa berjalan** tanpa kredensial MT5 yang valid.
docs/arsitektur-ai/18-Trade-Logger.md
+64 -50
View File
@@ -2,15 +2,43 @@
> **File:** `src/trade_logger.py`
> **Class:** `TradeLogger`
> **Storage:** PostgreSQL (primary) + CSV (fallback)
> **Storage:** PostgreSQL (primary) + CSV (*fallback*)
---
## Apa Itu Trade Logger?
## Apa Itu *Trade Logger*?
Trade Logger mencatat **setiap trade, sinyal, dan kondisi pasar** secara otomatis ke database dan file CSV. Data ini digunakan untuk analisis performa, retraining ML model, dan debugging.
*Trade Logger* mencatat **setiap trade, sinyal, dan kondisi pasar** secara otomatis ke database dan file CSV. Data ini digunakan untuk analisis performa, retraining ML model, dan debugging.
**Analogi:** Trade Logger seperti **black box di pesawat** — merekam semua yang terjadi untuk analisis setelah penerbangan (trading).
**Analogi:** *Trade Logger* seperti ***black box* di pesawat** — merekam semua yang terjadi untuk analisis setelah penerbangan (trading).
---
## Alur *Dual Storage*
```mermaid
flowchart TD
A["Event Terjadi\n(trade / signal / snapshot)"] --> B[TradeLogger]
B --> C{DB tersedia?}
C -- Ya --> D["PostgreSQL\n(Primary)"]
C -- Ya --> E["CSV\n(Backup)"]
C -- Tidak --> E
D --> F["trades table\nsignals table\nmarket_snapshots\nbot_status"]
E --> G["data/trade_logs/\ntrades/ | signals/ | snapshots/\n(file bulanan YYYY_MM.csv)"]
style A fill:#2d333b,stroke:#adbac7,color:#adbac7
style B fill:#1f6feb,stroke:#58a6ff,color:#fff
style C fill:#3d444d,stroke:#adbac7,color:#adbac7
style D fill:#238636,stroke:#3fb950,color:#fff
style E fill:#9e6a03,stroke:#d29922,color:#fff
style F fill:#238636,stroke:#3fb950,color:#fff
style G fill:#9e6a03,stroke:#d29922,color:#fff
```
**Prinsip *dual storage*:**
- **DB tersedia?** — Tulis ke PostgreSQL **DAN** CSV (double safety)
- **DB tidak tersedia?** — CSV saja (*graceful degradation*)
- CSV **selalu** ditulis sebagai *fallback*, tidak peduli status DB
---
@@ -27,14 +55,14 @@ Setiap trade dibuka/ditutup dicatat lengkap:
| **Hasil** | profit_usd, profit_pips, duration_seconds |
| **Waktu** | open_time, close_time |
| **Market** | regime, volatility, session, spread, ATR |
| **SMC** | signal, confidence, reason, FVG/OB/BOS/CHoCH flags |
| **ML** | signal, confidence |
| **SMC** | *signal*, confidence, reason, FVG/OB/BOS/CHoCH flags |
| **ML** | *signal*, confidence |
| **Dynamic** | market_quality, market_score, threshold |
| **Exit** | exit_reason, exit_regime, exit_ml_signal |
| **Balance** | balance_before, balance_after, equity_at_entry |
| **Features** | JSON snapshot fitur saat entry & exit |
| **Features** | JSON *snapshot* fitur saat entry & exit |
### 2. Signal Record (Per Sinyal)
### 2. *Signal* Record (Per Sinyal)
Setiap sinyal yang dihasilkan (termasuk yang **tidak** dieksekusi):
@@ -47,9 +75,9 @@ trade_executed (bool)
execution_reason ("executed" / "below_threshold" / "max_positions" / ...)
```
### 3. Market Snapshot (Periodik)
### 3. Market *Snapshot* (Periodik)
Snapshot kondisi pasar secara berkala:
*Snapshot* kondisi pasar secara berkala:
```
timestamp, symbol, price, OHLC
@@ -62,56 +90,38 @@ features (JSON)
---
## Dual Storage
## *Dual Storage*
```
Event Terjadi (trade/signal/snapshot)
|
v
┌─────────────────────┐ ┌──────────────────┐
│ PostgreSQL (Primary) │ │ CSV (Fallback) │
│ │ │ │
│ ├── trades table │ │ data/trade_logs/│
│ ├── signals table │ │ ├── trades/ │
│ ├── market_snapshots │ │ │ └── trades_2025_02.csv
│ └── bot_status │ │ ├── signals/ │
│ │ │ │ └── signals_2025_02.csv
│ Cepat, queryable, │ │ └── snapshots/ │
│ thread-safe pooling │ │ └── snapshots_2025_02.csv
└─────────────────────┘ │ │
│ Selalu ditulis │
│ (backup) │
└──────────────────┘
```mermaid
flowchart TD
EV["Event Terjadi<br/>(trade / signal / snapshot)"] --> PG["PostgreSQL (Primary)<br/>trades, signals,<br/>market_snapshots, bot_status<br/>Cepat, queryable, thread-safe pooling"]
EV --> CSV["CSV (Fallback)<br/>data/trade_logs/<br/>trades/, signals/, snapshots/<br/>Selalu ditulis (backup)"]
```
- **DB tidak tersedia?** → CSV saja (graceful degradation)
- **DB tidak tersedia?** → CSV saja (*graceful degradation*)
- **DB tersedia?** → Tulis ke DB **DAN** CSV (double safety)
---
## Proses Log Trade
```mermaid
flowchart TD
OPEN["Trade Dibuka"] --> LOG_OPEN["log_trade_open()<br/>ticket, entry_price, regime, smc, ml"]
LOG_OPEN --> MEM["Simpan ke _pending_trades di memory"]
LOG_OPEN --> DB_INS["INSERT ke database (trades table)"]
MEM --> WAIT["... trading berjalan ..."]
DB_INS --> WAIT
WAIT --> CLOSE["Trade Ditutup"]
CLOSE --> LOG_CLOSE["log_trade_close()<br/>ticket, exit_price, profit, exit_reason"]
LOG_CLOSE --> FETCH["Ambil data pending dari memory"]
LOG_CLOSE --> DUR["Hitung durasi: close - open"]
FETCH --> UPD["UPDATE database<br/>(exit_price, profit, duration)"]
DUR --> UPD
UPD --> CSV["APPEND ke CSV<br/>(trades_YYYY_MM.csv)"]
```
Trade Dibuka:
|
v
log_trade_open(ticket, entry_price, regime, smc_*, ml_*, ...)
|
├── Simpan ke _pending_trades[ticket] (di memory)
└── INSERT ke database (trades table)
... trading berjalan ...
Trade Ditutup:
|
v
log_trade_close(ticket, exit_price, profit, exit_reason, ...)
|
├── Ambil data pending dari memory
├── Hitung durasi: close_time - open_time
├── UPDATE database (exit_price, profit, duration, ...)
└── APPEND ke CSV (trades_YYYY_MM.csv)
```
Data *pending* disimpan dalam dictionary `_pending_trades[ticket]` selama trade masih terbuka. Ketika trade ditutup, data entry digabung dengan data exit menjadi satu `TradeRecord` lengkap sebelum ditulis ke CSV.
---
@@ -125,9 +135,11 @@ log_trade_close(ticket, exit_price, profit, exit_reason, ...)
| `get_trades_for_training(30)` | Data untuk ML retraining |
| `get_stats()` | Statistik logger |
Setiap helper method mencoba query dari PostgreSQL terlebih dahulu. Jika DB tidak tersedia, otomatis *fallback* ke pembacaan file CSV — konsisten dengan prinsip *graceful degradation*.
---
## Thread Safety
## *Thread Safety*
```python
self._lock = threading.Lock()
@@ -137,6 +149,8 @@ with self._lock:
# Write to CSV
```
Semua operasi tulis ke file CSV dilindungi oleh `threading.Lock()` untuk menjamin *thread safety*. Ini mencegah korupsi data ketika multiple thread mencoba menulis ke file yang sama secara bersamaan (misalnya log trade close dan log *signal* terjadi hampir bersamaan).
---
## File CSV (Terorganisir per Bulan)
docs/arsitektur-ai/19-Position-Manager.md
+122 -16
View File
@@ -2,15 +2,15 @@
> **File:** `src/position_manager.py`
> **Class:** `SmartPositionManager`, `SmartMarketCloseHandler`
> **Fitur:** Trailing SL, Profit Protection, Market Close Handler
> **Fitur:** *Trailing Stop*, Profit Protection, *Market Close Handler*
---
## Apa Itu Position Manager?
## Apa Itu *Position Manager*?
Position Manager mengelola posisi terbuka secara **aktif dan cerdas** — trailing stop loss, proteksi profit, dan keputusan otomatis saat market mendekati penutupan.
*Position Manager* mengelola posisi terbuka secara **aktif dan cerdas** — *trailing stop* loss, proteksi profit, dan keputusan otomatis saat market mendekati penutupan.
**Analogi:** Position Manager seperti **co-pilot yang mengawasi perjalanan** — mengamankan keuntungan saat angin baik, dan mengambil tindakan darurat saat cuaca memburuk.
**Analogi:** *Position Manager* seperti **co-pilot yang mengawasi perjalanan** — mengamankan keuntungan saat angin baik, dan mengambil tindakan darurat saat cuaca memburuk.
---
@@ -18,17 +18,62 @@ Position Manager mengelola posisi terbuka secara **aktif dan cerdas** — traili
### A. SmartPositionManager
Mengelola posisi aktif: trailing SL, breakeven, profit protection.
Mengelola posisi aktif: *trailing stop*, *breakeven*, profit protection.
### B. SmartMarketCloseHandler
Keputusan cerdas saat market mendekati penutupan (harian/weekend).
Keputusan cerdas saat market mendekati penutupan (harian/weekend) — *market close handler*.
---
## SmartPositionManager — 7 Kondisi Aksi
Untuk setiap posisi terbuka, dicek berurutan:
Untuk setiap posisi terbuka, dicek berurutan berdasarkan prioritas:
```mermaid
flowchart TD
START([Posisi Terbuka]) --> C0{0. Market Close Check}
C0 -->|Profit >= $10\n+ dekat close| CLOSE0[CLOSE\nAmankan profit]
C0 -->|Loss + weekend\n+ SL >50% hit| CLOSE0W[CLOSE\nHindari gap risk]
C0 -->|Loss + weekend\n+ loss > $100| CLOSE0W
C0 -->|Tidak terpicu| C1
C1{1. Regime Danger?}
C1 -->|Crisis/High Vol\n+ profit > $50| CLOSE1[CLOSE\nAmankan dari volatilitas]
C1 -->|Tidak| C2
C2{2. Opposite Signal?}
C2 -->|Sinyal berlawanan kuat\n+ profit > $25| CLOSE2[CLOSE\nAmankan sebelum reversal]
C2 -->|Tidak| C3
C3{3. Drawdown from Peak?}
C3 -->|Peak > $50\n+ drawdown > 30%| CLOSE3[CLOSE\nProfit sudah turun]
C3 -->|Tidak| C4
C4{4. High Urgency?}
C4 -->|Urgency >= 7\n+ profit > 0| CLOSE4[CLOSE\nBanyak sinyal bahaya]
C4 -->|Tidak| C5
C5{5. Breakeven\nProtection?}
C5 -->|Profit >= BE pips| TRAIL5[TRAIL SL\nPindah ke breakeven]
C5 -->|Tidak| C6
C6{6. Trailing Stop?}
C6 -->|Profit >= trail pips| TRAIL6[TRAIL SL\nIkuti harga]
C6 -->|Tidak| HOLD
HOLD([7. Default: HOLD])
style CLOSE0 fill:#e74c3c,color:#fff
style CLOSE0W fill:#e74c3c,color:#fff
style CLOSE1 fill:#e74c3c,color:#fff
style CLOSE2 fill:#e74c3c,color:#fff
style CLOSE3 fill:#e74c3c,color:#fff
style CLOSE4 fill:#e74c3c,color:#fff
style TRAIL5 fill:#f39c12,color:#fff
style TRAIL6 fill:#f39c12,color:#fff
style HOLD fill:#27ae60,color:#fff
```
### 0. Market Close Check (Prioritas Tertinggi)
@@ -57,7 +102,7 @@ Posisi SELL + sinyal bullish kuat + profit > $25:
→ CLOSE (amankan sebelum reversal)
```
### 3. Drawdown from Peak
### 3. *Drawdown* from Peak
```
Peak profit > $50 DAN drawdown > 30% dari peak:
@@ -73,20 +118,23 @@ Urgency score >= 7 (dari 10) DAN profit > 0:
→ CLOSE (banyak sinyal bahaya bersamaan)
```
### 5. Breakeven Protection
### 5. *Breakeven* Protection
```
Profit >= 15 pips:
→ Pindah SL ke breakeven + 2 poin buffer
Profit >= BE pips (ATR * 2.0, fallback 15 pips):
→ Pindah SL ke breakeven + 0.5*ATR buffer
(tidak bisa rugi lagi)
```
### 6. Trailing Stop
### 6. *Trailing Stop*
```
Profit >= 25 pips:
→ SL mengikuti harga dengan jarak 10 pips
Profit >= trail start pips (ATR * 4.0, fallback 25 pips):
→ SL mengikuti harga dengan jarak ATR * 3.0 (fallback 10 pips)
(kunci profit sambil biarkan profit berjalan)
Impulse candle (range > 1.5x ATR):
→ Trail diperketat ke 1.5x ATR
```
### 7. Default: HOLD
@@ -99,6 +147,8 @@ Tidak ada kondisi terpenuhi → HOLD posisi
## SmartMarketCloseHandler
*Market close handler* menentukan aksi cerdas saat market mendekati penutupan harian atau weekend.
### Market Hours (XAUUSD)
```
@@ -109,6 +159,41 @@ Daily close: 05:00 WIB (= 17:00 EST hari sebelumnya)
Weekend close: Sabtu 05:00 WIB (= Jumat 17:00 EST)
```
### Diagram Keputusan
```mermaid
flowchart TD
START([Posisi Terbuka\nDekat Close?]) --> NEAR{Dekat market close?\n2 jam sebelum}
NEAR -->|Tidak| NORMAL([NORMAL\nLanjut trading biasa])
NEAR -->|Ya| WEEKEND{Dekat weekend?\nJumat < 30 menit}
WEEKEND -->|Tidak, daily close| PROFIT_D{Profit >= $10?}
PROFIT_D -->|Ya| CLOSE_P([CLOSE_PROFIT\nAmankan profit\nsebelum daily close])
PROFIT_D -->|Tidak, profit kecil| VCLOSE{< 30 menit\nke close?}
VCLOSE -->|Ya, profit > 0| CLOSE_P
VCLOSE -->|Tidak / loss| LOSS_D{Loss > $100?}
LOSS_D -->|Ya| CUT_D([CUT_LOSS\nLoss terlalu besar])
LOSS_D -->|Tidak| HOLD_D([HOLD_LOSS\nBisa recovery besok])
WEEKEND -->|Ya, weekend| PROFIT_W{Profit >= $10?}
PROFIT_W -->|Ya| CLOSE_W([CLOSE_PROFIT\nAmankan sebelum\nweekend])
PROFIT_W -->|Tidak, loss| SL_CHECK{SL > 50% hit?}
SL_CHECK -->|Ya| CUT_W([CUT_LOSS_WEEKEND\nHindari gap risk])
SL_CHECK -->|Tidak| BIG_LOSS{Loss > $100?}
BIG_LOSS -->|Ya| CUT_W
BIG_LOSS -->|Tidak| HOLD_W([HOLD_LOSS\nBisa recovery Senin])
style CLOSE_P fill:#27ae60,color:#fff
style CLOSE_W fill:#27ae60,color:#fff
style CUT_D fill:#e74c3c,color:#fff
style CUT_W fill:#e74c3c,color:#fff
style HOLD_D fill:#3498db,color:#fff
style HOLD_W fill:#3498db,color:#fff
style NORMAL fill:#95a5a6,color:#fff
```
### Konfigurasi
```python
@@ -126,13 +211,15 @@ SmartMarketCloseHandler(
| Rekomendasi | Kondisi | Aksi |
|-------------|---------|------|
| **CLOSE_PROFIT** | Profit + dekat close | Tutup, amankan profit |
| **CUT_LOSS_WEEKEND** | Loss besar + dekat weekend | Tutup, hindari gap |
| **CUT_LOSS_WEEKEND** | Loss besar + dekat weekend | Tutup, hindari *gap risk* |
| **HOLD_LOSS** | Loss kecil + dekat close | Hold, bisa recovery |
| **NORMAL** | Belum dekat close | Lanjut normal |
---
## Market Analysis (Urgency Score)
## Market Analysis (*Urgency Score*)
*Urgency score* menghitung tingkat bahaya pasar pada skala 0-10. Skor tinggi memicu exit otomatis.
```
Score dimulai dari 0, lalu ditambah:
@@ -147,17 +234,36 @@ Total max: ~10
Score >= 7 = HIGH URGENCY → tutup jika ada profit
```
| Komponen | Skor | Kondisi |
|----------|------|---------|
| Regime berbahaya | +3 | Crisis atau high volatility |
| ML sinyal berlawanan | +2 | Confidence > 75% + arah berlawanan posisi |
| RSI *overbought* | +2 | RSI > 75 (bahaya untuk posisi BUY) |
| RSI *oversold* | +2 | RSI < 25 (bahaya untuk posisi SELL) |
| Trend + momentum berlawanan | +3 | Trend dan momentum searah melawan posisi |
| **Threshold** | **>= 7** | **Tutup posisi jika ada profit** |
---
## Konfigurasi SmartPositionManager
```python
SmartPositionManager(
# Fallback jika ATR tidak tersedia
breakeven_pips=15.0, # Breakeven setelah 15 pips profit
trail_start_pips=25.0, # Mulai trailing setelah 25 pips
trail_step_pips=10.0, # Trail distance: 10 pips
# ATR-adaptive exit multipliers (#24B)
atr_be_mult=2.0, # Breakeven = ATR * 2.0
atr_trail_start_mult=4.0, # Trail start = ATR * 4.0
atr_trail_step_mult=3.0, # Trail step = ATR * 3.0
# Proteksi profit
min_profit_to_protect=50.0, # Min $50 untuk proteksi profit
max_drawdown_from_peak=30.0, # Max 30% drawdown dari peak
# Market close handler
enable_market_close_handler=True, # Aktifkan market close handler
min_profit_before_close=10.0, # Take profit $10+ sebelum close
max_loss_to_hold=100.0, # Hold loss sampai $100
docs/arsitektur-ai/20-Risk-Engine.md
+1 -1
View File
@@ -168,7 +168,7 @@ RiskEngine (modul ini)
SmartRiskManager (05-Risk-Management.md)
├── 4 trading modes (NORMAL/RECOVERY/PROTECTED/STOPPED)
├── Smart exit logic (10 kondisi)
├── Smart exit logic (12 kondisi)
├── Position monitoring per-detik
└── Higher-level risk decisions
```
docs/arsitektur-ai/21-Database.md
+202 -25
View File
@@ -1,41 +1,54 @@
# Database Module — PostgreSQL Integration
# *Database Module* — *PostgreSQL* Integration
> **File:** `src/db/connection.py`, `src/db/repository.py`
> **Database:** PostgreSQL
> **Library:** psycopg2 (connection pooling)
> **Database:** *PostgreSQL*
> **Library:** psycopg2 (*connection pooling*)
---
## Apa Itu Database Module?
## Apa Itu *Database Module*?
Database Module menyediakan **penyimpanan persisten** untuk semua data trading — trade history, training log, sinyal, snapshot pasar, dan status bot. Menggunakan PostgreSQL dengan connection pooling untuk performa tinggi.
*Database Module* menyediakan **penyimpanan persisten** untuk semua data trading — trade history, training log, sinyal, snapshot pasar, dan status bot. Menggunakan *PostgreSQL* dengan *connection pooling* untuk performa tinggi.
**Analogi:** Database Module seperti **arsip perpustakaan** — menyimpan semua catatan trading secara terorganisir, bisa dicari kapan saja, dan tidak hilang meski bot di-restart.
**Analogi:** *Database Module* seperti **arsip perpustakaan** — menyimpan semua catatan trading secara terorganisir, bisa dicari kapan saja, dan tidak hilang meski bot di-restart.
---
## Arsitektur
```
Bot Components
├── TradeLogger → TradeRepository, SignalRepository, MarketSnapshotRepository
├── AutoTrainer → TrainingRepository
├── main_live.py → BotStatusRepository, DailySummaryRepository
└── Dashboard → Semua repository (READ)
|
v
DatabaseConnection (Singleton)
|
v
ThreadedConnectionPool (1-10 koneksi)
|
v
PostgreSQL Server
```mermaid
graph TD
TL[TradeLogger] -->|write| TR[TradeRepository]
TL -->|write| SigR[SignalRepository]
TL -->|write| MSR[MarketSnapshotRepository]
AT[AutoTrainer] -->|write| TrR[TrainingRepository]
ML[main_live.py] -->|write| BSR[BotStatusRepository]
ML -->|write| DSR[DailySummaryRepository]
DASH[Dashboard] -.->|read| TR
DASH -.->|read| SigR
DASH -.->|read| MSR
DASH -.->|read| TrR
DASH -.->|read| BSR
DASH -.->|read| DSR
TR --> DC[DatabaseConnection<br/><i>Singleton</i>]
SigR --> DC
MSR --> DC
TrR --> DC
BSR --> DC
DSR --> DC
DC --> POOL[ThreadedConnectionPool<br/>1 – 10 koneksi]
POOL --> PG[(PostgreSQL Server)]
style DC fill:#2d6a4f,stroke:#1b4332,color:#fff
style PG fill:#1b4332,stroke:#081c15,color:#fff
style POOL fill:#40916c,stroke:#2d6a4f,color:#fff
```
---
## Connection (Singleton + Pooling)
## Connection (*Singleton* + Pooling)
```python
class DatabaseConnection:
@@ -49,6 +62,8 @@ class DatabaseConnection:
"""
```
`DatabaseConnection` menerapkan pola *singleton* yang *thread-safe* — hanya satu instance yang pernah dibuat selama proses berjalan. Akses ke database dilakukan melalui *context manager* (`with db.get_cursor() as cur`) sehingga koneksi selalu dikembalikan ke pool setelah selesai.
### Konfigurasi
```
@@ -68,7 +83,7 @@ from src.db import get_db, init_db
if init_db():
db = get_db()
# Query
# Query dengan context manager
with db.get_cursor() as cur:
cur.execute("SELECT * FROM trades WHERE profit_usd > 0")
rows = cur.fetchall()
@@ -79,7 +94,9 @@ if init_db():
---
## 6 Repository
## 6 *Repository*
Setiap *repository* bertanggung jawab atas satu tabel dan menyediakan method khusus untuk operasi CRUD.
### 1. TradeRepository
@@ -140,6 +157,164 @@ if init_db():
## Tabel Database
### Entity-Relationship Diagram
```mermaid
erDiagram
trades {
bigint ticket PK
varchar symbol
varchar direction
float entry_price
float exit_price
float stop_loss
float take_profit
float lot_size
float profit_usd
float profit_pips
timestamp opened_at
timestamp closed_at
int duration_seconds
varchar entry_regime
float entry_volatility
varchar entry_session
varchar smc_signal
float smc_confidence
text smc_reason
bool smc_fvg_detected
bool smc_ob_detected
bool smc_bos_detected
bool smc_choch_detected
varchar ml_signal
float ml_confidence
varchar market_quality
float market_score
float dynamic_threshold
varchar exit_reason
varchar exit_regime
varchar exit_ml_signal
float balance_before
float balance_after
float equity_at_entry
json features_entry
json features_exit
varchar bot_version
varchar trade_mode
}
training_runs {
serial id PK
varchar training_type
int bars_used
int num_boost_rounds
bool hmm_trained
int hmm_n_regimes
bool xgb_trained
float train_auc
float test_auc
float train_accuracy
float test_accuracy
varchar model_path
varchar backup_path
bool success
text error_message
timestamp started_at
timestamp completed_at
int duration_seconds
bool rolled_back
text rollback_reason
timestamp rollback_at
}
signals {
serial id PK
timestamp signal_time
varchar symbol
float price
varchar signal_type
varchar signal_source
float combined_confidence
varchar regime
varchar session
float volatility
float market_score
bool executed
text execution_reason
bigint trade_ticket FK
}
market_snapshots {
serial id PK
timestamp snapshot_time
varchar symbol
float price
float open
float high
float low
float close
varchar regime
float volatility
varchar session
float atr
float spread
varchar ml_signal
float ml_confidence
varchar smc_signal
float smc_confidence
int open_positions
float floating_pnl
json features
}
bot_status {
serial id PK
timestamp status_time
bool is_running
varchar status
int loop_count
float avg_execution_ms
int uptime_seconds
float balance
float equity
float margin_used
int open_positions
float floating_pnl
float daily_pnl
varchar risk_mode
varchar current_session
bool is_golden_time
}
daily_summaries {
date summary_date PK
int total_trades
int winning_trades
int losing_trades
int breakeven_trades
float gross_profit
float gross_loss
float net_profit
float start_balance
float end_balance
float win_rate
float profit_factor
float avg_win
float avg_loss
int sydney_trades
int tokyo_trades
int london_trades
int ny_trades
int golden_trades
int fvg_trades
int fvg_wins
int ob_trades
int ob_wins
}
trades ||--o{ signals : "trade_ticket"
daily_summaries ||--o{ trades : "summary_date covers opened_at"
```
### trades
```sql
@@ -214,7 +389,7 @@ if init_db():
---
## Graceful Degradation
## *Graceful Degradation*
```
PostgreSQL tersedia?
@@ -223,3 +398,5 @@ PostgreSQL tersedia?
Bot TIDAK pernah crash karena database.
```
*Graceful degradation* memastikan bot tetap beroperasi penuh meskipun *PostgreSQL* tidak tersedia. Semua operasi database dibungkus dengan `try/except` — jika koneksi gagal, data ditulis ke CSV sebagai fallback. Saat database kembali online, bot otomatis menggunakan koneksi pool kembali tanpa restart.
docs/arsitektur-ai/22-Train-Models.md
+36 -12
View File
@@ -6,11 +6,11 @@
---
## Apa Itu Train Models?
## Apa Itu *Train Models*?
Train Models adalah script **pelatihan awal** yang dijalankan sekali sebelum bot mulai trading. Mengambil data historis dari MT5, melatih HMM dan XGBoost, lalu menyimpan model ke file `.pkl`.
*Train Models* adalah script **pelatihan awal** yang dijalankan sekali sebelum bot mulai trading. Mengambil data historis dari MT5, melatih HMM dan XGBoost, lalu menyimpan model ke file `.pkl`.
**Analogi:** Train Models seperti **sekolah penerbangan** — melatih pilot (model AI) sebelum terbang pertama kali. Setelah itu, pelatihan rutin dilakukan oleh Auto Trainer (13).
**Analogi:** *Train Models* seperti **sekolah penerbangan** — melatih pilot (model AI) sebelum terbang pertama kali. Setelah itu, pelatihan rutin dilakukan oleh Auto Trainer (13).
---
@@ -24,6 +24,30 @@ python train_models.py
## Pipeline Training
```mermaid
flowchart TD
A[Load Config] --> B[Connect MT5]
B --> C[Fetch Data]
C --> D[Feature Engineering]
D --> E[Train HMM]
E --> F[Train XGBoost]
F --> G[Save Models]
A:::config
B:::mt5
C:::data
D:::data
E:::model
F:::model
G:::save
classDef config fill:#4a90d9,color:#fff
classDef mt5 fill:#50c878,color:#fff
classDef data fill:#f5a623,color:#fff
classDef model fill:#d0021b,color:#fff
classDef save fill:#7b68ee,color:#fff
```
```
1. LOAD CONFIG
├── get_config() dari .env
@@ -54,9 +78,9 @@ python train_models.py
7. TRAIN XGBOOST
├── TradingModel(confidence_threshold=0.60)
├── fit(train_ratio=0.7, boost_rounds=50, early_stop=5)
├── Log: top 10 feature importance
├── Walk-forward validation (train=500, test=50, step=50)
├── Log: avg train/test AUC, overfitting ratio
├── Log: top 10 *feature importance*
├── *Walk-forward* validation (train=500, test=50, step=50)
├── Log: avg train/test *AUC*, overfitting ratio
└── Save → models/xgboost_model.pkl
8. DISCONNECT
@@ -70,11 +94,11 @@ python train_models.py
|-----------|-------|------------|
| Data | 10.000 bar M15 | ~104 hari |
| Train/Test Split | 70% / 30% | Lebih banyak test data |
| XGBoost Rounds | 50 | Anti-overfitting |
| Early Stopping | 5 rounds | Stop lebih awal |
| XGBoost Rounds | 50 | *Anti-overfitting* |
| *Early Stopping* | 5 rounds | Stop lebih awal |
| HMM Regimes | 3 | Low/Medium/High volatility |
| HMM Lookback | 500 bar | Window training |
| Walk-forward Window | 500 train / 50 test | Validasi robustness |
| *Walk-forward* Window | 500 train / 50 test | Validasi robustness |
---
@@ -82,7 +106,7 @@ python train_models.py
```
models/
├── xgboost_model.pkl # Model XGBoost (binary classifier)
├── xgboost_model.pkl # Model XGBoost (*binary classifier*)
└── hmm_regime.pkl # Model HMM (regime detector)
data/
@@ -158,7 +182,7 @@ logs/
| **Kapan** | Manual, 1x | Otomatis, harian |
| **Data** | 10K bar | 8K (daily) / 15K (weekend) |
| **Backup** | Tidak | Ya (5 terakhir) |
| **Rollback** | Tidak | Ya (AUC < 0.52) |
| **Rollback** | Tidak | Ya (*AUC* < 0.60) |
| **Database** | Tidak | Ya (PostgreSQL) |
| **Walk-forward** | Ya | Tidak |
| *Walk-forward* | Ya | Tidak |
| **Tujuan** | Setup awal | Maintenance rutin |
docs/arsitektur-ai/23-Main-Live-Orchestrator.md
+190 -121
View File
@@ -1,21 +1,82 @@
# Main Live — Orchestrator Utama
# Main Live — *Orchestrator* Utama
> **File:** `main_live.py`
> **Class:** `TradingBot`
> **Runtime:** Async event loop (asyncio)
> **Mode:** Candle-based (analisis penuh hanya saat candle baru M15)
> **Runtime:** *Async event loop* (asyncio)
> **Mode:** *Candle-based* (analisis penuh hanya saat candle baru M15)
> **Target:** < 0.05 detik per iterasi analisis
---
## Apa Itu Main Live?
Main Live adalah **otak pusat** yang mengorkestrasi semua komponen bot. Menjalankan loop **candle-based** — analisis penuh hanya dijalankan saat candle M15 baru terbentuk, dengan pengecekan posisi setiap 10 detik di antara candle. Mengkoordinasikan 15+ komponen dari data fetching hingga order execution.
Main Live adalah **otak pusat** yang mengorkestrasi semua komponen bot. Menjalankan loop *candle-based* — analisis penuh hanya dijalankan saat candle M15 baru terbentuk, dengan pengecekan posisi setiap 10 detik di antara candle. Mengkoordinasikan 15+ komponen dari data fetching hingga order execution.
**Analogi:** Main Live seperti **konduktor orkestra** — tidak memainkan alat musik sendiri, tapi mengarahkan semua pemain (komponen) agar bermain harmonis pada waktu yang tepat.
---
## Diagram Alur Main Loop
```mermaid
flowchart TD
A[STARTUP] --> B{Candle Baru?}
B -- Ya --> C["Phase 1: DATA\nFetch 200 bar, Feature Eng,\nSMC, HMM, XGBoost"]
B -- Tidak --> G["Position Check Only\n(setiap ~10 detik)"]
G --> H[Sleep ~5 detik]
C --> D["Phase 2: MONITORING\nCek posisi terbuka,\n12 kondisi exit"]
D --> E["Phase 3: ENTRY\n14 Filter harus PASS,\nExecute trade"]
E --> F["Phase 4: PERIODIK\nAuto-retrain, Market update,\nHourly analysis, Daily summary"]
F --> H
H --> B
style A fill:#2d6a4f,color:#fff
style C fill:#1b4332,color:#fff
style D fill:#40916c,color:#fff
style E fill:#52b788,color:#000
style F fill:#74c69d,color:#000
style G fill:#b7e4c7,color:#000
style H fill:#d8f3dc,color:#000
```
---
## Diagram *Startup* / *Shutdown*
```mermaid
flowchart LR
subgraph STARTUP
direction TB
S1[Load .env Config] --> S2[Connect MT5\nmax 3 retry]
S2 --> S3[Load HMM Model]
S3 --> S4[Load XGBoost Model]
S4 --> S5[Init SmartRiskManager]
S5 --> S6[Init SessionFilter\nWIB timezone]
S6 --> S7[Init Telegram + Logger]
S7 --> S8[Init AutoTrainer]
S8 --> S9["Telegram: BOT STARTED"]
S9 --> S10[Mulai Main Loop]
end
subgraph SHUTDOWN
direction TB
X1[Signal SIGINT/SIGTERM] --> X2[Hentikan Loop]
X2 --> X3["Telegram: BOT STOPPED\n(balance, trades, uptime)"]
X3 --> X4[Disconnect MT5]
X4 --> X5[Close DB Connections]
X5 --> X6[Exit]
end
STARTUP -.->|"Runtime\n(loop berjalan)"| SHUTDOWN
style S1 fill:#2d6a4f,color:#fff
style S10 fill:#52b788,color:#000
style X1 fill:#9d0208,color:#fff
style X6 fill:#d00000,color:#fff
```
---
## Komponen yang Dimuat
```python
@@ -34,7 +95,10 @@ class TradingBot:
self.features = FeatureEngineer() # 40+ fitur
self.dynamic_confidence = DynamicConfidenceManager(...) # Threshold
self.session_filter = SessionFilter(...) # Waktu trading
self.news_agent = NewsAgent(...) # Monitor berita
# self.news_agent = NewsAgent(...) # NONAKTIF (line 64)
# -> Dikomentari karena backtest membuktikan
# News Agent merugikan $178 profit.
# ML model sudah menangani volatilitas.
# Risiko
self.smart_risk = SmartRiskManager(...) # Risk management
@@ -49,98 +113,123 @@ class TradingBot:
self.flash_crash_detector = FlashCrashDetector(...) # Proteksi
```
> **Catatan:** `NewsAgent` **NONAKTIF** — import dikomentari di `main_live.py` line 64 (`# DISABLED`). Backtest membuktikan News Agent justru mengurangi profit sebesar $178 karena ML model sudah cukup menangani volatilitas pasar. Variabel `self.news_agent` di-set `None` (line 165).
---
## Main Loop (Candle-Based)
## Main Loop (*Candle-Based*)
```
STARTUP:
Load models → Connect MT5 → Send Telegram startup
Load models -> Connect MT5 -> Send Telegram startup
|
v
LOOP UTAMA (cek setiap ~5 detik, analisis pada candle baru):
|
├── Fetch 2 bar terakhir → cek apakah candle baru terbentuk
+-- Fetch 2 bar terakhir -> cek apakah candle baru terbentuk
|
|===[CANDLE BARU? YA → FULL ANALYSIS]=================
|===[CANDLE BARU? YA -> FULL ANALYSIS]=================
|
| |===[PHASE 1: DATA]================================
| |
| ├── Fetch 200 bar M15 XAUUSD dari MT5
| ├── Feature Engineering (40+ fitur)
| ├── SMC Analysis (Swing, FVG, OB, BOS, CHoCH)
| ├── HMM Regime Detection
| └── XGBoost Prediction
| +-- Fetch 200 bar M15 XAUUSD dari MT5
| +-- Feature Engineering (40+ fitur)
| +-- SMC Analysis (Swing, FVG, OB, BOS, CHoCH)
| +-- HMM Regime Detection
| +-- XGBoost Prediction
| |
| |===[PHASE 2: MONITORING]===========================
| |
| ├── Cek posisi terbuka
| │ └── Untuk setiap posisi:
| │ ├── Update profit & momentum
| │ ├── 10 kondisi exit (smart_risk.evaluate_position)
| │ └── Jika should_close → tutup → log → Telegram
| +-- Cek posisi terbuka
| | +-- Untuk setiap posisi:
| | +-- Update profit & momentum
| | +-- 12 kondisi exit (smart_risk.evaluate_position)
| | +-- Jika should_close -> tutup -> log -> Telegram
| |
| ├── Position Manager (trailing SL, breakeven)
| │ └── Smart Market Close Handler
| +-- Position Manager (trailing SL, breakeven)
| | +-- Smart Market Close Handler
| |
| |===[PHASE 3: ENTRY]================================
| |
| ├── [1] Session Filter → boleh trading?
| ├── [2] Risk Mode → bukan STOPPED?
| ├── [3] SMC Signal → ada setup?
| ├── [4] ML Confidence → >= threshold?
| ├── [5] ML Agreement → tidak strongly disagree?
| ├── [6] Dynamic Quality → bukan AVOID?
| ├── [7] Confirmation → 2x candle berturut?
| ├── [8] Pullback Filter → momentum selaras? (v5: ATR-based)
| ├── [9] Cooldown → 5 menit sejak trade terakhir?
| ├── [10] Position Limit → < 2 posisi?
| ├── [11] Lot Size → > 0?
| ├── SEMUA PASS → Execute trade
| ├── Slippage validation (v5: cek harga aktual vs expected)
| ├── Partial fill check (v5: cek volume aktual vs requested)
| └── Register position (harga & volume AKTUAL) → Log → Telegram
| +-- [1] Flash Crash Guard -> tidak ada crash?
| +-- [2] Regime Filter -> bukan SLEEP / CRISIS?
| +-- [3] Risk Check -> equity & drawdown aman?
| +-- [4] Session Filter -> boleh trading (WIB)?
| +-- [5] SMC Signal -> ada setup?
| +-- [6] Signal Combination -> sinyal valid (quality != AVOID)?
| +-- [7] H1 Bias (#31B) -> H1 EMA20 selaras?
| +-- [8] Time Filter (#34A) -> bukan jam 9/21 WIB?
| +-- [9] Trade Cooldown -> cukup jeda sejak trade terakhir?
| +-- [10] Smart Risk Gate -> mode bukan STOPPED?
| +-- [11] Lot Size -> > 0 setelah kalkulasi?
| +-- [12] Position Limit -> < 2 posisi terbuka?
| +-- [13] Slippage Validation -> harga aktual vs expected
| +-- [14] Partial Fill Check -> volume aktual vs requested
| +-- SEMUA PASS -> Execute trade
| +-- Register position (harga & volume AKTUAL) -> Log -> Telegram
| |
| |===[PHASE 4: PERIODIK]=============================
| |
| ├── Setiap 20 candle (~5 jam M15): Cek auto-retrain
| ├── Setiap 30 menit: Market update (Telegram)
| ├── Setiap 1 jam: Hourly analysis (Telegram)
| ├── Pergantian hari: Daily summary + reset
| └── News Agent: Monitor (non-blocking)
| +-- Setiap 20 candle (~5 jam M15): Cek auto-retrain
| +-- Setiap 30 menit: Market update (Telegram)
| +-- Setiap 1 jam: Hourly analysis (Telegram)
| +-- Pergantian hari: Daily summary + reset
|
|===[CANDLE BARU? TIDAK → POSITION CHECK ONLY]========
|===[CANDLE BARU? TIDAK -> POSITION CHECK ONLY]========
|
├── Setiap 10 detik: cek posisi terbuka saja
│ ├── Fetch 50 bar (minimal data)
│ ├── Hitung fitur untuk ML check
│ └── Evaluasi exit conditions per posisi
+-- Setiap 10 detik: cek posisi terbuka saja
| +-- Fetch 50 bar (minimal data)
| +-- Hitung fitur untuk ML check
| +-- Evaluasi exit conditions per posisi
|
v
Tunggu ~5 detik → Loop lagi
Tunggu ~5 detik -> Loop lagi
```
---
## Startup Sequence
## Detail 14 *Entry* Filter
| # | Filter | Deskripsi | Sumber |
|---|--------|-----------|--------|
| 1 | Flash Crash Guard | Deteksi pergerakan harga ekstrem (>X%) dalam 5 bar terakhir | `FlashCrashDetector` |
| 2 | Regime Filter | Blok *entry* jika HMM regime = SLEEP atau CRISIS | `regime_detector` |
| 3 | Risk Check | Validasi equity, drawdown, dan balance aman | `risk_engine` |
| 4 | Session Filter | Hanya trading di sesi aktif (Sydney/London/NY, WIB) | `session_filter` |
| 5 | SMC Signal | Harus ada setup SMC (Order Block, FVG, BOS, CHoCH) | `smc.generate_signal()` |
| 6 | Signal Combination | Gabung SMC + ML, blok jika market quality AVOID | `_combine_signals()` |
| 7 | H1 Bias (#31B) | BUY hanya jika H1 BULLISH, SELL hanya jika H1 BEARISH | `_get_h1_bias()` |
| 8 | Time Filter (#34A) | Skip jam 9 dan 21 WIB (likuiditas rendah / whipsaw) | Hardcoded WIB check |
| 9 | Trade Cooldown | Jeda minimum antar trade (mencegah overtrade) | `_last_trade_time` |
| 10 | Smart Risk Gate | Cek mode risk (NORMAL/CAUTIOUS/STOPPED) | `smart_risk` |
| 11 | Lot Size > 0 | Pastikan kalkulasi lot menghasilkan size > 0 | `smart_risk.calculate_lot_size()` |
| 12 | Position Limit | Maksimal 2 posisi terbuka bersamaan | `smart_risk.can_open_position()` |
| 13 | Slippage Validation | Cek harga eksekusi aktual vs harga yang diharapkan | Post-execution check |
| 14 | Partial Fill Check | Cek volume aktual yang terisi vs volume yang diminta | Post-execution check |
---
## *Startup* Sequence
```
1. Load konfigurasi dari .env
2. Connect ke MT5 (max 3 retry)
3. Load model HMM dari models/hmm_regime.pkl
4. Load model XGBoost dari models/xgboost_model.pkl
5. Initialize SmartRiskManager (set balance, limits)
6. Initialize SessionFilter (WIB timezone)
7. Initialize TelegramNotifier
8. Initialize TradeLogger
9. Initialize AutoTrainer
1. Load konfigurasi dari .env
2. Connect ke MT5 (max 3 retry)
3. Load model HMM dari models/hmm_regime.pkl
4. Load model XGBoost dari models/xgboost_model.pkl
5. Initialize SmartRiskManager (set balance, limits)
6. Initialize SessionFilter (WIB timezone)
7. Initialize TelegramNotifier
8. Initialize TradeLogger
9. Initialize AutoTrainer
10. Send Telegram: "BOT STARTED" (config, balance, risk settings)
11. Mulai main loop
```
> **Note:** NewsAgent **tidak** diinisialisasi saat *startup* — import dan inisialisasi dikomentari sejak backtest membuktikan kerugian $178.
---
## Shutdown Sequence
## *Shutdown* Sequence
```
1. Signal SIGINT/SIGTERM diterima
@@ -153,7 +242,7 @@ LOOP UTAMA (cek setiap ~5 detik, analisis pada candle baru):
---
## Error Handling
## Error Handling (*Fault Tolerance*)
```
Setiap iterasi loop dibungkus try-except:
@@ -161,22 +250,24 @@ Setiap iterasi loop dibungkus try-except:
try:
# Fetch data, analyze, trade
except ConnectionError:
# MT5 disconnected → reconnect()
# MT5 disconnected -> reconnect()
except Exception as e:
# Log error → lanjut loop berikutnya
# Log error -> lanjut loop berikutnya
# Bot TIDAK crash dari error tunggal
Prinsip: NEVER STOP TRADING karena error non-kritis
```
Bot dirancang dengan prinsip *fault tolerance* — satu error tidak menghentikan seluruh sistem. Setiap iterasi loop dibungkus `try-except` sehingga error pada satu candle tidak mempengaruhi candle berikutnya. Koneksi MT5 yang putus akan otomatis di-reconnect.
---
## Timer Periodik
## Timer *Periodik*
| Event | Interval | Aksi |
|-------|----------|------|
| Full analysis + entry | Setiap candle baru M15 | Saat candle terbentuk |
| Position monitoring | ~10 detik | Di antara candle |
| Full analysis + *entry* | Setiap candle baru M15 | Saat candle terbentuk |
| Position *monitoring* | ~10 detik | Di antara candle |
| Performance logging | 4 candle (~1 jam) | `loop_count % 4` |
| Auto-retrain check | 20 candle (~5 jam) | `loop_count % 20` |
| Market update Telegram | 30 menit | Timer |
@@ -191,23 +282,23 @@ Prinsip: NEVER STOP TRADING karena error non-kritis
Target: < 0.05 detik per iterasi analisis (50ms)
Full Analysis (saat candle baru):
├── MT5 data fetch: ~10ms (200 bar)
├── Feature engineering: ~5ms (Polars, vectorized)
├── SMC analysis: ~5ms (Polars native)
├── HMM predict: ~2ms
├── XGBoost predict: ~3ms
├── Position monitoring: ~5ms
├── Entry logic: ~5ms
└── Overhead: ~15ms
+-- MT5 data fetch: ~10ms (200 bar)
+-- Feature engineering: ~5ms (Polars, vectorized)
+-- SMC analysis: ~5ms (Polars native)
+-- HMM predict: ~2ms
+-- XGBoost predict: ~3ms
+-- Position monitoring: ~5ms
+-- Entry logic: ~5ms
+-- Overhead: ~15ms
------
~50ms total
Position Check Only (di antara candle):
├── MT5 data fetch: ~5ms (50 bar saja)
├── Feature engineering: ~3ms
├── ML prediction: ~3ms
├── Position evaluation: ~5ms
└── Overhead: ~5ms
+-- MT5 data fetch: ~5ms (50 bar saja)
+-- Feature engineering: ~3ms
+-- ML prediction: ~3ms
+-- Position evaluation: ~5ms
+-- Overhead: ~5ms
------
~21ms total
```
@@ -216,48 +307,26 @@ Position Check Only (di antara candle):
## Hubungan Semua Komponen
```
┌─────────────────────────────────────────────────────────┐
│ main_live.py │
│ (TradingBot) │
│ │
│ ┌─────────┐ ┌─────────┐ ┌──────────┐ ┌──────────┐ │
│ │ MT5 │ │ Feature │ │ SMC │ │ HMM │ │
│ │Connector│→ │ Eng │→ │ Analyzer │→ │ Detector │ │
│ └─────────┘ └─────────┘ └──────────┘ └──────────┘ │
│ ↑ ↓ ↓ │
│ │ ┌──────────────────────┐ │
│ │ │ Dynamic Confidence │ │
│ │ └──────────────────────┘ │
│ │ ↓ │
│ │ ┌──────────────────┐ │
│ │ │ XGBoost Model │ │
│ │ └──────────────────┘ │
│ │ ↓ │
│ │ ┌─────────────────────────────────┐ │
│ │ │ Entry Logic (11 Filters) │ │
│ │ │ Session, Risk, SMC, ML, ... │ │
│ │ └─────────────────────────────────┘ │
│ │ ↓ │
│ │ ┌────────────┐ ┌───────────────┐ │
│ ├────│ Risk Engine│ │Smart Risk Mgr │ │
│ │ └────────────┘ └───────────────┘ │
│ │ ↓ │
│ │←── Execute Order (BUY/SELL) │
│ │ ↓ │
│ │ ┌────────────┐ ┌───────────────┐ │
│ │ │ Position │ │ Trade Logger │ │
│ │ │ Manager │ │ (DB + CSV) │ │
│ │ └────────────┘ └───────────────┘ │
│ │ ↓ │
│ │ ┌────────────┐ ┌───────────────┐ │
│ │ │ Telegram │ │ Auto Trainer │ │
│ │ │ Notifier │ │ (retraining) │ │
│ │ └────────────┘ └───────────────┘ │
│ │ │
│ │ ┌────────────┐ ┌───────────────┐ │
│ │ │News Agent │ │Session Filter │ │
│ │ │(monitor) │ │(waktu trading)│ │
│ │ └────────────┘ └───────────────┘ │
└─────────────────────────────────────────────────────────┘
```mermaid
flowchart TD
subgraph BOT["main_live.py — TradingBot"]
direction TB
MT5["MT5 Connector"] --> FE["Feature Eng"]
FE --> SMC["SMC Analyzer"]
SMC --> HMM["HMM Detector"]
HMM --> DC["Dynamic Confidence"]
DC --> XGB["XGBoost Model"]
XGB --> ENTRY["Entry Logic (14 Filters)<br/>Flash, Regime, Risk, Session,<br/>SMC, H1 Bias, Time, Cooldown,<br/>Smart Risk, Lot, Pos Limit"]
ENTRY --> RE["Risk Engine"]
ENTRY --> SRM["Smart Risk Mgr"]
RE --> EXEC["Execute Order<br/>(BUY/SELL)"]
SRM --> EXEC
EXEC --> MT5
EXEC --> PM["Position Manager"]
EXEC --> TL["Trade Logger<br/>(DB + CSV)"]
EXEC --> TG["Telegram Notifier"]
PM ~~~ AT["Auto Trainer<br/>(retraining)"]
TG ~~~ NA["News Agent<br/>(NONAKTIF)"]
AT ~~~ SF["Session Filter<br/>(waktu trading)"]
end
```
docs/arsitektur-ai/README.md
+58 -140
View File
@@ -1,152 +1,70 @@
# Arsitektur AI — Smart Trading Bot
# Dokumentasi Arsitektur — XAUBot AI
> Dokumentasi lengkap semua komponen AI dan sistem pendukung.
**[ARSITEKTUR LENGKAP (1 Dokumen)](00-ARSITEKTUR-LENGKAP.md)** — Seluruh arsitektur bot dalam 1 file komprehensif: pipeline data, 11 filter entry, 10 kondisi exit, 4 lapis proteksi risiko, AI/ML engine, SMC, position lifecycle, auto-retraining, database, konfigurasi, dan error handling.
> Panduan lengkap arsitektur dan komponen sistem *trading bot* otomatis XAUUSD.
---
## Daftar Komponen
## Daftar Dokumen
### Inti AI & Analisis
| # | Komponen | File Source | Fungsi |
|---|----------|------------|--------|
| 1 | [HMM Regime Detector](01-HMM-Regime-Detector.md) | `src/regime_detector.py` | Deteksi kondisi pasar (radar cuaca) |
| 2 | [XGBoost Signal Predictor](02-XGBoost-Signal-Predictor.md) | `src/ml_model.py` | Prediksi arah harga (navigator AI) |
| 3 | [SMC Analyzer](03-SMC-Analyzer.md) | `src/smc_polars.py` | Analisis struktur pasar institusi (peta jalan) |
| 4 | [Feature Engineering](04-Feature-Engineering.md) | `src/feature_eng.py` | Pengolahan data mentah ke fitur ML (alat ukur) |
### Proteksi & Manajemen Risiko
| # | Komponen | File Source | Fungsi |
|---|----------|------------|--------|
| 5 | [Risk Management](05-Risk-Management.md) | `src/smart_risk_manager.py` | Perlindungan modal (sabuk pengaman) |
| 6 | [Session Filter](06-Session-Filter.md) | `src/session_filter.py` | Pengaturan waktu trading (jadwal kerja) |
| 7 | [Stop Loss (S/L)](07-Stop-Loss.md) | Multi-file | Proteksi 4 lapis dari kerugian |
| 8 | [Take Profit (T/P)](08-Take-Profit.md) | Multi-file | Pengambilan profit cerdas 6 layer |
### Proses Trading
| # | Komponen | File Source | Fungsi |
|---|----------|------------|--------|
| 9 | [Entry Trade](09-Entry-Trade.md) | `main_live.py` | Proses masuk posisi (11 filter) |
| 10 | [Exit Trade](10-Exit-Trade.md) | `main_live.py` | Proses keluar posisi (10 kondisi) |
### Koneksi & Konfigurasi
| # | Komponen | File Source | Fungsi |
|---|----------|------------|--------|
| 16 | [MT5 Connector](16-MT5-Connector.md) | `src/mt5_connector.py` | Jembatan ke broker MT5 (auto-reconnect) |
| 17 | [Configuration](17-Configuration.md) | `src/config.py` | Konfigurasi terpusat (6 sub-config) |
### Pendukung
| # | Komponen | File Source | Fungsi |
|---|----------|------------|--------|
| 11 | [News Agent](11-News-Agent.md) | `src/news_agent.py` | Monitoring berita ekonomi |
| 12 | [Telegram Notifications](12-Telegram-Notifications.md) | `src/telegram_notifier.py` | Notifikasi real-time ke Telegram |
| 18 | [Trade Logger](18-Trade-Logger.md) | `src/trade_logger.py` | Pencatatan trade dual-storage (DB + CSV) |
| 19 | [Position Manager](19-Position-Manager.md) | `src/position_manager.py` | Manajemen posisi aktif (trailing, breakeven) |
| 20 | [Risk Engine](20-Risk-Engine.md) | `src/risk_engine.py` | Mesin risiko & circuit breaker (Kelly Criterion) |
| 21 | [Database](21-Database.md) | `src/db/` | PostgreSQL integration (6 repository) |
### Training & Validasi
| # | Komponen | File Source | Fungsi |
|---|----------|------------|--------|
| 13 | [Auto Trainer](13-Auto-Trainer.md) | `src/auto_trainer.py` | Retraining model otomatis (pelatih malam) |
| 14 | [Backtest](14-Backtest.md) | `backtests/backtest_live_sync.py` | Simulasi trading 100% sync dengan live |
| 15 | [Dynamic Confidence](15-Dynamic-Confidence.md) | `src/dynamic_confidence.py` | Penyesuaian threshold otomatis (termometer) |
| 22 | [Train Models](22-Train-Models.md) | `train_models.py` | Script training awal (HMM + XGBoost) |
### Orchestrator
| # | Komponen | File Source | Fungsi |
|---|----------|------------|--------|
| 23 | [Main Live Orchestrator](23-Main-Live-Orchestrator.md) | `main_live.py` | Otak pusat bot, koordinasi semua komponen |
| # | Dokumen | Deskripsi |
|---|---------|-----------|
| 00 | **Arsitektur Lengkap** | Gambaran besar seluruh sistem — *data flow*, komponen, dan interaksi |
| 01 | **HMM *Regime Detector*** | Deteksi kondisi pasar menggunakan *Hidden Markov Model* 3 *state* |
| 02 | **XGBoost *Signal Predictor*** | Model *machine learning* untuk prediksi BUY/SELL/HOLD |
| 03 | **SMC *Analyzer*** | Analisis *Smart Money Concepts* — *Order Block*, FVG, BOS, CHoCH |
| 04 | ***Feature Engineering*** | 37 fitur teknikal — RSI, ATR, MACD, *Bollinger*, dll |
| 05 | **Manajemen Risiko** | Sistem manajemen risiko dinamis dengan mode kapital |
| 06 | **Filter Sesi** | Filter sesi perdagangan — Sydney, London, New York (WIB) |
| 07 | ***Stop Loss*** | Proteksi SL berbasis ATR dan *broker-level* |
| 08 | ***Take Profit*** | Target TP multi-level dengan ATR dan struktur pasar |
| 09 | ***Entry Trade*** | 14 filter *entry* dan logika eksekusi perdagangan |
| 10 | ***Exit Trade*** | 12 kondisi *exit* termasuk *trailing* SL, batas waktu, perubahan *regime* |
| 11 | ***News Agent*** | Filter berita ekonomi dan penilaian dampak *(saat ini nonaktif)* |
| 12 | **Notifikasi Telegram** | Notifikasi *trade* dan ringkasan harian via Telegram |
| 13 | ***Auto Trainer*** | *Pipeline retraining* otomatis saat kondisi pasar berubah |
| 14 | ***Backtest*** | *Framework backtesting* yang disinkronkan dengan logika *live* |
| 15 | ***Dynamic Confidence*** | Ambang batas *confidence* adaptif berdasarkan kondisi pasar |
| 16 | **Konektor MT5** | Lapisan koneksi *MetaTrader 5* dan eksekusi *order* |
| 17 | **Konfigurasi** | Konfigurasi *trading*, mode kapital, dan pengaturan *environment* |
| 18 | ***Trade Logger*** | Pencatatan *trade* ke *database* PostgreSQL |
| 19 | ***Position Manager*** | Pelacakan dan manajemen posisi terbuka |
| 20 | ***Risk Engine*** | Perhitungan risiko, *Kelly criterion*, dan *position sizing* |
| 21 | ***Database*** | Skema PostgreSQL dan penyimpanan data perdagangan |
| 22 | ***Train Models*** | *Pipeline* pelatihan model dan optimasi *hyperparameter* |
| 23 | **Orkestrator Utama** | *Async main loop* — inti dari *trading bot* |
---
## Pipeline Lengkap
## Diagram Arsitektur
```
Raw OHLCV dari MT5
|
v
[Feature Engineering] -> 40+ fitur numerik (RSI, ATR, MACD, BB, EMA, ...)
|
v
[SMC Analyzer] -> Swing, FVG, OB, BOS, CHoCH, Liquidity
| + Signal (entry, SL ATR-based, TP ATR-capped)
|
+---+---+
| |
v v
[HMM] [XGBoost]
Regime Signal
| |
+---+---+
|
v
[Signal Combination] -> SMC + ML harus setuju
|
v
[News Agent] -> Monitor berita (tidak blocking)
|
v
[Session Filter] -> Cek waktu boleh trading?
|
v
[ENTRY TRADE] -> 11 filter harus PASS:
| Session, Risk Mode, SMC Signal, ML Confirm,
| ML Agree, Quality, Confirmation 2x, Pullback,
| Cooldown, Position Limit, Lot Size
|
v
[Risk Management] -> Hitung lot aman, apply multiplier
|
v
[Execute Order] -> Kirim ke MT5 dengan broker SL & TP
|
v
[Telegram] -> Notifikasi trade open
|
v
[EXIT MONITORING] -> Setiap 1 detik, 10 kondisi exit:
| Smart TP, Early Exit, Golden Hold, ML Reversal,
| Max Loss, Stall, Daily Limit, Weekend, Time-based, Hold
|
v
[Close Position] -> Record result, update risk, notify
```mermaid
graph TD
A["MetaTrader 5<br/>XAUUSD M15"] -->|OHLCV| B["Data Pipeline<br/>Polars Engine"]
B --> C["SMC Analyzer<br/>OB / FVG / BOS"]
B --> D["Feature Engineering<br/>37 Fitur"]
B --> E["HMM Regime<br/>Detector"]
C --> F["XGBoost Model<br/>Signal + Confidence"]
D --> F
E --> F
F --> G["14 Entry Filters"]
F --> H["Risk Engine<br/>ATR + Kelly"]
G --> I["Eksekusi Trade<br/>MT5"]
H --> I
I --> J["Position Manager<br/>12 Exit Conditions"]
J --> K["Telegram + PostgreSQL<br/>Logging"]
```
---
## Status Komponen (Terkini)
## Ringkasan Peran Setiap Komponen
| Komponen | Pertanyaan yang Dijawab |
|----------|------------------------|
| Feature Engineering | "Data mentah ini berarti apa?" |
| SMC Analyzer | "Dimana institusi besar trading? Entry/SL/TP dimana?" |
| HMM | "Kondisi pasar bagaimana sekarang?" |
| XGBoost | "Harga akan naik atau turun?" |
| Session Filter | "Sekarang waktu yang tepat untuk trading?" |
| News Agent | "Ada berita high-impact yang perlu diperhatikan?" |
| Risk Management | "Berapa besar boleh trading? Sudah aman?" |
| Stop Loss | "Bagaimana melindungi dari kerugian?" |
| Take Profit | "Kapan mengambil profit?" |
| Entry Trade | "Apakah semua syarat terpenuhi untuk masuk?" |
| Exit Trade | "Apakah sudah waktunya keluar?" |
| Telegram | "Apa yang sedang terjadi?" |
| Auto Trainer | "Apakah model AI masih akurat? Perlu dilatih ulang?" |
| Backtest | "Apakah strategi ini profitable di data historis?" |
| Dynamic Confidence | "Seberapa selektif bot harus trading saat ini?" |
| MT5 Connector | "Bagaimana bot terhubung ke broker dan mengirim order?" |
| Configuration | "Bagaimana semua parameter dikonfigurasi?" |
| Trade Logger | "Dimana semua data trade disimpan?" |
| Position Manager | "Bagaimana posisi terbuka dikelola secara aktif?" |
| Risk Engine | "Berapa ukuran lot yang aman? Sudah lewat batas harian?" |
| Database | "Bagaimana data persisten disimpan dan di-query?" |
| Train Models | "Bagaimana model AI dilatih pertama kali?" |
| Main Live | "Siapa yang mengorkestrasi semua komponen?" |
| Komponen | Status | Catatan |
|----------|--------|---------|
| SMC *Analyzer* | **Aktif** | *Order Block*, FVG, BOS, CHoCH |
| XGBoost *Model* | **Aktif** | 37 fitur, *confidence calibrated* |
| HMM *Regime* | **Aktif** | 3 *state* — *low/medium/high volatility* |
| *Session Filter* | **Aktif** | WIB, Tokyo-London *overlap* diblokir |
| *Smart Risk Manager* | **Aktif** | Mode NORMAL/PROTECTED/RECOVERY/COOLDOWN |
| *Dynamic Confidence* | **Aktif** | *Threshold* adaptif per kondisi pasar |
| *Auto Trainer* | **Aktif** | *Retrain* otomatis tiap 7 hari |
| *News Agent* | **Nonaktif** | Dikomentari di `main_live.py` baris 64 |
| Telegram | **Aktif** | Notifikasi *entry/exit* + ringkasan harian |
| *Trade Logger* | **Aktif** | *Logging* ke PostgreSQL |