feat: add 5 dashboard features — dark mode, trade history, backtests, model insights, alerts

- Dark mode: class-based theme toggle with localStorage persistence and flash prevention
- Trade History (/trades): paginated table, stats cards, equity curve chart with DB API endpoints
- Backtest Viewer (/backtests): log parser for 35 backtest results, sidebar + detail + comparison tabs
- Model Insights: dashboard card + dialog showing feature importance, regime distribution, training history
- Alert/Signal Log (/alerts): signal stats, filterable table with execution tracking
- API: 8 new endpoints with psycopg2 DB connection pool
- Dark mode sweep across books page, about dialog, and all dashboard components
- Architecture docs rewritten with Mermaid diagrams (23 docs)
- README and FEATURES.md rewritten bilingual (Indonesian + English)
- main_live.py: write model_metrics.json on startup and retrain

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
GifariKemal
2026-02-09 05:46:54 +07:00
co-authored by Claude Opus 4.6
parent b2dc2dacd7
commit e8355b3f62
230 changed files with 69573 additions and 5673 deletions
+179 -280
View File
@@ -1,276 +1,210 @@
# Entry Trade — Proses Masuk Posisi
# *Entry Trade* — Proses Masuk Posisi
> **File utama:** `main_live.py`
> **File pendukung:** `src/smc_polars.py`, `src/ml_model.py`, `src/smart_risk_manager.py`, `src/session_filter.py`
---
## Apa Itu Entry Trade?
## Apa Itu *Entry Trade*?
Entry Trade adalah keseluruhan proses dari **mendeteksi peluang** hingga **mengirim order ke broker**. Bot menggunakan **10+ filter** yang harus SEMUA lolos sebelum satu trade dieksekusi.
*Entry Trade* adalah keseluruhan proses dari **mendeteksi peluang** hingga **mengirim *order* ke *broker***. Bot menggunakan **14 filter** yang harus **SEMUA lolos** sebelum satu *trade* dieksekusi.
**Analogi:** Entry Trade seperti **proses boarding pesawat** — harus punya tiket (signal), passport valid (confirmation), lulus security check (risk), tepat waktu (session), dan gate terbuka (position limit).
**Analogi:** *Entry Trade* seperti **proses *boarding* pesawat** — harus punya tiket (*signal*), *passport* valid (*confirmation*), lulus *security check* (risiko), tepat waktu (sesi), dan *gate* terbuka (*position limit*).
---
## Checklist Entry (Semua Harus PASS)
## Daftar *Checklist Entry* (Semua Harus PASS)
```
1. [SESSION] Session filter izinkan trading?
2. [RISK MODE] Trading mode bukan STOPPED?
3. [SMC SIGNAL] Ada signal dari SMC Analyzer?
4. [ML CONFIRM] XGBoost confidence >= 50%?
5. [ML AGREE] ML tidak strongly disagree (>65% berlawanan)?
6. [QUALITY] Market quality bukan AVOID/CRISIS?
7. [CONFIRM] Signal konsisten 2 bar berturut?
8. [PULLBACK] Bukan sedang pullback/retrace?
9. [COOLDOWN] Sudah 5 menit sejak trade terakhir?
10. [POS LIMIT] Posisi terbuka < 2?
11. [LOT SIZE] Lot > 0 setelah semua adjustment?
| # | Filter | Keterangan | Status |
|---|--------|------------|--------|
| 1 | ***Flash Crash Guard*** | Apakah ada pergerakan harga ekstrem? | **Aktif** |
| 2 | ***Regime Filter*** | Apakah *regime* HMM bukan SLEEP? | **Aktif** |
| 3 | ***Risk Check*** | Apakah `risk_metrics.can_trade` = `true`? | **Aktif** |
| 4 | ***Session Filter*** | Apakah sesi perdagangan mengizinkan *trading*? | **Aktif** |
| 5 | ***SMC Signal*** | Apakah ada sinyal valid dari SMC *Analyzer*? | **Aktif** |
| 6 | ***Signal Combination*** | Apakah kombinasi SMC + ML menghasilkan sinyal akhir? | **Aktif** |
| 7 | **H1 *Bias* (#31B)** | Apakah *bias* H1 EMA20 sejalan dengan sinyal? | **Aktif** |
| 8 | **Filter Waktu (#34A)** | Apakah bukan jam 9 atau 21 WIB? | **Aktif** |
| 9 | ***Trade Cooldown*** | Sudah 5 menit sejak *trade* terakhir? | **Aktif** |
| 10 | ***Pullback Filter*** | Apakah bukan sedang *pullback/retrace*? | **Nonaktif** |
| 11 | ***Smart Risk Gate*** | Mode *trading* bukan STOPPED/COOLDOWN? | **Aktif** |
| 12 | **Kalkulasi *Lot*** | Apakah *lot size* > 0 setelah semua *adjustment*? | **Aktif** |
| 13 | ***Spread* Validasi** | Apakah *spread* tidak terlalu lebar? | **Aktif** |
| 14 | **Batas Posisi** | Posisi terbuka < 2? | **Aktif** |
SEMUA PASS -> Execute Trade
SATU GAGAL -> Skip, tunggu loop berikutnya
```
> **Semua PASS** → Eksekusi *Trade*
> **Satu GAGAL** → *Skip*, tunggu *loop* berikutnya
---
## Step-by-Step Flow
## *Step-by-Step Flow*
### Step 1: Session Filter
### Filter 1: *Flash Crash Guard*
```python
# main_live.py
is_flash, move_pct = self.flash_crash.detect(df.tail(5))
if is_flash:
return # Pergerakan harga ekstrem terdeteksi
```
**Bisa *block*:** Pergerakan harga > 2.5% dalam 1 menit (*flash crash threshold* dari `config.py`).
---
### Filter 2: *Regime Filter*
```python
regime_sleep = regime_state and regime_state.recommendation == "SLEEP"
if regime_sleep:
return # HMM mendeteksi kondisi krisis
```
**Bisa *block*:** *Regime* HIGH_VOLATILITY / CRISIS — pasar terlalu bergejolak.
---
### Filter 3: *Risk Check*
```python
if not risk_metrics.can_trade:
return # Risiko di luar batas
```
---
### Filter 4: *Session Filter*
```python
# main_live.py Lines 472-483
session_ok, session_reason, session_multiplier = self.session_filter.can_trade()
if not session_ok:
return # Skip — bukan waktu trading
# Simpan multiplier untuk lot sizing nanti
self._current_session_multiplier = session_multiplier
return # Bukan waktu trading
```
**Bisa block:** Weekend, Friday >23:00, danger zone (00:00-06:00), low volatility session.
**Bisa *block*:** *Weekend*, Jumat > 23:00, zona bahaya (00:00-06:00), sesi *low volatility*.
**Tokyo-London *overlap*** (15:00-16:00 WIB) **diblokir** — hasil optimasi *backtest* #24B.
---
### Step 2: Risk Mode Check
### Filter 5: *SMC Signal*
```python
# main_live.py Lines 537-542
risk_rec = self.smart_risk.get_trading_recommendation()
if not risk_rec["can_trade"]:
return # STOPPED mode — daily/total limit tercapai
```
**Bisa block:** Mode STOPPED (daily loss >= $250, total loss >= $500).
---
### Step 3: SMC Signal Generation
```python
# main_live.py Lines 498-499
smc_signal = self.smc.generate_signal(df)
if smc_signal is None:
return # Tidak ada setup SMC yang valid
```
**SMC membutuhkan:**
- Market structure (bullish/bearish) ATAU BOS/CHoCH
- DAN (FVG ATAU Order Block)
- Minimum 2:1 risk/reward
- Struktur pasar (*bullish/bearish*) ATAU BOS/CHoCH
- DAN (FVG ATAU *Order Block*)
- Minimum 2:1 *risk/reward*
**Output:** Entry price, SL, TP, confidence (55-85%), reason.
**Output:** *Entry price*, SL, TP, *confidence* (55-85%), alasan.
---
### Step 4: ML Confidence Check
### Filter 6: *Signal Combination*
```python
# main_live.py Lines 419-425
ml_prediction = self.ml_model.predict(df, feature_cols)
# Lines 664-669
if ml_prediction.confidence < 0.50:
return # ML terlalu tidak yakin
final_signal = self._combine_signals(smc_signal, ml_prediction, regime_state)
if final_signal is None:
return # Sinyal terfilter
```
Menggabungkan **SMC + ML + *Regime*** menjadi satu sinyal akhir. ML harus *agree* atau minimal tidak *strongly disagree* (> 65% *confidence* berlawanan).
---
### Step 5: ML Agreement Check
### Filter 7: H1 *Bias* (#31B)
```python
# main_live.py Lines 676-684
# Jika SMC bilang BUY tapi ML bilang SELL dengan confidence > 65%:
if smc_signal.signal_type == "BUY":
if ml_prediction.signal == "SELL" and ml_prediction.confidence > 0.65:
return # ML strongly disagrees — VETO
if smc_signal.signal_type == "SELL":
if ml_prediction.signal == "BUY" and ml_prediction.confidence > 0.65:
return # ML strongly disagrees — VETO
# Backtest #31B: H1 EMA20 filter menambah +$345 profit
if h1_bias == "BULLISH" and final_signal.signal_type == "SELL":
return # BUY signal vs H1 bullish = blokir
if h1_bias == "BEARISH" and final_signal.signal_type == "BUY":
return # SELL signal vs H1 bearish = blokir
if h1_bias == "NEUTRAL":
return # Tidak ada bias jelas = blokir
```
**Tujuan:** Hanya masuk posisi yang sejalan dengan *trend* H1.
---
### Step 6: Dynamic Market Quality
### Filter 8: Filter Waktu (#34A)
```python
# main_live.py Lines 618-657
# Analisis kualitas pasar berdasarkan:
# - Session (London/NY = tinggi, Sydney = rendah)
# - Regime (low vol = bagus, crisis = block)
# - Volatility (medium = ideal)
# - Trend strength
# - SMC confluence
# - ML signal alignment
quality_score = analyze_market_quality(...)
# EXCELLENT (80+), GOOD (60+), MODERATE (40+), POOR (20+), AVOID (<20), CRISIS
if quality == "AVOID" or quality == "CRISIS":
return # Pasar tidak layak untuk trading
# Backtest #34A: skip jam 9 dan 21 WIB menambah +$356 profit
wib_hour = datetime.now(ZoneInfo("Asia/Jakarta")).hour
if wib_hour in (9, 21):
return # Jam transisi — volatilitas tidak optimal
```
**Tujuan:** Menghindari jam transisi sesi yang berpotensi *whipsaw*.
---
### Step 7: Signal Confirmation (2 Bar Berturut)
### Filter 9: *Trade Cooldown*
```python
# main_live.py Lines 686-709
signal_key = f"{smc_signal.signal_type}_{smc_signal.entry_price:.0f}"
if signal_key in self._signal_persistence:
self._signal_persistence[signal_key] += 1
else:
self._signal_persistence[signal_key] = 1
if self._signal_persistence[signal_key] < 2:
return # Belum dikonfirmasi — tunggu 1 loop lagi
# Signal sudah muncul 2x berturut -> CONFIRMED
```
**Tujuan:** Mencegah whipsaw — signal yang hanya muncul 1 detik kemungkinan noise.
---
### Step 8: Pullback Filter
```python
# main_live.py Lines 742-871
can_enter, pullback_reason = self._check_pullback_filter(df, signal.signal_type)
if not can_enter:
return # Sedang pullback, tunggu momentum selaras
```
**v5: Threshold sekarang ATR-based (bukan hardcoded)**
```
ATR diambil dari data (default $12 untuk XAUUSD)
bounce_threshold = ATR × 0.15 # ~$1.80 (sebelumnya: $2.00 fixed)
consolidation_threshold = ATR × 0.10 # ~$1.20 (sebelumnya: $1.50 fixed)
Kenapa ATR-based?
- Threshold menyesuaikan volatilitas pasar saat ini
- Saat volatilitas tinggi (ATR=$20): bounce=$3, consolidation=$2
- Saat volatilitas rendah (ATR=$8): bounce=$1.2, consolidation=$0.8
- Lebih akurat daripada threshold tetap
```
**Untuk signal BUY, block jika:**
- Harga turun > bounce_threshold (15% ATR) dalam 3 candle terakhir
- MACD bearish + harga turun
- Harga jauh di bawah EMA9 + terus turun
**Untuk signal SELL, block jika:**
- Harga naik > bounce_threshold (15% ATR) dalam 3 candle terakhir
- MACD bullish + harga naik
- Harga jauh di atas EMA9 + terus naik
**Komponen yang dicek:**
```
1. Short-term Momentum (3 candle terakhir)
-> Arah pergerakan harga terkini
-> Block jika bounce > 15% ATR (v5: dinamis)
2. MACD Histogram
-> Rising = bullish momentum
-> Falling = bearish momentum
3. Harga vs EMA9
-> Di atas = bullish bias
-> Di bawah = bearish bias
4. RSI Extreme
-> RSI > 80 = overbought (block BUY)
-> RSI < 20 = oversold (block SELL)
5. Consolidation Check
-> Jika movement < 10% ATR = consolidation → ALLOW
-> v5: dinamis, bukan fixed $1.5
```
---
### Step 9: Trade Cooldown
```python
# main_live.py Lines 520-524
trade_cooldown = 300 # 5 menit
if last_trade_time:
elapsed = (now - last_trade_time).total_seconds()
if elapsed < trade_cooldown:
return # Tunggu cooldown selesai
if last_trade_time and (now - last_trade_time).total_seconds() < 300:
return # Tunggu cooldown selesai
```
**Tujuan:** Mencegah overtrading — minimal 5 menit antar trade.
**Tujuan:** Mencegah *overtrading* — minimal 5 menit antar *trade*.
---
### Step 10: Position Limit
### Filter 10: *Pullback Filter* (NONAKTIF)
```python
# main_live.py Lines 588-592
can_open, limit_reason = self.smart_risk.can_open_position()
if not can_open:
return # Sudah 2 posisi terbuka (max)
# DISABLED — mode SMC-only
# Struktur SMC sudah memvalidasi zona entry
```
> Filter ini dinonaktifkan karena analisis SMC sudah mencakup validasi *pullback* dalam logika *Order Block* dan FVG.
---
### Step 11: Lot Size Calculation
### Filter 11: *Smart Risk Gate*
```python
# main_live.py Lines 544-560
safe_lot = self.smart_risk.calculate_lot_size(
entry_price=signal.entry_price,
confidence=signal.confidence, # SMC confidence
regime=regime_name, # HMM regime
ml_confidence=ml_prediction.confidence, # ML confidence
)
risk_rec = self.smart_risk.get_trading_recommendation()
if not risk_rec["can_trade"]:
return # Mode STOPPED/COOLDOWN
```
# Apply session multiplier
**4 mode** *Smart Risk*: NORMAL → PROTECTED → RECOVERY → COOLDOWN/STOPPED.
---
### Filter 12-14: *Lot*, *Spread*, dan Batas Posisi
```python
# Kalkulasi lot
safe_lot = self.smart_risk.calculate_lot_size(...)
safe_lot = max(0.01, safe_lot * session_multiplier)
if safe_lot <= 0:
return # Lot 0 = tidak boleh trade
# Validasi spread
if spread > max_allowed:
return # Spread terlalu lebar
# Batas posisi (max 2)
can_open, limit_reason = self.smart_risk.can_open_position()
if not can_open:
return # Sudah 2 posisi terbuka
```
---
## Eksekusi Order
## Eksekusi *Order*
Setelah semua 11 filter lolos:
Setelah semua **14 filter** lolos:
```python
# main_live.py Lines 985-1008
# Step A: Ambil harga real-time
tick = mt5.get_tick(symbol)
current_price = tick.ask if BUY else tick.bid
@@ -284,128 +218,93 @@ if jarak_terlalu_dekat:
result = mt5.send_order(
symbol="XAUUSD",
order_type="BUY" / "SELL",
volume=0.01 - 0.02, # Lot dari risk calculation
sl=broker_sl, # ATR-based SL (v3)
tp=signal.take_profit, # SMC TP (ATR-capped)
magic=123456, # ID bot
volume=0.01 - 0.05,
sl=broker_sl, # SL berbasis ATR
tp=signal.take_profit, # TP dari SMC (ATR-capped)
magic=123456,
comment="AI Safe v3",
)
# Step D: Fallback jika broker reject SL
if gagal dan error 10016:
result = mt5.send_order(sl=0, ...) # Tanpa broker SL
result = mt5.send_order(sl=0, ...)
# Step E: Slippage Validation (v5 BARU)
# Step E: Validasi slippage
if result.success:
actual_price = result.price
slippage = abs(actual_price - signal.entry_price)
max_slippage = signal.entry_price * 0.0015 # 0.15% dari harga
slippage = abs(result.price - signal.entry_price)
max_slippage = signal.entry_price * 0.0015 # 0.15%
if slippage > max_slippage:
log WARNING "HIGH SLIPPAGE" # Catat slippage tinggi
# Gunakan harga AKTUAL untuk tracking, bukan harga expected
log WARNING "HIGH SLIPPAGE"
# Step F: Partial Fill Check (v5 BARU)
filled_volume = result.volume
if filled_volume < requested_volume:
log WARNING "PARTIAL FILL"
# Update lot_size ke volume yang sebenarnya terisi
position.lot_size = filled_volume
# Step G: Register posisi (gunakan nilai AKTUAL)
# Step F: Register posisi (gunakan nilai AKTUAL)
smart_risk.register_position(
ticket=result.order_id,
entry_price=actual_price, # v5: harga aktual (bukan expected)
lot_size=filled_volume, # v5: volume aktual (bukan requested)
entry_price=result.price, # Harga aktual
lot_size=result.volume, # Volume aktual
direction=signal.signal_type,
)
```
### Slippage & Partial Fill (v5 Detail)
```
SLIPPAGE VALIDATION:
expected_price = signal.entry_price
actual_price = result.price (dari broker)
slippage = |actual - expected|
max_acceptable = 0.15% dari harga (~$4 untuk XAUUSD @$2650)
Jika slippage > max_acceptable:
-> LOG WARNING (untuk monitoring & analisis)
-> Tetap pakai harga aktual untuk position tracking
PARTIAL FILL HANDLING:
requested_volume = lot dari risk calculation
filled_volume = result.volume (dari broker)
Jika filled < requested:
-> LOG WARNING dengan fill ratio (%)
-> Update position.lot_size ke filled_volume
-> Risk calculation tetap akurat (berdasarkan volume sebenarnya)
```
---
## Post-Entry
## *Post-Entry*
```python
# Step H: Log trade detail
trade_logger.log_trade_open(
signal, ml_prediction, regime, market_quality, ...
)
# Log trade detail ke PostgreSQL
trade_logger.log_trade_open(signal, ml_prediction, regime, market_quality, ...)
# Step G: Kirim notifikasi Telegram
# Kirim notifikasi Telegram
await telegram.send_trade_open(trade_info)
# Step H: Update cooldown timer
# Update cooldown timer
last_trade_time = now
```
---
## Diagram Flow Lengkap
## Diagram *Flow* Lengkap
```
Loop setiap 1 detik
|
v
Fetch 200 bar M15 -> Feature Eng -> SMC -> HMM -> XGBoost
|
v
[1] Session OK? ----NO----> Skip
|YES
[2] Risk OK? -------NO----> Skip (STOPPED)
|YES
[3] SMC Signal? ----NO----> Skip (tidak ada setup)
|YES
[4] ML >= 50%? -----NO----> Skip (terlalu uncertain)
|YES
[5] ML Agree? ------NO----> Skip (ML veto)
|YES
[6] Quality OK? ----NO----> Skip (AVOID/CRISIS)
|YES
[7] Confirmed 2x? --NO----> Skip (tunggu konfirmasi)
|YES
[8] No Pullback? ---NO----> Skip (retrace)
|YES
[9] Cooldown OK? ---NO----> Skip (< 5 menit)
|YES
[10] Pos < 2? ------NO----> Skip (full)
|YES
[11] Lot > 0? ------NO----> Skip
|YES
v
EXECUTE TRADE -> Register -> Log -> Telegram
```mermaid
graph TD
A["Loop Setiap ~30 Detik"] --> B["Fetch 200 Bar M15"]
B --> C["Feature Eng + SMC + HMM + XGBoost"]
C --> F1{"1. Flash Crash?"}
F1 -->|Ya| SKIP["Skip ↩"]
F1 -->|Tidak| F2{"2. Regime SLEEP?"}
F2 -->|Ya| SKIP
F2 -->|Tidak| F3{"3. Risk OK?"}
F3 -->|Tidak| SKIP
F3 -->|Ya| F4{"4. Session OK?"}
F4 -->|Tidak| SKIP
F4 -->|Ya| F5{"5. SMC Signal?"}
F5 -->|Tidak| SKIP
F5 -->|Ya| F6{"6. Signal Combo?"}
F6 -->|Tidak| SKIP
F6 -->|Ya| F7{"7. H1 Bias OK?"}
F7 -->|Tidak| SKIP
F7 -->|Ya| F8{"8. Jam OK?"}
F8 -->|Tidak| SKIP
F8 -->|Ya| F9{"9. Cooldown OK?"}
F9 -->|Tidak| SKIP
F9 -->|Ya| F11{"10. Risk Gate?"}
F11 -->|Tidak| SKIP
F11 -->|Ya| F12{"11-14. Lot/Spread/Pos?"}
F12 -->|Tidak| SKIP
F12 -->|Ya| EXEC["EKSEKUSI TRADE"]
EXEC --> POST["Register + Log + Telegram"]
```
---
## Statistik Filter
Dalam kondisi normal, dari ratusan loop per jam:
- **~95%** diblokir oleh "tidak ada SMC signal" (pasar sideways)
- **~3%** diblokir oleh ML disagreement atau low confidence
- **~1%** diblokir oleh pullback filter atau session
- **<1%** lolos semua filter dan menghasilkan trade
Dalam kondisi normal, dari ratusan *loop* per jam:
**Rata-rata:** 3-8 trade per hari (sangat selektif).
| Sumber *Block* | Persentase | Keterangan |
|-----------------|-----------|------------|
| Tidak ada sinyal SMC | **~95%** | Pasar *sideways*, tidak ada *setup* |
| ML *disagreement* / *low confidence* | **~3%** | ML tidak yakin atau berlawanan |
| *Pullback*, sesi, H1 *bias* | **~1%** | Filter waktu dan arah |
| **Lolos semua → *Trade*** | **< 1%** | Sangat selektif |
**Rata-rata:** 3-8 *trade* per hari.