Commit Graph

19 Commits

Author SHA1 Message Date
GifariKemal e8355b3f62 feat: add 5 dashboard features — dark mode, trade history, backtests, model insights, alerts
- Dark mode: class-based theme toggle with localStorage persistence and flash prevention
- Trade History (/trades): paginated table, stats cards, equity curve chart with DB API endpoints
- Backtest Viewer (/backtests): log parser for 35 backtest results, sidebar + detail + comparison tabs
- Model Insights: dashboard card + dialog showing feature importance, regime distribution, training history
- Alert/Signal Log (/alerts): signal stats, filterable table with execution tracking
- API: 8 new endpoints with psycopg2 DB connection pool
- Dark mode sweep across books page, about dialog, and all dashboard components
- Architecture docs rewritten with Mermaid diagrams (23 docs)
- README and FEATURES.md rewritten bilingual (Indonesian + English)
- main_live.py: write model_metrics.json on startup and retrain

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 05:46:54 +07:00
GifariKemal b2dc2dacd7 refactor: reorganize project structure — consolidate Docker files, clean root
- Delete temp files: _tmp_analysis.py, nul, dashboard_screenshot.png
- Move ea/ to archive/ea/ (deprecated)
- Move 12 Docker helper scripts (.bat/.sh) to docker/scripts/
- Move 5 Docker docs to docker/docs/
- Move .env.docker.example, requirements-docker.txt to docker/
- Update all scripts with cd to project root for correct path resolution
- Update all doc references to new paths
- Update .gitignore with bot.pid, bot_output.log, *.png patterns
- Update CLAUDE.md, README.md directory trees

Root reduced from ~40 files to 12 essential files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 14:21:11 +07:00
GifariKemal 61877480b3 feat: add full dashboard monitoring + FEATURES.md documentation
- Create docs/FEATURES.md with complete feature reference (14 entry
  filters, 12 exit conditions, backtest history, risk modes, session
  rules, auto-trainer, active components table, architecture diagram)

- Extend main_live.py _write_dashboard_status() with 10 new data
  sections: entryFilters, riskMode, cooldown, timeFilter,
  sessionMultiplier, positionDetails, autoTrainer, performance,
  marketClose, h1BiasDetails. Add filter tracking at each checkpoint
  in _trading_iteration() and 7 helper methods.

- Add 9 TypeScript interfaces and extend TradingStatus in trading.ts

- Create BotStatusCard (risk mode, cooldown bar, AUC, uptime, market
  close) and EntryFilterCard (14 filters with pass/block/skip icons)

- Enhance SessionCard (lot multiplier badge + time filter status),
  RiskCard (risk mode badge + total loss progress bar), PositionsCard
  (expandable per-position details with momentum, TP probability)

- Update page.tsx layout: BotStatusCard replaces SettingsCard in Row 2,
  EntryFilterCard added to Row 3 sidebar

- Add API defaults for all new fields

Dashboard now monitors 100% of bot features. Verified: Next.js build
0 errors, bot + API + dashboard all run clean, Docker rebuilt OK.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 13:45:31 +07:00
GifariKemal cb41bfe5ba feat: apply #33B impulse trail + #34A skip hours 9,21 WIB
#33B Impulse Trail (position_manager.py):
- Tighten trailing SL to 1.5x ATR when candle range > 1.5x ATR
- Locks profit faster during volatile spikes (+$59, Sharpe 4.03)

#34A Time-of-Hour Filter (main_live.py):
- Skip entries at WIB hours 9 (02:00 UTC) and 21 (14:00 UTC)
- Hour 9 = end NY session (low liquidity), Hour 21 = London-NY transition (whipsaw)
- +$356 vs #31B, WR 82.6%, Sharpe 4.41, PF 2.43, DD 2.4%

Cumulative live: $3,163 net, 614 trades, 82.6% WR, Sharpe 4.41

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 12:55:34 +07:00
GifariKemal 214b64945d feat: apply #28B smart breakeven + #31B H1 EMA20 filter, add backtests #26-#32
Live trading optimizations (cumulative: $2,807 net, 81.8% WR, Sharpe 3.97):
- #28B: Smart breakeven locks profit at entry + 0.5x ATR instead of fixed $2
- #31B: H1 Price vs EMA20 filter — BUY only when H1 bullish, SELL only when bearish

Backtests #26-#32 (7 scripts testing sell improvement, regime-aware entry,
confluence scoring, dynamic RR, multi-TF H1, and ML exit optimizer).
Winners: #28B (+$229), #31B (+$343). Failed: #26, #27, #29, #30, #32.

Also includes: web dashboard redesign, Docker setup, startup scripts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 10:33:24 +07:00
GifariKemal 53d8cd26a2 feat: apply #24B optimizations — ATR-adaptive exit, skip Tokyo-London, relaxed early cut
Backtest #24B results: 739 trades, 80.4% WR, $2,235 PnL, 3.4% DD, Sharpe 2.87, PF 1.77 (+$785 vs baseline)

Three proven improvements:
- Skip Tokyo-London overlap session (15:00-16:00 WIB) — backtest +$345
- Relax early cut momentum threshold from -30 to -50 — backtest +$125
- ATR-adaptive breakeven/trail (BE=2.0x ATR, trail_start=4.0x ATR, trail_step=3.0x ATR) — backtest +$373

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 22:34:24 +07:00
GifariKemal 8a34dc7f6d feat: 5 critical improvements to trading bot
1. Activate SmartPositionManager (was dead code) - trailing SL, breakeven,
   market close handler, drawdown-from-peak protection now wired into live loop
2. Add flash crash detection between candles - _position_check_only() now
   checks for flash crashes every 5s instead of only on 15min candle close
3. Fix signal confirmation persistence - use direction-based key instead of
   exact price (which changed every candle), persist to file to survive restarts
4. Cache ML/features between candles - stop recalculating 37 features + XGBoost
   every 5 seconds when candle hasn't changed, reuse cached values
5. Add H1 multi-timeframe SMC bias filter - fetch H1 data, analyze BOS/CHoCH/OB,
   block M15 signals that contradict H1 bias, boost confidence when aligned

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 18:21:10 +07:00
GifariKemal 6e0db5274b fix: spread dict access and close_position retry logic
- Fix 'dict' has no attribute 'spread' by using .get("spread", 0)
- Add 3-retry loop to close_position() with fresh price each attempt
- Match retry pattern from send_order() for consistency

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 13:36:46 +07:00
GifariKemal 0d25548ed5 refactor: restructure repository and add README, CLAUDE.md, LICENSE
- Move utility scripts to scripts/ (check_market, check_positions, etc.)
- Move test files to tests/ (test_modules, test_mt5_connection, etc.)
- Move deprecated dashboards to archive/
- Move research files to docs/research/
- Add sys.path fix to all moved Python files
- Rewrite README.md with architecture diagram and badges
- Add CLAUDE.md project guide
- Add MIT LICENSE
- Update .gitignore with archive/ pattern

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 13:22:46 +07:00
GifariKemal 20dc1385c3 optimize: improve win rate with SELL filter and reduced cooldown
Changes:
- Add SELL filter: require ML agreement + 55% confidence for SELL signals
- Reduce trade cooldown: 300s -> 150s (more trade opportunities)
- Relax trend reversal threshold: 0.4 -> 0.6 (less premature exits)
- backtest_live_sync.py: add configurable params for optimization testing

Backtest Results (Jan 2025 - Feb 2026):
BASELINE: 535 trades, 44.1% WR, $994 profit, PF 1.31
OPTIMIZED: 459 trades, 49.2% WR, $1018 profit, PF 1.43

Improvements:
- Win Rate: +5.1% (44.1% -> 49.2%)
- Profit Factor: +0.12 (1.31 -> 1.43)
- Max Drawdown: -0.8% (5.7% -> 4.9%)
- Sharpe Ratio: +0.55 (1.28 -> 1.83)
- NY Session WR: +17.4% (41.6% -> 59.0%)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-06 11:46:36 +07:00
GifariKemal d99df49dfc sync: backtest_live_sync.py with all critical/major fixes
Synchronized elements:
- ATR-based pullback filter (no more hardcoded $2, $1.5)
- Smart time-based exit (checks profit_growing before exit)
- ATR-based trend reversal thresholds
- Signal persistence with index-based cleanup
- Matches main_live.py logic 100%

Backtest Results (Jan 2025 - Feb 2026):
- 534 trades, 44.2% WR
- Net P/L: +$1,056.94
- Profit Factor: 1.34
- Max Drawdown: 5.7%
- Expectancy: +$1.98/trade

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-06 10:25:17 +07:00
GifariKemal 757b499033 docs: sync architecture docs with v5 major issues fix
5 major issues reflected in documentation:

1. Confidence calibration (03-SMC, 00-ARSITEKTUR):
   - Base 55% + 10% each → base 40% + weighted scoring
   - Structure +15%, BOS/CHoCH +12%, FVG +8%, OB +10%, Trend +10%

2. ATR-based pullback filter (09-Entry, 00-ARSITEKTUR):
   - Hardcoded $2/$1.5 → bounce 15% ATR, consolidation 10% ATR

3. Smarter time-based exit (10-Exit, 05-Risk, 00-ARSITEKTUR):
   - 4h: check profit growth, not just profit<$5
   - 6h: extend to 8h if profit>$10 and growing + ML agrees

4. Slippage validation (09-Entry, 23-Main, 00-ARSITEKTUR):
   - Check actual vs expected price, log if >0.15%

5. Partial fill handling (09-Entry, 23-Main, 00-ARSITEKTUR):
   - Check filled volume, use actual values for tracking

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 10:06:43 +07:00
GifariKemal 64848a2b14 fix: major issues - calibrated confidence, ATR-based filters, smarter exits
Major Issue #1: Confidence Calculation Calibration
- Added calculate_confidence() method with weighted scoring
- Base 40% + Structure 15% + BOS/CHoCH 12% + FVG 8% + OB 10% + Trend 10%
- Capped at 85% (never 100% certain)

Major Issue #2: Pullback Filter ATR-based
- Replaced hardcoded $2, $1.5 thresholds
- Now uses bounce_threshold = 0.15 * ATR
- consolidation_threshold = 0.10 * ATR

Major Issue #3: Smarter Time-based Exit
- Don't cut winners short if profit growing
- Check ML agreement before timeout
- Extend time to 8h if profit > $10 and growing

Major Issue #4: Slippage Validation
- Check actual vs expected price after execution
- Log warning if slippage > 0.15% of price
- Use actual price for position tracking

Major Issue #5: Partial Fill Handling
- Check if filled volume < requested volume
- Log warning with fill ratio
- Use actual volume for position tracking

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-06 09:56:42 +07:00
GifariKemal e0ef14b08f docs: update comprehensive architecture doc to match v4 source code
- Loop: ~1 detik → candle-based (M15) + position check ~10 detik
- Exit Kondisi 3: Golden Time Hold → Early Cut (Smart Hold dihapus)
- AUC rollback threshold: 0.52 → 0.60
- Train/test: tambah 50-bar gap info
- Timer periodik: candle-based intervals
- Performance: split full analysis vs position-check-only
- Golden Time: hapus referensi hold losers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 09:51:50 +07:00
GifariKemal 092926415c chore: add training data, backups, and research docs
- Add model backups from training sessions
- Add training data parquet file
- Add risk state persistence file
- Add research documents (Gemini analysis)
- Update architecture docs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-06 09:44:49 +07:00
GifariKemal 7eff3f1a2b fix: critical improvements to trading logic and ML pipeline
main_live.py:
- Switch main loop from time-based (1s) to candle-based (M15)
- Add position-only checks between candles (every 10s)
- Fix memory leak in signal persistence dict (cleanup stale entries)
- Raise auto-retrain rollback AUC threshold from 0.52 to 0.60

src/ml_model.py:
- Add 50-bar gap between train/test split to prevent temporal leakage

src/smart_risk_manager.py:
- Remove dangerous "Smart Hold" behavior (holding losers waiting for golden time)
- Replace with proper early cut logic (loss >30% + negative momentum)

src/smc_polars.py:
- Fix lookahead bias in FVG detection (remove shift(-1), use confirmed bars only)
- Fix lookahead bias in Swing Points (use center=False rolling window)
- Fix lookahead bias in Order Blocks (validate with current bar, not future)
- Enforce minimum 1:2 Risk:Reward ratio on all signals
- Always use current_close as entry price (no stale FVG/OB zone prices)
- Add ATR sanity check with realistic XAUUSD default ($12)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 09:33:43 +07:00
GifariKemal 07e12f5229 docs: add comprehensive full architecture document (00-ARSITEKTUR-LENGKAP.md)
Complete system architecture in 1 document covering:
- System overview with 3 AI brains (SMC + XGBoost + HMM)
- Full architecture diagrams and data flow
- All 23 components table and relationships
- Data pipeline: OHLCV → Features → SMC → HMM → XGBoost → Decision
- 11 entry filters detailed flow
- 10 exit conditions detailed flow
- 4-layer risk protection system (Broker SL → Software → Emergency → Circuit Breaker)
- 4 trading modes (Normal → Recovery → Protected → Stopped)
- Kelly Criterion lot sizing with ML confidence boost
- SMC concepts explained (Swing, FVG, OB, BOS, CHoCH, Liquidity)
- Position lifecycle from signal to close
- Auto-retraining & model management
- Database schema & graceful degradation
- All configuration parameters & session schedule
- Performance targets (~50ms per loop)
- Error handling & fault tolerance (6 levels)
- Complete source code file listing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 09:27:33 +07:00
GifariKemal a240d974f6 docs: add architecture documentation for remaining 8 components (16-23)
New documentation files:
- 16-MT5-Connector: Broker bridge with auto-reconnect & Polars native
- 17-Configuration: 6 sub-configs with capital mode auto-adjustment
- 18-Trade-Logger: Dual storage (PostgreSQL + CSV), thread-safe
- 19-Position-Manager: 7 action conditions, trailing SL, market close handler
- 20-Risk-Engine: Kelly Criterion sizing, circuit breaker, order validation
- 21-Database: PostgreSQL integration with 6 repositories
- 22-Train-Models: Initial training script (HMM + XGBoost)
- 23-Main-Live-Orchestrator: Main loop coordinating 15+ components

Updated README.md with complete index of all 23 components.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 09:17:20 +07:00
GifariKemal 7af9183af3 feat: Smart AI Trading Bot for XAUUSD with ML and SMC
- XGBoost ML model with 37 features for market direction prediction
- Smart Money Concepts (SMC): Order Blocks, FVG, BOS, CHoCH
- HMM market regime detection (trending/ranging/volatile)
- ATR-based stop loss with 1.5 ATR minimum distance
- Broker-level SL protection with fallback
- Time-based exit (max 6 hours per trade)
- Session-aware trading optimized for London/NY overlap
- Auto-retraining based on market conditions
- Telegram notifications and web dashboard
- Backtest results: 63.9% win rate, 2.64 profit factor, 4.83 Sharpe

Backtest period: Jan 2025 - Feb 2026, 654 trades, $4,189 net P/L

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-06 09:01:35 +07:00