- Delete all old XGBoost/HMM models trained with the order-block look-ahead
leak (models/backups/* + root models). They reproduced a fake 63.9% WR /
2.64 PF that collapses to ~35% WR / 0.95 PF once the leak is fixed.
- scripts/collect_data.py: dedicated raw M1+M15 collector (paginated)
- scripts/fast_backtest.py: vectorized GPU backtest for honest validation
- backtest_live_sync.py: read SYMBOL from env (XM uses GOLD, not XAUUSD)
- stop tracking generated data/training_data.parquet
See upstream report: GifariKemal/xaubot-ai#4
Add M1+M15 multi-timeframe SMC scalping training pipeline (GPU XGBoost),
then fix data-leakage and non-stationarity issues found in a skeptical audit.
Pipeline:
- src/triple_barrier.py: TP/SL/time labeling (ATR-scaled, asymmetric RR)
- src/multi_tf_dataset.py: M1 base + M15 HTF context, point-in-time join_asof
(only CLOSED M15 candles visible to each M1 bar - proven no leakage)
- src/economic_calendar.py: point-in-time forecast/actual/surprise provider
- src/smc_polars.py: add premium/discount + displacement SMC features
- scripts/train_multitf_scalper.py: GPU (device=cuda) training + walk-forward
- scripts/download_training_data.py: 1y data downloader
Leakage / robustness fixes (audit):
- CRITICAL: order block signal was written to the ORIGIN bar (future info);
now assigned at the CONFIRMATION bar -> matches live conditions
- replace non-stationary absolute features (ema_9/21, macd*) with scale-free
forms (ema*_dist_atr, ema_spread_atr, macd_*_bps) -> valid at any price level
- drop constant-zero calendar features from defaults (recurring provider has
no real values); re-add when a real calendar CSV is configured
- walk-forward + train/test now embargo the max_holding label horizon and drop
warmup rows (NaN->0 artifacts)
- news calendar features remain point-in-time (actual only at/after release)
Honest result: after fixes the spurious +2.35% edge collapses to ~random
(AUC 0.49). The prior edge was caused by the order-block look-ahead. Pipeline
is now leakage-free; a real edge still needs more M1 history / better features.
Also: test infra (pytest.ini asyncio, hmmlearn), TRAIN_BARS, cleanup of dead
modules. 14 tests pass.
Make the bot connect to MetaTrader5 on Linux without manual setup or
launching extra programs.
- Add mt5linux rpyc bridge: MT5 terminal + Windows Python run under Wine,
Linux client talks to it over a socket
- scripts/mt5_bridge.sh: robust, idempotent launcher (up/down/status/restart/
login-gui/install-service) with .env loading, port health-checks and a
systemd --user unit for auto-start on login
- src/mt5_connector.py: auto-detect bridge backend (lazy, non-blocking import);
auto-start the bridge via mt5_bridge.sh when the port is down so the bot
self-connects; attach-mode initialize() first, then explicit login
- Use configured SYMBOL instead of hardcoded XAUUSD in connector + news_agent
(XM names spot gold 'GOLD')
- scripts/test_mt5_bridge.py: connectivity smoke test
- docs/MT5-ARCH-LINUX-SETUP.md: full setup, root-cause notes, troubleshooting
Verified live: account 345454551 @ XMGlobal-MT5 10, balance read, GOLD M15
data + ticks streaming, trade_allowed=True.
- Untrack .claude/settings.local.json (IDE-specific)
- Gitignore: data/bot_status.json, signal_persistence.json, model_metrics.json
- Gitignore: logs/ (all content, not just *.log)
- Gitignore: backtests/.claude/
- Add scripts/check_trade_detail.py utility
- Update data/risk_state.txt to current state
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Move utility scripts to scripts/ (check_market, check_positions, etc.)
- Move test files to tests/ (test_modules, test_mt5_connection, etc.)
- Move deprecated dashboards to archive/
- Move research files to docs/research/
- Add sys.path fix to all moved Python files
- Rewrite README.md with architecture diagram and badges
- Add CLAUDE.md project guide
- Add MIT LICENSE
- Update .gitignore with archive/ pattern
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Vanszs