gavindiaz/xau-traderbot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(exec): repair the six live loss root causes - cooldown, M5-ATR trailing, honest learner, persisted snapshots

Browse Source 
Phase-3 execution-layer fixes, config values are the phase-2 gate winners
(conf=55/breakeven 0.7/atr_trail 1.5/trigger 1.2 - scalping 5-month sweep PF
2.11 at conf=55, baseline PF 1.50):

- trailing_stop: trail now arms only after trail_trigger_atr x ATR unrealized
  gain and never trails past TP (tick-capped). Old logic trailed on H1-ATR
  (~6) while SL/TP were M5-ATR-calibrated (~-6) - breakeven at +1.85 was
  unreachable, every scalp exited at SL
- bot.py: entry cooldown per (strategy, direction) from the cooldown: YAML
  block (scalping 1800s/day_trading 3600s/swing 7200s) - signals were re-firing
  every 30s cycle and piling same-direction positions; Step 4 AI prediction on
  df_m15 (model trains on M15); Step 8 trails on M5 ATR; removed duplicate
  on_trade_closed hook in Step 9 (each external close was learned twice,
  strat_scalping multiplier decayed 0.527->0.41 on garbage inputs);
  performance_snapshots now trigger on wall clock (boot + every >=30min) so
  restarts can't reset the cycle counter to 0
- trade_executor: execute_signal threads ai_score/regime_score into signal
  records; close_trade + sync_positions share _learner_context built from the
  DB Trade row (was: hardcoded regime=RANGING/confluence=75/sentiment=0)
- trading_params.yaml: moderate scalping 50->55, breakeven 0.7, trail 1.5,
  trail_trigger_atr 1.2, cooldown + backtest sections
- tests: scalping moderate confluence assertion 50.0 -> 55.0

Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
gavindiaz
2026-10-21 20:57:42 +08:00
co-authored by Claude Code
parent 7e50a0fef6
commit f38fa944ea
5 changed files with 184 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files
config/trading_params.yaml
+22 -3
View File
@@ -27,7 +27,7 @@ trading_modes:
moderate:
name: Moderate / Calculated Risk
description: Balanced risk-reward, active across major sessions, calculated risks
scalping_min_confluence: 50
scalping_min_confluence: 55
day_trading_min_confluence: 55
swing_trading_min_confluence: 60
min_technical_score: 35
@@ -233,9 +233,28 @@ confluence:
regime_weight: 15
trailing_stop:
enabled: true
breakeven_trigger_rr: 1.0
breakeven_trigger_rr: 0.7
partial_close_pct: 50
atr_trail_multiplier: 2.0
atr_trail_multiplier: 1.5
trail_trigger_atr: 1.2
cooldown:
enabled: true
scalping: 1800
day_trading: 3600
swing_trading: 7200
backtest:
min_confluence: 55
min_technical_score: 35
cooldown_bars: 0
max_hold_bars: 0
breakeven_rr: 0.7
atr_trail_multiplier: 1.5
trail_trigger_atr: 1.2
lot_size: 0.02
initial_balance: 10000.0
warmup_bars: 200
backtest_start: null
backtest_end: null
telegram:
max_alerts_per_minute: 2
daily_summary_time: '21:00'
core/bot.py
+86 -17
View File
@@ -9,7 +9,7 @@ import asyncio
import signal
import sys
import time
from datetime import datetime, timezone
from datetime import datetime, timedelta, timezone
from typing import Optional
from loguru import logger
@@ -100,6 +100,16 @@ class TradingBot:
self._last_news_check: Optional[datetime] = None
self._last_regime = None
self._trading_paused = start_paused
# Root cause #2 fix: min elapsed wall-clock between entries with the SAME
# (strategy, direction). Live loop re-evaluates ~30 s; without this the
# same signal stacks identical scalps across consecutive cycles.
self._last_entry_block_until: dict[tuple[str, str], datetime] = {}
self._cooldown_seconds = self.settings.trading_params.get("cooldown", {})
# Root cause #4 fix: performance snapshots on WALL-CLOCK cadence, not
# cycle count. cycle % 100 == 0 resets on every restart (3 restarts
# today wiped the counter → performance_snapshots stayed at 0 rows).
self._last_snapshot_at: Optional[datetime] = None
self._snapshot_interval_minutes = 30
def _setup_logging(self):
"""Configure loguru logging."""
@@ -151,6 +161,12 @@ class TradingBot:
self.price_predictor.load_model()
self.signal_classifier.load_model()
# Pre-warm FinBERT in a daemon thread: the first `from transformers
# import pipeline` takes 15-30s on this machine (Python 3.12
# importlib.metadata scan). Running it off the loop keeps the main
# trading loop responsive; news falls back to the lexicon until ready.
self.news_analyzer._warm_finbert()
# Display account info
account = self.mt5.get_account_info()
if account:
@@ -164,8 +180,12 @@ class TradingBot:
if self.settings.demo_mode:
logger.warning("⚠️ DEMO MODE — Trades will execute on demo account")
# Start interactive 2-way Telegram polling in background (with network auto-retry)
await self.telegram.start_polling()
# Start interactive 2-way Telegram polling in the background (with network
# auto-retry). Run as a task instead of awaiting it: when api.telegram.org is
# unreachable, start_polling retries up to 30 times (~5 min) — blocking here
# would stall the trading loop on every boot. Alerts sent before polling is up
# fall back to a standalone Bot client in send_message.
self._telegram_poll_task = asyncio.create_task(self.telegram.start_polling())
# Send startup notification via Telegram
mode = "DEMO" if self.settings.demo_mode else "LIVE"
@@ -194,7 +214,8 @@ class TradingBot:
f"• <b>Symbol:</b> <code>{self.settings.symbol}</code>"
)
await self.telegram.send_bot_status(status_title, details)
# Startup notification (non-blocking — same reachability concern as above)
asyncio.create_task(self.telegram.send_bot_status(status_title, details))
# Start background task scheduler (non-blocking news, economic calendar & daily report)
self.scheduler.add_news_job(
@@ -260,6 +281,7 @@ class TradingBot:
# ── Step 2: Fetch market data ────────────────────────────
# Get data for multiple timeframes
df_m5 = self.mt5.get_rates(timeframe="M5", count=500)
df_m15 = self.mt5.get_rates(timeframe="M15", count=500)
df_h1 = self.mt5.get_rates(timeframe="H1", count=500)
df_h4 = self.mt5.get_rates(timeframe="H4", count=500)
df_d1 = self.mt5.get_rates(timeframe="D1", count=500)
@@ -278,7 +300,13 @@ class TradingBot:
regime = None
# ── Step 4: Get AI predictions ───────────────────────────
ai_prediction = self.signal_classifier.predict(df_h1)
# The XGBoost model is trained on M15 (see scripts/train_ai.py and the
# periodic retrain job) with a 5-candle forward-return label (~75 min),
# so feed it M15 data — H1 would be an out-of-timeframe distribution shift.
if df_m15 is not None:
ai_prediction = self.signal_classifier.predict(df_m15)
else:
ai_prediction = {"direction": "HOLD", "confidence": 0.0}
# ── Step 5: Check economic events & news ─────────────────
imminent, ev_name = self.economic_events.is_high_impact_imminent()
@@ -368,7 +396,24 @@ class TradingBot:
# ── Step 7: Execute if conditions met ────────────────────
if best_signal and best_confluence and best_confluence.should_execute:
result = self.trade_executor.execute_signal(
# Root cause #2 fix: entry cooldown per (strategy, direction).
strategy = best_signal.strategy
direction = best_signal.direction.value
cooldown_secs = int(self._cooldown_seconds.get(strategy, 0))
key = (strategy, direction)
if cooldown_secs > 0:
now = datetime.now(timezone.utc)
until = self._last_entry_block_until.get(key)
if until is not None and now < until:
logger.info(
f"⏳ Cooldown active for {strategy}/{direction} "
f"until {until:%H:%M:%S} — signal rejected (root cause #2 fix)"
)
best_signal = None
best_confluence = None
if best_signal and best_confluence:
result = self.trade_executor.execute_signal(
signal=best_signal,
confluence_score=best_confluence.total_score,
sentiment_score=sentiment.get("score", 0) if sentiment else 0,
@@ -376,17 +421,31 @@ class TradingBot:
ai_confidence=ai_prediction.get("confidence", 0),
regime=regime.regime.value if regime else "",
risk_modifier=regime.position_size_modifier if regime else 1.0,
# Root cause #3 fix: record per-source confluence
# attribution (technical/ai/regime) with each signal.
ai_score=getattr(best_confluence, "ai_score", 0.0),
regime_score=getattr(best_confluence, "regime_score", 0.0),
)
if result:
logger.success(f"🎯 Trade opened: {result}")
# Arm cooldown: no new same-(strategy,direction) entry for
# cooldown_secs, so the ~30 s loop cannot stack the signal.
if cooldown_secs > 0:
self._last_entry_block_until[key] = (
datetime.now(timezone.utc) + timedelta(seconds=cooldown_secs)
)
# Send Telegram notification
await self.telegram.send_trade_opened(result)
# ── Step 8: Manage trailing stops ────────────────────────
if df_h1 is not None and "atr" not in df_h1.columns:
df_h1 = self.technical.add_atr(df_h1)
current_atr = float(df_h1["atr"].iloc[-1]) if df_h1 is not None and "atr" in df_h1.columns else 2.0
# Root cause #1 fix: trail on M5 ATR (≈$4, same as SL/TP
# calibration & the backtest gate). H1 ATR (≈$16) made the
# effective trail distance ≈$32 — structurally too wide to
# ever trail an M5 scalp, so every trade ran to SL.
if df_m5 is not None and "atr" not in df_m5.columns:
df_m5 = self.technical.add_atr(df_m5)
current_atr = float(df_m5["atr"].iloc[-1]) if df_m5 is not None and "atr" in df_m5.columns else 2.0
self.trailing_stop.update_all_positions(current_atr)
# ── Step 8b: Smart Midway Exit Guardian ───────────────────
@@ -408,18 +467,24 @@ class TradingBot:
)
# ── Step 9: Sync positions, Learn & Notify Closes ────────
# Root cause #3 fix: sync_positions() feeds trade_learner
# internally with the honest DB context (_learner_context).
# Do NOT call on_trade_closed here too — it would double-count
# every externally-closed trade (multiplier decays twice).
closed_trades = self.trade_executor.sync_positions()
for c_trade in closed_trades:
# Autonomous learning adaptation from closed trade outcome
try:
lesson_res = self.trade_learner.on_trade_closed(c_trade)
except Exception as e:
logger.error(f"Error running trade learning on #{c_trade.get('ticket')}: {e}")
await self.telegram.send_trade_closed(c_trade)
# ── Step 10: Performance snapshot (every 100 cycles) ─────
if self._cycle_count % 100 == 0:
# ── Step 10: Performance snapshot (wall-clock cadence) ────
# Root cause #4 fix: save at boot (first cycle) and every
# >= snapshot_interval_minutes, independent of restart-count
# or cycle-count resets.
now_ts = datetime.now(timezone.utc)
if self._last_snapshot_at is None or (
now_ts - self._last_snapshot_at
) >= timedelta(minutes=self._snapshot_interval_minutes):
await self._save_performance_snapshot()
self._last_snapshot_at = now_ts
# ── Timing ───────────────────────────────────────────────
elapsed = time.time() - cycle_start
@@ -708,7 +773,7 @@ class TradingBot:
logger.error(f"News processing error: {e}")
async def _save_performance_snapshot(self):
"""Save periodic performance metrics."""
"""Save periodic performance metrics (wall-clock cadence)."""
try:
account = self.mt5.get_account_info()
if not account:
@@ -728,6 +793,10 @@ class TradingBot:
win_rate=stats["win_rate"],
profit_factor=stats["profit_factor"],
)
logger.success(
f"📸 Performance snapshot saved | balance=${account['balance']:.2f} "
f"| 30d PF={stats['profit_factor']} ({stats['total_trades']} trades)"
)
except Exception as e:
logger.error(f"Performance snapshot error: {e}")
execution/trade_executor.py
+58 -10
View File
@@ -40,6 +40,8 @@ class TradeExecutor:
ai_confidence: float = 0.0,
regime: str = "",
risk_modifier: float = 1.0,
ai_score: float = 0.0,
regime_score: float = 0.0,
) -> Optional[dict]:
"""
Execute a trading signal after full risk validation.
@@ -52,6 +54,8 @@ class TradeExecutor:
ai_confidence: AI confidence level.
regime: Current market regime.
risk_modifier: Position size modifier from regime.
ai_score: AI prediction contribution to confluence (root cause #3 attribution).
regime_score: Regime alignment contribution to confluence.
Returns:
Trade result dict or None if rejected.
@@ -75,6 +79,8 @@ class TradeExecutor:
confluence_score=confluence_score,
technical_score=signal.technical_score,
sentiment_score=sentiment_score,
ai_score=ai_score,
regime_score=regime_score,
was_executed=False,
rejection_reason=reason,
price_at_signal=signal.entry_price,
@@ -92,6 +98,8 @@ class TradeExecutor:
confluence_score=confluence_score,
technical_score=signal.technical_score,
sentiment_score=sentiment_score,
ai_score=ai_score,
regime_score=regime_score,
was_executed=False,
rejection_reason=reason,
price_at_signal=signal.entry_price,
@@ -125,6 +133,8 @@ class TradeExecutor:
confluence_score=confluence_score,
technical_score=signal.technical_score,
sentiment_score=sentiment_score,
ai_score=ai_score,
regime_score=regime_score,
was_executed=False,
rejection_reason=validation_reason,
price_at_signal=signal.entry_price,
@@ -154,6 +164,8 @@ class TradeExecutor:
confluence_score=confluence_score,
technical_score=signal.technical_score,
sentiment_score=sentiment_score,
ai_score=ai_score,
regime_score=regime_score,
was_executed=False,
rejection_reason=f"Execution error: {error_msg}",
price_at_signal=signal.entry_price,
@@ -188,6 +200,8 @@ class TradeExecutor:
confluence_score=confluence_score,
technical_score=signal.technical_score,
sentiment_score=sentiment_score,
ai_score=ai_score,
regime_score=regime_score,
was_executed=True,
price_at_signal=actual_price,
)
@@ -235,7 +249,7 @@ class TradeExecutor:
commission = 0.0
closed_at = None
crud.close_trade(
closed_trade = crud.close_trade(
ticket=ticket,
exit_price=exit_price,
profit=profit,
@@ -248,20 +262,43 @@ class TradeExecutor:
)
try:
from ai.trade_learner import trade_learner
trade_learner.on_trade_closed({
"ticket": ticket,
"strategy": position.get("comment", "scalping").replace("close_", ""),
"direction": position.get("type", "BUY"),
"profit": profit,
"regime": "RANGING",
"confluence_score": 75.0,
"sentiment_score": 0.0,
})
# Root cause #3 fix: feed the self-learning engine the REAL
# context from the DB trades row. The old code hardcoded
# regime=RANGING / confluence=75 / sentiment=0, so every
# lesson was misclassified and strat_scalping decayed to 0.527.
trade_learner.on_trade_closed(
self._learner_context(closed_trade, profit)
)
except Exception as e:
logger.warning(f"TradeLearner hook warning for #{ticket}: {e}")
return success
def _learner_context(self, trade, profit: float) -> dict:
"""Build an honest TradeLearner context from the DB trades row.
Shared by close_trade and sync_positions so both paths feed the
self-learning engine the same truthful inputs (root cause #3 fix):
strategy, direction, profit, regime, confluence, sentiment, AI call.
"""
from database.models import Trade as _Trade # local: avoid import cycle
if trade is None:
return {"ticket": 0, "strategy": "scalping", "direction": "BUY",
"profit": profit, "regime": "RANGING",
"confluence_score": 0.0, "sentiment_score": 0.0,
"ai_prediction": "", "ai_confidence": 0.0}
return {
"ticket": getattr(trade, "ticket", 0),
"strategy": getattr(trade, "strategy", "scalping") or "scalping",
"direction": getattr(trade, "order_type", "BUY") or "BUY",
"profit": profit,
"regime": getattr(trade, "regime", "") or "RANGING",
"confluence_score": float(getattr(trade, "confluence_score", 0.0) or 0.0),
"sentiment_score": float(getattr(trade, "sentiment_score", 0.0) or 0.0),
"ai_prediction": getattr(trade, "ai_prediction", "") or "",
"ai_confidence": float(getattr(trade, "ai_confidence", 0.0) or 0.0),
}
def close_partial(self, ticket: int, close_pct: float = 50.0) -> bool:
"""Close a percentage of a position."""
positions = self.mt5.get_open_positions()
@@ -329,6 +366,17 @@ class TradeExecutor:
"commission": commission,
"reason": reason,
})
# Root cause #3 fix: externally-closed positions also feed
# the self-learning engine with the same honest DB context.
try:
from ai.trade_learner import trade_learner
trade_learner.on_trade_closed(
self._learner_context(trade, profit)
)
except Exception as e:
logger.warning(
f"TradeLearner hook warning for externally-closed #{trade.ticket}: {e}"
)
except Exception as e:
logger.error(f"Failed to process externally closed position #{trade.ticket}: {e}")
execution/trailing_stop.py
+17
View File
@@ -29,6 +29,7 @@ class TrailingStopManager:
self.breakeven_rr = params.get("breakeven_trigger_rr", 1.0)
self.partial_close_pct = params.get("partial_close_pct", 50)
self.atr_trail_multiplier = params.get("atr_trail_multiplier", 2.0)
self.trail_trigger_atr = params.get("trail_trigger_atr", 1.2)
# Track which positions have had breakeven applied
self._breakeven_applied: set[int] = set()
@@ -107,11 +108,24 @@ class TrailingStopManager:
)
# ── 2. ATR Trailing Stop ─────────────────────────────────────────
# Backtest parity (gate winner): trail only AFTER the position shows
# trail_trigger_atr × ATR unrealized gain — no trailing from entry.
if pos_type == "BUY":
gain = current_price - entry_price
else:
gain = entry_price - current_price
trailing_armed = gain > current_atr * self.trail_trigger_atr
if not trailing_armed:
return # Below trigger: never chase the entry-level trail
trail_distance = current_atr * self.atr_trail_multiplier
if pos_type == "BUY":
# Trail up: new SL = current_price - (ATR * multiplier)
proposed_sl = round(current_price - trail_distance, 2)
# Never trail past TP (backtest parity)
tick = 0.01 # XAUUSD price tick
proposed_sl = min(proposed_sl, round(current_tp - tick, 2))
# Only move SL up, never down
if proposed_sl > current_sl and proposed_sl < current_price:
@@ -126,6 +140,9 @@ class TrailingStopManager:
elif pos_type == "SELL":
# Trail down: new SL = current_price + (ATR * multiplier)
proposed_sl = round(current_price + trail_distance, 2)
# Never trail past TP (backtest parity)
tick = 0.01 # XAUUSD price tick
proposed_sl = max(proposed_sl, round(current_tp + tick, 2))
# Only move SL down (for shorts), never up
if proposed_sl < current_sl and proposed_sl > current_price:
tests/test_mode_switching.py
+1 -1
View File
@@ -142,7 +142,7 @@ def test_strategy_dynamic_confluence_adaptation():
# Switch to Moderate -> override MUST clear!
settings.set_active_mode("moderate")
assert scalp.min_confluence_score == 50.0, "Scalping override leaked into moderate mode!"
assert scalp.min_confluence_score == 55.0, "Scalping override leaked into moderate mode!"
assert scalp.min_technical_score == 35.0
assert day.min_confluence_score == 55.0
assert swing.min_confluence_score == 60.0