mt5-ai-xauusd-trader/DEPLOYMENT_GUIDE.md

Production Deployment Guide: MT5 AI/ML XAUUSD Trading Bot

Executive Summary

Comprehensive guide for deploying the MT5 trading bot to production environments with zero-downtime deployment, automated rollback, comprehensive monitoring, and disaster recovery procedures.

1. Pre-Deployment Checklist

1.1 Code Readiness

• All tests passing (100% pass rate)
• Code coverage >85%
• Security scan passed (no critical issues)
• Code review approved (2+ reviewers)
• No hardcoded credentials or secrets
• All dependencies pinned to specific versions
• Documentation updated
• Release notes prepared

1.2 Infrastructure Readiness

• Staging environment matches production
• All services provisioned
• Database migrations prepared
• Backup systems verified
• Monitoring configured
• Alerting rules tested
• SSL certificates valid
• Firewall rules configured

1.3 Operational Readiness

• Runbooks prepared
• On-call team assigned
• Communication channels ready
• Rollback procedure tested
• Stakeholders notified
• Deployment window scheduled
• No other deployments planned
• Team availability confirmed

2. Deployment Strategy

2.1 Blue-Green Deployment

• Blue Environment: Current production (active)
• Green Environment: New release (standby)
• Process:
  1. Deploy to green environment
  2. Run smoke tests on green
  3. Route traffic to green
  4. Monitor green (2+ hours)
  5. Blue becomes backup
  6. Automatic rollback to blue if issues

2.2 Canary Deployment

• Phase 1: Route 10% traffic to new version
• Phase 2: Monitor metrics for 30 minutes
• Phase 3: Route 50% traffic if no issues
• Phase 4: Monitor metrics for 30 minutes
• Phase 5: Route 100% traffic if no issues
• Rollback: Automatic if error rate >1%

2.3 Rolling Update

• Pod-by-Pod: Update one pod at a time
• Availability: Always maintain quorum
• Health Checks: Verify each pod health
• Rollback: Automatic if health check fails
• Duration: 30-60 minutes for full deployment

3. Database Migrations

3.1 Zero-Downtime Migrations

• Plan: All migrations must be backward-compatible
• Staging: Test all migrations in staging first
• Naming: Use descriptive migration names
• Versioning: Track migration versions
• Reversibility: All migrations must be reversible

3.2 Migration Strategy

• Phase 1: Deploy new code (supports old & new schema)
• Phase 2: Run migration (backward-compatible)
• Phase 3: Monitor for 1+ hour
• Phase 4: Deploy code optimized for new schema
• Rollback: Reverse migration if issues

3.3 Data Backup

• Pre-Migration: Full database backup
• Point-in-Time: Enable PITR (7+ days)
• Verification: Backup verification 2x daily
• Testing: Test restore procedure monthly
• Storage: Backup in separate region

4. Configuration Management

4.1 Environment Configuration

• Development: Local development settings
• Staging: Pre-production configuration
• Production: Live trading configuration
• Secrets: Use HashiCorp Vault or AWS Secrets Manager
• Validation: Validate configs before deployment

4.2 Feature Flags

• Gradual Rollout: Use feature flags for new features
• A/B Testing: Compare feature variants
• Killswitch: Disable features without redeployment
• Observability: Track flag usage
• Cleanup: Remove stale flags regularly

4.3 Configuration Validation

The application implements a mandatory Startup Validation Layer (src/core/config_validator.py) that blocks launch if mission-critical production configuration is invalid or incomplete.

• Required Fields: Ensures all mandatory environment variables (e.g., MT5_PASSWORD, MT5_SERVER) are provided.
• Market Parameters: Validates that SYMBOL is non-empty/uppercase and TIMEFRAME matches MT5 standard periods.
• Credential Safety: Detects default/placeholder values in sensitive fields like DATABASE_URL, TELEGRAM_TOKEN, and REDIS_URL.
• Trading Mode Restrictions: Enforces explicit confirmation for LIVE mode (requires CONFIRM_LIVE_TRADING=YES).
• MT5 Format Enforcement: Prohibits spaces in MT5_SERVER when running in LIVE mode to prevent broker connection failures.
• Risk Perimeter Limits: Enforces hard limits from RISK_LIMITS.md at startup:
  • Critical Failure: Accuracy Floor < 0.50, Win Rate Floor < 0.45, Calibration Error (ECE) > 0.25.
  • Warning: Model Drift > 0.3, SQLite usage in LIVE mode.

5. Deployment Process

5.1 Pre-Deployment

# 1. Create deployment branch
git checkout -b deploy/v1.2.0

# 2. Build Docker images
docker build -t trading-bot:v1.2.0 .

# 3. Run pre-deployment tests
python scripts/verify_version_sync.py

# 4. Verify staging deployment
kubectl apply -f k8s/staging.yaml
kubectl rollout status deployment/trading-bot-staging

5.2 Production Deployment

# 1. Tag Docker image
docker tag trading-bot:v1.2.0 registry.io/trading-bot:v1.2.0
docker push registry.io/trading-bot:v1.2.0

# 2. Update Kubernetes manifests
sed -i 's/v1.1.0/v1.2.0/g' k8s/production.yaml

# 3. Apply zero-downtime deployment
kubectl apply -f k8s/production.yaml
kubectl rollout status deployment/trading-bot

# 4. Run smoke tests
python scripts/smoke_test.py --url http://localhost:8000 --audit-db audit.db

# 5. Monitor deployment (2+ hours)
kubectl logs -l app=trading-bot --follow

5.3 Post-Deployment

# 1. Verify all services running
kubectl get pods -l app=trading-bot

# 2. Check metrics in Grafana
# - Error rate should be <0.1%
# - Latency should be normal
# - CPU/Memory usage should be normal

# 3. Run end-to-end tests
# Verification of core trading logic via MT5 simulation (if applicable)

# 4. Update documentation
git tag -a v1.2.0 -m "Production release v1.2.0"
git push origin v1.2.0

6. Rollback Procedures

6.1 Automatic Rollback

• Error Rate: >1% error rate
• Latency: p95 latency >2x baseline
• Health Check: Failed health checks
• Disk Space: <10% free disk space
• Memory: OOM kills detected

6.2 Manual Rollback

# 1. Identify issue
kubectl logs -l app=trading-bot --tail=100

# 2. Check previous version
kubectl rollout history deployment/trading-bot

# 3. Rollback to previous version
kubectl rollout undo deployment/trading-bot

# 4. Verify rollback
kubectl rollout status deployment/trading-bot

# 5. Monitor metrics
kubectl logs -l app=trading-bot --follow

6.3 Rollback Testing

• Monthly: Test rollback procedure
• Documentation: Keep rollback runbook updated
• Automation: Automate common rollback scenarios
• Verification: Verify successful rollbacks

7. Infrastructure

7.1 Cloud Deployment

• Provider: AWS, Azure, or GCP
• Region: Primary + backup region
• Multi-AZ: Spread across availability zones
• Load Balancing: Auto-scaling load balancer
• CDN: CloudFlare for DDoS protection

7.2 Kubernetes Configuration

apiVersion: apps/v1
kind: Deployment
metadata:
  name: trading-bot
spec:
  replicas: 3
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    spec:
      containers:
      - name: trading-bot
        image: trading-bot:v1.2.0
        resources:
          requests:
            cpu: 500m
            memory: 512Mi
          limits:
            cpu: 1000m
            memory: 1Gi
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /ready
            port: 8080
          initialDelaySeconds: 10
          periodSeconds: 5

7.3 Load Balancing

• Type: Network Load Balancer (Layer 4)
• Algorithm: Round-robin with health checks
• Connection Draining: 30-second drain timeout
• Stickiness: None (stateless services)
• Health Check: Every 10 seconds

8. Monitoring & Observability

8.1 Deployment Monitoring

• Deployment Status: Track rollout status
• Pod Health: Monitor pod readiness via /health/readiness
• Service Status: Check service availability via /health/liveness
• Error Rate: Alert if >1% errors
• Latency: Alert if p95 latency > 500ms

8.2 Health Check Endpoints

The application provides enterprise-grade health monitoring endpoints:

• GET /health/liveness: Returns 200 OK if the process is running.
• GET /health/readiness: Performs deep dependency checks (DB, MT5, Redis) and returns 200 OK only if ready to trade. Returns 503 Service Unavailable on critical failures.
• GET /health/full: Returns a comprehensive report of all system components.
• GET /metrics: Exports Prometheus metrics, including system_component_health gauges.

8.2 Application Metrics

• Requests: Requests per second
• Errors: Error count and rate
• Latency: Response time distribution
• Trades: Trade execution count
• P&L: Profit/loss tracking

8.3 System Metrics

• CPU: CPU utilization (target <70%)
• Memory: Memory usage (target <80%)
• Disk: Disk usage (alert <10% free)
• Network: Network I/O (baseline expected)
• Database: Query latency, connection count

9. Disaster Recovery

9.1 Backup Strategy

• Database: Hourly incremental, daily full
• Configuration: Version controlled in Git
• Secrets: Encrypted backup in separate region
• RPO: Maximum 1 hour data loss
• RTO: 15-minute recovery time

9.2 Recovery Procedures

• Database Recovery: Restore from latest backup
• Configuration Recovery: Deploy from Git
• Secrets Recovery: Restore from encrypted backup
• Testing: Monthly disaster recovery drills

9.3 Multi-Region Deployment

• Primary Region: Active trading
• Secondary Region: Hot standby
• Failover: Automatic on primary failure
• Failback: Manual after primary recovery
• Testing: Quarterly failover drills

10. Documentation

10.1 Runbooks

• Deployment: Step-by-step deployment guide
• Rollback: Rollback procedures
• Troubleshooting: Common issues and solutions
• Incident Response: Incident handling procedures
• On-Call: On-call team responsibilities

10.2 Change Log

• Version: Semantic versioning (X.Y.Z)
• Features: New features added
• Fixes: Bug fixes included
• Breaking Changes: Incompatible changes
• Migration Guide: Data migration procedures

11. Compliance & Audit

11.1 Deployment Audit Trail

• Who: Deploying user
• What: Version deployed
• When: Deployment timestamp
• Where: Target environment
• Why: Deployment reason
• Status: Success/failure

11.2 Change Management

• Approval: Requires approval before production
• Review: All changes reviewed
• Testing: All changes tested
• Documentation: Changes documented
• Communication: Stakeholders informed

Conclusion

Production deployment requires discipline, planning, and automation. Follow this guide for reliable, safe deployments with minimal risk to trading operations.