2026-05-01 07:54:01 +05:00
# Service Level Objectives (SLO) & Reliability Targets
2026-05-09 08:32:39 +05:00
This document defines the formal, measurable reliability standards for the MT5 AI/ML Trading Bot. These targets translate "Enterprise Quality" into specific, trackable objectives to ensure production trust, capital safety, and operational excellence.
2026-05-01 07:54:01 +05:00
2026-05-08 08:10:36 +05:00
## 1. Availability SLOs (System Uptime)
2026-05-02 08:03:24 +05:00
2026-05-08 08:10:36 +05:00
Uptime is defined as the percentage of time the system is operational and capable of executing trades during active market hours (XAUUSD: Monday 00:00 - Friday 23:59 Server Time).
2026-05-02 08:03:24 +05:00
| Metric | Target | Measurement Method |
|--------|--------|--------------------|
2026-05-10 08:20:00 +05:00
| **System Uptime (Live Mode)** | **99.5%** | `/health/readiness` probe success rate (Prometheus: `system_component_health{component="liveness"}` ). |
2026-05-08 08:10:36 +05:00
| **API Availability** | 99.9% | Percentage of 2xx/3xx responses from health/metrics endpoints. |
2026-05-10 08:20:00 +05:00
| **MT5 Connectivity** | 99.0% | Prometheus: `system_component_health{component="mt5"}` status during market hours. |
2026-05-02 08:03:24 +05:00
2026-05-08 08:10:36 +05:00
**Acceptable Downtime:** ~3.6 hours per 30-day rolling window during market hours.
2026-05-02 08:03:24 +05:00
2026-05-08 08:10:36 +05:00
## 2. CI/CD & Engineering SLOs
2026-05-02 08:03:24 +05:00
2026-05-08 08:10:36 +05:00
Ensuring the stability of the delivery pipeline and the quality of the codebase.
2026-05-02 08:03:24 +05:00
| Metric | Target | Measurement Method |
|--------|--------|--------------------|
2026-05-08 08:10:36 +05:00
| **CI Pipeline Success Rate** | **95.0%** | Ratio of "Success" vs "Failure" in GitHub Actions `ci.yml` and `pre-deploy-validation.yml` . |
| **Test Coverage (Core)** | > 85% | Automated `pytest-cov` report (Gate 8 in `pre-deploy-validation.yml` ). |
| **Security Scan Cleanliness** | 100% | Zero `CRITICAL` or `HIGH` vulnerabilities in Trivy/Gitleaks scans. |
2026-05-02 08:03:24 +05:00
## 3. Performance & Latency SLOs
2026-05-08 08:10:36 +05:00
Inference and execution latency are measured via `src/core/profiler.py` and exported to Prometheus.
2026-05-02 08:03:24 +05:00
| Metric | P50 | P95 | P99 | Measurement Method |
|--------|-----|-----|-----|-------------------|
2026-05-06 08:24:57 +05:00
| **Model Inference** | < 10ms | < 50ms | < 100ms | `trading_block_duration_seconds{block_label="inference"}` |
| **Risk Approval** | < 20ms | < 50ms | < 100ms | `trading_block_duration_seconds{block_label="risk_check"}` |
| **End-to-End Latency** | < 100ms | < 500ms | < 1.5s | `trading_execution_latency_seconds` |
2026-05-09 08:32:39 +05:00
| **Backtest Generation** | ** < 5 min** | ** < 8 min** | ** < 12 min** | Audit Log: `action="backtest_completed"` (metadata: `duration_seconds` ). |
2026-05-02 08:03:24 +05:00
2026-05-08 08:10:36 +05:00
## 4. Operational Responsiveness (Alert Triage)
2026-05-02 08:03:24 +05:00
2026-05-08 08:10:36 +05:00
Response expectations for the operations team, aligned with the [Monitoring Runbook ](runbooks/06-monitoring-alert-triage.md ).
2026-05-02 08:03:24 +05:00
2026-05-11 08:02:45 +05:00
| Severity | Target Response (ACK) | Target Resolution | Measurement Method |
|----------|-----------------------|-------------------|--------------------|
| **P0 (Critical)** | ** < 5 mins** | < 1 hour | Time-to-Acknowledge (TTO) in Incident Tracker |
| **P1 (High)** | ** < 15 mins** | < 4 hours | Time-to-Acknowledge (TTO) in Incident Tracker |
| **P2 (Medium)** | < 2 hours | < 24 hours | Time-to-Resolution (TTR) in Incident Tracker |
| **P3 (Low)** | < 24 hours | 1 Week | JIRA/Issue Age Tracking |
2026-05-02 08:03:24 +05:00
## 5. Incident Recovery (RTO/RPO)
2026-05-08 08:10:36 +05:00
Defined in the [Disaster Recovery Plan ](DISASTER_RECOVERY.md ).
2026-05-01 07:54:01 +05:00
2026-05-06 08:24:57 +05:00
| Metric | Target | Measurement Method |
|--------|--------|--------------------|
2026-05-10 08:20:00 +05:00
| **Recovery Time Objective (RTO)** | **15 mins** | Time from incident detection to `system_restored` event in audit log (logged via `AuditLogger.log_system_restored` ). |
2026-05-11 08:02:45 +05:00
| **Recovery Point Objective (RPO)** | **1 hour** | Maximum data age in the most recent valid database backup (verified via `scripts/backup_verify.sh` logs). |
2026-05-01 07:54:01 +05:00
2026-05-02 08:03:24 +05:00
## 6. Error Budget Framework (30-Day Rolling Window)
2026-05-01 07:54:01 +05:00
2026-05-08 08:10:36 +05:00
The Error Budget is the maximum allowable unreliability. If the budget is exhausted, a "Stability Freeze" is triggered.
2026-05-01 07:54:01 +05:00
2026-05-13 10:35:48 +05:00
| Component | SLO | Error Budget | "Acceptable" Failure / Month | Description |
|-----------|-----|--------------|-----------------------------|-------------|
| **Availability** | 99.5% | 0.5% | 216 Minutes (3.6 Hours) | Total downtime across all market hours (equiv. to ~14 incidents at 15m RTO). |
| **CI Stability** | 95.0% | 5.0% | 5 failures per 100 commits | Non-transient pipeline failures (excludes infrastructure flakiness). |
| **Trade Execution** | 99.9% | 0.1% | 1 failure per 1,000 signals | Technical execution failures (e.g., timeout, rejection due to code bug). |
| **Data Integrity** | 100% | 0.0% | **Zero** incidents | No unrecoverable data loss or permanent database corruption. |
2026-05-01 07:54:01 +05:00
2026-05-08 08:10:36 +05:00
### 6.1 Stability Freeze Protocol
2026-05-03 08:01:33 +05:00
2026-05-08 08:10:36 +05:00
If any error budget reaches **0% remaining** within a 30-day window:
1. **Immediate Feature Freeze** : All PRs containing new features or non-critical refactors are blocked.
2. **Reliability Sprint** : The next development cycle is dedicated 100% to addressing the root cause of the SLO breach.
3. **Mandatory Post-Mortem** : A blameless post-mortem must be conducted and documented in `docs/audits/` .
2026-05-03 08:01:33 +05:00
2026-05-08 08:10:36 +05:00
## 7. Governance & Reporting
2026-05-03 08:01:33 +05:00
2026-05-08 08:10:36 +05:00
- **Real-time Tracking**: SLO compliance is monitored via Grafana dashboards pulling from Prometheus.
- **Weekly Review**: Reliability metrics are reviewed every Monday to assess remaining error budgets.
- **Audit Trail**: Every SLO breach is logged as a `critical_event` in the `audit.db` .
2026-05-11 08:02:45 +05:00
- **Stakeholder Report**: A monthly reliability summary is generated using `scripts/generate_research_report.py` (Enterprise Section).