gavindiaz/mt-data-mcp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
02740017efd482d885b0f3b09ad256356c4ff9f1
mt-data-mcp/docs/forecast/UNCERTAINTY.md
T

10 KiB
Raw Blame History

Uncertainty and confidence intervals

Audience: User

A point forecast without a range invites overconfidence. This page covers model intervals and conformal intervals so you can size risk and set levels with eyes open.

Dense terms: Confidence interval · Conformal intervals · CI alpha · Horizon

Related: Forecasting · Barriers · Volatility · Glossary

Types of uncertainty

Model confidence intervals

Intervals from the model’s assumptions (for example normal errors).

Limitation: Markets often have fat tails and regime shifts, so these bands can be too narrow.

Conformal intervals

Bands calibrated from historical forecast errors — empirical residual-quantile coverage without strong distributional assumptions.

Advantage: More realistic bounds based on actual performance.

Model Confidence Intervals

Request intervals with --ci-alpha:

mtdata-cli forecast_generate EURUSD --timeframe H1 --horizon 12 --method analog --ci-alpha 0.1 --json

Parameters:

  • --ci-alpha 0.1 → 90% confidence interval
  • --ci-alpha 0.05 → 95% confidence interval

Output includes:

{
  "uncertainty": {
    "status": "available",
    "mode": "interval",
    "alpha": 0.1,
    "intervals": [
      {"time": "2026-01-01T18:00Z", "forecast": 1.1755, "low": 1.1740, "high": 1.1770}
    ]
  }
}

Interpretation:

  • Each uncertainty.intervals[] row is a forecast step. low and high are the model interval bounds for that row (price or return, matching the requested quantity).
  • If model assumptions hold, the true value will fall between low and high about 90% (or 95%) of the time.

Caution: Financial markets have fat tails. Model CIs often underestimate extreme moves.

forecast_generate does not promote a bullish or bearish point estimate to direction unless the horizon interval excludes the last observed price. If a method cannot supply the requested interval, the response reports signal_status: not_actionable, direction_actionable: false, and preserves the model-only label as point_estimate_direction for research diagnostics.

Conformal Intervals (forecast_conformal_intervals)

Residual-quantile calibration builds intervals from rolling backtest residuals, making no distributional assumptions.

How It Works

  1. Run a rolling-origin backtest on historical data
  2. Collect actual forecast errors at each horizon step
  3. Use error quantiles to set interval width for new forecasts

Usage

mtdata-cli forecast_conformal_intervals EURUSD --timeframe H1 --method theta --horizon 12 --steps 50 --spacing 20 --json

Parameters:

Parameter Description Default
--method Forecasting method theta
--horizon Forecast horizon 12
--steps Number of calibration anchors (default 50 for stabler quantiles) 50
--spacing Bars between calibration anchors 20
--ci-alpha Miscoverage rate (0.05 = 95% interval; 0.1 = 90%) 0.05

When --steps > 1, --spacing must be >= --horizon so calibration windows do not overlap; otherwise the request is rejected.

Output

{
  "forecast": [
    {"time": "2026-01-01T18:00Z", "value": 1.1755, "lower": 1.1740, "upper": 1.1770}
  ],
  "conformal": {
    "interval_method": "rolling_residual_quantiles",
    "calibration_anchor_tests_planned": 50,
    "calibration_anchor_tests_succeeded": 50,
    "calibration_anchor_tests_failed": 0,
    "calibration_complete": true,
    "coverage_target": 0.95,
    "empirical_coverage": 0.92
  }
}

Interpretation:

  • Each forecast[] row has its point value and empirically calibrated lower / upper price bounds.
  • With the default alpha, nominal_confidence_level is 95%. This is a calibration target, not a finite-sample guarantee. Compare it with the top-level empirical_coverage and coverage_status fields before using the interval as evidence of historical calibration quality.
  • ci_available=true requires at least 30 calibration residuals for every forecast step and complete forecast paths from every planned anchor. Smaller samples return ci_status=insufficient_calibration. A failed or incomplete anchor returns ci_status=incomplete_anchor_coverage. Both cases set ci_available=false and keep any bounds diagnostic-only.
  • Check calibration_anchor_tests_failed=0 and calibration_complete=true. Empirical coverage from only the anchors where a model happened to fit can be biased, so mtdata does not promote such a subset to a decision-use band.

Use --detail full when you need the raw lower_price / upper_price arrays or calibration diagnostics such as conformal.per_step_q and per-step coverage.

When to Use

  • When you don't trust model-based intervals
  • For trading decisions where reliability matters
  • When historical data shows fat tails or regime changes

Triple-Barrier Labeling (labels_triple_barrier)

A different approach to uncertainty: instead of predicting where price goes, label what happened historically.

Concept

For each historical bar, ask: "Within the next N bars, did price hit the take-profit level, stop-loss level, or neither?"

Labels:

  • +1 (Win): TP hit first
  • -1 (Loss): SL hit first
  • 0 (Neutral): Neither hit within horizon

In high_low mode, same_bar_policy resolves a bar that touches both barriers. The default is conservatively sl_first; tp_first and neutral are explicit alternatives.

When denoising is enabled, the resolved close series anchors each barrier. high_low still uses raw intrabar highs and lows so an observed tradable touch is not smoothed away; labeling_spec.hit_price_source reports this as raw_high_low. Choose label_on=close to use only the resolved close series for both anchors and hits.

Label preprocessing is causal by default. zero_phase filters use future bars and are rejected unless --allow-noncausal-denoise is supplied. That override is for exploratory offline analysis only: the response sets denoise_lookahead_bias=true, suitable_as_training_target=false, and records the effective method, causality, parameters, and entry column under preprocessing.denoise.

Every triple-barrier outcome uses later bars by design. Responses therefore set label_uses_future_path=true, suitable_as_live_feature=false, and separately state whether the result remains a valid historical training target. Join these labels as targets, never as same-timestamp live features.

The label row is keyed by entry_bar_open_time, but its entry_price is the source bar's close. Use entry_price_available_at as the earliest decision time and only join features that were available by that instant. For example, an H1 row opened at 05:00Z has a close-derived entry price available at 06:00Z. tp_hit_bar_open_time and sl_hit_bar_open_time identify the candle containing the first observed touch; OHLC data cannot reveal the exact intrabar touch time. The machine-readable timestamp_contract carries the same rules in compact and full responses. Daily, weekly, and monthly availability uses broker-calendar bar boundaries, including configured daylight-saving changes.

Usage

mtdata-cli labels_triple_barrier EURUSD --timeframe H1 --horizon 12 --barriers '{"unit":"pct","take_profit":0.5,"stop_loss":0.3}' --json

Parameters:

Parameter Description
--horizon Maximum bars to wait
--barriers JSON object with unit, take_profit, and stop_loss
--allow-noncausal-denoise Explicitly permit look-ahead-contaminated zero-phase labels for offline exploration

A conventional FX pip is not the same unit as an MT5 tick. Convert pips using the symbol's quote precision before supplying tick distances (for many five-digit FX quotes, one pip is 10 ticks).

Output

{
  "data": [
    {
      "entry_bar_open_time": "2025-12-18T17:00Z",
      "entry_price_available_at": "2025-12-18T18:00Z",
      "label": 1,
      "outcome": "tp_first",
      "holding_bars": 5,
      "tp_hit_bar_open_time": "2025-12-18T22:00Z"
    }
  ],
  "summary": {
    "counts": {"tp": 45, "sl": 32, "neutral": 123}
  }
}

Compact output keeps at most 10 representative rows in data; summary counts cover the full requested lookback. Use --detail full when you need the parallel entry_bar_open_times, entry_price_available_at, labels, and holding_bars arrays for model training.

Interpretation:

  • Label distribution shows historical win/loss rates for these barrier levels
  • Use this to evaluate signal quality or train ML models

Practical Applications

Conservative Position Sizing

Use conformal intervals instead of model CIs:

# Get conformal intervals
mtdata-cli forecast_conformal_intervals EURUSD --horizon 12 --ci-alpha 0.1

# Use the first forecast row's lower value as a stop-loss floor
# Size position so max loss (if that lower bound is hit) is within risk budget

Validating Signal Quality

Use triple-barrier labels to evaluate entry signals:

# Label historical entry points
mtdata-cli labels_triple_barrier EURUSD --horizon 12 --barriers '{"unit":"pct","take_profit":0.5,"stop_loss":0.3}'

# Check resolved win rate: counts.tp / (counts.tp + counts.sl)
# If win rate < 50%, signal needs improvement

Comparing Forecast Methods

Backtest with conformal intervals to compare reliability:

# Method A
mtdata-cli forecast_conformal_intervals EURUSD --method theta --horizon 12 --steps 50

# Method B
mtdata-cli forecast_conformal_intervals EURUSD --method sf_autoarima --horizon 12 --steps 50

# Compare interval widths—narrower = more precise (if coverage is similar)

Quick Reference

Task Command
Method CI (90%) mtdata-cli forecast_generate EURUSD --method analog --ci-alpha 0.1
Conformal intervals mtdata-cli forecast_conformal_intervals EURUSD --method theta --horizon 12
Triple-barrier labels mtdata-cli labels_triple_barrier EURUSD --horizon 12 --barriers '{"unit":"pct","take_profit":0.5,"stop_loss":0.3}'

See Also

Reference in New Issue View Git Blame Copy Permalink