feat: Grafana copy publishing, dashboard examples, and optional OTel metrics

Implements three observability improvements:

#82 — publish_grafana_copy(): Uses SQLite online backup API (WAL-safe) to
atomically publish a consistent read-only copy beside the target. Adds
--publish-copy option to grafana-schema and snapshot CLI commands.

#83 — examples/grafana/: Minimal working Grafana setup with docker-compose,
provisioning datasource/dashboard YAML, and three dashboard JSON files
(mt5cli-overview, mt5cli-trades, mt5cli-market). All queries use grafana_*
views; no credentials or private paths included.

#84 — mt5cli/telemetry.py: Optional OTel metrics behind mt5cli[otel] extra.
Base install is unaffected. Adds _Mt5Metrics singleton (no-op until
configure_metrics() is called), wraps update_history() and
update_observability() with record_history_update / record_snapshot_update
context managers, and emits account/position gauges from snapshots.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
agent
2026-06-27 23:17:13 +00:00
co-authored by Claude Sonnet 4.6
parent d27da02f3f
commit 4c2e25af8a
18 changed files with 1609 additions and 35 deletions
+87
View File
@@ -0,0 +1,87 @@
# Grafana Integration for mt5cli
This directory contains example configuration and dashboard files for visualising
mt5cli SQLite data in [Grafana](https://grafana.com/) using the
[Grafana SQLite datasource plugin](https://grafana.com/grafana/plugins/frser-sqlite-datasource/).
## Prerequisites
- mt5cli installed and able to connect to MetaTrader 5
- Grafana 10+ with the `frser-sqlite-datasource` plugin installed
- (Optional) Docker and Docker Compose for the containerised setup
## Generating the SQLite database
Collect historical data and snapshot current account state:
```sh
# Collect OHLCV history
mt5cli -o history.db collect-history --symbol EURUSD --date-from 2024-01-01 --date-to 2024-12-31
# Create Grafana-ready views and indexes
mt5cli -o history.db grafana-schema
# Snapshot current account, positions, and orders
mt5cli -o history.db snapshot --with-grafana-schema
```
## Publishing a Grafana-readable copy
Grafana reads the SQLite file directly. To avoid read/write conflicts, publish
a consistent copy after each update:
```sh
mt5cli -o history.db grafana-schema --publish-copy history.grafana.db
mt5cli -o history.db snapshot --publish-copy history.grafana.db
```
The `--publish-copy` option uses the SQLite online backup API, which is safe
even when the source database uses WAL journal mode.
## Configuring the datasource path
Edit `provisioning/datasources/mt5cli-sqlite.yaml` and set the `path` field
to the absolute path of your published `.db` file:
```yaml
jsonData:
path: /absolute/path/to/history.grafana.db
```
## Running Grafana on Windows (native)
1. Download and install Grafana from <https://grafana.com/grafana/download/>.
2. Install the SQLite plugin: `grafana-cli plugins install frser-sqlite-datasource`.
3. Copy `provisioning/` into `%ProgramFiles%\GrafanaLabs\grafana\conf\provisioning\`.
4. Import the dashboards from `dashboards/` via the Grafana UI
(Dashboards → Import → Upload JSON file).
## Running with Docker Compose
```sh
# From the examples/grafana directory
docker compose up -d
```
Then open <http://localhost:3000> (default credentials: admin / admin).
The Compose file mounts this directory and the SQLite file into the container.
Edit `docker-compose.yml` to point `MT5CLI_DB_PATH` at your `.db` file.
## Dashboard overview
| Dashboard | Description |
|---|---|
| `mt5cli-overview.json` | Account balance, equity, margin, and snapshot freshness |
| `mt5cli-trades.json` | Trade P/L, win rate, symbol breakdown |
| `mt5cli-market.json` | OHLCV rates, spreads, and tick volume |
All panel queries use the `grafana_*` views; they do not read internal storage
tables directly.
## Importing dashboards
1. Open Grafana and navigate to **Dashboards → Import**.
2. Click **Upload JSON file** and select one of the files in `dashboards/`.
3. Select the `mt5cli-SQLite` datasource when prompted.
4. Click **Import**.
@@ -0,0 +1,113 @@
{
"__inputs": [
{
"name": "DS_MT5CLI_SQLITE",
"label": "mt5cli-SQLite",
"description": "",
"type": "datasource",
"pluginId": "frser-sqlite-datasource",
"pluginName": "SQLite"
}
],
"__requires": [
{
"type": "datasource",
"id": "frser-sqlite-datasource",
"name": "SQLite",
"version": "1.0.0"
}
],
"annotations": {"list": []},
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 0,
"id": null,
"links": [],
"panels": [
{
"datasource": "${DS_MT5CLI_SQLITE}",
"fieldConfig": {"defaults": {}, "overrides": []},
"gridPos": {"h": 8, "w": 24, "x": 0, "y": 0},
"id": 1,
"title": "Close Price Over Time",
"type": "timeseries",
"targets": [
{
"rawSql": "SELECT \"time\" AS time, \"symbol\", \"close\" FROM grafana_rates WHERE $__timeFilter(time) ORDER BY time",
"format": "time_series",
"refId": "A"
}
]
},
{
"datasource": "${DS_MT5CLI_SQLITE}",
"fieldConfig": {"defaults": {}, "overrides": []},
"gridPos": {"h": 8, "w": 12, "x": 0, "y": 8},
"id": 2,
"title": "Spread Over Time",
"type": "timeseries",
"targets": [
{
"rawSql": "SELECT \"time\" AS time, \"symbol\", \"spread\" FROM grafana_rates WHERE $__timeFilter(time) ORDER BY time",
"format": "time_series",
"refId": "A"
}
]
},
{
"datasource": "${DS_MT5CLI_SQLITE}",
"fieldConfig": {"defaults": {}, "overrides": []},
"gridPos": {"h": 8, "w": 12, "x": 12, "y": 8},
"id": 3,
"title": "Tick Volume Over Time",
"type": "timeseries",
"targets": [
{
"rawSql": "SELECT \"time\" AS time, \"symbol\", \"tick_volume\" FROM grafana_rates WHERE $__timeFilter(time) ORDER BY time",
"format": "time_series",
"refId": "A"
}
]
},
{
"datasource": "${DS_MT5CLI_SQLITE}",
"fieldConfig": {"defaults": {}, "overrides": []},
"gridPos": {"h": 8, "w": 24, "x": 0, "y": 16},
"id": 4,
"title": "Tick Bid/Ask Over Time",
"type": "timeseries",
"targets": [
{
"rawSql": "SELECT \"time\" AS time, \"symbol\", \"bid\", \"ask\" FROM grafana_ticks WHERE $__timeFilter(time) ORDER BY time LIMIT 5000",
"format": "time_series",
"refId": "A"
}
]
}
],
"refresh": "1m",
"schemaVersion": 36,
"tags": ["mt5cli", "market"],
"templating": {
"list": [
{
"current": {},
"hide": 0,
"includeAll": false,
"label": "Data Source",
"multi": false,
"name": "DS_MT5CLI_SQLITE",
"options": [],
"query": "frser-sqlite-datasource",
"refresh": 1,
"type": "datasource"
}
]
},
"time": {"from": "now-24h", "to": "now"},
"timepicker": {},
"timezone": "browser",
"title": "MT5CLI - Market Data",
"uid": "mt5cli-market",
"version": 1
}
@@ -0,0 +1,159 @@
{
"__inputs": [
{
"name": "DS_MT5CLI_SQLITE",
"label": "mt5cli-SQLite",
"description": "",
"type": "datasource",
"pluginId": "frser-sqlite-datasource",
"pluginName": "SQLite"
}
],
"__requires": [
{
"type": "datasource",
"id": "frser-sqlite-datasource",
"name": "SQLite",
"version": "1.0.0"
}
],
"annotations": {"list": []},
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 0,
"id": null,
"links": [],
"panels": [
{
"datasource": "${DS_MT5CLI_SQLITE}",
"fieldConfig": {
"defaults": {"unit": "currencyUSD"},
"overrides": []
},
"gridPos": {"h": 4, "w": 6, "x": 0, "y": 0},
"id": 1,
"options": {"reduceOptions": {"calcs": ["lastNotNull"]}},
"title": "Balance",
"type": "stat",
"targets": [
{
"rawSql": "SELECT \"time\" AS time, \"balance\" FROM grafana_account_snapshots ORDER BY time DESC LIMIT 1",
"format": "table",
"refId": "A"
}
]
},
{
"datasource": "${DS_MT5CLI_SQLITE}",
"fieldConfig": {
"defaults": {"unit": "currencyUSD"},
"overrides": []
},
"gridPos": {"h": 4, "w": 6, "x": 6, "y": 0},
"id": 2,
"options": {"reduceOptions": {"calcs": ["lastNotNull"]}},
"title": "Equity",
"type": "stat",
"targets": [
{
"rawSql": "SELECT \"time\" AS time, \"equity\" FROM grafana_account_snapshots ORDER BY time DESC LIMIT 1",
"format": "table",
"refId": "A"
}
]
},
{
"datasource": "${DS_MT5CLI_SQLITE}",
"fieldConfig": {
"defaults": {"unit": "currencyUSD"},
"overrides": []
},
"gridPos": {"h": 4, "w": 6, "x": 12, "y": 0},
"id": 3,
"options": {"reduceOptions": {"calcs": ["lastNotNull"]}},
"title": "Free Margin",
"type": "stat",
"targets": [
{
"rawSql": "SELECT \"time\" AS time, \"margin_free\" FROM grafana_account_snapshots ORDER BY time DESC LIMIT 1",
"format": "table",
"refId": "A"
}
]
},
{
"datasource": "${DS_MT5CLI_SQLITE}",
"fieldConfig": {
"defaults": {"unit": "percent"},
"overrides": []
},
"gridPos": {"h": 4, "w": 6, "x": 18, "y": 0},
"id": 4,
"options": {"reduceOptions": {"calcs": ["lastNotNull"]}},
"title": "Margin Level",
"type": "stat",
"targets": [
{
"rawSql": "SELECT \"time\" AS time, \"margin_level\" FROM grafana_account_snapshots ORDER BY time DESC LIMIT 1",
"format": "table",
"refId": "A"
}
]
},
{
"datasource": "${DS_MT5CLI_SQLITE}",
"fieldConfig": {"defaults": {}, "overrides": []},
"gridPos": {"h": 8, "w": 24, "x": 0, "y": 4},
"id": 5,
"title": "Account Balance Over Time",
"type": "timeseries",
"targets": [
{
"rawSql": "SELECT \"time\" AS time, \"balance\" FROM grafana_account_snapshots ORDER BY time",
"format": "time_series",
"refId": "A"
}
]
},
{
"datasource": "${DS_MT5CLI_SQLITE}",
"fieldConfig": {"defaults": {}, "overrides": []},
"gridPos": {"h": 8, "w": 24, "x": 0, "y": 12},
"id": 6,
"title": "Equity Over Time",
"type": "timeseries",
"targets": [
{
"rawSql": "SELECT \"time\" AS time, \"equity\" FROM grafana_account_snapshots ORDER BY time",
"format": "time_series",
"refId": "A"
}
]
}
],
"refresh": "1m",
"schemaVersion": 36,
"tags": ["mt5cli", "account"],
"templating": {
"list": [
{
"current": {},
"hide": 0,
"includeAll": false,
"label": "Data Source",
"multi": false,
"name": "DS_MT5CLI_SQLITE",
"options": [],
"query": "frser-sqlite-datasource",
"refresh": 1,
"type": "datasource"
}
]
},
"time": {"from": "now-7d", "to": "now"},
"timepicker": {},
"timezone": "browser",
"title": "MT5CLI - Account Overview",
"uid": "mt5cli-overview",
"version": 1
}
@@ -0,0 +1,113 @@
{
"__inputs": [
{
"name": "DS_MT5CLI_SQLITE",
"label": "mt5cli-SQLite",
"description": "",
"type": "datasource",
"pluginId": "frser-sqlite-datasource",
"pluginName": "SQLite"
}
],
"__requires": [
{
"type": "datasource",
"id": "frser-sqlite-datasource",
"name": "SQLite",
"version": "1.0.0"
}
],
"annotations": {"list": []},
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 0,
"id": null,
"links": [],
"panels": [
{
"datasource": "${DS_MT5CLI_SQLITE}",
"fieldConfig": {"defaults": {}, "overrides": []},
"gridPos": {"h": 8, "w": 12, "x": 0, "y": 0},
"id": 1,
"title": "Realized P/L by Symbol",
"type": "table",
"targets": [
{
"rawSql": "SELECT \"symbol\", \"cumulative_pnl\", \"deal_count\" FROM grafana_realized_pnl ORDER BY cumulative_pnl DESC",
"format": "table",
"refId": "A"
}
]
},
{
"datasource": "${DS_MT5CLI_SQLITE}",
"fieldConfig": {"defaults": {}, "overrides": []},
"gridPos": {"h": 8, "w": 12, "x": 12, "y": 0},
"id": 2,
"title": "Trade Statistics by Symbol",
"type": "table",
"targets": [
{
"rawSql": "SELECT \"symbol\", \"total_deals\", \"winning_deals\", \"losing_deals\", \"total_profit\", \"avg_profit\" FROM grafana_trade_stats ORDER BY total_profit DESC",
"format": "table",
"refId": "A"
}
]
},
{
"datasource": "${DS_MT5CLI_SQLITE}",
"fieldConfig": {"defaults": {}, "overrides": []},
"gridPos": {"h": 8, "w": 24, "x": 0, "y": 8},
"id": 3,
"title": "Open Position Profit Over Time",
"type": "timeseries",
"targets": [
{
"rawSql": "SELECT \"time\" AS time, \"symbol\", SUM(\"profit\") AS profit FROM grafana_position_snapshots GROUP BY time, symbol ORDER BY time",
"format": "time_series",
"refId": "A"
}
]
},
{
"datasource": "${DS_MT5CLI_SQLITE}",
"fieldConfig": {"defaults": {}, "overrides": []},
"gridPos": {"h": 8, "w": 24, "x": 0, "y": 16},
"id": 4,
"title": "Cash Events Over Time",
"type": "timeseries",
"targets": [
{
"rawSql": "SELECT \"time\" AS time, \"profit\" FROM grafana_cash_events ORDER BY time",
"format": "time_series",
"refId": "A"
}
]
}
],
"refresh": "5m",
"schemaVersion": 36,
"tags": ["mt5cli", "trades"],
"templating": {
"list": [
{
"current": {},
"hide": 0,
"includeAll": false,
"label": "Data Source",
"multi": false,
"name": "DS_MT5CLI_SQLITE",
"options": [],
"query": "frser-sqlite-datasource",
"refresh": 1,
"type": "datasource"
}
]
},
"time": {"from": "now-30d", "to": "now"},
"timepicker": {},
"timezone": "browser",
"title": "MT5CLI - Trade Analytics",
"uid": "mt5cli-trades",
"version": 1
}
+24
View File
@@ -0,0 +1,24 @@
# Docker Compose for Grafana with mt5cli SQLite datasource.
#
# Set MT5CLI_DB_PATH to the absolute host path of your published .db file
# before running `docker compose up -d`.
#
# Example:
# MT5CLI_DB_PATH=/home/user/history.grafana.db docker compose up -d
services:
grafana:
image: grafana/grafana:latest
ports:
- "3000:3000"
environment:
GF_PATHS_PROVISIONING: /etc/grafana/provisioning
volumes:
- ./provisioning:/etc/grafana/provisioning:ro
- ./dashboards:/var/lib/grafana/dashboards:ro
- grafana-storage:/var/lib/grafana
- ${MT5CLI_DB_PATH:-/tmp/mt5cli-placeholder.db}:/data/mt5cli.db:ro
user: "472"
volumes:
grafana-storage:
@@ -0,0 +1,13 @@
# Grafana dashboard provisioning for mt5cli dashboards.
apiVersion: 1
providers:
- name: mt5cli
type: file
disableDeletion: false
updateIntervalSeconds: 30
allowUiUpdates: true
options:
path: /var/lib/grafana/dashboards
foldersFromFilesStructure: false
@@ -0,0 +1,16 @@
# Grafana datasource provisioning for mt5cli SQLite.
#
# Requires the frser-sqlite-datasource plugin:
# grafana-cli plugins install frser-sqlite-datasource
#
# Set `path` to the absolute path of your published history.grafana.db file.
apiVersion: 1
datasources:
- name: mt5cli-SQLite
type: frser-sqlite-datasource
access: proxy
isDefault: true
jsonData:
path: /data/mt5cli.db
+33 -2
View File
@@ -803,7 +803,19 @@ def collect_history(
@app.command(rich_help_panel="Collection") @app.command(rich_help_panel="Collection")
def grafana_schema(ctx: typer.Context) -> None: def grafana_schema(
ctx: typer.Context,
publish_copy: Annotated[
Path | None,
typer.Option(
"--publish-copy",
help=(
"Publish a Grafana-ready SQLite copy to this path"
" after schema creation."
),
),
] = None,
) -> None:
"""Create or refresh Grafana-ready views and indexes in a SQLite database. """Create or refresh Grafana-ready views and indexes in a SQLite database.
Idempotent — safe to run repeatedly on the same database. Requires SQLite Idempotent — safe to run repeatedly on the same database. Requires SQLite
@@ -814,7 +826,11 @@ def grafana_schema(ctx: typer.Context) -> None:
""" """
import sqlite3 as _sqlite3 # noqa: PLC0415 import sqlite3 as _sqlite3 # noqa: PLC0415
from .grafana import create_snapshot_tables, ensure_grafana_schema # noqa: PLC0415 from .grafana import ( # noqa: PLC0415
create_snapshot_tables,
ensure_grafana_schema,
publish_grafana_copy,
)
export_ctx = _get_export_context(ctx) export_ctx = _get_export_context(ctx)
if export_ctx.output_format != "sqlite3": if export_ctx.output_format != "sqlite3":
@@ -829,6 +845,9 @@ def grafana_schema(ctx: typer.Context) -> None:
create_snapshot_tables(conn) create_snapshot_tables(conn)
ensure_grafana_schema(conn) ensure_grafana_schema(conn)
logger.info("Grafana schema applied to %s", export_ctx.output) logger.info("Grafana schema applied to %s", export_ctx.output)
if publish_copy is not None:
publish_grafana_copy(export_ctx.output, publish_copy)
logger.info("Grafana copy published to %s", publish_copy)
@app.command(rich_help_panel="Collection") @app.command(rich_help_panel="Collection")
@@ -867,6 +886,13 @@ def snapshot(
help="Ensure Grafana views and indexes exist.", help="Ensure Grafana views and indexes exist.",
), ),
] = False, ] = False,
publish_copy: Annotated[
Path | None,
typer.Option(
"--publish-copy",
help=("Publish a Grafana-ready SQLite copy to this path after snapshot."),
),
] = None,
) -> None: ) -> None:
"""Snapshot current account, position, order, and terminal state into SQLite. """Snapshot current account, position, order, and terminal state into SQLite.
@@ -894,6 +920,11 @@ def snapshot(
with_grafana_schema=with_grafana_schema, with_grafana_schema=with_grafana_schema,
) )
logger.info("Snapshot written to %s", export_ctx.output) logger.info("Snapshot written to %s", export_ctx.output)
if publish_copy is not None:
from .grafana import publish_grafana_copy # noqa: PLC0415
publish_grafana_copy(export_ctx.output, publish_copy)
logger.info("Grafana copy published to %s", publish_copy)
def main() -> None: def main() -> None:
+55
View File
@@ -2,9 +2,13 @@
from __future__ import annotations from __future__ import annotations
import contextlib
import datetime import datetime
import logging import logging
import os
import sqlite3 import sqlite3
import tempfile
from pathlib import Path
from typing import cast from typing import cast
from .history import get_table_columns from .history import get_table_columns
@@ -478,6 +482,57 @@ def ensure_grafana_schema(conn: sqlite3.Connection) -> None:
create_grafana_indexes(conn) create_grafana_indexes(conn)
def publish_grafana_copy(
source: str | Path,
target: str | Path,
) -> Path:
"""Publish a consistent SQLite copy for Grafana using the backup API.
Uses the SQLite online backup API for a WAL-safe, consistent snapshot of
the source database. Writes to a temporary file beside the target, then
atomically replaces it so that a previous published copy is preserved if
publishing fails.
Args:
source: Path to the source SQLite database.
target: Destination path for the published copy.
Returns:
The resolved absolute target path.
Raises:
FileNotFoundError: If the source database does not exist.
"""
source_path = Path(source)
target_path = Path(target)
if not source_path.exists():
raise FileNotFoundError(source_path)
target_path.parent.mkdir(parents=True, exist_ok=True)
tmp_fd, tmp_str = tempfile.mkstemp(
dir=target_path.parent,
suffix=".tmp",
prefix=target_path.name + ".",
)
tmp_path = Path(tmp_str)
try:
os.close(tmp_fd)
with (
sqlite3.connect(source_path) as src,
sqlite3.connect(tmp_path) as dst,
):
src.backup(dst)
tmp_path.replace(target_path)
except Exception:
with contextlib.suppress(OSError):
tmp_path.unlink()
raise
logger.info("Published Grafana copy: %s -> %s", source_path, target_path)
return target_path.resolve()
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Snapshot insert helpers # Snapshot insert helpers
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
+65 -30
View File
@@ -45,6 +45,7 @@ from .history import (
write_incremental_datasets, write_incremental_datasets,
) )
from .retry import retry_with_backoff from .retry import retry_with_backoff
from .telemetry import get_metrics
from .utils import ( from .utils import (
Dataset, Dataset,
IfExists, IfExists,
@@ -1026,20 +1027,21 @@ def update_history( # noqa: PLR0913
with closing(sqlite3.connect(request.output_path)) as conn, conn: with closing(sqlite3.connect(request.output_path)) as conn, conn:
conn.execute("PRAGMA journal_mode=WAL") conn.execute("PRAGMA journal_mode=WAL")
conn.execute("PRAGMA synchronous=NORMAL") conn.execute("PRAGMA synchronous=NORMAL")
write_incremental_datasets( with get_metrics().record_history_update(dataset="history"):
conn, write_incremental_datasets(
client, conn,
symbols, client,
request.selected, symbols,
request.resolved_timeframes, request.selected,
request.resolved_tick_flags, request.resolved_timeframes,
request.fallback_start, request.resolved_tick_flags,
request.end, request.fallback_start,
deduplicate=deduplicate, request.end,
create_rate_views=create_rate_views, deduplicate=deduplicate,
with_views=with_views, create_rate_views=create_rate_views,
include_account_events=include_account_events, with_views=with_views,
) include_account_events=include_account_events,
)
def update_history_with_config( # noqa: PLR0913 def update_history_with_config( # noqa: PLR0913
@@ -2171,6 +2173,36 @@ def mt5_summary_as_df(*, config: Mt5Config | None = None) -> pd.DataFrame:
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
def _emit_account_metrics(row: dict[str, object]) -> None:
login = str(row.get("login", ""))
server = str(row.get("server", ""))
get_metrics().record_account_state(
login=login,
server=server,
balance=float(row.get("balance") or 0.0), # type: ignore[arg-type]
equity=float(row.get("equity") or 0.0), # type: ignore[arg-type]
margin=float(row.get("margin") or 0.0), # type: ignore[arg-type]
margin_free=float(row.get("margin_free") or 0.0), # type: ignore[arg-type]
margin_level=float(row.get("margin_level") or 0.0), # type: ignore[arg-type]
)
def _emit_position_metrics(
rows: list[dict[str, object]],
login: int | None,
) -> None:
m = get_metrics()
login_str = str(login) if login is not None else ""
for r in rows:
m.record_position_state(
login=login_str,
server="",
symbol=str(r.get("symbol", "")),
profit=float(r.get("profit") or 0.0), # type: ignore[arg-type]
volume=float(r.get("volume") or 0.0), # type: ignore[arg-type]
)
def _snapshot_account( def _snapshot_account(
conn: sqlite3.Connection, conn: sqlite3.Connection,
client: Mt5DataClient, client: Mt5DataClient,
@@ -2184,6 +2216,7 @@ def _snapshot_account(
return None return None
row = cast("dict[str, object]", df.iloc[0].to_dict()) row = cast("dict[str, object]", df.iloc[0].to_dict())
insert_account_snapshot(conn, run_id, row) insert_account_snapshot(conn, run_id, row)
_emit_account_metrics(row)
login_val = row.get("login") login_val = row.get("login")
return int(login_val) if login_val is not None else None # type: ignore[arg-type] return int(login_val) if login_val is not None else None # type: ignore[arg-type]
@@ -2201,6 +2234,7 @@ def _snapshot_positions(
raw = df.to_dict(orient="records") if not df.empty else [] raw = df.to_dict(orient="records") if not df.empty else []
rows = cast("list[dict[str, object]]", raw) rows = cast("list[dict[str, object]]", raw)
insert_position_snapshots(conn, run_id, login, rows) insert_position_snapshots(conn, run_id, login, rows)
_emit_position_metrics(rows, login)
def _snapshot_orders( def _snapshot_orders(
@@ -2270,22 +2304,23 @@ def update_observability(
ensure_grafana_schema(conn) ensure_grafana_schema(conn)
else: else:
create_snapshot_tables(conn) create_snapshot_tables(conn)
run_id = start_snapshot_run(conn, observed_at) with get_metrics().record_snapshot_update():
login: int | None = None run_id = start_snapshot_run(conn, observed_at)
try: login: int | None = None
if include_account: try:
login = _snapshot_account(conn, client, run_id) if include_account:
if include_positions: login = _snapshot_account(conn, client, run_id)
_snapshot_positions(conn, client, run_id, login, symbols) if include_positions:
if include_orders: _snapshot_positions(conn, client, run_id, login, symbols)
_snapshot_orders(conn, client, run_id, login, symbols) if include_orders:
if include_terminal: _snapshot_orders(conn, client, run_id, login, symbols)
_snapshot_terminal(conn, client, run_id) if include_terminal:
record_snapshot_run(conn, run_id, "ok") _snapshot_terminal(conn, client, run_id)
except Exception: record_snapshot_run(conn, run_id, "ok")
record_snapshot_run(conn, run_id, "error") except Exception:
conn.commit() record_snapshot_run(conn, run_id, "error")
raise conn.commit()
raise
def update_observability_with_config( def update_observability_with_config(
+272
View File
@@ -0,0 +1,272 @@
"""Optional OpenTelemetry metrics for MT5 history and snapshot observability."""
from __future__ import annotations
import logging
import time
from contextlib import contextmanager
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from collections.abc import Iterator
logger = logging.getLogger(__name__)
_otel_available_flag = False
try:
import opentelemetry.metrics as _otel_metrics_mod
_otel_available_flag = True
except ImportError: # pragma: no cover
_otel_metrics_mod = None # type: ignore[assignment]
_OTEL_AVAILABLE: bool = _otel_available_flag
class _NoOp:
"""No-op instrument that silently ignores all calls."""
def add(
self,
amount: float,
attributes: dict[str, str] | None = None,
) -> None:
"""No-op add."""
def set(
self,
amount: float,
attributes: dict[str, str] | None = None,
) -> None:
"""No-op set."""
def record(
self,
amount: float,
attributes: dict[str, str] | None = None,
) -> None:
"""No-op record."""
_NOOP: _NoOp = _NoOp()
class _Mt5Metrics:
"""MT5 metric instrument registry.
Holds references to OTel instruments. All instruments are no-op until
:meth:`configure` is called with a compatible meter object.
"""
def __init__(self) -> None:
self._history_duration: Any = _NOOP
self._history_rows: Any = _NOOP
self._history_failures: Any = _NOOP
self._snapshot_duration: Any = _NOOP
self._snapshot_failures: Any = _NOOP
self._account_balance: Any = _NOOP
self._account_equity: Any = _NOOP
self._account_margin: Any = _NOOP
self._account_margin_free: Any = _NOOP
self._account_margin_level: Any = _NOOP
self._position_profit: Any = _NOOP
self._position_volume: Any = _NOOP
self._last_successful_update: Any = _NOOP
def configure(self, meter: Any) -> None: # noqa: ANN401
"""Set up metric instruments from a meter object.
Args:
meter: An OpenTelemetry ``Meter`` or duck-typed compatible object
that supports ``create_counter``, ``create_histogram``, and
``create_gauge``.
"""
self._history_duration = meter.create_histogram(
"mt5_history_update_duration_seconds",
unit="s",
description="Duration of incremental history update operations.",
)
self._history_rows = meter.create_counter(
"mt5_history_update_rows_total",
description="Rows written during incremental history updates.",
)
self._history_failures = meter.create_counter(
"mt5_history_update_failures_total",
description="Number of incremental history update failures.",
)
self._snapshot_duration = meter.create_histogram(
"mt5_snapshot_update_duration_seconds",
unit="s",
description="Duration of snapshot update operations.",
)
self._snapshot_failures = meter.create_counter(
"mt5_snapshot_update_failures_total",
description="Number of snapshot update failures.",
)
self._account_balance = meter.create_gauge(
"mt5_account_balance",
description="Account balance.",
)
self._account_equity = meter.create_gauge(
"mt5_account_equity",
description="Account equity.",
)
self._account_margin = meter.create_gauge(
"mt5_account_margin",
description="Account margin used.",
)
self._account_margin_free = meter.create_gauge(
"mt5_account_margin_free",
description="Account free margin.",
)
self._account_margin_level = meter.create_gauge(
"mt5_account_margin_level",
description="Account margin level as a percentage.",
)
self._position_profit = meter.create_gauge(
"mt5_position_profit",
description="Floating profit for an open position.",
)
self._position_volume = meter.create_gauge(
"mt5_position_volume",
description="Volume of an open position.",
)
self._last_successful_update = meter.create_gauge(
"mt5_last_successful_update_timestamp",
description="Unix timestamp of the last successful history update.",
)
@contextmanager
def record_history_update(
self,
*,
dataset: str,
) -> Iterator[None]:
"""Context manager recording history update duration and failures.
Args:
dataset: Dataset label (e.g. ``"rates"``).
Yields:
None inside the update operation.
"""
attrs = {"dataset": dataset}
start = time.monotonic()
try:
yield
self._history_duration.record(time.monotonic() - start, attrs)
self._last_successful_update.set(time.time(), attrs)
except Exception:
self._history_failures.add(1, attrs)
raise
@contextmanager
def record_snapshot_update(self) -> Iterator[None]:
"""Context manager recording snapshot update duration and failures.
Yields:
None inside the snapshot operation.
"""
start = time.monotonic()
try:
yield
self._snapshot_duration.record(time.monotonic() - start, {})
except Exception:
self._snapshot_failures.add(1, {})
raise
def record_account_state(
self,
*,
login: str,
server: str,
balance: float,
equity: float,
margin: float,
margin_free: float,
margin_level: float,
) -> None:
"""Emit account metric gauges.
Args:
login: Account login number (as string; not a password or secret).
server: Broker server name.
balance: Account balance.
equity: Account equity.
margin: Margin used.
margin_free: Free margin.
margin_level: Margin level percentage.
"""
attrs: dict[str, str] = {"login": login, "server": server}
self._account_balance.set(balance, attrs)
self._account_equity.set(equity, attrs)
self._account_margin.set(margin, attrs)
self._account_margin_free.set(margin_free, attrs)
self._account_margin_level.set(margin_level, attrs)
def record_position_state(
self,
*,
login: str,
server: str,
symbol: str,
profit: float,
volume: float,
) -> None:
"""Emit position metric gauges.
Args:
login: Account login number (as string).
server: Broker server name.
symbol: Position symbol.
profit: Floating profit/loss.
volume: Position volume.
"""
attrs: dict[str, str] = {"login": login, "server": server, "symbol": symbol}
self._position_profit.set(profit, attrs)
self._position_volume.set(volume, attrs)
_metrics = _Mt5Metrics()
def configure_metrics(meter: Any) -> None: # noqa: ANN401
"""Configure MT5 metrics using the provided meter.
Args:
meter: An OpenTelemetry ``Meter`` or duck-typed compatible object.
"""
_metrics.configure(meter)
def enable_otel_metrics(service_name: str = "mt5cli") -> None:
"""Enable OTel metrics using the global meter provider.
Requires the ``otel`` optional dependency group:
``pip install "mt5cli[otel]"``.
Args:
service_name: OTel meter/service name.
Raises:
ImportError: If ``opentelemetry-api`` is not installed.
"""
if not _OTEL_AVAILABLE:
msg = (
"opentelemetry-api is not installed. "
'Install it with: pip install "mt5cli[otel]"'
)
raise ImportError(msg)
meter = _otel_metrics_mod.get_meter(service_name) # type: ignore[union-attr]
configure_metrics(meter)
def get_metrics() -> _Mt5Metrics:
"""Return the global :class:`_Mt5Metrics` instance.
Returns:
The global metric registry (no-op until :func:`configure_metrics` is
called).
"""
return _metrics
+7
View File
@@ -26,6 +26,11 @@ classifiers = [
[project.optional-dependencies] [project.optional-dependencies]
parquet = ["pyarrow >= 19.0.0"] parquet = ["pyarrow >= 19.0.0"]
otel = [
"opentelemetry-api",
"opentelemetry-sdk",
"opentelemetry-exporter-otlp-proto-http",
]
[project.scripts] [project.scripts]
mt5cli = "mt5cli.cli:main" mt5cli = "mt5cli.cli:main"
@@ -45,6 +50,8 @@ dev = [
"pytest-cov >= 5.0.0", "pytest-cov >= 5.0.0",
"pandas-stubs >= 2.2.3.250527", "pandas-stubs >= 2.2.3.250527",
"pyarrow >= 19.0.0", "pyarrow >= 19.0.0",
"opentelemetry-api",
"opentelemetry-sdk",
"mkdocs >= 1.6.1", "mkdocs >= 1.6.1",
"mkdocs-material >= 9.7.6", "mkdocs-material >= 9.7.6",
"mkdocstrings[python] >= 1.0.4", "mkdocstrings[python] >= 1.0.4",
+71
View File
@@ -2022,6 +2022,77 @@ class TestSnapshotCommand:
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
assert updater.call_args.kwargs["with_grafana_schema"] is False assert updater.call_args.kwargs["with_grafana_schema"] is False
def test_snapshot_with_publish_copy(
self,
tmp_path: Path,
mocker: MockerFixture,
) -> None:
"""--publish-copy calls publish_grafana_copy after update_observability."""
mocker.patch("mt5cli.cli.sdk.update_observability_with_config")
mock_publish = mocker.patch("mt5cli.grafana.publish_grafana_copy")
copy_path = tmp_path / "grafana.db"
result = runner.invoke(
app,
[
"-o",
str(tmp_path / "out.db"),
"snapshot",
"--publish-copy",
str(copy_path),
],
)
assert result.exit_code == 0, result.output
mock_publish.assert_called_once()
def test_snapshot_no_publish_copy_by_default(
self,
tmp_path: Path,
mocker: MockerFixture,
) -> None:
"""Snapshot does not call publish_grafana_copy without --publish-copy."""
mocker.patch("mt5cli.cli.sdk.update_observability_with_config")
mock_publish = mocker.patch("mt5cli.grafana.publish_grafana_copy")
result = runner.invoke(app, ["-o", str(tmp_path / "out.db"), "snapshot"])
assert result.exit_code == 0, result.output
mock_publish.assert_not_called()
class TestGrafanaSchemaPublishCopy:
"""Tests for grafana-schema --publish-copy option."""
def test_grafana_schema_with_publish_copy(
self,
tmp_path: Path,
mocker: MockerFixture,
) -> None:
"""grafana-schema --publish-copy calls publish_grafana_copy."""
mock_publish = mocker.patch("mt5cli.grafana.publish_grafana_copy")
output = tmp_path / "out.db"
copy_path = tmp_path / "grafana.db"
result = runner.invoke(
app,
[
"-o",
str(output),
"grafana-schema",
"--publish-copy",
str(copy_path),
],
)
assert result.exit_code == 0, result.output
mock_publish.assert_called_once()
def test_grafana_schema_no_publish_copy_by_default(
self,
tmp_path: Path,
mocker: MockerFixture,
) -> None:
"""grafana-schema does not call publish_grafana_copy by default."""
mock_publish = mocker.patch("mt5cli.grafana.publish_grafana_copy")
result = runner.invoke(app, ["-o", str(tmp_path / "out.db"), "grafana-schema"])
assert result.exit_code == 0, result.output
mock_publish.assert_not_called()
class TestMain: class TestMain:
"""Tests for the main entry point.""" """Tests for the main entry point."""
+75
View File
@@ -0,0 +1,75 @@
"""Tests for example files in examples/grafana/."""
from __future__ import annotations
import json
from pathlib import Path
_EXAMPLES_DIR = Path(__file__).parent.parent / "examples" / "grafana"
_DASHBOARDS_DIR = _EXAMPLES_DIR / "dashboards"
class TestGrafanaExamples:
"""Validate structure and content of bundled Grafana example files."""
def test_dashboard_json_files_are_valid_json(self) -> None:
"""All dashboard JSON files parse without error."""
paths = list(_DASHBOARDS_DIR.glob("*.json"))
assert paths, "No dashboard JSON files found"
for path in paths:
content = path.read_text(encoding="utf-8")
obj = json.loads(content)
assert isinstance(obj, dict), f"{path.name} root must be a JSON object"
def test_dashboard_json_has_no_private_placeholders(self) -> None:
"""Dashboard JSON files contain no obvious credential placeholders."""
private_patterns = ["password", "api_key", "apikey"]
for path in _DASHBOARDS_DIR.glob("*.json"):
content = path.read_text(encoding="utf-8").lower()
for pat in private_patterns:
assert pat not in content, f"{path.name} contains {pat!r}"
def test_dashboard_json_uses_grafana_views(self) -> None:
"""All dashboard JSON files query grafana_* views."""
for path in _DASHBOARDS_DIR.glob("*.json"):
content = path.read_text(encoding="utf-8")
assert "grafana_" in content, (
f"{path.name} must contain queries against grafana_* views"
)
def test_dashboard_json_has_uid(self) -> None:
"""All dashboard JSON files have a non-empty uid field."""
for path in _DASHBOARDS_DIR.glob("*.json"):
obj = json.loads(path.read_text(encoding="utf-8"))
assert obj.get("uid"), f"{path.name} must have a uid"
def test_dashboard_json_has_title(self) -> None:
"""All dashboard JSON files have a non-empty title field."""
for path in _DASHBOARDS_DIR.glob("*.json"):
obj = json.loads(path.read_text(encoding="utf-8"))
assert obj.get("title"), f"{path.name} must have a title"
def test_expected_dashboards_present(self) -> None:
"""The three expected dashboard files are present."""
names = {p.name for p in _DASHBOARDS_DIR.glob("*.json")}
assert "mt5cli-overview.json" in names
assert "mt5cli-trades.json" in names
assert "mt5cli-market.json" in names
def test_readme_exists(self) -> None:
"""examples/grafana/README.md is present."""
assert (_EXAMPLES_DIR / "README.md").is_file()
def test_docker_compose_exists(self) -> None:
"""examples/grafana/docker-compose.yml is present."""
assert (_EXAMPLES_DIR / "docker-compose.yml").is_file()
def test_datasource_provisioning_exists(self) -> None:
"""Datasource provisioning YAML is present."""
assert (
_EXAMPLES_DIR / "provisioning" / "datasources" / "mt5cli-sqlite.yaml"
).is_file()
def test_dashboard_provisioning_exists(self) -> None:
"""Dashboard provisioning YAML is present."""
assert (_EXAMPLES_DIR / "provisioning" / "dashboards" / "mt5cli.yaml").is_file()
+121 -1
View File
@@ -4,8 +4,9 @@ from __future__ import annotations
import logging import logging
import sqlite3 import sqlite3
from pathlib import Path
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
from unittest.mock import MagicMock from unittest.mock import MagicMock, patch
import pandas as pd import pandas as pd
import pytest import pytest
@@ -24,6 +25,7 @@ from mt5cli.grafana import (
insert_order_snapshots, insert_order_snapshots,
insert_position_snapshots, insert_position_snapshots,
insert_terminal_snapshot, insert_terminal_snapshot,
publish_grafana_copy,
record_snapshot_run, record_snapshot_run,
start_snapshot_run, start_snapshot_run,
) )
@@ -836,3 +838,121 @@ class TestSnapshotInserts:
record_snapshot_run(conn, run_id, "ok") record_snapshot_run(conn, run_id, "ok")
row = conn.execute("SELECT status, detail FROM snapshot_runs").fetchone() row = conn.execute("SELECT status, detail FROM snapshot_runs").fetchone()
assert row == ("ok", None) assert row == ("ok", None)
# ---------------------------------------------------------------------------
# TestPublishGrafanaCopy
# ---------------------------------------------------------------------------
def _make_source_db(path: Path) -> None:
"""Create a minimal source SQLite database with snapshot tables."""
with sqlite3.connect(path) as conn:
conn.execute("PRAGMA journal_mode=WAL")
create_snapshot_tables(conn)
conn.execute(
"INSERT INTO snapshot_runs (observed_at, status) VALUES (?, 'ok')",
(1700000000,),
)
class TestPublishGrafanaCopy:
"""Tests for publish_grafana_copy."""
def test_publish_to_fresh_target(self, tmp_path: Path) -> None:
"""publish_grafana_copy creates the target file."""
source = tmp_path / "src.db"
target = tmp_path / "out" / "grafana.db"
_make_source_db(source)
result = publish_grafana_copy(source, target)
assert target.exists()
assert result == target.resolve()
def test_overwrite_existing_target(self, tmp_path: Path) -> None:
"""publish_grafana_copy replaces an existing target without error."""
source = tmp_path / "src.db"
target = tmp_path / "grafana.db"
_make_source_db(source)
target.write_bytes(b"stale")
publish_grafana_copy(source, target)
# Target must now be a valid SQLite file from source
with sqlite3.connect(target) as conn:
tables = {
row[0]
for row in conn.execute(
"SELECT name FROM sqlite_master WHERE type='table'"
).fetchall()
}
assert "snapshot_runs" in tables
def test_target_contains_source_tables(self, tmp_path: Path) -> None:
"""Published target contains the same tables as the source."""
source = tmp_path / "src.db"
target = tmp_path / "grafana.db"
_make_source_db(source)
publish_grafana_copy(source, target)
with sqlite3.connect(target) as conn:
tables = {
row[0]
for row in conn.execute(
"SELECT name FROM sqlite_master WHERE type='table'"
).fetchall()
}
assert {"snapshot_runs", "account_snapshots"}.issubset(tables)
def test_target_can_be_opened_readonly(self, tmp_path: Path) -> None:
"""Published target can be opened with uri=True in read-only mode."""
source = tmp_path / "src.db"
target = tmp_path / "grafana.db"
_make_source_db(source)
publish_grafana_copy(source, target)
uri = f"file:{target}?mode=ro"
with sqlite3.connect(uri, uri=True) as conn:
row = conn.execute("SELECT status FROM snapshot_runs").fetchone()
assert row == ("ok",)
def test_source_not_found_raises(self, tmp_path: Path) -> None:
"""publish_grafana_copy raises FileNotFoundError when source is absent."""
with pytest.raises(FileNotFoundError):
publish_grafana_copy(tmp_path / "missing.db", tmp_path / "out.db")
def test_preserve_old_target_on_backup_failure(self, tmp_path: Path) -> None:
"""Old target is preserved when the backup fails."""
source = tmp_path / "src.db"
target = tmp_path / "grafana.db"
_make_source_db(source)
original_content = b"original_data"
target.write_bytes(original_content)
with patch("sqlite3.connect") as mock_connect:
mock_src = MagicMock()
mock_src.__enter__ = MagicMock(return_value=mock_src)
mock_src.__exit__ = MagicMock(return_value=False)
mock_src.backup.side_effect = sqlite3.OperationalError("backup failed")
mock_connect.return_value = mock_src
with pytest.raises(sqlite3.OperationalError, match="backup failed"):
publish_grafana_copy(source, target)
assert target.read_bytes() == original_content
def test_temp_file_cleaned_up_on_failure(self, tmp_path: Path) -> None:
"""Temporary file is removed when backup raises an exception."""
source = tmp_path / "src.db"
target = tmp_path / "grafana.db"
_make_source_db(source)
with patch("sqlite3.connect") as mock_connect:
mock_src = MagicMock()
mock_src.__enter__ = MagicMock(return_value=mock_src)
mock_src.__exit__ = MagicMock(return_value=False)
mock_src.backup.side_effect = sqlite3.OperationalError("fail")
mock_connect.return_value = mock_src
with pytest.raises(sqlite3.OperationalError):
publish_grafana_copy(source, target)
tmp_files = list(tmp_path.glob("grafana.db.*.tmp"))
assert not tmp_files, "Temp file should be cleaned up on failure"
def test_returns_path_object(self, tmp_path: Path) -> None:
"""publish_grafana_copy returns a Path instance."""
source = tmp_path / "src.db"
target = tmp_path / "grafana.db"
_make_source_db(source)
result = publish_grafana_copy(source, target)
assert isinstance(result, Path)
+59
View File
@@ -3120,3 +3120,62 @@ class TestUpdateObservability:
call_kwargs = spy.call_args.kwargs call_kwargs = spy.call_args.kwargs
assert call_kwargs["symbols"] == ["EURUSD"] assert call_kwargs["symbols"] == ["EURUSD"]
assert call_kwargs["include_account"] is False assert call_kwargs["include_account"] is False
def test_update_observability_invokes_snapshot_telemetry(
self,
mock_client: MagicMock,
mocker: MockerFixture,
tmp_path: Path,
) -> None:
"""update_observability calls record_snapshot_update on the global metrics."""
mock_metrics = MagicMock()
mock_cm = MagicMock()
mock_cm.__enter__ = MagicMock(return_value=None)
mock_cm.__exit__ = MagicMock(return_value=False)
mock_metrics.record_snapshot_update.return_value = mock_cm
mocker.patch("mt5cli.sdk.get_metrics", return_value=mock_metrics)
update_observability(client=mock_client, output=tmp_path / "obs.db")
mock_metrics.record_snapshot_update.assert_called_once()
def test_update_observability_emits_account_metrics(
self,
mock_client: MagicMock,
mocker: MockerFixture,
tmp_path: Path,
) -> None:
"""_snapshot_account emits account gauges via get_metrics."""
mock_metrics = MagicMock()
mock_cm = MagicMock()
mock_cm.__enter__ = MagicMock(return_value=None)
mock_cm.__exit__ = MagicMock(return_value=False)
mock_metrics.record_snapshot_update.return_value = mock_cm
mocker.patch("mt5cli.sdk.get_metrics", return_value=mock_metrics)
update_observability(client=mock_client, output=tmp_path / "obs.db")
mock_metrics.record_account_state.assert_called_once()
class TestUpdateHistoryTelemetry:
"""Tests for telemetry hooks in update_history."""
def test_update_history_invokes_history_telemetry(
self,
mocker: MockerFixture,
tmp_path: Path,
) -> None:
"""update_history wraps write_incremental_datasets with telemetry."""
mock_client = MagicMock()
mock_client.copy_rates_range_as_df.return_value = pd.DataFrame()
mock_client.history_orders_get_as_df.return_value = pd.DataFrame()
mock_client.history_deals_get_as_df.return_value = pd.DataFrame()
mock_metrics = MagicMock()
mock_cm = MagicMock()
mock_cm.__enter__ = MagicMock(return_value=None)
mock_cm.__exit__ = MagicMock(return_value=False)
mock_metrics.record_history_update.return_value = mock_cm
mocker.patch("mt5cli.sdk.get_metrics", return_value=mock_metrics)
update_history(
client=mock_client,
output=tmp_path / "hist.db",
symbols=["EURUSD"],
)
mock_metrics.record_history_update.assert_called_once_with(dataset="history")
+204
View File
@@ -0,0 +1,204 @@
"""Tests for mt5cli.telemetry module."""
from __future__ import annotations
from unittest.mock import MagicMock
import pytest
from mt5cli.telemetry import (
_OTEL_AVAILABLE, # type: ignore[reportPrivateUsage]
_Mt5Metrics, # type: ignore[reportPrivateUsage]
_NoOp, # type: ignore[reportPrivateUsage]
configure_metrics,
enable_otel_metrics,
get_metrics,
)
class TestNoOp:
"""Tests for _NoOp no-op instrument."""
def test_add_is_noop(self) -> None:
"""_NoOp.add accepts amount and optional attributes without error."""
noop = _NoOp()
noop.add(1.0)
noop.add(1.0, {"key": "val"})
def test_set_is_noop(self) -> None:
"""_NoOp.set accepts amount and optional attributes without error."""
noop = _NoOp()
noop.set(2.0)
noop.set(2.0, {"key": "val"})
def test_record_is_noop(self) -> None:
"""_NoOp.record accepts amount and optional attributes without error."""
noop = _NoOp()
noop.record(3.0)
noop.record(3.0, {"key": "val"})
class TestMt5Metrics:
"""Tests for _Mt5Metrics."""
def test_default_instruments_are_noop(self) -> None:
"""Default _Mt5Metrics methods do not raise before configure is called."""
m = _Mt5Metrics()
m.record_account_state(
login="123",
server="demo",
balance=1000.0,
equity=1050.0,
margin=100.0,
margin_free=950.0,
margin_level=1050.0,
)
def test_configure_calls_meter(self) -> None:
"""configure() calls create_histogram, create_counter, create_gauge on meter."""
meter = MagicMock()
m = _Mt5Metrics()
m.configure(meter)
assert meter.create_histogram.called
assert meter.create_counter.called
assert meter.create_gauge.called
def test_record_history_update_success(self) -> None:
"""record_history_update records duration and timestamp on success."""
meter = MagicMock()
m = _Mt5Metrics()
m.configure(meter)
with m.record_history_update(dataset="rates"):
pass
m._history_duration.record.assert_called_once() # type: ignore[reportPrivateUsage]
m._last_successful_update.set.assert_called_once() # type: ignore[reportPrivateUsage]
m._history_failures.add.assert_not_called() # type: ignore[reportPrivateUsage]
def test_record_history_update_failure(self) -> None:
"""record_history_update increments failure counter and re-raises on error."""
meter = MagicMock()
m = _Mt5Metrics()
m.configure(meter)
exc = ValueError("boom")
with (
pytest.raises(ValueError, match="boom"),
m.record_history_update(dataset="rates"),
):
raise exc
m._history_failures.add.assert_called_once_with( # type: ignore[reportPrivateUsage]
1, {"dataset": "rates"}
)
m._history_duration.record.assert_not_called() # type: ignore[reportPrivateUsage]
def test_record_snapshot_update_success(self) -> None:
"""record_snapshot_update records duration on success."""
meter = MagicMock()
m = _Mt5Metrics()
m.configure(meter)
with m.record_snapshot_update():
pass
m._snapshot_duration.record.assert_called_once() # type: ignore[reportPrivateUsage]
m._snapshot_failures.add.assert_not_called() # type: ignore[reportPrivateUsage]
def test_record_snapshot_update_failure(self) -> None:
"""record_snapshot_update increments failure counter and re-raises on error."""
meter = MagicMock()
m = _Mt5Metrics()
m.configure(meter)
exc = RuntimeError("snap fail")
with (
pytest.raises(RuntimeError, match="snap fail"),
m.record_snapshot_update(),
):
raise exc
m._snapshot_failures.add.assert_called_once_with(1, {}) # type: ignore[reportPrivateUsage]
m._snapshot_duration.record.assert_not_called() # type: ignore[reportPrivateUsage]
def test_record_position_state(self) -> None:
"""record_position_state emits profit and volume gauges."""
meter = MagicMock()
m = _Mt5Metrics()
m.configure(meter)
m.record_position_state(
login="42",
server="demo",
symbol="EURUSD",
profit=12.5,
volume=0.01,
)
# Both profit and volume share the same gauge mock via create_gauge;
# verify that set was called exactly twice (once each).
assert m._position_profit.set.call_count == 2 # type: ignore[reportPrivateUsage]
def test_record_account_state_after_configure(self) -> None:
"""record_account_state emits all five account gauges."""
meter = MagicMock()
m = _Mt5Metrics()
m.configure(meter)
m.record_account_state(
login="99",
server="live",
balance=5000.0,
equity=5100.0,
margin=200.0,
margin_free=4800.0,
margin_level=2550.0,
)
# All five account gauges share the same gauge mock; set is called 5×.
assert m._account_balance.set.call_count == 5 # type: ignore[reportPrivateUsage]
def test_record_history_update_noop_before_configure(self) -> None:
"""record_history_update works without configure (no-op instruments)."""
m = _Mt5Metrics()
with m.record_history_update(dataset="ticks"):
pass
def test_record_snapshot_update_noop_before_configure(self) -> None:
"""record_snapshot_update works without configure (no-op instruments)."""
m = _Mt5Metrics()
with m.record_snapshot_update():
pass
class TestConfigureMetrics:
"""Tests for configure_metrics and get_metrics."""
def test_configure_metrics_updates_global(self) -> None:
"""configure_metrics wires up the global singleton."""
meter = MagicMock()
configure_metrics(meter)
assert get_metrics() is get_metrics()
def test_get_metrics_returns_mt5metrics(self) -> None:
"""get_metrics returns the global _Mt5Metrics instance."""
assert isinstance(get_metrics(), _Mt5Metrics)
class TestEnableOtelMetrics:
"""Tests for enable_otel_metrics."""
def test_enable_raises_when_unavailable(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""enable_otel_metrics raises ImportError when OTel is not installed."""
monkeypatch.setattr("mt5cli.telemetry._OTEL_AVAILABLE", False)
with pytest.raises(ImportError, match="opentelemetry-api"):
enable_otel_metrics()
def test_enable_uses_global_meter_provider(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""enable_otel_metrics calls get_meter on the OTel metrics module."""
mock_meter = MagicMock()
mock_mod = MagicMock()
mock_mod.get_meter.return_value = mock_meter
monkeypatch.setattr("mt5cli.telemetry._OTEL_AVAILABLE", True)
monkeypatch.setattr("mt5cli.telemetry._otel_metrics_mod", mock_mod)
enable_otel_metrics("my-service")
mock_mod.get_meter.assert_called_once_with("my-service")
def test_otel_available_flag_is_bool(self) -> None:
"""_OTEL_AVAILABLE is a boolean."""
assert isinstance(_OTEL_AVAILABLE, bool)
Generated
+122 -2
View File
@@ -223,6 +223,18 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/f7/ec/67fbef5d497f86283db54c22eec6f6140243aae73265799baaaa19cd17fb/ghp_import-2.1.0-py3-none-any.whl", hash = "sha256:8337dd7b50877f163d4c0289bc1f1c7f127550241988d568c1db512c4324a619", size = 11034, upload-time = "2022-05-02T15:47:14.552Z" }, { url = "https://files.pythonhosted.org/packages/f7/ec/67fbef5d497f86283db54c22eec6f6140243aae73265799baaaa19cd17fb/ghp_import-2.1.0-py3-none-any.whl", hash = "sha256:8337dd7b50877f163d4c0289bc1f1c7f127550241988d568c1db512c4324a619", size = 11034, upload-time = "2022-05-02T15:47:14.552Z" },
] ]
[[package]]
name = "googleapis-common-protos"
version = "1.75.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "protobuf" },
]
sdist = { url = "https://files.pythonhosted.org/packages/b5/c8/f439cffde755cffa462bfbb156278fa6f9d09119719af9814b858fd4f81f/googleapis_common_protos-1.75.0.tar.gz", hash = "sha256:53a062ff3c32552fbd62c11fe23768b78e4ddf0494d5e5fd97d3f4689c75fbbd", size = 151035, upload-time = "2026-05-07T08:04:49.423Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/e7/c8/e2645aa8ed02fd4c7a2f59d68783b65b1f3cbdfe39a6308e156509d1fee8/googleapis_common_protos-1.75.0-py3-none-any.whl", hash = "sha256:961ed60399c457ceb0ee8f285a84c870aabc9c6a832b9d37bb281b5bebde43ed", size = 300631, upload-time = "2026-05-07T08:03:30.345Z" },
]
[[package]] [[package]]
name = "griffelib" name = "griffelib"
version = "2.0.2" version = "2.0.2"
@@ -358,7 +370,7 @@ name = "metatrader5"
version = "5.0.5640" version = "5.0.5640"
source = { registry = "https://pypi.org/simple" } source = { registry = "https://pypi.org/simple" }
dependencies = [ dependencies = [
{ name = "numpy" }, { name = "numpy", marker = "sys_platform == 'win32'" },
] ]
wheels = [ wheels = [
{ url = "https://files.pythonhosted.org/packages/ef/a0/3b764c6743ef601ff12f7d8d62ca5768eb25e90d3758ac7e1d08e667af85/metatrader5-5.0.5640-cp311-cp311-win_amd64.whl", hash = "sha256:4057255f2d63138a3ea1a5d492715038a71d3177cad793fca97a5c58771b4eb1", size = 48091, upload-time = "2026-02-20T23:31:12.289Z" }, { url = "https://files.pythonhosted.org/packages/ef/a0/3b764c6743ef601ff12f7d8d62ca5768eb25e90d3758ac7e1d08e667af85/metatrader5-5.0.5640-cp311-cp311-win_amd64.whl", hash = "sha256:4057255f2d63138a3ea1a5d492715038a71d3177cad793fca97a5c58771b4eb1", size = 48091, upload-time = "2026-02-20T23:31:12.289Z" },
@@ -496,6 +508,11 @@ dependencies = [
] ]
[package.optional-dependencies] [package.optional-dependencies]
otel = [
{ name = "opentelemetry-api" },
{ name = "opentelemetry-exporter-otlp-proto-http" },
{ name = "opentelemetry-sdk" },
]
parquet = [ parquet = [
{ name = "pyarrow" }, { name = "pyarrow" },
] ]
@@ -505,6 +522,8 @@ dev = [
{ name = "mkdocs" }, { name = "mkdocs" },
{ name = "mkdocs-material" }, { name = "mkdocs-material" },
{ name = "mkdocstrings", extra = ["python"] }, { name = "mkdocstrings", extra = ["python"] },
{ name = "opentelemetry-api" },
{ name = "opentelemetry-sdk" },
{ name = "pandas-stubs" }, { name = "pandas-stubs" },
{ name = "pyarrow" }, { name = "pyarrow" },
{ name = "pymdown-extensions" }, { name = "pymdown-extensions" },
@@ -518,17 +537,22 @@ dev = [
[package.metadata] [package.metadata]
requires-dist = [ requires-dist = [
{ name = "click", specifier = ">=8.1.0" }, { name = "click", specifier = ">=8.1.0" },
{ name = "opentelemetry-api", marker = "extra == 'otel'" },
{ name = "opentelemetry-exporter-otlp-proto-http", marker = "extra == 'otel'" },
{ name = "opentelemetry-sdk", marker = "extra == 'otel'" },
{ name = "pdmt5", specifier = ">=1.0.0" }, { name = "pdmt5", specifier = ">=1.0.0" },
{ name = "pyarrow", marker = "extra == 'parquet'", specifier = ">=19.0.0" }, { name = "pyarrow", marker = "extra == 'parquet'", specifier = ">=19.0.0" },
{ name = "typer", specifier = ">=0.15.0" }, { name = "typer", specifier = ">=0.15.0" },
] ]
provides-extras = ["parquet"] provides-extras = ["parquet", "otel"]
[package.metadata.requires-dev] [package.metadata.requires-dev]
dev = [ dev = [
{ name = "mkdocs", specifier = ">=1.6.1" }, { name = "mkdocs", specifier = ">=1.6.1" },
{ name = "mkdocs-material", specifier = ">=9.7.6" }, { name = "mkdocs-material", specifier = ">=9.7.6" },
{ name = "mkdocstrings", extras = ["python"], specifier = ">=1.0.4" }, { name = "mkdocstrings", extras = ["python"], specifier = ">=1.0.4" },
{ name = "opentelemetry-api" },
{ name = "opentelemetry-sdk" },
{ name = "pandas-stubs", specifier = ">=2.2.3.250527" }, { name = "pandas-stubs", specifier = ">=2.2.3.250527" },
{ name = "pyarrow", specifier = ">=19.0.0" }, { name = "pyarrow", specifier = ">=19.0.0" },
{ name = "pymdown-extensions", specifier = ">=10.21.2" }, { name = "pymdown-extensions", specifier = ">=10.21.2" },
@@ -606,6 +630,87 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/57/a7/b35835e278c18b85206834b3aa3abe68e77a98769c59233d1f6300284781/numpy-2.4.3-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:4b42639cdde6d24e732ff823a3fa5b701d8acad89c4142bc1d0bd6dc85200ba5", size = 12504685, upload-time = "2026-03-09T07:58:50.525Z" }, { url = "https://files.pythonhosted.org/packages/57/a7/b35835e278c18b85206834b3aa3abe68e77a98769c59233d1f6300284781/numpy-2.4.3-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:4b42639cdde6d24e732ff823a3fa5b701d8acad89c4142bc1d0bd6dc85200ba5", size = 12504685, upload-time = "2026-03-09T07:58:50.525Z" },
] ]
[[package]]
name = "opentelemetry-api"
version = "1.43.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/ae/cc/e4c9584181f86494df0f6bdec1a4f3280c50db44704dc2a407e994fc87bb/opentelemetry_api-1.43.0.tar.gz", hash = "sha256:107d0d03857ea8fc7c5fcbbbd83f800c281f0d560553d61c1d675fccfd1761c1", size = 73476, upload-time = "2026-06-24T15:19:55.323Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/17/83/6dba32b85f31868400440dc7ad2ca1eab94cbbf3a7b0459ed39f8311a9e2/opentelemetry_api-1.43.0-py3-none-any.whl", hash = "sha256:20acf45e9b21851926835292e4045d290acade1edd2ff3de86d2f069687ba1fd", size = 61912, upload-time = "2026-06-24T15:19:35.434Z" },
]
[[package]]
name = "opentelemetry-exporter-otlp-proto-common"
version = "1.43.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "opentelemetry-proto" },
]
sdist = { url = "https://files.pythonhosted.org/packages/55/c1/e8098490ab15abf116dcaf9fa89ededcb35547c7d08d4b5a62f573dc1e63/opentelemetry_exporter_otlp_proto_common-1.43.0.tar.gz", hash = "sha256:c4e32ba6d6b13bdb2b8f6764c4fd28d00192826561aa04f6d14eedfce7ac076f", size = 20197, upload-time = "2026-06-24T15:20:00.247Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/d0/b2/41ebc74ae1d5859901f1b69305de58724bf043381103d6ef413521cbc35a/opentelemetry_exporter_otlp_proto_common-1.43.0-py3-none-any.whl", hash = "sha256:123c3f9cc87218562490c63b36f497bf3a722faf174a515d1443f31ababa6264", size = 17048, upload-time = "2026-06-24T15:19:41.264Z" },
]
[[package]]
name = "opentelemetry-exporter-otlp-proto-http"
version = "1.43.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "googleapis-common-protos" },
{ name = "opentelemetry-api" },
{ name = "opentelemetry-exporter-otlp-proto-common" },
{ name = "opentelemetry-proto" },
{ name = "opentelemetry-sdk" },
{ name = "requests" },
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/fc/92/0b9f56412483a8891d4843890294796c9df8ab42417bd9bad8035d840cb3/opentelemetry_exporter_otlp_proto_http-1.43.0.tar.gz", hash = "sha256:fa8a42bb7d00ee5391f4c0b04d8e6a46c03caa437903296ab73a81dc11ba118f", size = 25406, upload-time = "2026-06-24T15:20:01.515Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b3/20/b685ed7af2e17c29ffc8af56f1fa8bc2033258fc30fb0d2b722f49d13ba0/opentelemetry_exporter_otlp_proto_http-1.43.0-py3-none-any.whl", hash = "sha256:647f603aa8efdbdb4dbff842e0729d0406a6fff26b295a72d3d60e7d963b2610", size = 21795, upload-time = "2026-06-24T15:19:43.164Z" },
]
[[package]]
name = "opentelemetry-proto"
version = "1.43.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "protobuf" },
]
sdist = { url = "https://files.pythonhosted.org/packages/e0/b9/d357faefb40bda1d4799913e6af611171ff22a2dedcb93576bc92242d056/opentelemetry_proto-1.43.0.tar.gz", hash = "sha256:224778df17e1f3fafeaaa21d874236ca5f6ffc2f86e0899298ec7351aac27924", size = 46481, upload-time = "2026-06-24T15:20:07.625Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/ed/a7/3e5308cf548b8f72529c7db1afdb3a404211982376a12927fd7759f77bf3/opentelemetry_proto-1.43.0-py3-none-any.whl", hash = "sha256:c58f1f7ef84bc7dc2834016c0c37fe0081dde7ca9f6339be1970fbf9cdaaa90d", size = 72489, upload-time = "2026-06-24T15:19:51.164Z" },
]
[[package]]
name = "opentelemetry-sdk"
version = "1.43.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "opentelemetry-api" },
{ name = "opentelemetry-semantic-conventions" },
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/3e/eb/5041074274ac0956b03637cc039d434569112468e875eddfcc9a0674ce06/opentelemetry_sdk-1.43.0.tar.gz", hash = "sha256:d8187c81c162df9913e4003dd6485f7390d9a24fc17026ec7387b8b8218b08e9", size = 254744, upload-time = "2026-06-24T15:20:08.467Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/49/e3/b17be23af124201c9f52eececd4cc8ddfed1597d37b4ee771895d325805c/opentelemetry_sdk-1.43.0-py3-none-any.whl", hash = "sha256:d1323a547c1ce69d6a069a17a44b7da82bb8b332051ecb074041f87642c86823", size = 178852, upload-time = "2026-06-24T15:19:52.169Z" },
]
[[package]]
name = "opentelemetry-semantic-conventions"
version = "0.64b0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "opentelemetry-api" },
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/5a/30/5f26df29509eccd86b99b481ac9ffa39da49ba9577cc69071c552ae30447/opentelemetry_semantic_conventions-0.64b0.tar.gz", hash = "sha256:72f76fb2d1582d9d033dd1fcd84532e961e6ff3d90d24ba6fabc72975a83864c", size = 148340, upload-time = "2026-06-24T15:20:09.267Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/f2/ca/23ba87a221b574a7c5a99d48849d80bfe8b047624681357e2b002e566187/opentelemetry_semantic_conventions-0.64b0-py3-none-any.whl", hash = "sha256:ea77e85e354b8f604ddbe5f3d9135216f982fa4d77e5859ac30f6d8a50505aa6", size = 203713, upload-time = "2026-06-24T15:19:53.339Z" },
]
[[package]] [[package]]
name = "packaging" name = "packaging"
version = "26.0" version = "26.0"
@@ -721,6 +826,21 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" }, { url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" },
] ]
[[package]]
name = "protobuf"
version = "7.35.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/da/01/9ef0afd7999eb9badb3a768b4aedd78c86d4c65cfaf1958ab276199e76b4/protobuf-7.35.1.tar.gz", hash = "sha256:ce115a26fe0c39a2c29973d914d327e516a6455464489fe3cd1e51a1b354f81a", size = 458717, upload-time = "2026-06-11T21:55:40.257Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/10/03/8aeeb7458d22546bf64b5250ca1daeb5ff757d900e8e4a7476c6f0db843e/protobuf-7.35.1-cp310-abi3-macosx_10_9_universal2.whl", hash = "sha256:24f857477359a85c0c235261b8ba905fd51b2562f4a64ca1df5473f29850cbf6", size = 433226, upload-time = "2026-06-11T21:55:31.719Z" },
{ url = "https://files.pythonhosted.org/packages/37/4b/dfb89eb0e652a1ff073c39a59fb5e3a83cfe9b57a2c83fa6d78270101767/protobuf-7.35.1-cp310-abi3-manylinux2014_aarch64.whl", hash = "sha256:11d6b0ec246892d85215b0a13ca6e0233cf5284b68f0ac02646427f4ff88a799", size = 328847, upload-time = "2026-06-11T21:55:34.035Z" },
{ url = "https://files.pythonhosted.org/packages/0f/58/dc12f2cd484951524af6e3382c785869b9b3fb5e52ee95ae23add53ee8f9/protobuf-7.35.1-cp310-abi3-manylinux2014_s390x.whl", hash = "sha256:b73f9489a4b8b1c9cb1f8ed951c736392592edb24b9d6819f36d2e10b171d5b4", size = 344030, upload-time = "2026-06-11T21:55:34.941Z" },
{ url = "https://files.pythonhosted.org/packages/e4/be/5b3cfe508bfab6761414ff944e3366eb13be4fd71efcd69450f89ba39f43/protobuf-7.35.1-cp310-abi3-manylinux2014_x86_64.whl", hash = "sha256:74758715c53d7158fb76caf4f0cfdacc5329a4b1bb994f865d6cf302d413a1c4", size = 327130, upload-time = "2026-06-11T21:55:35.921Z" },
{ url = "https://files.pythonhosted.org/packages/d8/bc/6d6c7ba8709c85f8f2c390b2b118d6fb08a783676a572271851bf45a7d22/protobuf-7.35.1-cp310-abi3-win32.whl", hash = "sha256:353652e4efd0bca5b5fc2656abf8307ef351f0cf938c9eba09f0e09c20a25c30", size = 428945, upload-time = "2026-06-11T21:55:37.034Z" },
{ url = "https://files.pythonhosted.org/packages/0a/19/8d0cb6f20a1ef7b18f1c8986ad5783f22f84cce39c6ce9a6e645ea55192e/protobuf-7.35.1-cp310-abi3-win_amd64.whl", hash = "sha256:230a75ddfc2de4806e56696ce9640c1cdfdb6543b7cfce98d42a4c0a0e7bdb87", size = 439996, upload-time = "2026-06-11T21:55:38.123Z" },
{ url = "https://files.pythonhosted.org/packages/19/c7/5f7c636ec43e0c545e28d1f1db71990108306f7bdcb89f069ba97e428e7f/protobuf-7.35.1-py3-none-any.whl", hash = "sha256:4bc97768d8fe4ad6743c8a19403e314511ed9f6d13205b687e52421c023ac1b9", size = 171659, upload-time = "2026-06-11T21:55:39.155Z" },
]
[[package]] [[package]]
name = "pyarrow" name = "pyarrow"
version = "23.0.1" version = "23.0.1"