Files
mt5cli/tests/test_grafana.py
Daichi Narushima a5c2aa8e8f test: consolidate duplicate test cases into parameterized tests (#91)
* test: consolidate duplicate test cases into parameterized tests

- Combine test_symbol_info, test_symbol_info_tick, test_market_book into test_symbol_command with parametrize
- Combine test_order_check variants into test_order_request with parametrize
- Combine test_order_send variants into test_order_request with parametrize
- Consolidate grafana view skip tests into test_grafana_view_skipped_when_required_cols_missing
- Consolidate grafana index skip tests into test_index_skipped_when_cols_missing
- Consolidate _NoOp method tests into test_method_is_noop
- Consolidate record_*_update noop tests into test_record_update_noop_before_configure

This improves test maintainability by reducing duplication while maintaining coverage.

* test: parametrize remaining duplicated cases

* test: parameterize remaining duplicate test cases

* test: parameterize remaining duplicate test cases

* test: parameterize remaining duplicate test cases

* test: parameterize remaining duplicate test cases

* test: parameterize count and start_pos validation tests

* test: parameterize remaining duplicate test cases

- tests/test_utils.py: merge valid string/integer cases for
  parse_timeframe and parse_tick_flags.
- tests/test_cli.py: consolidate collect-history --if-exists append/fail
  tests and drop the duplicate collect-history --dataset ticks default
  flags=-1 assertion.
- tests/test_trading.py: parametrize buy/sell projected margin ratio,
  invalid place_market_order mode validation, core
  calculate_volume_by_margin boundary cases, and zero-ratio
  calculate_margin_and_volume sizing cases.

* test: parameterize remaining duplicate test cases

Consolidate the last high-value duplicate test cases outside the
already-changed areas while keeping semantics and 100% coverage.

- tests/test_sdk.py:TestMt5CliClient - merge copy_rates_range,
  copy_ticks_from, history_orders, and latest_rates delegation/
  normalization tests into test_method_delegates_with_normalization
  parameterized by call/expected_method/expected_kwargs with readable
  ids for each normalization path.
- tests/test_sdk.py:TestSubstituteEnvPlaceholders - merge
  brace/whole-dollar/plain/partial substitution cases into
  test_substitute_env_placeholders, and missing-env cases into
  test_substitute_env_placeholders_raises_on_missing_env.
- tests/test_trading.py:TestVolumeAndExecution - merge buy/sell
  trailing-stop main cases into test_calculate_trailing_stop_updates_by_side,
  opposite-side invalid-quote cases into
  test_calculate_trailing_stop_updates_ignores_opposite_side_price,
  and the two mixed-positions scenarios into the parametrized
  test_calculate_trailing_stop_updates_mixed_positions_skip_invalid_side.
  All ids explicitly describe the side and behavior.

* test: parameterize collect-history and telemetry update tests

Consolidate duplicate default-vs-ticks collect-history cases in CLI and
SDK tests, and merge history/snapshot record_*_update success and
failure tests in test_telemetry.py.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test: parameterize telemetry record_*_state gauge tests

Co-authored-by: Cursor <cursoragent@cursor.com>

* test: parameterize Grafana example file checks

* test: centralize remaining parametrized cases

* test: add parametrized replacements for remaining cases

* test: fix remaining parametrized lint

* Fix Ruff warnings

* test: remove duplicate parametrization sweep file

* test: remove duplicate-test deselection hook

* test: restore conftest formatting

* test: add remaining parametrized regression coverage

* test: fix lint issues in parametrization completion tests

* test: avoid dynamic SQL in parametrization completion tests

* test: format parametrization completion tests

* test: fix lint and pyright issues in parametrization coverage

* Fix a Pyright error

* Update .agents/skills/pr-feedback-triage/SKILL.md

* test: dissolve parametrize-completion catch-all into owning test files

Move the remaining consolidated cases from test_parametrize_completion.py
into their owning modules and delete the file:

- test_cli.py: merge snapshot/grafana-schema --publish-copy gating into
  one parametrized test.
- test_trading.py: merge calculate_spread_ratio numeric/numeric-string
  cases; fold non-finite volume_max into
  test_normalize_order_volume_deterministic; merge estimate_order_margin
  invalid-volume cases (0.0/nan/inf); merge calculate_positions_margin
  invalid-row filtering cases.
- test_history.py: parametrize resolve_history_tick_flags and
  resolve_granularity_name.
- test_sdk.py: parametrize collect_latest_closed_rates_for_accounts
  empty-effective-frame cases.
- test_grafana.py: merge account/terminal snapshot insert tests inside
  TestSnapshotInserts.

All remaining cases in the removed file were exact duplicates of
existing coverage, so no new test scenarios are introduced. 100%
coverage and all checks (ruff, pyright, pytest, local-qa) pass.

* test: parameterize remaining account-event, snapshot, and CLI edge cases

Consolidates six more pairs of duplicate tests flagged during PR #91
review into pytest.mark.parametrize tables (incremental history_deals
account-event edge cases, dedup scope column variants, snapshot view
status filtering, snapshot kwargs forwarding, injected-client
lifecycle, and close-positions --yes gate), preserving explicit ids
and 100% coverage.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* test: consolidate build_config login coercion into parametrized cases

Fold separate backward-compat and env-expansion tests into named
pytest.param rows for clearer failure output.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test: consolidate deduplication and publish test cases into parametrized variants

Merge related test cases into single parametrized tests:
- Consolidate collect-history validation tests into a single parametrized case
- Merge snapshot-view skip conditions into one test with parametrized inputs
- Combine publish_grafana_copy target variations into one parametrized test
- Consolidate incremental-start and drop-duplicates error cases
- Remove duplicate test_drop_duplicates_rejects_invalid_identifiers

Reduces test file duplication while maintaining full coverage.

* test: consolidate remaining review-identified parameterized duplicates

Merge close-position filter, margin-ratio suppress/reraise, SQLite path
validation, and grafana_symbol_pnl schema cases into clearer parametrized tests.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test: restore default if_exists coverage in SQLite append test

The default-append parametrized case now omits if_exists so the API default
remains exercised instead of passing IfExists.APPEND explicitly.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test: tighten trading parametrization cleanup

* test: consolidate remaining parameterized cases

* test: consolidate additional duplicate cases into parametrized tests

Merge overlapping success-path tests in contracts, history, SDK, and
trading modules while preserving 100% coverage and explicit test ids.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: agent <agent@localhost>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-03 03:54:26 +09:00

944 lines
34 KiB
Python

"""Tests for mt5cli.grafana module."""
from __future__ import annotations
import logging
import sqlite3
from pathlib import Path
from typing import TYPE_CHECKING
from unittest.mock import MagicMock, patch
import pandas as pd
import pytest
if TYPE_CHECKING:
from collections.abc import Callable, Iterator
from mt5cli.grafana import (
_build_snapshot_view, # type: ignore[reportPrivateUsage]
_create_view_safe, # type: ignore[reportPrivateUsage]
create_grafana_indexes,
create_grafana_views,
create_snapshot_tables,
ensure_grafana_schema,
insert_account_snapshot,
insert_order_snapshots,
insert_position_snapshots,
insert_terminal_snapshot,
publish_grafana_copy,
record_snapshot_run,
start_snapshot_run,
)
@pytest.fixture
def conn() -> Iterator[sqlite3.Connection]:
"""Yield an in-memory SQLite connection for each test."""
with sqlite3.connect(":memory:") as c:
yield c
def _get_names(conn: sqlite3.Connection, type_: str) -> set[str]:
return {
row[0]
for row in conn.execute(
"SELECT name FROM sqlite_master WHERE type=?",
(type_,),
).fetchall()
}
def _make_rates_table(conn: sqlite3.Connection) -> None:
conn.execute(
"CREATE TABLE rates"
" (time TEXT, symbol TEXT, timeframe INTEGER,"
" open REAL, high REAL, low REAL, close REAL)"
)
def _make_ticks_table(conn: sqlite3.Connection) -> None:
conn.execute("CREATE TABLE ticks (time TEXT, symbol TEXT, bid REAL, ask REAL)")
def _make_history_deals_full(conn: sqlite3.Connection) -> None:
conn.execute(
"CREATE TABLE history_deals"
" (time TEXT, symbol TEXT, profit REAL, type INTEGER,"
" entry INTEGER, volume REAL, price REAL, ticket INTEGER, position_id INTEGER)"
)
def _make_history_deals_minimal(conn: sqlite3.Connection) -> None:
"""history_deals with only time, type, symbol, profit — no entry/volume/price."""
conn.execute(
"CREATE TABLE history_deals (time TEXT, symbol TEXT, profit REAL, type INTEGER)"
)
def _make_history_deals_symbol_pnl_minimal(conn: sqlite3.Connection) -> None:
"""history_deals with entry but without volume and price columns."""
conn.execute(
"CREATE TABLE history_deals"
" (time TEXT, symbol TEXT, profit REAL, type INTEGER, entry INTEGER)"
)
def _make_history_orders_table(conn: sqlite3.Connection) -> None:
conn.execute(
"CREATE TABLE history_orders"
" (time_setup TEXT, symbol TEXT, ticket INTEGER, type INTEGER)"
)
_TIMESTAMP_TIME_SETUP: pd.Timestamp = pd.Timestamp("2024-01-15 10:30:00", tz="UTC")
# ---------------------------------------------------------------------------
# TestSnapshotTables
# ---------------------------------------------------------------------------
class TestSnapshotTables:
"""Tests for create_snapshot_tables."""
def test_creates_all_five_tables(self, conn: sqlite3.Connection) -> None:
"""All five snapshot tables are created."""
create_snapshot_tables(conn)
tables = _get_names(conn, "table")
assert "snapshot_runs" in tables
assert "account_snapshots" in tables
assert "position_snapshots" in tables
assert "order_snapshots" in tables
assert "terminal_snapshots" in tables
def test_is_idempotent(self, conn: sqlite3.Connection) -> None:
"""Calling create_snapshot_tables twice does not raise."""
create_snapshot_tables(conn)
create_snapshot_tables(conn)
tables = _get_names(conn, "table")
assert "snapshot_runs" in tables
# ---------------------------------------------------------------------------
# TestCreateViewSafe
# ---------------------------------------------------------------------------
class TestCreateViewSafe:
"""Tests for _create_view_safe."""
def test_creates_view_successfully(self, conn: sqlite3.Connection) -> None:
"""A valid select SQL creates the named view."""
_create_view_safe(conn, "test_view", "SELECT 1 AS val")
views = _get_names(conn, "view")
assert "test_view" in views
def test_replaces_existing_view(self, conn: sqlite3.Connection) -> None:
"""Calling again with a new SQL replaces the existing view."""
_create_view_safe(conn, "test_view", "SELECT 1 AS val")
_create_view_safe(conn, "test_view", "SELECT 2 AS val")
result = conn.execute("SELECT val FROM test_view").fetchone()
assert result == (2,)
def test_logs_warning_on_sqlite_error(
self,
caplog: pytest.LogCaptureFixture,
) -> None:
"""sqlite3.Error during CREATE VIEW logs a warning instead of raising."""
mock_conn = MagicMock()
mock_conn.execute.side_effect = [
None,
sqlite3.OperationalError("parse error"),
]
with caplog.at_level(logging.WARNING, logger="mt5cli.grafana"):
_create_view_safe(mock_conn, "bad_view", "SELECT 1")
assert "Skipping view bad_view" in caplog.text
assert "parse error" in caplog.text
# ---------------------------------------------------------------------------
# TestGrafanaViews
# ---------------------------------------------------------------------------
class TestGrafanaViews:
"""Tests for create_grafana_views and individual view builders."""
def test_all_views_created_with_full_schema(
self,
conn: sqlite3.Connection,
) -> None:
"""All 13 Grafana views are created when all source tables are present."""
_make_rates_table(conn)
_make_ticks_table(conn)
_make_history_deals_full(conn)
_make_history_orders_table(conn)
create_snapshot_tables(conn)
create_grafana_views(conn)
views = _get_names(conn, "view")
expected = {
"grafana_rates",
"grafana_ticks",
"grafana_history_deals",
"grafana_history_orders",
"grafana_trade_deals",
"grafana_cash_events",
"grafana_realized_pnl",
"grafana_symbol_pnl",
"grafana_trade_stats",
"grafana_account_snapshots",
"grafana_position_snapshots",
"grafana_order_snapshots",
"grafana_terminal_snapshots",
}
assert expected.issubset(views)
def test_stale_view_dropped_when_source_table_disappears(
self,
conn: sqlite3.Connection,
) -> None:
"""create_grafana_views drops a previously created view whose source is gone."""
_make_ticks_table(conn)
create_grafana_views(conn)
assert "grafana_ticks" in _get_names(conn, "view")
conn.execute("DROP TABLE ticks")
create_grafana_views(conn)
assert "grafana_ticks" not in _get_names(conn, "view")
def test_grafana_rates_skipped_when_table_absent(
self,
conn: sqlite3.Connection,
caplog: pytest.LogCaptureFixture,
) -> None:
"""grafana_rates is skipped when rates table is missing."""
with caplog.at_level(logging.WARNING, logger="mt5cli.grafana"):
create_grafana_views(conn)
assert "grafana_rates" not in _get_names(conn, "view")
@pytest.mark.parametrize(
("ddl", "view_name"),
[
("CREATE TABLE rates (open REAL)", "grafana_rates"),
("CREATE TABLE ticks (bid REAL)", "grafana_ticks"),
("CREATE TABLE history_deals (symbol TEXT)", "grafana_history_deals"),
("CREATE TABLE history_orders (symbol TEXT)", "grafana_history_orders"),
("CREATE TABLE history_deals (symbol TEXT)", "grafana_trade_deals"),
("CREATE TABLE history_deals (symbol TEXT)", "grafana_cash_events"),
(
"CREATE TABLE history_deals (time TEXT, type INTEGER)",
"grafana_realized_pnl",
),
(
(
"CREATE TABLE history_deals"
" (time TEXT, symbol TEXT, profit REAL, type INTEGER)"
),
"grafana_realized_pnl",
),
(
"CREATE TABLE history_deals (time TEXT, type INTEGER)",
"grafana_symbol_pnl",
),
("CREATE TABLE history_deals (time TEXT)", "grafana_trade_stats"),
],
ids=[
"rates-cols",
"ticks-cols",
"history_deals-time",
"history_orders-time_setup",
"trade_deals-cols",
"cash_events-cols",
"realized_pnl-cols",
"realized_pnl-entry",
"symbol_pnl-cols",
"trade_stats-cols",
],
)
def test_grafana_view_skipped_when_required_cols_missing(
self,
conn: sqlite3.Connection,
caplog: pytest.LogCaptureFixture,
ddl: str,
view_name: str,
) -> None:
"""A grafana view is skipped (with warning) when source columns are missing."""
conn.execute(ddl)
with caplog.at_level(logging.WARNING, logger="mt5cli.grafana"):
create_grafana_views(conn)
assert view_name not in _get_names(conn, "view")
assert f"Skipping {view_name}" in caplog.text
@pytest.mark.parametrize(
("setup_deals", "optional_cols"),
[
pytest.param(
_make_history_deals_symbol_pnl_minimal, set[str](), id="minimal"
),
pytest.param(
_make_history_deals_full,
{"volume", "price"},
id="full",
),
],
)
def test_grafana_symbol_pnl_schema(
self,
conn: sqlite3.Connection,
setup_deals: Callable[[sqlite3.Connection], None],
optional_cols: set[str],
) -> None:
"""grafana_symbol_pnl is created and includes optional columns when present."""
setup_deals(conn)
create_grafana_views(conn)
assert "grafana_symbol_pnl" in _get_names(conn, "view")
cols = {row[1] for row in conn.execute("PRAGMA table_info(grafana_symbol_pnl)")}
assert optional_cols.issubset(cols)
@pytest.mark.parametrize(
"setup_deals",
[_make_history_deals_minimal, _make_history_deals_full],
ids=["minimal", "full"],
)
def test_grafana_trade_stats_static_summary(
self,
conn: sqlite3.Connection,
setup_deals: Callable[[sqlite3.Connection], None],
) -> None:
"""grafana_trade_stats is a static summary view with no time column."""
setup_deals(conn)
create_grafana_views(conn)
assert "grafana_trade_stats" in _get_names(conn, "view")
cols = {
row[1] for row in conn.execute("PRAGMA table_info(grafana_trade_stats)")
}
assert "time" not in cols
assert "symbol" in cols
def test_snapshot_views_skipped_when_snapshot_tables_absent(
self,
conn: sqlite3.Connection,
caplog: pytest.LogCaptureFixture,
) -> None:
"""Snapshot views are skipped when snapshot tables are not created."""
with caplog.at_level(logging.WARNING, logger="mt5cli.grafana"):
create_grafana_views(conn)
views = _get_names(conn, "view")
assert "grafana_account_snapshots" not in views
assert "grafana_position_snapshots" not in views
assert "grafana_order_snapshots" not in views
assert "grafana_terminal_snapshots" not in views
def test_build_snapshot_view_with_only_run_id_col(
self,
conn: sqlite3.Connection,
) -> None:
"""_build_snapshot_view exposes time and run_id when table has only run_id."""
create_snapshot_tables(conn)
conn.execute("CREATE TABLE only_run (run_id INTEGER NOT NULL)")
run_id = start_snapshot_run(conn, 1000)
record_snapshot_run(conn, run_id, "ok")
conn.execute("INSERT INTO only_run (run_id) VALUES (?)", (run_id,))
_build_snapshot_view(conn, "test_view", "only_run")
assert "test_view" in _get_names(conn, "view")
cols = {row[1] for row in conn.execute("PRAGMA table_info(test_view)")}
assert "time" in cols
assert "run_id" in cols
@pytest.mark.parametrize(
("use_snapshot_tables", "table_ddl", "table_name", "log_fragment"),
[
pytest.param(
False,
"CREATE TABLE only_run (run_id INTEGER NOT NULL)",
"only_run",
"snapshot_runs missing required columns",
id="snapshot-runs-wrong-columns",
),
pytest.param(
True,
"CREATE TABLE no_run_id (symbol TEXT)",
"no_run_id",
"missing run_id column",
id="table-missing-run-id",
),
],
)
def test_build_snapshot_view_skips_negative_cases(
self,
conn: sqlite3.Connection,
caplog: pytest.LogCaptureFixture,
use_snapshot_tables: bool,
table_ddl: str,
table_name: str,
log_fragment: str,
) -> None:
"""_build_snapshot_view skips view creation for invalid table/run metadata."""
if use_snapshot_tables:
create_snapshot_tables(conn)
else:
conn.execute("CREATE TABLE snapshot_runs (foo TEXT)")
conn.execute(table_ddl)
with caplog.at_level(logging.WARNING, logger="mt5cli.grafana"):
_build_snapshot_view(conn, "test_view", table_name)
assert "test_view" not in _get_names(conn, "view")
assert log_fragment in caplog.text
@pytest.mark.parametrize(
("started_at", "status", "message", "keep_row"),
[
pytest.param(
1000,
"error",
"terminal offline",
False,
id="excludes-failed-run-rows",
),
pytest.param(2000, "ok", None, True, id="includes-ok-run-rows"),
],
)
def test_snapshot_view_filters_rows_by_run_status(
self,
conn: sqlite3.Connection,
started_at: int,
status: str,
message: str | None,
keep_row: bool,
) -> None:
"""Snapshot views expose rows only from successful runs."""
create_snapshot_tables(conn)
run_id = start_snapshot_run(conn, started_at)
conn.execute(
"INSERT INTO account_snapshots"
" (run_id, login, balance, equity, margin, margin_free, profit)"
" VALUES (?, 12345, 10000.0, 9800.0, 200.0, 9600.0, -200.0)",
(run_id,),
)
record_snapshot_run(conn, run_id, status, message)
create_grafana_views(conn)
rows = conn.execute(
"SELECT time, run_id, login FROM grafana_account_snapshots"
).fetchall()
if keep_row:
assert rows == [(started_at, run_id, 12345)]
cols = {
row[1]
for row in conn.execute("PRAGMA table_info(grafana_account_snapshots)")
}
assert "run_id" in cols
else:
assert rows == []
@pytest.mark.parametrize(
("observed_at", "runs", "expected_rows"),
[
pytest.param(
3000,
[("error", 99), ("ok", 12345)],
[(12345,)],
id="same-second-error-and-ok-exposes-only-ok-row",
),
pytest.param(
4000,
[("ok", 1), ("ok", 2)],
[(1,), (2,)],
id="same-second-two-ok-runs-no-duplication",
),
],
)
def test_snapshot_view_same_second_runs(
self,
conn: sqlite3.Connection,
observed_at: int,
runs: list[tuple[str, int]],
expected_rows: list[tuple[int]],
) -> None:
"""Snapshot views join same-second rows by run_id."""
create_snapshot_tables(conn)
for status, login in runs:
run_id = start_snapshot_run(conn, observed_at)
conn.execute(
"INSERT INTO account_snapshots (run_id, login) VALUES (?, ?)",
(run_id, login),
)
record_snapshot_run(conn, run_id, status)
create_grafana_views(conn)
rows = conn.execute("SELECT login FROM grafana_account_snapshots").fetchall()
assert rows == expected_rows
# ---------------------------------------------------------------------------
# TestGrafanaIndexes
# ---------------------------------------------------------------------------
class TestGrafanaIndexes:
"""Tests for create_grafana_indexes."""
def test_all_indexes_created_with_full_schema(
self,
conn: sqlite3.Connection,
) -> None:
"""All 9 indexes are created when all source tables are present."""
_make_rates_table(conn)
_make_ticks_table(conn)
_make_history_deals_full(conn)
_make_history_orders_table(conn)
create_snapshot_tables(conn)
create_grafana_indexes(conn)
indexes = _get_names(conn, "index")
assert "idx_rates_time_symbol_timeframe" in indexes
assert "idx_ticks_time_symbol" in indexes
assert "idx_history_deals_time_symbol" in indexes
assert "idx_history_deals_symbol_time" in indexes
assert "idx_history_orders_time_setup_symbol" in indexes
assert "idx_account_snapshots_time_login" in indexes
assert "idx_position_snapshots_time_symbol" in indexes
assert "idx_order_snapshots_time_symbol" in indexes
assert "idx_snapshot_runs_time_status" in indexes
def test_no_indexes_created_when_tables_absent(
self,
conn: sqlite3.Connection,
) -> None:
"""No indexes are created when tables are absent."""
create_grafana_indexes(conn)
indexes = _get_names(conn, "index")
assert not any(name.startswith("idx_") for name in indexes)
def test_indexes_for_snapshot_tables_skipped_when_absent(
self,
conn: sqlite3.Connection,
) -> None:
"""Snapshot table indexes are skipped when snapshot tables don't exist."""
_make_history_deals_full(conn)
create_grafana_indexes(conn)
indexes = _get_names(conn, "index")
assert "idx_account_snapshots_time_login" not in indexes
assert "idx_position_snapshots_time_symbol" not in indexes
assert "idx_order_snapshots_time_symbol" not in indexes
assert "idx_snapshot_runs_time_status" not in indexes
@pytest.mark.parametrize(
("ddl", "index_name"),
[
("CREATE TABLE rates (open REAL)", "idx_rates_time_symbol_timeframe"),
("CREATE TABLE ticks (bid REAL)", "idx_ticks_time_symbol"),
(
"CREATE TABLE history_deals (ticket INTEGER)",
"idx_history_deals_time_symbol",
),
(
"CREATE TABLE history_orders (ticket INTEGER)",
"idx_history_orders_time_setup_symbol",
),
],
ids=["rates", "ticks", "deals", "orders"],
)
def test_index_skipped_when_cols_missing(
self,
conn: sqlite3.Connection,
ddl: str,
index_name: str,
) -> None:
"""An index is skipped when the source table lacks required columns."""
conn.execute(ddl)
create_grafana_indexes(conn)
assert index_name not in _get_names(conn, "index")
def test_snapshot_indexes_skipped_when_cols_missing(
self,
conn: sqlite3.Connection,
) -> None:
"""Snapshot table indexes are skipped when required columns are absent."""
conn.execute("CREATE TABLE account_snapshots (foo TEXT)")
conn.execute("CREATE TABLE position_snapshots (foo TEXT)")
conn.execute("CREATE TABLE order_snapshots (foo TEXT)")
conn.execute("CREATE TABLE snapshot_runs (foo TEXT)")
create_grafana_indexes(conn)
indexes = _get_names(conn, "index")
assert "idx_account_snapshots_time_login" not in indexes
assert "idx_position_snapshots_time_symbol" not in indexes
assert "idx_order_snapshots_time_symbol" not in indexes
assert "idx_snapshot_runs_time_status" not in indexes
def test_indexes_are_idempotent(self, conn: sqlite3.Connection) -> None:
"""Creating indexes twice does not raise (IF NOT EXISTS)."""
_make_rates_table(conn)
create_grafana_indexes(conn)
create_grafana_indexes(conn)
indexes = _get_names(conn, "index")
assert "idx_rates_time_symbol_timeframe" in indexes
# ---------------------------------------------------------------------------
# TestEnsureGrafanaSchema
# ---------------------------------------------------------------------------
class TestEnsureGrafanaSchema:
"""Tests for ensure_grafana_schema."""
def test_creates_all_tables_views_and_indexes(
self,
conn: sqlite3.Connection,
) -> None:
"""ensure_grafana_schema creates snapshot tables, views, and indexes."""
_make_rates_table(conn)
_make_history_deals_full(conn)
ensure_grafana_schema(conn)
tables = _get_names(conn, "table")
assert "snapshot_runs" in tables
assert "account_snapshots" in tables
views = _get_names(conn, "view")
assert "grafana_rates" in views
assert "grafana_account_snapshots" in views
indexes = _get_names(conn, "index")
assert "idx_rates_time_symbol_timeframe" in indexes
def test_is_idempotent(self, conn: sqlite3.Connection) -> None:
"""Calling ensure_grafana_schema twice does not raise."""
ensure_grafana_schema(conn)
ensure_grafana_schema(conn)
# ---------------------------------------------------------------------------
# TestSnapshotInserts
# ---------------------------------------------------------------------------
class TestSnapshotInserts:
"""Tests for snapshot insert helpers."""
@pytest.fixture(autouse=True)
def setup_tables(self, conn: sqlite3.Connection) -> None:
"""Create snapshot tables before each insert test."""
create_snapshot_tables(conn)
@pytest.mark.parametrize(
("insert_func", "row", "select_sql", "expected"),
[
(
insert_account_snapshot,
{
"login": 12345,
"currency": "USD",
"balance": 10000.0,
"equity": 9800.0,
"margin": 200.0,
"margin_free": 9800.0,
"margin_level": 4900.0,
"profit": -200.0,
"leverage": 100,
},
"SELECT login, currency, balance FROM account_snapshots",
(12345, "USD", 10000.0),
),
(
insert_terminal_snapshot,
{
"name": "MetaTrader 5",
"connected": 1,
"community_account": 0,
"trade_allowed": 1,
"trade_expert": 1,
"path": "/mt5",
"company": "Broker",
"language": "en",
},
"SELECT name, connected FROM terminal_snapshots",
("MetaTrader 5", 1),
),
],
ids=["account", "terminal"],
)
def test_insert_single_snapshot(
self,
conn: sqlite3.Connection,
insert_func: Callable[[sqlite3.Connection, int, dict[str, object]], None],
row: dict[str, object],
select_sql: str,
expected: tuple[object, ...],
) -> None:
"""insert_account_snapshot and insert_terminal_snapshot append one row."""
run_id = start_snapshot_run(conn, 1700000000)
insert_func(conn, run_id, row)
result = conn.execute(select_sql).fetchone()
assert result == expected
def test_insert_account_snapshot_partial_row(
self,
conn: sqlite3.Connection,
) -> None:
"""insert_account_snapshot works when some fields are missing (uses None)."""
run_id = start_snapshot_run(conn, 1700000000)
insert_account_snapshot(conn, run_id, {"login": 1})
result = conn.execute(
"SELECT login, currency FROM account_snapshots"
).fetchone()
assert result == (1, None)
@pytest.mark.parametrize(
("insert_func", "table", "rows", "expected_count"),
[
(
insert_position_snapshots,
"position_snapshots",
[
{"ticket": 1, "symbol": "EURUSD", "volume": 0.1, "profit": 10.0},
{"ticket": 2, "symbol": "GBPUSD", "volume": 0.2, "profit": -5.0},
],
2,
),
(
insert_position_snapshots,
"position_snapshots",
[],
0,
),
(
insert_order_snapshots,
"order_snapshots",
[
{
"ticket": 10,
"symbol": "EURUSD",
"type": 2,
"volume_current": 0.1,
},
],
1,
),
(
insert_order_snapshots,
"order_snapshots",
[],
0,
),
],
ids=[
"positions-with-rows",
"positions-empty-noop",
"orders-with-rows",
"orders-empty-noop",
],
)
def test_insert_snapshot_rows(
self,
conn: sqlite3.Connection,
insert_func: Callable[
[sqlite3.Connection, int, int | None, list[dict[str, object]]],
None,
],
table: str,
rows: list[dict[str, object]],
expected_count: int,
) -> None:
"""insert_*_snapshots appends each row and is a no-op when empty."""
run_id = start_snapshot_run(conn, 1700000000)
insert_func(conn, run_id, 12345, rows)
count = conn.execute(
f"SELECT COUNT(*) FROM {table}" # noqa: S608
).fetchone()[0]
assert count == expected_count
@pytest.mark.parametrize(
("time_setup", "expected_stored"),
[
(_TIMESTAMP_TIME_SETUP, int(_TIMESTAMP_TIME_SETUP.timestamp())),
(1705314600, 1705314600),
("not_a_time", None),
],
ids=["timestamp", "int", "unknown-string"],
)
def test_insert_order_snapshots_normalizes_time_setup(
self,
conn: sqlite3.Connection,
time_setup: object,
expected_stored: int | None,
) -> None:
"""insert_order_snapshots stores epoch int, int as-is, or None for unknown."""
run_id = start_snapshot_run(conn, 1700000000)
rows: list[dict[str, object]] = [{"ticket": 10, "time_setup": time_setup}]
insert_order_snapshots(conn, run_id, 12345, rows)
stored = conn.execute("SELECT time_setup FROM order_snapshots").fetchone()[0]
assert stored == expected_stored
def test_start_snapshot_run_returns_incrementing_ids(
self,
conn: sqlite3.Connection,
) -> None:
"""start_snapshot_run returns a unique run_id for each call."""
run1 = start_snapshot_run(conn, 1700000000)
run2 = start_snapshot_run(conn, 1700000000)
assert run1 != run2
@pytest.mark.parametrize(
("status", "detail", "expected"),
[
pytest.param(
"error",
"RuntimeError: boom",
("error", "RuntimeError: boom"),
id="with-detail",
),
pytest.param("ok", None, ("ok", None), id="without-detail"),
],
)
def test_record_snapshot_run(
self,
conn: sqlite3.Connection,
status: str,
detail: str | None,
expected: tuple[str, str | None],
) -> None:
"""record_snapshot_run stores status and optional detail text."""
run_id = start_snapshot_run(conn, 1700000000)
record_snapshot_run(conn, run_id, status, detail)
row = conn.execute("SELECT status, detail FROM snapshot_runs").fetchone()
assert row == expected
# ---------------------------------------------------------------------------
# TestPublishGrafanaCopy
# ---------------------------------------------------------------------------
def _make_source_db(path: Path) -> None:
"""Create a minimal source SQLite database with snapshot tables."""
with sqlite3.connect(path) as conn:
conn.execute("PRAGMA journal_mode=WAL")
create_snapshot_tables(conn)
conn.execute(
"INSERT INTO snapshot_runs (observed_at, status) VALUES (?, 'ok')",
(1700000000,),
)
class TestPublishGrafanaCopy:
"""Tests for publish_grafana_copy."""
@pytest.mark.parametrize(
("target_rel", "stale_content", "required_tables"),
[
pytest.param(
Path("out") / "grafana.db",
None,
frozenset({"snapshot_runs", "account_snapshots"}),
id="fresh-target",
),
pytest.param(
Path("grafana.db"),
b"stale",
frozenset({"snapshot_runs"}),
id="overwrite-stale-target",
),
],
)
def test_publish_creates_valid_sqlite_target(
self,
tmp_path: Path,
target_rel: Path,
stale_content: bytes | None,
required_tables: frozenset[str],
) -> None:
"""publish_grafana_copy creates or replaces a valid SQLite target."""
source = tmp_path / "src.db"
target = tmp_path / target_rel
_make_source_db(source)
if stale_content is not None:
target.write_bytes(stale_content)
result = publish_grafana_copy(source, target)
assert target.exists()
assert isinstance(result, Path)
assert result == target.resolve()
with sqlite3.connect(target) as conn:
tables = {
row[0]
for row in conn.execute(
"SELECT name FROM sqlite_master WHERE type='table'"
).fetchall()
}
assert required_tables <= tables
def test_target_can_be_opened_readonly(self, tmp_path: Path) -> None:
"""Published target can be opened with uri=True in read-only mode."""
source = tmp_path / "src.db"
target = tmp_path / "grafana.db"
_make_source_db(source)
publish_grafana_copy(source, target)
uri = f"file:{target}?mode=ro"
with sqlite3.connect(uri, uri=True) as conn:
row = conn.execute("SELECT status FROM snapshot_runs").fetchone()
assert row == ("ok",)
def test_same_path_raises(self, tmp_path: Path) -> None:
"""publish_grafana_copy raises ValueError when source equals target."""
db = tmp_path / "history.db"
_make_source_db(db)
with pytest.raises(ValueError, match="must differ from the source"):
publish_grafana_copy(db, db)
def test_source_not_found_raises(self, tmp_path: Path) -> None:
"""publish_grafana_copy raises FileNotFoundError when source is absent."""
with pytest.raises(FileNotFoundError):
publish_grafana_copy(tmp_path / "missing.db", tmp_path / "out.db")
def test_preserve_old_target_on_backup_failure(self, tmp_path: Path) -> None:
"""Old target is preserved when the backup fails."""
source = tmp_path / "src.db"
target = tmp_path / "grafana.db"
_make_source_db(source)
original_content = b"original_data"
target.write_bytes(original_content)
with patch("sqlite3.connect") as mock_connect:
mock_src = MagicMock()
mock_src.__enter__ = MagicMock(return_value=mock_src)
mock_src.__exit__ = MagicMock(return_value=False)
mock_src.backup.side_effect = sqlite3.OperationalError("backup failed")
mock_connect.return_value = mock_src
with pytest.raises(sqlite3.OperationalError, match="backup failed"):
publish_grafana_copy(source, target)
assert target.read_bytes() == original_content
def test_temp_file_cleaned_up_on_failure(self, tmp_path: Path) -> None:
"""Temporary file is removed when backup raises an exception."""
source = tmp_path / "src.db"
target = tmp_path / "grafana.db"
_make_source_db(source)
with patch("sqlite3.connect") as mock_connect:
mock_src = MagicMock()
mock_src.__enter__ = MagicMock(return_value=mock_src)
mock_src.__exit__ = MagicMock(return_value=False)
mock_src.backup.side_effect = sqlite3.OperationalError("fail")
mock_connect.return_value = mock_src
with pytest.raises(sqlite3.OperationalError):
publish_grafana_copy(source, target)
tmp_files = list(tmp_path.glob("grafana.db.*.tmp"))
assert not tmp_files, "Temp file should be cleaned up on failure"
def test_fresh_target_has_readable_permissions(self, tmp_path: Path) -> None:
"""Published copy is readable by the owner."""
import stat as _stat # noqa: PLC0415
source = tmp_path / "src.db"
target = tmp_path / "grafana.db"
_make_source_db(source)
publish_grafana_copy(source, target)
mode = target.stat().st_mode & 0o777
assert bool(mode & _stat.S_IRUSR), "owner must be able to read"
@pytest.mark.skipif(
__import__("sys").platform == "win32",
reason="Windows does not support Unix-style group/other permission bits",
)
def test_overwrite_preserves_existing_target_mode(self, tmp_path: Path) -> None:
"""Overwriting an existing target preserves that target's file mode."""
source = tmp_path / "src.db"
target = tmp_path / "grafana.db"
_make_source_db(source)
target.write_bytes(b"old")
target.chmod(0o640)
publish_grafana_copy(source, target)
mode = target.stat().st_mode & 0o777
assert mode == 0o640