* feat: add Grafana-ready SQLite observability (#79, #80, #81)
New `mt5cli/grafana.py` module with idempotent DDL helpers:
- `create_snapshot_tables` — five SQLite tables for time-series account,
position, order, terminal, and run-status snapshots
- `create_grafana_views` — 13 `grafana_*` views with integer epoch-second
`time` columns; missing source tables emit warnings and are skipped
- `create_grafana_indexes` — 9 performance indexes guarded by column checks
- `ensure_grafana_schema` — convenience wrapper calling all three above
- Insert helpers: `insert_account_snapshot`, `insert_position_snapshots`,
`insert_order_snapshots`, `insert_terminal_snapshot`, `record_snapshot_run`
New stable SDK exports in `mt5cli.__init__` and `mt5cli.contract`:
- `update_observability` — appends a timestamped snapshot to a SQLite db
using an already-connected `Mt5DataClient`; never places orders
- `update_observability_with_config` — standalone wrapper that opens and
closes the MT5 connection automatically
New CLI commands (Collection panel):
- `grafana-schema` — idempotent schema setup, no MT5 connection required
- `snapshot` — append account/position/order/terminal rows; supports
`--symbol`, `--with-account/--no-account`, and equivalent flags
All public modules maintain 100 % branch coverage; 968 tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: apply mdformat to docs after grafana observability additions
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: bump version to 1.1.0
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address PR #86 review feedback
- Fix unaggregated time in grafana_realized_pnl GROUP BY query (MAX)
- Replace O(N) per-symbol API calls with single call + client-side filter
- Eliminate double create_snapshot_tables when with_grafana_schema=True
- Move grafana imports to module level in sdk.py (remove PLC0415 noqa)
- Default with_grafana_schema to False (run grafana-schema once for setup)
- Fix README position example to use snapshot_runs for latest snapshot
- Update tests to reflect new behavior and correct patch targets
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: apply ruff formatting and sync lock file
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address owner review feedback on PR #86
- Filter grafana_*_snapshots views to only expose rows from successful
runs (JOIN snapshot_runs WHERE status='ok'), closing the partial-snapshot
visibility gap raised in PRRT_kwDORzI_286MvDJ6
- Add issubset column guards for snapshot table index creation, consistent
with the rest of create_grafana_indexes (PRRT_kwDORzI_286MvUx1)
- Fix README example queries: views expose 'time' not 'observed_at'
(PRRT_kwDORzI_286MvUxw)
- Correct public-contract.md default for with_grafana_schema (False, not
True) and point to grafana-schema command (PRRT_kwDORzI_286MvUxy)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: add run_id to snapshot schema and fix stale-positions README query
Replace second-level observed_at as the join key between snapshot_runs
and snapshot tables with a stable run_id INTEGER PRIMARY KEY. Two runs
in the same second now get distinct run_ids, preventing view duplication
and cross-contamination from a failed run. Update README example to use
snapshot_runs for latest-snapshot lookup so zero-position runs return an
empty result instead of stale rows.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: format markdown tables
Align table column widths in README and public-contract documentation.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* fix: close sqlite connections deterministically
* fix: require entry filter in grafana_realized_pnl and add time to grafana_trade_stats
grafana_realized_pnl now requires the entry column and filters to
close-side deals (entry IN (1, 2, 3)), consistent with grafana_symbol_pnl
and grafana_trade_stats. grafana_trade_stats now requires the time column
and emits MAX(time_expr) AS "time" so it satisfies the documented Grafana
view contract (integer epoch-second time column throughout).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: normalize pd.Timestamp time_setup to epoch int in insert_order_snapshots
orders_get_as_df() returns datetime-converted columns by default, so
time_setup is a pd.Timestamp in normal use. Passing it directly to
sqlite3.executemany raises ProgrammingError. Added _to_epoch_int helper
that converts datetime.datetime subclasses (including pd.Timestamp) and
raw int/float values to integer epoch seconds, returning None for other
types. Regression tests cover all four input paths.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: pre-drop all grafana_* views at start of create_grafana_views
Previously, a builder that skipped due to a missing source table or column
would not drop the view it owned, leaving stale views referencing gone
tables. Now create_grafana_views drops all 13 known grafana_* views before
calling any builder, so a schema refresh always removes views whose source
has disappeared. Regression test covers the create → drop-source → refresh
cycle.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: expose run_id in snapshot views and drop time from summary views
Grafana snapshot views now expose run_id so latest-state queries can use
MAX(run_id) instead of the ambiguous second-level MAX(observed_at).
grafana_realized_pnl and grafana_trade_stats lose their MAX(time) column
and are reclassified as static summary views; their all-time aggregates
are not filterable by Grafana time-range selectors.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: guard snapshot views against missing run_id and fix README view docs
_build_snapshot_view now skips with a warning when the underlying
snapshot table exists but lacks a run_id column, preventing a broken
view that fails at query time. Adds a regression test for that path.
README Grafana section now qualifies that grafana_realized_pnl and
grafana_trade_stats are static summary views (no time column) and splits
the view table to match docs/api/public-contract.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: agent <agent@localhost>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Daichi Narushima
d27da02f3f