* feat: add MT5 order metadata, coverage report, and env-backed CLI config
* fix: align review-driven trading and history contracts
* Bump pdmt5 to 1.1.0
* test: stabilize history gaps CLI assertion
* fix: address review follow-ups for gaps and filling mode
* chore: upgrade pdmt5 to v1.0.4
pdmt5 1.0.4 removes Mt5TradingClient/Mt5TradingError entirely and wraps
Mt5Config.password in pydantic SecretStr. Drop the now-dead conditional
Mt5TradingError handling in mt5cli.exceptions/sdk/retry (mt5cli already
type-checks trading clients against its own protocol, so no functional
change), and unwrap SecretStr when forwarding a base config's password to
per-account configs. Update tests and docs accordingly.
* chore: declare pydantic as a direct runtime dependency
mt5cli.sdk imports SecretStr directly from pydantic, so pin it explicitly
instead of relying on pdmt5's transitive dependency.
---------
Co-authored-by: Claude <noreply@anthropic.com>
* fix: stabilize history timestamps and telemetry docs
* fix: preserve numeric epoch cursors in history SQLite queries
Normalize mixed ISO and unixepoch time values for incremental resume and scoped dedup so legacy numeric rows are not dropped by julianday filters.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Bump version to v1.1.2
* fix: aggregate incremental start timestamps in SQLite
Use MAX on the normalized time expression with GROUP BY so incremental
resume loaders stay O(groups) instead of materializing every history row.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test: parametrize duplicated incremental-start cases in TestIncrementalStart
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor: collapse repeated tests with pytest.mark.parametrize
Collapse 13 near-identical test methods into 4 parametrized tests
across test_cli.py and test_sdk.py, keeping all 1045 cases passing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: add fetch_recent_history_deals_for_trading_client to stable SDK
Adds a generic history deal retrieval helper for active trading clients,
a _HistoryDealsClientProtocol describing the minimal required interface,
clarified create_trading_client() docs (returns pdmt5.Mt5DataClient, not
MT5Client), 9 unit tests at 100% coverage, and updated trading.md and
public-contract.md with examples and out-of-scope strategy semantics note.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: narrow Mt5CliClient protocol claim and preserve empty deal DataFrame schema
- _HistoryDealsClientProtocol docstring and fetch_recent_history_deals_for_trading_client
docstring now explicitly state that Mt5CliClient (mt5_session) exposes
history_deals() not history_deals_get_as_df() and does not satisfy the protocol;
the function is for trading-client sessions (pdmt5.Mt5DataClient) only
- Empty DataFrames with columns are now passed through with reset_index rather
than replaced by a bare pd.DataFrame(), preserving schema for callers that rely
on stable column names even in no-deal windows
- Tests updated to assert schema preservation on empty results and bare empty on None
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: add combined protocol so create_trading_client() is type-safe with history deals helper
Adds _TradingHistoryDealsClientProtocol combining _Mt5ClientProtocol and
_HistoryDealsClientProtocol, and updates create_trading_client() and
mt5_trading_session() to return/yield this combined type so the natural SDK
flow `client = create_trading_client(...); fetch_recent_history_deals_for_trading_client(client)`
is type-safe under pyright strict without casts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: validate hours is finite before timedelta in fetch_recent_history_deals_for_trading_client
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Bump version to 1.1.1
---------
Co-authored-by: agent <agent@localhost>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: Grafana copy publishing, dashboard examples, and optional OTel metrics
Implements three observability improvements:
#82 — publish_grafana_copy(): Uses SQLite online backup API (WAL-safe) to
atomically publish a consistent read-only copy beside the target. Adds
--publish-copy option to grafana-schema and snapshot CLI commands.
#83 — examples/grafana/: Minimal working Grafana setup with docker-compose,
provisioning datasource/dashboard YAML, and three dashboard JSON files
(mt5cli-overview, mt5cli-trades, mt5cli-market). All queries use grafana_*
views; no credentials or private paths included.
#84 — mt5cli/telemetry.py: Optional OTel metrics behind mt5cli[otel] extra.
Base install is unaffected. Adds _Mt5Metrics singleton (no-op until
configure_metrics() is called), wraps update_history() and
update_observability() with record_history_update / record_snapshot_update
context managers, and emits account/position gauges from snapshots.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: replace ambiguous multiplication sign in comment
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: normalize markdown formatting in grafana README
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: preserve file mode on Grafana copy and fix unsupported time macro
- publish_grafana_copy: chmod temp file to match the existing target's
permissions (or 0o644 when no prior target exists) before atomic
replace, so Grafana running as a different OS user (e.g. UID 472 in
Docker) can read the published database
- mt5cli-market.json: replace unsupported \$__timeFilter(time) with the
epoch-based filter supported by frser-sqlite-datasource:
"time" >= \$__from / 1000 AND "time" < \$__to / 1000
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: skip Windows-incompatible mode test, rename compose file to compose.yaml
- Skip test_overwrite_preserves_existing_target_mode on win32 since
Windows chmod does not preserve Unix group/other permission bits
- Simplify test_fresh_target_has_readable_permissions to check owner
read bit only (portable across platforms)
- Rename docker-compose.yml -> compose.yaml (modern Compose convention)
- Update README and test reference to match new filename
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: rename *.yaml to *.yml in examples/grafana
Renames compose.yaml, mt5cli-sqlite.yaml, and mt5cli.yaml to .yml;
updates README and test references accordingly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: format Grafana dashboards and expand qa script to include JSON
- Update qa.sh prettier pattern to format JSON files alongside markdown
- Reformat Grafana dashboard JSONs with consistent spacing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address owner review comments before merge
- qa.sh: fix Prettier glob from `{,d,json}` to `{md,json}` so Markdown
files are actually formatted by local QA (P2)
- compose.yml: add GF_INSTALL_PLUGINS env var so the frser-sqlite-datasource
plugin is installed at container start (P1)
- telemetry.py: replace no-op get_meter() call with a real SDK MeterProvider
pipeline; add optional `readers` kwarg so callers can inject custom readers
(e.g. InMemoryMetricReader in tests) without needing the OTLP package (P1)
- sdk.py: aggregate profit and volume by symbol before emitting gauge values
so hedging accounts with multiple same-symbol positions emit one point per
symbol instead of overwriting with each row (P2)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: emit mt5_history_update_rows_total via conn.total_changes delta
The counter was registered but never incremented, making the advertised
history-update throughput metric permanently zero. Add add_history_rows()
to _Mt5Metrics and call it in update_history() using the SQLite
total_changes delta measured around write_incremental_datasets().
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address three owner review comments
- compose.yml: replace soft fallback with :? error expansion so Compose
refuses to start when MT5CLI_DB_PATH is unset or empty (P1)
- README.md: tell native Windows users to copy only the datasource
provisioning file; the dashboards yml contains a Docker-specific path
that is invalid on Windows (P2)
- telemetry.py / sdk.py: emit mt5_terminal_connected,
mt5_terminal_trade_allowed, and mt5_terminal_trade_expert gauges via a
new record_terminal_state() method called from _snapshot_terminal(),
completing the connection-status metric surface from issue #84 (P2)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: add snapshot freshness panel and win-rate column to dashboards
- mt5cli-overview.json: add a full-width "Last Snapshot" stat panel
(dateTimeFromNow unit) below the account stats, querying
MAX(time)*1000 from grafana_account_snapshots so users can tell
whether Grafana is reading a current published copy (#83)
- mt5cli-trades.json: add win_rate_pct computed column to the Trade
Statistics by Symbol table via 100.0 * winning_deals / NULLIF(
total_deals, 0), with a percent unit override and "Win Rate (%)"
display label (#83)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: reject same source and target path in publish_grafana_copy
Adds an early same-path guard to publish_grafana_copy: resolves both
paths before any I/O and raises ValueError if they are identical,
preventing the function from overwriting the live source database with
its own backup copy. Also adds a unit test for the rejected case.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address ruff EM102/TRY003/E501 in same-path guard
Assigns the ValueError message to a variable before raising and
shortens the test docstring to stay within the 88-char line limit.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: apply ruff format to publish_grafana_copy error message
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: remove grafana_ticks panel from default market dashboard
The Tick Bid/Ask panel queried grafana_ticks which only exists when users
collect tick data (opt-in). Users following the default OHLCV-only setup
path hit "no such table: grafana_ticks" on dashboard load.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: close SQLite connections before atomic replace in publish_grafana_copy
Wrap both src and dst connections with contextlib.closing() so they are
explicitly closed before tmp_path.replace(target_path) runs. Without
this, sqlite3.Connection's context manager only commits/rolls back but
leaves the file handle open, which can cause PermissionError on Windows.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: rename history.grafana.db to history.mt5cli.db in Grafana examples
frser-sqlite-datasource blocks paths containing "grafana.db" via its
internal blocklist. Rename the recommended published filename in the
README, compose comment, and datasource provisioning comment to avoid
a blocked/denied datasource for native Windows users.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: update Docker Compose quick-start to pass MT5CLI_DB_PATH
The compose.yml already required MT5CLI_DB_PATH via ${MT5CLI_DB_PATH:?...},
but the README still showed bare `docker compose up -d`. Update the section
to show the env-var-prefixed invocation and document the .env file alternative.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: agent <agent@localhost>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: add Grafana-ready SQLite observability (#79, #80, #81)
New `mt5cli/grafana.py` module with idempotent DDL helpers:
- `create_snapshot_tables` — five SQLite tables for time-series account,
position, order, terminal, and run-status snapshots
- `create_grafana_views` — 13 `grafana_*` views with integer epoch-second
`time` columns; missing source tables emit warnings and are skipped
- `create_grafana_indexes` — 9 performance indexes guarded by column checks
- `ensure_grafana_schema` — convenience wrapper calling all three above
- Insert helpers: `insert_account_snapshot`, `insert_position_snapshots`,
`insert_order_snapshots`, `insert_terminal_snapshot`, `record_snapshot_run`
New stable SDK exports in `mt5cli.__init__` and `mt5cli.contract`:
- `update_observability` — appends a timestamped snapshot to a SQLite db
using an already-connected `Mt5DataClient`; never places orders
- `update_observability_with_config` — standalone wrapper that opens and
closes the MT5 connection automatically
New CLI commands (Collection panel):
- `grafana-schema` — idempotent schema setup, no MT5 connection required
- `snapshot` — append account/position/order/terminal rows; supports
`--symbol`, `--with-account/--no-account`, and equivalent flags
All public modules maintain 100 % branch coverage; 968 tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: apply mdformat to docs after grafana observability additions
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: bump version to 1.1.0
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address PR #86 review feedback
- Fix unaggregated time in grafana_realized_pnl GROUP BY query (MAX)
- Replace O(N) per-symbol API calls with single call + client-side filter
- Eliminate double create_snapshot_tables when with_grafana_schema=True
- Move grafana imports to module level in sdk.py (remove PLC0415 noqa)
- Default with_grafana_schema to False (run grafana-schema once for setup)
- Fix README position example to use snapshot_runs for latest snapshot
- Update tests to reflect new behavior and correct patch targets
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: apply ruff formatting and sync lock file
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address owner review feedback on PR #86
- Filter grafana_*_snapshots views to only expose rows from successful
runs (JOIN snapshot_runs WHERE status='ok'), closing the partial-snapshot
visibility gap raised in PRRT_kwDORzI_286MvDJ6
- Add issubset column guards for snapshot table index creation, consistent
with the rest of create_grafana_indexes (PRRT_kwDORzI_286MvUx1)
- Fix README example queries: views expose 'time' not 'observed_at'
(PRRT_kwDORzI_286MvUxw)
- Correct public-contract.md default for with_grafana_schema (False, not
True) and point to grafana-schema command (PRRT_kwDORzI_286MvUxy)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: add run_id to snapshot schema and fix stale-positions README query
Replace second-level observed_at as the join key between snapshot_runs
and snapshot tables with a stable run_id INTEGER PRIMARY KEY. Two runs
in the same second now get distinct run_ids, preventing view duplication
and cross-contamination from a failed run. Update README example to use
snapshot_runs for latest-snapshot lookup so zero-position runs return an
empty result instead of stale rows.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: format markdown tables
Align table column widths in README and public-contract documentation.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* fix: close sqlite connections deterministically
* fix: require entry filter in grafana_realized_pnl and add time to grafana_trade_stats
grafana_realized_pnl now requires the entry column and filters to
close-side deals (entry IN (1, 2, 3)), consistent with grafana_symbol_pnl
and grafana_trade_stats. grafana_trade_stats now requires the time column
and emits MAX(time_expr) AS "time" so it satisfies the documented Grafana
view contract (integer epoch-second time column throughout).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: normalize pd.Timestamp time_setup to epoch int in insert_order_snapshots
orders_get_as_df() returns datetime-converted columns by default, so
time_setup is a pd.Timestamp in normal use. Passing it directly to
sqlite3.executemany raises ProgrammingError. Added _to_epoch_int helper
that converts datetime.datetime subclasses (including pd.Timestamp) and
raw int/float values to integer epoch seconds, returning None for other
types. Regression tests cover all four input paths.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: pre-drop all grafana_* views at start of create_grafana_views
Previously, a builder that skipped due to a missing source table or column
would not drop the view it owned, leaving stale views referencing gone
tables. Now create_grafana_views drops all 13 known grafana_* views before
calling any builder, so a schema refresh always removes views whose source
has disappeared. Regression test covers the create → drop-source → refresh
cycle.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: expose run_id in snapshot views and drop time from summary views
Grafana snapshot views now expose run_id so latest-state queries can use
MAX(run_id) instead of the ambiguous second-level MAX(observed_at).
grafana_realized_pnl and grafana_trade_stats lose their MAX(time) column
and are reclassified as static summary views; their all-time aggregates
are not filterable by Grafana time-range selectors.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: guard snapshot views against missing run_id and fix README view docs
_build_snapshot_view now skips with a warning when the underlying
snapshot table exists but lacks a run_id column, preventing a broken
view that fails at query time. Adds a regression test for that path.
README Grafana section now qualifies that grafana_realized_pnl and
grafana_trade_stats are static summary views (no time column) and splits
the view table to match docs/api/public-contract.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: agent <agent@localhost>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: add copy_rates_from_pos_as_df fallback in fetch_latest_closed_rates_for_trading_client
Mt5DataClient (returned by create_trading_client) exposes copy_rates_from_pos_as_df,
not fetch_latest_rates_as_df. Adds a fallback path that resolves the granularity string
to an integer timeframe via parse_timeframe, fetches count+1 bars from start_pos=0,
and applies the same drop_forming_rate_bar + tail(count) logic so callers that use
the client returned by create_trading_client no longer need a compatibility shim.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Bump version to v1.0.3
* fix: hoist parse_timeframe before dispatch and add invalid-granularity test
Hoisting parse_timeframe(granularity) before the fetch_latest_rates_as_df /
copy_rates_from_pos_as_df dispatch ensures invalid granularity strings fail
consistently on both paths with a clear ValueError, rather than only when
the fallback branch is taken.
Adds test_copy_rates_from_pos_fallback_raises_on_invalid_granularity to pin
the early-validation contract and confirm the underlying method is never called.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: agent <agent@localhost>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: make SQLite tick history opt-in for collect-history
Ticks can grow SQLite databases quickly, so they are excluded from the
default dataset selection. The new DEFAULT_HISTORY_DATASETS constant
(rates, history-orders, history-deals) drives resolve_history_datasets(None),
collect_history(), and update_history(). Callers must pass
--dataset ticks (CLI) or datasets={Dataset.ticks} (SDK) to include ticks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ALK71tg75JWrrCKaiShb7b
* chore: reformat docs/index.md table column widths
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ALK71tg75JWrrCKaiShb7b
* fix: update stale docstrings and tighten CLI None check
- update_history and ThrottledHistoryUpdater.__init__ docstrings now
state that ticks are opt-in, matching collect_history's wording
- cli.py collect-history uses `is not None` for explicit empty-list safety
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ALK71tg75JWrrCKaiShb7b
* docs: update README collect-history to reflect ticks opt-in default
The command table and section intro previously stated ticks were
collected by default ("all four", "rates, ticks, history-orders, and
history-deals"). Both now reflect the new default (rates, history-orders,
history-deals) and note that --dataset ticks is required to include ticks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ALK71tg75JWrrCKaiShb7b
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat: clarify CLI/docs scope as generic MT5 data and execution infrastructure
- Update app help text and module docstring to describe mt5cli as MT5 data
and execution utilities rather than export-only tooling
- Group CLI commands under rich_help_panel sections: Data / Export, Execution,
and Collection; command names are unchanged for compatibility
- Expand order-send docstring to explicitly flag it as the expert raw-request
live-trading path; preserve --yes gate
- Split docs/index.md Trading section into "Trading State" (read-only) and
"Execution (live / mutating)" with close-positions now documented
- Add TestHelpText tests verifying top-level panel grouping, order-send
expert/live language, and close-positions safety gate coverage
Closes#78
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018yH6esaqc5D1cmo1dK2Ur9
* chore: trim trailing whitespace in docs/index.md table
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018yH6esaqc5D1cmo1dK2Ur9
* chore: bump version to 1.0.1
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018yH6esaqc5D1cmo1dK2Ur9
* chore: update uv.lock for version 1.0.1
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018yH6esaqc5D1cmo1dK2Ur9
* fix: address review feedback on CLI/docs scope PR
- Remove dead help invocation in test_order_send_help_mentions_expert_and_raw
(the result was immediately overwritten by result2)
- Strengthen assertion from `or` to `and`; both "raw" and "expert" are present
in the docstring so disjunction masked a potential regression
- Split into two `assert` statements to satisfy PT018 (ruff)
- Fix docs/index.md inaccuracy: order-check has no --yes gate; clarify that
only order-send and close-positions require confirmation for live execution
- Move order-check from "Execution" rich_help_panel to "Data / Export" so the
Execution panel name is truthful (order-check is read-only)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018yH6esaqc5D1cmo1dK2Ur9
* docs: move order-check out of Execution section into Trading State
order-check is read-only and now lives in the CLI's Data / Export panel,
so documenting it under "Execution (live / mutating)" was inconsistent.
Moved it to the Trading State table. The Execution section now only lists
order-send and close-positions, both of which require --yes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018yH6esaqc5D1cmo1dK2Ur9
---------
Co-authored-by: Claude <noreply@anthropic.com>
* fix: decouple mt5cli from pdmt5 high-level trading helpers
- Replace Mt5TradingClient type annotations with internal _Mt5ClientProtocol
- Lazy-import Mt5TradingClient in create_trading_client to avoid hard dependency
- Replace Mt5TradingError with Mt5OperationError in mt5cli validation paths
- Update exception handling to support future pdmt5 versions without Mt5TradingError
- Add test to enforce that mt5cli doesn't import high-level symbols at module level
- Update documentation to clarify dependency boundaries
mt5cli now relies only on low-level MT5 primitives:
- Mt5Config for configuration
- Mt5RuntimeError for runtime errors
- Raw MT5 methods (order_send, order_check, account_info, etc.)
This aligns with pdmt5's direction to remove high-level trading helpers and focus
on low-level MT5 access plus DataFrame/dict conversion.
Fixes#75 (dceoy/mt5cli#75)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PcGVFTVgyqzse3LLw38ber
* fix: address PR #76 review feedback on pdmt5 decoupling
- Replace Mt5TradingClient with Mt5DataClient in create_trading_client()
so the function no longer depends on the high-level trading client
- Fix _RECOVERABLE_MT5_ERRORS in exceptions.py to use tuple unpacking
form, removing the incorrect ternary assignment
- Add pragma: no cover to except ImportError branches in exceptions.py
and sdk.py (dead code when pdmt5 is installed)
- Switch coverage exclude_lines to exclude_also so the default
pragma: no cover pattern is preserved; also exclude bare ... stubs
(Protocol method bodies) from coverage
- Correct inaccurate note in docs/api/public-contract.md: Mt5TradingClient
is no longer required internally; Mt5TradingError is conditionally
available but mt5cli raises Mt5OperationError for trading failures
- Update all mock patches from pdmt5.Mt5TradingClient to
mt5cli.trading.Mt5DataClient to match the new module-level import
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat: add close-positions CLI command and replace_symbol projection mode (#65#66)
Part 1 — close-positions CLI (#65):
- Add `close-positions` subcommand delegating to `close_open_positions()`.
- Accepts repeated `--symbol` and `--ticket` filters (AND semantics).
- Supports `--dry-run` (no `--yes` required); live execution requires `--yes`.
- Fails closed with `BadParameter` when neither `--symbol` nor `--ticket` is given.
- Exports normalized `OrderExecutionResult` list as a DataFrame (request/response
serialized as JSON strings for clean CSV/JSON/Parquet/SQLite output).
- `order-send` remains the raw expert path; `close-positions` is the safer
high-level helper that builds correct close requests automatically.
Part 2 — ProjectionMode and replace_symbol (#66):
- Add `ProjectionMode = Literal["add", "replace_symbol"]` type alias.
- Add optional `projection_mode` parameter to `calculate_symbol_group_margin_ratio`.
Default `"add"` preserves existing additive behavior.
`"replace_symbol"` subtracts current margin for `new_symbol`, then adds
candidate margin — the subtraction and addition are atomic (suppressed together).
- Export `ProjectionMode` from `mt5cli` and add to `STABLE_SDK_EXPORTS`.
- No mteor-specific strategy, risk-threshold, or policy logic added.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: remove unused ProjectionMode import in test_contracts.py
The parametrized test_stable_exports_are_importable_from_package_root
already covers ProjectionMode via hasattr(mt5cli, name). Ruff correctly
flagged the explicit top-level import as unused (F401).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Bump version to v0.9.6
* fix: address PR #67 review feedback
- Floor replace_symbol margin subtraction at zero to prevent negative ratio
- Serialize response unconditionally via json.dumps (null for dry-run rows)
- Return a schema-preserving empty DataFrame when results list is empty
- Add test: --dry-run --yes precedence (dry-run wins, no order_send)
- Add test: zero-match filter produces empty JSON array with exit 0
- Move projection_mode prose to stable trading section in docs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: add runtime validation for projection_mode in calculate_symbol_group_margin_ratio
Unsupported values previously silently fell through as "add". The new
_validate_projection_mode helper raises ValueError with a message that
names the bad value and the two accepted modes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: agent <agent@localhost>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: support string login in build_config and add substitute_mapping_values (#61, #62)
Extend build_config() to accept login: int | str | None. String logins
are coerced via the existing coerce_login() helper (empty/whitespace →
None, numeric strings → int, non-numeric → ValueError). When
allow_whole_dollar_env=True, ${ENV} and $ENV placeholders are expanded
before coercion, consistent with path/password/server behavior.
Add substitute_mapping_values(), a generic recursive helper that
substitutes environment placeholders in nested dicts/lists only for
caller-selected mapping keys. Non-selected fields (including literal
dollar signs) are preserved exactly. Supports blank_string_keys_as_none
to normalise empty strings to None after substitution. No application-
specific key names (e.g. mt5_login) are hard-coded in mt5cli.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Bump version to v0.9.5
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs+test: clarify substitute_mapping_values docstring and pin tuple behaviour
- Adds sentence noting list-element strings are never substituted (only
immediate dict values are), addressing reviewer finding #1.
- Rewrites Returns section to accurately describe scalar pass-through
behaviour, addressing reviewer finding #2.
- Adds recursion-depth caveat to the generic-utility docstring,
addressing reviewer finding #4.
- Adds test_tuple_container_not_traversed to pin the existing silent
tuple-exclusion contract, addressing reviewer finding #3.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs: update public contract and README for build_config login coercion and substitute_mapping_values
- Expands build_config row to document login: int | str | None,
numeric-string coercion, blank-string handling, and env placeholder
expansion when allow_whole_dollar_env=True.
- Adds substitute_mapping_values to the stable SDK table with a note
that key names are never hard-coded in mt5cli.
- Extends allow_whole_dollar_env paragraph to list substitute_mapping_values.
- README: adds build_config env-placeholder example and imports to the
trading lifecycle snippet.
- README: extends credential-resolution bullet with a substitute_mapping_values
usage example using generic key names.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: agent <agent@localhost>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* test: add explicit unit tests for calculate_positions_margin_by_symbol and calculate_positions_margin_safe (#50)
Covers all acceptance criteria: partial failure with warning log, all-fail,
empty symbol list with no-broker-call assertion, duplicate deduplication,
successful aggregation with first-seen key order, suppress_errors=False
propagation, and three calculate_positions_margin_safe cases (partial skip,
all-fail → 0.0, empty list → 0.0).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test: fix warning log assertion and parametrize suppress_errors=False test
- Use record.getMessage() + levelno check instead of record.message, which
is only populated after formatting and can return an empty string.
- Parametrize test_one_symbol_fails_suppress_errors_false over all three
exception types caught by the implementation (Mt5TradingError,
Mt5RuntimeError, AttributeError) so any future narrowing of the except
tuple would be caught by tests.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style: shorten docstring to fit 88-char line limit
* style: shorten docstring to fit 88-char line limit
---------
Co-authored-by: agent <agent@localhost>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces manual isinstance/<=0 checks in calculate_spread_ratio() and
determine_order_limits() with _valid_tick_price(), ensuring NaN, inf,
-inf, zero, negative, bool, and invalid-string tick values are
consistently rejected across all trading helpers.
Adds regression tests covering numeric-string acceptance and every
invalid-value category for both functions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add _valid_tick_price() internal helper that returns a positive finite float
from a tick dict or None for any invalid value (missing, None, NaN, infinite,
zero, negative, or unsupported type). Refactor five existing bid/ask validation
sites in trading.py to use it, removing duplicated isinstance/isfinite checks.
Add calculate_positions_margin_by_symbol() which computes margin per unique
symbol independently using the existing strict calculate_positions_margin(),
with first-seen deduplication and configurable error suppression
(Mt5TradingError, Mt5RuntimeError, AttributeError) via suppress_errors=.
Add calculate_positions_margin_safe() as a thin sum wrapper with
suppress_errors=True, returning 0.0 on empty or fully-failed inputs.
Both new helpers are exported from mt5cli, added to STABLE_SDK_EXPORTS, and
documented in docs/api/public-contract.md. Existing strict behavior of
calculate_positions_margin() is unchanged.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: re-verify normalized volume margin before returning from calculate_volume_by_margin
For CFDs, index products, and tiered-margin instruments, the initial
min-lot margin estimate can be optimistic; the normalized stepped volume
may require more margin than available_margin. After computing the
normalized volume, step down by volume_step until order_calc_margin
confirms affordability, or return 0.0 if no step is affordable.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test: fix ruff line-length violations in calculate_volume_by_margin tests
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: use integer step index and add actual>0 guard in calculate_volume_by_margin
Replace float-subtraction loop with integer step index to eliminate
accumulation rounding error and add `actual > 0` guard so a broker
returning zero/negative margin is never accepted as affordable.
Inline `capped` to keep local-variable count within Ruff PLR0914 limit.
Update docstring to reflect re-verification behaviour and 0.0 fallback.
Tighten test assertion from `volume > 0` to the symbol's valid range.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Bump version to v0.9.1
* perf: replace linear step-down scan with binary search in calculate_volume_by_margin
Resolves the P2 review finding: the previous O(n) loop called
order_calc_margin once per volume step, making sizing appear hung for
symbols with a large step range or small volume_step.
Binary search over the integer step index finds the largest affordable
step in O(log n) IPC calls (≈17 for a 99 999-step range vs up to 99 999
in the worst case). Monotonicity of broker margin with volume is assumed,
which holds for standard linear margin schedules.
To stay within the PLR0914 local-variable limit the steps variable is
inlined into hi and the tick temporary is eliminated by accessing the
snapshot dict directly. Error messages still go via msg to satisfy EM102.
Two existing tests are updated to match the binary-search call sequence.
A new regression test (volume_min=0.01, volume_max=1000.0) configures
a tiered-margin mock with its threshold at step 50000 and asserts that
the total order_calc_margin call count does not exceed 25.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: remove obsolete TC003 per-file-ignore for history.py
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: agent <agent@localhost>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: add fetch_latest_closed_rates_indexed and allow_whole_dollar_env opt-in (#43, #44)
Closes#43: add fetch_latest_closed_rates_indexed(client, *, symbol,
granularity, count) -> pd.DataFrame to mt5cli/trading.py. Internally
reuses fetch_latest_closed_rates_for_trading_client(), converts the
"time" column to a UTC-aware DatetimeIndex named "time", and drops the
original column. Exported from trading.__all__, mt5cli.__init__, and
STABLE_SDK_EXPORTS.
Closes#44: extend substitute_env_placeholders() with opt-in
allow_whole_dollar_env=False that expands whole-value $ENV_NAME strings
(entire string must be exactly $IDENTIFIER). Threaded through
build_config(), resolve_account_spec(), and resolve_account_specs() with
the same default=False. Partial strings like "plan$pass", "abc$ENV", or
"$ENV-suffix" are never expanded.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: align Markdown table columns in docs and skill file
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: treat numeric (float64) epoch seconds as UTC in _rate_time_to_utc
After DataFrame concat or NA upcast the time column becomes float64, which
is still epoch seconds. Using is_numeric_dtype instead of is_integer_dtype
fixes the silent misalignment. Using series.to_numpy() before passing to
pd.to_datetime avoids the redundant pd.DatetimeIndex() wrapper and aligns
with how existing rate-time normalization in schemas.py handles numeric
timestamps.
Add test_converts_float_epoch_seconds_to_utc_datetime_index to cover the
regression. Add a doc note clarifying that build_config cannot expand
login since that parameter is int | None.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: reject NaT values after rate timestamp conversion in _rate_time_to_utc
pd.to_datetime() silently produces NaT for None/NaN inputs rather than
raising, so the function could return a DatetimeIndex containing NaT
despite documenting invalid timestamps as a ValueError. Check any(idx.isna())
after conversion and raise with a clear message.
Add test_raises_on_nat_time_column to cover the regression.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Bump version to v0.9.0
* fix: handle object numeric rate timestamps
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat: add stable SDK helpers for volume, margin, and closed bars (#39, #40, #41)
Expose generic trading utilities in the stable downstream SDK so applications
like mteor can drop local MT5 adapter code:
- normalize_order_volume() for broker step/min/max sizing
- estimate_order_margin() and calculate_positions_margin() for margin totals
- fetch_latest_closed_rates_for_trading_client() for closed bars from Mt5TradingClient
Update STABLE_SDK_EXPORTS, package-root exports, docs, and unit tests.
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* chore: bump version to 0.8.3
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* fix: address PR review feedback on volume cap, rate time, and margin grouping
- Re-apply volume_max after step normalization in normalize_order_volume()
- Drop misleading non-time index reset branch in _ensure_rate_time_column()
- Group positions by (symbol, side) before margin estimation
- Add branch-coverage tests for tick price validation and volume cap edge case
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* fix: address remaining PR review threads on docs and DatetimeIndex
- Rename unnamed DatetimeIndex column to time after reset_index()
- Guard estimate_order_margin example on positive normalized volume
- Document calculate_positions_margin skip vs error propagation behavior
- Add test for unnamed DatetimeIndex branch coverage
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* fix: harden stable SDK margin, rate fetch, and volume normalization
- Wrap order_calc_margin conversion and reject None/non-numeric results
- Validate fetched rate objects are DataFrames before time normalization
- Return 0.0 for non-finite volume inputs and constraints in normalize_order_volume
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* fix: reject non-finite volumes in margin estimation helpers
Use _is_positive_finite_number() in estimate_order_margin() and
calculate_positions_margin() so NaN/inf volumes never reach broker calls.
Add focused tests and document non-finite volume skipping in trading.md.
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* fix: guard symbol filter in calculate_positions_margin for empty frames
Return 0.0 before filtering when positions are empty or lack a symbol column.
Add regression tests for filtered calls on malformed position frames.
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* feat: add injectable update_backend to ThrottledHistoryUpdater
Allow downstream applications to substitute the history update backend via
the ThrottledHistoryUpdater constructor without monkey-patching
mt5cli.sdk.update_history. Defaults to update_history for backward
compatibility.
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* chore: fix lint and format after QA
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* chore: bump version to 0.8.2
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* fix: use explicit None check for ThrottledHistoryUpdater backend
Only None selects the default update_history backend so falsy callable
objects with __bool__ returning False are preserved as custom backends.
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
Align agent instructions with the streamlined project structure, QA workflow, and security notes used elsewhere in the repo.
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat: add stable MT5Client public API and infrastructure layer
Introduce a reusable public API for downstream trading applications:
- MT5Client as the primary client abstraction with order_check/order_send
- schemas module with DataKind contracts, validation, and normalization
- converters, exceptions, retry, and storage facade modules
- CLI order commands now route through MT5Client
- connected_client made public; retry logic centralized
- Contract tests for API surface, schemas, and storage round-trips
- README and docs updated with Python API usage examples
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* fix: correct time coercion, broker-safe symbols, and execution docs
- Normalize MT5 time columns with correct second/millisecond units
- Coerce all present known MT5 time fields, including optional order times
- Preserve broker symbol casing in normalize_symbol()
- Document order_send() as a live execution primitive with clear scope boundaries
- Add contract tests for timestamp and symbol normalization behavior
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* Consolidate duplicated export and history streaming helpers.
Reduce repeated CLI export plumbing, shared per-symbol SQLite writes, and test mock setup without changing public behavior.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Bump version from 0.7.0 to 0.7.1.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* Refactor MT5 constant parsing to delegate to pdmt5 >= 0.3.0
Replace local TIMEFRAME_MAP, TICK_FLAG_MAP, and parser helpers with thin
compatibility wrappers around pdmt5. COPY_TICKS flags now use real MT5 values
(ALL=-1, INFO=1, TRADE=2). Click parameter types validate all inputs through
the wrappers. Update tests and docs to describe the pdmt5/mt5cli/mt5api layering.
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* Fix timeframe defaults and COPY_TICKS flag defaults after pdmt5 migration
Use short timeframe aliases for default history collection and granularity
naming via pdmt5.get_timeframe_name. Set CLI/SDK default tick flags to ALL
(-1) instead of the legacy mt5cli-only value.
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* Address CI lint failure and PR review feedback
Fix ruff import ordering in history.py. Use ALL string defaults for CLI tick
flags, isolate TICK_FLAG_MAP as a dict snapshot, derive flag names from pdmt5,
reuse TIMEFRAME_NAMES for default history timeframes, and add tests for prefix
stripping and TIMEFRAME_ key filtering.
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* Bump version to 0.7.0
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* Add trading session helpers and extend ThrottledHistoryUpdater
Introduce mt5cli.trading with mt5_trading_session() for Mt5TradingClient
lifecycle management and reusable operational helpers for position-side
detection, margin/volume sizing, and protective order price derivation.
Extend ThrottledHistoryUpdater to validate inputs before updates and to
optionally suppress ValueError, OSError, and missing-method errors without
advancing the throttle timestamp.
Export the new helpers from mt5cli.__init__, add unit tests with mocked
clients, and document migration guidance for downstream projects such as
mteor.
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* Narrow ThrottledHistoryUpdater suppress_errors handling (#27)
* Narrow ThrottledHistoryUpdater suppress_errors for MT5 capability only
Remove broad AttributeError/TypeError handling from recoverable errors.
Add _is_mt5_client_capability_error() to detect missing history API methods
or non-callable client attributes by message and attribute name.
Generic AttributeError/TypeError values always propagate even when
suppress_errors=True. Update docs and tests accordingly.
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* Detect non-callable history client methods in suppress_errors
Address review feedback: when a history API attribute exists but is not
callable, Python raises a generic TypeError. Inspect the traceback for
mt5cli.history client call sites so these capability mismatches are still
suppressed without matching all TypeError values.
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* Address PR review feedback on trading helpers
- Resolve history module path once at import time
- Only treat non-callable TypeErrors as capability errors at the raise site
- Validate SL/TP ratios in determine_order_limits
- Add tests for margin_free edge cases, body-raise shutdown, and internal TypeError propagation
- Clarify ThrottledHistoryUpdater suppress_errors docs
- Split README migration example into trading vs read-only history sessions
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* Tighten protective ratio validation and clamp negative margin_free
Add _require_protective_ratio enforcing 0 <= ratio < 1 for SL/TP limits so
a ratio of 1.0 cannot produce zero protective prices. Clamp negative
margin_free to 0.0 in calculate_margin_and_volume before sizing.
Add boundary and negative-margin tests; document constraints in trading API
docs.
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* Add closed-bar rate helpers and bump version to 0.6.0.
Expose drop_forming_rate_bar and multi-account collectors so downstream apps no longer need count+1 fetches and manual bar trimming.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Bump pygments to 2.20.0 to fix CVE-2026-4539 ReDoS advisory.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address PR review feedback on closed-bar rate collection.
Validate count and start_pos before MT5 fetches, avoid redundant frame copies, clarify empty-series errors, and expand test coverage.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Include symbol and timeframe in empty closed-rate error messages.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
dceoy