c2cf0656ddb113ebcc41bf366b98f691ed1c63e5
46 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
c2cf0656dd |
Drop Mt5TradingError support; require pdmt5>=1.0.4 (#101)
* chore: upgrade pdmt5 to v1.0.4 pdmt5 1.0.4 removes Mt5TradingClient/Mt5TradingError entirely and wraps Mt5Config.password in pydantic SecretStr. Drop the now-dead conditional Mt5TradingError handling in mt5cli.exceptions/sdk/retry (mt5cli already type-checks trading clients against its own protocol, so no functional change), and unwrap SecretStr when forwarding a base config's password to per-account configs. Update tests and docs accordingly. * chore: declare pydantic as a direct runtime dependency mt5cli.sdk imports SecretStr directly from pydantic, so pin it explicitly instead of relying on pdmt5's transitive dependency. --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
efc0de230a |
fix: stabilize history timestamps and telemetry docs (#100)
* fix: stabilize history timestamps and telemetry docs * fix: preserve numeric epoch cursors in history SQLite queries Normalize mixed ISO and unixepoch time values for incremental resume and scoped dedup so legacy numeric rows are not dropped by julianday filters. Co-authored-by: Cursor <cursoragent@cursor.com> * Bump version to v1.1.2 * fix: aggregate incremental start timestamps in SQLite Use MAX on the normalized time expression with GROUP BY so incremental resume loaders stay O(groups) instead of materializing every history row. Co-authored-by: Cursor <cursoragent@cursor.com> * test: parametrize duplicated incremental-start cases in TestIncrementalStart Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
a5c2aa8e8f |
test: consolidate duplicate test cases into parameterized tests (#91)
* test: consolidate duplicate test cases into parameterized tests - Combine test_symbol_info, test_symbol_info_tick, test_market_book into test_symbol_command with parametrize - Combine test_order_check variants into test_order_request with parametrize - Combine test_order_send variants into test_order_request with parametrize - Consolidate grafana view skip tests into test_grafana_view_skipped_when_required_cols_missing - Consolidate grafana index skip tests into test_index_skipped_when_cols_missing - Consolidate _NoOp method tests into test_method_is_noop - Consolidate record_*_update noop tests into test_record_update_noop_before_configure This improves test maintainability by reducing duplication while maintaining coverage. * test: parametrize remaining duplicated cases * test: parameterize remaining duplicate test cases * test: parameterize remaining duplicate test cases * test: parameterize remaining duplicate test cases * test: parameterize remaining duplicate test cases * test: parameterize count and start_pos validation tests * test: parameterize remaining duplicate test cases - tests/test_utils.py: merge valid string/integer cases for parse_timeframe and parse_tick_flags. - tests/test_cli.py: consolidate collect-history --if-exists append/fail tests and drop the duplicate collect-history --dataset ticks default flags=-1 assertion. - tests/test_trading.py: parametrize buy/sell projected margin ratio, invalid place_market_order mode validation, core calculate_volume_by_margin boundary cases, and zero-ratio calculate_margin_and_volume sizing cases. * test: parameterize remaining duplicate test cases Consolidate the last high-value duplicate test cases outside the already-changed areas while keeping semantics and 100% coverage. - tests/test_sdk.py:TestMt5CliClient - merge copy_rates_range, copy_ticks_from, history_orders, and latest_rates delegation/ normalization tests into test_method_delegates_with_normalization parameterized by call/expected_method/expected_kwargs with readable ids for each normalization path. - tests/test_sdk.py:TestSubstituteEnvPlaceholders - merge brace/whole-dollar/plain/partial substitution cases into test_substitute_env_placeholders, and missing-env cases into test_substitute_env_placeholders_raises_on_missing_env. - tests/test_trading.py:TestVolumeAndExecution - merge buy/sell trailing-stop main cases into test_calculate_trailing_stop_updates_by_side, opposite-side invalid-quote cases into test_calculate_trailing_stop_updates_ignores_opposite_side_price, and the two mixed-positions scenarios into the parametrized test_calculate_trailing_stop_updates_mixed_positions_skip_invalid_side. All ids explicitly describe the side and behavior. * test: parameterize collect-history and telemetry update tests Consolidate duplicate default-vs-ticks collect-history cases in CLI and SDK tests, and merge history/snapshot record_*_update success and failure tests in test_telemetry.py. Co-authored-by: Cursor <cursoragent@cursor.com> * test: parameterize telemetry record_*_state gauge tests Co-authored-by: Cursor <cursoragent@cursor.com> * test: parameterize Grafana example file checks * test: centralize remaining parametrized cases * test: add parametrized replacements for remaining cases * test: fix remaining parametrized lint * Fix Ruff warnings * test: remove duplicate parametrization sweep file * test: remove duplicate-test deselection hook * test: restore conftest formatting * test: add remaining parametrized regression coverage * test: fix lint issues in parametrization completion tests * test: avoid dynamic SQL in parametrization completion tests * test: format parametrization completion tests * test: fix lint and pyright issues in parametrization coverage * Fix a Pyright error * Update .agents/skills/pr-feedback-triage/SKILL.md * test: dissolve parametrize-completion catch-all into owning test files Move the remaining consolidated cases from test_parametrize_completion.py into their owning modules and delete the file: - test_cli.py: merge snapshot/grafana-schema --publish-copy gating into one parametrized test. - test_trading.py: merge calculate_spread_ratio numeric/numeric-string cases; fold non-finite volume_max into test_normalize_order_volume_deterministic; merge estimate_order_margin invalid-volume cases (0.0/nan/inf); merge calculate_positions_margin invalid-row filtering cases. - test_history.py: parametrize resolve_history_tick_flags and resolve_granularity_name. - test_sdk.py: parametrize collect_latest_closed_rates_for_accounts empty-effective-frame cases. - test_grafana.py: merge account/terminal snapshot insert tests inside TestSnapshotInserts. All remaining cases in the removed file were exact duplicates of existing coverage, so no new test scenarios are introduced. 100% coverage and all checks (ruff, pyright, pytest, local-qa) pass. * test: parameterize remaining account-event, snapshot, and CLI edge cases Consolidates six more pairs of duplicate tests flagged during PR #91 review into pytest.mark.parametrize tables (incremental history_deals account-event edge cases, dedup scope column variants, snapshot view status filtering, snapshot kwargs forwarding, injected-client lifecycle, and close-positions --yes gate), preserving explicit ids and 100% coverage. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * test: consolidate build_config login coercion into parametrized cases Fold separate backward-compat and env-expansion tests into named pytest.param rows for clearer failure output. Co-authored-by: Cursor <cursoragent@cursor.com> * test: consolidate deduplication and publish test cases into parametrized variants Merge related test cases into single parametrized tests: - Consolidate collect-history validation tests into a single parametrized case - Merge snapshot-view skip conditions into one test with parametrized inputs - Combine publish_grafana_copy target variations into one parametrized test - Consolidate incremental-start and drop-duplicates error cases - Remove duplicate test_drop_duplicates_rejects_invalid_identifiers Reduces test file duplication while maintaining full coverage. * test: consolidate remaining review-identified parameterized duplicates Merge close-position filter, margin-ratio suppress/reraise, SQLite path validation, and grafana_symbol_pnl schema cases into clearer parametrized tests. Co-authored-by: Cursor <cursoragent@cursor.com> * test: restore default if_exists coverage in SQLite append test The default-append parametrized case now omits if_exists so the API default remains exercised instead of passing IfExists.APPEND explicitly. Co-authored-by: Cursor <cursoragent@cursor.com> * test: tighten trading parametrization cleanup * test: consolidate remaining parameterized cases * test: consolidate additional duplicate cases into parametrized tests Merge overlapping success-path tests in contracts, history, SDK, and trading modules while preserving 100% coverage and explicit test ids. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: agent <agent@localhost> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
|
513eb7617d |
feat: add fetch_recent_history_deals_for_trading_client to stable SDK (#90)
* refactor: collapse repeated tests with pytest.mark.parametrize Collapse 13 near-identical test methods into 4 parametrized tests across test_cli.py and test_sdk.py, keeping all 1045 cases passing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add fetch_recent_history_deals_for_trading_client to stable SDK Adds a generic history deal retrieval helper for active trading clients, a _HistoryDealsClientProtocol describing the minimal required interface, clarified create_trading_client() docs (returns pdmt5.Mt5DataClient, not MT5Client), 9 unit tests at 100% coverage, and updated trading.md and public-contract.md with examples and out-of-scope strategy semantics note. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: narrow Mt5CliClient protocol claim and preserve empty deal DataFrame schema - _HistoryDealsClientProtocol docstring and fetch_recent_history_deals_for_trading_client docstring now explicitly state that Mt5CliClient (mt5_session) exposes history_deals() not history_deals_get_as_df() and does not satisfy the protocol; the function is for trading-client sessions (pdmt5.Mt5DataClient) only - Empty DataFrames with columns are now passed through with reset_index rather than replaced by a bare pd.DataFrame(), preserving schema for callers that rely on stable column names even in no-deal windows - Tests updated to assert schema preservation on empty results and bare empty on None Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: add combined protocol so create_trading_client() is type-safe with history deals helper Adds _TradingHistoryDealsClientProtocol combining _Mt5ClientProtocol and _HistoryDealsClientProtocol, and updates create_trading_client() and mt5_trading_session() to return/yield this combined type so the natural SDK flow `client = create_trading_client(...); fetch_recent_history_deals_for_trading_client(client)` is type-safe under pyright strict without casts. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: validate hours is finite before timedelta in fetch_recent_history_deals_for_trading_client Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Bump version to 1.1.1 --------- Co-authored-by: agent <agent@localhost> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
1ffac45d57 |
feat: add publish_grafana_copy, Grafana examples, and optional OTel metrics (#89)
* feat: Grafana copy publishing, dashboard examples, and optional OTel metrics Implements three observability improvements: #82 — publish_grafana_copy(): Uses SQLite online backup API (WAL-safe) to atomically publish a consistent read-only copy beside the target. Adds --publish-copy option to grafana-schema and snapshot CLI commands. #83 — examples/grafana/: Minimal working Grafana setup with docker-compose, provisioning datasource/dashboard YAML, and three dashboard JSON files (mt5cli-overview, mt5cli-trades, mt5cli-market). All queries use grafana_* views; no credentials or private paths included. #84 — mt5cli/telemetry.py: Optional OTel metrics behind mt5cli[otel] extra. Base install is unaffected. Adds _Mt5Metrics singleton (no-op until configure_metrics() is called), wraps update_history() and update_observability() with record_history_update / record_snapshot_update context managers, and emits account/position gauges from snapshots. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: replace ambiguous multiplication sign in comment Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: normalize markdown formatting in grafana README Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: preserve file mode on Grafana copy and fix unsupported time macro - publish_grafana_copy: chmod temp file to match the existing target's permissions (or 0o644 when no prior target exists) before atomic replace, so Grafana running as a different OS user (e.g. UID 472 in Docker) can read the published database - mt5cli-market.json: replace unsupported \$__timeFilter(time) with the epoch-based filter supported by frser-sqlite-datasource: "time" >= \$__from / 1000 AND "time" < \$__to / 1000 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: skip Windows-incompatible mode test, rename compose file to compose.yaml - Skip test_overwrite_preserves_existing_target_mode on win32 since Windows chmod does not preserve Unix group/other permission bits - Simplify test_fresh_target_has_readable_permissions to check owner read bit only (portable across platforms) - Rename docker-compose.yml -> compose.yaml (modern Compose convention) - Update README and test reference to match new filename Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: rename *.yaml to *.yml in examples/grafana Renames compose.yaml, mt5cli-sqlite.yaml, and mt5cli.yaml to .yml; updates README and test references accordingly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: format Grafana dashboards and expand qa script to include JSON - Update qa.sh prettier pattern to format JSON files alongside markdown - Reformat Grafana dashboard JSONs with consistent spacing Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address owner review comments before merge - qa.sh: fix Prettier glob from `{,d,json}` to `{md,json}` so Markdown files are actually formatted by local QA (P2) - compose.yml: add GF_INSTALL_PLUGINS env var so the frser-sqlite-datasource plugin is installed at container start (P1) - telemetry.py: replace no-op get_meter() call with a real SDK MeterProvider pipeline; add optional `readers` kwarg so callers can inject custom readers (e.g. InMemoryMetricReader in tests) without needing the OTLP package (P1) - sdk.py: aggregate profit and volume by symbol before emitting gauge values so hedging accounts with multiple same-symbol positions emit one point per symbol instead of overwriting with each row (P2) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: emit mt5_history_update_rows_total via conn.total_changes delta The counter was registered but never incremented, making the advertised history-update throughput metric permanently zero. Add add_history_rows() to _Mt5Metrics and call it in update_history() using the SQLite total_changes delta measured around write_incremental_datasets(). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address three owner review comments - compose.yml: replace soft fallback with :? error expansion so Compose refuses to start when MT5CLI_DB_PATH is unset or empty (P1) - README.md: tell native Windows users to copy only the datasource provisioning file; the dashboards yml contains a Docker-specific path that is invalid on Windows (P2) - telemetry.py / sdk.py: emit mt5_terminal_connected, mt5_terminal_trade_allowed, and mt5_terminal_trade_expert gauges via a new record_terminal_state() method called from _snapshot_terminal(), completing the connection-status metric surface from issue #84 (P2) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add snapshot freshness panel and win-rate column to dashboards - mt5cli-overview.json: add a full-width "Last Snapshot" stat panel (dateTimeFromNow unit) below the account stats, querying MAX(time)*1000 from grafana_account_snapshots so users can tell whether Grafana is reading a current published copy (#83) - mt5cli-trades.json: add win_rate_pct computed column to the Trade Statistics by Symbol table via 100.0 * winning_deals / NULLIF( total_deals, 0), with a percent unit override and "Win Rate (%)" display label (#83) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: reject same source and target path in publish_grafana_copy Adds an early same-path guard to publish_grafana_copy: resolves both paths before any I/O and raises ValueError if they are identical, preventing the function from overwriting the live source database with its own backup copy. Also adds a unit test for the rejected case. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address ruff EM102/TRY003/E501 in same-path guard Assigns the ValueError message to a variable before raising and shortens the test docstring to stay within the 88-char line limit. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: apply ruff format to publish_grafana_copy error message Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: remove grafana_ticks panel from default market dashboard The Tick Bid/Ask panel queried grafana_ticks which only exists when users collect tick data (opt-in). Users following the default OHLCV-only setup path hit "no such table: grafana_ticks" on dashboard load. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: close SQLite connections before atomic replace in publish_grafana_copy Wrap both src and dst connections with contextlib.closing() so they are explicitly closed before tmp_path.replace(target_path) runs. Without this, sqlite3.Connection's context manager only commits/rolls back but leaves the file handle open, which can cause PermissionError on Windows. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: rename history.grafana.db to history.mt5cli.db in Grafana examples frser-sqlite-datasource blocks paths containing "grafana.db" via its internal blocklist. Rename the recommended published filename in the README, compose comment, and datasource provisioning comment to avoid a blocked/denied datasource for native Windows users. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: update Docker Compose quick-start to pass MT5CLI_DB_PATH The compose.yml already required MT5CLI_DB_PATH via ${MT5CLI_DB_PATH:?...}, but the README still showed bare `docker compose up -d`. Update the section to show the env-var-prefixed invocation and document the .env file alternative. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: agent <agent@localhost> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
d27da02f3f |
feat: add Grafana-ready SQLite observability (#86)
* feat: add Grafana-ready SQLite observability (#79, #80, #81) New `mt5cli/grafana.py` module with idempotent DDL helpers: - `create_snapshot_tables` — five SQLite tables for time-series account, position, order, terminal, and run-status snapshots - `create_grafana_views` — 13 `grafana_*` views with integer epoch-second `time` columns; missing source tables emit warnings and are skipped - `create_grafana_indexes` — 9 performance indexes guarded by column checks - `ensure_grafana_schema` — convenience wrapper calling all three above - Insert helpers: `insert_account_snapshot`, `insert_position_snapshots`, `insert_order_snapshots`, `insert_terminal_snapshot`, `record_snapshot_run` New stable SDK exports in `mt5cli.__init__` and `mt5cli.contract`: - `update_observability` — appends a timestamped snapshot to a SQLite db using an already-connected `Mt5DataClient`; never places orders - `update_observability_with_config` — standalone wrapper that opens and closes the MT5 connection automatically New CLI commands (Collection panel): - `grafana-schema` — idempotent schema setup, no MT5 connection required - `snapshot` — append account/position/order/terminal rows; supports `--symbol`, `--with-account/--no-account`, and equivalent flags All public modules maintain 100 % branch coverage; 968 tests pass. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: apply mdformat to docs after grafana observability additions Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: bump version to 1.1.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address PR #86 review feedback - Fix unaggregated time in grafana_realized_pnl GROUP BY query (MAX) - Replace O(N) per-symbol API calls with single call + client-side filter - Eliminate double create_snapshot_tables when with_grafana_schema=True - Move grafana imports to module level in sdk.py (remove PLC0415 noqa) - Default with_grafana_schema to False (run grafana-schema once for setup) - Fix README position example to use snapshot_runs for latest snapshot - Update tests to reflect new behavior and correct patch targets Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: apply ruff formatting and sync lock file Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address owner review feedback on PR #86 - Filter grafana_*_snapshots views to only expose rows from successful runs (JOIN snapshot_runs WHERE status='ok'), closing the partial-snapshot visibility gap raised in PRRT_kwDORzI_286MvDJ6 - Add issubset column guards for snapshot table index creation, consistent with the rest of create_grafana_indexes (PRRT_kwDORzI_286MvUx1) - Fix README example queries: views expose 'time' not 'observed_at' (PRRT_kwDORzI_286MvUxw) - Correct public-contract.md default for with_grafana_schema (False, not True) and point to grafana-schema command (PRRT_kwDORzI_286MvUxy) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: add run_id to snapshot schema and fix stale-positions README query Replace second-level observed_at as the join key between snapshot_runs and snapshot tables with a stable run_id INTEGER PRIMARY KEY. Two runs in the same second now get distinct run_ids, preventing view duplication and cross-contamination from a failed run. Update README example to use snapshot_runs for latest-snapshot lookup so zero-position runs return an empty result instead of stale rows. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: format markdown tables Align table column widths in README and public-contract documentation. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> * fix: close sqlite connections deterministically * fix: require entry filter in grafana_realized_pnl and add time to grafana_trade_stats grafana_realized_pnl now requires the entry column and filters to close-side deals (entry IN (1, 2, 3)), consistent with grafana_symbol_pnl and grafana_trade_stats. grafana_trade_stats now requires the time column and emits MAX(time_expr) AS "time" so it satisfies the documented Grafana view contract (integer epoch-second time column throughout). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: normalize pd.Timestamp time_setup to epoch int in insert_order_snapshots orders_get_as_df() returns datetime-converted columns by default, so time_setup is a pd.Timestamp in normal use. Passing it directly to sqlite3.executemany raises ProgrammingError. Added _to_epoch_int helper that converts datetime.datetime subclasses (including pd.Timestamp) and raw int/float values to integer epoch seconds, returning None for other types. Regression tests cover all four input paths. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: pre-drop all grafana_* views at start of create_grafana_views Previously, a builder that skipped due to a missing source table or column would not drop the view it owned, leaving stale views referencing gone tables. Now create_grafana_views drops all 13 known grafana_* views before calling any builder, so a schema refresh always removes views whose source has disappeared. Regression test covers the create → drop-source → refresh cycle. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: expose run_id in snapshot views and drop time from summary views Grafana snapshot views now expose run_id so latest-state queries can use MAX(run_id) instead of the ambiguous second-level MAX(observed_at). grafana_realized_pnl and grafana_trade_stats lose their MAX(time) column and are reclassified as static summary views; their all-time aggregates are not filterable by Grafana time-range selectors. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: guard snapshot views against missing run_id and fix README view docs _build_snapshot_view now skips with a warning when the underlying snapshot table exists but lacks a run_id column, preventing a broken view that fails at query time. Adds a regression test for that path. README Grafana section now qualifies that grafana_realized_pnl and grafana_trade_stats are static summary views (no time column) and splits the view table to match docs/api/public-contract.md. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: agent <agent@localhost> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
4bc36d09d2 |
fix: add copy_rates_from_pos_as_df fallback for trading client rate fetch (#88)
* fix: add copy_rates_from_pos_as_df fallback in fetch_latest_closed_rates_for_trading_client Mt5DataClient (returned by create_trading_client) exposes copy_rates_from_pos_as_df, not fetch_latest_rates_as_df. Adds a fallback path that resolves the granularity string to an integer timeframe via parse_timeframe, fetches count+1 bars from start_pos=0, and applies the same drop_forming_rate_bar + tail(count) logic so callers that use the client returned by create_trading_client no longer need a compatibility shim. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Bump version to v1.0.3 * fix: hoist parse_timeframe before dispatch and add invalid-granularity test Hoisting parse_timeframe(granularity) before the fetch_latest_rates_as_df / copy_rates_from_pos_as_df dispatch ensures invalid granularity strings fail consistently on both paths with a clear ValueError, rather than only when the fallback branch is taken. Adds test_copy_rates_from_pos_fallback_raises_on_invalid_granularity to pin the early-validation contract and confirm the underlying method is never called. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: agent <agent@localhost> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
80c3f3f65e |
Make ticks dataset opt-in for collect-history (#87)
* feat: make SQLite tick history opt-in for collect-history
Ticks can grow SQLite databases quickly, so they are excluded from the
default dataset selection. The new DEFAULT_HISTORY_DATASETS constant
(rates, history-orders, history-deals) drives resolve_history_datasets(None),
collect_history(), and update_history(). Callers must pass
--dataset ticks (CLI) or datasets={Dataset.ticks} (SDK) to include ticks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ALK71tg75JWrrCKaiShb7b
* chore: reformat docs/index.md table column widths
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ALK71tg75JWrrCKaiShb7b
* fix: update stale docstrings and tighten CLI None check
- update_history and ThrottledHistoryUpdater.__init__ docstrings now
state that ticks are opt-in, matching collect_history's wording
- cli.py collect-history uses `is not None` for explicit empty-list safety
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ALK71tg75JWrrCKaiShb7b
* docs: update README collect-history to reflect ticks opt-in default
The command table and section intro previously stated ticks were
collected by default ("all four", "rates, ticks, history-orders, and
history-deals"). Both now reflect the new default (rates, history-orders,
history-deals) and note that --dataset ticks is required to include ticks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ALK71tg75JWrrCKaiShb7b
---------
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|
|
43f632bc40 |
Reorganize CLI help text and command grouping for data/execution clarity (#85)
* feat: clarify CLI/docs scope as generic MT5 data and execution infrastructure - Update app help text and module docstring to describe mt5cli as MT5 data and execution utilities rather than export-only tooling - Group CLI commands under rich_help_panel sections: Data / Export, Execution, and Collection; command names are unchanged for compatibility - Expand order-send docstring to explicitly flag it as the expert raw-request live-trading path; preserve --yes gate - Split docs/index.md Trading section into "Trading State" (read-only) and "Execution (live / mutating)" with close-positions now documented - Add TestHelpText tests verifying top-level panel grouping, order-send expert/live language, and close-positions safety gate coverage Closes #78 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018yH6esaqc5D1cmo1dK2Ur9 * chore: trim trailing whitespace in docs/index.md table Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018yH6esaqc5D1cmo1dK2Ur9 * chore: bump version to 1.0.1 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018yH6esaqc5D1cmo1dK2Ur9 * chore: update uv.lock for version 1.0.1 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018yH6esaqc5D1cmo1dK2Ur9 * fix: address review feedback on CLI/docs scope PR - Remove dead help invocation in test_order_send_help_mentions_expert_and_raw (the result was immediately overwritten by result2) - Strengthen assertion from `or` to `and`; both "raw" and "expert" are present in the docstring so disjunction masked a potential regression - Split into two `assert` statements to satisfy PT018 (ruff) - Fix docs/index.md inaccuracy: order-check has no --yes gate; clarify that only order-send and close-positions require confirmation for live execution - Move order-check from "Execution" rich_help_panel to "Data / Export" so the Execution panel name is truthful (order-check is read-only) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018yH6esaqc5D1cmo1dK2Ur9 * docs: move order-check out of Execution section into Trading State order-check is read-only and now lives in the CLI's Data / Export panel, so documenting it under "Execution (live / mutating)" was inconsistent. Moved it to the Trading State table. The Execution section now only lists order-send and close-positions, both of which require --yes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018yH6esaqc5D1cmo1dK2Ur9 --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
93565681e1 |
fix: decouple mt5cli from pdmt5 high-level trading helpers (#76)
* fix: decouple mt5cli from pdmt5 high-level trading helpers - Replace Mt5TradingClient type annotations with internal _Mt5ClientProtocol - Lazy-import Mt5TradingClient in create_trading_client to avoid hard dependency - Replace Mt5TradingError with Mt5OperationError in mt5cli validation paths - Update exception handling to support future pdmt5 versions without Mt5TradingError - Add test to enforce that mt5cli doesn't import high-level symbols at module level - Update documentation to clarify dependency boundaries mt5cli now relies only on low-level MT5 primitives: - Mt5Config for configuration - Mt5RuntimeError for runtime errors - Raw MT5 methods (order_send, order_check, account_info, etc.) This aligns with pdmt5's direction to remove high-level trading helpers and focus on low-level MT5 access plus DataFrame/dict conversion. Fixes #75 (dceoy/mt5cli#75) Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PcGVFTVgyqzse3LLw38ber * fix: address PR #76 review feedback on pdmt5 decoupling - Replace Mt5TradingClient with Mt5DataClient in create_trading_client() so the function no longer depends on the high-level trading client - Fix _RECOVERABLE_MT5_ERRORS in exceptions.py to use tuple unpacking form, removing the incorrect ternary assignment - Add pragma: no cover to except ImportError branches in exceptions.py and sdk.py (dead code when pdmt5 is installed) - Switch coverage exclude_lines to exclude_also so the default pragma: no cover pattern is preserved; also exclude bare ... stubs (Protocol method bodies) from coverage - Correct inaccurate note in docs/api/public-contract.md: Mt5TradingClient is no longer required internally; Mt5TradingError is conditionally available but mt5cli raises Mt5OperationError for trading failures - Update all mock patches from pdmt5.Mt5TradingClient to mt5cli.trading.Mt5DataClient to match the new module-level import --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
f435544f07 | Shrink public API surface and remove storage re-export module (#74) | ||
|
|
668f38d8aa | feat: reduce package-root API surface and require pdmt5>=1.0.0 (closes #70) (#73) | ||
|
|
9dbb46fbb1 | feat: make pyarrow optional via mt5cli[parquet] extra (#69) | ||
|
|
dfe80ce500 |
feat: add close-positions CLI and replace_symbol projection mode (#65 #66) (#67)
* feat: add close-positions CLI command and replace_symbol projection mode (#65 #66) Part 1 — close-positions CLI (#65): - Add `close-positions` subcommand delegating to `close_open_positions()`. - Accepts repeated `--symbol` and `--ticket` filters (AND semantics). - Supports `--dry-run` (no `--yes` required); live execution requires `--yes`. - Fails closed with `BadParameter` when neither `--symbol` nor `--ticket` is given. - Exports normalized `OrderExecutionResult` list as a DataFrame (request/response serialized as JSON strings for clean CSV/JSON/Parquet/SQLite output). - `order-send` remains the raw expert path; `close-positions` is the safer high-level helper that builds correct close requests automatically. Part 2 — ProjectionMode and replace_symbol (#66): - Add `ProjectionMode = Literal["add", "replace_symbol"]` type alias. - Add optional `projection_mode` parameter to `calculate_symbol_group_margin_ratio`. Default `"add"` preserves existing additive behavior. `"replace_symbol"` subtracts current margin for `new_symbol`, then adds candidate margin — the subtraction and addition are atomic (suppressed together). - Export `ProjectionMode` from `mt5cli` and add to `STABLE_SDK_EXPORTS`. - No mteor-specific strategy, risk-threshold, or policy logic added. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: remove unused ProjectionMode import in test_contracts.py The parametrized test_stable_exports_are_importable_from_package_root already covers ProjectionMode via hasattr(mt5cli, name). Ruff correctly flagged the explicit top-level import as unused (F401). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Bump version to v0.9.6 * fix: address PR #67 review feedback - Floor replace_symbol margin subtraction at zero to prevent negative ratio - Serialize response unconditionally via json.dumps (null for dry-run rows) - Return a schema-preserving empty DataFrame when results list is empty - Add test: --dry-run --yes precedence (dry-run wins, no order_send) - Add test: zero-match filter produces empty JSON array with exit 0 - Move projection_mode prose to stable trading section in docs Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add runtime validation for projection_mode in calculate_symbol_group_margin_ratio Unsupported values previously silently fell through as "add". The new _validate_projection_mode helper raises ValueError with a message that names the bad value and the two accepted modes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: agent <agent@localhost> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
15bfd17db3 |
test: reduce test_trading.py duplication with parametrize (#64)
* test: reduce test_trading.py duplication with parametrize Collapse repetitive individual tests in test_trading.py into parametrized equivalents, cutting 267 lines without losing any cases. - TestExtractTickPrice: 13 tests → 2 parametrized (×3 valid, ×10 None) - TestEstimateOrderMargin: 4 invalid-margin tests → 1 parametrized ×4; nan/inf volume tests → 1 parametrized ×2 - TestNormalizeOrderVolume: multi-assert bodies split into parametrized cases for non-finite volume and constraints - TestVolumeAndExecution: 9 place_market_order retcode tests → 1 ×11; 5 update_sltp retcode tests → 1 ×5 - test_calculate_trailing_stop_updates_missing_symbol_digits: inline double-assert body → 1 parametrized ×2 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test: further reduce test_trading.py duplication with parametrize Merge six broker stop-level tests into two parametrized tests, collapse two default-digits fallback tests and three symbol-filter zero-margin tests into one each. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test: address claude[bot] review on PR #64 - Consolidate _MISSING_RETCODE sentinel to one line with corrected comment - Add comment explaining ids list is required for deterministic node IDs - Document intentional narrower retcode coverage in update_sltp test Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test: reduce duplication in test_sdk, test_history, test_contracts - TestBuildConfigWholeDollarEnv: 3 field tests (server/password/path) → 1 parametrized ×3 - TestResolveAccountSpec: whole-dollar expand/no-expand pair → 1 parametrized ×2 - test_normalize_mt5_exception_maps_types: 2 isinstance asserts → parametrized ×2 - test_resolve_history_tick_flags_invalid: 2 pytest.raises blocks → parametrized ×2 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: agent <agent@localhost> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
37eef16e99 |
feat: support string login in build_config and add substitute_mapping_values (#63)
* feat: support string login in build_config and add substitute_mapping_values (#61, #62) Extend build_config() to accept login: int | str | None. String logins are coerced via the existing coerce_login() helper (empty/whitespace → None, numeric strings → int, non-numeric → ValueError). When allow_whole_dollar_env=True, ${ENV} and $ENV placeholders are expanded before coercion, consistent with path/password/server behavior. Add substitute_mapping_values(), a generic recursive helper that substitutes environment placeholders in nested dicts/lists only for caller-selected mapping keys. Non-selected fields (including literal dollar signs) are preserved exactly. Supports blank_string_keys_as_none to normalise empty strings to None after substitution. No application- specific key names (e.g. mt5_login) are hard-coded in mt5cli. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Bump version to v0.9.5 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs+test: clarify substitute_mapping_values docstring and pin tuple behaviour - Adds sentence noting list-element strings are never substituted (only immediate dict values are), addressing reviewer finding #1. - Rewrites Returns section to accurately describe scalar pass-through behaviour, addressing reviewer finding #2. - Adds recursion-depth caveat to the generic-utility docstring, addressing reviewer finding #4. - Adds test_tuple_container_not_traversed to pin the existing silent tuple-exclusion contract, addressing reviewer finding #3. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: update public contract and README for build_config login coercion and substitute_mapping_values - Expands build_config row to document login: int | str | None, numeric-string coercion, blank-string handling, and env placeholder expansion when allow_whole_dollar_env=True. - Adds substitute_mapping_values to the stable SDK table with a note that key names are never hard-coded in mt5cli. - Extends allow_whole_dollar_env paragraph to list substitute_mapping_values. - README: adds build_config env-placeholder example and imports to the trading lifecycle snippet. - README: extends credential-resolution bullet with a substitute_mapping_values usage example using generic key names. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: agent <agent@localhost> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
96c75f7852 |
Add account-wide projected margin ratio helper (#60)
* feat: add account projected margin ratio helper * Bump version to v0.9.4 * fix: address account margin ratio review feedback * fix: simplify account margin ratio errors |
||
|
|
292fac899a |
Add generic trading helpers and reduce public API tiers (#58)
* feat: add generic trading helpers and API tiers * Bump version to v0.9.3 * fix: require symbol digits for trailing stops * fix: allow side-specific trailing stop ticks * test: enforce complete public export tiers * docs: align public contract tiers * refactor: remove legacy public supports |
||
|
|
9ac3b885c3 |
test: add explicit unit tests for calculate_positions_margin_by_symbol and calculate_positions_margin_safe (#50) (#53)
* test: add explicit unit tests for calculate_positions_margin_by_symbol and calculate_positions_margin_safe (#50) Covers all acceptance criteria: partial failure with warning log, all-fail, empty symbol list with no-broker-call assertion, duplicate deduplication, successful aggregation with first-seen key order, suppress_errors=False propagation, and three calculate_positions_margin_safe cases (partial skip, all-fail → 0.0, empty list → 0.0). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test: fix warning log assertion and parametrize suppress_errors=False test - Use record.getMessage() + levelno check instead of record.message, which is only populated after formatting and can return an empty string. - Parametrize test_one_symbol_fails_suppress_errors_false over all three exception types caught by the implementation (Mt5TradingError, Mt5RuntimeError, AttributeError) so any future narrowing of the except tuple would be caught by tests. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style: shorten docstring to fit 88-char line limit * style: shorten docstring to fit 88-char line limit --------- Co-authored-by: agent <agent@localhost> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
1c57be5c44 |
fix: centralize tick price validation in calculate_spread_ratio and determine_order_limits (#52)
Replaces manual isinstance/<=0 checks in calculate_spread_ratio() and determine_order_limits() with _valid_tick_price(), ensuring NaN, inf, -inf, zero, negative, bool, and invalid-string tick values are consistently rejected across all trading helpers. Adds regression tests covering numeric-string acceptance and every invalid-value category for both functions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
d292fbb9d9 |
feat: centralize tick price validation and add resilient position margin helpers (#49, #50)
Add _valid_tick_price() internal helper that returns a positive finite float from a tick dict or None for any invalid value (missing, None, NaN, infinite, zero, negative, or unsupported type). Refactor five existing bid/ask validation sites in trading.py to use it, removing duplicated isinstance/isfinite checks. Add calculate_positions_margin_by_symbol() which computes margin per unique symbol independently using the existing strict calculate_positions_margin(), with first-seen deduplication and configurable error suppression (Mt5TradingError, Mt5RuntimeError, AttributeError) via suppress_errors=. Add calculate_positions_margin_safe() as a thin sum wrapper with suppress_errors=True, returning 0.0 on empty or fully-failed inputs. Both new helpers are exported from mt5cli, added to STABLE_SDK_EXPORTS, and documented in docs/api/public-contract.md. Existing strict behavior of calculate_positions_margin() is unchanged. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
8e53212a24 | fix: always use mt5cli calculate_volume_by_margin to prevent LACK OF FUNDS (#48) | ||
|
|
b878a61c07 |
fix: re-verify normalized volume margin in calculate_volume_by_margin (#46)
* fix: re-verify normalized volume margin before returning from calculate_volume_by_margin For CFDs, index products, and tiered-margin instruments, the initial min-lot margin estimate can be optimistic; the normalized stepped volume may require more margin than available_margin. After computing the normalized volume, step down by volume_step until order_calc_margin confirms affordability, or return 0.0 if no step is affordable. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test: fix ruff line-length violations in calculate_volume_by_margin tests Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: use integer step index and add actual>0 guard in calculate_volume_by_margin Replace float-subtraction loop with integer step index to eliminate accumulation rounding error and add `actual > 0` guard so a broker returning zero/negative margin is never accepted as affordable. Inline `capped` to keep local-variable count within Ruff PLR0914 limit. Update docstring to reflect re-verification behaviour and 0.0 fallback. Tighten test assertion from `volume > 0` to the symbol's valid range. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Bump version to v0.9.1 * perf: replace linear step-down scan with binary search in calculate_volume_by_margin Resolves the P2 review finding: the previous O(n) loop called order_calc_margin once per volume step, making sizing appear hung for symbols with a large step range or small volume_step. Binary search over the integer step index finds the largest affordable step in O(log n) IPC calls (≈17 for a 99 999-step range vs up to 99 999 in the worst case). Monotonicity of broker margin with volume is assumed, which holds for standard linear margin schedules. To stay within the PLR0914 local-variable limit the steps variable is inlined into hi and the tick temporary is eliminated by accessing the snapshot dict directly. Error messages still go via msg to satisfy EM102. Two existing tests are updated to match the binary-search call sequence. A new regression test (volume_min=0.01, volume_max=1000.0) configures a tiered-margin mock with its threshold at step 50000 and asserts that the total order_calc_margin call count does not exceed 25. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: remove obsolete TC003 per-file-ignore for history.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: agent <agent@localhost> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
0610ea732c | fix: handle NumPy object rate timestamps | ||
|
|
82a39731ed |
feat: add fetch_latest_closed_rates_indexed and allow_whole_dollar_env opt-in (#45)
* feat: add fetch_latest_closed_rates_indexed and allow_whole_dollar_env opt-in (#43, #44) Closes #43: add fetch_latest_closed_rates_indexed(client, *, symbol, granularity, count) -> pd.DataFrame to mt5cli/trading.py. Internally reuses fetch_latest_closed_rates_for_trading_client(), converts the "time" column to a UTC-aware DatetimeIndex named "time", and drops the original column. Exported from trading.__all__, mt5cli.__init__, and STABLE_SDK_EXPORTS. Closes #44: extend substitute_env_placeholders() with opt-in allow_whole_dollar_env=False that expands whole-value $ENV_NAME strings (entire string must be exactly $IDENTIFIER). Threaded through build_config(), resolve_account_spec(), and resolve_account_specs() with the same default=False. Partial strings like "plan$pass", "abc$ENV", or "$ENV-suffix" are never expanded. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: align Markdown table columns in docs and skill file Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: treat numeric (float64) epoch seconds as UTC in _rate_time_to_utc After DataFrame concat or NA upcast the time column becomes float64, which is still epoch seconds. Using is_numeric_dtype instead of is_integer_dtype fixes the silent misalignment. Using series.to_numpy() before passing to pd.to_datetime avoids the redundant pd.DatetimeIndex() wrapper and aligns with how existing rate-time normalization in schemas.py handles numeric timestamps. Add test_converts_float_epoch_seconds_to_utc_datetime_index to cover the regression. Add a doc note clarifying that build_config cannot expand login since that parameter is int | None. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: reject NaT values after rate timestamp conversion in _rate_time_to_utc pd.to_datetime() silently produces NaT for None/NaN inputs rather than raising, so the function could return a DatetimeIndex containing NaT despite documenting invalid timestamps as a ValueError. Check any(idx.isna()) after conversion and raise with a clear message. Add test_raises_on_nat_time_column to cover the regression. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Bump version to v0.9.0 * fix: handle object numeric rate timestamps --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
c4a4253fbc |
feat: stable SDK helpers for volume, margin, and closed bars (#39–#41) (#42)
* feat: add stable SDK helpers for volume, margin, and closed bars (#39, #40, #41) Expose generic trading utilities in the stable downstream SDK so applications like mteor can drop local MT5 adapter code: - normalize_order_volume() for broker step/min/max sizing - estimate_order_margin() and calculate_positions_margin() for margin totals - fetch_latest_closed_rates_for_trading_client() for closed bars from Mt5TradingClient Update STABLE_SDK_EXPORTS, package-root exports, docs, and unit tests. Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * chore: bump version to 0.8.3 Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * fix: address PR review feedback on volume cap, rate time, and margin grouping - Re-apply volume_max after step normalization in normalize_order_volume() - Drop misleading non-time index reset branch in _ensure_rate_time_column() - Group positions by (symbol, side) before margin estimation - Add branch-coverage tests for tick price validation and volume cap edge case Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * fix: address remaining PR review threads on docs and DatetimeIndex - Rename unnamed DatetimeIndex column to time after reset_index() - Guard estimate_order_margin example on positive normalized volume - Document calculate_positions_margin skip vs error propagation behavior - Add test for unnamed DatetimeIndex branch coverage Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * fix: harden stable SDK margin, rate fetch, and volume normalization - Wrap order_calc_margin conversion and reject None/non-numeric results - Validate fetched rate objects are DataFrames before time normalization - Return 0.0 for non-finite volume inputs and constraints in normalize_order_volume Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * fix: reject non-finite volumes in margin estimation helpers Use _is_positive_finite_number() in estimate_order_margin() and calculate_positions_margin() so NaN/inf volumes never reach broker calls. Add focused tests and document non-finite volume skipping in trading.md. Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * fix: guard symbol filter in calculate_positions_margin for empty frames Return 0.0 before filtering when positions are empty or lack a symbol column. Add regression tests for filtered calls on malformed position frames. Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> |
||
|
|
7de3ce0b7a |
feat: add injectable update_backend to ThrottledHistoryUpdater (#38)
* feat: add injectable update_backend to ThrottledHistoryUpdater Allow downstream applications to substitute the history update backend via the ThrottledHistoryUpdater constructor without monkey-patching mt5cli.sdk.update_history. Defaults to update_history for backward compatibility. Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * chore: fix lint and format after QA Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * chore: bump version to 0.8.2 Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * fix: use explicit None check for ThrottledHistoryUpdater backend Only None selects the default update_history backend so falsy callable objects with __bool__ returning False are preserved as custom backends. Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> |
||
|
|
897f7f0a0d | docs: stable SDK contract and strategy-neutral order helpers (#37) | ||
|
|
d156dd7176 |
[codex] fix mt5 adapter APIs (#36)
* fix mt5 adapter APIs * address PR feedback * fix zero ratio minimum volume sizing * Bump version to v0.8.0 |
||
|
|
78c49238cf |
feat: stable MT5Client public API and infrastructure layer (#30)
* feat: add stable MT5Client public API and infrastructure layer Introduce a reusable public API for downstream trading applications: - MT5Client as the primary client abstraction with order_check/order_send - schemas module with DataKind contracts, validation, and normalization - converters, exceptions, retry, and storage facade modules - CLI order commands now route through MT5Client - connected_client made public; retry logic centralized - Contract tests for API surface, schemas, and storage round-trips - README and docs updated with Python API usage examples Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * fix: correct time coercion, broker-safe symbols, and execution docs - Normalize MT5 time columns with correct second/millisecond units - Coerce all present known MT5 time fields, including optional order times - Preserve broker symbol casing in normalize_symbol() - Document order_send() as a live execution primitive with clear scope boundaries - Add contract tests for timestamp and symbol normalization behavior Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> |
||
|
|
9356d5dcdf |
Consolidate duplicated export and history streaming helpers (#29)
* Consolidate duplicated export and history streaming helpers. Reduce repeated CLI export plumbing, shared per-symbol SQLite writes, and test mock setup without changing public behavior. Co-authored-by: Cursor <cursoragent@cursor.com> * Bump version from 0.7.0 to 0.7.1. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
0fad55d609 |
Refactor MT5 constant parsing to delegate to pdmt5 >= 0.3.0 (#28)
* Refactor MT5 constant parsing to delegate to pdmt5 >= 0.3.0 Replace local TIMEFRAME_MAP, TICK_FLAG_MAP, and parser helpers with thin compatibility wrappers around pdmt5. COPY_TICKS flags now use real MT5 values (ALL=-1, INFO=1, TRADE=2). Click parameter types validate all inputs through the wrappers. Update tests and docs to describe the pdmt5/mt5cli/mt5api layering. Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * Fix timeframe defaults and COPY_TICKS flag defaults after pdmt5 migration Use short timeframe aliases for default history collection and granularity naming via pdmt5.get_timeframe_name. Set CLI/SDK default tick flags to ALL (-1) instead of the legacy mt5cli-only value. Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * Address CI lint failure and PR review feedback Fix ruff import ordering in history.py. Use ALL string defaults for CLI tick flags, isolate TICK_FLAG_MAP as a dict snapshot, derive flag names from pdmt5, reuse TIMEFRAME_NAMES for default history timeframes, and add tests for prefix stripping and TIMEFRAME_ key filtering. Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * Bump version to 0.7.0 Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> |
||
|
|
b5e82e71c7 |
Add trading session helpers and extend ThrottledHistoryUpdater (#25)
* Add trading session helpers and extend ThrottledHistoryUpdater Introduce mt5cli.trading with mt5_trading_session() for Mt5TradingClient lifecycle management and reusable operational helpers for position-side detection, margin/volume sizing, and protective order price derivation. Extend ThrottledHistoryUpdater to validate inputs before updates and to optionally suppress ValueError, OSError, and missing-method errors without advancing the throttle timestamp. Export the new helpers from mt5cli.__init__, add unit tests with mocked clients, and document migration guidance for downstream projects such as mteor. Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * Narrow ThrottledHistoryUpdater suppress_errors handling (#27) * Narrow ThrottledHistoryUpdater suppress_errors for MT5 capability only Remove broad AttributeError/TypeError handling from recoverable errors. Add _is_mt5_client_capability_error() to detect missing history API methods or non-callable client attributes by message and attribute name. Generic AttributeError/TypeError values always propagate even when suppress_errors=True. Update docs and tests accordingly. Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * Detect non-callable history client methods in suppress_errors Address review feedback: when a history API attribute exists but is not callable, Python raises a generic TypeError. Inspect the traceback for mt5cli.history client call sites so these capability mismatches are still suppressed without matching all TypeError values. Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * Address PR review feedback on trading helpers - Resolve history module path once at import time - Only treat non-callable TypeErrors as capability errors at the raise site - Validate SL/TP ratios in determine_order_limits - Add tests for margin_free edge cases, body-raise shutdown, and internal TypeError propagation - Clarify ThrottledHistoryUpdater suppress_errors docs - Split README migration example into trading vs read-only history sessions Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * Tighten protective ratio validation and clamp negative margin_free Add _require_protective_ratio enforcing 0 <= ratio < 1 for SL/TP limits so a ratio of 1.0 cannot produce zero protective prices. Clamp negative margin_free to 0.0 in calculate_margin_and_volume before sizing. Add boundary and negative-margin tests; document constraints in trading API docs. Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> |
||
|
|
18df96872b |
Add closed-bar rate helpers (v0.6.0) (#26)
* Add closed-bar rate helpers and bump version to 0.6.0. Expose drop_forming_rate_bar and multi-account collectors so downstream apps no longer need count+1 fetches and manual bar trimming. Co-authored-by: Cursor <cursoragent@cursor.com> * Bump pygments to 2.20.0 to fix CVE-2026-4539 ReDoS advisory. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback on closed-bar rate collection. Validate count and start_pos before MT5 fetches, avoid redundant frame copies, clarify empty-series errors, and expand test coverage. Co-authored-by: Cursor <cursoragent@cursor.com> * Include symbol and timeframe in empty closed-rate error messages. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
5b1d54bfe9 |
Add resilient multi-account orchestration helpers (#22)
* Add SDK orchestration helpers for resilient multi-account collection
- collect_latest_rates_for_accounts_with_retries(): exponential-backoff
retries around collect_latest_rates_for_accounts(), retrying only
Mt5TradingError/Mt5RuntimeError and re-raising on exhaustion.
- resolve_account_spec()/resolve_account_specs() and
substitute_env_placeholders(): merge explicit overrides over AccountSpec
fields and expand ${ENV_VAR} placeholders, raising ValueError on missing
variables.
- ThrottledHistoryUpdater: monotonic-clock throttled wrapper around
update_history() with should_update()/update() and opt-in suppress_errors.
- load_rate_series_by_granularity(): rate-series loader keyed by
(symbol | None, granularity_name).
- Export new APIs, add unit tests (100% coverage), and document in README
and docs/api.
* chore: bump version from 0.5.1 to 0.5.3 (#24)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* fix: resolve leftover merge conflict markers in version files
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
* fix: address PR review feedback on SDK orchestration helpers
- Use single-pass env substitution to avoid TOCTOU KeyError
- Apply backoff_base to all retry delays (backoff_base ** (attempt + 1))
- Preserve integer logins in resolve_account_spec; hide login in repr
- Fix docs examples (env ordering, while True loop, backoff comment)
- Parametrize suppress_errors tests for MT5 and SQLite errors
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
|
||
|
|
ad9e513253 |
[codex] Guard dedup scopes by written columns (#23)
* Guard dedup scopes by written columns * Address dedup scope review feedback * Remove legacy dedup scope support * Remove stale legacy descriptions * chore: bump version from 0.5.1 to 0.5.2 |
||
|
|
1b69e8f08e | Add generic MT5 rate-loading SDK APIs for downstream reuse (#20) | ||
|
|
9957b0a1de |
[codex] Add generic MT5 SDK and SQLite rate loader (#19)
* Add generic MT5 SDK and SQLite rate loader * Fix MT5 latest rates connection reuse * Make MT5 summary export safe * Address PR review feedback for SDK and SQLite rate loader. Reuse parse_sqlite_timestamp for rate time parsing, document empty-table errors, tighten tests, and align docs with require_existing=True. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
b2bb2ad0a0 |
Add rate view resolution and downstream SDK helpers (#18)
* Add public helpers to resolve rate compatibility view names. Expose resolve_rate_view_name and resolve_rate_view_names in mt5cli.history so consumers can derive mt5cli-managed SQLite view names from stored rates metadata without reimplementing the naming rules. Co-authored-by: Cursor <cursoragent@cursor.com> * Add reusable export, tick-window, and margin helpers for downstream tools. Expose SQLite append/dedup export, recent tick retrieval, and minimum margin summary through the SDK and CLI so projects like mteor can depend on mt5cli instead of duplicating MT5 data plumbing. Co-authored-by: Cursor <cursoragent@cursor.com> * Bump version to 0.4.3. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback for rate view resolution and SDK helpers. Harden SQLite read-only connections, tighten view discovery, improve recent_ticks fetch efficiency, default SQLite export to append, and expand tests and docs. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix read-only SQLite URI construction on Windows. Use Path.as_uri() so encoded file URIs work cross-platform with mode=ro. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
756faf747b |
Rename sqlite_history module to history (#17)
* Rename sqlite_history module to history. Drop the sqlite-specific prefix now that history collection is the primary module name across SDK, tests, and docs. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback for history module rename. Add a sqlite_history compatibility shim, clarify docs naming, and align the module docstring with the collect-history SQLite scope. Co-authored-by: Cursor <cursoragent@cursor.com> * Remove sqlite_history compatibility shim. The rename to mt5cli.history is intentionally breaking; downstream code should update imports rather than rely on a deprecated re-export path. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
c4232bf44d |
Add incremental SQLite history SDK (#16)
* Add incremental SQLite history SDK for automated pipelines. Extract sqlite history helpers into a dedicated module and expose update_history APIs that resume from existing MAX(time) values instead of re-fetching fixed date ranges. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix incremental history deals and stale rate view cleanup. Fetch account events once during incremental updates, drop stale rate_* views when timeframes change, and avoid SQLite variable limits on wide frames. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix incremental deal filtering edge cases Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR review feedback for incremental SQLite history. Make rate views collision-free, batch incremental resume queries, scope deduplication to appended boundaries, validate before opening MT5, use atomic SQLite transactions, and expand docs/tests for the new helpers. Co-authored-by: Cursor <cursoragent@cursor.com> * Document collect-history SQLite schema with ER diagram. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix account-event filtering and drop legacy rates resume. Account events must follow only account_event_start, not per-symbol trade cursors. Require normalized rates schema and fail fast when timeframe is missing. Co-authored-by: Cursor <cursoragent@cursor.com> * Validate normalized rates schema before incremental resume. Require symbol, timeframe, and time on existing rates tables with clear ValueError messages, and add regression tests for malformed schemas. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
5b44318d55 |
Add programmatic SDK and refactor mt5cli into cli, sdk, and utils (#15)
* Refactor cli.py into cli and utils modules Extract constants, enums, Click parameter types, and parse/export utility functions into a new mt5cli/utils.py module, keeping the typer app, commands, and collect-history SQLite helpers in cli.py. https://claude.ai/code/session_016JwSEhPyq6phXySktQ1FGU * Address review comments * Add programmatic SDK layer for read-only MT5 data collection. Expose Mt5CliClient and collect_history through the package API while keeping CLI commands as thin adapters over the SDK. Co-authored-by: Cursor <cursoragent@cursor.com> * Harden SDK connection lifecycle and scope internal helpers as private. Co-authored-by: Cursor <cursoragent@cursor.com> * Export build_config in the public API and bump version to 0.4.0. Co-authored-by: Cursor <cursoragent@cursor.com> * Remove duplicate scripts/ in favor of local-qa skill script. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
da74c11087 |
Add collect-history command for bulk data collection (#14)
* Add collect-history command for bulk SQLite export Bundles rates, ticks, history-orders, and history-deals for one or more symbols into a single SQLite database. Tick collection uses copy_ticks_range_as_df with a --flags option defaulting to ALL. With --with-views, optional cash_events and positions_reconstructed views are derived from history_deals when the required columns are present. * Extend collect-history with datasets, if-exists, timeframe, view fixes - Fetch history-orders and history-deals per symbol so --symbol applies consistently across all four datasets. - Add repeatable --dataset (rates, ticks, history-orders, history-deals) so ticks are no longer required and any subset can be collected. - Add --if-exists append|replace|fail to control SQLite table conflict behavior instead of hard-coding replace. - Record the requested timeframe in a timeframe column on the rates table so appended runs at different timeframes stay distinguishable. - Fix positions_reconstructed to exclude positions with no closing deals, use volume-weighted open/close prices, and report reversal deals (DEAL_ENTRY_INOUT) via volume_reversal / reversal_count without contributing to weighted prices. - Update tests, README, docs, and skill to match. * Address collect-history review feedback * Stream collect-history writes per symbol * Address PR cleanup for collect-history --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
b8ce76c9d6 |
Add CLI commands for remaining pdmt5.dataframe methods (#9)
* Add CLI commands for remaining pdmt5.dataframe methods Add the following subcommands so the CLI fully covers pdmt5.dataframe's public *_as_df methods: - version - last-error - symbol-info-tick - market-book - order-check (request via inline JSON or @path/to/file.json) - order-send (request via inline JSON or @path/to/file.json) Also export a new parse_request helper for parsing JSON order requests, add tests for every new command, and update the README and docs command tables. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Bump version to 0.2.0 Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> * Address PR review feedback Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Harden CLI error assertions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com> |
||
|
|
5e2b71977d |
Fix QA issues: add pyarrow dep, docstring, type stubs, and coverage config
- Add pyarrow dependency for parquet export support - Add docstring to tests/__init__.py (ruff D104) - Disable reportMissingTypeStubs for pdmt5 in pyright config - Exclude __main__.py from coverage measurement - Format markdown tables with prettier - Add uv.lock https://claude.ai/code/session_01YW3YHru8wRH9dvHnBX7xf1 |
||
|
|
527f9213da |
Add mt5cli package with CLI for MetaTrader 5 data export
Create a standalone CLI package that uses pdmt5 as a dependency to export MetaTrader 5 data to CSV, JSON, Parquet, and SQLite3 formats. Includes 12 subcommands for rates, ticks, account/terminal info, symbols, orders, positions, and trading history, along with comprehensive test coverage. https://claude.ai/code/session_01YW3YHru8wRH9dvHnBX7xf1 |