Compare commits

...

30 Commits

Author SHA1 Message Date
Daichi Narushima 43f632bc40 Reorganize CLI help text and command grouping for data/execution clarity (#85)
* feat: clarify CLI/docs scope as generic MT5 data and execution infrastructure

- Update app help text and module docstring to describe mt5cli as MT5 data
  and execution utilities rather than export-only tooling
- Group CLI commands under rich_help_panel sections: Data / Export, Execution,
  and Collection; command names are unchanged for compatibility
- Expand order-send docstring to explicitly flag it as the expert raw-request
  live-trading path; preserve --yes gate
- Split docs/index.md Trading section into "Trading State" (read-only) and
  "Execution (live / mutating)" with close-positions now documented
- Add TestHelpText tests verifying top-level panel grouping, order-send
  expert/live language, and close-positions safety gate coverage

Closes #78

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018yH6esaqc5D1cmo1dK2Ur9

* chore: trim trailing whitespace in docs/index.md table

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018yH6esaqc5D1cmo1dK2Ur9

* chore: bump version to 1.0.1

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018yH6esaqc5D1cmo1dK2Ur9

* chore: update uv.lock for version 1.0.1

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018yH6esaqc5D1cmo1dK2Ur9

* fix: address review feedback on CLI/docs scope PR

- Remove dead help invocation in test_order_send_help_mentions_expert_and_raw
  (the result was immediately overwritten by result2)
- Strengthen assertion from `or` to `and`; both "raw" and "expert" are present
  in the docstring so disjunction masked a potential regression
- Split into two `assert` statements to satisfy PT018 (ruff)
- Fix docs/index.md inaccuracy: order-check has no --yes gate; clarify that
  only order-send and close-positions require confirmation for live execution
- Move order-check from "Execution" rich_help_panel to "Data / Export" so the
  Execution panel name is truthful (order-check is read-only)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018yH6esaqc5D1cmo1dK2Ur9

* docs: move order-check out of Execution section into Trading State

order-check is read-only and now lives in the CLI's Data / Export panel,
so documenting it under "Execution (live / mutating)" was inconsistent.
Moved it to the Trading State table. The Execution section now only lists
order-send and close-positions, both of which require --yes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018yH6esaqc5D1cmo1dK2Ur9

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-28 01:23:17 +09:00
dceoy 8028263b24 docs: format public contract table
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-27 02:04:29 +09:00
dceoy 63a8d67419 chore: bump version to 1.0.0
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-27 02:03:46 +09:00
Daichi Narushima 93565681e1 fix: decouple mt5cli from pdmt5 high-level trading helpers (#76)
* fix: decouple mt5cli from pdmt5 high-level trading helpers

- Replace Mt5TradingClient type annotations with internal _Mt5ClientProtocol
- Lazy-import Mt5TradingClient in create_trading_client to avoid hard dependency
- Replace Mt5TradingError with Mt5OperationError in mt5cli validation paths
- Update exception handling to support future pdmt5 versions without Mt5TradingError
- Add test to enforce that mt5cli doesn't import high-level symbols at module level
- Update documentation to clarify dependency boundaries

mt5cli now relies only on low-level MT5 primitives:
- Mt5Config for configuration
- Mt5RuntimeError for runtime errors
- Raw MT5 methods (order_send, order_check, account_info, etc.)

This aligns with pdmt5's direction to remove high-level trading helpers and focus
on low-level MT5 access plus DataFrame/dict conversion.

Fixes #75 (dceoy/mt5cli#75)

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PcGVFTVgyqzse3LLw38ber

* fix: address PR #76 review feedback on pdmt5 decoupling

- Replace Mt5TradingClient with Mt5DataClient in create_trading_client()
  so the function no longer depends on the high-level trading client
- Fix _RECOVERABLE_MT5_ERRORS in exceptions.py to use tuple unpacking
  form, removing the incorrect ternary assignment
- Add pragma: no cover to except ImportError branches in exceptions.py
  and sdk.py (dead code when pdmt5 is installed)
- Switch coverage exclude_lines to exclude_also so the default
  pragma: no cover pattern is preserved; also exclude bare ... stubs
  (Protocol method bodies) from coverage
- Correct inaccurate note in docs/api/public-contract.md: Mt5TradingClient
  is no longer required internally; Mt5TradingError is conditionally
  available but mt5cli raises Mt5OperationError for trading failures
- Update all mock patches from pdmt5.Mt5TradingClient to
  mt5cli.trading.Mt5DataClient to match the new module-level import

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-26 22:26:54 +09:00
Daichi Narushima f435544f07 Shrink public API surface and remove storage re-export module (#74) 2026-06-26 18:23:30 +09:00
Daichi Narushima 668f38d8aa feat: reduce package-root API surface and require pdmt5>=1.0.0 (closes #70) (#73) 2026-06-26 12:08:00 +09:00
dceoy 8da5ee9242 Bump version to v0.9.7
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-25 14:10:23 +09:00
Daichi Narushima 9dbb46fbb1 feat: make pyarrow optional via mt5cli[parquet] extra (#69) 2026-06-25 14:03:24 +09:00
Daichi Narushima dfe80ce500 feat: add close-positions CLI and replace_symbol projection mode (#65 #66) (#67)
* feat: add close-positions CLI command and replace_symbol projection mode (#65 #66)

Part 1 — close-positions CLI (#65):
- Add `close-positions` subcommand delegating to `close_open_positions()`.
- Accepts repeated `--symbol` and `--ticket` filters (AND semantics).
- Supports `--dry-run` (no `--yes` required); live execution requires `--yes`.
- Fails closed with `BadParameter` when neither `--symbol` nor `--ticket` is given.
- Exports normalized `OrderExecutionResult` list as a DataFrame (request/response
  serialized as JSON strings for clean CSV/JSON/Parquet/SQLite output).
- `order-send` remains the raw expert path; `close-positions` is the safer
  high-level helper that builds correct close requests automatically.

Part 2 — ProjectionMode and replace_symbol (#66):
- Add `ProjectionMode = Literal["add", "replace_symbol"]` type alias.
- Add optional `projection_mode` parameter to `calculate_symbol_group_margin_ratio`.
  Default `"add"` preserves existing additive behavior.
  `"replace_symbol"` subtracts current margin for `new_symbol`, then adds
  candidate margin — the subtraction and addition are atomic (suppressed together).
- Export `ProjectionMode` from `mt5cli` and add to `STABLE_SDK_EXPORTS`.
- No mteor-specific strategy, risk-threshold, or policy logic added.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: remove unused ProjectionMode import in test_contracts.py

The parametrized test_stable_exports_are_importable_from_package_root
already covers ProjectionMode via hasattr(mt5cli, name). Ruff correctly
flagged the explicit top-level import as unused (F401).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Bump version to v0.9.6

* fix: address PR #67 review feedback

- Floor replace_symbol margin subtraction at zero to prevent negative ratio
- Serialize response unconditionally via json.dumps (null for dry-run rows)
- Return a schema-preserving empty DataFrame when results list is empty
- Add test: --dry-run --yes precedence (dry-run wins, no order_send)
- Add test: zero-match filter produces empty JSON array with exit 0
- Move projection_mode prose to stable trading section in docs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: add runtime validation for projection_mode in calculate_symbol_group_margin_ratio

Unsupported values previously silently fell through as "add". The new
_validate_projection_mode helper raises ValueError with a message that
names the bad value and the two accepted modes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: agent <agent@localhost>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-25 10:39:49 +09:00
Daichi Narushima 15bfd17db3 test: reduce test_trading.py duplication with parametrize (#64)
* test: reduce test_trading.py duplication with parametrize

Collapse repetitive individual tests in test_trading.py into
parametrized equivalents, cutting 267 lines without losing any cases.

- TestExtractTickPrice: 13 tests → 2 parametrized (×3 valid, ×10 None)
- TestEstimateOrderMargin: 4 invalid-margin tests → 1 parametrized ×4;
  nan/inf volume tests → 1 parametrized ×2
- TestNormalizeOrderVolume: multi-assert bodies split into parametrized
  cases for non-finite volume and constraints
- TestVolumeAndExecution: 9 place_market_order retcode tests → 1 ×11;
  5 update_sltp retcode tests → 1 ×5
- test_calculate_trailing_stop_updates_missing_symbol_digits:
  inline double-assert body → 1 parametrized ×2

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: further reduce test_trading.py duplication with parametrize

Merge six broker stop-level tests into two parametrized tests, collapse
two default-digits fallback tests and three symbol-filter zero-margin
tests into one each.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: address claude[bot] review on PR #64

- Consolidate _MISSING_RETCODE sentinel to one line with corrected comment
- Add comment explaining ids list is required for deterministic node IDs
- Document intentional narrower retcode coverage in update_sltp test

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: reduce duplication in test_sdk, test_history, test_contracts

- TestBuildConfigWholeDollarEnv: 3 field tests (server/password/path) → 1
  parametrized ×3
- TestResolveAccountSpec: whole-dollar expand/no-expand pair → 1 parametrized ×2
- test_normalize_mt5_exception_maps_types: 2 isinstance asserts → parametrized ×2
- test_resolve_history_tick_flags_invalid: 2 pytest.raises blocks → parametrized ×2

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: agent <agent@localhost>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-25 01:55:04 +09:00
Daichi Narushima 37eef16e99 feat: support string login in build_config and add substitute_mapping_values (#63)
* feat: support string login in build_config and add substitute_mapping_values (#61, #62)

Extend build_config() to accept login: int | str | None. String logins
are coerced via the existing coerce_login() helper (empty/whitespace →
None, numeric strings → int, non-numeric → ValueError). When
allow_whole_dollar_env=True, ${ENV} and $ENV placeholders are expanded
before coercion, consistent with path/password/server behavior.

Add substitute_mapping_values(), a generic recursive helper that
substitutes environment placeholders in nested dicts/lists only for
caller-selected mapping keys. Non-selected fields (including literal
dollar signs) are preserved exactly. Supports blank_string_keys_as_none
to normalise empty strings to None after substitution. No application-
specific key names (e.g. mt5_login) are hard-coded in mt5cli.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Bump version to v0.9.5

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs+test: clarify substitute_mapping_values docstring and pin tuple behaviour

- Adds sentence noting list-element strings are never substituted (only
  immediate dict values are), addressing reviewer finding #1.
- Rewrites Returns section to accurately describe scalar pass-through
  behaviour, addressing reviewer finding #2.
- Adds recursion-depth caveat to the generic-utility docstring,
  addressing reviewer finding #4.
- Adds test_tuple_container_not_traversed to pin the existing silent
  tuple-exclusion contract, addressing reviewer finding #3.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs: update public contract and README for build_config login coercion and substitute_mapping_values

- Expands build_config row to document login: int | str | None,
  numeric-string coercion, blank-string handling, and env placeholder
  expansion when allow_whole_dollar_env=True.
- Adds substitute_mapping_values to the stable SDK table with a note
  that key names are never hard-coded in mt5cli.
- Extends allow_whole_dollar_env paragraph to list substitute_mapping_values.
- README: adds build_config env-placeholder example and imports to the
  trading lifecycle snippet.
- README: extends credential-resolution bullet with a substitute_mapping_values
  usage example using generic key names.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: agent <agent@localhost>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-25 01:04:32 +09:00
Daichi Narushima 96c75f7852 Add account-wide projected margin ratio helper (#60)
* feat: add account projected margin ratio helper

* Bump version to v0.9.4

* fix: address account margin ratio review feedback

* fix: simplify account margin ratio errors
2026-06-24 03:43:52 +09:00
Daichi Narushima 292fac899a Add generic trading helpers and reduce public API tiers (#58)
* feat: add generic trading helpers and API tiers

* Bump version to v0.9.3

* fix: require symbol digits for trailing stops

* fix: allow side-specific trailing stop ticks

* test: enforce complete public export tiers

* docs: align public contract tiers

* refactor: remove legacy public supports
2026-06-24 01:58:32 +09:00
Daichi Narushima 9ac3b885c3 test: add explicit unit tests for calculate_positions_margin_by_symbol and calculate_positions_margin_safe (#50) (#53)
* test: add explicit unit tests for calculate_positions_margin_by_symbol and calculate_positions_margin_safe (#50)

Covers all acceptance criteria: partial failure with warning log, all-fail,
empty symbol list with no-broker-call assertion, duplicate deduplication,
successful aggregation with first-seen key order, suppress_errors=False
propagation, and three calculate_positions_margin_safe cases (partial skip,
all-fail → 0.0, empty list → 0.0).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: fix warning log assertion and parametrize suppress_errors=False test

- Use record.getMessage() + levelno check instead of record.message, which
  is only populated after formatting and can return an empty string.
- Parametrize test_one_symbol_fails_suppress_errors_false over all three
  exception types caught by the implementation (Mt5TradingError,
  Mt5RuntimeError, AttributeError) so any future narrowing of the except
  tuple would be caught by tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* style: shorten docstring to fit 88-char line limit

* style: shorten docstring to fit 88-char line limit

---------

Co-authored-by: agent <agent@localhost>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-23 21:41:30 +09:00
Daichi Narushima 823cb5b0a4 Revert "Bump version to v0.9.3 (#51)" (#52)
This reverts commit f1ada55bce.
2026-06-23 19:17:42 +09:00
agent 1c57be5c44 fix: centralize tick price validation in calculate_spread_ratio and determine_order_limits (#52)
Replaces manual isinstance/<=0 checks in calculate_spread_ratio() and
determine_order_limits() with _valid_tick_price(), ensuring NaN, inf,
-inf, zero, negative, bool, and invalid-string tick values are
consistently rejected across all trading helpers.

Adds regression tests covering numeric-string acceptance and every
invalid-value category for both functions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-23 09:48:13 +00:00
Daichi Narushima f1ada55bce Bump version to v0.9.3 (#51) 2026-06-23 18:30:18 +09:00
agent d292fbb9d9 feat: centralize tick price validation and add resilient position margin helpers (#49, #50)
Add _valid_tick_price() internal helper that returns a positive finite float
from a tick dict or None for any invalid value (missing, None, NaN, infinite,
zero, negative, or unsupported type). Refactor five existing bid/ask validation
sites in trading.py to use it, removing duplicated isinstance/isfinite checks.

Add calculate_positions_margin_by_symbol() which computes margin per unique
symbol independently using the existing strict calculate_positions_margin(),
with first-seen deduplication and configurable error suppression
(Mt5TradingError, Mt5RuntimeError, AttributeError) via suppress_errors=.

Add calculate_positions_margin_safe() as a thin sum wrapper with
suppress_errors=True, returning 0.0 on empty or fully-failed inputs.

Both new helpers are exported from mt5cli, added to STABLE_SDK_EXPORTS, and
documented in docs/api/public-contract.md. Existing strict behavior of
calculate_positions_margin() is unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-23 07:30:02 +00:00
Daichi Narushima 8e53212a24 fix: always use mt5cli calculate_volume_by_margin to prevent LACK OF FUNDS (#48) 2026-06-23 14:01:46 +09:00
Daichi Narushima b878a61c07 fix: re-verify normalized volume margin in calculate_volume_by_margin (#46)
* fix: re-verify normalized volume margin before returning from calculate_volume_by_margin

For CFDs, index products, and tiered-margin instruments, the initial
min-lot margin estimate can be optimistic; the normalized stepped volume
may require more margin than available_margin.  After computing the
normalized volume, step down by volume_step until order_calc_margin
confirms affordability, or return 0.0 if no step is affordable.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: fix ruff line-length violations in calculate_volume_by_margin tests

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: use integer step index and add actual>0 guard in calculate_volume_by_margin

Replace float-subtraction loop with integer step index to eliminate
accumulation rounding error and add `actual > 0` guard so a broker
returning zero/negative margin is never accepted as affordable.
Inline `capped` to keep local-variable count within Ruff PLR0914 limit.
Update docstring to reflect re-verification behaviour and 0.0 fallback.
Tighten test assertion from `volume > 0` to the symbol's valid range.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Bump version to v0.9.1

* perf: replace linear step-down scan with binary search in calculate_volume_by_margin

Resolves the P2 review finding: the previous O(n) loop called
order_calc_margin once per volume step, making sizing appear hung for
symbols with a large step range or small volume_step.

Binary search over the integer step index finds the largest affordable
step in O(log n) IPC calls (≈17 for a 99 999-step range vs up to 99 999
in the worst case). Monotonicity of broker margin with volume is assumed,
which holds for standard linear margin schedules.

To stay within the PLR0914 local-variable limit the steps variable is
inlined into hi and the tick temporary is eliminated by accessing the
snapshot dict directly. Error messages still go via msg to satisfy EM102.

Two existing tests are updated to match the binary-search call sequence.
A new regression test (volume_min=0.01, volume_max=1000.0) configures
a tiered-margin mock with its threshold at step 50000 and asserts that
the total order_calc_margin call count does not exceed 25.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: remove obsolete TC003 per-file-ignore for history.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: agent <agent@localhost>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-23 04:40:00 +09:00
dceoy 0610ea732c fix: handle NumPy object rate timestamps 2026-06-22 23:01:31 +09:00
Daichi Narushima 82a39731ed feat: add fetch_latest_closed_rates_indexed and allow_whole_dollar_env opt-in (#45)
* feat: add fetch_latest_closed_rates_indexed and allow_whole_dollar_env opt-in (#43, #44)

Closes #43: add fetch_latest_closed_rates_indexed(client, *, symbol,
granularity, count) -> pd.DataFrame to mt5cli/trading.py. Internally
reuses fetch_latest_closed_rates_for_trading_client(), converts the
"time" column to a UTC-aware DatetimeIndex named "time", and drops the
original column. Exported from trading.__all__, mt5cli.__init__, and
STABLE_SDK_EXPORTS.

Closes #44: extend substitute_env_placeholders() with opt-in
allow_whole_dollar_env=False that expands whole-value $ENV_NAME strings
(entire string must be exactly $IDENTIFIER). Threaded through
build_config(), resolve_account_spec(), and resolve_account_specs() with
the same default=False. Partial strings like "plan$pass", "abc$ENV", or
"$ENV-suffix" are never expanded.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: align Markdown table columns in docs and skill file

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: treat numeric (float64) epoch seconds as UTC in _rate_time_to_utc

After DataFrame concat or NA upcast the time column becomes float64, which
is still epoch seconds. Using is_numeric_dtype instead of is_integer_dtype
fixes the silent misalignment. Using series.to_numpy() before passing to
pd.to_datetime avoids the redundant pd.DatetimeIndex() wrapper and aligns
with how existing rate-time normalization in schemas.py handles numeric
timestamps.

Add test_converts_float_epoch_seconds_to_utc_datetime_index to cover the
regression. Add a doc note clarifying that build_config cannot expand
login since that parameter is int | None.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: reject NaT values after rate timestamp conversion in _rate_time_to_utc

pd.to_datetime() silently produces NaT for None/NaN inputs rather than
raising, so the function could return a DatetimeIndex containing NaT
despite documenting invalid timestamps as a ValueError. Check any(idx.isna())
after conversion and raise with a clear message.

Add test_raises_on_nat_time_column to cover the regression.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Bump version to v0.9.0

* fix: handle object numeric rate timestamps

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-22 22:52:19 +09:00
Daichi Narushima c4a4253fbc feat: stable SDK helpers for volume, margin, and closed bars (#39–#41) (#42)
* feat: add stable SDK helpers for volume, margin, and closed bars (#39, #40, #41)

Expose generic trading utilities in the stable downstream SDK so applications
like mteor can drop local MT5 adapter code:

- normalize_order_volume() for broker step/min/max sizing
- estimate_order_margin() and calculate_positions_margin() for margin totals
- fetch_latest_closed_rates_for_trading_client() for closed bars from Mt5TradingClient

Update STABLE_SDK_EXPORTS, package-root exports, docs, and unit tests.

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* chore: bump version to 0.8.3

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* fix: address PR review feedback on volume cap, rate time, and margin grouping

- Re-apply volume_max after step normalization in normalize_order_volume()
- Drop misleading non-time index reset branch in _ensure_rate_time_column()
- Group positions by (symbol, side) before margin estimation
- Add branch-coverage tests for tick price validation and volume cap edge case

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* fix: address remaining PR review threads on docs and DatetimeIndex

- Rename unnamed DatetimeIndex column to time after reset_index()
- Guard estimate_order_margin example on positive normalized volume
- Document calculate_positions_margin skip vs error propagation behavior
- Add test for unnamed DatetimeIndex branch coverage

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* fix: harden stable SDK margin, rate fetch, and volume normalization

- Wrap order_calc_margin conversion and reject None/non-numeric results
- Validate fetched rate objects are DataFrames before time normalization
- Return 0.0 for non-finite volume inputs and constraints in normalize_order_volume

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* fix: reject non-finite volumes in margin estimation helpers

Use _is_positive_finite_number() in estimate_order_margin() and
calculate_positions_margin() so NaN/inf volumes never reach broker calls.
Add focused tests and document non-finite volume skipping in trading.md.

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* fix: guard symbol filter in calculate_positions_margin for empty frames

Return 0.0 before filtering when positions are empty or lack a symbol column.
Add regression tests for filtered calls on malformed position frames.

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
2026-06-19 01:02:18 +09:00
dceoy 9f2968cc98 Update .agents/skills/pr-feedback-triage/SKILL.md 2026-06-19 00:21:33 +09:00
Daichi Narushima 7de3ce0b7a feat: add injectable update_backend to ThrottledHistoryUpdater (#38)
* feat: add injectable update_backend to ThrottledHistoryUpdater

Allow downstream applications to substitute the history update backend via
the ThrottledHistoryUpdater constructor without monkey-patching
mt5cli.sdk.update_history. Defaults to update_history for backward
compatibility.

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* chore: fix lint and format after QA

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* chore: bump version to 0.8.2

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* fix: use explicit None check for ThrottledHistoryUpdater backend

Only None selects the default update_history backend so falsy callable
objects with __bool__ returning False are preserved as custom backends.

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
2026-06-18 22:58:09 +09:00
Daichi Narushima 897f7f0a0d docs: stable SDK contract and strategy-neutral order helpers (#37) 2026-06-18 19:12:11 +09:00
Daichi Narushima d156dd7176 [codex] fix mt5 adapter APIs (#36)
* fix mt5 adapter APIs

* address PR feedback

* fix zero ratio minimum volume sizing

* Bump version to v0.8.0
2026-06-15 02:47:05 +09:00
dceoy 307d6f5320 docs: restructure AGENTS.md with concise repository guidance
Align agent instructions with the streamlined project structure, QA workflow, and security notes used elsewhere in the repo.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-14 23:07:13 +09:00
Daichi Narushima 8031389a67 Add GitHub CodeQL analysis to CI workflow (#35)
* chore: add GitHub CodeQL analysis to CI workflow

Enable automated security scanning with GitHub CodeQL to detect potential vulnerabilities in Python code.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

* chore: run CodeQL analysis on pull requests

Co-authored-by: Cursor <cursoragent@cursor.com>

* Add checks and statuses read permissions for dependabot auto-merge.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-14 22:54:28 +09:00
dceoy fdf5e08d31 Add .agents/skills/pr-feedback-triage/SKILL.md 2026-06-14 21:16:52 +09:00
30 changed files with 7943 additions and 635 deletions
+201
View File
@@ -0,0 +1,201 @@
---
name: pr-feedback-triage
description: Triage pull request review comments into fixes, replies, clarification requests, or open follow-ups while respecting safe execution modes.
---
# PR Feedback Triage
Triage pull request review feedback, decide what action each thread needs, make focused fixes when allowed, and report or resolve only what is actually handled.
## When to Use
- A PR has review comments, requested changes, unresolved review threads, or bot review findings.
- The user asks to address, respond to, or resolve PR feedback.
- The user provides a PR URL/number, a branch with an associated PR, or copied comments.
Do not use this skill for a first-pass code review with no existing feedback; use a code review skill instead.
## Inputs
- Pull request URL or number, or a current branch that has an associated pull request.
- Repository checkout or platform access sufficient to inspect the PR diff and review feedback.
- Optional reviewer priorities from the user, such as "only address blocking comments" or "do not reply on the PR platform".
- Optional operating mode flags: `dry_run`, `no_push`, and `no_reply`.
If no PR or review comments are identifiable, ask for the target PR or the copied comments before proceeding.
## Modes
- `dry_run`: inspect review feedback and report the triage only. Do not edit files, run write-mode formatters, commit, push, post replies, or resolve review threads.
- `no_push`: local edits and verification are allowed, but do not push commits or otherwise update the remote branch. Report the local diff or local commits that still need to be pushed. Do not resolve threads whose resolution depends on unpushed local edits.
- `no_reply`: do not post replies, submit reviews, or resolve review threads. Provide suggested replies and resolution actions in the final report instead.
When a mode disables an action, skip that destructive or externally visible action even if normal workflow text would otherwise allow it.
## Preflight
1. Identify the current branch and target PR.
2. Check tracked local changes with `git diff --name-only` and `git diff --cached --name-only`. Ignore untracked files unless the review feedback explicitly concerns them.
3. Check unpushed commits before relying on remote review feedback.
4. If tracked local changes or unpushed commits exist, warn that existing PR comments may not cover the latest local state. In `normal` mode, push only when the user request or repository workflow allows it; otherwise continue with a clearly reported limitation.
## Feedback Collection
Gather the complete feedback set before editing:
- Fetch unresolved review threads, requested-change reviews, PR-level summary comments, and copied comments.
- Use platform-native APIs/CLI when available. Paginate results; do not inspect only the first page of threads or comments.
- For bot reviewers that post both summary comments and inline comments, collect both. Summary comments often contain severity, rationale, and fix instructions; inline comments contain the exact file and line context.
- Preserve every thread/comment identifier needed to reply or resolve later.
- Compare each comment with the current diff and file contents because review lines can become outdated.
## Deduplication and Ordering
Build one triage record per distinct finding:
- Prefer exact review-thread identity when available.
- For duplicate bot findings appearing in both summary and inline comments, merge by exact issue title first, then by file path plus line range as a fallback.
- Prefer inline comments for location and current code context.
- Prefer summary comments for severity, category, rationale, and detailed agent prompts.
- Preserve the reviewers exact issue title and original wording where practical. Do not rename findings in a way that would make replies hard to map back to comments.
- Preserve the reviewers original ordering unless the user asks for priority reordering. Many review bots already order findings by severity.
Each triage record should track: original title, reviewer, source IDs, location, current applicability, severity/priority if available, disposition, planned action, verification, reply text if any, resolution decision, platform action attempted, and final platform state.
## Resolution Policy
In normal mode, `Resolve conversation` is the default action for any review thread that has been fully handled. A thread is handled when the requested change is implemented and verified, the current code already satisfies the comment, the comment is outdated and no longer applies, or a deliberate deferral/won't-fix response has been posted with a clear reason.
Keep a thread open only when it still needs reviewer, maintainer, or product input, the fix is local-only and not pushed, verification is missing for a material change, or the user explicitly requested `dry_run`, `no_push`, or `no_reply` behavior that prevents resolution.
When resolving a thread, add a concise reply first only if it provides useful context, such as what changed, why no code change was needed, why a finding was intentionally deferred, or why the original comment is now outdated. Do not add noisy replies for self-evident fixes unless project norms require them.
## Platform Action Contract
Do not treat triage as complete until every collected source ID reaches an explicit terminal state:
- `resolved`: a platform resolve action succeeded, or a re-check shows the thread is already resolved.
- `replied_left_open`: a reply or question was posted and the thread is intentionally left unresolved.
- `not_resolvable`: the source is a PR-level summary comment or copied comment that has no platform-level resolve action; reply or post a PR summary when useful.
- `skipped_by_mode`: `dry_run`, `no_push`, or `no_reply` prevented the external action.
- `failed_action`: a reply or resolve action was attempted and failed; include the attempted action and failure in the final summary.
In normal mode, build and execute a platform action queue after fixes are verified and pushed when needed:
- `reply_then_resolve`: use for handled threads where the reviewer needs context before resolution.
- `resolve_only`: use for self-evident fixes and already-addressed or outdated threads where an extra reply would add noise.
- `reply_leave_open`: use only for clarification requests, blocked work, or intentionally open follow-ups.
- `reply_only`: use for PR-level comments or summaries that cannot be resolved as review threads.
For duplicate findings, execute the terminal action for every source thread ID, not only the primary triage record. If one finding is represented by three unresolved inline threads, all three must be resolved or explicitly left open.
## GitHub Action Guidance
Prefer platform-native APIs or `gh` commands that expose review-thread resolution state. For GitHub inline review threads, use the thread node ID and the GraphQL `resolveReviewThread` mutation rather than assuming that a reply resolves the conversation.
A reliable pattern is:
1. Re-fetch review threads and comments immediately before acting.
2. Reply to the thread when the action queue says a reply is needed.
3. Resolve the review thread by node ID when the terminal state should be `resolved`.
4. Re-fetch unresolved review threads after the action queue completes.
5. Retry any expected-to-be-resolved thread that is still unresolved once; if it still remains unresolved, mark it `failed_action` instead of claiming completion.
Example GraphQL mutation shape:
```graphql
mutation ($threadId: ID!) {
resolveReviewThread(input: { threadId: $threadId }) {
thread {
id
isResolved
}
}
}
```
A posted reply alone is sufficient only for `reply_leave_open`, `reply_only`, or `not_resolvable` sources. For handled inline review threads, reply and resolve are separate actions.
## Flow
```mermaid
flowchart TD
A[Identify PR and branch state] --> B[Collect all review feedback]
B --> C[Deduplicate and preserve source IDs]
C --> D[Inspect current diff and code]
D --> E{Classify each triage record}
E -->|Fix| F[Implement minimal change]
E -->|Answer| G[Prepare concise reply]
E -->|Clarify| H[Prepare question and leave open]
E -->|Already addressed or Outdated| I[Prepare evidence]
E -->|Defer or Won't fix| J[Document reason]
F --> K[Verify]
G --> L{Mode}
H --> L
I --> L
J --> L
K --> L
L -->|dry_run| M[Report triage only]
L -->|no_push| N[Report local diff or commits]
L -->|no_reply| O[Report suggested replies/actions]
L -->|normal| P[Commit/push if changed]
P --> R[Execute reply/resolve action queue]
R --> S[Re-fetch threads and retry unresolved handled threads once]
M --> Q[Final summary]
N --> Q
O --> Q
S --> Q
```
## Compact Workflow
1. **Collect all relevant feedback**
- Identify the PR and gather unresolved review threads, requested-change reviews, PR-level summaries, inline comments, and copied comments.
- Paginate all platform calls and keep comment/thread IDs for later replies and resolution.
- For bot reviews, collect both summary and inline comments, then merge duplicates rather than fixing the same finding twice.
2. **Classify each triage record**
- **Fix**: Valid requested change; make the smallest focused edit when not in `dry_run`.
- **Answer**: No code change needed; prepare a concise explanation.
- **Clarify**: Ambiguous, conflicting, or missing context; reply with the question and leave unresolved.
- **Already addressed**: Current code already satisfies it; prepare evidence.
- **Outdated**: Commented code or issue no longer exists; prepare evidence.
- **Defer / Won't fix**: Valid concern intentionally not changed now; document a specific reason.
3. **Act according to the classification and mode**
- Keep edits scoped to the review feedback.
- Follow reviewer-provided fix instructions literally when they are still applicable; deviate only when the current code proves the instruction is stale or unsafe.
- In `dry_run`, stop at triage, proposed fixes, suggested replies, and verification plan.
- In `no_push`, local edits are allowed, but do not push or resolve threads whose fix is only local. Reply or resolve non-code, already-addressed, or outdated threads only when the action does not depend on unpushed work and `no_reply` is not set.
- In `no_reply`, do not post replies or resolve threads; report suggested replies/actions instead.
- In normal mode, commit and push changed code when appropriate, then execute the platform action queue for every collected source ID.
4. **Verify before claiming completion**
- For fixes, run appropriate checks or explain why they could not run.
- Re-inspect the updated diff and comment context to confirm the concern is resolved.
- Re-fetch review threads after reply/resolve actions and confirm all expected-to-be-resolved thread IDs are resolved.
- Do not mark a thread resolved if it still needs reviewer, maintainer, or product input.
- If a resolve or reply operation fails, retry once when safe; then report `failed_action` with the affected source ID and reason.
5. **Finish**
- Normal mode: commit/push changes when appropriate, post useful replies or a summary, resolve all handled threads by default, and reconcile the final unresolved set.
- Safe modes: report the local state and the exact replies/resolution actions a human could take.
## Reply Guidance
- Keep inline replies short and tied to the original title or concern.
- For fixed findings, mention the concrete change or commit if useful.
- For already-addressed or outdated findings, cite the current code path or behavior that makes the finding no longer applicable.
- For deferred or won't-fix findings, provide the reason and any follow-up issue or owner if known.
- If a reply or resolve operation fails, continue with the remaining threads and report the failure in the final summary.
## Final Summary Checklist
- Mode used: `normal`, `dry_run`, `no_push`, or `no_reply`
- Counts by disposition: fixed, answered, clarified/left open, already addressed, outdated, deferred/won't-fix
- Counts by platform terminal state: resolved, replied-left-open, not-resolvable, skipped-by-mode, failed-action
- Threads resolved, intentionally left open, already resolved, or resolution actions skipped by mode
- Any expected-to-be-resolved thread that remained unresolved after retry
- Verification run or planned
- Commits pushed, local diff/commits, or "none"
- Remaining open items and who needs to respond
+15
View File
@@ -62,6 +62,19 @@ jobs:
runs-on: ubuntu-slim
secrets:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
github-codeql-analysis:
if: >
github.event_name == 'push'
|| github.event_name == 'pull_request'
|| (github.event_name == 'workflow_dispatch' && inputs.workflow == 'lint-and-test')
permissions:
contents: read
security-events: write
actions: read
uses: dceoy/gh-actions-for-devops/.github/workflows/github-codeql-analysis.yml@main # zizmor: ignore[unpinned-uses]
with:
language: >
["python"]
dependabot-auto-merge:
if: >
github.event_name == 'pull_request' && github.actor == 'dependabot[bot]'
@@ -73,5 +86,7 @@ jobs:
contents: write
pull-requests: write
actions: read
checks: read
statuses: read
with:
unconditional: true
+20 -64
View File
@@ -1,81 +1,37 @@
# Repository Guidelines
## Commands
## Project Structure & Module Organization
### Development Setup
`mt5cli/` contains the package source. Important modules include `cli.py` for the Typer command-line app, `client.py` and `sdk.py` for public MT5 client/session APIs, `history.py` for SQLite history collection, `storage.py` and `converters.py` for export behavior, and `schemas.py` for normalized dataset contracts. `tests/` holds pytest coverage for CLI behavior, SDK contracts, trading helpers, history, and utilities. `docs/` and `mkdocs.yml` define the MkDocs site and API reference. `skills/mt5cli/SKILL.md` documents the mt5cli agent skill.
```bash
uv sync
```
## Build, Test, and Development Commands
### Code Quality and Documentation
- `uv sync` installs runtime and development dependencies from `pyproject.toml` and `uv.lock`.
- `uv run mt5cli --help` runs the local CLI entry point.
- `uv run ruff format .` formats Python files.
- `uv run ruff check --fix .` lints and applies safe fixes.
- `uv run pyright .` runs strict type checking.
- `uv run pytest` runs doctests, branch coverage, and the test suite.
- `uv run mkdocs serve` previews documentation locally; `uv run mkdocs build` validates the docs build.
**Important**: Run these before committing or creating a PR.
Use `.agents/skills/local-qa/SKILL.md` for pre-handoff QA. It runs `.agents/skills/local-qa/scripts/qa.sh`, which formats, lints, type-checks, tests, formats Markdown, and checks GitHub workflows.
1. **format, lint, and test**: Use `local-qa` skill.
2. **Documentation build** (if any public API changes): `uv run mkdocs build`
## Coding Style & Naming Conventions
## Architecture
Target Python `>=3.11,<3.14`. Use Ruffs configured 88-character line length and Google-style docstrings. Pyright is strict, so prefer explicit public type annotations and narrow exception handling. Keep module, function, and variable names in `snake_case`; classes and enums use `PascalCase`. Preserve the packages small, typed helper style rather than adding broad abstractions.
### Key Dependencies
## Design Principles
- **pdmt5**: Pandas-based data handler for MetaTrader 5 (core library)
- **typer**: CLI framework for building command-line interfaces
- **click**: Parameter type customization for CLI options
- **pandas**: Core data manipulation and analysis
Apply KISS, DRY, and YAGNI when changing code. Prefer the simplest implementation that satisfies the current CLI/API contract. Remove duplication when shared behavior is already proven by at least two concrete call sites, but avoid generic helpers for speculative reuse. Do not add configuration flags, extension hooks, or alternate backends until a real repository use case requires them.
### Package Structure
## Testing Guidelines
- `mt5cli/`: Main package directory
- `__init__.py`: Package initialization and exports (`detect_format`, `export_dataframe`)
- `cli.py`: CLI application with typer-based commands for data export
- `utils.py`: Constants, enums, parameter types, parsers, and export utilities
- `__main__.py`: Entry point for `python -m mt5cli`
- `tests/`: Comprehensive test suite (pytest-based)
- `test_cli.py`: Tests for CLI commands and collect-history behavior
- `test_utils.py`: Tests for utility constants, parameter types, parsers, and export functions
- `docs/`: MkDocs documentation with API reference
- `docs/index.md`: Main documentation
- `docs/api/`: Auto-generated API documentation for all modules
- Modern Python packaging with `pyproject.toml` and uv dependency management
### Quality Standards
- Type hints required (pyright strict mode)
- Comprehensive linting with 35+ rule categories (ruff)
- Test coverage tracking with 100% (pytest-cov)
- Parametrized tests for input/result matrices using `pytest.mark.parametrize` (pytest)
- Test doubles (mocks, stubs) using `pytest_mock` for external dependencies (pytest-mock)
- Pydantic models for data validation and configuration
### Documentation workflow
1. Add Google-style docstrings to functions/classes
2. Local preview: `uv run mkdocs serve`
3. Build: `uv run mkdocs build`
4. Deploy: `uv run mkdocs gh-deploy`
Tests use pytest, pytest-mock, doctests, and pytest-cov. Test files should match `tests/test_*.py`, classes `Test*`, and functions `test_*`. Coverage is configured with `fail_under = 100`, so add focused tests for every behavior change. Mock MT5/pdmt5 boundaries; do not require a live MetaTrader terminal in unit tests.
## Commit & Pull Request Guidelines
- Run QA checks using `local-qa` skill before committing or creating a PR.
- Branch names use appropriate prefixes on creation (e.g., `feature/...`, `bugfix/...`, `refactor/...`, `docs/...`, `chore/...`).
- When instructed to create a PR, create it as a draft with appropriate labels by default.
Recent history uses concise imperative commits, sometimes with conventional prefixes such as `feat:` or `chore:` and PR numbers appended by GitHub. Keep commits scoped to one logical change. Pull requests should describe behavior changes, note tests run, link related issues, and call out MT5/live-trading risk where relevant.
## Code Design Principles
## Security & Configuration Tips
Always prefer the simplest design that works.
- **KISS**: Choose straightforward solutions and avoid unnecessary abstraction.
- **DRY**: Remove duplication when it improves clarity and maintainability.
- **YAGNI**: Do not add features, hooks, or flexibility until they are needed.
- **SOLID/Clean Code**: Apply these as tools, only when they keep the design simpler and easier to change.
## Development Methodology
Keep delivery incremental, test-backed, and easy to review.
- Make small, safe, reversible changes.
- Prefer `Red -> Green -> Refactor`.
- Do not mix feature work and refactoring in the same commit.
- Refactor when it improves clarity or removes real duplication (Rule of Three).
- Keep tests fast, focused, and self-validating.
Never commit account credentials, broker passwords, exported private data, or local `.venv` contents. Treat `order_send` and CLI `order-send --yes` as live execution paths; gate examples and tests so they cannot place real trades accidentally.
+62 -16
View File
@@ -4,6 +4,8 @@
Generic MT5 data and execution infrastructure for Python applications. Export from the CLI or import a small, stable Python API in downstream packages.
The [Public API Contract](docs/api/public-contract.md) lists stable SDK exports (`mt5cli.STABLE_SDK_EXPORTS`), CLI commands, internal helpers, and responsibilities that remain out of scope (strategy logic, backtests, optimization).
Built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data handler for MetaTrader 5.
## Architecture
@@ -27,25 +29,29 @@ Built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data han
pip install -U mt5cli MetaTrader5
```
Parquet export is not included by default. To enable it, install the `parquet` extra:
```bash
pip install -U "mt5cli[parquet]" MetaTrader5
```
## Python API (downstream packages)
Import `MT5Client` for generic MT5 data access, schema normalization, and optional order primitives. `Mt5CliClient` remains available as a backward-compatible alias.
Import `MT5Client` for generic MT5 data access, schema normalization, and optional order primitives.
```python
from datetime import UTC, datetime
from pathlib import Path
from mt5cli import (
DataKind,
Dataset,
MT5Client,
build_config,
collect_history,
export_dataframe,
mt5_session,
normalize_dataframe,
update_history_with_config,
)
from mt5cli.schemas import DataKind, normalize_dataframe
from mt5cli.utils import Dataset, export_dataframe
# Persistent session for multiple calls
with mt5_session(build_config(login=12345, server="Broker-Demo")) as client:
@@ -81,10 +87,46 @@ update_history_with_config(
)
```
Schema contracts live in `mt5cli.schemas` (`DataKind`, `validate_schema`, `normalize_dataframe`). Storage helpers are re-exported from `mt5cli.storage` and the package root.
Schema contracts live in `mt5cli.schemas` (`DataKind`, `validate_schema`, `normalize_dataframe`). Export and storage helpers are in `mt5cli.utils` (`Dataset`, `export_dataframe`) and `mt5cli.history`.
`MT5Client.order_send()` is a live execution primitive: it can place real trades on the connected account. mt5cli does not implement strategy logic, signal generation, backtesting, or optimization — downstream applications must gate live execution explicitly.
### Trading lifecycle and state helpers
Trading applications can depend on `mt5cli` imports only; terminal path,
credentials, server, and timeout are forwarded to `pdmt5.Mt5Config`, numeric
login strings are coerced to integers, and empty login strings are treated as
unset. Pass `allow_whole_dollar_env=True` to expand `${ENV_VAR}` and bare
`$ENV_NAME` placeholders in connection string parameters before coercion.
```python
from mt5cli import (
build_config,
calculate_spread_ratio,
create_trading_client,
get_account_snapshot,
mt5_trading_session,
)
# Login from environment — numeric string is coerced to int automatically
config = build_config(login="$MT5_LOGIN", allow_whole_dollar_env=True)
with mt5_trading_session(
path=r"C:\Program Files\MetaTrader 5\terminal64.exe",
login="12345",
password="from-env-or-secret-store",
server="Broker-Demo",
) as client:
account = get_account_snapshot(client)
spread = calculate_spread_ratio(client, "EURUSD")
client = create_trading_client(login=12345, server="Broker-Demo")
try:
positions = client.positions_get_as_df(symbol="EURUSD")
finally:
client.shutdown()
```
## CLI usage
```bash
@@ -140,10 +182,13 @@ python -m mt5cli -o account.csv account-info
| `recent-history-deals` | Export historical deals from a recent trailing window |
| `mt5-summary` | Export terminal/account status summary |
| `order-check` | Check funds sufficiency for a trade request |
| `order-send` | Send a trade request to the trade server (`--yes` required) |
| `order-send` | Send a raw trade request to the trade server (`--yes` required; expert path) |
| `close-positions` | Close open positions by `--symbol` or `--ticket` (`--yes` required for live; `--dry-run` available) |
| `collect-history` | Bundle rates, ticks, history-orders, and history-deals for one or more symbols into a single SQLite database |
Use `order-check` to validate a request payload before running `order-send --yes`.
`close-positions` is the safer high-level alternative that builds correct close
requests automatically. At least one `--symbol` or `--ticket` must be provided.
### `collect-history`
@@ -165,7 +210,8 @@ For automated pipelines, use the importable incremental API instead of re-fetchi
```python
from pdmt5 import Mt5Config, Mt5DataClient
from mt5cli import Dataset, update_history, update_history_with_config
from mt5cli import update_history, update_history_with_config
from mt5cli.utils import Dataset
# Reuse an already-connected pdmt5 client (does not open/close MT5)
client = Mt5DataClient(config=Mt5Config(login=12345))
@@ -196,11 +242,11 @@ update_history_with_config(
- **`collect-history`**: explicit date-range export into SQLite.
- **`update_history`**: incremental append based on existing SQLite `MAX(time)` per symbol (and timeframe for rates); account-level deals use a separate cursor when `include_account_events=True`.
- **`rates` table**: normalized storage with `symbol` and `timeframe` columns.
- **Rate compatibility views**: mt5cli manages all `rate_*` views. Naming is `rate_<symbol>__<timeframe>` when a symbol has one timeframe, otherwise `rate_<symbol>__<granularity>_<timeframe>` (for example `rate_EURUSD__M1_1`). Stale `rate_*` views are dropped and recreated when rates change for offline tools such as mteor optimize.
- **Rate compatibility views**: mt5cli manages all `rate_*` views. Naming is `rate_<symbol>__<timeframe>` when a symbol has one timeframe, otherwise `rate_<symbol>__<granularity>_<timeframe>` (for example `rate_EURUSD__M1_1`). Stale `rate_*` views are dropped and recreated when rates change for offline downstream tools.
- **Rate view resolution**: use `resolve_rate_view_name()` / `resolve_rate_view_names()` to map symbols and granularities to existing SQLite compatibility views without creating databases. Both accept `None` (or a missing path) and return deterministic default names unless `require_existing=True`.
- **Rate view loading**: use `load_rate_data()` / `load_rate_data_from_connection()` to load a SQLite rate table or view into a `DatetimeIndex` DataFrame.
- **Multi-series rate loading**: use `build_rate_targets()` to build neutral `RateTarget(symbol, timeframe)` pairs, `resolve_rate_tables()` to map them to table/view names (pass `require_existing=True` for strict resolution), and `load_rate_series_from_sqlite()` to load them into a mapping keyed by `(symbol, integer timeframe)`. The loader requires existing managed views unless `explicit_tables` is supplied, and rejects duplicate `(symbol, timeframe)` targets.
- **Multi-account latest rates**: use `collect_latest_rates_for_accounts()` with `AccountSpec` to read the latest bars for several account groups, merged into a `(symbol, integer timeframe)` mapping. For long-running pollers, `collect_latest_rates_for_accounts_with_retries()` adds bounded exponential backoff that retries only `pdmt5.Mt5TradingError` / `pdmt5.Mt5RuntimeError` and re-raises once `retry_count` is exhausted.
- **Multi-account latest rates**: use `collect_latest_rates_for_accounts()` with `AccountSpec` to read the latest bars for several account groups, merged into a `(symbol, integer timeframe)` mapping. For long-running pollers, `collect_latest_rates_for_accounts_with_retries()` adds bounded exponential backoff that retries only recoverable MT5 errors and re-raises once `retry_count` is exhausted.
- **Latest closed bars**: use `collect_latest_closed_rates_for_accounts()` when downstream logic must exclude the still-forming current bar. It fetches `count + 1` bars at `start_pos=0`, drops the last row with `drop_forming_rate_bar()`, and validates each series is non-empty. `collect_latest_closed_rates_by_granularity()` returns the same data keyed by `(symbol, granularity_name)` such as `("EURUSD", "M1")`.
```python
@@ -215,9 +261,9 @@ rates = collect_latest_closed_rates_by_granularity(
eurusd_m1 = rates["EURUSD", "M1"] # closed bars only
```
- **Credential resolution**: use `resolve_account_spec()` / `resolve_account_specs()` to merge explicit override values over `AccountSpec` fields and expand `${ENV_VAR}` placeholders (via `substitute_env_placeholders()`), raising `ValueError` for missing variables. This keeps secrets out of plan/config files without coupling to any strategy code.
- **Throttled history updates**: use `ThrottledHistoryUpdater` to wrap `update_history()` with a minimum `interval_seconds` between successful runs (monotonic clock). Call `should_update()` / `update(client, symbols)` from an application loop; errors propagate by default, or pass `suppress_errors=True` to swallow recoverable `Mt5*Error`, `sqlite3.Error`, `ValueError`, `OSError`, and MT5 client capability errors for history API methods without advancing the throttle (other `AttributeError` / `TypeError` values always propagate).
- **Trading session helpers**: use `mt5_trading_session()` for a trading-capable `pdmt5.Mt5TradingClient` that initializes/logs in via `Mt5Config.path` and always shuts down safely. Pair with `detect_position_side()`, `calculate_margin_and_volume()`, and `determine_order_limits()` for generic position and sizing utilities. The read-only `mt5_session()` / `Mt5CliClient` SDK is unchanged.
- **Credential resolution**: use `resolve_account_spec()` / `resolve_account_specs()` to merge explicit override values over `AccountSpec` fields and expand `${ENV_VAR}` placeholders (via `substitute_env_placeholders()`), raising `ValueError` for missing variables. This keeps secrets out of plan/config files without coupling to any strategy code. For config dicts or nested structures loaded from YAML/TOML, use `substitute_mapping_values(data, keys={"login", "password"})` to expand placeholders only for caller-specified keys — key names are never hard-coded in mt5cli.
- **Throttled history updates**: use `ThrottledHistoryUpdater` to wrap `update_history()` with a minimum `interval_seconds` between successful runs (monotonic clock). Call `should_update()` / `update(client, symbols)` from an application loop; errors propagate by default, or pass `suppress_errors=True` to swallow recoverable `Mt5*Error`, `sqlite3.Error`, `ValueError`, `OSError`, and MT5 client capability errors for history API methods without advancing the throttle (other `AttributeError` / `TypeError` values always propagate). Pass `update_backend` to inject a custom history update callable (same keyword arguments as `update_history`) instead of monkey-patching `mt5cli.sdk.update_history`.
- **Trading session helpers**: use `mt5_trading_session()` for a trading-capable client that initializes/logs in via `Mt5Config.path` and always shuts down safely. Pair with `detect_position_side()`, `calculate_margin_and_volume()`, and `determine_order_limits()` for generic position and sizing utilities. Keep read-only collection on `mt5_session()` / `MT5Client`.
- **Granularity-keyed rate loading**: `load_rate_series_by_granularity()` builds targets with `build_rate_targets()`, loads them with `load_rate_series_from_sqlite()`, and returns a mapping keyed by `(symbol | None, granularity_name)` such as `("EURUSD", "M1")` to reduce downstream boilerplate.
- **MT5 session helper**: use the `mt5_session()` context manager to attach to (or, when `Mt5Config.path` is set, launch) an MT5 terminal, log in, and yield a connected `MT5Client` that shuts down on exit.
- **SQLite export helpers**: use `export_dataframe_to_sqlite()` for append mode, optional index export, and post-write deduplication by key columns.
@@ -229,12 +275,12 @@ eurusd_m1 = rates["EURUSD", "M1"] # closed bars only
- Windows OS (MetaTrader 5 requirement)
- MetaTrader 5 platform installed
### Migration note for mteor
### Migration note for downstream trading apps
Replace local MT5 lifecycle and trading helper code with mt5cli imports:
```python
# Before (local mteor helpers)
# Before (local application helpers)
# with local_mt5_trading_session(config) as client:
# side = local_detect_position_side(client, symbol)
# sizing = local_calculate_margin_and_volume(client, symbol, unit_ratio, preserved_ratio)
@@ -284,7 +330,7 @@ finally:
client.shutdown()
```
Read-only collectors can keep using `mt5_session()` and `MT5Client` (or the `Mt5CliClient` alias) without changes.
Read-only collectors can keep using `mt5_session()` and `MT5Client`.
## Development
+32 -4
View File
@@ -167,19 +167,47 @@ Resolution rules:
### Rate data loading
Use `load_rate_data()` to load a table or view from a SQLite path, or
`load_rate_data_from_connection()` when you already have a connection:
The canonical normalized rate table is `rates`; compatibility views are named
with `rate_<symbol>__<timeframe>` for single-timeframe symbols or
`rate_<symbol>__<granularity>_<timeframe>` when a symbol has multiple stored
timeframes. `resolve_rate_table_name()` returns `rates`, while
`resolve_rate_view_name()` returns the per-symbol compatibility view name.
Use `load_rate_data()` or `load_rate_series_from_sqlite(..., table=...)` to load
a single table or view from a SQLite path. Use
`load_rate_series_by_granularity()` to load multiple instrument/granularity
targets without hard-coding view names:
```python
from pathlib import Path
from mt5cli import load_rate_data
from mt5cli.history import resolve_rate_view_name
from mt5cli import (
load_rate_series_by_granularity,
load_rate_series_from_sqlite,
)
from mt5cli.history import (
load_rate_data,
resolve_rate_table_name,
resolve_rate_view_name,
)
view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1", require_existing=True)
rates = load_rate_data(Path("history.db"), view, count=1000)
same_rates = load_rate_series_from_sqlite(Path("history.db"), table=view, count=1000)
table = resolve_rate_table_name("EURUSD", "M1") # "rates"
series = load_rate_series_by_granularity(
Path("history.db"),
symbols=["EURUSD", "GBPUSD"],
granularities=["M1", "H1"],
count=500,
)
```
`count` returns the latest rows while preserving chronological order. Missing
tables/views and mismatched `explicit_tables` lengths raise `ValueError` with
the requested database target in the message.
The loader accepts close-based OHLC rate data or tick-like bid/ask data. It
validates that `time` exists, parses timestamps with pandas, and returns a
DataFrame indexed by ascending `DatetimeIndex` named `time`.
+10 -4
View File
@@ -2,13 +2,17 @@
This section documents the mt5cli public Python API and CLI modules.
Start with the [Public API Contract](public-contract.md) for the stable
downstream SDK surface, CLI boundary, internal modules, and out-of-scope strategy
responsibilities.
## Public API layers
| Module | Purpose |
| ----------------------------------------- | ------------------------------------------------------------------------- |
| [Public API Contract](public-contract.md) | Stable downstream SDK exports, CLI boundary, and out-of-scope items |
| [Client](client.md) | `MT5Client` session abstraction for data access and order primitives |
| [Schemas](schemas.md) | Canonical DataFrame contracts and normalization helpers |
| [Storage](storage.md) | CSV/JSON/Parquet/SQLite export and history collection helpers |
| [Converters](converters.md) | Symbol, timeframe, timezone, and date-range utilities |
| [Exceptions](exceptions.md) | Stable mt5cli exception types and MT5 error normalization |
| [SDK](sdk.md) | Module-level fetch helpers, multi-account collectors, incremental history |
@@ -25,12 +29,14 @@ flowchart TD
CLI["mt5cli CLI"] --> Client
Client --> SDK["sdk / pdmt5"]
Client --> Schemas["schemas"]
Storage["storage"] --> History["history SQLite"]
Storage --> Utils["utils export"]
History["history SQLite"] --> Utils["utils export"]
SDK --> PDMT5["pdmt5.Mt5DataClient"]
```
Downstream packages should depend on the package root exports (`MT5Client`, `DataKind`, `normalize_dataframe`, `export_dataframe`, `collect_history`, etc.) rather than private modules.
Downstream packages should depend on the package root exports documented in the
[Public API Contract](public-contract.md) (`MT5Client`,
`collect_history`, `load_rate_series_from_sqlite`, etc.) rather than private
modules. Lower-level helpers are accessible directly from their owning modules.
`MT5Client.order_send()` is a live execution primitive that can place real trades. mt5cli exposes minimal execution helpers only; strategy logic, signals, backtests, and optimization remain out of scope and must be implemented downstream with explicit execution gating.
+202
View File
@@ -0,0 +1,202 @@
# Public API Contract
mt5cli is the canonical operational trading SDK and CLI/batch layer over pdmt5.
The intended dependency direction is:
```text
downstream app -> mt5cli -> pdmt5 -> MetaTrader 5
```
## Responsibility boundary
| Layer | Owns |
| -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **pdmt5** | MT5 core wrapper; DataFrame/dict conversion; canonical MT5 constants and parsers; direct low-level order primitives |
| **mt5cli** | CLI/batch workflows; SQLite history collection; normalized datasets; closed-bar helpers; small downstream operational SDK; generic broker-facing margin/volume/order orchestration |
| **downstream** | Strategy logic; signals; risk policy; backtesting; optimization; YAML/application semantics |
Downstream code should import raw pdmt5 types and constants (such as
`Mt5Config`, `Mt5RuntimeError`, `TIMEFRAME_MAP`, `COPY_TICKS_MAP`) directly
from `pdmt5` when needed. mt5cli does not serve as a pass-through compatibility
namespace for pdmt5. mt5cli's trading helpers type their client parameter against
an internal protocol backed by `pdmt5.Mt5DataClient`; `Mt5TradingClient` is no
longer required. `Mt5TradingError` is conditionally imported where still present
in pdmt5, but mt5cli raises `Mt5OperationError` for all trading-related failures.
Note: the former `mt5cli` re-export `TICK_FLAG_MAP` corresponds to `COPY_TICKS_MAP`
in pdmt5 — the name changed, it was not simply moved.
Downstream packages should import from the package root (`from mt5cli import
...`). The contract set `STABLE_SDK_EXPORTS` in `mt5cli.contract` enumerates
every package-root symbol. Lower-level helpers (schema utilities, export
functions, parser helpers, low-level MT5 wrappers) are available directly from
their owning modules (`mt5cli.schemas`, `mt5cli.utils`, `mt5cli.converters`,
`mt5cli.sdk`, etc.) and are not part of the root SDK surface.
## Stable downstream SDK API
These names are exported from `mt5cli` and enumerated in
`mt5cli.STABLE_SDK_EXPORTS` (defined in `mt5cli.contract`).
### Session lifecycle and configuration
| Symbol | Role |
| ----------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `MT5Client` | Read-only data client with optional `order_check` / `order_send` |
| `build_config` | Build `pdmt5.Mt5Config` from connection fields; `login` accepts `int \| str \| None` — numeric strings are coerced to `int`, blank strings are treated as unset, and `${ENV_VAR}` / `$ENV_NAME` placeholders in string parameters are expanded when `allow_whole_dollar_env=True` |
| `mt5_session` | Context manager: initialize, login, yield client, shutdown |
| `create_trading_client`, `mt5_trading_session` | Trading-capable MT5 client lifecycle; returns a client supporting order execution and account management |
| `AccountSpec` | Generic account group: symbols plus optional credentials |
| `resolve_account_spec`, `resolve_account_specs` | Merge overrides and expand `${ENV_VAR}` placeholders; opt-in `allow_whole_dollar_env` for bare `$NAME` |
### Closed-bar rate helpers
MetaTrader 5 returns the still-forming bar as the last row when
`start_pos=0`. Use these helpers instead of reimplementing bar trimming or
timestamp normalization in downstream apps.
| Symbol | Role |
| ------------------------------------------------ | ------------------------------------------------------------------------------- |
| `drop_forming_rate_bar` | Remove the last row from chronologically ordered rate data |
| `fetch_latest_closed_rates` | Single connected client: fetch `count + 1`, drop forming bar |
| `fetch_latest_closed_rates_for_trading_client` | Closed bars from an active trading client session; returns RangeIndex |
| `fetch_latest_closed_rates_indexed` | Same as above but returns a UTC `DatetimeIndex` named `"time"` (no time column) |
| `collect_latest_closed_rates_for_accounts` | Multi-account closed bars with optional retry wrapper |
| `collect_latest_closed_rates_by_granularity` | Same data keyed by `(symbol, granularity_name)` |
| `collect_latest_rates_for_accounts_with_retries` | Bounded exponential backoff for transient MT5 errors |
### SQLite history collection and rate loading
| Symbol | Role |
| ----------------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| `collect_history` | One-shot date-range export into SQLite |
| `update_history`, `update_history_with_config` | Incremental append from `MAX(time)` cursors |
| `ThrottledHistoryUpdater` | Minimum interval between successful incremental updates; optional `update_backend` injection |
| `RateTarget`, `build_rate_targets` | Neutral `(symbol, timeframe)` series descriptors |
| `load_rate_series_from_sqlite`, `load_rate_series_by_granularity` | Load one or many series; fail clearly when managed views are missing |
See [History Collection (SQLite)](history.md) for schema, view naming, and ER
diagrams.
### Trading and sizing primitives (generic)
These helpers implement broker-facing calculations only. They do not encode
strategy entries, exits, Kelly sizing, or signal logic.
| Symbol | Role |
| ------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------- |
| `get_account_snapshot`, `get_symbol_snapshot`, `get_tick_snapshot`, `get_positions_frame` | Normalized account/symbol/tick/position views |
| `extract_tick_price` | Positive finite bid/ask extraction from tick mappings |
| `detect_position_side` | Net long / short / flat from open positions |
| `calculate_spread_ratio` | Relative bid-ask spread |
| `calculate_margin_and_volume`, `calculate_volume_by_margin`, `calculate_new_position_margin_ratio` | Margin budget and volume sizing |
| `normalize_order_volume`, `estimate_order_margin`, `calculate_positions_margin` | Broker volume normalization and margin totals |
| `calculate_positions_margin_by_symbol` | Per-symbol margin map (resilient, first-seen order) |
| `calculate_positions_margin_safe` | Summed total margin across symbols (failed symbols skipped) |
| `calculate_projected_margin_ratio` | Estimated symbol-scoped margin/equity after optional new exposure |
| `calculate_account_projected_margin_ratio` | Account snapshot margin/equity after optional new exposure |
| `calculate_symbol_group_margin_ratio` | Estimated symbol-group margin/equity with optional exposure |
| `determine_order_limits` | SL/TP price levels from ratios |
| `calculate_trailing_stop_updates` | Per-ticket generic trailing stop-loss update plan |
| `ensure_symbol_selected` | Select/verify Market Watch visibility |
| `place_market_order`, `close_open_positions`, `update_sltp_for_open_positions`, `update_trailing_stop_loss_for_open_positions` | Order execution helpers (`dry_run` supported) |
| `MarginVolume`, `OrderLimits`, `OrderExecutionResult` | Typed return contracts for order helpers |
| `OrderSide`, `OrderFillingMode`, `OrderTimeMode`, `PositionSide`, `ExecutionStatus` | Typed enums for order helpers |
| `ProjectionMode` | Literal type for `calculate_symbol_group_margin_ratio` projection |
`calculate_symbol_group_margin_ratio` accepts an optional `projection_mode`
parameter (`"add"` by default). Pass `projection_mode="replace_symbol"` to
subtract current exposure for `new_symbol` before adding the candidate margin —
useful for reversal-style projections. mt5cli only calculates broker-facing
exposure; downstream applications own thresholds, risk guard actions, and
strategy policy.
`MT5Client.order_send()` and CLI `order-send --yes` are live execution paths.
Order helpers validate broker stop-level distance in `determine_order_limits()` and
raise `Mt5OperationError` when computed SL/TP prices are too close to the entry
quote. Validation uses `trade_stops_level * point` from the current quote and
symbol metadata as a pre-check only; it does not guarantee live order acceptance
after price movement and does not inspect `trade_freeze_level`. Live
`place_market_order()` and SL/TP updates call
`ensure_symbol_selected()` so hidden symbols are added to Market Watch before
sending requests. Failed, malformed, or unknown broker retcodes are fail-closed
and returned as `status="failed"` with normalized `request` / `response` details;
`dry_run=True` never calls `ensure_symbol_selected()` or `order_send()`.
### Errors
| Symbol | Role |
| -------------------------------------------------------------------------- | ----------------------------- |
| `Mt5CliError`, `Mt5ConnectionError`, `Mt5OperationError`, `Mt5SchemaError` | Stable mt5cli exception types |
## Module-scoped helpers
Lower-level helpers are available from their owning modules and are not part
of the package-root stable surface. Import them directly when needed:
| Module | Examples |
| ------------------- | ---------------------------------------------------------------------------------------------- |
| `mt5cli.history` | `resolve_rate_view_name`, `resolve_rate_tables`, `load_rate_data`, `build_rate_view_name` |
| `mt5cli.sdk` | `copy_rates_from`, `copy_ticks_from`, `account_info`, `symbols`, `mt5_summary`, `latest_rates` |
| `mt5cli.schemas` | `DataKind`, `normalize_dataframe`, `validate_schema`, `DEDUP_KEYS` |
| `mt5cli.utils` | `Dataset`, `IfExists`, `detect_format`, `export_dataframe`, `export_dataframe_to_sqlite` |
| `mt5cli.converters` | `normalize_symbol`, `ensure_utc`, `parse_date_range`, `granularity_name` |
| `mt5cli.exceptions` | `normalize_mt5_exception`, `call_with_normalized_errors`, `is_recoverable_mt5_error` |
## CLI commands
The Typer application in `mt5cli.cli` exposes file-export commands documented in
[CLI Module](cli.md) and the project README. CLI commands:
- Require `-o/--output` and write CSV, JSON, Parquet, or SQLite.
- Accept global MT5 connection options (`--login`, `--password`, `--server`,
`--path`, `--timeout`).
- Delegate to the same Python APIs described here; they are not duplicated
business logic.
`order-send` is the expert raw-request path; it requires `--yes` and a fully
constructed request payload. `close-positions` is the safer high-level helper
that closes open positions by `--symbol` or `--ticket` using
`close_open_positions()`. Both `order-send --yes` and `close-positions --yes`
are live execution paths. `close-positions --dry-run` previews close orders
without placing them and does not require `--yes`.
## Internal helpers (not stable)
Do not import these for downstream contracts; they may change without a semver
notice:
| Module | Examples |
| ------------------------ | ------------------------------------------------------------------------- |
| `mt5cli.sdk` | `connected_client`, `_run_with_client`, private coercion helpers |
| `mt5cli.history` | `write_*_dataset`, `deduplicate_history_tables`, `parse_sqlite_timestamp` |
| `mt5cli.retry` | `retry_with_backoff` |
| `mt5cli.cli` | Typer command handlers and Click parameter types |
| Leading-underscore names | Any `_`-prefixed function or method |
Use the package-root stable exports instead of reaching into submodule
internals.
## Explicitly out of scope
mt5cli must **not** implement downstream strategy or research responsibilities.
The following belong in consuming applications, not in mt5cli:
- Signal detection (for example AR-GARCH or other model-specific triggers)
- Backtesting, walk-forward analysis, or parameter optimization
- Strategy-specific risk policy, position sizing systems, or Kelly fractions
- Entry/exit decision logic or YAML strategy semantics
- Application-specific credential schema keys wired into mt5cli internals
mt5cli provides connection lifecycle, normalized data access, SQLite history
machinery, closed-bar helpers, generic margin/volume/spread/SL/TP utilities, and
optional order primitives so downstream apps can focus on strategy code behind
their own adapter layer.
## Contract verification
`tests/test_contracts.py` asserts that every name in `STABLE_SDK_EXPORTS` is
importable from `mt5cli`, that all package-root exports are covered by the
stable set, and documents key closed-bar, SQLite loading, account-resolution,
and trading-session behaviors.
+69 -8
View File
@@ -31,13 +31,26 @@ rates = collect_latest_rates_for_accounts_with_retries(
### Latest closed rate bars
MetaTrader 5 `start_pos=0` includes the still-forming current bar as the last
row. `collect_latest_closed_rates_for_accounts()` fetches `count + 1` bars,
drops that row with `drop_forming_rate_bar()`, and validates each series is
non-empty. Use `collect_latest_closed_rates_by_granularity()` when callers
prefer keys such as `("EURUSD", "M1")` instead of integer timeframes.
row. `fetch_latest_closed_rates()` handles one connected `MT5Client`; use
`fetch_latest_closed_rates_for_trading_client()` from an active
`Mt5TradingClient` session. Multi-account helpers fetch `count + 1` bars, drop
that row with `drop_forming_rate_bar()`, and validate each series is non-empty. Returned frames are ordered
oldest-to-newest and may contain fewer than `count` rows only when MT5 returns
fewer closed bars.
```python
from mt5cli import AccountSpec, collect_latest_closed_rates_by_granularity
from mt5cli import (
AccountSpec,
collect_latest_closed_rates_by_granularity,
fetch_latest_closed_rates,
)
closed = fetch_latest_closed_rates(
client,
symbol="EURUSD",
granularity="M1",
count=500,
)
rates = collect_latest_closed_rates_by_granularity(
[AccountSpec(symbols=["EURUSD"], login=12345)],
@@ -48,6 +61,9 @@ rates = collect_latest_closed_rates_by_granularity(
closed_m1 = rates["EURUSD", "M1"]
```
Use `collect_latest_closed_rates_by_granularity()` when callers prefer keys such
as `("EURUSD", "M1")` instead of integer timeframes.
### Resolving credentials and `${ENV_VAR}` placeholders
`resolve_account_spec()` / `resolve_account_specs()` merge explicit override
@@ -70,6 +86,28 @@ resolved = resolve_account_specs(accounts, server="Broker-Demo")
# resolved[0].login == "12345", resolved[0].server == "Broker-Demo"
```
Pass `allow_whole_dollar_env=True` to also expand strings whose **entire value**
is a bare `$ENV_NAME` identifier (no braces). This opt-in covers
`substitute_env_placeholders()`, `resolve_account_spec()`,
`resolve_account_specs()`, and `build_config()`. Note: `build_config` cannot
expand `login` because that parameter is `int | None`; use
`resolve_account_spec` for a string `login` placeholder. Partial strings such as
`"plan$pass"`, `"abc$ENV"`, or `"$ENV-suffix"` are never expanded — only an
exact `$IDENTIFIER` whole-string match qualifies. The default is `False` to
preserve backward compatibility.
```python
import os
from mt5cli import AccountSpec, resolve_account_specs
os.environ["MT5_PASSWORD"] = "secret"
accounts = [AccountSpec(symbols=["EURUSD"], password="$MT5_PASSWORD")]
resolved = resolve_account_specs(accounts, allow_whole_dollar_env=True)
# resolved[0].password == "secret"
```
### Throttled incremental history updates
`ThrottledHistoryUpdater` wraps `update_history()` with a minimum interval
@@ -79,7 +117,8 @@ call it every iteration without over-fetching.
```python
from pdmt5 import Mt5Config, Mt5DataClient
from mt5cli import Dataset, ThrottledHistoryUpdater
from mt5cli import ThrottledHistoryUpdater
from mt5cli.utils import Dataset
updater = ThrottledHistoryUpdater(
output="history.db",
@@ -98,6 +137,28 @@ finally:
client.shutdown()
```
Pass `update_backend` to substitute the default `update_history` implementation
without monkey-patching `mt5cli.sdk.update_history`. The callable receives the
same keyword arguments as `update_history` (`client`, `output`, `symbols`,
`datasets`, `timeframes`, `flags`, `lookback_hours`, `with_views`,
`include_account_events`). The resolved backend is stored on
`updater.update_backend` for inspection or subclassing.
```python
from mt5cli import ThrottledHistoryUpdater, update_history
def app_update_history(**kwargs) -> None:
update_history(**kwargs) # or delegate to application-specific logic
updater = ThrottledHistoryUpdater(
output="history.db",
interval_seconds=60,
update_backend=app_update_history,
)
```
By default recoverable errors (`Mt5TradingError`, `Mt5RuntimeError`,
`sqlite3.Error`, `ValueError`, `OSError`, and MT5 client capability
`AttributeError` / `TypeError` for history API methods) propagate so the caller
@@ -110,5 +171,5 @@ resulting `ValueError` is suppressed along with other recoverable errors.
## Trading-capable sessions
For order placement and trading calculations, use the dedicated
[Trading module](trading.md). The read-only `Mt5CliClient` and `mt5_session()`
helpers in this module are unchanged.
[Trading module](trading.md). Use `mt5_session()` / `MT5Client` for read-only
collection.
-3
View File
@@ -1,3 +0,0 @@
# Storage
::: mt5cli.storage
+152 -22
View File
@@ -4,38 +4,90 @@
## Trading-capable MT5 sessions
`mt5_trading_session()` complements the read-only `mt5_session()` helper in
`sdk.py`. It yields a connected `pdmt5.Mt5TradingClient`, uses
`Mt5Config.path` to launch the terminal when configured, and always calls
`shutdown()` on exit.
`create_trading_client()` and `mt5_trading_session()` complement the read-only
`mt5_session()` helper in `sdk.py`. They return or yield an initialized
client supporting order execution and account management, use `Mt5Config.path`
to launch the terminal when configured, and `mt5_trading_session()` always
calls `shutdown()` on exit.
```python
from pdmt5 import Mt5Config
from mt5cli import mt5_trading_session
from mt5cli import create_trading_client, mt5_trading_session
with mt5_trading_session(
Mt5Config(path=r"C:\Program Files\MetaTrader 5\terminal64.exe", login=12345),
path=r"C:\Program Files\MetaTrader 5\terminal64.exe",
login="12345",
password="secret",
server="Broker-Demo",
retry_count=2,
) as client:
positions = client.positions_get_as_df(symbol="EURUSD")
client = create_trading_client(login=12345, server="Broker-Demo")
try:
account = client.account_info_as_dict()
finally:
client.shutdown()
```
The read-only `Mt5CliClient` / `mt5_session()` API is unchanged.
`login` accepts `int`, numeric `str`, or an empty string; empty strings are
treated as unset. `path`, `password`, `server`, and `timeout` are forwarded to
`pdmt5.Mt5Config`, and omitted `timeout` values keep the lower-level default.
Use `mt5_session()` / `MT5Client` for read-only data collection.
## Operational trading helpers
## State and order helpers
These helpers are strategy-agnostic and do not depend on signal detection,
betting logic, or scheduling code in downstream applications.
```python
from mt5cli import (
calculate_positions_margin,
calculate_spread_ratio,
calculate_margin_and_volume,
close_open_positions,
detect_position_side,
determine_order_limits,
estimate_order_margin,
fetch_latest_closed_rates_for_trading_client,
fetch_latest_closed_rates_indexed,
get_account_snapshot,
get_positions_frame,
get_symbol_snapshot,
get_tick_snapshot,
normalize_order_volume,
place_market_order,
)
account = get_account_snapshot(client)
symbol = get_symbol_snapshot(client, "EURUSD")
tick = get_tick_snapshot(client, "EURUSD")
positions = get_positions_frame(client, "EURUSD")
side = detect_position_side(client, "EURUSD")
spread_ratio = calculate_spread_ratio(client, "EURUSD")
volume = normalize_order_volume(
0.15,
volume_min=symbol["volume_min"],
volume_max=symbol["volume_max"],
volume_step=symbol["volume_step"],
)
buy_margin = (
estimate_order_margin(client, "EURUSD", "BUY", volume) if volume > 0 else 0.0
)
open_margin = calculate_positions_margin(client, symbols=["EURUSD"])
closed_bars = fetch_latest_closed_rates_for_trading_client(
client,
symbol="EURUSD",
granularity="M1",
count=100,
)
# Or fetch with a UTC DatetimeIndex instead of a "time" column:
indexed_bars = fetch_latest_closed_rates_indexed(
client,
symbol="EURUSD",
granularity="M1",
count=100,
)
# indexed_bars.index is a UTC-aware DatetimeIndex named "time"
sizing = calculate_margin_and_volume(
client,
"EURUSD",
@@ -49,22 +101,100 @@ limits = determine_order_limits(
stop_loss_limit_ratio=0.01,
take_profit_limit_ratio=0.02,
)
preview = place_market_order(
client,
symbol="EURUSD",
volume=sizing["buy_volume"],
order_side="BUY",
sl=limits["stop_loss"],
tp=limits["take_profit"],
dry_run=True,
)
closed = close_open_positions(client, symbols="EURUSD", dry_run=True)
```
Protective ratios must satisfy `0 <= ratio < 1`; `0` omits that level.
`calculate_margin_and_volume()` clamps negative `margin_free` to `0.0`
before sizing.
`detect_position_side()` returns `long` for buy-only exposure, `short` for
sell-only exposure, and `None` for no positions or mixed long/short exposure.
`calculate_spread_ratio()` uses `(ask - bid) / ((ask + bid) / 2)` and raises
`Mt5OperationError` when bid or ask is missing or non-positive.
`normalize_order_volume()` returns `0.0` for invalid constraints or
sub-minimum requests; check the result before calling `estimate_order_margin()`,
which requires a positive finite volume. `calculate_positions_margin()` silently
skips rows with missing symbols, non-positive volumes, non-finite volumes, or
unsupported position types, but propagates `Mt5OperationError` from `estimate_order_margin()` when a valid row
encounters invalid tick data or margin results from the broker.
## Migration from mteor-local helpers
SL/TP ratios for `determine_order_limits()` must satisfy `0 <= ratio < 1`; `0`
omits that level. SL/TP prices are rounded with symbol `digits` metadata when
available. `determine_order_limits()` pre-validates computed SL/TP prices against
available `trade_stops_level * point` metadata when present; violations raise
`Mt5OperationError`. This is a planning helper only: it does not guarantee broker
acceptance because live validation can still depend on price movement, bid/ask
side, freeze levels, and server-side rules, and it does not validate
`trade_freeze_level`. When symbol metadata cannot be loaded, protective prices
still round with `digits=8` and stop-level validation is skipped.
`unit_margin_ratio` and `preserved_margin_ratio` for `calculate_margin_and_volume()`
accept `0 <= ratio <= 1`; `unit_margin_ratio=0` requests one minimum valid unit
when the post-reserve margin can afford it. Negative `margin_free` is clamped to
`0.0` before sizing. Execution helpers return normalized `OrderExecutionResult`
dictionaries containing the request, response, status, retcode, and `dry_run`
flag; `dry_run=True` never sends an order or mutates Market Watch visibility.
`ensure_symbol_selected()` adds hidden symbols to Market Watch before live order
placement and SL/TP updates. Failed, malformed, or unknown broker retcodes are
fail-closed and returned as `status="failed"` while keeping the normalized
response for inspection.
| mteor-local concern | mt5cli replacement |
| -------------------------------------------------------- | ----------------------------------------------- |
| Manual terminal spawn/kill around trading code | `mt5_trading_session()` |
| Local position-side detection | `detect_position_side()` |
| Local margin/volume sizing | `calculate_margin_and_volume()` |
| Local SL/TP price derivation | `determine_order_limits()` |
| Throttled SQLite history loop with ad-hoc error handling | `ThrottledHistoryUpdater(suppress_errors=True)` |
## Order planning return contracts
Keep read-only data collection on `mt5_session()` / `Mt5CliClient`; use
```python
from mt5cli import MarginVolume, OrderLimits, OrderExecutionResult
sizing: MarginVolume = calculate_margin_and_volume(
client,
"EURUSD",
unit_margin_ratio=0.5,
preserved_margin_ratio=0.2,
)
limits: OrderLimits = determine_order_limits(
client,
"EURUSD",
side="long",
stop_loss_limit_ratio=0.01,
take_profit_limit_ratio=0.02,
)
preview: OrderExecutionResult = place_market_order(
client,
symbol="EURUSD",
volume=sizing["buy_volume"],
order_side="BUY",
sl=limits["stop_loss"],
tp=limits["take_profit"],
dry_run=True,
)
updates: list[OrderExecutionResult] = update_sltp_for_open_positions(
client,
symbol="EURUSD",
stop_loss=limits["stop_loss"],
dry_run=True,
)
```
Closes issue #33: strategy-neutral order planning and execution helpers exposed
through the stable package root without embedding entry/exit policy.
## Migration from application-local helpers
| Application-local concern | mt5cli replacement |
| -------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| Manual terminal spawn/kill around trading code | `mt5_trading_session()` |
| Local position-side detection | `detect_position_side()` |
| Local margin/volume sizing | `calculate_margin_and_volume()` |
| Local broker volume step normalization | `normalize_order_volume()` |
| Local order or position margin estimation | `estimate_order_margin()`, `calculate_positions_margin()` |
| Local closed-bar fetch from a trading session | `fetch_latest_closed_rates_for_trading_client()`, `fetch_latest_closed_rates_indexed()` |
| Local SL/TP price derivation | `determine_order_limits()` |
| Throttled SQLite history loop with ad-hoc error handling | `ThrottledHistoryUpdater(suppress_errors=True)` |
Keep read-only data collection on `mt5_session()` / `MT5Client`; use
`mt5_trading_session()` only where order placement or trading calculations are
required.
+37 -23
View File
@@ -27,28 +27,30 @@ mt5cli provides a stable `MT5Client` Python API, standardized dataset schemas, s
pip install mt5cli
```
Parquet export is not included by default. To enable it, install the `parquet` extra:
```bash
pip install "mt5cli[parquet]"
```
## Python API for downstream packages
Import `MT5Client` for generic MT5 data access, schema normalization, and optional order primitives. `Mt5CliClient` remains available as a backward-compatible alias.
Import `MT5Client` for generic MT5 data access, schema normalization, and optional order primitives.
```python
from datetime import UTC, datetime
from pathlib import Path
from mt5cli import (
DataKind,
Dataset,
MT5Client,
build_config,
collect_history,
export_dataframe,
load_rate_data,
minimum_margins,
mt5_session,
normalize_dataframe,
recent_ticks,
resolve_rate_view_name,
)
from mt5cli.history import load_rate_data, resolve_rate_view_name
from mt5cli.schemas import DataKind, normalize_dataframe
from mt5cli.sdk import minimum_margins, recent_ticks
from mt5cli.utils import Dataset, export_dataframe
# Persistent session for multiple calls
with mt5_session(build_config(login=12345, server="Broker-Demo")) as client:
@@ -84,7 +86,7 @@ collect_history(
)
```
Schema contracts live in `mt5cli.schemas` (`DataKind`, `validate_schema`, `normalize_dataframe`). Storage helpers are re-exported from `mt5cli.storage` and the package root.
Schema contracts live in `mt5cli.schemas` (`DataKind`, `validate_schema`, `normalize_dataframe`). Export and storage helpers are in `mt5cli.utils` (`Dataset`, `export_dataframe`) and `mt5cli.history`.
`MT5Client.order_send()` is a live execution primitive: it can place real trades on the connected account. mt5cli does not implement strategy logic, signal generation, backtesting, or optimization — downstream applications must gate live execution explicitly (the CLI requires `--yes` for `order-send`).
@@ -145,20 +147,32 @@ mt5cli --login 12345 --password mypass --server MyBroker-Demo \
| `minimum-margins` | Export minimum-volume margin summary |
| `market-book` | Export market depth (order book) |
### Trading
### Trading State
| Command | Description |
| ---------------------- | ----------------------------------------------------------- |
| `orders` | Export active orders |
| `positions` | Export open positions |
| `history-orders` | Export historical orders |
| `history-deals` | Export historical deals |
| `recent-history-deals` | Export historical deals from a trailing window |
| `mt5-summary` | Export terminal/account status summary |
| `order-check` | Check funds sufficiency for a trade request |
| `order-send` | Send a trade request to the trade server (`--yes` required) |
| Command | Description |
| ---------------------- | --------------------------------------------------------------------- |
| `orders` | Export active orders |
| `positions` | Export open positions |
| `history-orders` | Export historical orders |
| `history-deals` | Export historical deals |
| `recent-history-deals` | Export historical deals from a trailing window |
| `mt5-summary` | Export terminal/account status summary |
| `order-check` | Check funds sufficiency for a trade request (read-only, no `--yes`) |
Use `order-check` to validate a request payload before running `order-send --yes`.
### Execution (live / mutating)
These commands send requests to the live trade server and can place or close
real trades. Both require `--yes` for live execution.
| Command | Description |
| ----------------- | ---------------------------------------------------------------------------------------------------- |
| `order-send` | Send a **raw** trade request directly to MT5 (`--yes` required; expert path — no extra validation) |
| `close-positions` | Close open positions by `--symbol` or `--ticket` (`--yes` required for live; `--dry-run` to preview) |
Use `order-check` (Trading State) to validate funds before running `order-send --yes`.
`close-positions` is the safer high-level alternative that builds correct close
requests automatically. `order-send` is the expert raw path — downstream
applications should prefer dedicated closing helpers or their own risk controls.
### Bulk Collection
@@ -215,7 +229,7 @@ See the [History schema diagram](api/history.md#entity-relationship-diagram) for
Browse the API documentation for detailed module information:
- [CLI Module](api/cli.md) - CLI application with export commands
- [CLI Module](api/cli.md) - CLI application with data export and execution commands
- [SDK Module](api/sdk.md) - Programmatic read-only data collection API
- [Utils Module](api/utils.md) - Constants, parameter types, parsers, and export utilities
+1 -1
View File
@@ -56,9 +56,9 @@ nav:
- Home: index.md
- API Reference:
- Overview: api/index.md
- Public API Contract: api/public-contract.md
- Client: api/client.md
- Schemas: api/schemas.md
- Storage: api/storage.md
- Converters: api/converters.md
- Exceptions: api/exceptions.md
- CLI: api/cli.md
+79 -139
View File
@@ -1,205 +1,145 @@
"""mt5cli: Generic MT5 data and execution infrastructure for Python applications."""
"""mt5cli: Generic MT5 data and execution infrastructure for Python applications.
Downstream packages should import from this module (``from mt5cli import ...``)
rather than private submodule helpers. See ``docs/api/public-contract.md`` for
the stable SDK contract, CLI surface, internal modules, and out-of-scope
strategy responsibilities.
"""
from importlib.metadata import version
from .client import MT5Client, build_config, mt5_session
from .converters import (
ensure_utc,
granularity_name,
normalize_symbol,
normalize_symbols,
parse_date_range,
recent_window,
)
from .contract import STABLE_SDK_EXPORTS
from .exceptions import (
Mt5CliError,
Mt5ConnectionError,
Mt5OperationError,
Mt5SchemaError,
call_with_normalized_errors,
is_recoverable_mt5_error,
normalize_mt5_exception,
)
from .history import (
RateTarget,
build_rate_targets,
build_rate_view_name,
drop_forming_rate_bar,
load_rate_data,
load_rate_data_from_connection,
load_rate_series_by_granularity,
load_rate_series_from_sqlite,
resolve_history_datasets,
resolve_history_tick_flags,
resolve_history_timeframes,
resolve_rate_tables,
resolve_rate_view_name,
resolve_rate_view_names,
)
from .schemas import (
DEDUP_KEYS,
KNOWN_MT5_TIME_COLUMNS,
REQUIRED_COLUMNS,
TIME_COLUMNS,
DataKind,
normalize_dataframe,
normalize_time_columns,
schema_columns,
validate_schema,
)
from .sdk import (
AccountSpec,
Mt5CliClient,
ThrottledHistoryUpdater,
account_info,
collect_history,
collect_latest_closed_rates_by_granularity,
collect_latest_closed_rates_for_accounts,
collect_latest_rates,
collect_latest_rates_for_accounts,
collect_latest_rates_for_accounts_with_retries,
copy_rates_from,
copy_rates_from_pos,
copy_rates_range,
copy_ticks_from,
copy_ticks_range,
history_deals,
history_orders,
last_error,
latest_rates,
market_book,
minimum_margins,
mt5_summary,
mt5_summary_as_df,
orders,
positions,
recent_history_deals,
recent_ticks,
fetch_latest_closed_rates,
resolve_account_spec,
resolve_account_specs,
substitute_env_placeholders,
symbol_info,
symbol_info_tick,
symbols,
terminal_info,
update_history,
update_history_with_config,
)
from .sdk import (
version as mt5_version,
)
from .storage import (
Dataset,
IfExists,
detect_format,
export_dataframe,
export_dataframe_to_sqlite,
)
from .trading import (
ExecutionStatus,
MarginVolume,
OrderExecutionResult,
OrderFillingMode,
OrderLimits,
OrderSide,
OrderTimeMode,
PositionSide,
ProjectionMode,
calculate_account_projected_margin_ratio,
calculate_margin_and_volume,
calculate_new_position_margin_ratio,
calculate_positions_margin,
calculate_positions_margin_by_symbol,
calculate_positions_margin_safe,
calculate_projected_margin_ratio,
calculate_spread_ratio,
calculate_symbol_group_margin_ratio,
calculate_trailing_stop_updates,
calculate_volume_by_margin,
close_open_positions,
create_trading_client,
detect_position_side,
determine_order_limits,
ensure_symbol_selected,
estimate_order_margin,
extract_tick_price,
fetch_latest_closed_rates_for_trading_client,
fetch_latest_closed_rates_indexed,
get_account_snapshot,
get_positions_frame,
get_symbol_snapshot,
get_tick_snapshot,
mt5_trading_session,
)
from .utils import (
TICK_FLAG_MAP,
TIMEFRAME_MAP,
parse_datetime,
parse_tick_flags,
parse_timeframe,
normalize_order_volume,
place_market_order,
update_sltp_for_open_positions,
update_trailing_stop_loss_for_open_positions,
)
__version__ = version(__package__) if __package__ else None
__all__ = [
"DEDUP_KEYS",
"KNOWN_MT5_TIME_COLUMNS",
"REQUIRED_COLUMNS",
"TICK_FLAG_MAP",
"TIMEFRAME_MAP",
"TIME_COLUMNS",
"STABLE_SDK_EXPORTS",
"AccountSpec",
"DataKind",
"Dataset",
"IfExists",
"ExecutionStatus",
"MT5Client",
"Mt5CliClient",
"MarginVolume",
"Mt5CliError",
"Mt5ConnectionError",
"Mt5OperationError",
"Mt5SchemaError",
"OrderExecutionResult",
"OrderFillingMode",
"OrderLimits",
"OrderSide",
"OrderTimeMode",
"PositionSide",
"ProjectionMode",
"RateTarget",
"ThrottledHistoryUpdater",
"account_info",
"build_config",
"build_rate_targets",
"build_rate_view_name",
"calculate_account_projected_margin_ratio",
"calculate_margin_and_volume",
"call_with_normalized_errors",
"calculate_new_position_margin_ratio",
"calculate_positions_margin",
"calculate_positions_margin_by_symbol",
"calculate_positions_margin_safe",
"calculate_projected_margin_ratio",
"calculate_spread_ratio",
"calculate_symbol_group_margin_ratio",
"calculate_trailing_stop_updates",
"calculate_volume_by_margin",
"close_open_positions",
"collect_history",
"collect_latest_closed_rates_by_granularity",
"collect_latest_closed_rates_for_accounts",
"collect_latest_rates",
"collect_latest_rates_for_accounts",
"collect_latest_rates_for_accounts_with_retries",
"copy_rates_from",
"copy_rates_from_pos",
"copy_rates_range",
"copy_ticks_from",
"copy_ticks_range",
"detect_format",
"create_trading_client",
"detect_position_side",
"determine_order_limits",
"drop_forming_rate_bar",
"ensure_utc",
"export_dataframe",
"export_dataframe_to_sqlite",
"granularity_name",
"history_deals",
"history_orders",
"is_recoverable_mt5_error",
"last_error",
"latest_rates",
"load_rate_data",
"load_rate_data_from_connection",
"ensure_symbol_selected",
"estimate_order_margin",
"extract_tick_price",
"fetch_latest_closed_rates",
"fetch_latest_closed_rates_for_trading_client",
"fetch_latest_closed_rates_indexed",
"get_account_snapshot",
"get_positions_frame",
"get_symbol_snapshot",
"get_tick_snapshot",
"load_rate_series_by_granularity",
"load_rate_series_from_sqlite",
"market_book",
"minimum_margins",
"mt5_session",
"mt5_summary",
"mt5_summary_as_df",
"mt5_trading_session",
"mt5_version",
"normalize_dataframe",
"normalize_mt5_exception",
"normalize_symbol",
"normalize_symbols",
"normalize_time_columns",
"orders",
"parse_date_range",
"parse_datetime",
"parse_tick_flags",
"parse_timeframe",
"positions",
"recent_history_deals",
"recent_ticks",
"recent_window",
"normalize_order_volume",
"place_market_order",
"resolve_account_spec",
"resolve_account_specs",
"resolve_history_datasets",
"resolve_history_tick_flags",
"resolve_history_timeframes",
"resolve_rate_tables",
"resolve_rate_view_name",
"resolve_rate_view_names",
"schema_columns",
"substitute_env_placeholders",
"symbol_info",
"symbol_info_tick",
"symbols",
"terminal_info",
"update_history",
"update_history_with_config",
"validate_schema",
"update_sltp_for_open_positions",
"update_trailing_stop_loss_for_open_positions",
]
+133 -31
View File
@@ -1,18 +1,21 @@
"""Command-line interface for MetaTrader 5 data export."""
"""Command-line interface for MetaTrader 5 data and execution utilities."""
from __future__ import annotations
import json
import logging
from dataclasses import dataclass
from datetime import datetime # noqa: TC003
from pathlib import Path # noqa: TC003
from typing import TYPE_CHECKING, Annotated, Any, cast
import pandas as pd
import typer
from pdmt5 import Mt5Config
from . import sdk
from .client import MT5Client
from .trading import OrderExecutionResult, close_open_positions, create_trading_client
from .utils import (
DATETIME_TYPE,
REQUEST_TYPE,
@@ -29,8 +32,6 @@ from .utils import (
if TYPE_CHECKING:
from collections.abc import Callable
import pandas as pd
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
@@ -54,7 +55,12 @@ class _ExportContext:
app = typer.Typer(
name="mt5cli",
help="Export MetaTrader5 data to CSV, JSON, Parquet, or SQLite3.",
help=(
"MT5 data and execution utilities — read market data, inspect account"
" state, and send trade requests. Data commands write to CSV, JSON,"
" Parquet, or SQLite3. Execution commands (order-send, close-positions)"
" require --yes for live mutations."
),
)
_REQUEST_OPTION_HELP = (
@@ -150,7 +156,7 @@ def _callback( # pyright: ignore[reportUnusedFunction]
typer.Option("--log-level", help="Logging level."),
] = LogLevel.WARNING,
) -> None:
"""Configure shared options for all export commands.
"""Configure shared connection and output options.
Raises:
typer.BadParameter: If the output format cannot be determined.
@@ -182,7 +188,7 @@ def _callback( # pyright: ignore[reportUnusedFunction]
# ---------------------------------------------------------------------------
@app.command()
@app.command(rich_help_panel="Data / Export")
def rates_from(
ctx: typer.Context,
symbol: Annotated[str, typer.Option(help="Symbol name.")],
@@ -209,7 +215,7 @@ def rates_from(
)
@app.command()
@app.command(rich_help_panel="Data / Export")
def rates_from_pos(
ctx: typer.Context,
symbol: Annotated[str, typer.Option(help="Symbol name.")],
@@ -235,7 +241,7 @@ def rates_from_pos(
)
@app.command()
@app.command(rich_help_panel="Data / Export")
def latest_rates(
ctx: typer.Context,
symbol: Annotated[str, typer.Option(help="Symbol name.")],
@@ -264,7 +270,7 @@ def latest_rates(
)
@app.command()
@app.command(rich_help_panel="Data / Export")
def rates_range(
ctx: typer.Context,
symbol: Annotated[str, typer.Option(help="Symbol name.")],
@@ -291,7 +297,7 @@ def rates_range(
)
@app.command()
@app.command(rich_help_panel="Data / Export")
def ticks_from(
ctx: typer.Context,
symbol: Annotated[str, typer.Option(help="Symbol name.")],
@@ -315,7 +321,7 @@ def ticks_from(
)
@app.command()
@app.command(rich_help_panel="Data / Export")
def ticks_range(
ctx: typer.Context,
symbol: Annotated[str, typer.Option(help="Symbol name.")],
@@ -339,7 +345,7 @@ def ticks_range(
)
@app.command()
@app.command(rich_help_panel="Data / Export")
def ticks_recent(
ctx: typer.Context,
symbol: Annotated[str, typer.Option(help="Symbol name.")],
@@ -376,19 +382,19 @@ def ticks_recent(
)
@app.command()
@app.command(rich_help_panel="Data / Export")
def account_info(ctx: typer.Context) -> None:
"""Export account information."""
_export_command(ctx, lambda client: client.account_info())
@app.command()
@app.command(rich_help_panel="Data / Export")
def terminal_info(ctx: typer.Context) -> None:
"""Export terminal information."""
_export_command(ctx, lambda client: client.terminal_info())
@app.command()
@app.command(rich_help_panel="Data / Export")
def symbols(
ctx: typer.Context,
group: Annotated[
@@ -400,7 +406,7 @@ def symbols(
_export_command(ctx, lambda client: client.symbols(group=group))
@app.command()
@app.command(rich_help_panel="Data / Export")
def symbol_info(
ctx: typer.Context,
symbol: Annotated[str, typer.Option(help="Symbol name.")],
@@ -409,7 +415,7 @@ def symbol_info(
_export_command(ctx, lambda client: client.symbol_info(symbol))
@app.command()
@app.command(rich_help_panel="Data / Export")
def minimum_margins(
ctx: typer.Context,
symbol: Annotated[str, typer.Option(help="Symbol name.")],
@@ -418,7 +424,7 @@ def minimum_margins(
_export_command(ctx, lambda client: client.minimum_margins(symbol))
@app.command()
@app.command(rich_help_panel="Data / Export")
def orders(
ctx: typer.Context,
symbol: Annotated[str | None, typer.Option(help="Symbol filter.")] = None,
@@ -432,7 +438,7 @@ def orders(
)
@app.command()
@app.command(rich_help_panel="Data / Export")
def positions(
ctx: typer.Context,
symbol: Annotated[str | None, typer.Option(help="Symbol filter.")] = None,
@@ -446,7 +452,7 @@ def positions(
)
@app.command()
@app.command(rich_help_panel="Data / Export")
def history_orders(
ctx: typer.Context,
date_from: Annotated[
@@ -476,7 +482,7 @@ def history_orders(
)
@app.command()
@app.command(rich_help_panel="Data / Export")
def history_deals(
ctx: typer.Context,
date_from: Annotated[
@@ -506,7 +512,7 @@ def history_deals(
)
@app.command()
@app.command(rich_help_panel="Data / Export")
def recent_history_deals(
ctx: typer.Context,
hours: Annotated[float, typer.Option(help="Lookback window in hours.")],
@@ -529,25 +535,25 @@ def recent_history_deals(
)
@app.command()
@app.command(rich_help_panel="Data / Export")
def mt5_summary(ctx: typer.Context) -> None:
"""Export a compact terminal/account status summary."""
_export_command(ctx, lambda client: client.mt5_summary_as_df())
@app.command()
@app.command(rich_help_panel="Data / Export")
def version(ctx: typer.Context) -> None:
"""Export MetaTrader5 version information."""
_export_command(ctx, lambda client: client.version())
@app.command()
@app.command(rich_help_panel="Data / Export")
def last_error(ctx: typer.Context) -> None:
"""Export the last error information."""
_export_command(ctx, lambda client: client.last_error())
@app.command()
@app.command(rich_help_panel="Data / Export")
def symbol_info_tick(
ctx: typer.Context,
symbol: Annotated[str, typer.Option(help="Symbol name.")],
@@ -556,7 +562,7 @@ def symbol_info_tick(
_export_command(ctx, lambda client: client.symbol_info_tick(symbol))
@app.command()
@app.command(rich_help_panel="Data / Export")
def market_book(
ctx: typer.Context,
symbol: Annotated[str, typer.Option(help="Symbol name.")],
@@ -565,7 +571,7 @@ def market_book(
_export_command(ctx, lambda client: client.market_book(symbol))
@app.command()
@app.command(rich_help_panel="Data / Export")
def order_check(
ctx: typer.Context,
request: Annotated[
@@ -577,7 +583,7 @@ def order_check(
_export_command(ctx, lambda client: client.order_check(request))
@app.command()
@app.command(rich_help_panel="Execution")
def order_send(
ctx: typer.Context,
request: Annotated[
@@ -589,7 +595,13 @@ def order_send(
typer.Option("--yes", help="Confirm the live trade request."),
] = False,
) -> None:
"""Send a trading operation request to the trade server.
"""Send a raw trade request to the trade server (expert path, live execution).
Passes the request JSON directly to MT5 ``order_send``. This is the
low-level expert path — it places real trades on the connected account
with no additional validation beyond what MT5 itself performs. Use
``order-check`` first to validate funds sufficiency. Prefer
``close-positions`` for closing open positions. ``--yes`` is required.
Raises:
typer.BadParameter: If --yes is not provided.
@@ -600,7 +612,97 @@ def order_send(
_export_command(ctx, lambda client: client.order_send(request))
@app.command()
_EXECUTION_RESULT_COLUMNS: list[str] = [
"status",
"symbol",
"order_side",
"volume",
"retcode",
"comment",
"request",
"response",
"dry_run",
]
def _execution_results_to_df(results: list[OrderExecutionResult]) -> pd.DataFrame:
if not results:
return pd.DataFrame(columns=_EXECUTION_RESULT_COLUMNS)
rows = [
{
**r,
"request": json.dumps(r["request"]),
"response": json.dumps(r["response"]),
}
for r in results
]
return pd.DataFrame(rows)
@app.command(rich_help_panel="Execution")
def close_positions(
ctx: typer.Context,
symbol: Annotated[
list[str] | None,
typer.Option(
"--symbol",
"-s",
help="Symbol to close (repeat for multiple symbols).",
),
] = None,
ticket: Annotated[
list[int] | None,
typer.Option(
"--ticket",
"-t",
help="Position ticket to close (repeat for multiple tickets).",
),
] = None,
dry_run: Annotated[
bool,
typer.Option("--dry-run", help="Preview close orders without executing them."),
] = False,
yes: Annotated[
bool,
typer.Option("--yes", help="Confirm live position closing."),
] = False,
) -> None:
"""Close open positions by symbol or ticket.
Delegates to :func:`mt5cli.trading.close_open_positions`. At least one
``--symbol`` or ``--ticket`` must be provided to avoid accidentally closing
all positions. Use ``--dry-run`` to preview without executing; ``--yes`` is
required for live execution.
``order-send`` is the expert raw-request path. ``close-positions`` is the
safer high-level helper that builds correct close requests automatically.
Raises:
typer.BadParameter: If neither ``--symbol`` nor ``--ticket`` is given,
or if ``--yes`` is missing for a live (non-dry-run) run.
"""
if not symbol and not ticket:
msg = "Provide at least one --symbol or --ticket to close positions."
raise typer.BadParameter(msg)
if not dry_run and not yes:
msg = "Pass --yes to close live positions."
raise typer.BadParameter(msg, param_hint="--yes")
export_ctx = _get_export_context(ctx)
client = create_trading_client(config=export_ctx.config)
try:
results = close_open_positions(
client,
symbols=list(symbol) if symbol else None,
tickets=list(ticket) if ticket else None,
dry_run=dry_run,
)
finally:
client.shutdown()
df = _execution_results_to_df(results)
_execute_export(ctx, lambda: df)
@app.command(rich_help_panel="Collection")
def collect_history(
ctx: typer.Context,
symbol: Annotated[
+1 -3
View File
@@ -24,9 +24,7 @@ class MT5Client(Mt5CliClient):
"""Public client for generic MT5 data access and order primitives.
Extends the read-only SDK client with optional order check/send helpers and
exposes the same connection lifecycle as :class:`~mt5cli.sdk.Mt5CliClient`.
Downstream applications such as private trading packages should prefer this
type over the legacy ``Mt5CliClient`` name.
exposes the same connection lifecycle as :func:`mt5_session`.
mt5cli intentionally exposes minimal execution primitives only. Trading
decisions, signals, strategies, backtests, and optimization remain the
+69
View File
@@ -0,0 +1,69 @@
"""Downstream SDK export tier for mt5cli."""
from __future__ import annotations
STABLE_SDK_EXPORTS: frozenset[str] = frozenset({
"AccountSpec",
"MT5Client",
"Mt5CliError",
"Mt5ConnectionError",
"Mt5OperationError",
"Mt5SchemaError",
"OrderFillingMode",
"OrderSide",
"OrderTimeMode",
"PositionSide",
"ProjectionMode",
"ExecutionStatus",
"MarginVolume",
"OrderExecutionResult",
"OrderLimits",
"RateTarget",
"ThrottledHistoryUpdater",
"build_config",
"build_rate_targets",
"calculate_account_projected_margin_ratio",
"calculate_margin_and_volume",
"calculate_new_position_margin_ratio",
"calculate_projected_margin_ratio",
"calculate_positions_margin",
"calculate_positions_margin_by_symbol",
"calculate_positions_margin_safe",
"calculate_spread_ratio",
"calculate_symbol_group_margin_ratio",
"calculate_trailing_stop_updates",
"calculate_volume_by_margin",
"close_open_positions",
"collect_history",
"collect_latest_closed_rates_by_granularity",
"collect_latest_closed_rates_for_accounts",
"collect_latest_rates_for_accounts_with_retries",
"create_trading_client",
"detect_position_side",
"determine_order_limits",
"drop_forming_rate_bar",
"ensure_symbol_selected",
"estimate_order_margin",
"extract_tick_price",
"fetch_latest_closed_rates",
"fetch_latest_closed_rates_for_trading_client",
"fetch_latest_closed_rates_indexed",
"get_account_snapshot",
"get_positions_frame",
"get_symbol_snapshot",
"get_tick_snapshot",
"load_rate_series_by_granularity",
"load_rate_series_from_sqlite",
"mt5_session",
"mt5_trading_session",
"normalize_order_volume",
"place_market_order",
"resolve_account_spec",
"resolve_account_specs",
"update_history",
"update_history_with_config",
"update_sltp_for_open_positions",
"update_trailing_stop_loss_for_open_positions",
})
__all__ = ["STABLE_SDK_EXPORTS"]
+9 -4
View File
@@ -4,11 +4,16 @@ from __future__ import annotations
from typing import TYPE_CHECKING, TypeVar
from pdmt5 import Mt5RuntimeError, Mt5TradingError
from pdmt5 import Mt5RuntimeError
if TYPE_CHECKING:
from collections.abc import Callable
try:
from pdmt5 import Mt5TradingError
except ImportError: # pragma: no cover
Mt5TradingError = None # type: ignore[assignment]
T = TypeVar("T")
__all__ = [
@@ -22,7 +27,7 @@ __all__ = [
]
_RECOVERABLE_MT5_ERRORS: tuple[type[BaseException], ...] = (
Mt5TradingError,
*([Mt5TradingError] if Mt5TradingError is not None else []), # type: ignore[misc]
Mt5RuntimeError,
)
@@ -50,7 +55,7 @@ def is_recoverable_mt5_error(exc: BaseException) -> bool:
exc: Exception raised by MT5 or pdmt5.
Returns:
True for ``Mt5RuntimeError`` and ``Mt5TradingError``.
True for ``Mt5RuntimeError`` and ``Mt5TradingError`` (if available).
"""
return isinstance(exc, _RECOVERABLE_MT5_ERRORS)
@@ -65,7 +70,7 @@ def normalize_mt5_exception(exc: BaseException) -> Mt5CliError:
``Mt5ConnectionError`` for runtime failures, ``Mt5OperationError`` for
trading failures, or the original exception when it is not recognized.
"""
if isinstance(exc, Mt5TradingError):
if Mt5TradingError is not None and isinstance(exc, Mt5TradingError):
return Mt5OperationError(str(exc))
if isinstance(exc, Mt5RuntimeError):
return Mt5ConnectionError(str(exc))
+72 -10
View File
@@ -7,7 +7,7 @@ import sqlite3
from dataclasses import dataclass
from datetime import UTC, datetime
from pathlib import Path
from typing import TYPE_CHECKING, Literal, cast
from typing import TYPE_CHECKING, Literal, cast, overload
import pandas as pd
from pdmt5 import get_timeframe_name as _get_timeframe_name
@@ -142,6 +142,26 @@ def build_rate_view_name(
return f"rate_{symbol}__{granularity}_{timeframe}"
def resolve_rate_table_name(symbol: str, granularity: str) -> str:
"""Return the canonical normalized SQLite rate table name.
The normalized history table stores all symbols and timeframes in
``rates``; use :func:`resolve_rate_view_name` for per-symbol compatibility
view names.
Returns:
Canonical normalized rates table name.
Raises:
ValueError: If ``symbol`` or ``granularity`` is invalid.
"""
parse_timeframe(granularity)
if not symbol.strip():
msg = "symbol must not be empty."
raise ValueError(msg)
return Dataset.rates.table_name
SqliteConnOrPath = sqlite3.Connection | Path | str
@@ -653,34 +673,76 @@ def resolve_rate_tables(
conn.close()
if TYPE_CHECKING:
@overload
def load_rate_series_from_sqlite(
conn_or_path: SqliteConnOrPath,
targets: None = None,
count: int | None = None,
explicit_tables: None = None,
*,
table: str,
) -> pd.DataFrame: ...
@overload
def load_rate_series_from_sqlite(
conn_or_path: SqliteConnOrPath,
targets: None = None,
count: int | None = None,
explicit_tables: Sequence[str] | None = None,
*,
table: None = None,
) -> dict[tuple[str | None, int], pd.DataFrame]: ...
@overload
def load_rate_series_from_sqlite(
conn_or_path: SqliteConnOrPath,
targets: Sequence[RateTarget],
count: int,
explicit_tables: Sequence[str] | None = None,
*,
table: None = None,
) -> dict[tuple[str | None, int], pd.DataFrame]: ...
def load_rate_series_from_sqlite(
conn_or_path: SqliteConnOrPath,
targets: Sequence[RateTarget],
count: int,
targets: Sequence[RateTarget] | None = None,
count: int | None = None,
explicit_tables: Sequence[str] | None = None,
) -> dict[tuple[str | None, int], pd.DataFrame]:
"""Load multiple rate series from a SQLite database.
*,
table: str | None = None,
) -> dict[tuple[str | None, int], pd.DataFrame] | pd.DataFrame:
"""Load one table/view or multiple rate series from a SQLite database.
Args:
conn_or_path: SQLite database path or open connection.
targets: Rate targets to load. Each ``(symbol, timeframe_int)`` pair
must be unique.
count: Number of most recent rows to load per series.
targets: Rate targets to load. Each ``(symbol, timeframe_int)`` pair must
be unique. Omit when loading a single explicit ``table``.
count: Optional number of most recent rows to load per series.
explicit_tables: Optional explicit table or view names matching targets.
When omitted, managed ``rate_*`` compatibility views must already
exist in the database.
table: Optional single table or view name to load directly.
Returns:
Mapping keyed by ``(symbol, timeframe_int)`` to each rate DataFrame.
A DataFrame when ``table`` is provided, otherwise a mapping keyed by
``(symbol, timeframe_int)`` to each rate DataFrame.
Raises:
ValueError: If ``count`` is not positive, targets are empty, duplicate
``(symbol, timeframe_int)`` pairs are present, or table resolution
fails.
"""
if count <= 0:
if table is not None:
return load_rate_data(conn_or_path, table, count=count)
if count is None or count <= 0:
msg = "count must be positive."
raise ValueError(msg)
if targets is None:
msg = "targets are required when table is not provided."
raise ValueError(msg)
target_list = list(targets)
if not target_list:
msg = "At least one rate target is required."
+216 -33
View File
@@ -15,7 +15,12 @@ from pathlib import Path
from typing import TYPE_CHECKING, Self, TypeVar, cast
import pandas as pd
from pdmt5 import Mt5Config, Mt5DataClient, Mt5RuntimeError, Mt5TradingError
from pdmt5 import Mt5Config, Mt5DataClient, Mt5RuntimeError
try:
from pdmt5 import Mt5TradingError
except ImportError: # pragma: no cover
Mt5TradingError = None # type: ignore[assignment]
from .history import (
create_cash_events_view,
@@ -37,16 +42,19 @@ from .utils import (
parse_tick_flags,
parse_timeframe,
)
from .utils import coerce_login as _coerce_login
if TYPE_CHECKING:
from collections.abc import Callable, Iterator, Sequence
from collections.abc import Callable, Collection, Iterator, Sequence
UpdateHistoryBackend = Callable[..., None]
T = TypeVar("T")
logger = logging.getLogger(__name__)
_RECOVERABLE_HISTORY_UPDATE_ERRORS: tuple[type[BaseException], ...] = (
Mt5TradingError,
*([Mt5TradingError] if Mt5TradingError is not None else []), # type: ignore[assignment]
Mt5RuntimeError,
sqlite3.Error,
ValueError,
@@ -122,6 +130,7 @@ __all__ = [
"copy_rates_range",
"copy_ticks_from",
"copy_ticks_range",
"fetch_latest_closed_rates",
"history_deals",
"history_orders",
"last_error",
@@ -138,6 +147,7 @@ __all__ = [
"resolve_account_spec",
"resolve_account_specs",
"substitute_env_placeholders",
"substitute_mapping_values",
"symbol_info",
"symbol_info_tick",
"symbols",
@@ -301,19 +311,47 @@ def _fetch_minimum_margins(client: Mt5DataClient, symbol: str) -> pd.DataFrame:
def build_config(
*,
path: str | None = None,
login: int | None = None,
login: int | str | None = None,
password: str | None = None,
server: str | None = None,
timeout: int | None = None,
allow_whole_dollar_env: bool = False,
) -> Mt5Config:
"""Build an ``Mt5Config`` from optional connection parameters.
Args:
path: Optional terminal executable path.
login: Optional trading account login. Integers are preserved. String
values are coerced: empty or whitespace-only strings become
``None``; numeric strings such as ``"12345"`` are converted to
``int``; non-numeric strings raise ``ValueError``. When
``allow_whole_dollar_env=True``, ``$ENV_NAME`` and
``${ENV_NAME}`` placeholders are expanded before coercion.
password: Optional trading account password.
server: Optional trading server name.
timeout: Optional connection timeout in milliseconds.
allow_whole_dollar_env: When ``True``, string parameters that are
exactly ``$ENV_NAME`` are expanded from the environment. Applies
to ``path``, ``login``, ``password``, and ``server``. Default
``False`` preserves existing behavior.
Returns:
Configured ``Mt5Config`` instance.
"""
if allow_whole_dollar_env:
if path is not None:
path = substitute_env_placeholders(path, allow_whole_dollar_env=True)
if isinstance(login, str):
login = substitute_env_placeholders(login, allow_whole_dollar_env=True)
if password is not None:
password = substitute_env_placeholders(
password, allow_whole_dollar_env=True
)
if server is not None:
server = substitute_env_placeholders(server, allow_whole_dollar_env=True)
return Mt5Config(
path=path,
login=login,
login=_coerce_login(login),
password=password,
server=server,
timeout=timeout,
@@ -1042,10 +1080,16 @@ def update_history_with_config( # noqa: PLR0913
class ThrottledHistoryUpdater:
"""Throttled incremental SQLite history updater for long-running apps.
Wraps :func:`update_history` with a minimum interval between successful
updates, so a tight application loop can call :meth:`update` every
iteration without re-fetching MT5 history more often than desired. Timing
uses a monotonic clock, so it is unaffected by wall-clock changes.
Wraps :func:`update_history` (or a custom ``update_backend``) with a minimum
interval between successful updates, so a tight application loop can call
:meth:`update` every iteration without re-fetching MT5 history more often
than desired. Timing uses a monotonic clock, so it is unaffected by
wall-clock changes.
Downstream applications may pass ``update_backend`` to substitute the
default :func:`update_history` implementationfor example to add
application-specific logging, metrics, or test doubleswithout monkey-
patching ``mt5cli.sdk.update_history``.
"""
def __init__(
@@ -1060,6 +1104,7 @@ class ThrottledHistoryUpdater:
include_account_events: bool = True,
interval_seconds: float = 0.0,
suppress_errors: bool = False,
update_backend: UpdateHistoryBackend | None = None,
) -> None:
"""Initialize the throttled updater.
@@ -1083,6 +1128,12 @@ class ThrottledHistoryUpdater:
the throttle. Other ``AttributeError`` / ``TypeError`` values
always propagate. When False (default), recoverable errors
propagate so callers control logging.
update_backend: Callable invoked instead of :func:`update_history`
when :meth:`update` runs. Receives the same keyword arguments as
:func:`update_history` (``client``, ``output``, ``symbols``,
``datasets``, ``timeframes``, ``flags``, ``lookback_hours``,
``with_views``, ``include_account_events``). Defaults to
:func:`update_history`.
"""
self.output = output
self.datasets = datasets
@@ -1093,6 +1144,9 @@ class ThrottledHistoryUpdater:
self.include_account_events = include_account_events
self.interval_seconds = interval_seconds
self.suppress_errors = suppress_errors
self.update_backend = (
update_history if update_backend is None else update_backend
)
self._last_update_monotonic: float | None = None
@property
@@ -1143,7 +1197,7 @@ class ThrottledHistoryUpdater:
lookback_hours=self.lookback_hours,
date_to=None,
)
update_history(
self.update_backend(
client=client,
output=self.output,
symbols=symbols,
@@ -1289,6 +1343,33 @@ def latest_rates(
)
def fetch_latest_closed_rates(
client: Mt5CliClient,
*,
symbol: str,
granularity: str,
count: int,
) -> pd.DataFrame:
"""Fetch up to ``count`` most recent closed bars, oldest to newest.
Returns:
Closed rate bars ordered oldest to newest.
Raises:
ValueError: If ``count`` is not positive or no closed bars are returned.
"""
_require_positive(count, "count")
frame = client.latest_rates(symbol, granularity, count + 1, start_pos=0)
closed = drop_forming_rate_bar(frame)
if closed.empty:
msg = (
f"Rate data is empty for {symbol!r} at granularity {granularity!r} "
f"with count {count}."
)
raise ValueError(msg)
return closed.tail(count).reset_index(drop=True)
def collect_latest_rates(
symbols: Sequence[str],
timeframes: Sequence[int | str],
@@ -1329,13 +1410,22 @@ class AccountSpec:
_ENV_PLACEHOLDER_PATTERN = re.compile(r"\$\{(?P<name>[A-Za-z_][A-Za-z0-9_]*)\}")
_WHOLE_DOLLAR_PATTERN = re.compile(r"^\$(?P<name>[A-Za-z_][A-Za-z0-9_]*)$")
def substitute_env_placeholders(value: str) -> str:
def substitute_env_placeholders(
value: str,
*,
allow_whole_dollar_env: bool = False,
) -> str:
"""Replace ``${ENV_VAR}`` placeholders in a string with environment values.
Args:
value: String that may contain one or more ``${ENV_VAR}`` placeholders.
allow_whole_dollar_env: When ``True``, a string that is exactly
``$ENV_NAME`` (the whole value and nothing else) is also expanded
from the environment. Partial occurrences such as ``"plan$pass"``
or ``"$ENV-suffix"`` are left unchanged.
Returns:
The string with every placeholder replaced by its environment value.
@@ -1343,6 +1433,14 @@ def substitute_env_placeholders(value: str) -> str:
Raises:
ValueError: If a referenced environment variable is not set.
"""
if allow_whole_dollar_env:
m = _WHOLE_DOLLAR_PATTERN.match(value)
if m:
name = m.group("name")
if name not in os.environ:
msg = f"Environment variable {name!r} is not set."
raise ValueError(msg)
return os.environ[name]
parts: list[str] = []
last_end = 0
for match in _ENV_PLACEHOLDER_PATTERN.finditer(value):
@@ -1357,7 +1455,81 @@ def substitute_env_placeholders(value: str) -> str:
return "".join(parts)
def _resolve_field(override: str | None, account_value: str | None) -> str | None:
def substitute_mapping_values(
data: object,
*,
keys: Collection[str],
allow_whole_dollar_env: bool = False,
blank_string_keys_as_none: Collection[str] = (),
) -> object:
"""Recursively substitute environment placeholders for selected mapping keys.
Traverses nested dicts and lists, expanding ``${ENV_VAR}`` (and
``$ENV_NAME`` when ``allow_whole_dollar_env=True``) in string values
whose immediate parent dict key is in ``keys``. Fields whose key is
not in ``keys`` are preserved exactly, including literal dollar signs.
Strings that are direct elements of a list are never substituted;
substitution only applies to strings that are immediate dict values.
This is a generic downstream config utility. Key names such as
``mt5_login`` or ``mt5_password`` must be supplied by the caller;
mt5cli does not hard-code any application-specific key names.
Callers are responsible for ensuring ``data`` has bounded nesting depth;
deeply nested or self-referential structures will hit Python's recursion
limit.
Args:
data: Arbitrarily nested dict/list/scalar value to process.
keys: Mapping keys whose string values receive placeholder
substitution.
allow_whole_dollar_env: When ``True``, a string that is exactly
``$ENV_NAME`` (whole value) is also expanded from the
environment in addition to ``${ENV_NAME}`` placeholders.
Default ``False`` expands ``${ENV_NAME}`` only.
blank_string_keys_as_none: Mapping keys for which blank strings
(after any substitution) are normalised to ``None``. A key
may appear in ``blank_string_keys_as_none`` without also
appearing in ``keys``.
Returns:
The processed value. Dicts and lists are rebuilt into new
containers with selected string values substituted and
blank-normalised. Scalar inputs (non-dict, non-list) are
returned as-is.
"""
keys_set: frozenset[str] = frozenset(keys)
blank_keys_set: frozenset[str] = frozenset(blank_string_keys_as_none)
def _visit(node: object, current_key: str | None) -> object:
if isinstance(node, dict):
typed = cast("dict[object, object]", node)
return {
k: _visit(v, k if isinstance(k, str) else None)
for k, v in typed.items()
}
if isinstance(node, list):
typed_list = cast("list[object]", node)
return [_visit(item, None) for item in typed_list]
if not isinstance(node, str):
return node
text = node
if current_key in keys_set:
text = substitute_env_placeholders(
node, allow_whole_dollar_env=allow_whole_dollar_env
)
if current_key in blank_keys_set and not text.strip():
return None
return text
return _visit(data, None)
def _resolve_field(
override: str | None,
account_value: str | None,
*,
allow_whole_dollar_env: bool = False,
) -> str | None:
"""Resolve a string field from an override or account value with env subst.
Returns:
@@ -1367,12 +1539,16 @@ def _resolve_field(override: str | None, account_value: str | None) -> str | Non
value = override if override is not None else account_value
if value is None:
return None
return substitute_env_placeholders(value)
return substitute_env_placeholders(
value, allow_whole_dollar_env=allow_whole_dollar_env
)
def _resolve_login(
override: int | str | None,
account_login: int | str | None,
*,
allow_whole_dollar_env: bool = False,
) -> int | str | None:
"""Resolve a login from an override or account value with env substitution.
@@ -1384,10 +1560,14 @@ def _resolve_login(
if override is not None:
if isinstance(override, int):
return override
return substitute_env_placeholders(override)
return substitute_env_placeholders(
override, allow_whole_dollar_env=allow_whole_dollar_env
)
if account_login is None or isinstance(account_login, int):
return account_login
return substitute_env_placeholders(account_login)
return substitute_env_placeholders(
account_login, allow_whole_dollar_env=allow_whole_dollar_env
)
def resolve_account_spec(
@@ -1398,6 +1578,7 @@ def resolve_account_spec(
server: str | None = None,
path: str | None = None,
timeout: int | None = None,
allow_whole_dollar_env: bool = False,
) -> AccountSpec:
"""Resolve an account's credentials from overrides and ``${ENV_VAR}`` values.
@@ -1413,6 +1594,9 @@ def resolve_account_spec(
server: Optional explicit server override.
path: Optional explicit terminal path override.
timeout: Optional explicit connection timeout override.
allow_whole_dollar_env: When ``True``, string fields that are exactly
``$ENV_NAME`` are also expanded from the environment. Default
``False`` preserves existing behavior.
Returns:
A new :class:`AccountSpec` with resolved credentials and the original
@@ -1422,10 +1606,18 @@ def resolve_account_spec(
"""
return AccountSpec(
symbols=account.symbols,
login=_resolve_login(login, account.login),
password=_resolve_field(password, account.password),
server=_resolve_field(server, account.server),
path=_resolve_field(path, account.path),
login=_resolve_login(
login, account.login, allow_whole_dollar_env=allow_whole_dollar_env
),
password=_resolve_field(
password, account.password, allow_whole_dollar_env=allow_whole_dollar_env
),
server=_resolve_field(
server, account.server, allow_whole_dollar_env=allow_whole_dollar_env
),
path=_resolve_field(
path, account.path, allow_whole_dollar_env=allow_whole_dollar_env
),
timeout=timeout if timeout is not None else account.timeout,
)
@@ -1438,6 +1630,7 @@ def resolve_account_specs(
server: str | None = None,
path: str | None = None,
timeout: int | None = None,
allow_whole_dollar_env: bool = False,
) -> list[AccountSpec]:
"""Resolve credentials for multiple accounts.
@@ -1451,6 +1644,9 @@ def resolve_account_specs(
server: Optional explicit server override applied to each account.
path: Optional explicit terminal path override applied to each account.
timeout: Optional explicit timeout override applied to each account.
allow_whole_dollar_env: When ``True``, string fields that are exactly
``$ENV_NAME`` are also expanded from the environment. Default
``False`` preserves existing behavior.
Returns:
Resolved account specifications in the original order. Raises
@@ -1465,25 +1661,12 @@ def resolve_account_specs(
server=server,
path=path,
timeout=timeout,
allow_whole_dollar_env=allow_whole_dollar_env,
)
for account in accounts
]
def _coerce_login(login: int | str | None) -> int | None:
"""Coerce a login value to int, treating empty strings as unset.
Returns:
Integer login, or None when unset or an empty string.
"""
if login is None or isinstance(login, int):
return login
text = login.strip()
if not text:
return None
return int(text)
def _build_account_config(
account: AccountSpec,
base_config: Mt5Config | None,
-49
View File
@@ -1,49 +0,0 @@
"""Generic storage helpers for MT5 market and account history."""
from __future__ import annotations
from .history import (
RateTarget,
build_rate_targets,
build_rate_view_name,
drop_forming_rate_bar,
load_rate_data,
load_rate_data_from_connection,
load_rate_series_by_granularity,
load_rate_series_from_sqlite,
resolve_rate_tables,
resolve_rate_view_name,
resolve_rate_view_names,
)
from .sdk import collect_history, update_history, update_history_with_config
from .utils import (
Dataset,
IfExists,
OutputFormat,
detect_format,
export_dataframe,
export_dataframe_to_sqlite,
)
__all__ = [
"Dataset",
"IfExists",
"OutputFormat",
"RateTarget",
"build_rate_targets",
"build_rate_view_name",
"collect_history",
"detect_format",
"drop_forming_rate_bar",
"export_dataframe",
"export_dataframe_to_sqlite",
"load_rate_data",
"load_rate_data_from_connection",
"load_rate_series_by_granularity",
"load_rate_series_from_sqlite",
"resolve_rate_tables",
"resolve_rate_view_name",
"resolve_rate_view_names",
"update_history",
"update_history_with_config",
]
+1624 -52
View File
File diff suppressed because it is too large Load Diff
+27 -6
View File
@@ -10,7 +10,8 @@ from pathlib import Path
from typing import TYPE_CHECKING, Any, TypeGuard
import click
from pdmt5 import COPY_TICKS_MAP, TIMEFRAME_MAP
from pdmt5 import COPY_TICKS_MAP as _COPY_TICKS_MAP
from pdmt5 import TIMEFRAME_MAP as _TIMEFRAME_MAP
from pdmt5 import parse_copy_ticks as _parse_copy_ticks
from pdmt5 import parse_timeframe as _parse_timeframe
@@ -23,14 +24,11 @@ if TYPE_CHECKING:
# Constants
# ---------------------------------------------------------------------------
# Backward-compatible snapshot; prefer ``COPY_TICKS_MAP`` from pdmt5 directly.
TICK_FLAG_MAP: dict[str, int] = dict(COPY_TICKS_MAP)
TIMEFRAME_NAMES: tuple[str, ...] = tuple(
name for name in TIMEFRAME_MAP if not name.startswith("TIMEFRAME_")
name for name in _TIMEFRAME_MAP if not name.startswith("TIMEFRAME_")
)
_TICK_FLAG_NAMES: tuple[str, ...] = tuple(
name for name in COPY_TICKS_MAP if not name.startswith("COPY_TICKS_")
name for name in _COPY_TICKS_MAP if not name.startswith("COPY_TICKS_")
)
_FORMAT_EXTENSIONS: dict[str, str] = {
@@ -241,6 +239,20 @@ def detect_format(
raise ValueError(msg)
def coerce_login(login: int | str | None) -> int | None:
"""Coerce a login value to int, treating empty strings as unset.
Returns:
Integer login, or None when unset or an empty string.
"""
if login is None or isinstance(login, int):
return login
text = login.strip()
if not text:
return None
return int(text)
def export_dataframe_to_sqlite(
df: pd.DataFrame,
output_path: Path,
@@ -300,6 +312,7 @@ def export_dataframe(
table_name: Table name for SQLite3 output.
Raises:
ImportError: If the parquet format is requested but pyarrow is not installed.
ValueError: If the output format is not supported.
"""
if output_format == "csv":
@@ -312,6 +325,14 @@ def export_dataframe(
indent=2,
)
elif output_format == "parquet":
try:
__import__("pyarrow")
except ImportError as exc:
msg = (
"Parquet export requires the optional dependency pyarrow. "
'Install it with: pip install "mt5cli[parquet]"'
)
raise ImportError(msg) from exc
df.to_parquet(output_path, index=False)
elif output_format == "sqlite3":
export_dataframe_to_sqlite(
+10 -5
View File
@@ -1,6 +1,6 @@
[project]
name = "mt5cli"
version = "0.7.2"
version = "1.0.1"
description = "Generic MT5 data and execution infrastructure for Python applications"
authors = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
maintainers = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
@@ -9,9 +9,8 @@ license-files = ["LICENSE"]
readme = "README.md"
requires-python = ">= 3.11, < 3.14"
dependencies = [
"pdmt5>=0.3.0",
"pdmt5>=1.0.0",
"click >= 8.1.0",
"pyarrow >= 19.0.0",
"typer >= 0.15.0",
]
classifiers = [
@@ -25,6 +24,9 @@ classifiers = [
"Topic :: Office/Business :: Financial :: Investment",
]
[project.optional-dependencies]
parquet = ["pyarrow >= 19.0.0"]
[project.scripts]
mt5cli = "mt5cli.cli:main"
@@ -42,6 +44,7 @@ dev = [
"pytest-mock >= 3.12.0",
"pytest-cov >= 5.0.0",
"pandas-stubs >= 2.2.3.250527",
"pyarrow >= 19.0.0",
"mkdocs >= 1.6.1",
"mkdocs-material >= 9.7.6",
"mkdocstrings[python] >= 1.0.4",
@@ -124,7 +127,6 @@ ignore = [
]
[tool.ruff.lint.per-file-ignores]
"mt5cli/history.py" = ["TC003"]
"tests/**/*.py" = [
"DOC201", # Missing return documentation
"DOC501", # Raised exception missing from docstring
@@ -176,7 +178,10 @@ omit = [
[tool.coverage.report]
show_missing = true
fail_under = 100
exclude_lines = ["if TYPE_CHECKING:"]
exclude_also = [
"if TYPE_CHECKING:",
"^\\s+\\.\\.\\.$",
]
[build-system]
requires = ["hatchling"]
+360
View File
@@ -740,6 +740,366 @@ class TestCommands:
assert "must be a JSON object" in normalize_cli_output(result.output)
# ---------------------------------------------------------------------------
# Help text / scope tests
# ---------------------------------------------------------------------------
class TestHelpText:
"""Tests verifying CLI help text matches the documented scope."""
def test_top_level_help_mentions_execution(self) -> None:
"""Top-level help must describe execution utilities, not export only."""
result = runner.invoke(app, ["--help"])
assert result.exit_code == 0
output = normalize_cli_output(result.output)
assert "execution" in output.lower()
def test_top_level_help_has_execution_panel(self) -> None:
"""Top-level help must show an Execution command group."""
result = runner.invoke(app, ["--help"])
assert result.exit_code == 0
assert "Execution" in result.output
def test_top_level_help_has_data_export_panel(self) -> None:
"""Top-level help must show a Data / Export command group."""
result = runner.invoke(app, ["--help"])
assert result.exit_code == 0
assert "Data / Export" in result.output
def test_order_send_help_mentions_expert_and_raw(self) -> None:
"""order-send help must communicate it is the expert raw-request path."""
result2 = runner.invoke(
app,
["-o", "out.csv", "order-send", "--help"],
)
assert result2.exit_code == 0
output = normalize_cli_output(result2.output)
assert "raw" in output.lower()
assert "expert" in output.lower()
def test_order_send_help_mentions_live_execution(self) -> None:
"""order-send help must warn about live execution."""
result = runner.invoke(
app,
["-o", "out.csv", "order-send", "--help"],
)
assert result.exit_code == 0
output = normalize_cli_output(result.output)
assert "live" in output.lower()
def test_close_positions_help_mentions_dry_run_and_yes(self) -> None:
"""close-positions help must document both safety gates."""
result = runner.invoke(
app,
["-o", "out.csv", "close-positions", "--help"],
)
assert result.exit_code == 0
output = normalize_cli_output(result.output)
assert "--dry-run" in output
assert "--yes" in output
# ---------------------------------------------------------------------------
# close-positions command
# ---------------------------------------------------------------------------
def _build_mock_trading_client() -> MagicMock:
"""Return a MagicMock Mt5TradingClient with trading constants set."""
client = MagicMock()
client.mt5.POSITION_TYPE_BUY = 0
client.mt5.POSITION_TYPE_SELL = 1
client.mt5.ORDER_TYPE_BUY = 10
client.mt5.ORDER_TYPE_SELL = 11
client.mt5.TRADE_ACTION_DEAL = 20
client.mt5.ORDER_FILLING_IOC = 30
client.mt5.ORDER_TIME_GTC = 40
client.mt5.TRADE_RETCODE_DONE = 10009
client.mt5.TRADE_RETCODE_PLACED = 10008
client.mt5.TRADE_RETCODE_DONE_PARTIAL = 10010
return client
class TestClosePositions:
"""Tests for the close-positions command."""
@pytest.fixture
def trading_client(self, mocker: MockerFixture) -> MagicMock:
"""Patch create_trading_client and return a mock trading client."""
client = _build_mock_trading_client()
client.positions_get_as_df.return_value = pd.DataFrame([
{"ticket": 1, "symbol": "JP225", "type": 0, "volume": 1.0},
{"ticket": 2, "symbol": "EURUSD", "type": 1, "volume": 0.5},
])
client.symbol_info_tick_as_dict.return_value = {"ask": 1.2, "bid": 1.1}
mocker.patch("mt5cli.cli.create_trading_client", return_value=client)
return client
def test_dry_run_does_not_require_yes(
self,
tmp_path: Path,
trading_client: MagicMock,
) -> None:
"""Test --dry-run mode succeeds without --yes."""
output = tmp_path / "close.json"
result = runner.invoke(
app,
["-o", str(output), "close-positions", "--symbol", "JP225", "--dry-run"],
)
assert result.exit_code == 0, result.output
assert output.exists()
trading_client.order_send.assert_not_called()
trading_client.shutdown.assert_called_once()
def test_live_requires_yes(
self,
tmp_path: Path,
trading_client: MagicMock,
) -> None:
"""Test live close-positions fails without --yes."""
output = tmp_path / "close.json"
result = runner.invoke(
app,
["-o", str(output), "close-positions", "--symbol", "JP225"],
)
assert result.exit_code != 0
assert "Pass --yes" in normalize_cli_output(result.output)
trading_client.order_send.assert_not_called()
def test_live_with_yes_calls_order_send(
self,
tmp_path: Path,
trading_client: MagicMock,
) -> None:
"""Test --yes triggers live execution for matching positions."""
trading_client.order_send.return_value = {"retcode": 10009, "comment": "ok"}
output = tmp_path / "close.json"
result = runner.invoke(
app,
["-o", str(output), "close-positions", "--symbol", "JP225", "--yes"],
)
assert result.exit_code == 0, result.output
trading_client.order_send.assert_called_once()
trading_client.shutdown.assert_called_once()
def test_symbol_filter_passed_through(
self,
tmp_path: Path,
trading_client: MagicMock,
) -> None:
"""Test --symbol values are used to filter positions."""
output = tmp_path / "close.json"
result = runner.invoke(
app,
[
"-o",
str(output),
"close-positions",
"--symbol",
"JP225",
"--dry-run",
],
)
assert result.exit_code == 0, result.output
data = json.loads(output.read_text())
assert len(data) == 1
assert data[0]["symbol"] == "JP225"
trading_client.shutdown.assert_called_once()
def test_multiple_symbols_filter(
self,
tmp_path: Path,
trading_client: MagicMock,
) -> None:
"""Test multiple --symbol options are combined."""
output = tmp_path / "close.json"
result = runner.invoke(
app,
[
"-o",
str(output),
"close-positions",
"--symbol",
"JP225",
"--symbol",
"EURUSD",
"--dry-run",
],
)
assert result.exit_code == 0, result.output
data = json.loads(output.read_text())
assert len(data) == 2
symbols = {row["symbol"] for row in data}
assert symbols == {"JP225", "EURUSD"}
trading_client.shutdown.assert_called_once()
def test_ticket_filter_passed_through(
self,
tmp_path: Path,
trading_client: MagicMock,
) -> None:
"""Test --ticket values are used to filter positions."""
output = tmp_path / "close.json"
result = runner.invoke(
app,
[
"-o",
str(output),
"close-positions",
"--ticket",
"2",
"--dry-run",
],
)
assert result.exit_code == 0, result.output
data = json.loads(output.read_text())
assert len(data) == 1
assert data[0]["symbol"] == "EURUSD"
trading_client.shutdown.assert_called_once()
def test_symbol_and_ticket_combined(
self,
tmp_path: Path,
trading_client: MagicMock,
) -> None:
"""Test --symbol and --ticket apply AND semantics when combined."""
output = tmp_path / "close.json"
result = runner.invoke(
app,
[
"-o",
str(output),
"close-positions",
"--symbol",
"JP225",
"--ticket",
"1",
"--dry-run",
],
)
assert result.exit_code == 0, result.output
data = json.loads(output.read_text())
# symbol=JP225 AND ticket=1 → exactly one match
assert len(data) == 1
assert data[0]["symbol"] == "JP225"
trading_client.shutdown.assert_called_once()
def test_missing_symbol_and_ticket_fails(
self,
tmp_path: Path,
mocker: MockerFixture,
) -> None:
"""Test that omitting both --symbol and --ticket fails closed."""
mocker.patch("mt5cli.cli.create_trading_client")
output = tmp_path / "close.json"
result = runner.invoke(
app,
["-o", str(output), "close-positions", "--dry-run"],
)
assert result.exit_code != 0
assert "symbol" in normalize_cli_output(result.output).lower()
def test_output_export_dry_run(
self,
tmp_path: Path,
trading_client: MagicMock,
) -> None:
"""Test dry-run results export with status=dry_run."""
output = tmp_path / "close.json"
result = runner.invoke(
app,
["-o", str(output), "close-positions", "--symbol", "JP225", "--dry-run"],
)
assert result.exit_code == 0, result.output
trading_client.shutdown.assert_called_once()
data = json.loads(output.read_text())
assert data[0]["status"] == "dry_run"
assert data[0]["dry_run"] is True
assert data[0]["order_side"] == "SELL"
def test_order_send_unchanged(
self,
tmp_path: Path,
mock_client: MagicMock,
) -> None:
"""Test that order-send behavior is unchanged by close-positions addition."""
output = tmp_path / "out.csv"
request = json.dumps({"action": 1, "symbol": "EURUSD", "volume": 0.1})
result = runner.invoke(
app,
["-o", str(output), "order-send", "--request", request, "--yes"],
)
assert result.exit_code == 0, result.output
mock_client.order_send_as_df.assert_called_once()
def test_shutdown_called_on_close_error(
self,
tmp_path: Path,
mocker: MockerFixture,
) -> None:
"""Test that shutdown is called even when close_open_positions raises."""
client = _build_mock_trading_client()
client.positions_get_as_df.side_effect = RuntimeError("connection lost")
mocker.patch("mt5cli.cli.create_trading_client", return_value=client)
output = tmp_path / "close.json"
result = runner.invoke(
app,
["-o", str(output), "close-positions", "--symbol", "JP225", "--dry-run"],
)
assert result.exit_code != 0
client.shutdown.assert_called_once()
def test_dry_run_wins_over_yes(
self,
tmp_path: Path,
trading_client: MagicMock,
) -> None:
"""Test that --dry-run takes precedence when combined with --yes."""
output = tmp_path / "close.json"
result = runner.invoke(
app,
[
"-o",
str(output),
"close-positions",
"--symbol",
"JP225",
"--dry-run",
"--yes",
],
)
assert result.exit_code == 0, result.output
trading_client.order_send.assert_not_called()
trading_client.shutdown.assert_called_once()
def test_no_matching_positions_exports_empty_result(
self,
tmp_path: Path,
trading_client: MagicMock,
) -> None:
"""Test that zero filter matches produces an empty JSON array."""
trading_client.positions_get_as_df.return_value = pd.DataFrame([
{"ticket": 1, "symbol": "JP225", "type": 0, "volume": 1.0},
])
output = tmp_path / "close.json"
result = runner.invoke(
app,
[
"-o",
str(output),
"close-positions",
"--symbol",
"NONEXISTENT",
"--dry-run",
],
)
assert result.exit_code == 0, result.output
trading_client.shutdown.assert_called_once()
assert output.exists()
assert json.loads(output.read_text()) == []
# ---------------------------------------------------------------------------
# Callback / shared options
# ---------------------------------------------------------------------------
+405 -56
View File
@@ -2,48 +2,93 @@
from __future__ import annotations
import importlib
import sqlite3
from datetime import UTC, datetime
from typing import TYPE_CHECKING
from importlib.metadata import requires
from typing import TYPE_CHECKING, get_type_hints
from unittest.mock import MagicMock
if TYPE_CHECKING:
from pathlib import Path
import pandas as pd
import pytest
from pdmt5 import Mt5RuntimeError, Mt5TradingError
from pytest_mock import MockerFixture # noqa: TC002
import mt5cli
from mt5cli import (
DEDUP_KEYS,
REQUIRED_COLUMNS,
TIME_COLUMNS,
DataKind,
Dataset,
STABLE_SDK_EXPORTS,
AccountSpec,
ExecutionStatus,
MarginVolume,
MT5Client,
Mt5CliError,
Mt5ConnectionError,
Mt5OperationError,
Mt5SchemaError,
OrderExecutionResult,
OrderLimits,
RateTarget,
build_config,
call_with_normalized_errors,
detect_format,
ensure_utc,
export_dataframe,
export_dataframe_to_sqlite,
granularity_name,
is_recoverable_mt5_error,
build_rate_targets,
calculate_account_projected_margin_ratio,
calculate_margin_and_volume,
calculate_positions_margin,
calculate_projected_margin_ratio,
calculate_symbol_group_margin_ratio,
calculate_trailing_stop_updates,
drop_forming_rate_bar,
ensure_symbol_selected,
extract_tick_price,
fetch_latest_closed_rates,
fetch_latest_closed_rates_for_trading_client,
fetch_latest_closed_rates_indexed,
load_rate_series_from_sqlite,
mt5_session,
normalize_dataframe,
normalize_mt5_exception,
mt5_trading_session,
normalize_order_volume,
place_market_order,
resolve_account_spec,
resolve_account_specs,
)
from mt5cli.converters import (
ensure_utc,
granularity_name,
normalize_symbol,
normalize_symbols,
parse_date_range,
recent_window,
)
from mt5cli.exceptions import (
call_with_normalized_errors,
is_recoverable_mt5_error,
normalize_mt5_exception,
)
from mt5cli.history import (
create_rate_compatibility_views,
load_rate_data,
resolve_rate_view_name,
)
from mt5cli.retry import retry_with_backoff
from mt5cli.schemas import (
DEDUP_KEYS,
REQUIRED_COLUMNS,
TIME_COLUMNS,
DataKind,
ensure_utc_columns,
normalize_dataframe,
normalize_time_columns,
schema_columns,
validate_schema,
)
from mt5cli.retry import retry_with_backoff
from mt5cli.schemas import ensure_utc_columns, normalize_time_columns
if TYPE_CHECKING:
from pathlib import Path
from mt5cli.utils import (
Dataset,
detect_format,
export_dataframe,
export_dataframe_to_sqlite,
)
def _sample_frame(kind: DataKind) -> pd.DataFrame:
@@ -201,16 +246,19 @@ def test_is_recoverable_mt5_error(exc: Exception) -> None:
assert is_recoverable_mt5_error(exc)
def test_normalize_mt5_exception_maps_types() -> None:
@pytest.mark.parametrize(
("exc", "expected_type"),
[
(Mt5RuntimeError("x"), Mt5ConnectionError),
(Mt5TradingError("x"), Mt5OperationError),
],
)
def test_normalize_mt5_exception_maps_types(
exc: Exception,
expected_type: type[Mt5ConnectionError | Mt5OperationError],
) -> None:
"""MT5 exceptions map to stable mt5cli types."""
assert isinstance(
normalize_mt5_exception(Mt5RuntimeError("x")),
Mt5ConnectionError,
)
assert isinstance(
normalize_mt5_exception(Mt5TradingError("x")),
Mt5OperationError,
)
assert isinstance(normalize_mt5_exception(exc), expected_type)
def test_call_with_normalized_errors_reraises_mapped_type() -> None:
@@ -387,26 +435,24 @@ def test_normalize_time_columns_skips_absent_time_fields() -> None:
assert list(result.columns) == ["open"]
def test_normalize_time_columns_converts_unix_seconds() -> None:
"""Numeric MT5 ``time`` values are interpreted as Unix seconds."""
frame = pd.DataFrame({"time": [1704067200]})
result = normalize_time_columns(frame, DataKind.rates)
assert result.loc[0, "time"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
def test_normalize_time_columns_converts_unix_milliseconds() -> None:
"""Numeric MT5 ``time_msc`` values are interpreted as Unix milliseconds."""
frame = pd.DataFrame({"time_msc": [1704067200000]})
result = normalize_time_columns(frame, DataKind.ticks)
assert result.loc[0, "time_msc"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
def test_normalize_time_columns_preserves_utc_datetimes() -> None:
"""Already-converted datetime values remain UTC-normalized."""
aware = datetime(2024, 1, 1, tzinfo=UTC)
frame = pd.DataFrame({"time": [aware]})
result = normalize_time_columns(frame, DataKind.rates)
assert result.loc[0, "time"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
@pytest.mark.parametrize(
("col", "value", "kind"),
[
("time", 1704067200, DataKind.rates),
("time_msc", 1704067200000, DataKind.ticks),
("time", datetime(2024, 1, 1, tzinfo=UTC), DataKind.rates),
("time", "2024-01-01T00:00:00+00:00", DataKind.rates),
],
)
def test_normalize_time_columns_coerces_value(
col: str,
value: object,
kind: DataKind,
) -> None:
"""Time column values are coerced to UTC timestamps regardless of input type."""
frame = pd.DataFrame({col: [value]})
result = normalize_time_columns(frame, kind)
assert result.loc[0, col] == pd.Timestamp("2024-01-01T00:00:00+00:00")
def test_normalize_time_columns_handles_optional_order_times() -> None:
@@ -456,13 +502,6 @@ def test_ensure_utc_columns_skips_missing_columns() -> None:
assert "time" in result.columns
def test_normalize_time_columns_coerces_string_timestamps() -> None:
"""String timestamps are parsed with timezone-aware datetime coercion."""
frame = pd.DataFrame({"time": ["2024-01-01T00:00:00+00:00"]})
result = normalize_time_columns(frame, DataKind.rates)
assert result.loc[0, "time"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
def test_ensure_utc_columns_coerces_non_mt5_columns() -> None:
"""Non-MT5 columns still coerce to UTC datetimes."""
frame = pd.DataFrame({"created_at": ["2024-01-01T00:00:00+00:00"]})
@@ -510,3 +549,313 @@ def test_storage_export_round_trip_sqlite(tmp_path: Path) -> None:
with __import__("sqlite3").connect(output) as conn:
count = conn.execute("SELECT COUNT(*) FROM rates").fetchone()[0]
assert count == 1
def test_storage_module_does_not_exist() -> None:
"""mt5cli.storage re-export module has been removed."""
with pytest.raises(ModuleNotFoundError):
importlib.import_module("mt5cli.storage")
class TestStableSdkContract:
"""Tests for the documented stable downstream SDK contract."""
def test_stable_exports_are_subset_of_all(self) -> None:
"""Every stable export is also listed in the package __all__."""
missing = sorted(STABLE_SDK_EXPORTS - set(mt5cli.__all__))
assert not missing, f"STABLE_SDK_EXPORTS missing from __all__: {missing}"
def test_stable_exports_cover_root_api(self) -> None:
"""STABLE_SDK_EXPORTS classifies every package-root symbol."""
tier_metadata = {"STABLE_SDK_EXPORTS"}
root_exports = set(mt5cli.__all__)
missing_from_root = sorted(STABLE_SDK_EXPORTS - root_exports)
assert not missing_from_root, (
f"STABLE_SDK_EXPORTS missing from __all__: {missing_from_root}"
)
unclassified = sorted(root_exports - STABLE_SDK_EXPORTS - tier_metadata)
assert not unclassified, (
f"Root exports not in STABLE_SDK_EXPORTS: {unclassified}"
)
@pytest.mark.parametrize("name", sorted(STABLE_SDK_EXPORTS))
def test_stable_exports_are_importable_from_package_root(self, name: str) -> None:
"""Stable SDK names resolve through ``from mt5cli import ...``."""
assert hasattr(mt5cli, name), f"{name!r} missing from mt5cli package root"
def test_drop_forming_rate_bar_from_package_root(self) -> None:
"""Closed-bar trimming is available from the stable package surface."""
frame = pd.DataFrame({"time": [1, 2, 3], "close": [1.0, 1.1, 1.2]})
closed = drop_forming_rate_bar(frame)
assert list(closed["close"]) == [1.0, 1.1]
assert len(closed) == 2
def test_fetch_latest_closed_rates_from_package_root(self) -> None:
"""Single-client closed-bar helper drops the forming row."""
client = MagicMock()
client.latest_rates.return_value = pd.DataFrame(
{"time": [1, 2, 3], "close": [1.0, 1.1, 1.2]},
)
result = fetch_latest_closed_rates(
client,
symbol="EURUSD",
granularity="M1",
count=2,
)
client.latest_rates.assert_called_once_with("EURUSD", "M1", 3, start_pos=0)
assert list(result["close"]) == [1.0, 1.1]
def test_fetch_latest_closed_rates_for_trading_client_from_package_root(
self,
) -> None:
"""Trading-client closed-bar helper is importable from the stable surface."""
client = MagicMock()
client.fetch_latest_rates_as_df.return_value = pd.DataFrame(
{"time": [1, 2, 3], "close": [1.0, 1.1, 1.2]},
)
result = fetch_latest_closed_rates_for_trading_client(
client,
symbol="EURUSD",
granularity="M1",
count=2,
)
assert list(result["close"]) == [1.0, 1.1]
def test_normalize_order_volume_from_package_root(self) -> None:
"""Volume normalization helper is importable from the stable surface."""
result = normalize_order_volume(
0.25,
volume_min=0.1,
volume_max=1.0,
volume_step=0.1,
)
assert abs(result - 0.2) < 1e-9
def test_calculate_positions_margin_from_package_root(self) -> None:
"""Position margin helper is importable from the stable surface."""
client = MagicMock()
client.mt5.POSITION_TYPE_BUY = 0
client.mt5.POSITION_TYPE_SELL = 1
client.mt5.ORDER_TYPE_BUY = 10
client.mt5.ORDER_TYPE_SELL = 11
client.positions_get_as_df.return_value = pd.DataFrame()
assert calculate_positions_margin(client) == 0
def test_generic_trading_helpers_from_package_root(self) -> None:
"""New generic trading helpers resolve through the stable surface."""
price = extract_tick_price({"bid": "1.2"}, "bid")
assert price is not None
assert abs(price - 1.2) < 1e-9
assert callable(calculate_trailing_stop_updates)
assert callable(calculate_account_projected_margin_ratio)
assert callable(calculate_projected_margin_ratio)
assert callable(calculate_symbol_group_margin_ratio)
def test_load_rate_series_from_sqlite_requires_managed_views(
self,
tmp_path: Path,
) -> None:
"""Multi-series loading fails clearly when managed views are absent."""
db_path = tmp_path / "empty-views.db"
with sqlite3.connect(db_path) as conn:
conn.execute(
"CREATE TABLE rates("
" symbol TEXT, timeframe INTEGER, time TEXT, close REAL)",
)
targets = build_rate_targets(["EURUSD"], ["M1"])
with pytest.raises(ValueError, match="No rate compatibility view exists"):
load_rate_series_from_sqlite(db_path, targets, count=10)
assert targets == [RateTarget(symbol="EURUSD", timeframe=1)]
def test_resolve_account_spec_from_package_root(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Account credential resolution uses generic ${ENV_VAR} placeholders."""
monkeypatch.setenv("APP_MT5_LOGIN", "555")
monkeypatch.setenv("APP_MT5_PASSWORD", "secret")
account = AccountSpec(
symbols=["EURUSD"],
login="${APP_MT5_LOGIN}",
password="${APP_MT5_PASSWORD}",
server="Broker-Demo",
)
resolved = resolve_account_spec(account, timeout=3000)
assert resolved.login == "555"
assert resolved.password == "secret" # noqa: S105
assert resolved.timeout == 3000
batch = resolve_account_specs([account], server="Override")
assert batch[0].server == "Override"
def test_mt5_trading_session_lifecycle_from_package_root(
self,
mocker: MockerFixture,
) -> None:
"""Trading session helper initializes and always shuts down."""
mock_client = MagicMock()
mocker.patch(
"mt5cli.trading.Mt5DataClient",
return_value=mock_client,
)
with mt5_trading_session(login=12345, server="Broker-Demo") as client:
assert client is mock_client
mock_client.initialize_and_login_mt5.assert_called_once()
mock_client.shutdown.assert_called_once()
def test_trading_order_helpers_importable_from_package_root(self) -> None:
"""Order planning helpers resolve through the stable package surface."""
assert callable(calculate_margin_and_volume)
assert callable(ensure_symbol_selected)
assert callable(place_market_order)
margin_hints = get_type_hints(MarginVolume)
limits_hints = get_type_hints(OrderLimits)
execution_hints = get_type_hints(OrderExecutionResult)
assert margin_hints["buy_volume"] is float
assert limits_hints["stop_loss"] == float | None
assert execution_hints["status"] == ExecutionStatus
def test_mt5_trading_session_shuts_down_on_exception(
self,
mocker: MockerFixture,
) -> None:
"""Trading session helper shuts down even when the body raises."""
mock_client = MagicMock()
mocker.patch(
"mt5cli.trading.Mt5DataClient",
return_value=mock_client,
)
message = "strategy error"
with (
pytest.raises(RuntimeError, match=message),
mt5_trading_session(login=12345, server="Broker-Demo"),
):
raise RuntimeError(message)
mock_client.shutdown.assert_called_once()
def test_fetch_latest_closed_rates_indexed_from_package_root(
self,
mocker: MockerFixture,
) -> None:
"""Indexed closed-bar helper returns a UTC DatetimeIndex named 'time'."""
client = MagicMock()
mocker.patch(
"mt5cli.trading.fetch_latest_closed_rates_for_trading_client",
return_value=pd.DataFrame(
{
"time": [1704067200, 1704153600, 1704240000],
"close": [1.0, 1.1, 1.2],
},
),
)
result = fetch_latest_closed_rates_indexed(
client,
symbol="EURUSD",
granularity="M1",
count=2,
)
assert isinstance(result.index, pd.DatetimeIndex)
assert result.index.name == "time"
assert result.index.tz is not None
assert "time" not in result.columns
assert "close" in result.columns
def test_rate_view_helpers_in_history_module(self, tmp_path: Path) -> None:
"""Rate view helpers are available from mt5cli.history."""
db_path = tmp_path / "rates.db"
with sqlite3.connect(db_path) as conn:
conn.execute(
"CREATE TABLE rates("
" symbol TEXT, timeframe INTEGER, time TEXT, close REAL)",
)
conn.execute(
"INSERT INTO rates(symbol, timeframe, time, close) VALUES (?, ?, ?, ?)",
("EURUSD", 1, "2024-01-01T00:00:00+00:00", 1.0),
)
create_rate_compatibility_views(conn)
assert resolve_rate_view_name(db_path, "EURUSD", "M1") == "rate_EURUSD__1"
missing = tmp_path / "missing.db"
with pytest.raises(ValueError, match="SQLite database not found"):
resolve_rate_view_name(missing, "EURUSD", "M1", require_existing=True)
def test_load_rate_data_in_history_module(self, tmp_path: Path) -> None:
"""SQLite rate loading normalizes timestamps through mt5cli.history."""
db_path = tmp_path / "view.db"
with sqlite3.connect(db_path) as conn:
conn.execute(
'CREATE VIEW "rate_EURUSD__1" AS'
" SELECT '2024-01-01T00:00:00+00:00' AS time, 1.1 AS close",
)
frame = load_rate_data(db_path, "rate_EURUSD__1")
assert frame.index.name == "time"
assert abs(float(frame.iloc[0]["close"]) - 1.1) < 1e-9
@pytest.mark.parametrize(
"name",
[
"Mt5Config",
"Mt5RuntimeError",
"Mt5TradingClient",
"Mt5TradingError",
"TICK_FLAG_MAP",
"TIMEFRAME_MAP",
],
)
def test_pdmt5_pass_through_names_removed_from_public_contract(name: str) -> None:
"""Removed pdmt5 pass-through names are not part of the public contract."""
assert name not in STABLE_SDK_EXPORTS, (
f"{name!r} should not be in STABLE_SDK_EXPORTS"
)
assert name not in mt5cli.__all__, f"{name!r} should not be in mt5cli.__all__"
def test_mt5cli_does_not_import_high_level_trading_symbols() -> None:
"""mt5cli doesn't import Mt5TradingClient or Mt5TradingError at module level."""
trading_module = importlib.import_module("mt5cli.trading")
module_dict = vars(trading_module)
assert "Mt5TradingClient" not in module_dict, (
"mt5cli.trading should not import Mt5TradingClient at module level"
)
assert "Mt5TradingError" not in module_dict, (
"mt5cli.trading should not import Mt5TradingError at module level"
)
# ---------------------------------------------------------------------------
# Packaging metadata
# ---------------------------------------------------------------------------
def test_parquet_extra_declares_pyarrow() -> None:
"""Package metadata lists pyarrow under the parquet optional extra."""
reqs = requires("mt5cli") or []
parquet_reqs = [r for r in reqs if "pyarrow" in r and "parquet" in r]
assert parquet_reqs, "pyarrow not found in parquet optional extra"
def test_pyarrow_not_in_core_dependencies() -> None:
"""Pyarrow is not a core dependency; it belongs only in the parquet extra."""
reqs = requires("mt5cli") or []
core_reqs = [r for r in reqs if "extra ==" not in r]
assert not any("pyarrow" in r for r in core_reqs), (
"pyarrow should not appear in core dependencies"
)
+58 -55
View File
@@ -15,6 +15,8 @@ from pytest_mock import MockerFixture # noqa: TC002
if TYPE_CHECKING:
from pathlib import Path
from pdmt5 import TIMEFRAME_MAP
from mt5cli import history
from mt5cli.history import (
DEFAULT_HISTORY_TIMEFRAMES,
@@ -48,6 +50,7 @@ from mt5cli.history import (
resolve_history_datasets,
resolve_history_tick_flags,
resolve_history_timeframes,
resolve_rate_table_name,
resolve_rate_tables,
resolve_rate_view_name,
resolve_rate_view_names,
@@ -57,12 +60,21 @@ from mt5cli.history import (
write_rates_dataset,
write_streamed_frame,
)
from mt5cli.utils import TIMEFRAME_MAP, Dataset, IfExists
from mt5cli.utils import Dataset, IfExists
class TestResolveRateViewName:
"""Tests for resolve_rate_view_name and resolve_rate_view_names."""
def test_resolve_rate_table_name_returns_normalized_table(self) -> None:
"""Test canonical normalized rates table name is stable."""
assert resolve_rate_table_name("EURUSD", "M1") == "rates"
def test_resolve_rate_table_name_rejects_empty_symbol(self) -> None:
"""Test canonical rate table resolution validates symbols."""
with pytest.raises(ValueError, match="symbol must not be empty"):
resolve_rate_table_name(" ", "M1")
def test_missing_database_path_does_not_create_file(self, tmp_path: Path) -> None:
"""Test resolving against a missing path does not create a database."""
db_path = tmp_path / "missing.db"
@@ -416,6 +428,32 @@ class TestLoadRateData:
frame = load_rate_data_from_connection(conn, "rate_view")
assert list(frame["close"]) == [1.0]
def test_load_rate_series_from_sqlite_table_style(
self,
tmp_path: Path,
) -> None:
"""Test public table-style loader returns one rate DataFrame."""
db_path = tmp_path / "table-style.db"
with sqlite3.connect(db_path) as conn:
conn.execute("CREATE TABLE rates(time TEXT, close REAL)")
conn.executemany(
"INSERT INTO rates(time, close) VALUES (?, ?)",
[
("2024-01-01T00:00:00+00:00", 1.0),
("2024-01-01T00:01:00+00:00", 1.1),
],
)
frame = load_rate_series_from_sqlite(db_path, table="rates", count=1)
assert isinstance(frame, pd.DataFrame)
assert list(frame["close"]) == [1.1]
def test_load_rate_series_from_sqlite_requires_targets_without_table(self) -> None:
"""Test multi-series loading requires targets when table is omitted."""
with pytest.raises(ValueError, match="targets are required"):
load_rate_series_from_sqlite("unused.db", count=1)
def test_loads_quoted_identifier(self, tmp_path: Path) -> None:
"""Test table names are quoted safely."""
db_path = tmp_path / "quoted.db"
@@ -669,19 +707,25 @@ class TestIncrementalStart:
assert starts["EURUSD", 1] == datetime(2024, 1, 2, tzinfo=UTC)
assert starts["GBPUSD", 1] == datetime(2024, 1, 3, tzinfo=UTC)
def test_load_incremental_start_datetimes_requires_timeframe_column(
@pytest.mark.parametrize(
("ddl", "missing_col"),
[
("CREATE TABLE rates(symbol TEXT, time TEXT, open REAL)", "timeframe"),
("CREATE TABLE rates(timeframe INTEGER, time TEXT, open REAL)", "symbol"),
("CREATE TABLE rates(symbol TEXT, timeframe INTEGER, open REAL)", "time"),
],
)
def test_load_incremental_start_datetimes_requires_column(
self,
tmp_path: Path,
ddl: str,
missing_col: str,
) -> None:
"""Test rates tables without timeframe fail fast during incremental resume."""
"""Test rates tables missing a required column fail fast."""
fallback = datetime(2024, 1, 1, tzinfo=UTC)
with sqlite3.connect(tmp_path / "rates-without-timeframe.db") as conn:
conn.execute("CREATE TABLE rates(symbol TEXT, time TEXT, open REAL)")
conn.execute(
"INSERT INTO rates(symbol, time, open) VALUES (?, ?, ?)",
("EURUSD", "2024-01-02T00:00:00+00:00", 1.0),
)
with pytest.raises(ValueError, match="missing: timeframe") as exc_info:
with sqlite3.connect(tmp_path / f"rates-no-{missing_col}.db") as conn:
conn.execute(ddl)
with pytest.raises(ValueError, match=f"missing: {missing_col}") as exc_info:
load_incremental_start_datetimes(
conn,
Dataset.rates,
@@ -689,47 +733,7 @@ class TestIncrementalStart:
timeframes=[1],
fallback_start=fallback,
)
assert "timeframe" in str(exc_info.value)
def test_load_incremental_start_datetimes_requires_symbol_column(
self,
tmp_path: Path,
) -> None:
"""Test rates tables without symbol fail fast during incremental resume."""
fallback = datetime(2024, 1, 1, tzinfo=UTC)
with sqlite3.connect(tmp_path / "rates-no-symbol.db") as conn:
conn.execute(
"CREATE TABLE rates(timeframe INTEGER, time TEXT, open REAL)",
)
with pytest.raises(ValueError, match="missing: symbol") as exc_info:
load_incremental_start_datetimes(
conn,
Dataset.rates,
symbols=["EURUSD"],
timeframes=[1],
fallback_start=fallback,
)
assert "symbol" in str(exc_info.value)
def test_load_incremental_start_datetimes_requires_time_column(
self,
tmp_path: Path,
) -> None:
"""Test rates tables without time fail fast during incremental resume."""
fallback = datetime(2024, 1, 1, tzinfo=UTC)
with sqlite3.connect(tmp_path / "rates-no-time.db") as conn:
conn.execute(
"CREATE TABLE rates(symbol TEXT, timeframe INTEGER, open REAL)",
)
with pytest.raises(ValueError, match="missing: time") as exc_info:
load_incremental_start_datetimes(
conn,
Dataset.rates,
symbols=["EURUSD"],
timeframes=[1],
fallback_start=fallback,
)
assert "time" in str(exc_info.value)
assert missing_col in str(exc_info.value)
def test_load_incremental_start_datetimes_rejects_unrelated_rates_columns(
self,
@@ -1764,12 +1768,11 @@ class TestIncrementalIntegration:
)
assert written_tables == set()
def test_resolve_history_tick_flags_invalid(self) -> None:
@pytest.mark.parametrize("flags", ["BAD", 7])
def test_resolve_history_tick_flags_invalid(self, flags: str | int) -> None:
"""Test invalid tick flags raise ValueError."""
with pytest.raises(ValueError, match="Invalid tick flags"):
resolve_history_tick_flags("BAD")
with pytest.raises(ValueError, match="Invalid tick flags"):
resolve_history_tick_flags(7)
resolve_history_tick_flags(flags)
def test_resolve_history_timeframes_invalid(self) -> None:
"""Test invalid timeframes raise ValueError."""
+645 -3
View File
@@ -37,6 +37,7 @@ from mt5cli.sdk import (
copy_rates_range,
copy_ticks_from,
copy_ticks_range,
fetch_latest_closed_rates,
history_deals,
history_orders,
last_error,
@@ -53,6 +54,7 @@ from mt5cli.sdk import (
resolve_account_spec,
resolve_account_specs,
substitute_env_placeholders,
substitute_mapping_values,
symbol_info,
symbol_info_tick,
symbols,
@@ -61,7 +63,7 @@ from mt5cli.sdk import (
update_history_with_config,
version,
)
from mt5cli.utils import Dataset, IfExists
from mt5cli.utils import Dataset, IfExists, coerce_login
class _TerminalInfo(NamedTuple):
@@ -1301,12 +1303,12 @@ class TestAccountSpec:
expected: int | None,
) -> None:
"""Test login values are normalized for account configs."""
assert sdk._coerce_login(login) == expected # type: ignore[reportPrivateUsage]
assert coerce_login(login) == expected
def test_coerce_login_rejects_non_numeric_string(self) -> None:
"""Test non-numeric login strings raise ValueError."""
with pytest.raises(ValueError, match="invalid literal"):
sdk._coerce_login("abc") # type: ignore[reportPrivateUsage]
coerce_login("abc")
class TestCollectLatestRatesForAccounts:
@@ -1662,6 +1664,62 @@ class TestCollectLatestClosedRatesForAccounts:
)
class TestFetchLatestClosedRates:
"""Tests for fetch_latest_closed_rates."""
def test_fetches_extra_bar_and_drops_forming_row(self) -> None:
"""Test single-symbol closed-bar helper hides the forming bar."""
client = MagicMock()
client.latest_rates.return_value = pd.DataFrame(
{
"time": [1, 2, 3],
"close": [1.0, 1.1, 1.2],
},
)
result = fetch_latest_closed_rates(
client,
symbol="EURUSD",
granularity="M1",
count=2,
)
client.latest_rates.assert_called_once_with(
"EURUSD",
"M1",
3,
start_pos=0,
)
assert list(result["close"]) == [1.0, 1.1]
def test_raises_when_no_closed_bars_are_available(self) -> None:
"""Test empty closed-bar results raise an actionable ValueError."""
client = MagicMock()
client.latest_rates.return_value = pd.DataFrame({"close": [1.0]})
with pytest.raises(ValueError, match="Rate data is empty"):
fetch_latest_closed_rates(
client,
symbol="EURUSD",
granularity="M1",
count=1,
)
def test_rejects_non_positive_count_before_fetching(self) -> None:
"""Test invalid count values fail before calling MT5."""
client = MagicMock()
with pytest.raises(ValueError, match="count must be positive"):
fetch_latest_closed_rates(
client,
symbol="EURUSD",
granularity="M1",
count=0,
)
client.latest_rates.assert_not_called()
class TestCollectLatestClosedRatesByGranularity:
"""Tests for collect_latest_closed_rates_by_granularity."""
@@ -1720,6 +1778,80 @@ class TestSubstituteEnvPlaceholders:
with pytest.raises(ValueError, match="'MT5_MISSING' is not set"):
substitute_env_placeholders("${MT5_MISSING}")
def test_whole_dollar_not_substituted_by_default(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test $ENV_NAME is not expanded without allow_whole_dollar_env=True."""
monkeypatch.setenv("MT5_PASSWORD", "secret")
assert substitute_env_placeholders("$MT5_PASSWORD") == "$MT5_PASSWORD"
def test_whole_dollar_substituted_with_opt_in(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test $ENV_NAME is expanded when allow_whole_dollar_env=True."""
monkeypatch.setenv("MT5_PASSWORD", "secret")
result = substitute_env_placeholders(
"$MT5_PASSWORD", allow_whole_dollar_env=True
)
assert result == "secret"
def test_whole_dollar_missing_variable_raises_value_error(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test missing $ENV_NAME raises ValueError when opt-in is enabled."""
monkeypatch.delenv("MT5_MISSING", raising=False)
with pytest.raises(ValueError, match="'MT5_MISSING' is not set"):
substitute_env_placeholders("$MT5_MISSING", allow_whole_dollar_env=True)
def test_partial_dollar_not_expanded_with_opt_in(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test $ENV embedded in a larger string is not expanded."""
monkeypatch.setenv("pass", "secret")
monkeypatch.setenv("ENV", "val")
assert (
substitute_env_placeholders("plan$pass", allow_whole_dollar_env=True)
== "plan$pass"
)
assert (
substitute_env_placeholders("abc$ENV", allow_whole_dollar_env=True)
== "abc$ENV"
)
def test_dollar_with_suffix_not_expanded_with_opt_in(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test $ENV-suffix is not expanded (not a whole-value placeholder)."""
monkeypatch.setenv("ENV", "val")
assert (
substitute_env_placeholders("$ENV-suffix", allow_whole_dollar_env=True)
== "$ENV-suffix"
)
def test_brace_format_works_with_opt_in(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test ${ENV_VAR} substitution still works when allow_whole_dollar_env=True."""
monkeypatch.setenv("MT5_LOGIN", "12345")
result = substitute_env_placeholders(
"${MT5_LOGIN}", allow_whole_dollar_env=True
)
assert result == "12345"
class TestResolveAccountSpec:
"""Tests for resolve_account_spec and resolve_account_specs."""
@@ -1806,6 +1938,105 @@ class TestResolveAccountSpec:
assert [a.server for a in resolved] == ["Shared", "Fixed"]
assert all(a.timeout == 1000 for a in resolved)
@pytest.mark.parametrize(
("allow_whole_dollar_env", "expected"),
[
(True, "secret"),
(False, "$MT5_PASSWORD"),
],
)
def test_resolve_account_spec_whole_dollar_password(
self,
monkeypatch: pytest.MonkeyPatch,
allow_whole_dollar_env: bool,
expected: str,
) -> None:
"""Test resolve_account_spec expands $ENV_NAME password only with opt-in."""
monkeypatch.setenv("MT5_PASSWORD", "secret")
account = AccountSpec(symbols=["EURUSD"], password="$MT5_PASSWORD")
resolved = resolve_account_spec(
account, allow_whole_dollar_env=allow_whole_dollar_env
)
assert resolved.password == expected
def test_resolve_account_specs_with_whole_dollar_env(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test resolve_account_specs threads allow_whole_dollar_env to each account."""
monkeypatch.setenv("MT5_SERVER", "Broker-Demo")
accounts = [
AccountSpec(symbols=["EURUSD"], server="$MT5_SERVER"),
AccountSpec(symbols=["GBPUSD"], server="Fixed"),
]
resolved = resolve_account_specs(accounts, allow_whole_dollar_env=True)
assert resolved[0].server == "Broker-Demo"
assert resolved[1].server == "Fixed"
def test_resolve_account_spec_whole_dollar_login(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test $ENV_NAME login string is expanded when allow_whole_dollar_env=True."""
monkeypatch.setenv("MT5_LOGIN", "12345")
account = AccountSpec(symbols=["EURUSD"], login="$MT5_LOGIN")
resolved = resolve_account_spec(account, allow_whole_dollar_env=True)
assert resolved.login == "12345"
class TestBuildConfigWholeDollarEnv:
"""Tests for build_config with allow_whole_dollar_env."""
@pytest.mark.parametrize(
("env_var", "field", "env_value"),
[
("MT5_SERVER", "server", "Broker-Demo"),
("MT5_PASSWORD", "password", "secret"),
("MT5_PATH", "path", "/opt/mt5/terminal64.exe"),
],
)
def test_build_config_substitutes_field_with_opt_in(
self,
monkeypatch: pytest.MonkeyPatch,
env_var: str,
field: str,
env_value: str,
) -> None:
"""Test build_config expands $ENV_NAME fields when opt-in is enabled."""
monkeypatch.setenv(env_var, env_value)
config = build_config(**{field: f"${env_var}"}, allow_whole_dollar_env=True) # type: ignore[arg-type]
assert getattr(config, field) == env_value
def test_build_config_leaves_dollar_literal_by_default(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test build_config does not substitute $ENV without opt-in."""
monkeypatch.setenv("MT5_SERVER", "Broker-Demo")
config = build_config(server="$MT5_SERVER")
assert config.server == "$MT5_SERVER"
def test_build_config_none_params_not_substituted(
self,
monkeypatch: pytest.MonkeyPatch, # noqa: ARG002
) -> None:
"""Test build_config with None params does not raise even with opt-in."""
config = build_config(allow_whole_dollar_env=True)
assert config.server is None
assert config.password is None
assert config.path is None
class TestThrottledHistoryUpdater:
"""Tests for the throttled incremental history updater."""
@@ -2043,3 +2274,414 @@ class TestThrottledHistoryUpdater:
assert updater.update(MagicMock(), []) is False
update.assert_not_called()
assert updater.last_update_monotonic is None
def test_default_update_backend_is_update_history(self) -> None:
"""Test the default backend resolves to update_history."""
updater = ThrottledHistoryUpdater(output="history.db")
assert updater.update_backend is update_history
def test_falsy_callable_update_backend_is_preserved(
self,
mocker: MockerFixture,
) -> None:
"""Test only None selects the default backend, not falsy callables."""
class FalsyCallable:
def __init__(self) -> None:
self.calls: list[dict[str, object]] = []
def __bool__(self) -> bool:
return False
def __call__(self, **kwargs: object) -> None:
self.calls.append(kwargs)
falsy_backend = FalsyCallable()
default_backend = mocker.patch("mt5cli.sdk.update_history")
updater = ThrottledHistoryUpdater(
output="history.db",
update_backend=falsy_backend,
)
assert updater.update_backend is falsy_backend
client = MagicMock()
assert updater.update(client, ["EURUSD"]) is True
assert len(falsy_backend.calls) == 1
assert falsy_backend.calls[0]["client"] is client
assert falsy_backend.calls[0]["symbols"] == ["EURUSD"]
default_backend.assert_not_called()
def test_custom_update_backend_receives_expected_kwargs(
self,
mocker: MockerFixture,
) -> None:
"""Test a custom backend receives update_history keyword arguments."""
backend = mocker.Mock()
client = MagicMock()
updater = ThrottledHistoryUpdater(
output="history.db",
datasets={Dataset.rates},
timeframes=["M1", "H1"],
flags="INFO",
lookback_hours=12.0,
with_views=True,
include_account_events=False,
update_backend=backend,
)
updater.update(client, ["EURUSD", "GBPUSD"])
backend.assert_called_once_with(
client=client,
output="history.db",
symbols=["EURUSD", "GBPUSD"],
datasets={Dataset.rates},
timeframes=["M1", "H1"],
flags="INFO",
lookback_hours=12.0,
with_views=True,
include_account_events=False,
)
def test_throttled_calls_do_not_invoke_custom_backend(
self,
mocker: MockerFixture,
) -> None:
"""Test throttled update cycles skip the injected backend."""
backend = mocker.Mock()
monotonic = mocker.patch("mt5cli.sdk.time.monotonic")
monotonic.side_effect = [100.0, 105.0, 200.0, 200.0]
client = MagicMock()
updater = ThrottledHistoryUpdater(
output="history.db",
interval_seconds=60,
update_backend=backend,
)
assert updater.update(client, ["EURUSD"]) is True
assert updater.update(client, ["EURUSD"]) is False
assert updater.update(client, ["EURUSD"]) is True
assert backend.call_count == 2
def test_successful_custom_backend_advances_throttle(
self,
mocker: MockerFixture,
) -> None:
"""Test a successful custom backend updates _last_update_monotonic."""
backend = mocker.Mock()
monotonic = mocker.patch("mt5cli.sdk.time.monotonic", return_value=42.0)
updater = ThrottledHistoryUpdater(
output="history.db",
update_backend=backend,
)
assert updater.update(MagicMock(), ["EURUSD"]) is True
assert updater.last_update_monotonic is monotonic.return_value
monotonic.assert_called_once()
def test_failed_custom_backend_does_not_advance_throttle(
self,
mocker: MockerFixture,
) -> None:
"""Test a failing custom backend leaves _last_update_monotonic unchanged."""
backend = mocker.Mock(side_effect=Mt5RuntimeError("boom"))
updater = ThrottledHistoryUpdater(
output="history.db",
update_backend=backend,
)
with pytest.raises(Mt5RuntimeError, match="boom"):
updater.update(MagicMock(), ["EURUSD"])
assert updater.last_update_monotonic is None
def test_custom_backend_suppresses_recoverable_errors_when_requested(
self,
mocker: MockerFixture,
) -> None:
"""Test suppress_errors swallows recoverable custom backend errors."""
backend = mocker.Mock(side_effect=Mt5RuntimeError("boom"))
updater = ThrottledHistoryUpdater(
output="history.db",
suppress_errors=True,
update_backend=backend,
)
assert updater.update(MagicMock(), ["EURUSD"]) is False
assert updater.last_update_monotonic is None
def test_custom_backend_propagates_errors_when_not_suppressed(
self,
mocker: MockerFixture,
) -> None:
"""Test recoverable custom backend errors propagate by default."""
backend = mocker.Mock(side_effect=Mt5RuntimeError("boom"))
updater = ThrottledHistoryUpdater(
output="history.db",
update_backend=backend,
)
with pytest.raises(Mt5RuntimeError, match="boom"):
updater.update(MagicMock(), ["EURUSD"])
assert updater.last_update_monotonic is None
class TestBuildConfigStringLogin:
"""Tests for build_config() string login coercion (issue #61)."""
@pytest.mark.parametrize(
("login", "expected"),
[
(None, None),
(12345, 12345),
("12345", 12345),
(" 12345 ", 12345),
("", None),
(" ", None),
],
)
def test_coerces_login_from_string(
self,
login: int | str | None,
expected: int | None,
) -> None:
"""Test build_config coerces string login to int or None."""
config = build_config(login=login)
assert config.login == expected
def test_rejects_non_numeric_string_login(self) -> None:
"""Test build_config raises ValueError for non-numeric string login."""
with pytest.raises(ValueError, match="invalid literal"):
build_config(login="abc")
def test_expands_dollar_brace_login_with_opt_in(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test build_config expands ${MT5_LOGIN} and coerces with opt-in."""
monkeypatch.setenv("MT5_LOGIN", "12345")
config = build_config(login="${MT5_LOGIN}", allow_whole_dollar_env=True)
assert config.login == 12345
def test_expands_whole_dollar_login_with_opt_in(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test build_config expands $MT5_LOGIN and coerces with opt-in."""
monkeypatch.setenv("MT5_LOGIN", "99999")
config = build_config(login="$MT5_LOGIN", allow_whole_dollar_env=True)
assert config.login == 99999
def test_missing_env_variable_raises(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test build_config raises ValueError when referenced env var is not set."""
monkeypatch.delenv("MT5_LOGIN", raising=False)
with pytest.raises(ValueError, match="'MT5_LOGIN' is not set"):
build_config(login="${MT5_LOGIN}", allow_whole_dollar_env=True)
def test_env_expands_to_blank_becomes_none(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test build_config coerces blank env-expanded login to None."""
monkeypatch.setenv("MT5_LOGIN", "")
config = build_config(login="${MT5_LOGIN}", allow_whole_dollar_env=True)
assert config.login is None
def test_dollar_brace_login_not_expanded_without_opt_in(self) -> None:
"""Test ${MT5_LOGIN} is not expanded when allow_whole_dollar_env=False."""
with pytest.raises(ValueError, match="invalid literal"):
build_config(login="${MT5_LOGIN}")
def test_integer_login_preserved_backward_compat(self) -> None:
"""Test existing int login callers remain backward-compatible."""
config = build_config(login=54321)
assert config.login == 54321
def test_none_login_preserved_backward_compat(self) -> None:
"""Test existing None login callers remain backward-compatible."""
config = build_config(login=None)
assert config.login is None
class TestSubstituteMappingValues:
"""Tests for substitute_mapping_values() (issue #62)."""
def test_substitutes_selected_keys_in_flat_dict(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test selected keys are substituted in a flat mapping."""
monkeypatch.setenv("MT5_LOGIN", "12345")
data: dict[str, object] = {
"mt5_login": "${MT5_LOGIN}",
"strategy_name": "${MT5_LOGIN}",
}
result = substitute_mapping_values(data, keys={"mt5_login"})
assert result == {"mt5_login": "12345", "strategy_name": "${MT5_LOGIN}"}
def test_preserves_non_selected_literal_dollar_signs(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test literal dollar signs in non-selected fields are preserved exactly."""
monkeypatch.setenv("MT5_PASSWORD", "secret")
data: dict[str, object] = {
"mt5_password": "${MT5_PASSWORD}",
"notes": "$NOT_EXPANDED",
}
result = substitute_mapping_values(data, keys={"mt5_password"})
assert result == {"mt5_password": "secret", "notes": "$NOT_EXPANDED"}
def test_nested_dict_traversal_substitutes_selected_keys(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test selected keys inside nested dicts are substituted."""
monkeypatch.setenv("MT5_SERVER", "Broker-Demo")
data: dict[str, object] = {
"outer": {
"mt5_server": "${MT5_SERVER}",
"other": "${MT5_SERVER}",
}
}
result = substitute_mapping_values(data, keys={"mt5_server"})
assert result == {
"outer": {"mt5_server": "Broker-Demo", "other": "${MT5_SERVER}"}
}
def test_nested_list_traversal_substitutes_selected_keys(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test selected keys inside list elements are substituted."""
monkeypatch.setenv("MT5_LOGIN", "42")
data: dict[str, object] = {
"accounts": [
{"mt5_login": "${MT5_LOGIN}", "name": "${MT5_LOGIN}"},
{"mt5_login": "${MT5_LOGIN}", "name": "fixed"},
]
}
result = substitute_mapping_values(data, keys={"mt5_login"})
assert result == {
"accounts": [
{"mt5_login": "42", "name": "${MT5_LOGIN}"},
{"mt5_login": "42", "name": "fixed"},
]
}
def test_whole_dollar_expanded_with_opt_in(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test $ENV_NAME is expanded when allow_whole_dollar_env=True."""
monkeypatch.setenv("MT5_PASSWORD", "secret")
data: dict[str, object] = {"mt5_password": "$MT5_PASSWORD"}
result = substitute_mapping_values(
data,
keys={"mt5_password"},
allow_whole_dollar_env=True,
)
assert result == {"mt5_password": "secret"}
def test_whole_dollar_not_expanded_by_default(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test $ENV_NAME in a selected key is preserved when opt-in is False."""
monkeypatch.setenv("MT5_PASSWORD", "secret")
data: dict[str, object] = {"mt5_password": "$MT5_PASSWORD"}
result = substitute_mapping_values(data, keys={"mt5_password"})
assert result == {"mt5_password": "$MT5_PASSWORD"}
def test_blank_string_becomes_none_for_blank_keys(self) -> None:
"""Test blank strings are normalised to None for blank_string_keys_as_none."""
data: dict[str, object] = {
"mt5_login": "",
"mt5_password": " ",
"other": "",
}
result = substitute_mapping_values(
data,
keys=set(),
blank_string_keys_as_none={"mt5_login", "mt5_password"},
)
assert result == {"mt5_login": None, "mt5_password": None, "other": ""}
def test_env_expanded_blank_becomes_none(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test env-expanded blank string is normalised to None."""
monkeypatch.setenv("MT5_LOGIN", "")
data: dict[str, object] = {"mt5_login": "${MT5_LOGIN}"}
result = substitute_mapping_values(
data,
keys={"mt5_login"},
blank_string_keys_as_none={"mt5_login"},
)
assert result == {"mt5_login": None}
def test_missing_env_variable_raises_for_selected_key(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test missing env var for a selected key raises ValueError."""
monkeypatch.delenv("MT5_MISSING", raising=False)
data: dict[str, object] = {"mt5_login": "${MT5_MISSING}"}
with pytest.raises(ValueError, match="'MT5_MISSING' is not set"):
substitute_mapping_values(data, keys={"mt5_login"})
def test_non_string_values_preserved(self) -> None:
"""Test non-string values under selected or non-selected keys are preserved."""
data: dict[str, object] = {
"mt5_login": 12345,
"timeout": 5000,
"enabled": True,
"ratio": 1.5,
"nothing": None,
}
result = substitute_mapping_values(
data, keys={"mt5_login", "timeout", "enabled", "ratio", "nothing"}
)
assert result == data
def test_caller_supplied_key_set_substitutes_correctly(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test helper works with any caller-supplied key set."""
monkeypatch.setenv("APP_LOGIN", "77777")
monkeypatch.setenv("APP_PASSWORD", "p4ss")
data: dict[str, object] = {
"app_login": "${APP_LOGIN}",
"app_password": "${APP_PASSWORD}",
"unrelated": "${APP_LOGIN}",
}
credential_keys = {"app_login", "app_password"}
result = substitute_mapping_values(data, keys=credential_keys)
assert result == {
"app_login": "77777",
"app_password": "p4ss",
"unrelated": "${APP_LOGIN}",
}
def test_scalar_data_returned_unchanged(self) -> None:
"""Test a scalar (non-dict, non-list) value is returned as-is."""
assert substitute_mapping_values("hello", keys={"x"}) == "hello"
assert substitute_mapping_values(42, keys={"x"}) == 42
assert substitute_mapping_values(None, keys={"x"}) is None
def test_tuple_container_not_traversed(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test tuple containers are returned as-is without traversal."""
monkeypatch.setenv("MT5_LOGIN", "42")
data: dict[str, object] = {"accounts": ({"mt5_login": "${MT5_LOGIN}"},)}
result = substitute_mapping_values(data, keys={"mt5_login"})
# tuple is returned as-is; inner dict is NOT visited
assert result == {"accounts": ({"mt5_login": "${MT5_LOGIN}"},)}
+3400 -25
View File
File diff suppressed because it is too large Load Diff
+20 -12
View File
@@ -4,21 +4,22 @@ from __future__ import annotations
import json
import sqlite3
import sys
from datetime import UTC, datetime
from typing import TYPE_CHECKING
import pandas as pd
import pytest
import mt5cli.utils
if TYPE_CHECKING:
from pathlib import Path
from mt5cli.utils import (
DATETIME_TYPE,
REQUEST_TYPE,
TICK_FLAG_MAP,
TICK_FLAGS_TYPE,
TIMEFRAME_MAP,
TIMEFRAME_TYPE,
Dataset,
IfExists,
@@ -111,6 +112,17 @@ class TestExportDataframe:
result = pd.read_parquet(output)
pd.testing.assert_frame_equal(result, sample_df)
def test_export_parquet_without_pyarrow(
self,
tmp_path: Path,
sample_df: pd.DataFrame,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test that a clear error is raised when pyarrow is not installed."""
monkeypatch.setitem(sys.modules, "pyarrow", None)
with pytest.raises(ImportError, match="mt5cli\\[parquet\\]"):
export_dataframe(sample_df, tmp_path / "out.parquet", "parquet")
def test_export_sqlite3(self, tmp_path: Path, sample_df: pd.DataFrame) -> None:
"""Test SQLite3 export."""
output = tmp_path / "out.db"
@@ -361,17 +373,13 @@ class TestParseRequest:
class TestConstants:
"""Tests for module constants."""
def test_timeframe_map_has_expected_keys(self) -> None:
"""Test that TIMEFRAME_MAP contains standard timeframes."""
for key in ("M1", "M5", "M15", "M30", "H1", "H4", "D1", "W1", "MN1"):
assert key in TIMEFRAME_MAP
def test_timeframe_map_is_private_in_utils(self) -> None:
"""TIMEFRAME_MAP is a private implementation detail; not a public attribute."""
assert not hasattr(mt5cli.utils, "TIMEFRAME_MAP")
def test_tick_flag_map_has_expected_keys(self) -> None:
"""Test that TICK_FLAG_MAP contains standard flags with MT5 values."""
assert {"ALL", "INFO", "TRADE"} <= set(TICK_FLAG_MAP)
assert TICK_FLAG_MAP["ALL"] == -1
assert TICK_FLAG_MAP["INFO"] == 1
assert TICK_FLAG_MAP["TRADE"] == 2
def test_tick_flag_map_absent_from_utils(self) -> None:
"""TICK_FLAG_MAP is not exposed by mt5cli.utils."""
assert not hasattr(mt5cli.utils, "TICK_FLAG_MAP")
@pytest.mark.parametrize(
("dataset", "expected"),
Generated
+14 -7
View File
@@ -487,21 +487,26 @@ wheels = [
[[package]]
name = "mt5cli"
version = "0.7.2"
version = "1.0.1"
source = { editable = "." }
dependencies = [
{ name = "click" },
{ name = "pdmt5" },
{ name = "pyarrow" },
{ name = "typer" },
]
[package.optional-dependencies]
parquet = [
{ name = "pyarrow" },
]
[package.dev-dependencies]
dev = [
{ name = "mkdocs" },
{ name = "mkdocs-material" },
{ name = "mkdocstrings", extra = ["python"] },
{ name = "pandas-stubs" },
{ name = "pyarrow" },
{ name = "pymdown-extensions" },
{ name = "pyright" },
{ name = "pytest" },
@@ -513,10 +518,11 @@ dev = [
[package.metadata]
requires-dist = [
{ name = "click", specifier = ">=8.1.0" },
{ name = "pdmt5", specifier = ">=0.3.0" },
{ name = "pyarrow", specifier = ">=19.0.0" },
{ name = "pdmt5", specifier = ">=1.0.0" },
{ name = "pyarrow", marker = "extra == 'parquet'", specifier = ">=19.0.0" },
{ name = "typer", specifier = ">=0.15.0" },
]
provides-extras = ["parquet"]
[package.metadata.requires-dev]
dev = [
@@ -524,6 +530,7 @@ dev = [
{ name = "mkdocs-material", specifier = ">=9.7.6" },
{ name = "mkdocstrings", extras = ["python"], specifier = ">=1.0.4" },
{ name = "pandas-stubs", specifier = ">=2.2.3.250527" },
{ name = "pyarrow", specifier = ">=19.0.0" },
{ name = "pymdown-extensions", specifier = ">=10.21.2" },
{ name = "pyright", specifier = ">=1.1.407" },
{ name = "pytest", specifier = ">=9.0.3" },
@@ -684,16 +691,16 @@ wheels = [
[[package]]
name = "pdmt5"
version = "0.3.0"
version = "1.0.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "metatrader5", marker = "sys_platform == 'win32'" },
{ name = "pandas" },
{ name = "pydantic" },
]
sdist = { url = "https://files.pythonhosted.org/packages/bf/cc/c8fa3a01e0e34178fec8527992f7bb8eda5881477ce23aaacaa9b2ef7bec/pdmt5-0.3.0.tar.gz", hash = "sha256:bb612d5c2695eafac9b2a7b74756e13bd383d7e5517bd90c9a2efa92492c484c", size = 215100, upload-time = "2026-06-11T13:26:46.976Z" }
sdist = { url = "https://files.pythonhosted.org/packages/21/6d/b51d2d0ec4636e914210be03a7da20e5078bc8cd7a351edd13bc30b7d2b1/pdmt5-1.0.0.tar.gz", hash = "sha256:ba53a1a5db41fdf4c022ef55353f5a4f6884f625c9310de2b0ddb20457956716", size = 123155, upload-time = "2026-06-25T23:41:50.503Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/f2/03/b12cc4c9db983d971c9172b3765161b6d91136d0624e6718a04dd815e7a1/pdmt5-0.3.0-py3-none-any.whl", hash = "sha256:5388b406cc583202600cfe22c9d781679b1d931b1ed5a2b5dcf37c566149b49f", size = 26250, upload-time = "2026-06-11T13:26:45.689Z" },
{ url = "https://files.pythonhosted.org/packages/5e/38/27b712c572d8146efddf571a0e4e7b6d2314a335eb0f47dec1f499ab2189/pdmt5-1.0.0-py3-none-any.whl", hash = "sha256:f969c17902f9ffcbf56d7ccf9285aab39ececd676fcca50c094abcf9d728d517", size = 23992, upload-time = "2026-06-25T23:41:49.067Z" },
]
[[package]]