Compare commits

...

7 Commits

Author SHA1 Message Date
dceoy 254c159ad5 Bump version to v0.7.2 2026-06-13 01:34:03 +09:00
Daichi Narushima 78c49238cf feat: stable MT5Client public API and infrastructure layer (#30)
* feat: add stable MT5Client public API and infrastructure layer

Introduce a reusable public API for downstream trading applications:

- MT5Client as the primary client abstraction with order_check/order_send
- schemas module with DataKind contracts, validation, and normalization
- converters, exceptions, retry, and storage facade modules
- CLI order commands now route through MT5Client
- connected_client made public; retry logic centralized
- Contract tests for API surface, schemas, and storage round-trips
- README and docs updated with Python API usage examples

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* fix: correct time coercion, broker-safe symbols, and execution docs

- Normalize MT5 time columns with correct second/millisecond units
- Coerce all present known MT5 time fields, including optional order times
- Preserve broker symbol casing in normalize_symbol()
- Document order_send() as a live execution primitive with clear scope boundaries
- Add contract tests for timestamp and symbol normalization behavior

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
2026-06-13 01:32:03 +09:00
Daichi Narushima 9356d5dcdf Consolidate duplicated export and history streaming helpers (#29)
* Consolidate duplicated export and history streaming helpers.

Reduce repeated CLI export plumbing, shared per-symbol SQLite writes, and test mock setup without changing public behavior.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Bump version from 0.7.0 to 0.7.1.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-12 23:14:34 +09:00
Daichi Narushima 0fad55d609 Refactor MT5 constant parsing to delegate to pdmt5 >= 0.3.0 (#28)
* Refactor MT5 constant parsing to delegate to pdmt5 >= 0.3.0

Replace local TIMEFRAME_MAP, TICK_FLAG_MAP, and parser helpers with thin
compatibility wrappers around pdmt5. COPY_TICKS flags now use real MT5 values
(ALL=-1, INFO=1, TRADE=2). Click parameter types validate all inputs through
the wrappers. Update tests and docs to describe the pdmt5/mt5cli/mt5api layering.

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* Fix timeframe defaults and COPY_TICKS flag defaults after pdmt5 migration

Use short timeframe aliases for default history collection and granularity
naming via pdmt5.get_timeframe_name. Set CLI/SDK default tick flags to ALL
(-1) instead of the legacy mt5cli-only value.

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* Address CI lint failure and PR review feedback

Fix ruff import ordering in history.py. Use ALL string defaults for CLI tick
flags, isolate TICK_FLAG_MAP as a dict snapshot, derive flag names from pdmt5,
reuse TIMEFRAME_NAMES for default history timeframes, and add tests for prefix
stripping and TIMEFRAME_ key filtering.

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* Bump version to 0.7.0

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
2026-06-11 23:22:50 +09:00
dceoy d654b82f9d Bump version from 0.6.0 to 0.6.1.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-11 19:36:34 +09:00
Daichi Narushima b5e82e71c7 Add trading session helpers and extend ThrottledHistoryUpdater (#25)
* Add trading session helpers and extend ThrottledHistoryUpdater

Introduce mt5cli.trading with mt5_trading_session() for Mt5TradingClient
lifecycle management and reusable operational helpers for position-side
detection, margin/volume sizing, and protective order price derivation.

Extend ThrottledHistoryUpdater to validate inputs before updates and to
optionally suppress ValueError, OSError, and missing-method errors without
advancing the throttle timestamp.

Export the new helpers from mt5cli.__init__, add unit tests with mocked
clients, and document migration guidance for downstream projects such as
mteor.

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* Narrow ThrottledHistoryUpdater suppress_errors handling (#27)

* Narrow ThrottledHistoryUpdater suppress_errors for MT5 capability only

Remove broad AttributeError/TypeError handling from recoverable errors.
Add _is_mt5_client_capability_error() to detect missing history API methods
or non-callable client attributes by message and attribute name.

Generic AttributeError/TypeError values always propagate even when
suppress_errors=True. Update docs and tests accordingly.

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* Detect non-callable history client methods in suppress_errors

Address review feedback: when a history API attribute exists but is not
callable, Python raises a generic TypeError. Inspect the traceback for
mt5cli.history client call sites so these capability mismatches are still
suppressed without matching all TypeError values.

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* Address PR review feedback on trading helpers

- Resolve history module path once at import time
- Only treat non-callable TypeErrors as capability errors at the raise site
- Validate SL/TP ratios in determine_order_limits
- Add tests for margin_free edge cases, body-raise shutdown, and internal TypeError propagation
- Clarify ThrottledHistoryUpdater suppress_errors docs
- Split README migration example into trading vs read-only history sessions

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* Tighten protective ratio validation and clamp negative margin_free

Add _require_protective_ratio enforcing 0 <= ratio < 1 for SL/TP limits so
a ratio of 1.0 cannot produce zero protective prices. Clamp negative
margin_free to 0.0 in calculate_margin_and_volume before sizing.

Add boundary and negative-margin tests; document constraints in trading API
docs.

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
2026-06-11 19:32:52 +09:00
Daichi Narushima 18df96872b Add closed-bar rate helpers (v0.6.0) (#26)
* Add closed-bar rate helpers and bump version to 0.6.0.

Expose drop_forming_rate_bar and multi-account collectors so downstream apps no longer need count+1 fetches and manual bar trimming.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Bump pygments to 2.20.0 to fix CVE-2026-4539 ReDoS advisory.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address PR review feedback on closed-bar rate collection.

Validate count and start_pos before MT5 fetches, avoid redundant frame copies, clarify empty-series errors, and expand test coverage.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Include symbol and timeframe in empty closed-rate error messages.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-11 02:30:48 +09:00
32 changed files with 3200 additions and 472 deletions
+140 -4
View File
@@ -2,10 +2,16 @@
[![CI/CD](https://github.com/dceoy/mt5cli/actions/workflows/ci.yml/badge.svg)](https://github.com/dceoy/mt5cli/actions/workflows/ci.yml)
Command-line tool for exporting MetaTrader 5 data to CSV, JSON, Parquet, and SQLite3.
Generic MT5 data and execution infrastructure for Python applications. Export from the CLI or import a small, stable Python API in downstream packages.
Built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data handler for MetaTrader 5.
## Architecture
- **pdmt5** — canonical MT5 client, DataFrame/trading primitives, and MT5 constant parsing (`TIMEFRAME_*`, `COPY_TICKS_*`, order types).
- **mt5cli** — public `MT5Client` API, standardized dataset schemas, storage helpers, CLI commands, and SQLite history collection built on pdmt5.
- **mt5api** — sibling HTTP adapter for remote MT5 access; not a dependency of mt5cli.
## Features
- **Multi-format export**: CSV, JSON, Parquet, and SQLite3 output formats
@@ -21,7 +27,65 @@ Built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data han
pip install -U mt5cli MetaTrader5
```
## Usage
## Python API (downstream packages)
Import `MT5Client` for generic MT5 data access, schema normalization, and optional order primitives. `Mt5CliClient` remains available as a backward-compatible alias.
```python
from datetime import UTC, datetime
from pathlib import Path
from mt5cli import (
DataKind,
Dataset,
MT5Client,
build_config,
collect_history,
export_dataframe,
mt5_session,
normalize_dataframe,
update_history_with_config,
)
# Persistent session for multiple calls
with mt5_session(build_config(login=12345, server="Broker-Demo")) as client:
rates = client.copy_rates_range(
"EURUSD",
timeframe="H1",
date_from="2024-01-01",
date_to="2024-02-01",
)
positions = client.positions()
check = client.order_check({"action": 1, "symbol": "EURUSD", "volume": 0.1})
# Normalize MT5 frames to the public schema contract before storage
closed_rates = normalize_dataframe(
rates, DataKind.rates, symbol="EURUSD", timeframe="H1"
)
export_dataframe(closed_rates, Path("rates.csv"), "csv")
# Bulk SQLite history (same behavior as collect-history CLI command)
collect_history(
Path("history.db"),
symbols=["EURUSD"],
date_from=datetime(2024, 1, 1, tzinfo=UTC),
date_to=datetime(2024, 2, 1, tzinfo=UTC),
datasets={Dataset.rates, Dataset.history_deals},
)
# Incremental append for automated pipelines
update_history_with_config(
output="history.db",
symbols=["EURUSD"],
config=build_config(login=12345),
)
```
Schema contracts live in `mt5cli.schemas` (`DataKind`, `validate_schema`, `normalize_dataframe`). Storage helpers are re-exported from `mt5cli.storage` and the package root.
`MT5Client.order_send()` is a live execution primitive: it can place real trades on the connected account. mt5cli does not implement strategy logic, signal generation, backtesting, or optimization — downstream applications must gate live execution explicitly.
## CLI usage
```bash
# Export account information to CSV
@@ -137,10 +201,25 @@ update_history_with_config(
- **Rate view loading**: use `load_rate_data()` / `load_rate_data_from_connection()` to load a SQLite rate table or view into a `DatetimeIndex` DataFrame.
- **Multi-series rate loading**: use `build_rate_targets()` to build neutral `RateTarget(symbol, timeframe)` pairs, `resolve_rate_tables()` to map them to table/view names (pass `require_existing=True` for strict resolution), and `load_rate_series_from_sqlite()` to load them into a mapping keyed by `(symbol, integer timeframe)`. The loader requires existing managed views unless `explicit_tables` is supplied, and rejects duplicate `(symbol, timeframe)` targets.
- **Multi-account latest rates**: use `collect_latest_rates_for_accounts()` with `AccountSpec` to read the latest bars for several account groups, merged into a `(symbol, integer timeframe)` mapping. For long-running pollers, `collect_latest_rates_for_accounts_with_retries()` adds bounded exponential backoff that retries only `pdmt5.Mt5TradingError` / `pdmt5.Mt5RuntimeError` and re-raises once `retry_count` is exhausted.
- **Latest closed bars**: use `collect_latest_closed_rates_for_accounts()` when downstream logic must exclude the still-forming current bar. It fetches `count + 1` bars at `start_pos=0`, drops the last row with `drop_forming_rate_bar()`, and validates each series is non-empty. `collect_latest_closed_rates_by_granularity()` returns the same data keyed by `(symbol, granularity_name)` such as `("EURUSD", "M1")`.
```python
from mt5cli import AccountSpec, collect_latest_closed_rates_by_granularity
rates = collect_latest_closed_rates_by_granularity(
[AccountSpec(symbols=["EURUSD", "GBPUSD"], login=12345)],
["M1", "H1"],
count=500,
retry_count=3,
)
eurusd_m1 = rates["EURUSD", "M1"] # closed bars only
```
- **Credential resolution**: use `resolve_account_spec()` / `resolve_account_specs()` to merge explicit override values over `AccountSpec` fields and expand `${ENV_VAR}` placeholders (via `substitute_env_placeholders()`), raising `ValueError` for missing variables. This keeps secrets out of plan/config files without coupling to any strategy code.
- **Throttled history updates**: use `ThrottledHistoryUpdater` to wrap `update_history()` with a minimum `interval_seconds` between successful runs (monotonic clock). Call `should_update()` / `update(client, symbols)` from an application loop; errors propagate by default, or pass `suppress_errors=True` to swallow recoverable `Mt5*Error`/`sqlite3.Error` and let the caller decide logging.
- **Throttled history updates**: use `ThrottledHistoryUpdater` to wrap `update_history()` with a minimum `interval_seconds` between successful runs (monotonic clock). Call `should_update()` / `update(client, symbols)` from an application loop; errors propagate by default, or pass `suppress_errors=True` to swallow recoverable `Mt5*Error`, `sqlite3.Error`, `ValueError`, `OSError`, and MT5 client capability errors for history API methods without advancing the throttle (other `AttributeError` / `TypeError` values always propagate).
- **Trading session helpers**: use `mt5_trading_session()` for a trading-capable `pdmt5.Mt5TradingClient` that initializes/logs in via `Mt5Config.path` and always shuts down safely. Pair with `detect_position_side()`, `calculate_margin_and_volume()`, and `determine_order_limits()` for generic position and sizing utilities. The read-only `mt5_session()` / `Mt5CliClient` SDK is unchanged.
- **Granularity-keyed rate loading**: `load_rate_series_by_granularity()` builds targets with `build_rate_targets()`, loads them with `load_rate_series_from_sqlite()`, and returns a mapping keyed by `(symbol | None, granularity_name)` such as `("EURUSD", "M1")` to reduce downstream boilerplate.
- **MT5 session helper**: use the `mt5_session()` context manager to attach to (or, when `Mt5Config.path` is set, launch) an MT5 terminal, log in, and yield a connected `Mt5CliClient` that shuts down on exit.
- **MT5 session helper**: use the `mt5_session()` context manager to attach to (or, when `Mt5Config.path` is set, launch) an MT5 terminal, log in, and yield a connected `MT5Client` that shuts down on exit.
- **SQLite export helpers**: use `export_dataframe_to_sqlite()` for append mode, optional index export, and post-write deduplication by key columns.
- **Recent ticks and margins**: `recent_ticks()` and `minimum_margins()` SDK helpers (and matching CLI commands) cover common downstream read-only queries.
@@ -150,6 +229,63 @@ update_history_with_config(
- Windows OS (MetaTrader 5 requirement)
- MetaTrader 5 platform installed
### Migration note for mteor
Replace local MT5 lifecycle and trading helper code with mt5cli imports:
```python
# Before (local mteor helpers)
# with local_mt5_trading_session(config) as client:
# side = local_detect_position_side(client, symbol)
# sizing = local_calculate_margin_and_volume(client, symbol, unit_ratio, preserved_ratio)
# limits = local_determine_order_limits(client, symbol, side, sl_ratio, tp_ratio)
# After (mt5cli shared layer)
from pdmt5 import Mt5Config
from mt5cli import (
calculate_margin_and_volume,
detect_position_side,
determine_order_limits,
mt5_trading_session,
)
with mt5_trading_session(
Mt5Config(path=terminal_path, login=login), retry_count=2
) as client:
side = detect_position_side(client, symbol)
sizing = calculate_margin_and_volume(
client, symbol, unit_margin_ratio=0.5, preserved_margin_ratio=0.2
)
if side is not None:
limits = determine_order_limits(
client,
symbol,
side,
stop_loss_limit_ratio=0.01,
take_profit_limit_ratio=0.02,
)
```
Throttled history updates use a separate read-only session:
```python
from pdmt5 import Mt5Config, Mt5DataClient
from mt5cli import ThrottledHistoryUpdater
updater = ThrottledHistoryUpdater(
output="history.db", interval_seconds=60, suppress_errors=True
)
client = Mt5DataClient(config=Mt5Config(login=login))
client.initialize_and_login_mt5()
try:
updater.update(client, ["EURUSD"])
finally:
client.shutdown()
```
Read-only collectors can keep using `mt5_session()` and `MT5Client` (or the `Mt5CliClient` alias) without changes.
## Development
```bash
+3
View File
@@ -0,0 +1,3 @@
# Client
::: mt5cli.client
+3
View File
@@ -0,0 +1,3 @@
# Converters
::: mt5cli.converters
+3
View File
@@ -0,0 +1,3 @@
# Exceptions
::: mt5cli.exceptions
+39 -106
View File
@@ -1,120 +1,53 @@
# API Reference
This section contains the complete API documentation for mt5cli.
This section documents the mt5cli public Python API and CLI modules.
## Modules
## Public API layers
The mt5cli package consists of the following modules:
| Module | Purpose |
| ----------------------------------------- | ------------------------------------------------------------------------- |
| [Client](client.md) | `MT5Client` session abstraction for data access and order primitives |
| [Schemas](schemas.md) | Canonical DataFrame contracts and normalization helpers |
| [Storage](storage.md) | CSV/JSON/Parquet/SQLite export and history collection helpers |
| [Converters](converters.md) | Symbol, timeframe, timezone, and date-range utilities |
| [Exceptions](exceptions.md) | Stable mt5cli exception types and MT5 error normalization |
| [SDK](sdk.md) | Module-level fetch helpers, multi-account collectors, incremental history |
| [Trading](trading.md) | Trading-capable sessions and operational helpers |
| [History Collection (SQLite)](history.md) | SQLite schema, incremental writes, dedup, and rate views |
| [CLI](cli.md) | Typer commands that delegate to the Python API |
| [Utils](utils.md) | Parsing helpers and Click parameter types |
### [CLI](cli.md)
## Architecture overview
Command-line interface module providing typer-based commands for exporting MetaTrader 5 data to CSV, JSON, Parquet, and SQLite3 formats.
### [Utils](utils.md)
Utility module providing constants, enums, Click parameter types, and helper functions for parsing and exporting data.
### [SDK](sdk.md)
Programmatic SDK for read-only MetaTrader 5 data collection. Returns pandas DataFrames and provides `collect_history` for SQLite bulk collection.
### [History Collection (SQLite)](history.md)
SQLite storage helpers for the `collect-history` command schema, incremental updates, deduplication, indexes, and optional views.
## Architecture Overview
The package follows a simple architecture built on top of pdmt5:
1. **CLI Layer** (`cli.py`): Typer application with subcommands that delegate to the SDK and export results.
2. **SDK Layer** (`sdk.py`): Read-only data access functions, `Mt5CliClient`, and `collect_history` orchestration.
3. **Utils Layer** (`utils.py`): Constants, enums, custom Click parameter types, parsing helpers, and format detection/export utilities.
4. **Data Layer** (via `pdmt5`): Uses `Mt5DataClient` and `Mt5Config` from the pdmt5 package for all MetaTrader 5 data access.
## Usage Guidelines
All modules follow these conventions:
- **Type Safety**: All functions include comprehensive type hints
- **Error Handling**: User-friendly error messages via typer
- **Documentation**: Google-style docstrings with examples
- **Validation**: Custom Click parameter types for input validation
## Quick Start
```bash
# Export account information to CSV
mt5cli -o account.csv account-info
# Export EURUSD H1 rates to Parquet
mt5cli -o rates.parquet rates-from --symbol EURUSD --timeframe H1 \
--date-from 2024-01-01 --count 1000
# Export ticks to JSON
mt5cli -o ticks.json ticks-from --symbol EURUSD \
--date-from 2024-01-01 --count 500 --flags ALL
# Export to SQLite3 with custom table name
mt5cli -o data.db --table symbols symbols --group "*USD*"
```mermaid
flowchart TD
App["Downstream application"] --> Client["MT5Client"]
CLI["mt5cli CLI"] --> Client
Client --> SDK["sdk / pdmt5"]
Client --> Schemas["schemas"]
Storage["storage"] --> History["history SQLite"]
Storage --> Utils["utils export"]
SDK --> PDMT5["pdmt5.Mt5DataClient"]
```
## Python API
Downstream packages should depend on the package root exports (`MT5Client`, `DataKind`, `normalize_dataframe`, `export_dataframe`, `collect_history`, etc.) rather than private modules.
`MT5Client.order_send()` is a live execution primitive that can place real trades. mt5cli exposes minimal execution helpers only; strategy logic, signals, backtests, and optimization remain out of scope and must be implemented downstream with explicit execution gating.
## Quick start
```python
from datetime import UTC, datetime
from pathlib import Path
from mt5cli import MT5Client, build_config, mt5_session
from mt5cli import (
Dataset,
IfExists,
Mt5CliClient,
collect_history,
copy_rates_range,
detect_format,
export_dataframe,
export_dataframe_to_sqlite,
minimum_margins,
recent_ticks,
)
from mt5cli.history import resolve_rate_view_name
# Fetch rates programmatically
rates = copy_rates_range(
"EURUSD",
timeframe="H1",
date_from="2024-01-01",
date_to="2024-02-01",
)
# Detect output format from file extension
fmt = detect_format(Path("output.parquet")) # Returns "parquet"
# Export a DataFrame
export_dataframe(rates, Path("output.csv"), "csv")
# Append to SQLite with deduplication
export_dataframe_to_sqlite(
rates,
Path("history.db"),
"rates",
if_exists=IfExists.APPEND,
deduplicate_on=("symbol", "timeframe", "time"),
)
# Resolve rate compatibility views and fetch recent ticks
view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1")
ticks = recent_ticks("EURUSD", seconds=300)
margins = minimum_margins("EURUSD")
# Collect history into SQLite
collect_history(
Path("history.db"),
symbols=["EURUSD"],
date_from=datetime(2024, 1, 1, tzinfo=UTC),
date_to=datetime(2024, 2, 1, tzinfo=UTC),
)
with mt5_session(build_config(login=12345)) as client:
rates = client.copy_rates_range("EURUSD", "H1", "2024-01-01", "2024-02-01")
positions = client.positions()
```
## Examples
```bash
mt5cli -o account.csv account-info
mt5cli -o rates.parquet rates-range --symbol EURUSD --timeframe H1 \
--date-from 2024-01-01 --date-to 2024-02-01
```
See individual module pages for detailed usage examples and code samples.
See individual module pages for detailed usage examples.
+3
View File
@@ -0,0 +1,3 @@
# Schemas
::: mt5cli.schemas
+34 -3
View File
@@ -28,6 +28,26 @@ rates = collect_latest_rates_for_accounts_with_retries(
)
```
### Latest closed rate bars
MetaTrader 5 `start_pos=0` includes the still-forming current bar as the last
row. `collect_latest_closed_rates_for_accounts()` fetches `count + 1` bars,
drops that row with `drop_forming_rate_bar()`, and validates each series is
non-empty. Use `collect_latest_closed_rates_by_granularity()` when callers
prefer keys such as `("EURUSD", "M1")` instead of integer timeframes.
```python
from mt5cli import AccountSpec, collect_latest_closed_rates_by_granularity
rates = collect_latest_closed_rates_by_granularity(
[AccountSpec(symbols=["EURUSD"], login=12345)],
["M1", "H1"],
count=500,
retry_count=3,
)
closed_m1 = rates["EURUSD", "M1"]
```
### Resolving credentials and `${ENV_VAR}` placeholders
`resolve_account_spec()` / `resolve_account_specs()` merge explicit override
@@ -78,6 +98,17 @@ finally:
client.shutdown()
```
By default `Mt5TradingError`, `Mt5RuntimeError`, and `sqlite3.Error` propagate so
the caller controls logging; pass `suppress_errors=True` to swallow them and
return `False` without advancing the throttle.
By default recoverable errors (`Mt5TradingError`, `Mt5RuntimeError`,
`sqlite3.Error`, `ValueError`, `OSError`, and MT5 client capability
`AttributeError` / `TypeError` for history API methods) propagate so the caller
controls logging; pass `suppress_errors=True` to swallow them and return
`False` without advancing the throttle. Other `AttributeError` / `TypeError`
values always propagate. Input validation (`_resolve_update_history_request`)
runs before any MT5 or SQLite calls, but when `suppress_errors=True` the
resulting `ValueError` is suppressed along with other recoverable errors.
## Trading-capable sessions
For order placement and trading calculations, use the dedicated
[Trading module](trading.md). The read-only `Mt5CliClient` and `mt5_session()`
helpers in this module are unchanged.
+3
View File
@@ -0,0 +1,3 @@
# Storage
::: mt5cli.storage
+70
View File
@@ -0,0 +1,70 @@
# Trading Module
::: mt5cli.trading
## Trading-capable MT5 sessions
`mt5_trading_session()` complements the read-only `mt5_session()` helper in
`sdk.py`. It yields a connected `pdmt5.Mt5TradingClient`, uses
`Mt5Config.path` to launch the terminal when configured, and always calls
`shutdown()` on exit.
```python
from pdmt5 import Mt5Config
from mt5cli import mt5_trading_session
with mt5_trading_session(
Mt5Config(path=r"C:\Program Files\MetaTrader 5\terminal64.exe", login=12345),
retry_count=2,
) as client:
positions = client.positions_get_as_df(symbol="EURUSD")
```
The read-only `Mt5CliClient` / `mt5_session()` API is unchanged.
## Operational trading helpers
These helpers are strategy-agnostic and do not depend on signal detection,
betting logic, or scheduling code in downstream applications.
```python
from mt5cli import (
calculate_margin_and_volume,
detect_position_side,
determine_order_limits,
)
side = detect_position_side(client, "EURUSD")
sizing = calculate_margin_and_volume(
client,
"EURUSD",
unit_margin_ratio=0.5,
preserved_margin_ratio=0.2,
)
limits = determine_order_limits(
client,
"EURUSD",
side="long",
stop_loss_limit_ratio=0.01,
take_profit_limit_ratio=0.02,
)
```
Protective ratios must satisfy `0 <= ratio < 1`; `0` omits that level.
`calculate_margin_and_volume()` clamps negative `margin_free` to `0.0`
before sizing.
## Migration from mteor-local helpers
| mteor-local concern | mt5cli replacement |
| -------------------------------------------------------- | ----------------------------------------------- |
| Manual terminal spawn/kill around trading code | `mt5_trading_session()` |
| Local position-side detection | `detect_position_side()` |
| Local margin/volume sizing | `calculate_margin_and_volume()` |
| Local SL/TP price derivation | `determine_order_limits()` |
| Throttled SQLite history loop with ad-hoc error handling | `ThrottledHistoryUpdater(suppress_errors=True)` |
Keep read-only data collection on `mt5_session()` / `Mt5CliClient`; use
`mt5_trading_session()` only where order placement or trading calculations are
required.
+39 -31
View File
@@ -1,10 +1,16 @@
# mt5cli
Command-line tool for MetaTrader 5 data export.
Generic MT5 data and execution infrastructure for Python applications.
## Overview
mt5cli is a CLI application that exports MetaTrader 5 trading data to multiple file formats. It is built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data handler for MetaTrader 5.
mt5cli provides a stable `MT5Client` Python API, standardized dataset schemas, storage helpers, and a CLI for exporting MetaTrader 5 data. It is built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data handler for MetaTrader 5.
## Architecture
- **pdmt5** — canonical MT5 client, DataFrame/trading primitives, and MT5 constant parsing (`TIMEFRAME_*`, `COPY_TICKS_*`, order types).
- **mt5cli** — public `MT5Client` API, schema contracts, storage helpers, CLI commands, and SQLite history collection built on pdmt5.
- **mt5api** — sibling HTTP adapter for remote MT5 access; not a dependency of mt5cli.
## Features
@@ -21,66 +27,68 @@ mt5cli is a CLI application that exports MetaTrader 5 trading data to multiple f
pip install mt5cli
```
## Programmatic usage / SDK usage
## Python API for downstream packages
mt5cli can be used as a small Python SDK for read-only MetaTrader 5 data collection. SDK functions return pandas DataFrames without writing files. Use `export_dataframe` or `export_dataframe_to_sqlite` when you need to persist results.
Import `MT5Client` for generic MT5 data access, schema normalization, and optional order primitives. `Mt5CliClient` remains available as a backward-compatible alias.
```python
from datetime import UTC, datetime
from pathlib import Path
from mt5cli import (
Mt5CliClient,
DataKind,
Dataset,
MT5Client,
build_config,
collect_history,
copy_rates_range,
export_dataframe,
export_dataframe_to_sqlite,
load_rate_data,
minimum_margins,
mt5_session,
normalize_dataframe,
recent_ticks,
resolve_rate_view_name,
)
from mt5cli.history import resolve_rate_view_name
# One-off fetch with module-level helpers
rates = copy_rates_range(
"EURUSD",
timeframe="H1",
date_from="2024-01-01",
date_to="2024-02-01",
# Persistent session for multiple calls
with mt5_session(build_config(login=12345, server="Broker-Demo")) as client:
rates = client.copy_rates_range(
"EURUSD",
timeframe="H1",
date_from="2024-01-01",
date_to="2024-02-01",
)
positions = client.positions()
check = client.order_check({"action": 1, "symbol": "EURUSD", "volume": 0.1})
# Normalize MT5 frames to the public schema contract before storage
closed_rates = normalize_dataframe(
rates, DataKind.rates, symbol="EURUSD", timeframe="H1"
)
export_dataframe(rates, Path("rates.csv"), "csv")
export_dataframe(closed_rates, Path("rates.csv"), "csv")
# Resolve SQLite rate compatibility views for downstream tools
# Offline rate loading from mt5cli-managed SQLite history
view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1", require_existing=True)
offline_rates = load_rate_data(Path("history.db"), view, count=1000)
# Recent tick window and minimum margin summary
# One-off helpers still work without instantiating a client
ticks = recent_ticks("EURUSD", seconds=300)
margins = minimum_margins("EURUSD")
# Reuse one MT5 connection for multiple calls
with Mt5CliClient(login=12345, password="secret", server="Broker-Demo") as client:
account = client.account_info()
positions = client.positions()
latest = client.latest_rates("EURUSD", "M1", count=100)
summary = client.mt5_summary()
summary_table = client.mt5_summary_as_df()
# Bulk SQLite collection (same behavior as the collect-history CLI command)
collect_history(
Path("history.db"),
symbols=["EURUSD", "GBPUSD"],
date_from=datetime(2024, 1, 1, tzinfo=UTC),
date_to=datetime(2024, 2, 1, tzinfo=UTC),
timeframe="M1",
flags="ALL",
with_views=True,
datasets={Dataset.rates, Dataset.history_deals},
)
```
Timeframes, tick flags, and ISO 8601 date strings are accepted wherever noted in the SDK API.
Schema contracts live in `mt5cli.schemas` (`DataKind`, `validate_schema`, `normalize_dataframe`). Storage helpers are re-exported from `mt5cli.storage` and the package root.
`Mt5CliClient.mt5_summary()` returns the SDK structured form as plain nested Python values. Use `Mt5CliClient.mt5_summary_as_df()` when you need a one-row DataFrame for export. The `mt5-summary` CLI command uses this tabular form, so nested terminal/account fields are JSON-encoded strings that are safe for CSV, JSON, Parquet, and SQLite output.
`MT5Client.order_send()` is a live execution primitive: it can place real trades on the connected account. mt5cli does not implement strategy logic, signal generation, backtesting, or optimization — downstream applications must gate live execution explicitly (the CLI requires `--yes` for `order-send`).
`MT5Client.mt5_summary()` returns structured nested Python values. Use `MT5Client.mt5_summary_as_df()` when you need a one-row DataFrame for export.
## Quick Start
+7 -1
View File
@@ -1,5 +1,5 @@
site_name: mt5cli API Documentation
site_description: Command-line tool for MetaTrader 5
site_description: Generic MT5 data and execution infrastructure for Python
site_author: dceoy
site_url: https://github.com/dceoy/mt5cli
@@ -56,8 +56,14 @@ nav:
- Home: index.md
- API Reference:
- Overview: api/index.md
- Client: api/client.md
- Schemas: api/schemas.md
- Storage: api/storage.md
- Converters: api/converters.md
- Exceptions: api/exceptions.md
- CLI: api/cli.md
- SDK: api/sdk.md
- Trading: api/trading.md
- History Collection (SQLite): api/history.md
- Utils: api/utils.md
+74 -6
View File
@@ -1,11 +1,30 @@
"""mt5cli: Command-line tool and SDK for MetaTrader 5."""
"""mt5cli: Generic MT5 data and execution infrastructure for Python applications."""
from importlib.metadata import version
from .client import MT5Client, build_config, mt5_session
from .converters import (
ensure_utc,
granularity_name,
normalize_symbol,
normalize_symbols,
parse_date_range,
recent_window,
)
from .exceptions import (
Mt5CliError,
Mt5ConnectionError,
Mt5OperationError,
Mt5SchemaError,
call_with_normalized_errors,
is_recoverable_mt5_error,
normalize_mt5_exception,
)
from .history import (
RateTarget,
build_rate_targets,
build_rate_view_name,
drop_forming_rate_bar,
load_rate_data,
load_rate_data_from_connection,
load_rate_series_by_granularity,
@@ -17,13 +36,25 @@ from .history import (
resolve_rate_view_name,
resolve_rate_view_names,
)
from .schemas import (
DEDUP_KEYS,
KNOWN_MT5_TIME_COLUMNS,
REQUIRED_COLUMNS,
TIME_COLUMNS,
DataKind,
normalize_dataframe,
normalize_time_columns,
schema_columns,
validate_schema,
)
from .sdk import (
AccountSpec,
Mt5CliClient,
ThrottledHistoryUpdater,
account_info,
build_config,
collect_history,
collect_latest_closed_rates_by_granularity,
collect_latest_closed_rates_for_accounts,
collect_latest_rates,
collect_latest_rates_for_accounts,
collect_latest_rates_for_accounts_with_retries,
@@ -38,7 +69,6 @@ from .sdk import (
latest_rates,
market_book,
minimum_margins,
mt5_session,
mt5_summary,
mt5_summary_as_df,
orders,
@@ -58,14 +88,22 @@ from .sdk import (
from .sdk import (
version as mt5_version,
)
from .utils import (
TICK_FLAG_MAP,
TIMEFRAME_MAP,
from .storage import (
Dataset,
IfExists,
detect_format,
export_dataframe,
export_dataframe_to_sqlite,
)
from .trading import (
calculate_margin_and_volume,
detect_position_side,
determine_order_limits,
mt5_trading_session,
)
from .utils import (
TICK_FLAG_MAP,
TIMEFRAME_MAP,
parse_datetime,
parse_tick_flags,
parse_timeframe,
@@ -74,19 +112,33 @@ from .utils import (
__version__ = version(__package__) if __package__ else None
__all__ = [
"DEDUP_KEYS",
"KNOWN_MT5_TIME_COLUMNS",
"REQUIRED_COLUMNS",
"TICK_FLAG_MAP",
"TIMEFRAME_MAP",
"TIME_COLUMNS",
"AccountSpec",
"DataKind",
"Dataset",
"IfExists",
"MT5Client",
"Mt5CliClient",
"Mt5CliError",
"Mt5ConnectionError",
"Mt5OperationError",
"Mt5SchemaError",
"RateTarget",
"ThrottledHistoryUpdater",
"account_info",
"build_config",
"build_rate_targets",
"build_rate_view_name",
"calculate_margin_and_volume",
"call_with_normalized_errors",
"collect_history",
"collect_latest_closed_rates_by_granularity",
"collect_latest_closed_rates_for_accounts",
"collect_latest_rates",
"collect_latest_rates_for_accounts",
"collect_latest_rates_for_accounts_with_retries",
@@ -96,10 +148,16 @@ __all__ = [
"copy_ticks_from",
"copy_ticks_range",
"detect_format",
"detect_position_side",
"determine_order_limits",
"drop_forming_rate_bar",
"ensure_utc",
"export_dataframe",
"export_dataframe_to_sqlite",
"granularity_name",
"history_deals",
"history_orders",
"is_recoverable_mt5_error",
"last_error",
"latest_rates",
"load_rate_data",
@@ -111,14 +169,22 @@ __all__ = [
"mt5_session",
"mt5_summary",
"mt5_summary_as_df",
"mt5_trading_session",
"mt5_version",
"normalize_dataframe",
"normalize_mt5_exception",
"normalize_symbol",
"normalize_symbols",
"normalize_time_columns",
"orders",
"parse_date_range",
"parse_datetime",
"parse_tick_flags",
"parse_timeframe",
"positions",
"recent_history_deals",
"recent_ticks",
"recent_window",
"resolve_account_spec",
"resolve_account_specs",
"resolve_history_datasets",
@@ -127,6 +193,7 @@ __all__ = [
"resolve_rate_tables",
"resolve_rate_view_name",
"resolve_rate_view_names",
"schema_columns",
"substitute_env_placeholders",
"symbol_info",
"symbol_info_tick",
@@ -134,4 +201,5 @@ __all__ = [
"terminal_info",
"update_history",
"update_history_with_config",
"validate_schema",
]
+60 -74
View File
@@ -12,6 +12,7 @@ import typer
from pdmt5 import Mt5Config
from . import sdk
from .client import MT5Client
from .utils import (
DATETIME_TYPE,
REQUEST_TYPE,
@@ -91,9 +92,18 @@ def _execute_export(
)
def _sdk_client(ctx: typer.Context) -> sdk.Mt5CliClient:
def _sdk_client(ctx: typer.Context) -> MT5Client:
export_ctx = _get_export_context(ctx)
return sdk.Mt5CliClient(config=export_ctx.config)
return MT5Client(config=export_ctx.config)
def _export_command(
ctx: typer.Context,
fetch_fn: Callable[[MT5Client], pd.DataFrame],
) -> None:
"""Create an SDK client, fetch a DataFrame, and export it."""
client = _sdk_client(ctx)
_execute_export(ctx, lambda: fetch_fn(client))
@app.callback()
@@ -193,10 +203,9 @@ def rates_from(
count: Annotated[int, typer.Option(help="Number of records.")],
) -> None:
"""Export rates from a start date."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.copy_rates_from(symbol, timeframe, date_from, count),
lambda client: client.copy_rates_from(symbol, timeframe, date_from, count),
)
@@ -215,10 +224,14 @@ def rates_from_pos(
count: Annotated[int, typer.Option(help="Number of records.")],
) -> None:
"""Export rates from a start position."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.copy_rates_from_pos(symbol, timeframe, start_pos, count),
lambda client: client.copy_rates_from_pos(
symbol,
timeframe,
start_pos,
count,
),
)
@@ -240,10 +253,14 @@ def latest_rates(
] = 0,
) -> None:
"""Export latest rates from a start position."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.latest_rates(symbol, timeframe, count, start_pos=start_pos),
lambda client: client.latest_rates(
symbol,
timeframe,
count,
start_pos=start_pos,
),
)
@@ -268,10 +285,9 @@ def rates_range(
],
) -> None:
"""Export rates for a date range."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.copy_rates_range(symbol, timeframe, date_from, date_to),
lambda client: client.copy_rates_range(symbol, timeframe, date_from, date_to),
)
@@ -293,10 +309,9 @@ def ticks_from(
],
) -> None:
"""Export ticks from a start date."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.copy_ticks_from(symbol, date_from, count, flags),
lambda client: client.copy_ticks_from(symbol, date_from, count, flags),
)
@@ -318,10 +333,9 @@ def ticks_range(
],
) -> None:
"""Export ticks for a date range."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.copy_ticks_range(symbol, date_from, date_to, flags),
lambda client: client.copy_ticks_range(symbol, date_from, date_to, flags),
)
@@ -347,13 +361,12 @@ def ticks_recent(
click_type=TICK_FLAGS_TYPE,
help="Tick flags (ALL, INFO, TRADE, or integer).",
),
] = 1,
] = "ALL", # pyright: ignore[reportArgumentType]
) -> None:
"""Export ticks from a recent time window."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.recent_ticks(
lambda client: client.recent_ticks(
symbol,
seconds,
date_to=date_to,
@@ -366,13 +379,13 @@ def ticks_recent(
@app.command()
def account_info(ctx: typer.Context) -> None:
"""Export account information."""
_execute_export(ctx, _sdk_client(ctx).account_info)
_export_command(ctx, lambda client: client.account_info())
@app.command()
def terminal_info(ctx: typer.Context) -> None:
"""Export terminal information."""
_execute_export(ctx, _sdk_client(ctx).terminal_info)
_export_command(ctx, lambda client: client.terminal_info())
@app.command()
@@ -384,8 +397,7 @@ def symbols(
] = None,
) -> None:
"""Export symbol list."""
client = _sdk_client(ctx)
_execute_export(ctx, lambda: client.symbols(group=group))
_export_command(ctx, lambda client: client.symbols(group=group))
@app.command()
@@ -394,8 +406,7 @@ def symbol_info(
symbol: Annotated[str, typer.Option(help="Symbol name.")],
) -> None:
"""Export symbol details."""
client = _sdk_client(ctx)
_execute_export(ctx, lambda: client.symbol_info(symbol))
_export_command(ctx, lambda client: client.symbol_info(symbol))
@app.command()
@@ -404,8 +415,7 @@ def minimum_margins(
symbol: Annotated[str, typer.Option(help="Symbol name.")],
) -> None:
"""Export minimum-volume buy and sell margin requirements."""
client = _sdk_client(ctx)
_execute_export(ctx, lambda: client.minimum_margins(symbol))
_export_command(ctx, lambda client: client.minimum_margins(symbol))
@app.command()
@@ -416,10 +426,9 @@ def orders(
ticket: Annotated[int | None, typer.Option(help="Ticket filter.")] = None,
) -> None:
"""Export active orders."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.orders(symbol=symbol, group=group, ticket=ticket),
lambda client: client.orders(symbol=symbol, group=group, ticket=ticket),
)
@@ -431,10 +440,9 @@ def positions(
ticket: Annotated[int | None, typer.Option(help="Ticket filter.")] = None,
) -> None:
"""Export open positions."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.positions(symbol=symbol, group=group, ticket=ticket),
lambda client: client.positions(symbol=symbol, group=group, ticket=ticket),
)
@@ -455,10 +463,9 @@ def history_orders(
position: Annotated[int | None, typer.Option(help="Position ticket.")] = None,
) -> None:
"""Export historical orders."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.history_orders(
lambda client: client.history_orders(
date_from=date_from,
date_to=date_to,
group=group,
@@ -486,10 +493,9 @@ def history_deals(
position: Annotated[int | None, typer.Option(help="Position ticket.")] = None,
) -> None:
"""Export historical deals."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.history_deals(
lambda client: client.history_deals(
date_from=date_from,
date_to=date_to,
group=group,
@@ -512,10 +518,9 @@ def recent_history_deals(
symbol: Annotated[str | None, typer.Option(help="Symbol filter.")] = None,
) -> None:
"""Export historical deals from a recent trailing window."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.recent_history_deals(
lambda client: client.recent_history_deals(
hours,
date_to=date_to,
group=group,
@@ -527,20 +532,19 @@ def recent_history_deals(
@app.command()
def mt5_summary(ctx: typer.Context) -> None:
"""Export a compact terminal/account status summary."""
client = _sdk_client(ctx)
_execute_export(ctx, client.mt5_summary_as_df)
_export_command(ctx, lambda client: client.mt5_summary_as_df())
@app.command()
def version(ctx: typer.Context) -> None:
"""Export MetaTrader5 version information."""
_execute_export(ctx, _sdk_client(ctx).version)
_export_command(ctx, lambda client: client.version())
@app.command()
def last_error(ctx: typer.Context) -> None:
"""Export the last error information."""
_execute_export(ctx, _sdk_client(ctx).last_error)
_export_command(ctx, lambda client: client.last_error())
@app.command()
@@ -549,8 +553,7 @@ def symbol_info_tick(
symbol: Annotated[str, typer.Option(help="Symbol name.")],
) -> None:
"""Export the last tick for a symbol."""
client = _sdk_client(ctx)
_execute_export(ctx, lambda: client.symbol_info_tick(symbol))
_export_command(ctx, lambda client: client.symbol_info_tick(symbol))
@app.command()
@@ -559,8 +562,7 @@ def market_book(
symbol: Annotated[str, typer.Option(help="Symbol name.")],
) -> None:
"""Export market depth (order book) for a symbol."""
client = _sdk_client(ctx)
_execute_export(ctx, lambda: client.market_book(symbol))
_export_command(ctx, lambda client: client.market_book(symbol))
@app.command()
@@ -572,15 +574,7 @@ def order_check(
],
) -> None:
"""Check funds sufficiency for a trading operation."""
export_ctx = _get_export_context(ctx)
def _fetch() -> pd.DataFrame:
return sdk._run_with_client( # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
export_ctx.config,
lambda c: c.order_check_as_df(request=request),
)
_execute_export(ctx, _fetch)
_export_command(ctx, lambda client: client.order_check(request))
@app.command()
@@ -603,15 +597,7 @@ def order_send(
if not yes:
msg = "Pass --yes to send a live trade request."
raise typer.BadParameter(msg, param_hint="--yes")
export_ctx = _get_export_context(ctx)
def _fetch() -> pd.DataFrame:
return sdk._run_with_client( # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
export_ctx.config,
lambda c: c.order_send_as_df(request=request),
)
_execute_export(ctx, _fetch)
_export_command(ctx, lambda client: client.order_send(request))
@app.command()
@@ -656,7 +642,7 @@ def collect_history(
click_type=TICK_FLAGS_TYPE,
help="Tick copy flags (ALL, INFO, TRADE, or integer).",
),
] = 1,
] = "ALL", # pyright: ignore[reportArgumentType]
if_exists: Annotated[
IfExists,
typer.Option(
+88
View File
@@ -0,0 +1,88 @@
"""Stable public client abstraction for MT5 data and execution operations."""
from __future__ import annotations
from contextlib import contextmanager
from typing import TYPE_CHECKING, Any, Self
from .sdk import Mt5CliClient, build_config, connected_client
if TYPE_CHECKING:
from collections.abc import Iterator
import pandas as pd
from pdmt5 import Mt5Config, Mt5DataClient
__all__ = [
"MT5Client",
"build_config",
"mt5_session",
]
class MT5Client(Mt5CliClient):
"""Public client for generic MT5 data access and order primitives.
Extends the read-only SDK client with optional order check/send helpers and
exposes the same connection lifecycle as :class:`~mt5cli.sdk.Mt5CliClient`.
Downstream applications such as private trading packages should prefer this
type over the legacy ``Mt5CliClient`` name.
mt5cli intentionally exposes minimal execution primitives only. Trading
decisions, signals, strategies, backtests, and optimization remain the
responsibility of downstream applications.
"""
def order_check(self, request: dict[str, Any]) -> pd.DataFrame:
"""Check funds sufficiency for a trade request.
Args:
request: MT5 order request dictionary.
Returns:
One-row DataFrame with the order-check result.
"""
return self._fetch(lambda client: client.order_check_as_df(request=request))
def order_send(self, request: dict[str, Any]) -> pd.DataFrame:
"""Send a live trade request to the MT5 trade server.
Warning:
This is a live execution primitive. A successful call can place,
modify, or close real trades on the connected account. Downstream
applications must gate usage explicitly (for example behind manual
confirmation or application-specific risk controls). mt5cli does
not implement strategy logic, signal generation, or trade sizing.
Args:
request: MT5 order request dictionary.
Returns:
One-row DataFrame with the order-send result.
"""
return self._fetch(lambda client: client.order_send_as_df(request=request))
@classmethod
def from_connected_client(cls, client: Mt5DataClient) -> Self:
"""Bind to an already-connected ``Mt5DataClient`` without owning it.
Returns:
Client wrapper bound to the injected connection.
"""
return cls(client=client)
@contextmanager
def mt5_session(config: Mt5Config | None = None) -> Iterator[MT5Client]:
"""Open an MT5 terminal session and yield a connected :class:`MT5Client`.
Args:
config: MT5 connection configuration. Defaults to an empty config that
attaches to a running terminal.
Yields:
Connected :class:`MT5Client` bound to the session.
"""
mt5_config = config or build_config()
with connected_client(mt5_config) as client:
yield MT5Client.from_connected_client(client)
+162
View File
@@ -0,0 +1,162 @@
"""Shared conversion helpers for MT5 symbols, timeframes, and date ranges."""
from __future__ import annotations
from datetime import UTC, datetime, timedelta
from typing import TYPE_CHECKING
from pdmt5 import get_timeframe_name as _get_timeframe_name
from .utils import parse_datetime, parse_tick_flags, parse_timeframe
if TYPE_CHECKING:
from collections.abc import Sequence
__all__ = [
"ensure_utc",
"granularity_name",
"normalize_symbol",
"normalize_symbols",
"parse_date_range",
"parse_datetime",
"parse_tick_flags",
"parse_timeframe",
"recent_window",
]
def normalize_symbol(symbol: str) -> str:
"""Normalize a broker symbol name for MT5 API calls.
Strips surrounding whitespace while preserving broker-specific casing and
suffixes (for example ``XAUUSDm``, ``US500.cash``, or ``EURUSD.r``).
Args:
symbol: Raw symbol name.
Returns:
Normalized symbol string.
Raises:
ValueError: If the symbol is empty after normalization.
"""
normalized = symbol.strip()
if not normalized:
msg = "Symbol must not be empty."
raise ValueError(msg)
return normalized
def normalize_symbols(symbols: Sequence[str]) -> list[str]:
"""Normalize a sequence of broker symbol names.
Args:
symbols: Raw symbol names.
Returns:
List of normalized, de-duplicated symbols preserving first-seen order.
"""
seen: set[str] = set()
resolved: list[str] = []
for symbol in symbols:
normalized = normalize_symbol(symbol)
if normalized not in seen:
seen.add(normalized)
resolved.append(normalized)
return resolved
def ensure_utc(value: datetime | str) -> datetime:
"""Return a timezone-aware UTC datetime.
Args:
value: Datetime instance or ISO 8601 string.
Returns:
UTC-aware datetime.
"""
if isinstance(value, str):
return parse_datetime(value)
if value.tzinfo is None:
return value.replace(tzinfo=UTC)
return value.astimezone(UTC)
def parse_date_range(
date_from: datetime | str,
date_to: datetime | str,
) -> tuple[datetime, datetime]:
"""Parse and validate an inclusive UTC date range.
Args:
date_from: Range start as datetime or ISO 8601 string.
date_to: Range end as datetime or ISO 8601 string.
Returns:
Tuple of UTC-aware ``(start, end)`` datetimes.
Raises:
ValueError: If ``date_from`` is after ``date_to``.
"""
start = ensure_utc(date_from)
end = ensure_utc(date_to)
if start > end:
msg = (
f"date_from ({start.isoformat()}) must not be after "
f"date_to ({end.isoformat()})."
)
raise ValueError(msg)
return start, end
def recent_window(
*,
hours: float | None = None,
seconds: float | None = None,
date_to: datetime | str | None = None,
) -> tuple[datetime, datetime]:
"""Build a trailing UTC window ending at ``date_to`` or now.
Exactly one of ``hours`` or ``seconds`` must be provided.
Args:
hours: Trailing window length in hours.
seconds: Trailing window length in seconds.
date_to: Window end. Defaults to current UTC time.
Returns:
Tuple of UTC-aware ``(start, end)`` datetimes.
Raises:
ValueError: If neither or both window lengths are provided, or if a
length is not positive.
"""
if (hours is None) == (seconds is None):
msg = "Provide exactly one of hours or seconds."
raise ValueError(msg)
if hours is not None:
length = timedelta(hours=hours)
else:
length = timedelta(seconds=seconds if seconds is not None else 0)
if length.total_seconds() <= 0:
msg = "Window length must be positive."
raise ValueError(msg)
end = ensure_utc(date_to) if date_to is not None else datetime.now(UTC)
return end - length, end
def granularity_name(timeframe: int | str) -> str:
"""Return a short granularity label for a timeframe integer or name.
Args:
timeframe: MT5 timeframe as integer or name (for example ``M1``).
Returns:
Short name such as ``M1`` or the stringified integer when unknown.
"""
tf = parse_timeframe(timeframe)
try:
name = _get_timeframe_name(tf)
except ValueError:
return str(tf)
return name.removeprefix("TIMEFRAME_")
+90
View File
@@ -0,0 +1,90 @@
"""Normalized exception types for MT5 and mt5cli operations."""
from __future__ import annotations
from typing import TYPE_CHECKING, TypeVar
from pdmt5 import Mt5RuntimeError, Mt5TradingError
if TYPE_CHECKING:
from collections.abc import Callable
T = TypeVar("T")
__all__ = [
"Mt5CliError",
"Mt5ConnectionError",
"Mt5OperationError",
"Mt5SchemaError",
"call_with_normalized_errors",
"is_recoverable_mt5_error",
"normalize_mt5_exception",
]
_RECOVERABLE_MT5_ERRORS: tuple[type[BaseException], ...] = (
Mt5TradingError,
Mt5RuntimeError,
)
class Mt5CliError(Exception):
"""Base exception for mt5cli public API errors."""
class Mt5ConnectionError(Mt5CliError):
"""Raised when MT5 initialization, login, or shutdown fails."""
class Mt5OperationError(Mt5CliError):
"""Raised when an MT5 data or trading operation fails."""
class Mt5SchemaError(Mt5CliError):
"""Raised when a DataFrame does not match an expected dataset schema."""
def is_recoverable_mt5_error(exc: BaseException) -> bool:
"""Return whether an exception is a transient MT5 failure worth retrying.
Args:
exc: Exception raised by MT5 or pdmt5.
Returns:
True for ``Mt5RuntimeError`` and ``Mt5TradingError``.
"""
return isinstance(exc, _RECOVERABLE_MT5_ERRORS)
def normalize_mt5_exception(exc: BaseException) -> Mt5CliError:
"""Map pdmt5/MT5 exceptions to stable mt5cli exception types.
Args:
exc: Original exception from MT5 or pdmt5.
Returns:
``Mt5ConnectionError`` for runtime failures, ``Mt5OperationError`` for
trading failures, or the original exception when it is not recognized.
"""
if isinstance(exc, Mt5TradingError):
return Mt5OperationError(str(exc))
if isinstance(exc, Mt5RuntimeError):
return Mt5ConnectionError(str(exc))
if isinstance(exc, Mt5CliError):
return exc
return Mt5CliError(str(exc))
def call_with_normalized_errors(fn: Callable[[], T]) -> T:
"""Run ``fn`` and map recoverable MT5 errors to mt5cli types.
Args:
fn: Callable performing MT5 work.
Returns:
Value returned by ``fn``.
"""
try:
return fn()
except _RECOVERABLE_MT5_ERRORS as exc:
normalized = normalize_mt5_exception(exc)
raise normalized from exc
+124 -63
View File
@@ -10,9 +10,11 @@ from pathlib import Path
from typing import TYPE_CHECKING, Literal, cast
import pandas as pd
from pdmt5 import get_timeframe_name as _get_timeframe_name
from .schemas import DEDUP_KEYS, DataKind
from .utils import (
TIMEFRAME_MAP,
TIMEFRAME_NAMES,
Dataset,
IfExists,
parse_datetime,
@@ -27,13 +29,13 @@ if TYPE_CHECKING:
logger = logging.getLogger(__name__)
DEFAULT_HISTORY_TIMEFRAMES: tuple[str, ...] = tuple(TIMEFRAME_MAP)
DEFAULT_HISTORY_TIMEFRAMES: tuple[str, ...] = TIMEFRAME_NAMES
_HISTORY_DEDUP_KEYS: dict[Dataset, tuple[tuple[str, ...], ...]] = {
Dataset.rates: (("symbol", "timeframe", "time"), ("symbol", "time")),
Dataset.ticks: (("symbol", "time_msc"), ("symbol", "time")),
Dataset.history_orders: (("ticket",), ("symbol", "time", "type")),
Dataset.history_deals: (("ticket",), ("symbol", "time", "type", "entry")),
Dataset.rates: DEDUP_KEYS[DataKind.rates],
Dataset.ticks: DEDUP_KEYS[DataKind.ticks],
Dataset.history_orders: DEDUP_KEYS[DataKind.history_orders],
Dataset.history_deals: DEDUP_KEYS[DataKind.history_deals],
}
_TRADE_DEAL_TYPES: tuple[int, int] = (0, 1)
@@ -80,7 +82,7 @@ def resolve_history_timeframes(
seen: set[int] = set()
resolved: list[int] = []
for value in raw:
tf = value if isinstance(value, int) else parse_timeframe(str(value))
tf = parse_timeframe(value)
if tf not in seen:
seen.add(tf)
resolved.append(tf)
@@ -93,17 +95,33 @@ def resolve_history_tick_flags(flags: int | str) -> int:
Returns:
Integer tick flag value.
"""
if isinstance(flags, int):
return flags
return parse_tick_flags(flags)
def resolve_granularity_name(timeframe: int) -> str:
"""Return a granularity name for a timeframe integer when known."""
for name, value in TIMEFRAME_MAP.items():
if value == timeframe:
return name
return str(timeframe)
try:
name = _get_timeframe_name(timeframe)
except ValueError:
return str(timeframe)
return name.removeprefix("TIMEFRAME_")
def drop_forming_rate_bar(df_rate: pd.DataFrame) -> pd.DataFrame:
"""Return closed bars from chronologically ordered MT5 rate data.
MetaTrader 5 ``copy_rates_from_pos(start_pos=0)`` includes the still-forming
current bar as the last row. Slice it off so downstream logic only sees
completed bars. Empty frames and single-row frames return empty results.
Args:
df_rate: Rate data ordered oldest-to-newest with the forming bar last.
Returns:
A new DataFrame with all rows except the last. Index and columns are
preserved. The input frame is not modified.
"""
return df_rate.iloc[:-1].copy()
def build_rate_view_name(
@@ -1315,6 +1333,50 @@ def create_rate_compatibility_views(conn: sqlite3.Connection) -> None:
)
def _stream_symbol_frames(
conn: sqlite3.Connection,
symbols: Sequence[str],
dataset: Dataset,
if_exists: IfExists,
written_columns: dict[Dataset, set[str]],
fetch_frame: Callable[[str], pd.DataFrame],
) -> bool:
"""Stream per-symbol frames into SQLite.
Returns:
True if the dataset table was written.
"""
table_exists = False
for sym in symbols:
table_exists = write_streamed_frame(
conn,
fetch_frame(sym),
dataset,
table_exists,
if_exists,
written_columns,
)
return table_exists
def _record_symbol_time_dedup(
dedup_scopes: dict[Dataset, list[DedupScope]],
written_tables: set[Dataset],
dataset: Dataset,
symbol: str,
start_date: datetime,
) -> None:
"""Record a symbol-scoped deduplication window after an incremental write."""
written_tables.add(dataset)
_record_dedup_scope(
dedup_scopes,
dataset,
"symbol = ? AND time >= ?",
(symbol, start_date),
frozenset({"symbol", "time"}),
)
def write_rates_dataset(
conn: sqlite3.Connection,
client: Mt5DataClient,
@@ -1330,8 +1392,8 @@ def write_rates_dataset(
Returns:
True if the rates table was written.
"""
table_exists = False
for sym in symbols:
def _fetch_rates_frame(sym: str) -> pd.DataFrame:
frame = client.copy_rates_range_as_df(
symbol=sym,
timeframe=timeframe,
@@ -1341,15 +1403,16 @@ def write_rates_dataset(
if len(frame.columns) != 0:
frame.insert(0, "symbol", sym)
frame.insert(1, "timeframe", timeframe)
table_exists = write_streamed_frame(
conn,
frame,
Dataset.rates,
table_exists,
if_exists,
written_columns,
)
return table_exists
return frame
return _stream_symbol_frames(
conn,
symbols,
Dataset.rates,
if_exists,
written_columns,
_fetch_rates_frame,
)
def write_ticks_dataset(
@@ -1367,8 +1430,8 @@ def write_ticks_dataset(
Returns:
True if the ticks table was written.
"""
table_exists = False
for sym in symbols:
def _fetch_ticks_frame(sym: str) -> pd.DataFrame:
frame = client.copy_ticks_range_as_df(
symbol=sym,
date_from=date_from,
@@ -1377,15 +1440,16 @@ def write_ticks_dataset(
).drop(columns=["symbol"], errors="ignore")
if len(frame.columns) != 0:
frame.insert(0, "symbol", sym)
table_exists = write_streamed_frame(
conn,
frame,
Dataset.ticks,
table_exists,
if_exists,
written_columns,
)
return table_exists
return frame
return _stream_symbol_frames(
conn,
symbols,
Dataset.ticks,
if_exists,
written_columns,
_fetch_ticks_frame,
)
def write_history_dataset(
@@ -1420,22 +1484,22 @@ def write_history_dataset(
if_exists,
written_columns,
)
for sym in symbols:
frame = fetch(date_from=date_from, date_to=date_to, symbol=sym)
frame = filter_trade_history_frame(
frame,
def _fetch_history_frame(sym: str) -> pd.DataFrame:
return filter_trade_history_frame(
fetch(date_from=date_from, date_to=date_to, symbol=sym),
[sym],
include_account_events=False,
)
table_exists = write_streamed_frame(
conn,
frame,
dataset,
table_exists,
if_exists,
written_columns,
)
return table_exists
return _stream_symbol_frames(
conn,
symbols,
dataset,
if_exists,
written_columns,
_fetch_history_frame,
)
def _write_incremental_rates(
@@ -1508,13 +1572,12 @@ def _write_incremental_ticks(
IfExists.APPEND,
written_columns,
):
written_tables.add(Dataset.ticks)
_record_dedup_scope(
_record_symbol_time_dedup(
dedup_scopes,
written_tables,
Dataset.ticks,
"symbol = ? AND time >= ?",
(symbol, start_date),
frozenset({"symbol", "time"}),
symbol,
start_date,
)
@@ -1547,13 +1610,12 @@ def _write_incremental_history_orders(
written_columns,
include_account_events=False,
):
written_tables.add(Dataset.history_orders)
_record_dedup_scope(
_record_symbol_time_dedup(
dedup_scopes,
written_tables,
Dataset.history_orders,
"symbol = ? AND time >= ?",
(symbol, start_date),
frozenset({"symbol", "time"}),
symbol,
start_date,
)
@@ -1645,13 +1707,12 @@ def _write_incremental_history_deals(
written_columns,
include_account_events=False,
):
written_tables.add(Dataset.history_deals)
_record_dedup_scope(
_record_symbol_time_dedup(
dedup_scopes,
written_tables,
Dataset.history_deals,
"symbol = ? AND time >= ?",
(symbol, start_date),
frozenset({"symbol", "time"}),
symbol,
start_date,
)
+64
View File
@@ -0,0 +1,64 @@
"""Retry and reconnect helpers for transient MT5 failures."""
from __future__ import annotations
import logging
import time
from typing import TYPE_CHECKING, TypeVar
from .exceptions import is_recoverable_mt5_error
if TYPE_CHECKING:
from collections.abc import Callable
T = TypeVar("T")
logger = logging.getLogger(__name__)
__all__ = [
"retry_with_backoff",
]
def retry_with_backoff(
fn: Callable[[], T],
*,
retry_count: int = 0,
backoff_base: float = 2.0,
operation: str = "MT5 operation",
) -> T:
"""Call ``fn`` with bounded exponential backoff on recoverable MT5 errors.
Only ``pdmt5.Mt5RuntimeError`` and ``pdmt5.Mt5TradingError`` are retried.
Other exceptions propagate immediately. The final failure is re-raised once
retries are exhausted.
Args:
fn: Callable performing MT5 work.
retry_count: Maximum number of retries after the first attempt. ``0``
disables retries.
backoff_base: Base for exponential backoff. The delay before retry
attempt ``n`` (1-indexed) is ``backoff_base ** n`` seconds.
operation: Label used in warning logs.
Returns:
Value returned by ``fn`` on success.
"""
attempts = max(retry_count, 0) + 1
for attempt in range(attempts - 1):
try:
return fn()
except Exception as exc:
if not is_recoverable_mt5_error(exc):
raise
delay = backoff_base ** (attempt + 1)
logger.warning(
"%s failed (attempt %d/%d): %s; retrying in %.1fs",
operation,
attempt + 1,
attempts,
exc,
delay,
)
time.sleep(delay)
return fn()
+291
View File
@@ -0,0 +1,291 @@
"""Canonical DataFrame schemas for MT5 market and account datasets."""
from __future__ import annotations
from enum import StrEnum
from typing import TYPE_CHECKING, Final
import pandas as pd
from .converters import normalize_symbol, parse_timeframe
from .exceptions import Mt5SchemaError
if TYPE_CHECKING:
from collections.abc import Iterable
__all__ = [
"DEDUP_KEYS",
"KNOWN_MT5_TIME_COLUMNS",
"REQUIRED_COLUMNS",
"TIME_COLUMNS",
"DataKind",
"normalize_dataframe",
"normalize_time_columns",
"schema_columns",
"validate_schema",
]
KNOWN_MT5_TIME_COLUMNS: Final[frozenset[str]] = frozenset({
"time",
"time_setup",
"time_setup_msc",
"time_done",
"time_done_msc",
"time_msc",
})
_TIME_COLUMN_NAMES = KNOWN_MT5_TIME_COLUMNS
class DataKind(StrEnum):
"""Supported MT5 dataset kinds with canonical column contracts."""
rates = "rates"
ticks = "ticks"
orders = "orders"
positions = "positions"
history_orders = "history_orders"
history_deals = "history_deals"
REQUIRED_COLUMNS: dict[DataKind, frozenset[str]] = {
DataKind.rates: frozenset({
"time",
"open",
"high",
"low",
"close",
"tick_volume",
"spread",
"real_volume",
}),
DataKind.ticks: frozenset({
"time",
"bid",
"ask",
"last",
"volume",
"time_msc",
"flags",
"volume_real",
}),
DataKind.orders: frozenset({
"ticket",
"time_setup",
"type",
"state",
"symbol",
"volume_current",
"price_open",
}),
DataKind.positions: frozenset({
"ticket",
"time",
"type",
"symbol",
"volume",
"price_open",
"price_current",
"profit",
}),
DataKind.history_orders: frozenset({
"ticket",
"time_setup",
"type",
"state",
"symbol",
"volume_initial",
"price_open",
}),
DataKind.history_deals: frozenset({
"ticket",
"order",
"time",
"type",
"entry",
"symbol",
"volume",
"price",
"profit",
}),
}
_OPTIONAL_TIME_COLUMNS_BY_KIND: dict[DataKind, frozenset[str]] = {
DataKind.orders: frozenset({
"time_setup_msc",
"time_done",
"time_done_msc",
}),
DataKind.history_orders: frozenset({
"time_setup_msc",
"time_done",
"time_done_msc",
}),
DataKind.positions: frozenset({"time_msc"}),
}
TIME_COLUMNS: dict[DataKind, frozenset[str]] = {
kind: (REQUIRED_COLUMNS[kind] & _TIME_COLUMN_NAMES)
| _OPTIONAL_TIME_COLUMNS_BY_KIND.get(kind, frozenset())
for kind in DataKind
}
DEDUP_KEYS: dict[DataKind, tuple[tuple[str, ...], ...]] = {
DataKind.rates: (("symbol", "timeframe", "time"), ("symbol", "time")),
DataKind.ticks: (("symbol", "time_msc"), ("symbol", "time")),
DataKind.history_orders: (("ticket",), ("symbol", "time", "type")),
DataKind.history_deals: (("ticket",), ("symbol", "time", "type", "entry")),
}
def schema_columns(kind: DataKind) -> frozenset[str]:
"""Return required column names for a dataset kind.
Args:
kind: Dataset kind.
Returns:
Required column names for ``kind``.
"""
return REQUIRED_COLUMNS[kind]
def validate_schema(
frame: pd.DataFrame,
kind: DataKind,
*,
extra_required: Iterable[str] | None = None,
) -> None:
"""Validate that a DataFrame includes required columns for a dataset kind.
Args:
frame: DataFrame to validate.
kind: Expected dataset kind.
extra_required: Additional columns that must be present (for example
``symbol`` and ``timeframe`` on stored rate history).
Raises:
Mt5SchemaError: If required columns are missing.
"""
if frame.empty and len(frame.columns) == 0:
return
required = set(REQUIRED_COLUMNS[kind])
if extra_required is not None:
required.update(extra_required)
missing = required - set(frame.columns)
if missing:
msg = (
f"{kind.value} schema is missing required columns: "
f"{', '.join(sorted(missing))}."
)
raise Mt5SchemaError(msg)
def _coerce_mt5_time_column(series: pd.Series, column: str) -> pd.Series:
"""Coerce one MT5 time column to UTC-aware datetimes.
Returns:
Series with UTC-aware datetime values.
"""
if pd.api.types.is_datetime64_any_dtype(series):
return pd.to_datetime(series, utc=True, errors="coerce")
if pd.api.types.is_numeric_dtype(series):
unit = "ms" if column.endswith("_msc") else "s"
return pd.to_datetime(series, unit=unit, utc=True, errors="coerce")
return pd.to_datetime(series, utc=True, errors="coerce")
def normalize_time_columns(frame: pd.DataFrame, kind: DataKind) -> pd.DataFrame:
"""Coerce dataset time columns to UTC-aware datetimes when present.
Any column in :data:`KNOWN_MT5_TIME_COLUMNS` that is present in ``frame``
is normalized. Numeric MT5 epoch values use seconds for ``time``,
``time_setup``, and ``time_done``, and milliseconds for ``*_msc`` columns.
Args:
frame: Source DataFrame from MT5 or pdmt5.
kind: Dataset kind (retained for API compatibility).
Returns:
DataFrame copy with normalized time columns.
"""
del kind
normalized = frame.copy()
for column in normalized.columns:
if column not in _TIME_COLUMN_NAMES:
continue
normalized[column] = _coerce_mt5_time_column(normalized[column], column)
return normalized
def normalize_dataframe(
frame: pd.DataFrame,
kind: DataKind,
*,
symbol: str | None = None,
timeframe: int | str | None = None,
sort: bool = True,
) -> pd.DataFrame:
"""Normalize MT5 DataFrame columns, timestamps, and storage metadata.
Ensures UTC timestamps, optionally injects ``symbol`` / ``timeframe`` for
storage-oriented datasets, and sorts chronologically when a ``time`` column
exists.
Args:
frame: Source DataFrame from MT5 or pdmt5.
kind: Dataset kind guiding normalization rules.
symbol: Optional symbol to inject when missing.
timeframe: Optional timeframe integer or name to inject for rates.
sort: Whether to sort by ``time`` or ``time_msc`` when present.
Returns:
Normalized DataFrame copy.
"""
if frame.empty and len(frame.columns) == 0:
return frame.copy()
normalized = normalize_time_columns(frame, kind)
if symbol is not None and "symbol" not in normalized.columns:
normalized.insert(0, "symbol", normalize_symbol(symbol))
if timeframe is not None and kind is DataKind.rates:
tf = parse_timeframe(timeframe)
if "timeframe" not in normalized.columns:
insert_at = 1 if "symbol" in normalized.columns else 0
normalized.insert(insert_at, "timeframe", tf)
validate_schema(normalized, kind)
if sort:
if "time" in normalized.columns:
normalized = normalized.sort_values("time", kind="stable")
elif "time_msc" in normalized.columns:
normalized = normalized.sort_values("time_msc", kind="stable")
normalized = normalized.reset_index(drop=True)
return normalized
def ensure_utc_columns(frame: pd.DataFrame, columns: Iterable[str]) -> pd.DataFrame:
"""Return a copy with selected columns coerced to UTC datetimes.
Args:
frame: Source DataFrame.
columns: Column names to coerce.
Returns:
DataFrame copy with UTC-aware datetime columns.
"""
normalized = frame.copy()
for column in columns:
if column not in normalized.columns:
continue
if column in _TIME_COLUMN_NAMES:
normalized[column] = _coerce_mt5_time_column(normalized[column], column)
else:
normalized[column] = pd.to_datetime(
normalized[column], utc=True, errors="coerce"
)
return normalized
+226 -35
View File
@@ -21,12 +21,15 @@ from .history import (
create_cash_events_view,
create_history_indexes,
create_positions_reconstructed_view,
drop_forming_rate_bar,
resolve_granularity_name,
resolve_history_datasets,
resolve_history_tick_flags,
resolve_history_timeframes,
write_collected_datasets,
write_incremental_datasets,
)
from .retry import retry_with_backoff
from .utils import (
Dataset,
IfExists,
@@ -42,6 +45,65 @@ T = TypeVar("T")
logger = logging.getLogger(__name__)
_RECOVERABLE_HISTORY_UPDATE_ERRORS: tuple[type[BaseException], ...] = (
Mt5TradingError,
Mt5RuntimeError,
sqlite3.Error,
ValueError,
OSError,
)
_MT5_CLIENT_CAPABILITY_METHODS: frozenset[str] = frozenset({
"copy_rates_range_as_df",
"copy_ticks_range_as_df",
"history_deals_get_as_df",
"history_orders_get_as_df",
})
_MT5_HISTORY_MODULE = Path(__file__).with_name("history.py").resolve()
_MT5_HISTORY_CLIENT_CALL_FUNCTIONS: frozenset[str] = frozenset({
"write_rates_dataset",
"write_ticks_dataset",
"write_history_dataset",
"_write_incremental_history_deals",
"_fetch_rates_frame",
"_fetch_ticks_frame",
"_fetch_history_frame",
})
_NON_CALLABLE_TYPE_ERROR = re.compile(r"^'[^']+' object is not callable$")
def _is_non_callable_history_client_type_error(exc: TypeError) -> bool:
"""Return whether a TypeError came from calling a history client API attribute."""
if not _NON_CALLABLE_TYPE_ERROR.match(str(exc)):
return False
tb = exc.__traceback__
if tb is None:
return False
while tb.tb_next is not None:
tb = tb.tb_next
frame = tb.tb_frame
return (
frame.f_code.co_name in _MT5_HISTORY_CLIENT_CALL_FUNCTIONS
and Path(frame.f_code.co_filename).resolve() == _MT5_HISTORY_MODULE
)
def _is_mt5_client_capability_error(exc: BaseException) -> bool:
"""Return whether an error indicates an incompatible MT5 client API surface."""
if isinstance(exc, AttributeError):
msg = str(exc)
if msg.startswith("MT5 client is missing required method:"):
return True
name = getattr(exc, "name", None)
return isinstance(name, str) and name in _MT5_CLIENT_CAPABILITY_METHODS
if isinstance(exc, TypeError):
msg = str(exc)
if msg.startswith("MT5 client attribute is not callable:"):
return True
return _is_non_callable_history_client_type_error(exc)
return False
__all__ = [
"AccountSpec",
"Mt5CliClient",
@@ -49,9 +111,12 @@ __all__ = [
"account_info",
"build_config",
"collect_history",
"collect_latest_closed_rates_by_granularity",
"collect_latest_closed_rates_for_accounts",
"collect_latest_rates",
"collect_latest_rates_for_accounts",
"collect_latest_rates_for_accounts_with_retries",
"connected_client",
"copy_rates_from",
"copy_rates_from_pos",
"copy_rates_range",
@@ -84,14 +149,10 @@ __all__ = [
def _coerce_timeframe(timeframe: int | str) -> int:
if isinstance(timeframe, int):
return timeframe
return parse_timeframe(timeframe)
def _coerce_tick_flags(flags: int | str) -> int:
if isinstance(flags, int):
return flags
return parse_tick_flags(flags)
@@ -129,6 +190,12 @@ def _require_positive(value: float, name: str) -> None:
raise ValueError(msg)
def _require_non_negative(value: int, name: str) -> None:
if value < 0:
msg = f"{name} must be non-negative."
raise ValueError(msg)
def _call_required_client_method(client: Mt5DataClient, name: str) -> object:
try:
method = getattr(client, name)
@@ -254,7 +321,7 @@ def build_config(
@contextmanager
def _connected_client(config: Mt5Config) -> Iterator[Mt5DataClient]:
def connected_client(config: Mt5Config) -> Iterator[Mt5DataClient]:
"""Initialize MT5, yield a connected client, and always shut down.
Args:
@@ -284,7 +351,7 @@ def _run_with_client(
Returns:
Value returned by ``fetch_fn``.
"""
with _connected_client(config) as client:
with connected_client(config) as client:
return fetch_fn(client)
@@ -304,7 +371,7 @@ def mt5_session(config: Mt5Config | None = None) -> Iterator[Mt5CliClient]:
Connected ``Mt5CliClient`` bound to the session.
"""
mt5_config = config or build_config()
with _connected_client(mt5_config) as client:
with connected_client(mt5_config) as client:
yield Mt5CliClient.from_connected_client(client)
@@ -319,6 +386,7 @@ class Mt5CliClient:
password: str | None = None,
server: str | None = None,
timeout: int | None = None,
retry_count: int = 3,
config: Mt5Config | None = None,
client: Mt5DataClient | None = None,
) -> None:
@@ -330,6 +398,8 @@ class Mt5CliClient:
password: Trading account password.
server: Trading server name.
timeout: Connection timeout in milliseconds.
retry_count: Number of MT5 initialization retries for sessions
opened by this client.
config: Optional pre-built ``Mt5Config`` (overrides other args).
client: Optional already-connected ``Mt5DataClient``. Injected
clients are reused as-is and are not initialized or shut down.
@@ -341,6 +411,7 @@ class Mt5CliClient:
server=server,
timeout=timeout,
)
self._retry_count = retry_count
self._client = client
self._owns_client = client is None
@@ -369,7 +440,7 @@ class Mt5CliClient:
"""
if self._client is not None:
return self
client = Mt5DataClient(config=self._config)
client = Mt5DataClient(config=self._config, retry_count=self._retry_count)
try:
client.initialize_and_login_mt5()
except Exception:
@@ -951,7 +1022,7 @@ def update_history_with_config( # noqa: PLR0913
if request is None:
return
mt5_config = config or build_config()
with _connected_client(mt5_config) as client:
with connected_client(mt5_config) as client:
update_history(
client=client,
output=output,
@@ -1004,10 +1075,14 @@ class ThrottledHistoryUpdater:
include_account_events: Include account-level cash events.
interval_seconds: Minimum seconds between successful updates. Values
``<= 0`` update on every call.
suppress_errors: When True, ``Mt5TradingError``, ``Mt5RuntimeError``,
and ``sqlite3.Error`` raised during an update are swallowed and
:meth:`update` returns False without advancing the throttle. When
False (default), such errors propagate so callers control logging.
suppress_errors: When True, recoverable errors (``Mt5TradingError``,
``Mt5RuntimeError``, ``sqlite3.Error``, ``ValueError``,
``OSError``, and MT5 client capability ``AttributeError`` /
``TypeError`` for history API methods) raised during an update
are swallowed and :meth:`update` returns False without advancing
the throttle. Other ``AttributeError`` / ``TypeError`` values
always propagate. When False (default), recoverable errors
propagate so callers control logging.
"""
self.output = output
self.datasets = datasets
@@ -1047,16 +1122,27 @@ class ThrottledHistoryUpdater:
Returns:
True if an update ran successfully, False if it was throttled or
(when ``suppress_errors`` is True) failed with a recoverable error.
When ``suppress_errors`` is False, recoverable update failures
propagate to the caller.
Raises:
Mt5TradingError: If the update fails and ``suppress_errors`` is False.
Mt5RuntimeError: If the update fails and ``suppress_errors`` is False.
sqlite3.Error: If the SQLite write fails and ``suppress_errors`` is
False.
AttributeError: MT5 client capability mismatch when
``suppress_errors`` is False, or any other attribute error.
TypeError: MT5 client capability mismatch when ``suppress_errors``
is False, or any other type error.
"""
if not self.should_update():
return False
try:
_resolve_update_history_request(
output=self.output,
symbols=symbols,
datasets=self.datasets,
timeframes=self.timeframes,
flags=self.flags,
lookback_hours=self.lookback_hours,
date_to=None,
)
update_history(
client=client,
output=self.output,
@@ -1068,11 +1154,16 @@ class ThrottledHistoryUpdater:
with_views=self.with_views,
include_account_events=self.include_account_events,
)
except (Mt5TradingError, Mt5RuntimeError, sqlite3.Error):
except _RECOVERABLE_HISTORY_UPDATE_ERRORS:
if self.suppress_errors:
logger.warning("Suppressed history update error", exc_info=True)
return False
raise
except (AttributeError, TypeError) as exc:
if self.suppress_errors and _is_mt5_client_capability_error(exc):
logger.warning("Suppressed history update error", exc_info=True)
return False
raise
self._last_update_monotonic = time.monotonic()
return True
@@ -1085,7 +1176,7 @@ def collect_history(
*,
datasets: set[Dataset] | None = None,
timeframe: int | str = 1,
flags: int | str = 1,
flags: int | str = "ALL",
if_exists: IfExists = IfExists.FAIL,
with_views: bool = False,
config: Mt5Config | None = None,
@@ -1110,7 +1201,7 @@ def collect_history(
tf = _coerce_timeframe(timeframe)
tick_flags = _coerce_tick_flags(flags)
mt5_config = config or build_config()
with _connected_client(mt5_config) as client, sqlite3.connect(output) as conn:
with connected_client(mt5_config) as client, sqlite3.connect(output) as conn:
conn.execute("PRAGMA journal_mode=WAL")
conn.execute("PRAGMA synchronous=NORMAL")
written_tables, written_columns = write_collected_datasets(
@@ -1506,7 +1597,6 @@ def collect_latest_rates_for_accounts_with_retries(
re-raises the last ``pdmt5.Mt5TradingError`` or ``pdmt5.Mt5RuntimeError``
once retries are exhausted.
"""
attempts = max(retry_count, 0) + 1
def _collect() -> dict[tuple[str, int], pd.DataFrame]:
return collect_latest_rates_for_accounts(
@@ -1517,20 +1607,121 @@ def collect_latest_rates_for_accounts_with_retries(
base_config=base_config,
)
for attempt in range(attempts - 1):
try:
return _collect()
except (Mt5TradingError, Mt5RuntimeError) as exc:
delay = backoff_base ** (attempt + 1)
logger.warning(
"Rate collection failed (attempt %d/%d): %s; retrying in %.1fs",
attempt + 1,
attempts,
exc,
delay,
)
time.sleep(delay)
return _collect()
return retry_with_backoff(
_collect,
retry_count=retry_count,
backoff_base=backoff_base,
operation="Rate collection",
)
def collect_latest_closed_rates_for_accounts(
accounts: Sequence[AccountSpec],
timeframes: Sequence[int | str],
count: int,
*,
start_pos: int = 0,
base_config: Mt5Config | None = None,
retry_count: int = 0,
backoff_base: float = 2.0,
) -> dict[tuple[str, int], pd.DataFrame]:
"""Collect latest closed rate bars across multiple MT5 account groups.
When ``start_pos`` is ``0`` (the default), MetaTrader 5 includes the
still-forming current bar as the last row. This helper fetches
``count + 1`` bars, drops that bar with :func:`drop_forming_rate_bar`, and
validates that each resulting frame is non-empty. When ``start_pos`` is
greater than zero the forming bar is not in range, so only ``count`` bars
are fetched and no row is dropped.
Wraps :func:`collect_latest_rates_for_accounts_with_retries` for transient
MT5 error handling.
Args:
accounts: Account groups to read. Each must define at least one symbol.
timeframes: MT5 timeframes as integers or names (for example ``M1``).
count: Number of closed bars to return per symbol/timeframe.
start_pos: Initial bar position offset passed to the underlying collector.
base_config: Optional base configuration whose fields fill any value not
set on an individual account.
retry_count: Maximum number of retries after the first attempt. ``0``
disables retries.
backoff_base: Base for exponential backoff between retry attempts.
Returns:
Mapping keyed by ``(symbol, timeframe_int)``.
Raises:
ValueError: If inputs are invalid, or any series is empty (after
dropping the still-forming bar when ``start_pos`` is ``0``).
"""
_require_positive(count, "count")
_require_non_negative(start_pos, "start_pos")
fetch_count = count + 1 if start_pos == 0 else count
loaded = collect_latest_rates_for_accounts_with_retries(
accounts,
timeframes,
fetch_count,
start_pos=start_pos,
base_config=base_config,
retry_count=retry_count,
backoff_base=backoff_base,
)
result: dict[tuple[str, int], pd.DataFrame] = {}
for key, df_rate in loaded.items():
closed = drop_forming_rate_bar(df_rate) if start_pos == 0 else df_rate
if closed.empty:
symbol, timeframe = key
msg = f"Rate data is empty for {symbol!r} at timeframe {timeframe}."
raise ValueError(msg)
result[key] = closed
return result
def collect_latest_closed_rates_by_granularity(
accounts: Sequence[AccountSpec],
granularities: Sequence[int | str],
count: int,
*,
start_pos: int = 0,
base_config: Mt5Config | None = None,
retry_count: int = 0,
backoff_base: float = 2.0,
) -> dict[tuple[str, str], pd.DataFrame]:
"""Collect latest closed rate bars keyed by symbol and granularity name.
Thin wrapper around :func:`collect_latest_closed_rates_for_accounts` that
rekeys the result by granularity name (for example ``M1``) instead of the
integer timeframe.
Args:
accounts: Account groups to read. Each must define at least one symbol.
granularities: MT5 timeframes as integers or names (for example ``M1``).
count: Number of closed bars to return per symbol/timeframe.
start_pos: Initial bar position offset passed to the underlying collector.
base_config: Optional base configuration whose fields fill any value not
set on an individual account.
retry_count: Maximum number of retries after the first attempt. ``0``
disables retries.
backoff_base: Base for exponential backoff between retry attempts.
Returns:
Mapping keyed by ``(symbol, granularity_name)``. Propagates
``ValueError`` from :func:`collect_latest_closed_rates_for_accounts`.
"""
loaded = collect_latest_closed_rates_for_accounts(
accounts,
granularities,
count,
start_pos=start_pos,
base_config=base_config,
retry_count=retry_count,
backoff_base=backoff_base,
)
return {
(symbol, resolve_granularity_name(timeframe)): frame
for (symbol, timeframe), frame in loaded.items()
}
def copy_rates_range(
+49
View File
@@ -0,0 +1,49 @@
"""Generic storage helpers for MT5 market and account history."""
from __future__ import annotations
from .history import (
RateTarget,
build_rate_targets,
build_rate_view_name,
drop_forming_rate_bar,
load_rate_data,
load_rate_data_from_connection,
load_rate_series_by_granularity,
load_rate_series_from_sqlite,
resolve_rate_tables,
resolve_rate_view_name,
resolve_rate_view_names,
)
from .sdk import collect_history, update_history, update_history_with_config
from .utils import (
Dataset,
IfExists,
OutputFormat,
detect_format,
export_dataframe,
export_dataframe_to_sqlite,
)
__all__ = [
"Dataset",
"IfExists",
"OutputFormat",
"RateTarget",
"build_rate_targets",
"build_rate_view_name",
"collect_history",
"detect_format",
"drop_forming_rate_bar",
"export_dataframe",
"export_dataframe_to_sqlite",
"load_rate_data",
"load_rate_data_from_connection",
"load_rate_series_by_granularity",
"load_rate_series_from_sqlite",
"resolve_rate_tables",
"resolve_rate_view_name",
"resolve_rate_view_names",
"update_history",
"update_history_with_config",
]
+210
View File
@@ -0,0 +1,210 @@
"""Trading-capable MetaTrader 5 session helpers and operational utilities."""
from __future__ import annotations
from contextlib import contextmanager
from typing import TYPE_CHECKING, Literal
from pdmt5 import Mt5Config, Mt5TradingClient
from .sdk import build_config
if TYPE_CHECKING:
from collections.abc import Iterator
import pandas as pd
PositionSide = Literal["long", "short"]
OrderSide = Literal["long", "short"]
__all__ = [
"OrderSide",
"PositionSide",
"calculate_margin_and_volume",
"detect_position_side",
"determine_order_limits",
"mt5_trading_session",
]
def _require_unit_ratio(value: float, name: str) -> None:
if not 0.0 <= value <= 1.0:
msg = f"{name} must be between 0 and 1 inclusive."
raise ValueError(msg)
def _require_protective_ratio(value: float, name: str) -> None:
if not 0.0 <= value < 1.0:
msg = f"{name} must be at least 0 and less than 1."
raise ValueError(msg)
def _sum_position_volume(positions: pd.DataFrame, position_type: object) -> float:
matched = positions.loc[positions["type"] == position_type, "volume"]
if matched.empty:
return 0.0
return float(matched.to_numpy(dtype=float).sum())
def _normalize_order_side(side: str) -> OrderSide:
normalized = side.lower()
if normalized in {"long", "buy"}:
return "long"
if normalized in {"short", "sell"}:
return "short"
msg = (
f"Unsupported order side: {side!r}. Expected 'long', 'short', 'buy', or 'sell'."
)
raise ValueError(msg)
def detect_position_side(
client: Mt5TradingClient,
symbol: str,
) -> PositionSide | None:
"""Detect the net open position side for a symbol.
Args:
client: Connected ``Mt5TradingClient`` instance.
symbol: Symbol to inspect.
Returns:
``"long"`` when net buy volume exceeds sell volume, ``"short"`` when
net sell volume exceeds buy volume, or ``None`` when no positions exist
or buy/sell volumes are exactly balanced.
"""
positions = client.positions_get_as_df(symbol=symbol)
if positions.empty:
return None
buy_type = client.mt5.POSITION_TYPE_BUY
sell_type = client.mt5.POSITION_TYPE_SELL
buy_volume = _sum_position_volume(positions, buy_type)
sell_volume = _sum_position_volume(positions, sell_type)
net_volume = buy_volume - sell_volume
if net_volume > 0:
return "long"
if net_volume < 0:
return "short"
return None
def calculate_margin_and_volume(
client: Mt5TradingClient,
symbol: str,
unit_margin_ratio: float,
preserved_margin_ratio: float,
) -> dict[str, float]:
"""Calculate tradable margin and volumes from account free margin.
Applies ``preserved_margin_ratio`` to keep a reserve off ``margin_free``,
then allocates ``unit_margin_ratio`` of the remainder as the margin budget
for volume sizing on both buy and sell sides.
Args:
client: Connected ``Mt5TradingClient`` instance.
symbol: Symbol used for minimum-lot margin and volume calculations.
unit_margin_ratio: Fraction of post-reserve margin to allocate per unit.
preserved_margin_ratio: Fraction of ``margin_free`` to preserve.
Returns:
Dictionary with ``margin_free``, ``available_margin``, ``trade_margin``,
``buy_volume``, and ``sell_volume``. Negative ``margin_free`` values are
clamped to ``0.0`` before sizing.
"""
_require_unit_ratio(unit_margin_ratio, "unit_margin_ratio")
_require_unit_ratio(preserved_margin_ratio, "preserved_margin_ratio")
account = client.account_info_as_dict()
margin_free = max(0.0, float(account.get("margin_free") or 0.0))
available_margin = margin_free * (1.0 - preserved_margin_ratio)
trade_margin = available_margin * unit_margin_ratio
buy_volume = client.calculate_volume_by_margin(symbol, trade_margin, "BUY")
sell_volume = client.calculate_volume_by_margin(symbol, trade_margin, "SELL")
return {
"margin_free": margin_free,
"available_margin": available_margin,
"trade_margin": trade_margin,
"buy_volume": buy_volume,
"sell_volume": sell_volume,
}
def determine_order_limits(
client: Mt5TradingClient,
symbol: str,
side: OrderSide | str,
stop_loss_limit_ratio: float,
take_profit_limit_ratio: float,
) -> dict[str, float | None]:
"""Derive entry and protective order prices from current market quotes.
Args:
client: Connected ``Mt5TradingClient`` instance.
symbol: Symbol used for the quote lookup.
side: Position side as ``"long"``/``"short"`` (``"buy"``/``"sell"``
aliases are accepted).
stop_loss_limit_ratio: Relative distance from entry for stop loss in
``[0, 1)``. A value of ``0`` omits the stop loss.
take_profit_limit_ratio: Relative distance from entry for take profit in
``[0, 1)``. A value of ``0`` omits the take profit.
Returns:
Dictionary with ``entry``, ``stop_loss``, and ``take_profit`` keys.
Omitted protective levels are returned as ``None``.
"""
_require_protective_ratio(stop_loss_limit_ratio, "stop_loss_limit_ratio")
_require_protective_ratio(take_profit_limit_ratio, "take_profit_limit_ratio")
normalized_side = _normalize_order_side(side)
tick = client.symbol_info_tick_as_dict(symbol=symbol)
entry = float(tick["ask"] if normalized_side == "long" else tick["bid"])
stop_loss: float | None = None
if stop_loss_limit_ratio > 0:
if normalized_side == "long":
stop_loss = entry * (1.0 - stop_loss_limit_ratio)
else:
stop_loss = entry * (1.0 + stop_loss_limit_ratio)
take_profit: float | None = None
if take_profit_limit_ratio > 0:
if normalized_side == "long":
take_profit = entry * (1.0 + take_profit_limit_ratio)
else:
take_profit = entry * (1.0 - take_profit_limit_ratio)
return {
"entry": entry,
"stop_loss": stop_loss,
"take_profit": take_profit,
}
@contextmanager
def mt5_trading_session(
config: Mt5Config | None = None,
retry_count: int = 0,
) -> Iterator[Mt5TradingClient]:
"""Open a trading-capable MT5 session and always shut down safely.
Launches the MetaTrader 5 terminal using ``Mt5Config.path`` when set,
initializes and logs in via ``initialize_and_login_mt5()``, yields a
connected :class:`~pdmt5.Mt5TradingClient`, and calls ``shutdown()`` on
exit even when an error is raised inside the context.
Args:
config: MT5 connection configuration. Defaults to an empty config that
attaches to a running terminal.
retry_count: Number of initialization retries passed to
``Mt5TradingClient``.
Yields:
Connected ``Mt5TradingClient`` bound to the session.
"""
mt5_config = config or build_config()
client = Mt5TradingClient(config=mt5_config, retry_count=retry_count)
try:
client.initialize_and_login_mt5()
yield client
finally:
client.shutdown()
+31 -50
View File
@@ -10,6 +10,9 @@ from pathlib import Path
from typing import TYPE_CHECKING, Any, TypeGuard
import click
from pdmt5 import COPY_TICKS_MAP, TIMEFRAME_MAP
from pdmt5 import parse_copy_ticks as _parse_copy_ticks
from pdmt5 import parse_timeframe as _parse_timeframe
if TYPE_CHECKING:
from collections.abc import Sequence
@@ -20,35 +23,15 @@ if TYPE_CHECKING:
# Constants
# ---------------------------------------------------------------------------
TIMEFRAME_MAP: dict[str, int] = {
"M1": 1,
"M2": 2,
"M3": 3,
"M4": 4,
"M5": 5,
"M6": 6,
"M10": 10,
"M12": 12,
"M15": 15,
"M20": 20,
"M30": 30,
"H1": 16385,
"H2": 16386,
"H3": 16387,
"H4": 16388,
"H6": 16390,
"H8": 16392,
"H12": 16396,
"D1": 16408,
"W1": 32769,
"MN1": 49153,
}
# Backward-compatible snapshot; prefer ``COPY_TICKS_MAP`` from pdmt5 directly.
TICK_FLAG_MAP: dict[str, int] = dict(COPY_TICKS_MAP)
TICK_FLAG_MAP: dict[str, int] = {
"ALL": 1,
"INFO": 2,
"TRADE": 4,
}
TIMEFRAME_NAMES: tuple[str, ...] = tuple(
name for name in TIMEFRAME_MAP if not name.startswith("TIMEFRAME_")
)
_TICK_FLAG_NAMES: tuple[str, ...] = tuple(
name for name in COPY_TICKS_MAP if not name.startswith("COPY_TICKS_")
)
_FORMAT_EXTENSIONS: dict[str, str] = {
".csv": "csv",
@@ -160,10 +143,8 @@ class _TimeframeType(click.ParamType):
Returns:
Integer timeframe value.
"""
if isinstance(value, int):
return value
try:
return parse_timeframe(str(value))
return parse_timeframe(value)
except ValueError as exc:
self.fail(str(exc), param, ctx)
@@ -189,10 +170,8 @@ class _TickFlagsType(click.ParamType):
Returns:
Integer tick flag value.
"""
if isinstance(value, int):
return value
try:
return parse_tick_flags(str(value))
return parse_tick_flags(value)
except ValueError as exc:
self.fail(str(exc), param, ctx)
@@ -370,7 +349,7 @@ def parse_datetime(value: str) -> datetime:
return dt
def parse_timeframe(value: str) -> int:
def parse_timeframe(value: object) -> int:
"""Parse a timeframe string or integer value.
Args:
@@ -382,37 +361,39 @@ def parse_timeframe(value: str) -> int:
Raises:
ValueError: If the timeframe is invalid.
"""
upper = value.upper()
if upper in TIMEFRAME_MAP:
return TIMEFRAME_MAP[upper]
try:
return int(value)
return _parse_timeframe(value)
except ValueError:
valid = ", ".join(TIMEFRAME_MAP)
msg = f"Invalid timeframe: '{value}'. Use one of: {valid}, or an integer."
display = value if isinstance(value, str) else repr(value)
valid = ", ".join(TIMEFRAME_NAMES)
msg = (
f"Invalid timeframe: '{display}'. "
f"Use one of: {valid}, or a supported integer."
)
raise ValueError(msg) from None
def parse_tick_flags(value: str) -> int:
def parse_tick_flags(value: object) -> int:
"""Parse tick flags string or integer value.
Args:
value: Tick flag name (ALL, INFO, TRADE) or integer value.
value: Tick flag name (ALL, INFO, TRADE, COPY_TICKS_*) or integer value.
Returns:
Integer tick flag value.
Integer tick flag value compatible with MetaTrader 5 ``COPY_TICKS_*``.
Raises:
ValueError: If the flag is invalid.
"""
upper = value.upper()
if upper in TICK_FLAG_MAP:
return TICK_FLAG_MAP[upper]
try:
return int(value)
return _parse_copy_ticks(value)
except ValueError:
valid = ", ".join(TICK_FLAG_MAP)
msg = f"Invalid tick flags: '{value}'. Use one of: {valid}, or an integer."
display = value if isinstance(value, str) else repr(value)
valid = ", ".join(_TICK_FLAG_NAMES)
msg = (
f"Invalid tick flags: '{display}'. "
f"Use one of: {valid}, or a supported integer."
)
raise ValueError(msg) from None
+3 -3
View File
@@ -1,7 +1,7 @@
[project]
name = "mt5cli"
version = "0.5.3"
description = "Command-line tool for MetaTrader 5"
version = "0.7.2"
description = "Generic MT5 data and execution infrastructure for Python applications"
authors = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
maintainers = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
license = "MIT"
@@ -9,7 +9,7 @@ license-files = ["LICENSE"]
readme = "README.md"
requires-python = ">= 3.11, < 3.14"
dependencies = [
"pdmt5 >= 0.2.3",
"pdmt5>=0.3.0",
"click >= 8.1.0",
"pyarrow >= 19.0.0",
"typer >= 0.15.0",
+52
View File
@@ -0,0 +1,52 @@
"""Shared pytest fixtures for mt5cli tests."""
from __future__ import annotations
from unittest.mock import MagicMock
import pandas as pd
import pytest
from pytest_mock import MockerFixture # noqa: TC002
_DATAFRAME_METHODS = (
"copy_rates_from_as_df",
"copy_rates_from_pos_as_df",
"copy_rates_range_as_df",
"copy_ticks_from_as_df",
"copy_ticks_range_as_df",
"account_info_as_df",
"terminal_info_as_df",
"symbols_get_as_df",
"symbol_info_as_df",
"orders_get_as_df",
"positions_get_as_df",
"history_orders_get_as_df",
"history_deals_get_as_df",
"version_as_df",
"last_error_as_df",
"symbol_info_tick_as_df",
"market_book_get_as_df",
"order_check_as_df",
"order_send_as_df",
)
def build_mock_mt5_data_client() -> MagicMock:
"""Return a MagicMock Mt5DataClient with common DataFrame stubs."""
client = MagicMock()
sample_df = pd.DataFrame({"col": [1]})
for method_name in _DATAFRAME_METHODS:
getattr(client, method_name).return_value = sample_df
client.version.return_value = (5, 0, 1)
client.terminal_info.return_value = {"connected": True, "paths": ["terminal.exe"]}
client.account_info.return_value = {"login": 123, "limits": {"modes": ["demo"]}}
client.symbols_total.return_value = 42
return client
@pytest.fixture
def mock_client(mocker: MockerFixture) -> MagicMock:
"""Create and patch a mock Mt5DataClient for CLI and SDK tests."""
client = build_mock_mt5_data_client()
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=client)
return client
+5 -37
View File
@@ -69,38 +69,6 @@ class TestExecuteExport:
# ---------------------------------------------------------------------------
@pytest.fixture
def mock_client(mocker: MockerFixture) -> MagicMock:
"""Create and patch a mock Mt5DataClient for CLI tests."""
client = MagicMock()
sample_df = pd.DataFrame({"col": [1]})
client.copy_rates_from_as_df.return_value = sample_df
client.copy_rates_from_pos_as_df.return_value = sample_df
client.copy_rates_range_as_df.return_value = sample_df
client.copy_ticks_from_as_df.return_value = sample_df
client.copy_ticks_range_as_df.return_value = sample_df
client.account_info_as_df.return_value = sample_df
client.terminal_info_as_df.return_value = sample_df
client.symbols_get_as_df.return_value = sample_df
client.symbol_info_as_df.return_value = sample_df
client.orders_get_as_df.return_value = sample_df
client.positions_get_as_df.return_value = sample_df
client.history_orders_get_as_df.return_value = sample_df
client.history_deals_get_as_df.return_value = sample_df
client.version_as_df.return_value = sample_df
client.last_error_as_df.return_value = sample_df
client.symbol_info_tick_as_df.return_value = sample_df
client.market_book_get_as_df.return_value = sample_df
client.order_check_as_df.return_value = sample_df
client.order_send_as_df.return_value = sample_df
client.version.return_value = (5, 0, 1)
client.terminal_info.return_value = {"connected": True, "paths": ["terminal.exe"]}
client.account_info.return_value = {"login": 123, "limits": {"modes": ["demo"]}}
client.symbols_total.return_value = 42
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=client)
return client
class TestCommands:
"""Tests for all CLI subcommands via CliRunner."""
@@ -317,7 +285,7 @@ class TestCommands:
symbol="EURUSD",
date_from=datetime(2024, 1, 1, tzinfo=UTC),
count=100,
flags=1,
flags=-1,
)
def test_ticks_range(
@@ -348,7 +316,7 @@ class TestCommands:
symbol="EURUSD",
date_from=datetime(2024, 1, 1, tzinfo=UTC),
date_to=datetime(2024, 2, 1, tzinfo=UTC),
flags=2,
flags=1,
)
def test_ticks_recent(
@@ -381,7 +349,7 @@ class TestCommands:
symbol="EURUSD",
date_from=datetime(2024, 1, 2, tzinfo=UTC) - timedelta(seconds=120),
count=500,
flags=1,
flags=-1,
)
mock_client.copy_ticks_range_as_df.assert_not_called()
@@ -1000,7 +968,7 @@ class TestCollectHistory:
symbol="EURUSD",
date_from=datetime(2024, 1, 1, tzinfo=UTC),
date_to=datetime(2024, 2, 1, tzinfo=UTC),
flags=1,
flags=-1,
)
with sqlite3.connect(output) as conn:
tables = {
@@ -1213,7 +1181,7 @@ class TestCollectHistory:
symbol="EURUSD",
date_from=datetime(2024, 1, 1, tzinfo=UTC),
date_to=datetime(2024, 2, 1, tzinfo=UTC),
flags=1,
flags=-1,
)
def test_collect_history_with_views(
+512
View File
@@ -0,0 +1,512 @@
"""Contract tests for the mt5cli public API and dataset schemas."""
from __future__ import annotations
from datetime import UTC, datetime
from typing import TYPE_CHECKING
import pandas as pd
import pytest
from pdmt5 import Mt5RuntimeError, Mt5TradingError
from pytest_mock import MockerFixture # noqa: TC002
from mt5cli import (
DEDUP_KEYS,
REQUIRED_COLUMNS,
TIME_COLUMNS,
DataKind,
Dataset,
MT5Client,
Mt5CliError,
Mt5ConnectionError,
Mt5OperationError,
Mt5SchemaError,
build_config,
call_with_normalized_errors,
detect_format,
ensure_utc,
export_dataframe,
export_dataframe_to_sqlite,
granularity_name,
is_recoverable_mt5_error,
mt5_session,
normalize_dataframe,
normalize_mt5_exception,
normalize_symbol,
normalize_symbols,
parse_date_range,
recent_window,
schema_columns,
validate_schema,
)
from mt5cli.retry import retry_with_backoff
from mt5cli.schemas import ensure_utc_columns, normalize_time_columns
if TYPE_CHECKING:
from pathlib import Path
def _sample_frame(kind: DataKind) -> pd.DataFrame:
if kind is DataKind.rates:
return pd.DataFrame({
"time": [datetime(2024, 1, 1, tzinfo=UTC)],
"open": [1.1],
"high": [1.2],
"low": [1.0],
"close": [1.15],
"tick_volume": [10],
"spread": [1],
"real_volume": [0],
})
if kind is DataKind.ticks:
return pd.DataFrame({
"time": [datetime(2024, 1, 1, tzinfo=UTC)],
"bid": [1.1],
"ask": [1.11],
"last": [1.105],
"volume": [1],
"time_msc": [datetime(2024, 1, 1, tzinfo=UTC)],
"flags": [2],
"volume_real": [0.0],
})
if kind is DataKind.orders:
return pd.DataFrame({
"ticket": [1],
"time_setup": [datetime(2024, 1, 1, tzinfo=UTC)],
"type": [0],
"state": [1],
"symbol": ["EURUSD"],
"volume_current": [0.1],
"price_open": [1.1],
})
if kind is DataKind.positions:
return pd.DataFrame({
"ticket": [1],
"time": [datetime(2024, 1, 1, tzinfo=UTC)],
"type": [0],
"symbol": ["EURUSD"],
"volume": [0.1],
"price_open": [1.1],
"price_current": [1.11],
"profit": [1.0],
})
if kind is DataKind.history_orders:
return pd.DataFrame({
"ticket": [1],
"time_setup": [datetime(2024, 1, 1, tzinfo=UTC)],
"type": [0],
"state": [3],
"symbol": ["EURUSD"],
"volume_initial": [0.1],
"price_open": [1.1],
})
return pd.DataFrame({
"ticket": [1],
"order": [2],
"time": [datetime(2024, 1, 1, tzinfo=UTC)],
"type": [0],
"entry": [0],
"symbol": ["EURUSD"],
"volume": [0.1],
"price": [1.1],
"profit": [0.0],
})
@pytest.mark.parametrize("kind", list(DataKind))
def test_required_columns_contract(kind: DataKind) -> None:
"""Each dataset kind exposes a non-empty required column contract."""
assert REQUIRED_COLUMNS[kind]
validate_schema(_sample_frame(kind), kind)
@pytest.mark.parametrize("kind", list(DataKind))
def test_normalize_dataframe_injects_storage_metadata(kind: DataKind) -> None:
"""Normalization accepts MT5 frames and optional storage metadata."""
frame = _sample_frame(kind)
normalized = normalize_dataframe(
frame,
kind,
symbol="eurusd",
timeframe="M1" if kind is DataKind.rates else None,
)
if kind is DataKind.rates:
assert normalized.loc[0, "symbol"] == "eurusd"
assert normalized.loc[0, "timeframe"] == 1
validate_schema(normalized, kind)
def test_validate_schema_raises_for_missing_columns() -> None:
"""Schema validation fails fast on missing required columns."""
with pytest.raises(Mt5SchemaError, match="missing required columns"):
validate_schema(pd.DataFrame({"time": [1]}), DataKind.rates)
def test_history_dedup_keys_match_schema_contract() -> None:
"""SQLite history dedup keys stay aligned with schema contracts."""
assert DEDUP_KEYS[DataKind.rates][0] == ("symbol", "timeframe", "time")
assert DEDUP_KEYS[DataKind.ticks][0] == ("symbol", "time_msc")
assert Dataset.rates.table_name == "rates"
@pytest.mark.parametrize(
("raw", "expected"),
[
(" eurusd ", "eurusd"),
("GbpJpy", "GbpJpy"),
("XAUUSDm", "XAUUSDm"),
("US500.cash", "US500.cash"),
("EURUSD.r", "EURUSD.r"),
],
)
def test_normalize_symbol(raw: str, expected: str) -> None:
"""Symbol normalization trims whitespace and preserves broker casing."""
assert normalize_symbol(raw) == expected
def test_normalize_symbols_deduplicates() -> None:
"""Symbol lists are normalized and de-duplicated in order."""
assert normalize_symbols(["XAUUSDm", " XAUUSDm ", "EURUSD.r", "eurusd"]) == [
"XAUUSDm",
"EURUSD.r",
"eurusd",
]
def test_parse_date_range_rejects_inverted_bounds() -> None:
"""Date ranges must not be inverted."""
with pytest.raises(ValueError, match="must not be after"):
parse_date_range("2024-02-01", "2024-01-01")
def test_recent_window_builds_trailing_bounds() -> None:
"""Recent windows end at the provided timestamp."""
end = datetime(2024, 1, 2, tzinfo=UTC)
start, resolved_end = recent_window(hours=24, date_to=end)
assert resolved_end == end
assert start < end
def test_granularity_name_maps_timeframe_alias() -> None:
"""Granularity labels resolve MT5 timeframe aliases."""
assert granularity_name("M1") == "M1"
@pytest.mark.parametrize(
"exc",
[Mt5RuntimeError("init failed"), Mt5TradingError("trade failed")],
)
def test_is_recoverable_mt5_error(exc: Exception) -> None:
"""Recoverable MT5 errors are classified consistently."""
assert is_recoverable_mt5_error(exc)
def test_normalize_mt5_exception_maps_types() -> None:
"""MT5 exceptions map to stable mt5cli types."""
assert isinstance(
normalize_mt5_exception(Mt5RuntimeError("x")),
Mt5ConnectionError,
)
assert isinstance(
normalize_mt5_exception(Mt5TradingError("x")),
Mt5OperationError,
)
def test_call_with_normalized_errors_reraises_mapped_type() -> None:
"""Normalized error helper re-raises mapped mt5cli exceptions."""
def _raise() -> None:
message = "boom"
raise Mt5RuntimeError(message)
with pytest.raises(Mt5ConnectionError):
call_with_normalized_errors(_raise)
def test_retry_with_backoff_retries_recoverable_errors(
mocker: MockerFixture,
) -> None:
"""Retry helper retries recoverable MT5 failures."""
calls = {"count": 0}
def _flaky() -> str:
calls["count"] += 1
if calls["count"] == 1:
message = "transient"
raise Mt5RuntimeError(message)
return "ok"
mocker.patch("mt5cli.retry.time.sleep")
assert retry_with_backoff(_flaky, retry_count=1) == "ok"
assert calls["count"] == 2
def test_public_api_exports_mt5_client() -> None:
"""MT5Client is the primary importable client abstraction."""
client = MT5Client(config=build_config())
assert isinstance(client, MT5Client)
assert isinstance(client, MT5Client.__mro__[1])
def test_mt5_client_order_primitives_use_connected_client(
mock_client: object,
) -> None:
"""Order check/send route through the same client fetch path as exports."""
request = {"action": 1}
client = MT5Client()
client.order_check(request)
client.order_send(request)
assert mock_client.order_check_as_df.call_count == 1 # type: ignore[attr-defined]
assert mock_client.order_send_as_df.call_count == 1 # type: ignore[attr-defined]
def test_storage_export_round_trip_csv(tmp_path: Path) -> None:
"""Storage helpers export normalized rate frames to CSV."""
frame = normalize_dataframe(
_sample_frame(DataKind.rates),
DataKind.rates,
symbol="EURUSD",
timeframe="M1",
)
output = tmp_path / "rates.csv"
export_dataframe(frame, output, detect_format(output))
loaded = pd.read_csv(output)
assert len(loaded) == 1
assert "close" in loaded.columns
def test_normalize_symbol_rejects_empty_value() -> None:
"""Empty symbols are rejected after trimming."""
with pytest.raises(ValueError, match="must not be empty"):
normalize_symbol(" ")
def test_ensure_utc_handles_naive_and_aware_datetimes() -> None:
"""UTC coercion accepts naive and timezone-aware datetimes."""
naive = datetime(2024, 1, 1, tzinfo=UTC).replace(tzinfo=None)
aware = datetime(2024, 1, 1, tzinfo=UTC)
assert ensure_utc(naive).tzinfo == UTC
assert ensure_utc(aware).tzinfo == UTC
assert ensure_utc("2024-01-01T00:00:00+00:00").tzinfo == UTC
def test_recent_window_validation_errors() -> None:
"""Recent window helpers validate mutually exclusive length arguments."""
with pytest.raises(ValueError, match="exactly one"):
recent_window()
with pytest.raises(ValueError, match="exactly one"):
recent_window(hours=1, seconds=1)
with pytest.raises(ValueError, match="positive"):
recent_window(hours=0)
def test_recent_window_supports_seconds_argument() -> None:
"""Recent windows can be built from a seconds-based length."""
end = datetime(2024, 1, 2, tzinfo=UTC)
start, resolved_end = recent_window(seconds=3600, date_to=end)
assert resolved_end == end
assert start < end
def test_parse_date_range_returns_ordered_bounds() -> None:
"""Valid date ranges return UTC-aware bounds."""
start, end = parse_date_range("2024-01-01", "2024-02-01")
assert start < end
def test_granularity_name_falls_back_for_unknown_timeframe(
mocker: MockerFixture,
) -> None:
"""Unknown timeframe integers stringify as granularity labels."""
mocker.patch(
"mt5cli.converters._get_timeframe_name",
side_effect=ValueError("unknown"),
)
assert granularity_name(1) == "1"
def test_normalize_mt5_exception_passthrough_and_generic() -> None:
"""Normalization preserves mt5cli errors and wraps unknown exceptions."""
original = Mt5CliError("known")
assert normalize_mt5_exception(original) is original
assert isinstance(normalize_mt5_exception(ValueError("x")), Mt5CliError)
def test_schema_columns_and_extra_required_validation() -> None:
"""Schema helpers expose contracts and honor extra required columns."""
assert schema_columns(DataKind.rates) == REQUIRED_COLUMNS[DataKind.rates]
validate_schema(pd.DataFrame(), DataKind.rates)
frame = _sample_frame(DataKind.rates)
with pytest.raises(Mt5SchemaError, match="storage_symbol"):
validate_schema(frame, DataKind.rates, extra_required=["storage_symbol"])
def test_normalize_dataframe_empty_and_tick_sort_paths() -> None:
"""Normalization handles empty frames and tick time_msc sorting."""
empty = pd.DataFrame()
assert normalize_dataframe(empty, DataKind.rates).empty
ticks = _sample_frame(DataKind.ticks)
ticks = pd.concat([ticks, ticks], ignore_index=True)
sorted_ticks = normalize_dataframe(ticks, DataKind.ticks, sort=True)
assert len(sorted_ticks) == 2
unsorted_ticks = normalize_dataframe(ticks, DataKind.ticks, sort=False)
assert len(unsorted_ticks) == 2
def test_normalize_dataframe_rate_timeframe_without_symbol() -> None:
"""Rate normalization can inject timeframe without symbol metadata."""
frame = _sample_frame(DataKind.rates)
normalized = normalize_dataframe(frame, DataKind.rates, timeframe="M1")
assert "timeframe" in normalized.columns
def test_normalize_dataframe_keeps_existing_symbol_and_timeframe() -> None:
"""Normalization does not duplicate existing storage metadata columns."""
frame = normalize_dataframe(
_sample_frame(DataKind.rates),
DataKind.rates,
symbol="EURUSD",
timeframe="M1",
)
normalized = normalize_dataframe(
frame,
DataKind.rates,
symbol="GBPUSD",
timeframe="H1",
)
assert normalized.loc[0, "symbol"] == "EURUSD"
assert normalized.loc[0, "timeframe"] == 1
def test_normalize_time_columns_skips_absent_time_fields() -> None:
"""Time normalization ignores absent optional time columns."""
frame = pd.DataFrame({"open": [1.0]})
result = normalize_time_columns(frame, DataKind.rates)
assert list(result.columns) == ["open"]
def test_normalize_time_columns_converts_unix_seconds() -> None:
"""Numeric MT5 ``time`` values are interpreted as Unix seconds."""
frame = pd.DataFrame({"time": [1704067200]})
result = normalize_time_columns(frame, DataKind.rates)
assert result.loc[0, "time"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
def test_normalize_time_columns_converts_unix_milliseconds() -> None:
"""Numeric MT5 ``time_msc`` values are interpreted as Unix milliseconds."""
frame = pd.DataFrame({"time_msc": [1704067200000]})
result = normalize_time_columns(frame, DataKind.ticks)
assert result.loc[0, "time_msc"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
def test_normalize_time_columns_preserves_utc_datetimes() -> None:
"""Already-converted datetime values remain UTC-normalized."""
aware = datetime(2024, 1, 1, tzinfo=UTC)
frame = pd.DataFrame({"time": [aware]})
result = normalize_time_columns(frame, DataKind.rates)
assert result.loc[0, "time"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
def test_normalize_time_columns_handles_optional_order_times() -> None:
"""Optional order/history time columns are normalized when present."""
frame = pd.DataFrame({
"time_setup": [1704067200],
"time_setup_msc": [1704067200000],
"time_done": [1704153600],
"time_done_msc": [1704153600000],
})
result = normalize_time_columns(frame, DataKind.orders)
assert result.loc[0, "time_setup"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
assert result.loc[0, "time_setup_msc"] == pd.Timestamp(
"2024-01-01T00:00:00+00:00",
)
assert result.loc[0, "time_done"] == pd.Timestamp("2024-01-02T00:00:00+00:00")
assert result.loc[0, "time_done_msc"] == pd.Timestamp(
"2024-01-02T00:00:00+00:00",
)
def test_time_columns_include_optional_order_fields() -> None:
"""Schema contracts document optional MT5 time columns per dataset kind."""
assert "time_done" in TIME_COLUMNS[DataKind.orders]
assert "time_setup_msc" in TIME_COLUMNS[DataKind.history_orders]
def test_normalize_dataframe_sorts_ticks_by_time_msc(
mocker: MockerFixture,
) -> None:
"""Tick frames without ``time`` can still sort on ``time_msc``."""
mocker.patch("mt5cli.schemas.validate_schema")
ticks = pd.concat([_sample_frame(DataKind.ticks)] * 2, ignore_index=True).drop(
columns=["time"],
)
ticks.loc[0, "time_msc"] = datetime(2024, 1, 1, tzinfo=UTC)
ticks.loc[1, "time_msc"] = datetime(2024, 1, 2, tzinfo=UTC)
ticks = pd.concat([ticks.iloc[[1]], ticks.iloc[[0]]], ignore_index=True)
normalized = normalize_dataframe(ticks, DataKind.ticks, sort=True)
assert normalized.iloc[0]["time_msc"] <= normalized.iloc[1]["time_msc"]
def test_ensure_utc_columns_skips_missing_columns() -> None:
"""UTC column coercion ignores absent columns."""
frame = _sample_frame(DataKind.rates)
result = ensure_utc_columns(frame, ["time", "missing"])
assert "time" in result.columns
def test_normalize_time_columns_coerces_string_timestamps() -> None:
"""String timestamps are parsed with timezone-aware datetime coercion."""
frame = pd.DataFrame({"time": ["2024-01-01T00:00:00+00:00"]})
result = normalize_time_columns(frame, DataKind.rates)
assert result.loc[0, "time"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
def test_ensure_utc_columns_coerces_non_mt5_columns() -> None:
"""Non-MT5 columns still coerce to UTC datetimes."""
frame = pd.DataFrame({"created_at": ["2024-01-01T00:00:00+00:00"]})
result = ensure_utc_columns(frame, ["created_at"])
assert result.loc[0, "created_at"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
def test_mt5_session_yields_connected_client(mocker: MockerFixture) -> None:
"""Public mt5_session yields an MT5Client bound to a connected session."""
connected = mocker.MagicMock()
context = mocker.MagicMock()
context.__enter__.return_value = connected
context.__exit__.return_value = False
mocker.patch("mt5cli.client.connected_client", return_value=context)
with mt5_session(build_config()) as client:
assert isinstance(client, MT5Client)
def test_retry_with_backoff_reraises_non_recoverable_errors() -> None:
"""Non-MT5 errors are not retried."""
def _raise() -> None:
message = "fatal"
raise ValueError(message)
with pytest.raises(ValueError, match="fatal"):
retry_with_backoff(_raise, retry_count=2)
def test_storage_export_round_trip_sqlite(tmp_path: Path) -> None:
"""Storage helpers append deduplicated frames to SQLite."""
frame = normalize_dataframe(
_sample_frame(DataKind.rates),
DataKind.rates,
symbol="EURUSD",
timeframe="M1",
)
output = tmp_path / "rates.db"
export_dataframe_to_sqlite(
frame,
output,
"rates",
deduplicate_on=DEDUP_KEYS[DataKind.rates][0],
)
with __import__("sqlite3").connect(output) as conn:
count = conn.execute("SELECT COUNT(*) FROM rates").fetchone()[0]
assert count == 1
+58 -1
View File
@@ -30,6 +30,7 @@ from mt5cli.history import (
create_rate_compatibility_views,
deduplicate_history_tables,
drop_duplicates_in_table,
drop_forming_rate_bar,
filter_incremental_history_deals_frame,
filter_trade_history_frame,
get_history_deals_account_event_start_datetime,
@@ -516,6 +517,9 @@ class TestResolveHistorySettings:
"""Test default timeframes include all fixed MT5 values."""
resolved = resolve_history_timeframes(None)
assert len(resolved) == len(DEFAULT_HISTORY_TIMEFRAMES)
assert not any(
name.startswith("TIMEFRAME_") for name in DEFAULT_HISTORY_TIMEFRAMES
)
assert 1 in resolved
assert TIMEFRAME_MAP["H1"] in resolved
@@ -525,7 +529,7 @@ class TestResolveHistorySettings:
def test_resolve_history_tick_flags(self) -> None:
"""Test tick flag resolution."""
assert resolve_history_tick_flags("ALL") == 1
assert resolve_history_tick_flags("ALL") == -1
assert resolve_history_tick_flags(2) == 2
def test_resolve_granularity_name_falls_back_to_integer(self) -> None:
@@ -533,6 +537,57 @@ class TestResolveHistorySettings:
assert resolve_granularity_name(999) == "999"
assert resolve_granularity_name(1) == "M1"
def test_resolve_granularity_name_strips_official_prefix(
self,
mocker: MockerFixture,
) -> None:
"""Test official pdmt5 timeframe names are normalized to short aliases."""
mocker.patch(
"mt5cli.history._get_timeframe_name",
return_value="TIMEFRAME_H1",
)
assert resolve_granularity_name(16385) == "H1"
class TestDropFormingRateBar:
"""Tests for drop_forming_rate_bar."""
def test_drops_still_forming_last_bar(self) -> None:
"""Test the still-forming last bar is removed."""
df_rate = pd.DataFrame(
{"time": [1, 2, 3], "close": [1.1, 1.2, 1.3]},
index=pd.Index(["a", "b", "c"], name="idx"),
)
result = drop_forming_rate_bar(df_rate)
pd.testing.assert_frame_equal(
result,
pd.DataFrame(
{"time": [1, 2], "close": [1.1, 1.2]},
index=pd.Index(["a", "b"], name="idx"),
),
)
assert df_rate.shape == (3, 2)
def test_returns_empty_frame_when_input_empty(self) -> None:
"""Test empty frames stay empty."""
df_rate = pd.DataFrame(columns=["time", "close"])
result = drop_forming_rate_bar(df_rate)
assert result.empty
assert list(result.columns) == ["time", "close"]
def test_returns_empty_frame_when_only_forming_bar_present(self) -> None:
"""Test a single-bar frame becomes empty after dropping the forming bar."""
df_rate = pd.DataFrame({"time": [1], "close": [1.1]})
result = drop_forming_rate_bar(df_rate)
assert result.empty
assert list(result.columns) == ["time", "close"]
class TestParseSqliteTimestamp:
"""Tests for parse_sqlite_timestamp."""
@@ -1713,6 +1768,8 @@ class TestIncrementalIntegration:
"""Test invalid tick flags raise ValueError."""
with pytest.raises(ValueError, match="Invalid tick flags"):
resolve_history_tick_flags("BAD")
with pytest.raises(ValueError, match="Invalid tick flags"):
resolve_history_tick_flags(7)
def test_resolve_history_timeframes_invalid(self) -> None:
"""Test invalid timeframes raise ValueError."""
+342 -36
View File
@@ -19,7 +19,7 @@ if TYPE_CHECKING:
from pdmt5 import Mt5Config, Mt5DataClient
from mt5cli import sdk
from mt5cli.history import DEFAULT_HISTORY_TIMEFRAMES
from mt5cli.history import DEFAULT_HISTORY_TIMEFRAMES, write_rates_dataset
from mt5cli.sdk import (
AccountSpec,
Mt5CliClient,
@@ -27,6 +27,8 @@ from mt5cli.sdk import (
account_info,
build_config,
collect_history,
collect_latest_closed_rates_by_granularity,
collect_latest_closed_rates_for_accounts,
collect_latest_rates,
collect_latest_rates_for_accounts,
collect_latest_rates_for_accounts_with_retries,
@@ -59,7 +61,7 @@ from mt5cli.sdk import (
update_history_with_config,
version,
)
from mt5cli.utils import Dataset
from mt5cli.utils import Dataset, IfExists
class _TerminalInfo(NamedTuple):
@@ -130,32 +132,6 @@ _DEALS_FIXTURE: dict[str, list[object]] = {
}
@pytest.fixture
def mock_client(mocker: MockerFixture) -> MagicMock:
"""Create and patch a mock Mt5DataClient for SDK tests."""
client = MagicMock()
sample_df = pd.DataFrame({"col": [1]})
client.copy_rates_from_as_df.return_value = sample_df
client.copy_rates_from_pos_as_df.return_value = sample_df
client.copy_rates_range_as_df.return_value = sample_df
client.copy_ticks_from_as_df.return_value = sample_df
client.copy_ticks_range_as_df.return_value = sample_df
client.account_info_as_df.return_value = sample_df
client.terminal_info_as_df.return_value = sample_df
client.symbols_get_as_df.return_value = sample_df
client.symbol_info_as_df.return_value = sample_df
client.orders_get_as_df.return_value = sample_df
client.positions_get_as_df.return_value = sample_df
client.history_orders_get_as_df.return_value = sample_df
client.history_deals_get_as_df.return_value = sample_df
client.version_as_df.return_value = sample_df
client.last_error_as_df.return_value = sample_df
client.symbol_info_tick_as_df.return_value = sample_df
client.market_book_get_as_df.return_value = sample_df
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=client)
return client
def _build_history_client(mocker: MockerFixture) -> MagicMock:
"""Build a mocked Mt5DataClient with per-symbol history results."""
client = MagicMock()
@@ -199,7 +175,7 @@ class TestConnectionLifecycle:
mock_client = MagicMock()
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=mock_client)
config = MagicMock()
with sdk._connected_client(config): # type: ignore[reportPrivateUsage]
with sdk.connected_client(config): # type: ignore[reportPrivateUsage]
mock_client.initialize_and_login_mt5.assert_called_once()
mock_client.shutdown.assert_called_once()
@@ -215,7 +191,7 @@ class TestConnectionLifecycle:
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=mock_client)
with (
pytest.raises(RuntimeError, match="login failed"),
sdk._connected_client(MagicMock()), # type: ignore[reportPrivateUsage]
sdk.connected_client(MagicMock()), # type: ignore[reportPrivateUsage]
):
pass
mock_client.shutdown.assert_called_once()
@@ -388,7 +364,7 @@ class TestMt5CliClient:
symbol="EURUSD",
date_from=datetime(2024, 1, 1, tzinfo=UTC),
count=100,
flags=2,
flags=1,
)
def test_history_orders_accepts_string_dates(
@@ -949,7 +925,7 @@ class TestUpdateHistory:
assert kwargs["symbol"] == "EURUSD"
assert kwargs["date_from"] == expected_start
assert kwargs["date_to"] == date_to
assert kwargs["flags"] == 1
assert kwargs["flags"] == -1
return pd.DataFrame({
"time": ["2024-01-01T12:00:00+00:00"],
"time_msc": [1_704_110_400_000],
@@ -1117,7 +1093,7 @@ class TestRecentTicks:
symbol="EURUSD",
date_from=end - timedelta(seconds=60),
count=100,
flags=2,
flags=1,
)
client.copy_ticks_range_as_df.assert_not_called()
@@ -1147,7 +1123,7 @@ class TestRecentTicks:
assert kwargs["symbol"] == "EURUSD"
assert kwargs["date_to"] == tick.time
assert kwargs["date_from"] == tick.time - timedelta(seconds=30)
assert kwargs["flags"] == 1
assert kwargs["flags"] == -1
def test_recent_ticks_rejects_unsupported_tick_time(
self,
@@ -1217,7 +1193,7 @@ class TestRecentTicks:
symbol="EURUSD",
date_from=end - timedelta(seconds=60),
date_to=end,
flags=1,
flags=-1,
)
@@ -1366,7 +1342,7 @@ class TestCollectLatestRatesForAccounts:
"""Test account fields override base_config, empty login falls back."""
configs: list[object] = []
def _record_config(*, config: object) -> MagicMock:
def _record_config(*, config: object, **_: object) -> MagicMock:
configs.append(config)
return mock_client
@@ -1523,6 +1499,199 @@ class TestCollectLatestRatesForAccountsWithRetries:
sleep.assert_not_called()
class TestCollectLatestClosedRatesForAccounts:
"""Tests for collect_latest_closed_rates_for_accounts."""
def test_fetches_count_plus_one_and_drops_forming_bar(
self,
mocker: MockerFixture,
) -> None:
"""Test closed-bar collection requests one extra bar at start_pos=0."""
df_rate = pd.DataFrame({"time": [1, 2, 3], "close": [1.1, 1.2, 1.3]})
wrapped = mocker.patch(
"mt5cli.sdk.collect_latest_rates_for_accounts_with_retries",
return_value={("EURUSD", 1): df_rate},
)
accounts = [AccountSpec(symbols=["EURUSD"])]
result = collect_latest_closed_rates_for_accounts(
accounts,
["M1"],
count=2,
retry_count=1,
backoff_base=3,
)
wrapped.assert_called_once_with(
accounts,
["M1"],
3,
start_pos=0,
base_config=None,
retry_count=1,
backoff_base=3,
)
pd.testing.assert_frame_equal(
result["EURUSD", 1],
pd.DataFrame({"time": [1, 2], "close": [1.1, 1.2]}),
)
def test_rejects_forming_bar_only_frames(self, mocker: MockerFixture) -> None:
"""Test empty results after dropping the forming bar raise ValueError."""
mocker.patch(
"mt5cli.sdk.collect_latest_rates_for_accounts_with_retries",
return_value={("EURUSD", 1): pd.DataFrame({"time": [1], "close": [1.1]})},
)
with pytest.raises(ValueError, match="Rate data is empty"):
collect_latest_closed_rates_for_accounts(
[AccountSpec(symbols=["EURUSD"])],
["M1"],
count=1,
)
def test_skips_extra_fetch_when_start_pos_nonzero(
self,
mocker: MockerFixture,
) -> None:
"""Test start_pos > 0 fetches count bars without dropping the last row."""
df_rate = pd.DataFrame({"time": [1, 2], "close": [1.1, 1.2]})
wrapped = mocker.patch(
"mt5cli.sdk.collect_latest_rates_for_accounts_with_retries",
return_value={("EURUSD", 1): df_rate},
)
result = collect_latest_closed_rates_for_accounts(
[AccountSpec(symbols=["EURUSD"])],
["M1"],
count=2,
start_pos=1,
)
wrapped.assert_called_once_with(
[AccountSpec(symbols=["EURUSD"])],
["M1"],
2,
start_pos=1,
base_config=None,
retry_count=0,
backoff_base=2.0,
)
pd.testing.assert_frame_equal(result["EURUSD", 1], df_rate)
def test_rejects_zero_count_before_fetching(self, mocker: MockerFixture) -> None:
"""Test count=0 is rejected before any MT5 collection attempt."""
wrapped = mocker.patch(
"mt5cli.sdk.collect_latest_rates_for_accounts_with_retries",
)
with pytest.raises(ValueError, match="count must be positive"):
collect_latest_closed_rates_for_accounts(
[AccountSpec(symbols=["EURUSD"])],
["M1"],
count=0,
)
wrapped.assert_not_called()
def test_rejects_negative_start_pos(self, mocker: MockerFixture) -> None:
"""Test negative start_pos is rejected before any MT5 collection attempt."""
wrapped = mocker.patch(
"mt5cli.sdk.collect_latest_rates_for_accounts_with_retries",
)
with pytest.raises(ValueError, match="start_pos must be non-negative"):
collect_latest_closed_rates_for_accounts(
[AccountSpec(symbols=["EURUSD"])],
["M1"],
count=1,
start_pos=-1,
)
wrapped.assert_not_called()
def test_rejects_empty_frames_with_start_pos_nonzero(
self,
mocker: MockerFixture,
) -> None:
"""Test empty upstream frames raise ValueError when start_pos > 0."""
mocker.patch(
"mt5cli.sdk.collect_latest_rates_for_accounts_with_retries",
return_value={("EURUSD", 1): pd.DataFrame(columns=["time", "close"])},
)
with pytest.raises(ValueError, match="Rate data is empty"):
collect_latest_closed_rates_for_accounts(
[AccountSpec(symbols=["EURUSD"])],
["M1"],
count=1,
start_pos=1,
)
def test_processes_multiple_symbol_timeframe_pairs(
self,
mocker: MockerFixture,
) -> None:
"""Test each returned series is trimmed and validated independently."""
mocker.patch(
"mt5cli.sdk.collect_latest_rates_for_accounts_with_retries",
return_value={
("EURUSD", 1): pd.DataFrame(
{"time": [1, 2, 3], "close": [1.1, 1.2, 1.3]},
),
("GBPUSD", 16385): pd.DataFrame(
{"time": [4, 5, 6], "close": [2.1, 2.2, 2.3]},
),
},
)
result = collect_latest_closed_rates_for_accounts(
[AccountSpec(symbols=["EURUSD", "GBPUSD"])],
["M1", "H1"],
count=2,
)
assert set(result) == {("EURUSD", 1), ("GBPUSD", 16385)}
pd.testing.assert_frame_equal(
result["EURUSD", 1],
pd.DataFrame({"time": [1, 2], "close": [1.1, 1.2]}),
)
pd.testing.assert_frame_equal(
result["GBPUSD", 16385],
pd.DataFrame({"time": [4, 5], "close": [2.1, 2.2]}),
)
class TestCollectLatestClosedRatesByGranularity:
"""Tests for collect_latest_closed_rates_by_granularity."""
def test_rekeys_by_granularity_name(self, mocker: MockerFixture) -> None:
"""Test closed rates are keyed by symbol and granularity name."""
df_rate = pd.DataFrame({"time": [1, 2], "close": [1.1, 1.2]})
wrapped = mocker.patch(
"mt5cli.sdk.collect_latest_closed_rates_for_accounts",
return_value={("EURUSD", 1): df_rate},
)
result = collect_latest_closed_rates_by_granularity(
[AccountSpec(symbols=["EURUSD"])],
["M1"],
count=2,
)
wrapped.assert_called_once_with(
[AccountSpec(symbols=["EURUSD"])],
["M1"],
2,
start_pos=0,
base_config=None,
retry_count=0,
backoff_base=2.0,
)
assert ("EURUSD", "M1") in result
pd.testing.assert_frame_equal(result["EURUSD", "M1"], df_rate)
class TestSubstituteEnvPlaceholders:
"""Tests for ${ENV_VAR} substitution."""
@@ -1718,6 +1887,16 @@ class TestThrottledHistoryUpdater:
Mt5RuntimeError("boom"),
Mt5TradingError("trade failed"),
sqlite3.OperationalError("locked"),
ValueError("invalid symbols"),
OSError("disk full"),
AttributeError(
"'StubClient' object has no attribute 'copy_rates_range_as_df'",
name="copy_rates_range_as_df",
),
AttributeError(
"MT5 client is missing required method: copy_ticks_range_as_df"
),
TypeError("MT5 client attribute is not callable: history_orders_get_as_df"),
],
)
def test_suppresses_errors_when_requested(
@@ -1737,3 +1916,130 @@ class TestThrottledHistoryUpdater:
assert updater.update(MagicMock(), ["EURUSD"]) is False
assert updater.last_update_monotonic is None
@pytest.mark.parametrize(
"error",
[
AttributeError("'dict' object has no attribute 'typo'"),
TypeError("unsupported operand types"),
],
)
def test_suppress_errors_does_not_hide_programming_errors(
self,
mocker: MockerFixture,
error: Exception,
) -> None:
"""Test generic AttributeError/TypeError still propagate when suppressed."""
mocker.patch(
"mt5cli.sdk.update_history",
side_effect=error,
)
updater = ThrottledHistoryUpdater(
output="history.db",
suppress_errors=True,
)
with pytest.raises(type(error)):
updater.update(MagicMock(), ["EURUSD"])
assert updater.last_update_monotonic is None
@pytest.mark.parametrize(
("error", "expected"),
[
(AttributeError("MT5 client is missing required method: version"), True),
(
AttributeError(
"'Stub' object has no attribute 'copy_rates_range_as_df'",
name="copy_rates_range_as_df",
),
True,
),
(AttributeError("'dict' object has no attribute 'typo'"), False),
(TypeError("MT5 client attribute is not callable: version"), True),
(TypeError("unsupported operand types"), False),
(TypeError("'NoneType' object is not callable"), False),
(ValueError("invalid"), False),
],
)
def test_is_mt5_client_capability_error(
self,
error: BaseException,
expected: bool,
) -> None:
"""Test MT5 client capability error detection."""
assert sdk._is_mt5_client_capability_error(error) is expected # type: ignore[reportPrivateUsage]
def test_is_mt5_client_capability_error_for_non_callable_history_client(
self,
) -> None:
"""Test non-callable history client attributes are capability errors."""
client = MagicMock()
client.copy_rates_range_as_df = None
with (
sqlite3.connect(":memory:") as conn,
pytest.raises(TypeError, match="not callable") as exc_info,
):
write_rates_dataset(
conn,
client,
["EURUSD"],
1,
datetime.now(UTC),
datetime.now(UTC),
IfExists.APPEND,
{},
)
assert sdk._is_mt5_client_capability_error(exc_info.value) is True # type: ignore[reportPrivateUsage]
def test_suppresses_non_callable_history_client_method(
self,
tmp_path: Path,
) -> None:
"""Test suppress_errors swallows non-callable history client API attributes."""
client = MagicMock()
client.copy_rates_range_as_df = None
updater = ThrottledHistoryUpdater(
output=tmp_path / "history.db",
datasets={Dataset.rates},
timeframes=["M1"],
suppress_errors=True,
)
assert updater.update(client, ["EURUSD"]) is False
assert updater.last_update_monotonic is None
def test_suppress_errors_does_not_hide_internal_client_type_error(
self,
mocker: MockerFixture,
) -> None:
"""Test TypeError raised inside a callable client method still propagates."""
mocker.patch(
"mt5cli.sdk.update_history",
side_effect=TypeError("'int' object is not callable"),
)
updater = ThrottledHistoryUpdater(
output="history.db",
suppress_errors=True,
)
with pytest.raises(TypeError, match="not callable"):
updater.update(MagicMock(), ["EURUSD"])
assert updater.last_update_monotonic is None
def test_suppresses_validation_errors_before_update(
self,
mocker: MockerFixture,
) -> None:
"""Test validation failures are suppressed without calling update_history."""
update = mocker.patch("mt5cli.sdk.update_history")
updater = ThrottledHistoryUpdater(
output="history.db",
suppress_errors=True,
)
assert updater.update(MagicMock(), []) is False
update.assert_not_called()
assert updater.last_update_monotonic is None
+356
View File
@@ -0,0 +1,356 @@
"""Tests for trading session helpers and operational utilities."""
from __future__ import annotations
from unittest.mock import MagicMock
import pandas as pd
import pytest
from pdmt5 import Mt5RuntimeError
from pytest_mock import MockerFixture # noqa: TC002
from mt5cli.sdk import build_config
from mt5cli.trading import (
calculate_margin_and_volume,
detect_position_side,
determine_order_limits,
mt5_trading_session,
)
class TestDetectPositionSide:
"""Tests for detect_position_side."""
def test_returns_none_when_no_positions(self) -> None:
"""Test None is returned when no open positions exist."""
client = MagicMock()
client.positions_get_as_df.return_value = pd.DataFrame()
assert detect_position_side(client, "EURUSD") is None
def test_returns_long_for_net_buy_volume(self) -> None:
"""Test long is returned when buy volume exceeds sell volume."""
client = MagicMock()
client.mt5.POSITION_TYPE_BUY = 0
client.mt5.POSITION_TYPE_SELL = 1
client.positions_get_as_df.return_value = pd.DataFrame(
{
"type": [0, 0, 1],
"volume": [0.2, 0.1, 0.05],
},
)
assert detect_position_side(client, "EURUSD") == "long"
def test_returns_short_for_net_sell_volume(self) -> None:
"""Test short is returned when sell volume exceeds buy volume."""
client = MagicMock()
client.mt5.POSITION_TYPE_BUY = 0
client.mt5.POSITION_TYPE_SELL = 1
client.positions_get_as_df.return_value = pd.DataFrame(
{
"type": [1, 1],
"volume": [0.3, 0.1],
},
)
assert detect_position_side(client, "EURUSD") == "short"
def test_returns_none_for_balanced_hedged_positions(self) -> None:
"""Test None is returned when buy and sell volumes net to zero."""
client = MagicMock()
client.mt5.POSITION_TYPE_BUY = 0
client.mt5.POSITION_TYPE_SELL = 1
client.positions_get_as_df.return_value = pd.DataFrame(
{
"type": [0, 1],
"volume": [0.2, 0.2],
},
)
assert detect_position_side(client, "EURUSD") is None
class TestCalculateMarginAndVolume:
"""Tests for calculate_margin_and_volume."""
def test_calculates_margin_budget_and_volumes(self) -> None:
"""Test margin budget and buy/sell volumes are derived from ratios."""
client = MagicMock()
client.account_info_as_dict.return_value = {"margin_free": 1000.0}
client.calculate_volume_by_margin.side_effect = [0.3, 0.2]
result = calculate_margin_and_volume(
client,
"EURUSD",
unit_margin_ratio=0.5,
preserved_margin_ratio=0.2,
)
assert result == {
"margin_free": 1000.0,
"available_margin": 800.0,
"trade_margin": 400.0,
"buy_volume": 0.3,
"sell_volume": 0.2,
}
client.calculate_volume_by_margin.assert_any_call("EURUSD", 400.0, "BUY")
client.calculate_volume_by_margin.assert_any_call("EURUSD", 400.0, "SELL")
@pytest.mark.parametrize(
("account_dict", "expected_margin_free"),
[
({"margin_free": 0.0}, 0.0),
({}, 0.0),
({"margin_free": None}, 0.0),
],
)
def test_zero_or_missing_margin_free(
self,
account_dict: dict[str, float | None],
expected_margin_free: float,
) -> None:
"""Test missing or zero margin_free yields zero trade margin."""
client = MagicMock()
client.account_info_as_dict.return_value = account_dict
client.calculate_volume_by_margin.return_value = 0.0
result = calculate_margin_and_volume(
client,
"EURUSD",
unit_margin_ratio=0.5,
preserved_margin_ratio=0.2,
)
assert result["margin_free"] == expected_margin_free
client.calculate_volume_by_margin.assert_any_call("EURUSD", 0.0, "BUY")
client.calculate_volume_by_margin.assert_any_call("EURUSD", 0.0, "SELL")
def test_clamps_negative_margin_free_to_zero(self) -> None:
"""Test negative margin_free is clamped to zero before sizing."""
client = MagicMock()
client.account_info_as_dict.return_value = {"margin_free": -500.0}
client.calculate_volume_by_margin.return_value = 0.0
result = calculate_margin_and_volume(
client,
"EURUSD",
unit_margin_ratio=0.5,
preserved_margin_ratio=0.2,
)
expected_margin_free = 0.0
assert result["margin_free"] == expected_margin_free
client.calculate_volume_by_margin.assert_any_call("EURUSD", 0.0, "BUY")
client.calculate_volume_by_margin.assert_any_call("EURUSD", 0.0, "SELL")
@pytest.mark.parametrize(
("unit_ratio", "preserved_ratio"),
[
(-0.1, 0.0),
(1.1, 0.0),
(0.5, -0.1),
(0.5, 1.1),
],
)
def test_rejects_invalid_ratios(
self,
unit_ratio: float,
preserved_ratio: float,
) -> None:
"""Test invalid ratio values raise ValueError."""
with pytest.raises(ValueError, match="must be between 0 and 1"):
calculate_margin_and_volume(
MagicMock(),
"EURUSD",
unit_margin_ratio=unit_ratio,
preserved_margin_ratio=preserved_ratio,
)
class TestDetermineOrderLimits:
"""Tests for determine_order_limits."""
@pytest.mark.parametrize(
("side", "expected_entry_key"),
[
("long", "ask"),
("short", "bid"),
("buy", "ask"),
("sell", "bid"),
],
)
def test_uses_expected_quote_for_entry(
self,
side: str,
expected_entry_key: str,
) -> None:
"""Test entry price is taken from ask for long/buy and bid for short/sell."""
client = MagicMock()
client.symbol_info_tick_as_dict.return_value = {"ask": 1.1010, "bid": 1.1000}
result = determine_order_limits(
client,
"EURUSD",
side,
stop_loss_limit_ratio=0.0,
take_profit_limit_ratio=0.0,
)
assert (
result["entry"]
== client.symbol_info_tick_as_dict.return_value[expected_entry_key]
)
assert result["stop_loss"] is None
assert result["take_profit"] is None
def test_calculates_long_protective_levels(self) -> None:
"""Test long stop loss and take profit are placed below/above entry."""
client = MagicMock()
client.symbol_info_tick_as_dict.return_value = {"ask": 100.0, "bid": 99.0}
result = determine_order_limits(
client,
"EURUSD",
"long",
stop_loss_limit_ratio=0.02,
take_profit_limit_ratio=0.03,
)
assert result == {
"entry": 100.0,
"stop_loss": 98.0,
"take_profit": 103.0,
}
def test_calculates_short_protective_levels(self) -> None:
"""Test short stop loss and take profit are placed above/below entry."""
client = MagicMock()
client.symbol_info_tick_as_dict.return_value = {"ask": 100.0, "bid": 99.0}
result = determine_order_limits(
client,
"EURUSD",
"short",
stop_loss_limit_ratio=0.02,
take_profit_limit_ratio=0.03,
)
assert result == {
"entry": 99.0,
"stop_loss": 100.98,
"take_profit": 96.03,
}
def test_rejects_unknown_side(self) -> None:
"""Test unsupported side values raise ValueError."""
with pytest.raises(ValueError, match="Unsupported order side"):
determine_order_limits(
MagicMock(),
"EURUSD",
"flat",
stop_loss_limit_ratio=0.01,
take_profit_limit_ratio=0.01,
)
@pytest.mark.parametrize(
("stop_loss_ratio", "take_profit_ratio"),
[
(-0.05, 0.01),
(0.01, 2.0),
],
)
def test_rejects_invalid_protective_ratios(
self,
stop_loss_ratio: float,
take_profit_ratio: float,
) -> None:
"""Test out-of-range protective ratios raise ValueError."""
with pytest.raises(ValueError, match="must be at least 0 and less than 1"):
determine_order_limits(
MagicMock(),
"EURUSD",
"long",
stop_loss_limit_ratio=stop_loss_ratio,
take_profit_limit_ratio=take_profit_ratio,
)
@pytest.mark.parametrize(
("field", "ratio"),
[
("stop_loss_limit_ratio", 1.0),
("take_profit_limit_ratio", 1.0),
],
)
def test_rejects_unit_boundary_protective_ratios(
self,
field: str,
ratio: float,
) -> None:
"""Test protective ratios of exactly 1.0 are rejected."""
kwargs = {
"stop_loss_limit_ratio": 0.01,
"take_profit_limit_ratio": 0.01,
field: ratio,
}
with pytest.raises(ValueError, match="must be at least 0 and less than 1"):
determine_order_limits(
MagicMock(),
"EURUSD",
"long",
**kwargs,
)
class TestMt5TradingSession:
"""Tests for the mt5_trading_session context manager."""
def test_yields_connected_client_and_shuts_down(
self,
mocker: MockerFixture,
) -> None:
"""Test mt5_trading_session connects, yields a client, and shuts down."""
mock_client = MagicMock()
trading_client = mocker.patch(
"mt5cli.trading.Mt5TradingClient",
return_value=mock_client,
)
with mt5_trading_session(
build_config(path="/opt/mt5/terminal64.exe"),
retry_count=2,
) as client:
mock_client.initialize_and_login_mt5.assert_called_once()
assert client is mock_client
trading_client.assert_called_once()
assert trading_client.call_args.kwargs["retry_count"] == 2
assert (
trading_client.call_args.kwargs["config"].path == "/opt/mt5/terminal64.exe"
)
mock_client.shutdown.assert_called_once()
def test_shuts_down_when_initialize_raises(
self,
mocker: MockerFixture,
) -> None:
"""Test shutdown is called when initialization fails."""
mock_client = MagicMock()
mock_client.initialize_and_login_mt5.side_effect = Mt5RuntimeError("boom")
mocker.patch("mt5cli.trading.Mt5TradingClient", return_value=mock_client)
with pytest.raises(Mt5RuntimeError, match="boom"), mt5_trading_session():
pass
mock_client.shutdown.assert_called_once()
def test_shuts_down_when_body_raises(self, mocker: MockerFixture) -> None:
"""Test shutdown is called when the context body raises."""
mock_client = MagicMock()
mocker.patch("mt5cli.trading.Mt5TradingClient", return_value=mock_client)
body_error = "body error"
with pytest.raises(RuntimeError, match=body_error), mt5_trading_session():
raise RuntimeError(body_error)
mock_client.shutdown.assert_called_once()
+51 -14
View File
@@ -274,8 +274,14 @@ class TestParseTimeframe:
assert parse_timeframe(value) == expected
def test_integer_timeframe(self) -> None:
"""Test parsing integer timeframe."""
assert parse_timeframe("42") == 42
"""Test parsing supported integer timeframes."""
assert parse_timeframe("1") == 1
assert parse_timeframe(16385) == 16385
def test_unsupported_integer_timeframe_raises(self) -> None:
"""Test that unsupported integer timeframes raise ValueError."""
with pytest.raises(ValueError, match="Invalid timeframe"):
parse_timeframe("42")
def test_invalid_timeframe_raises(self) -> None:
"""Test that invalid timeframe raises ValueError."""
@@ -288,15 +294,21 @@ class TestParseTickFlags:
@pytest.mark.parametrize(
("value", "expected"),
[("ALL", 1), ("info", 2), ("TRADE", 4)],
[("ALL", -1), ("info", 1), ("TRADE", 2), ("COPY_TICKS_ALL", -1)],
)
def test_named_flag(self, value: str, expected: int) -> None:
"""Test parsing named tick flags."""
assert parse_tick_flags(value) == expected
def test_integer_flag(self) -> None:
"""Test parsing integer tick flag."""
assert parse_tick_flags("7") == 7
"""Test parsing supported integer tick flags."""
assert parse_tick_flags("-1") == -1
assert parse_tick_flags(2) == 2
def test_unsupported_integer_flag_raises(self) -> None:
"""Test that unsupported integer tick flags raise ValueError."""
with pytest.raises(ValueError, match="Invalid tick flags"):
parse_tick_flags("7")
def test_invalid_flag_raises(self) -> None:
"""Test that invalid flag raises ValueError."""
@@ -355,8 +367,11 @@ class TestConstants:
assert key in TIMEFRAME_MAP
def test_tick_flag_map_has_expected_keys(self) -> None:
"""Test that TICK_FLAG_MAP contains standard flags."""
assert set(TICK_FLAG_MAP) == {"ALL", "INFO", "TRADE"}
"""Test that TICK_FLAG_MAP contains standard flags with MT5 values."""
assert {"ALL", "INFO", "TRADE"} <= set(TICK_FLAG_MAP)
assert TICK_FLAG_MAP["ALL"] == -1
assert TICK_FLAG_MAP["INFO"] == 1
assert TICK_FLAG_MAP["TRADE"] == 2
@pytest.mark.parametrize(
("dataset", "expected"),
@@ -403,26 +418,48 @@ class TestTimeframeType:
"""Test converting a string to timeframe integer."""
assert TIMEFRAME_TYPE.convert("H1", None, None) == 16385
def test_convert_int_passthrough(self) -> None:
"""Test that integer values pass through unchanged."""
assert TIMEFRAME_TYPE.convert(42, None, None) == 42
def test_convert_int(self) -> None:
"""Test converting supported integer timeframe values."""
assert TIMEFRAME_TYPE.convert(16385, None, None) == 16385
def test_convert_unsupported_int(self) -> None:
"""Test that unsupported integer values raise BadParameter."""
with pytest.raises(Exception, match="Invalid timeframe"):
TIMEFRAME_TYPE.convert(42, None, None)
def test_convert_invalid(self) -> None:
"""Test that invalid values raise BadParameter."""
with pytest.raises(Exception, match="Invalid timeframe"):
TIMEFRAME_TYPE.convert("bad", None, None)
@pytest.mark.parametrize("value", [True, False, None, 1.5])
def test_convert_invalid_types(self, value: object) -> None:
"""Test that bool, float, and None values raise BadParameter."""
with pytest.raises(Exception, match="Invalid timeframe"):
TIMEFRAME_TYPE.convert(value, None, None)
class TestTickFlagsType:
"""Tests for _TickFlagsType."""
def test_convert_string(self) -> None:
"""Test converting a string to tick flags integer."""
assert TICK_FLAGS_TYPE.convert("ALL", None, None) == 1
assert TICK_FLAGS_TYPE.convert("ALL", None, None) == -1
def test_convert_int_passthrough(self) -> None:
"""Test that integer values pass through unchanged."""
assert TICK_FLAGS_TYPE.convert(7, None, None) == 7
def test_convert_int(self) -> None:
"""Test converting supported integer tick flag values."""
assert TICK_FLAGS_TYPE.convert(2, None, None) == 2
def test_convert_unsupported_int(self) -> None:
"""Test that unsupported integer values raise BadParameter."""
with pytest.raises(Exception, match="Invalid tick flags"):
TICK_FLAGS_TYPE.convert(7, None, None)
@pytest.mark.parametrize("value", [True, False, None, 1.5])
def test_convert_invalid_types(self, value: object) -> None:
"""Test that bool, float, and None values raise BadParameter."""
with pytest.raises(Exception, match="Invalid tick flags"):
TICK_FLAGS_TYPE.convert(value, None, None)
def test_convert_invalid(self) -> None:
"""Test that invalid values raise BadParameter."""
Generated
+8 -8
View File
@@ -487,7 +487,7 @@ wheels = [
[[package]]
name = "mt5cli"
version = "0.5.3"
version = "0.7.2"
source = { editable = "." }
dependencies = [
{ name = "click" },
@@ -513,7 +513,7 @@ dev = [
[package.metadata]
requires-dist = [
{ name = "click", specifier = ">=8.1.0" },
{ name = "pdmt5", specifier = ">=0.2.3" },
{ name = "pdmt5", specifier = ">=0.3.0" },
{ name = "pyarrow", specifier = ">=19.0.0" },
{ name = "typer", specifier = ">=0.15.0" },
]
@@ -684,16 +684,16 @@ wheels = [
[[package]]
name = "pdmt5"
version = "0.2.3"
version = "0.3.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "metatrader5", marker = "sys_platform == 'win32'" },
{ name = "pandas" },
{ name = "pydantic" },
]
sdist = { url = "https://files.pythonhosted.org/packages/02/25/52d9d954504ccdd0fe91f715ab74c424d61234b237cc4160d3ebe20070f1/pdmt5-0.2.3.tar.gz", hash = "sha256:21384f5826fb0125fee3f93c90b108340f55ab53b1c819d229ceac162289d2ec", size = 226665, upload-time = "2026-02-05T13:28:21.071Z" }
sdist = { url = "https://files.pythonhosted.org/packages/bf/cc/c8fa3a01e0e34178fec8527992f7bb8eda5881477ce23aaacaa9b2ef7bec/pdmt5-0.3.0.tar.gz", hash = "sha256:bb612d5c2695eafac9b2a7b74756e13bd383d7e5517bd90c9a2efa92492c484c", size = 215100, upload-time = "2026-06-11T13:26:46.976Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/c1/75/c5e52a9cf459b85b2dd52f83e70857571b1b45805c9fe610b3959a26ac15/pdmt5-0.2.3-py3-none-any.whl", hash = "sha256:f92246a05cfc3b7feb3ab0cc5b48768a4d84aad6b02e7a68060948f5828718a1", size = 22967, upload-time = "2026-02-05T13:28:19.523Z" },
{ url = "https://files.pythonhosted.org/packages/f2/03/b12cc4c9db983d971c9172b3765161b6d91136d0624e6718a04dd815e7a1/pdmt5-0.3.0-py3-none-any.whl", hash = "sha256:5388b406cc583202600cfe22c9d781679b1d931b1ed5a2b5dcf37c566149b49f", size = 26250, upload-time = "2026-06-11T13:26:45.689Z" },
]
[[package]]
@@ -836,11 +836,11 @@ wheels = [
[[package]]
name = "pygments"
version = "2.19.2"
version = "2.20.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/b0/77/a5b8c569bf593b0140bde72ea885a803b82086995367bf2037de0159d924/pygments-2.19.2.tar.gz", hash = "sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887", size = 4968631, upload-time = "2025-06-21T13:39:12.283Z" }
sdist = { url = "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", size = 4955991, upload-time = "2026-03-29T13:29:33.898Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/c7/21/705964c7812476f378728bdf590ca4b771ec72385c533964653c68e86bdc/pygments-2.19.2-py3-none-any.whl", hash = "sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b", size = 1225217, upload-time = "2025-06-21T13:39:07.939Z" },
{ url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" },
]
[[package]]