Compare commits
5 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 0fad55d609 | |||
| d654b82f9d | |||
| b5e82e71c7 | |||
| 18df96872b | |||
| 5b1d54bfe9 |
@@ -6,6 +6,12 @@ Command-line tool for exporting MetaTrader 5 data to CSV, JSON, Parquet, and SQL
|
||||
|
||||
Built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data handler for MetaTrader 5.
|
||||
|
||||
## Architecture
|
||||
|
||||
- **pdmt5** — canonical MT5 client, DataFrame/trading primitives, and MT5 constant parsing (`TIMEFRAME_*`, `COPY_TICKS_*`, order types).
|
||||
- **mt5cli** — CLI commands, CSV/JSON/Parquet/SQLite export, SQLite history collection, rate views, and local batch/automation SDK helpers built on pdmt5.
|
||||
- **mt5api** — sibling HTTP adapter for remote MT5 access; not a dependency of mt5cli.
|
||||
|
||||
## Features
|
||||
|
||||
- **Multi-format export**: CSV, JSON, Parquet, and SQLite3 output formats
|
||||
@@ -136,7 +142,25 @@ update_history_with_config(
|
||||
- **Rate view resolution**: use `resolve_rate_view_name()` / `resolve_rate_view_names()` to map symbols and granularities to existing SQLite compatibility views without creating databases. Both accept `None` (or a missing path) and return deterministic default names unless `require_existing=True`.
|
||||
- **Rate view loading**: use `load_rate_data()` / `load_rate_data_from_connection()` to load a SQLite rate table or view into a `DatetimeIndex` DataFrame.
|
||||
- **Multi-series rate loading**: use `build_rate_targets()` to build neutral `RateTarget(symbol, timeframe)` pairs, `resolve_rate_tables()` to map them to table/view names (pass `require_existing=True` for strict resolution), and `load_rate_series_from_sqlite()` to load them into a mapping keyed by `(symbol, integer timeframe)`. The loader requires existing managed views unless `explicit_tables` is supplied, and rejects duplicate `(symbol, timeframe)` targets.
|
||||
- **Multi-account latest rates**: use `collect_latest_rates_for_accounts()` with `AccountSpec` to read the latest bars for several account groups, merged into a `(symbol, integer timeframe)` mapping.
|
||||
- **Multi-account latest rates**: use `collect_latest_rates_for_accounts()` with `AccountSpec` to read the latest bars for several account groups, merged into a `(symbol, integer timeframe)` mapping. For long-running pollers, `collect_latest_rates_for_accounts_with_retries()` adds bounded exponential backoff that retries only `pdmt5.Mt5TradingError` / `pdmt5.Mt5RuntimeError` and re-raises once `retry_count` is exhausted.
|
||||
- **Latest closed bars**: use `collect_latest_closed_rates_for_accounts()` when downstream logic must exclude the still-forming current bar. It fetches `count + 1` bars at `start_pos=0`, drops the last row with `drop_forming_rate_bar()`, and validates each series is non-empty. `collect_latest_closed_rates_by_granularity()` returns the same data keyed by `(symbol, granularity_name)` such as `("EURUSD", "M1")`.
|
||||
|
||||
```python
|
||||
from mt5cli import AccountSpec, collect_latest_closed_rates_by_granularity
|
||||
|
||||
rates = collect_latest_closed_rates_by_granularity(
|
||||
[AccountSpec(symbols=["EURUSD", "GBPUSD"], login=12345)],
|
||||
["M1", "H1"],
|
||||
count=500,
|
||||
retry_count=3,
|
||||
)
|
||||
eurusd_m1 = rates["EURUSD", "M1"] # closed bars only
|
||||
```
|
||||
|
||||
- **Credential resolution**: use `resolve_account_spec()` / `resolve_account_specs()` to merge explicit override values over `AccountSpec` fields and expand `${ENV_VAR}` placeholders (via `substitute_env_placeholders()`), raising `ValueError` for missing variables. This keeps secrets out of plan/config files without coupling to any strategy code.
|
||||
- **Throttled history updates**: use `ThrottledHistoryUpdater` to wrap `update_history()` with a minimum `interval_seconds` between successful runs (monotonic clock). Call `should_update()` / `update(client, symbols)` from an application loop; errors propagate by default, or pass `suppress_errors=True` to swallow recoverable `Mt5*Error`, `sqlite3.Error`, `ValueError`, `OSError`, and MT5 client capability errors for history API methods without advancing the throttle (other `AttributeError` / `TypeError` values always propagate).
|
||||
- **Trading session helpers**: use `mt5_trading_session()` for a trading-capable `pdmt5.Mt5TradingClient` that initializes/logs in via `Mt5Config.path` and always shuts down safely. Pair with `detect_position_side()`, `calculate_margin_and_volume()`, and `determine_order_limits()` for generic position and sizing utilities. The read-only `mt5_session()` / `Mt5CliClient` SDK is unchanged.
|
||||
- **Granularity-keyed rate loading**: `load_rate_series_by_granularity()` builds targets with `build_rate_targets()`, loads them with `load_rate_series_from_sqlite()`, and returns a mapping keyed by `(symbol | None, granularity_name)` such as `("EURUSD", "M1")` to reduce downstream boilerplate.
|
||||
- **MT5 session helper**: use the `mt5_session()` context manager to attach to (or, when `Mt5Config.path` is set, launch) an MT5 terminal, log in, and yield a connected `Mt5CliClient` that shuts down on exit.
|
||||
- **SQLite export helpers**: use `export_dataframe_to_sqlite()` for append mode, optional index export, and post-write deduplication by key columns.
|
||||
- **Recent ticks and margins**: `recent_ticks()` and `minimum_margins()` SDK helpers (and matching CLI commands) cover common downstream read-only queries.
|
||||
@@ -147,6 +171,63 @@ update_history_with_config(
|
||||
- Windows OS (MetaTrader 5 requirement)
|
||||
- MetaTrader 5 platform installed
|
||||
|
||||
### Migration note for mteor
|
||||
|
||||
Replace local MT5 lifecycle and trading helper code with mt5cli imports:
|
||||
|
||||
```python
|
||||
# Before (local mteor helpers)
|
||||
# with local_mt5_trading_session(config) as client:
|
||||
# side = local_detect_position_side(client, symbol)
|
||||
# sizing = local_calculate_margin_and_volume(client, symbol, unit_ratio, preserved_ratio)
|
||||
# limits = local_determine_order_limits(client, symbol, side, sl_ratio, tp_ratio)
|
||||
|
||||
# After (mt5cli shared layer)
|
||||
from pdmt5 import Mt5Config
|
||||
from mt5cli import (
|
||||
calculate_margin_and_volume,
|
||||
detect_position_side,
|
||||
determine_order_limits,
|
||||
mt5_trading_session,
|
||||
)
|
||||
|
||||
with mt5_trading_session(
|
||||
Mt5Config(path=terminal_path, login=login), retry_count=2
|
||||
) as client:
|
||||
side = detect_position_side(client, symbol)
|
||||
sizing = calculate_margin_and_volume(
|
||||
client, symbol, unit_margin_ratio=0.5, preserved_margin_ratio=0.2
|
||||
)
|
||||
if side is not None:
|
||||
limits = determine_order_limits(
|
||||
client,
|
||||
symbol,
|
||||
side,
|
||||
stop_loss_limit_ratio=0.01,
|
||||
take_profit_limit_ratio=0.02,
|
||||
)
|
||||
```
|
||||
|
||||
Throttled history updates use a separate read-only session:
|
||||
|
||||
```python
|
||||
from pdmt5 import Mt5Config, Mt5DataClient
|
||||
|
||||
from mt5cli import ThrottledHistoryUpdater
|
||||
|
||||
updater = ThrottledHistoryUpdater(
|
||||
output="history.db", interval_seconds=60, suppress_errors=True
|
||||
)
|
||||
client = Mt5DataClient(config=Mt5Config(login=login))
|
||||
client.initialize_and_login_mt5()
|
||||
try:
|
||||
updater.update(client, ["EURUSD"])
|
||||
finally:
|
||||
client.shutdown()
|
||||
```
|
||||
|
||||
Read-only collectors can keep using `mt5_session()` and `Mt5CliClient` without changes.
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
|
||||
@@ -215,3 +215,15 @@ frame = series["EURUSD", 1] # keyed by (symbol, integer timeframe)
|
||||
requires existing managed `rate_*` compatibility views and raises
|
||||
`ValueError` when they are missing. Duplicate `(symbol, timeframe)` targets
|
||||
are rejected.
|
||||
- `load_rate_series_by_granularity()` is a thin wrapper that builds the targets,
|
||||
loads the series, and rekeys the result by granularity name to avoid
|
||||
converting integer timeframes downstream:
|
||||
|
||||
```python
|
||||
from mt5cli import load_rate_series_by_granularity
|
||||
|
||||
series = load_rate_series_by_granularity(
|
||||
"history.db", ["EURUSD"], ["M1", "H1"], count=1000
|
||||
)
|
||||
frame = series["EURUSD", "M1"] # keyed by (symbol | None, granularity_name)
|
||||
```
|
||||
|
||||
+7
-2
@@ -18,6 +18,10 @@ Utility module providing constants, enums, Click parameter types, and helper fun
|
||||
|
||||
Programmatic SDK for read-only MetaTrader 5 data collection. Returns pandas DataFrames and provides `collect_history` for SQLite bulk collection.
|
||||
|
||||
### [Trading](trading.md)
|
||||
|
||||
Trading-capable session management and operational helpers built on `pdmt5.Mt5TradingClient`. Complements the read-only SDK without changing existing `Mt5CliClient` behavior.
|
||||
|
||||
### [History Collection (SQLite)](history.md)
|
||||
|
||||
SQLite storage helpers for the `collect-history` command schema, incremental updates, deduplication, indexes, and optional views.
|
||||
@@ -28,8 +32,9 @@ The package follows a simple architecture built on top of pdmt5:
|
||||
|
||||
1. **CLI Layer** (`cli.py`): Typer application with subcommands that delegate to the SDK and export results.
|
||||
2. **SDK Layer** (`sdk.py`): Read-only data access functions, `Mt5CliClient`, and `collect_history` orchestration.
|
||||
3. **Utils Layer** (`utils.py`): Constants, enums, custom Click parameter types, parsing helpers, and format detection/export utilities.
|
||||
4. **Data Layer** (via `pdmt5`): Uses `Mt5DataClient` and `Mt5Config` from the pdmt5 package for all MetaTrader 5 data access.
|
||||
3. **Trading Layer** (`trading.py`): Trading-capable sessions and operational helpers on `Mt5TradingClient`.
|
||||
4. **Utils Layer** (`utils.py`): Constants, enums, custom Click parameter types, parsing helpers, and format detection/export utilities.
|
||||
5. **Data Layer** (via `pdmt5`): Uses `Mt5DataClient`, `Mt5TradingClient`, and `Mt5Config` from the pdmt5 package for MetaTrader 5 access.
|
||||
|
||||
## Usage Guidelines
|
||||
|
||||
|
||||
+111
@@ -1,3 +1,114 @@
|
||||
# SDK Module
|
||||
|
||||
::: mt5cli.sdk
|
||||
|
||||
## Resilient multi-account orchestration
|
||||
|
||||
The SDK ships strategy-agnostic helpers for building long-running collectors on
|
||||
top of the read-only client. None of them depend on a particular trading
|
||||
application.
|
||||
|
||||
### Retrying transient rate collection
|
||||
|
||||
`collect_latest_rates_for_accounts_with_retries()` wraps
|
||||
`collect_latest_rates_for_accounts()` with bounded exponential backoff. Only
|
||||
`pdmt5.Mt5TradingError` and `pdmt5.Mt5RuntimeError` are retried; the final
|
||||
failure is re-raised once `retry_count` is exhausted.
|
||||
|
||||
```python
|
||||
from mt5cli import AccountSpec, collect_latest_rates_for_accounts_with_retries
|
||||
|
||||
accounts = [AccountSpec(symbols=["EURUSD"], login=12345)]
|
||||
rates = collect_latest_rates_for_accounts_with_retries(
|
||||
accounts,
|
||||
["M1", "H1"],
|
||||
count=500,
|
||||
retry_count=3,
|
||||
backoff_base=2, # sleeps 2s, 4s, 8s between attempts
|
||||
)
|
||||
```
|
||||
|
||||
### Latest closed rate bars
|
||||
|
||||
MetaTrader 5 `start_pos=0` includes the still-forming current bar as the last
|
||||
row. `collect_latest_closed_rates_for_accounts()` fetches `count + 1` bars,
|
||||
drops that row with `drop_forming_rate_bar()`, and validates each series is
|
||||
non-empty. Use `collect_latest_closed_rates_by_granularity()` when callers
|
||||
prefer keys such as `("EURUSD", "M1")` instead of integer timeframes.
|
||||
|
||||
```python
|
||||
from mt5cli import AccountSpec, collect_latest_closed_rates_by_granularity
|
||||
|
||||
rates = collect_latest_closed_rates_by_granularity(
|
||||
[AccountSpec(symbols=["EURUSD"], login=12345)],
|
||||
["M1", "H1"],
|
||||
count=500,
|
||||
retry_count=3,
|
||||
)
|
||||
closed_m1 = rates["EURUSD", "M1"]
|
||||
```
|
||||
|
||||
### Resolving credentials and `${ENV_VAR}` placeholders
|
||||
|
||||
`resolve_account_spec()` / `resolve_account_specs()` merge explicit override
|
||||
values over `AccountSpec` fields and expand `${ENV_VAR}` placeholders, keeping
|
||||
secrets out of plan/config files. A missing environment variable raises
|
||||
`ValueError`.
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
from mt5cli import AccountSpec, resolve_account_specs
|
||||
|
||||
os.environ["MT5_LOGIN"] = "12345"
|
||||
os.environ["MT5_PASSWORD"] = "secret"
|
||||
accounts = [
|
||||
AccountSpec(symbols=["EURUSD"], login="${MT5_LOGIN}", password="${MT5_PASSWORD}")
|
||||
]
|
||||
|
||||
resolved = resolve_account_specs(accounts, server="Broker-Demo")
|
||||
# resolved[0].login == "12345", resolved[0].server == "Broker-Demo"
|
||||
```
|
||||
|
||||
### Throttled incremental history updates
|
||||
|
||||
`ThrottledHistoryUpdater` wraps `update_history()` with a minimum interval
|
||||
between successful runs (using a monotonic clock), so an application loop can
|
||||
call it every iteration without over-fetching.
|
||||
|
||||
```python
|
||||
from pdmt5 import Mt5Config, Mt5DataClient
|
||||
|
||||
from mt5cli import Dataset, ThrottledHistoryUpdater
|
||||
|
||||
updater = ThrottledHistoryUpdater(
|
||||
output="history.db",
|
||||
datasets={Dataset.rates},
|
||||
timeframes=["M1"],
|
||||
interval_seconds=60, # <= 0 updates on every call
|
||||
)
|
||||
|
||||
client = Mt5DataClient(config=Mt5Config(login=12345))
|
||||
client.initialize_and_login_mt5()
|
||||
try:
|
||||
while True:
|
||||
updater.update(client, ["EURUSD", "GBPUSD"]) # no-op until 60s elapse
|
||||
# ... do other work; break when shutting down ...
|
||||
finally:
|
||||
client.shutdown()
|
||||
```
|
||||
|
||||
By default recoverable errors (`Mt5TradingError`, `Mt5RuntimeError`,
|
||||
`sqlite3.Error`, `ValueError`, `OSError`, and MT5 client capability
|
||||
`AttributeError` / `TypeError` for history API methods) propagate so the caller
|
||||
controls logging; pass `suppress_errors=True` to swallow them and return
|
||||
`False` without advancing the throttle. Other `AttributeError` / `TypeError`
|
||||
values always propagate. Input validation (`_resolve_update_history_request`)
|
||||
runs before any MT5 or SQLite calls, but when `suppress_errors=True` the
|
||||
resulting `ValueError` is suppressed along with other recoverable errors.
|
||||
|
||||
## Trading-capable sessions
|
||||
|
||||
For order placement and trading calculations, use the dedicated
|
||||
[Trading module](trading.md). The read-only `Mt5CliClient` and `mt5_session()`
|
||||
helpers in this module are unchanged.
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
# Trading Module
|
||||
|
||||
::: mt5cli.trading
|
||||
|
||||
## Trading-capable MT5 sessions
|
||||
|
||||
`mt5_trading_session()` complements the read-only `mt5_session()` helper in
|
||||
`sdk.py`. It yields a connected `pdmt5.Mt5TradingClient`, uses
|
||||
`Mt5Config.path` to launch the terminal when configured, and always calls
|
||||
`shutdown()` on exit.
|
||||
|
||||
```python
|
||||
from pdmt5 import Mt5Config
|
||||
|
||||
from mt5cli import mt5_trading_session
|
||||
|
||||
with mt5_trading_session(
|
||||
Mt5Config(path=r"C:\Program Files\MetaTrader 5\terminal64.exe", login=12345),
|
||||
retry_count=2,
|
||||
) as client:
|
||||
positions = client.positions_get_as_df(symbol="EURUSD")
|
||||
```
|
||||
|
||||
The read-only `Mt5CliClient` / `mt5_session()` API is unchanged.
|
||||
|
||||
## Operational trading helpers
|
||||
|
||||
These helpers are strategy-agnostic and do not depend on signal detection,
|
||||
betting logic, or scheduling code in downstream applications.
|
||||
|
||||
```python
|
||||
from mt5cli import (
|
||||
calculate_margin_and_volume,
|
||||
detect_position_side,
|
||||
determine_order_limits,
|
||||
)
|
||||
|
||||
side = detect_position_side(client, "EURUSD")
|
||||
sizing = calculate_margin_and_volume(
|
||||
client,
|
||||
"EURUSD",
|
||||
unit_margin_ratio=0.5,
|
||||
preserved_margin_ratio=0.2,
|
||||
)
|
||||
limits = determine_order_limits(
|
||||
client,
|
||||
"EURUSD",
|
||||
side="long",
|
||||
stop_loss_limit_ratio=0.01,
|
||||
take_profit_limit_ratio=0.02,
|
||||
)
|
||||
```
|
||||
|
||||
Protective ratios must satisfy `0 <= ratio < 1`; `0` omits that level.
|
||||
`calculate_margin_and_volume()` clamps negative `margin_free` to `0.0`
|
||||
before sizing.
|
||||
|
||||
## Migration from mteor-local helpers
|
||||
|
||||
| mteor-local concern | mt5cli replacement |
|
||||
| -------------------------------------------------------- | ----------------------------------------------- |
|
||||
| Manual terminal spawn/kill around trading code | `mt5_trading_session()` |
|
||||
| Local position-side detection | `detect_position_side()` |
|
||||
| Local margin/volume sizing | `calculate_margin_and_volume()` |
|
||||
| Local SL/TP price derivation | `determine_order_limits()` |
|
||||
| Throttled SQLite history loop with ad-hoc error handling | `ThrottledHistoryUpdater(suppress_errors=True)` |
|
||||
|
||||
Keep read-only data collection on `mt5_session()` / `Mt5CliClient`; use
|
||||
`mt5_trading_session()` only where order placement or trading calculations are
|
||||
required.
|
||||
@@ -6,6 +6,12 @@ Command-line tool for MetaTrader 5 data export.
|
||||
|
||||
mt5cli is a CLI application that exports MetaTrader 5 trading data to multiple file formats. It is built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data handler for MetaTrader 5.
|
||||
|
||||
## Architecture
|
||||
|
||||
- **pdmt5** — canonical MT5 client, DataFrame/trading primitives, and MT5 constant parsing (`TIMEFRAME_*`, `COPY_TICKS_*`, order types).
|
||||
- **mt5cli** — CLI commands, CSV/JSON/Parquet/SQLite export, SQLite history collection, rate views, and local batch/automation SDK helpers built on pdmt5.
|
||||
- **mt5api** — sibling HTTP adapter for remote MT5 access; not a dependency of mt5cli.
|
||||
|
||||
## Features
|
||||
|
||||
- **Multi-format export**: CSV, JSON, Parquet, and SQLite3 output formats
|
||||
|
||||
@@ -58,6 +58,7 @@ nav:
|
||||
- Overview: api/index.md
|
||||
- CLI: api/cli.md
|
||||
- SDK: api/sdk.md
|
||||
- Trading: api/trading.md
|
||||
- History Collection (SQLite): api/history.md
|
||||
- Utils: api/utils.md
|
||||
|
||||
|
||||
@@ -6,8 +6,10 @@ from .history import (
|
||||
RateTarget,
|
||||
build_rate_targets,
|
||||
build_rate_view_name,
|
||||
drop_forming_rate_bar,
|
||||
load_rate_data,
|
||||
load_rate_data_from_connection,
|
||||
load_rate_series_by_granularity,
|
||||
load_rate_series_from_sqlite,
|
||||
resolve_history_datasets,
|
||||
resolve_history_tick_flags,
|
||||
@@ -19,11 +21,15 @@ from .history import (
|
||||
from .sdk import (
|
||||
AccountSpec,
|
||||
Mt5CliClient,
|
||||
ThrottledHistoryUpdater,
|
||||
account_info,
|
||||
build_config,
|
||||
collect_history,
|
||||
collect_latest_closed_rates_by_granularity,
|
||||
collect_latest_closed_rates_for_accounts,
|
||||
collect_latest_rates,
|
||||
collect_latest_rates_for_accounts,
|
||||
collect_latest_rates_for_accounts_with_retries,
|
||||
copy_rates_from,
|
||||
copy_rates_from_pos,
|
||||
copy_rates_range,
|
||||
@@ -42,6 +48,9 @@ from .sdk import (
|
||||
positions,
|
||||
recent_history_deals,
|
||||
recent_ticks,
|
||||
resolve_account_spec,
|
||||
resolve_account_specs,
|
||||
substitute_env_placeholders,
|
||||
symbol_info,
|
||||
symbol_info_tick,
|
||||
symbols,
|
||||
@@ -52,6 +61,12 @@ from .sdk import (
|
||||
from .sdk import (
|
||||
version as mt5_version,
|
||||
)
|
||||
from .trading import (
|
||||
calculate_margin_and_volume,
|
||||
detect_position_side,
|
||||
determine_order_limits,
|
||||
mt5_trading_session,
|
||||
)
|
||||
from .utils import (
|
||||
TICK_FLAG_MAP,
|
||||
TIMEFRAME_MAP,
|
||||
@@ -75,19 +90,27 @@ __all__ = [
|
||||
"IfExists",
|
||||
"Mt5CliClient",
|
||||
"RateTarget",
|
||||
"ThrottledHistoryUpdater",
|
||||
"account_info",
|
||||
"build_config",
|
||||
"build_rate_targets",
|
||||
"build_rate_view_name",
|
||||
"calculate_margin_and_volume",
|
||||
"collect_history",
|
||||
"collect_latest_closed_rates_by_granularity",
|
||||
"collect_latest_closed_rates_for_accounts",
|
||||
"collect_latest_rates",
|
||||
"collect_latest_rates_for_accounts",
|
||||
"collect_latest_rates_for_accounts_with_retries",
|
||||
"copy_rates_from",
|
||||
"copy_rates_from_pos",
|
||||
"copy_rates_range",
|
||||
"copy_ticks_from",
|
||||
"copy_ticks_range",
|
||||
"detect_format",
|
||||
"detect_position_side",
|
||||
"determine_order_limits",
|
||||
"drop_forming_rate_bar",
|
||||
"export_dataframe",
|
||||
"export_dataframe_to_sqlite",
|
||||
"history_deals",
|
||||
@@ -96,12 +119,14 @@ __all__ = [
|
||||
"latest_rates",
|
||||
"load_rate_data",
|
||||
"load_rate_data_from_connection",
|
||||
"load_rate_series_by_granularity",
|
||||
"load_rate_series_from_sqlite",
|
||||
"market_book",
|
||||
"minimum_margins",
|
||||
"mt5_session",
|
||||
"mt5_summary",
|
||||
"mt5_summary_as_df",
|
||||
"mt5_trading_session",
|
||||
"mt5_version",
|
||||
"orders",
|
||||
"parse_datetime",
|
||||
@@ -110,12 +135,15 @@ __all__ = [
|
||||
"positions",
|
||||
"recent_history_deals",
|
||||
"recent_ticks",
|
||||
"resolve_account_spec",
|
||||
"resolve_account_specs",
|
||||
"resolve_history_datasets",
|
||||
"resolve_history_tick_flags",
|
||||
"resolve_history_timeframes",
|
||||
"resolve_rate_tables",
|
||||
"resolve_rate_view_name",
|
||||
"resolve_rate_view_names",
|
||||
"substitute_env_placeholders",
|
||||
"symbol_info",
|
||||
"symbol_info_tick",
|
||||
"symbols",
|
||||
|
||||
+2
-2
@@ -347,7 +347,7 @@ def ticks_recent(
|
||||
click_type=TICK_FLAGS_TYPE,
|
||||
help="Tick flags (ALL, INFO, TRADE, or integer).",
|
||||
),
|
||||
] = 1,
|
||||
] = "ALL", # pyright: ignore[reportArgumentType]
|
||||
) -> None:
|
||||
"""Export ticks from a recent time window."""
|
||||
client = _sdk_client(ctx)
|
||||
@@ -656,7 +656,7 @@ def collect_history(
|
||||
click_type=TICK_FLAGS_TYPE,
|
||||
help="Tick copy flags (ALL, INFO, TRADE, or integer).",
|
||||
),
|
||||
] = 1,
|
||||
] = "ALL", # pyright: ignore[reportArgumentType]
|
||||
if_exists: Annotated[
|
||||
IfExists,
|
||||
typer.Option(
|
||||
|
||||
+75
-9
@@ -10,9 +10,10 @@ from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Literal, cast
|
||||
|
||||
import pandas as pd
|
||||
from pdmt5 import get_timeframe_name as _get_timeframe_name
|
||||
|
||||
from .utils import (
|
||||
TIMEFRAME_MAP,
|
||||
TIMEFRAME_NAMES,
|
||||
Dataset,
|
||||
IfExists,
|
||||
parse_datetime,
|
||||
@@ -27,7 +28,7 @@ if TYPE_CHECKING:
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
DEFAULT_HISTORY_TIMEFRAMES: tuple[str, ...] = tuple(TIMEFRAME_MAP)
|
||||
DEFAULT_HISTORY_TIMEFRAMES: tuple[str, ...] = TIMEFRAME_NAMES
|
||||
|
||||
_HISTORY_DEDUP_KEYS: dict[Dataset, tuple[tuple[str, ...], ...]] = {
|
||||
Dataset.rates: (("symbol", "timeframe", "time"), ("symbol", "time")),
|
||||
@@ -80,7 +81,7 @@ def resolve_history_timeframes(
|
||||
seen: set[int] = set()
|
||||
resolved: list[int] = []
|
||||
for value in raw:
|
||||
tf = value if isinstance(value, int) else parse_timeframe(str(value))
|
||||
tf = parse_timeframe(value)
|
||||
if tf not in seen:
|
||||
seen.add(tf)
|
||||
resolved.append(tf)
|
||||
@@ -93,17 +94,33 @@ def resolve_history_tick_flags(flags: int | str) -> int:
|
||||
Returns:
|
||||
Integer tick flag value.
|
||||
"""
|
||||
if isinstance(flags, int):
|
||||
return flags
|
||||
return parse_tick_flags(flags)
|
||||
|
||||
|
||||
def resolve_granularity_name(timeframe: int) -> str:
|
||||
"""Return a granularity name for a timeframe integer when known."""
|
||||
for name, value in TIMEFRAME_MAP.items():
|
||||
if value == timeframe:
|
||||
return name
|
||||
return str(timeframe)
|
||||
try:
|
||||
name = _get_timeframe_name(timeframe)
|
||||
except ValueError:
|
||||
return str(timeframe)
|
||||
return name.removeprefix("TIMEFRAME_")
|
||||
|
||||
|
||||
def drop_forming_rate_bar(df_rate: pd.DataFrame) -> pd.DataFrame:
|
||||
"""Return closed bars from chronologically ordered MT5 rate data.
|
||||
|
||||
MetaTrader 5 ``copy_rates_from_pos(start_pos=0)`` includes the still-forming
|
||||
current bar as the last row. Slice it off so downstream logic only sees
|
||||
completed bars. Empty frames and single-row frames return empty results.
|
||||
|
||||
Args:
|
||||
df_rate: Rate data ordered oldest-to-newest with the forming bar last.
|
||||
|
||||
Returns:
|
||||
A new DataFrame with all rows except the last. Index and columns are
|
||||
preserved. The input frame is not modified.
|
||||
"""
|
||||
return df_rate.iloc[:-1].copy()
|
||||
|
||||
|
||||
def build_rate_view_name(
|
||||
@@ -706,6 +723,55 @@ def load_rate_series_from_sqlite(
|
||||
conn.close()
|
||||
|
||||
|
||||
def load_rate_series_by_granularity(
|
||||
conn_or_path: SqliteConnOrPath,
|
||||
symbols: Sequence[str],
|
||||
granularities: Sequence[int | str],
|
||||
count: int,
|
||||
*,
|
||||
explicit_tables: Sequence[str] | None = None,
|
||||
allow_missing_symbol: bool = False,
|
||||
) -> dict[tuple[str | None, str], pd.DataFrame]:
|
||||
"""Load rate series keyed by symbol and string granularity name.
|
||||
|
||||
Builds targets with :func:`build_rate_targets` and loads them with
|
||||
:func:`load_rate_series_from_sqlite`, then rekeys the result by granularity
|
||||
name (for example ``M1``) instead of the integer timeframe to reduce
|
||||
downstream boilerplate.
|
||||
|
||||
Args:
|
||||
conn_or_path: SQLite database path or open connection.
|
||||
symbols: MT5 symbol names. May be empty when ``allow_missing_symbol``.
|
||||
granularities: MT5 timeframes as integers or names (for example ``M1``).
|
||||
count: Number of most recent rows to load per series.
|
||||
explicit_tables: Optional explicit table or view names matching the
|
||||
built targets in row-major order. Required when symbols are omitted.
|
||||
allow_missing_symbol: When True and ``symbols`` is empty, build targets
|
||||
with ``symbol=None`` for each granularity instead of raising.
|
||||
|
||||
Returns:
|
||||
Mapping keyed by ``(symbol | None, granularity_name)`` to each rate
|
||||
DataFrame. Propagates ``ValueError`` (via :func:`build_rate_targets` and
|
||||
:func:`load_rate_series_from_sqlite`) when inputs are empty or invalid,
|
||||
table resolution fails, or duplicate targets are present.
|
||||
"""
|
||||
targets = build_rate_targets(
|
||||
symbols,
|
||||
granularities,
|
||||
allow_missing_symbol=allow_missing_symbol,
|
||||
)
|
||||
series = load_rate_series_from_sqlite(
|
||||
conn_or_path,
|
||||
targets,
|
||||
count,
|
||||
explicit_tables=explicit_tables,
|
||||
)
|
||||
return {
|
||||
(symbol, resolve_granularity_name(timeframe)): frame
|
||||
for (symbol, timeframe), frame in series.items()
|
||||
}
|
||||
|
||||
|
||||
def get_table_columns(conn: sqlite3.Connection, table: str) -> set[str]:
|
||||
"""Return existing SQLite columns for a table."""
|
||||
quoted_table = quote_sqlite_identifier(table)
|
||||
|
||||
+519
-7
@@ -4,7 +4,10 @@ from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import sqlite3
|
||||
import time
|
||||
from contextlib import contextmanager
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import UTC, datetime, timedelta
|
||||
@@ -12,12 +15,14 @@ from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Self, TypeVar, cast
|
||||
|
||||
import pandas as pd
|
||||
from pdmt5 import Mt5Config, Mt5DataClient
|
||||
from pdmt5 import Mt5Config, Mt5DataClient, Mt5RuntimeError, Mt5TradingError
|
||||
|
||||
from .history import (
|
||||
create_cash_events_view,
|
||||
create_history_indexes,
|
||||
create_positions_reconstructed_view,
|
||||
drop_forming_rate_bar,
|
||||
resolve_granularity_name,
|
||||
resolve_history_datasets,
|
||||
resolve_history_tick_flags,
|
||||
resolve_history_timeframes,
|
||||
@@ -39,14 +44,74 @@ T = TypeVar("T")
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_RECOVERABLE_HISTORY_UPDATE_ERRORS: tuple[type[BaseException], ...] = (
|
||||
Mt5TradingError,
|
||||
Mt5RuntimeError,
|
||||
sqlite3.Error,
|
||||
ValueError,
|
||||
OSError,
|
||||
)
|
||||
|
||||
_MT5_CLIENT_CAPABILITY_METHODS: frozenset[str] = frozenset({
|
||||
"copy_rates_range_as_df",
|
||||
"copy_ticks_range_as_df",
|
||||
"history_deals_get_as_df",
|
||||
"history_orders_get_as_df",
|
||||
})
|
||||
_MT5_HISTORY_MODULE = Path(__file__).with_name("history.py").resolve()
|
||||
_MT5_HISTORY_CLIENT_CALL_FUNCTIONS: frozenset[str] = frozenset({
|
||||
"write_rates_dataset",
|
||||
"write_ticks_dataset",
|
||||
"write_history_dataset",
|
||||
"_write_incremental_history_deals",
|
||||
})
|
||||
_NON_CALLABLE_TYPE_ERROR = re.compile(r"^'[^']+' object is not callable$")
|
||||
|
||||
|
||||
def _is_non_callable_history_client_type_error(exc: TypeError) -> bool:
|
||||
"""Return whether a TypeError came from calling a history client API attribute."""
|
||||
if not _NON_CALLABLE_TYPE_ERROR.match(str(exc)):
|
||||
return False
|
||||
tb = exc.__traceback__
|
||||
if tb is None:
|
||||
return False
|
||||
while tb.tb_next is not None:
|
||||
tb = tb.tb_next
|
||||
frame = tb.tb_frame
|
||||
return (
|
||||
frame.f_code.co_name in _MT5_HISTORY_CLIENT_CALL_FUNCTIONS
|
||||
and Path(frame.f_code.co_filename).resolve() == _MT5_HISTORY_MODULE
|
||||
)
|
||||
|
||||
|
||||
def _is_mt5_client_capability_error(exc: BaseException) -> bool:
|
||||
"""Return whether an error indicates an incompatible MT5 client API surface."""
|
||||
if isinstance(exc, AttributeError):
|
||||
msg = str(exc)
|
||||
if msg.startswith("MT5 client is missing required method:"):
|
||||
return True
|
||||
name = getattr(exc, "name", None)
|
||||
return isinstance(name, str) and name in _MT5_CLIENT_CAPABILITY_METHODS
|
||||
if isinstance(exc, TypeError):
|
||||
msg = str(exc)
|
||||
if msg.startswith("MT5 client attribute is not callable:"):
|
||||
return True
|
||||
return _is_non_callable_history_client_type_error(exc)
|
||||
return False
|
||||
|
||||
|
||||
__all__ = [
|
||||
"AccountSpec",
|
||||
"Mt5CliClient",
|
||||
"ThrottledHistoryUpdater",
|
||||
"account_info",
|
||||
"build_config",
|
||||
"collect_history",
|
||||
"collect_latest_closed_rates_by_granularity",
|
||||
"collect_latest_closed_rates_for_accounts",
|
||||
"collect_latest_rates",
|
||||
"collect_latest_rates_for_accounts",
|
||||
"collect_latest_rates_for_accounts_with_retries",
|
||||
"copy_rates_from",
|
||||
"copy_rates_from_pos",
|
||||
"copy_rates_range",
|
||||
@@ -65,6 +130,9 @@ __all__ = [
|
||||
"positions",
|
||||
"recent_history_deals",
|
||||
"recent_ticks",
|
||||
"resolve_account_spec",
|
||||
"resolve_account_specs",
|
||||
"substitute_env_placeholders",
|
||||
"symbol_info",
|
||||
"symbol_info_tick",
|
||||
"symbols",
|
||||
@@ -76,14 +144,10 @@ __all__ = [
|
||||
|
||||
|
||||
def _coerce_timeframe(timeframe: int | str) -> int:
|
||||
if isinstance(timeframe, int):
|
||||
return timeframe
|
||||
return parse_timeframe(timeframe)
|
||||
|
||||
|
||||
def _coerce_tick_flags(flags: int | str) -> int:
|
||||
if isinstance(flags, int):
|
||||
return flags
|
||||
return parse_tick_flags(flags)
|
||||
|
||||
|
||||
@@ -121,6 +185,12 @@ def _require_positive(value: float, name: str) -> None:
|
||||
raise ValueError(msg)
|
||||
|
||||
|
||||
def _require_non_negative(value: int, name: str) -> None:
|
||||
if value < 0:
|
||||
msg = f"{name} must be non-negative."
|
||||
raise ValueError(msg)
|
||||
|
||||
|
||||
def _call_required_client_method(client: Mt5DataClient, name: str) -> object:
|
||||
try:
|
||||
method = getattr(client, name)
|
||||
@@ -960,6 +1030,135 @@ def update_history_with_config( # noqa: PLR0913
|
||||
)
|
||||
|
||||
|
||||
class ThrottledHistoryUpdater:
|
||||
"""Throttled incremental SQLite history updater for long-running apps.
|
||||
|
||||
Wraps :func:`update_history` with a minimum interval between successful
|
||||
updates, so a tight application loop can call :meth:`update` every
|
||||
iteration without re-fetching MT5 history more often than desired. Timing
|
||||
uses a monotonic clock, so it is unaffected by wall-clock changes.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
output: Path | str,
|
||||
datasets: set[Dataset] | None = None,
|
||||
timeframes: Sequence[int | str] | None = None,
|
||||
flags: int | str = "ALL",
|
||||
lookback_hours: float = 24.0,
|
||||
with_views: bool = False,
|
||||
include_account_events: bool = True,
|
||||
interval_seconds: float = 0.0,
|
||||
suppress_errors: bool = False,
|
||||
) -> None:
|
||||
"""Initialize the throttled updater.
|
||||
|
||||
Args:
|
||||
output: SQLite database path.
|
||||
datasets: Datasets to include (defaults to all).
|
||||
timeframes: Rate timeframes to update (defaults to all fixed MT5
|
||||
timeframes).
|
||||
flags: Tick copy flags as integer or name (e.g. ``ALL``).
|
||||
lookback_hours: First-run lookback when a table has no prior rows.
|
||||
with_views: Create ``cash_events`` and ``positions_reconstructed``
|
||||
views.
|
||||
include_account_events: Include account-level cash events.
|
||||
interval_seconds: Minimum seconds between successful updates. Values
|
||||
``<= 0`` update on every call.
|
||||
suppress_errors: When True, recoverable errors (``Mt5TradingError``,
|
||||
``Mt5RuntimeError``, ``sqlite3.Error``, ``ValueError``,
|
||||
``OSError``, and MT5 client capability ``AttributeError`` /
|
||||
``TypeError`` for history API methods) raised during an update
|
||||
are swallowed and :meth:`update` returns False without advancing
|
||||
the throttle. Other ``AttributeError`` / ``TypeError`` values
|
||||
always propagate. When False (default), recoverable errors
|
||||
propagate so callers control logging.
|
||||
"""
|
||||
self.output = output
|
||||
self.datasets = datasets
|
||||
self.timeframes = timeframes
|
||||
self.flags = flags
|
||||
self.lookback_hours = lookback_hours
|
||||
self.with_views = with_views
|
||||
self.include_account_events = include_account_events
|
||||
self.interval_seconds = interval_seconds
|
||||
self.suppress_errors = suppress_errors
|
||||
self._last_update_monotonic: float | None = None
|
||||
|
||||
@property
|
||||
def last_update_monotonic(self) -> float | None:
|
||||
"""Return the monotonic timestamp of the last successful update."""
|
||||
return self._last_update_monotonic
|
||||
|
||||
def should_update(self) -> bool:
|
||||
"""Return whether enough time has elapsed to run another update.
|
||||
|
||||
Returns:
|
||||
True when ``interval_seconds <= 0``, when no update has succeeded
|
||||
yet, or when at least ``interval_seconds`` have elapsed since the
|
||||
last successful update.
|
||||
"""
|
||||
if self.interval_seconds <= 0 or self._last_update_monotonic is None:
|
||||
return True
|
||||
return (time.monotonic() - self._last_update_monotonic) >= self.interval_seconds
|
||||
|
||||
def update(self, client: Mt5DataClient, symbols: Sequence[str]) -> bool:
|
||||
"""Run a throttled incremental history update.
|
||||
|
||||
Args:
|
||||
client: Connected MT5 data client.
|
||||
symbols: Symbols to update.
|
||||
|
||||
Returns:
|
||||
True if an update ran successfully, False if it was throttled or
|
||||
(when ``suppress_errors`` is True) failed with a recoverable error.
|
||||
When ``suppress_errors`` is False, recoverable update failures
|
||||
propagate to the caller.
|
||||
|
||||
Raises:
|
||||
AttributeError: MT5 client capability mismatch when
|
||||
``suppress_errors`` is False, or any other attribute error.
|
||||
TypeError: MT5 client capability mismatch when ``suppress_errors``
|
||||
is False, or any other type error.
|
||||
"""
|
||||
if not self.should_update():
|
||||
return False
|
||||
try:
|
||||
_resolve_update_history_request(
|
||||
output=self.output,
|
||||
symbols=symbols,
|
||||
datasets=self.datasets,
|
||||
timeframes=self.timeframes,
|
||||
flags=self.flags,
|
||||
lookback_hours=self.lookback_hours,
|
||||
date_to=None,
|
||||
)
|
||||
update_history(
|
||||
client=client,
|
||||
output=self.output,
|
||||
symbols=symbols,
|
||||
datasets=self.datasets,
|
||||
timeframes=self.timeframes,
|
||||
flags=self.flags,
|
||||
lookback_hours=self.lookback_hours,
|
||||
with_views=self.with_views,
|
||||
include_account_events=self.include_account_events,
|
||||
)
|
||||
except _RECOVERABLE_HISTORY_UPDATE_ERRORS:
|
||||
if self.suppress_errors:
|
||||
logger.warning("Suppressed history update error", exc_info=True)
|
||||
return False
|
||||
raise
|
||||
except (AttributeError, TypeError) as exc:
|
||||
if self.suppress_errors and _is_mt5_client_capability_error(exc):
|
||||
logger.warning("Suppressed history update error", exc_info=True)
|
||||
return False
|
||||
raise
|
||||
self._last_update_monotonic = time.monotonic()
|
||||
return True
|
||||
|
||||
|
||||
def collect_history(
|
||||
output: Path,
|
||||
symbols: list[str],
|
||||
@@ -968,7 +1167,7 @@ def collect_history(
|
||||
*,
|
||||
datasets: set[Dataset] | None = None,
|
||||
timeframe: int | str = 1,
|
||||
flags: int | str = 1,
|
||||
flags: int | str = "ALL",
|
||||
if_exists: IfExists = IfExists.FAIL,
|
||||
with_views: bool = False,
|
||||
config: Mt5Config | None = None,
|
||||
@@ -1113,13 +1312,155 @@ class AccountSpec:
|
||||
"""
|
||||
|
||||
symbols: Sequence[str]
|
||||
login: int | str | None = None
|
||||
login: int | str | None = field(default=None, repr=False)
|
||||
password: str | None = field(default=None, repr=False)
|
||||
server: str | None = None
|
||||
path: str | None = None
|
||||
timeout: int | None = None
|
||||
|
||||
|
||||
_ENV_PLACEHOLDER_PATTERN = re.compile(r"\$\{(?P<name>[A-Za-z_][A-Za-z0-9_]*)\}")
|
||||
|
||||
|
||||
def substitute_env_placeholders(value: str) -> str:
|
||||
"""Replace ``${ENV_VAR}`` placeholders in a string with environment values.
|
||||
|
||||
Args:
|
||||
value: String that may contain one or more ``${ENV_VAR}`` placeholders.
|
||||
|
||||
Returns:
|
||||
The string with every placeholder replaced by its environment value.
|
||||
|
||||
Raises:
|
||||
ValueError: If a referenced environment variable is not set.
|
||||
"""
|
||||
parts: list[str] = []
|
||||
last_end = 0
|
||||
for match in _ENV_PLACEHOLDER_PATTERN.finditer(value):
|
||||
parts.append(value[last_end : match.start()])
|
||||
name = match.group("name")
|
||||
if name not in os.environ:
|
||||
msg = f"Environment variable {name!r} is not set."
|
||||
raise ValueError(msg)
|
||||
parts.append(os.environ[name])
|
||||
last_end = match.end()
|
||||
parts.append(value[last_end:])
|
||||
return "".join(parts)
|
||||
|
||||
|
||||
def _resolve_field(override: str | None, account_value: str | None) -> str | None:
|
||||
"""Resolve a string field from an override or account value with env subst.
|
||||
|
||||
Returns:
|
||||
The explicit override when provided, otherwise the account value, with
|
||||
any ``${ENV_VAR}`` placeholders substituted.
|
||||
"""
|
||||
value = override if override is not None else account_value
|
||||
if value is None:
|
||||
return None
|
||||
return substitute_env_placeholders(value)
|
||||
|
||||
|
||||
def _resolve_login(
|
||||
override: int | str | None,
|
||||
account_login: int | str | None,
|
||||
) -> int | str | None:
|
||||
"""Resolve a login from an override or account value with env substitution.
|
||||
|
||||
Returns:
|
||||
The explicit override when provided, otherwise the account login.
|
||||
Integer values are preserved; string values have ``${ENV_VAR}``
|
||||
placeholders substituted.
|
||||
"""
|
||||
if override is not None:
|
||||
if isinstance(override, int):
|
||||
return override
|
||||
return substitute_env_placeholders(override)
|
||||
if account_login is None or isinstance(account_login, int):
|
||||
return account_login
|
||||
return substitute_env_placeholders(account_login)
|
||||
|
||||
|
||||
def resolve_account_spec(
|
||||
account: AccountSpec,
|
||||
*,
|
||||
login: int | str | None = None,
|
||||
password: str | None = None,
|
||||
server: str | None = None,
|
||||
path: str | None = None,
|
||||
timeout: int | None = None,
|
||||
) -> AccountSpec:
|
||||
"""Resolve an account's credentials from overrides and ``${ENV_VAR}`` values.
|
||||
|
||||
Explicit override arguments take precedence over the corresponding
|
||||
:class:`AccountSpec` fields. The resolved string fields (``login``,
|
||||
``password``, ``server``, ``path``) have any ``${ENV_VAR}`` placeholders
|
||||
substituted from the environment.
|
||||
|
||||
Args:
|
||||
account: Source account specification.
|
||||
login: Optional explicit login override.
|
||||
password: Optional explicit password override.
|
||||
server: Optional explicit server override.
|
||||
path: Optional explicit terminal path override.
|
||||
timeout: Optional explicit connection timeout override.
|
||||
|
||||
Returns:
|
||||
A new :class:`AccountSpec` with resolved credentials and the original
|
||||
symbols preserved. Raises ``ValueError`` (via
|
||||
:func:`substitute_env_placeholders`) if a referenced environment
|
||||
variable is not set.
|
||||
"""
|
||||
return AccountSpec(
|
||||
symbols=account.symbols,
|
||||
login=_resolve_login(login, account.login),
|
||||
password=_resolve_field(password, account.password),
|
||||
server=_resolve_field(server, account.server),
|
||||
path=_resolve_field(path, account.path),
|
||||
timeout=timeout if timeout is not None else account.timeout,
|
||||
)
|
||||
|
||||
|
||||
def resolve_account_specs(
|
||||
accounts: Sequence[AccountSpec],
|
||||
*,
|
||||
login: int | str | None = None,
|
||||
password: str | None = None,
|
||||
server: str | None = None,
|
||||
path: str | None = None,
|
||||
timeout: int | None = None,
|
||||
) -> list[AccountSpec]:
|
||||
"""Resolve credentials for multiple accounts.
|
||||
|
||||
Applies the same overrides and ``${ENV_VAR}`` substitution as
|
||||
:func:`resolve_account_spec` to every account.
|
||||
|
||||
Args:
|
||||
accounts: Source account specifications.
|
||||
login: Optional explicit login override applied to each account.
|
||||
password: Optional explicit password override applied to each account.
|
||||
server: Optional explicit server override applied to each account.
|
||||
path: Optional explicit terminal path override applied to each account.
|
||||
timeout: Optional explicit timeout override applied to each account.
|
||||
|
||||
Returns:
|
||||
Resolved account specifications in the original order. Raises
|
||||
``ValueError`` (via :func:`substitute_env_placeholders`) if a referenced
|
||||
environment variable is not set.
|
||||
"""
|
||||
return [
|
||||
resolve_account_spec(
|
||||
account,
|
||||
login=login,
|
||||
password=password,
|
||||
server=server,
|
||||
path=path,
|
||||
timeout=timeout,
|
||||
)
|
||||
for account in accounts
|
||||
]
|
||||
|
||||
|
||||
def _coerce_login(login: int | str | None) -> int | None:
|
||||
"""Coerce a login value to int, treating empty strings as unset.
|
||||
|
||||
@@ -1212,6 +1553,177 @@ def collect_latest_rates_for_accounts(
|
||||
return result
|
||||
|
||||
|
||||
def collect_latest_rates_for_accounts_with_retries(
|
||||
accounts: Sequence[AccountSpec],
|
||||
timeframes: Sequence[int | str],
|
||||
count: int,
|
||||
*,
|
||||
start_pos: int = 0,
|
||||
base_config: Mt5Config | None = None,
|
||||
retry_count: int = 0,
|
||||
backoff_base: float = 2.0,
|
||||
) -> dict[tuple[str, int], pd.DataFrame]:
|
||||
"""Collect latest rates across accounts, retrying transient MT5 failures.
|
||||
|
||||
Wraps :func:`collect_latest_rates_for_accounts` with bounded exponential
|
||||
backoff. Only ``pdmt5.Mt5TradingError`` and ``pdmt5.Mt5RuntimeError`` are
|
||||
retried; other exceptions propagate immediately. The final failure is
|
||||
re-raised once retries are exhausted.
|
||||
|
||||
Args:
|
||||
accounts: Account groups to read. Each must define at least one symbol.
|
||||
timeframes: MT5 timeframes as integers or names (for example ``M1``).
|
||||
count: Number of most recent bars to read per symbol/timeframe.
|
||||
start_pos: Initial bar position offset.
|
||||
base_config: Optional base configuration whose fields fill any value not
|
||||
set on an individual account.
|
||||
retry_count: Maximum number of retries after the first attempt. ``0``
|
||||
disables retries.
|
||||
backoff_base: Base for exponential backoff. The delay before retry
|
||||
attempt ``n`` (1-indexed) is ``backoff_base ** n`` seconds.
|
||||
|
||||
Returns:
|
||||
Mapping keyed by ``(symbol, timeframe_int)``. Propagates ``ValueError``
|
||||
for invalid inputs (see :func:`collect_latest_rates_for_accounts`) and
|
||||
re-raises the last ``pdmt5.Mt5TradingError`` or ``pdmt5.Mt5RuntimeError``
|
||||
once retries are exhausted.
|
||||
"""
|
||||
attempts = max(retry_count, 0) + 1
|
||||
|
||||
def _collect() -> dict[tuple[str, int], pd.DataFrame]:
|
||||
return collect_latest_rates_for_accounts(
|
||||
accounts,
|
||||
timeframes,
|
||||
count,
|
||||
start_pos=start_pos,
|
||||
base_config=base_config,
|
||||
)
|
||||
|
||||
for attempt in range(attempts - 1):
|
||||
try:
|
||||
return _collect()
|
||||
except (Mt5TradingError, Mt5RuntimeError) as exc:
|
||||
delay = backoff_base ** (attempt + 1)
|
||||
logger.warning(
|
||||
"Rate collection failed (attempt %d/%d): %s; retrying in %.1fs",
|
||||
attempt + 1,
|
||||
attempts,
|
||||
exc,
|
||||
delay,
|
||||
)
|
||||
time.sleep(delay)
|
||||
return _collect()
|
||||
|
||||
|
||||
def collect_latest_closed_rates_for_accounts(
|
||||
accounts: Sequence[AccountSpec],
|
||||
timeframes: Sequence[int | str],
|
||||
count: int,
|
||||
*,
|
||||
start_pos: int = 0,
|
||||
base_config: Mt5Config | None = None,
|
||||
retry_count: int = 0,
|
||||
backoff_base: float = 2.0,
|
||||
) -> dict[tuple[str, int], pd.DataFrame]:
|
||||
"""Collect latest closed rate bars across multiple MT5 account groups.
|
||||
|
||||
When ``start_pos`` is ``0`` (the default), MetaTrader 5 includes the
|
||||
still-forming current bar as the last row. This helper fetches
|
||||
``count + 1`` bars, drops that bar with :func:`drop_forming_rate_bar`, and
|
||||
validates that each resulting frame is non-empty. When ``start_pos`` is
|
||||
greater than zero the forming bar is not in range, so only ``count`` bars
|
||||
are fetched and no row is dropped.
|
||||
|
||||
Wraps :func:`collect_latest_rates_for_accounts_with_retries` for transient
|
||||
MT5 error handling.
|
||||
|
||||
Args:
|
||||
accounts: Account groups to read. Each must define at least one symbol.
|
||||
timeframes: MT5 timeframes as integers or names (for example ``M1``).
|
||||
count: Number of closed bars to return per symbol/timeframe.
|
||||
start_pos: Initial bar position offset passed to the underlying collector.
|
||||
base_config: Optional base configuration whose fields fill any value not
|
||||
set on an individual account.
|
||||
retry_count: Maximum number of retries after the first attempt. ``0``
|
||||
disables retries.
|
||||
backoff_base: Base for exponential backoff between retry attempts.
|
||||
|
||||
Returns:
|
||||
Mapping keyed by ``(symbol, timeframe_int)``.
|
||||
|
||||
Raises:
|
||||
ValueError: If inputs are invalid, or any series is empty (after
|
||||
dropping the still-forming bar when ``start_pos`` is ``0``).
|
||||
"""
|
||||
_require_positive(count, "count")
|
||||
_require_non_negative(start_pos, "start_pos")
|
||||
fetch_count = count + 1 if start_pos == 0 else count
|
||||
loaded = collect_latest_rates_for_accounts_with_retries(
|
||||
accounts,
|
||||
timeframes,
|
||||
fetch_count,
|
||||
start_pos=start_pos,
|
||||
base_config=base_config,
|
||||
retry_count=retry_count,
|
||||
backoff_base=backoff_base,
|
||||
)
|
||||
result: dict[tuple[str, int], pd.DataFrame] = {}
|
||||
for key, df_rate in loaded.items():
|
||||
closed = drop_forming_rate_bar(df_rate) if start_pos == 0 else df_rate
|
||||
if closed.empty:
|
||||
symbol, timeframe = key
|
||||
msg = f"Rate data is empty for {symbol!r} at timeframe {timeframe}."
|
||||
raise ValueError(msg)
|
||||
result[key] = closed
|
||||
return result
|
||||
|
||||
|
||||
def collect_latest_closed_rates_by_granularity(
|
||||
accounts: Sequence[AccountSpec],
|
||||
granularities: Sequence[int | str],
|
||||
count: int,
|
||||
*,
|
||||
start_pos: int = 0,
|
||||
base_config: Mt5Config | None = None,
|
||||
retry_count: int = 0,
|
||||
backoff_base: float = 2.0,
|
||||
) -> dict[tuple[str, str], pd.DataFrame]:
|
||||
"""Collect latest closed rate bars keyed by symbol and granularity name.
|
||||
|
||||
Thin wrapper around :func:`collect_latest_closed_rates_for_accounts` that
|
||||
rekeys the result by granularity name (for example ``M1``) instead of the
|
||||
integer timeframe.
|
||||
|
||||
Args:
|
||||
accounts: Account groups to read. Each must define at least one symbol.
|
||||
granularities: MT5 timeframes as integers or names (for example ``M1``).
|
||||
count: Number of closed bars to return per symbol/timeframe.
|
||||
start_pos: Initial bar position offset passed to the underlying collector.
|
||||
base_config: Optional base configuration whose fields fill any value not
|
||||
set on an individual account.
|
||||
retry_count: Maximum number of retries after the first attempt. ``0``
|
||||
disables retries.
|
||||
backoff_base: Base for exponential backoff between retry attempts.
|
||||
|
||||
Returns:
|
||||
Mapping keyed by ``(symbol, granularity_name)``. Propagates
|
||||
``ValueError`` from :func:`collect_latest_closed_rates_for_accounts`.
|
||||
"""
|
||||
loaded = collect_latest_closed_rates_for_accounts(
|
||||
accounts,
|
||||
granularities,
|
||||
count,
|
||||
start_pos=start_pos,
|
||||
base_config=base_config,
|
||||
retry_count=retry_count,
|
||||
backoff_base=backoff_base,
|
||||
)
|
||||
return {
|
||||
(symbol, resolve_granularity_name(timeframe)): frame
|
||||
for (symbol, timeframe), frame in loaded.items()
|
||||
}
|
||||
|
||||
|
||||
def copy_rates_range(
|
||||
symbol: str,
|
||||
timeframe: int | str,
|
||||
|
||||
@@ -0,0 +1,210 @@
|
||||
"""Trading-capable MetaTrader 5 session helpers and operational utilities."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from contextlib import contextmanager
|
||||
from typing import TYPE_CHECKING, Literal
|
||||
|
||||
from pdmt5 import Mt5Config, Mt5TradingClient
|
||||
|
||||
from .sdk import build_config
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterator
|
||||
|
||||
import pandas as pd
|
||||
|
||||
PositionSide = Literal["long", "short"]
|
||||
OrderSide = Literal["long", "short"]
|
||||
|
||||
__all__ = [
|
||||
"OrderSide",
|
||||
"PositionSide",
|
||||
"calculate_margin_and_volume",
|
||||
"detect_position_side",
|
||||
"determine_order_limits",
|
||||
"mt5_trading_session",
|
||||
]
|
||||
|
||||
|
||||
def _require_unit_ratio(value: float, name: str) -> None:
|
||||
if not 0.0 <= value <= 1.0:
|
||||
msg = f"{name} must be between 0 and 1 inclusive."
|
||||
raise ValueError(msg)
|
||||
|
||||
|
||||
def _require_protective_ratio(value: float, name: str) -> None:
|
||||
if not 0.0 <= value < 1.0:
|
||||
msg = f"{name} must be at least 0 and less than 1."
|
||||
raise ValueError(msg)
|
||||
|
||||
|
||||
def _sum_position_volume(positions: pd.DataFrame, position_type: object) -> float:
|
||||
matched = positions.loc[positions["type"] == position_type, "volume"]
|
||||
if matched.empty:
|
||||
return 0.0
|
||||
return float(matched.to_numpy(dtype=float).sum())
|
||||
|
||||
|
||||
def _normalize_order_side(side: str) -> OrderSide:
|
||||
normalized = side.lower()
|
||||
if normalized in {"long", "buy"}:
|
||||
return "long"
|
||||
if normalized in {"short", "sell"}:
|
||||
return "short"
|
||||
msg = (
|
||||
f"Unsupported order side: {side!r}. Expected 'long', 'short', 'buy', or 'sell'."
|
||||
)
|
||||
raise ValueError(msg)
|
||||
|
||||
|
||||
def detect_position_side(
|
||||
client: Mt5TradingClient,
|
||||
symbol: str,
|
||||
) -> PositionSide | None:
|
||||
"""Detect the net open position side for a symbol.
|
||||
|
||||
Args:
|
||||
client: Connected ``Mt5TradingClient`` instance.
|
||||
symbol: Symbol to inspect.
|
||||
|
||||
Returns:
|
||||
``"long"`` when net buy volume exceeds sell volume, ``"short"`` when
|
||||
net sell volume exceeds buy volume, or ``None`` when no positions exist
|
||||
or buy/sell volumes are exactly balanced.
|
||||
"""
|
||||
positions = client.positions_get_as_df(symbol=symbol)
|
||||
if positions.empty:
|
||||
return None
|
||||
|
||||
buy_type = client.mt5.POSITION_TYPE_BUY
|
||||
sell_type = client.mt5.POSITION_TYPE_SELL
|
||||
buy_volume = _sum_position_volume(positions, buy_type)
|
||||
sell_volume = _sum_position_volume(positions, sell_type)
|
||||
net_volume = buy_volume - sell_volume
|
||||
if net_volume > 0:
|
||||
return "long"
|
||||
if net_volume < 0:
|
||||
return "short"
|
||||
return None
|
||||
|
||||
|
||||
def calculate_margin_and_volume(
|
||||
client: Mt5TradingClient,
|
||||
symbol: str,
|
||||
unit_margin_ratio: float,
|
||||
preserved_margin_ratio: float,
|
||||
) -> dict[str, float]:
|
||||
"""Calculate tradable margin and volumes from account free margin.
|
||||
|
||||
Applies ``preserved_margin_ratio`` to keep a reserve off ``margin_free``,
|
||||
then allocates ``unit_margin_ratio`` of the remainder as the margin budget
|
||||
for volume sizing on both buy and sell sides.
|
||||
|
||||
Args:
|
||||
client: Connected ``Mt5TradingClient`` instance.
|
||||
symbol: Symbol used for minimum-lot margin and volume calculations.
|
||||
unit_margin_ratio: Fraction of post-reserve margin to allocate per unit.
|
||||
preserved_margin_ratio: Fraction of ``margin_free`` to preserve.
|
||||
|
||||
Returns:
|
||||
Dictionary with ``margin_free``, ``available_margin``, ``trade_margin``,
|
||||
``buy_volume``, and ``sell_volume``. Negative ``margin_free`` values are
|
||||
clamped to ``0.0`` before sizing.
|
||||
"""
|
||||
_require_unit_ratio(unit_margin_ratio, "unit_margin_ratio")
|
||||
_require_unit_ratio(preserved_margin_ratio, "preserved_margin_ratio")
|
||||
|
||||
account = client.account_info_as_dict()
|
||||
margin_free = max(0.0, float(account.get("margin_free") or 0.0))
|
||||
available_margin = margin_free * (1.0 - preserved_margin_ratio)
|
||||
trade_margin = available_margin * unit_margin_ratio
|
||||
buy_volume = client.calculate_volume_by_margin(symbol, trade_margin, "BUY")
|
||||
sell_volume = client.calculate_volume_by_margin(symbol, trade_margin, "SELL")
|
||||
return {
|
||||
"margin_free": margin_free,
|
||||
"available_margin": available_margin,
|
||||
"trade_margin": trade_margin,
|
||||
"buy_volume": buy_volume,
|
||||
"sell_volume": sell_volume,
|
||||
}
|
||||
|
||||
|
||||
def determine_order_limits(
|
||||
client: Mt5TradingClient,
|
||||
symbol: str,
|
||||
side: OrderSide | str,
|
||||
stop_loss_limit_ratio: float,
|
||||
take_profit_limit_ratio: float,
|
||||
) -> dict[str, float | None]:
|
||||
"""Derive entry and protective order prices from current market quotes.
|
||||
|
||||
Args:
|
||||
client: Connected ``Mt5TradingClient`` instance.
|
||||
symbol: Symbol used for the quote lookup.
|
||||
side: Position side as ``"long"``/``"short"`` (``"buy"``/``"sell"``
|
||||
aliases are accepted).
|
||||
stop_loss_limit_ratio: Relative distance from entry for stop loss in
|
||||
``[0, 1)``. A value of ``0`` omits the stop loss.
|
||||
take_profit_limit_ratio: Relative distance from entry for take profit in
|
||||
``[0, 1)``. A value of ``0`` omits the take profit.
|
||||
|
||||
Returns:
|
||||
Dictionary with ``entry``, ``stop_loss``, and ``take_profit`` keys.
|
||||
Omitted protective levels are returned as ``None``.
|
||||
"""
|
||||
_require_protective_ratio(stop_loss_limit_ratio, "stop_loss_limit_ratio")
|
||||
_require_protective_ratio(take_profit_limit_ratio, "take_profit_limit_ratio")
|
||||
normalized_side = _normalize_order_side(side)
|
||||
tick = client.symbol_info_tick_as_dict(symbol=symbol)
|
||||
entry = float(tick["ask"] if normalized_side == "long" else tick["bid"])
|
||||
|
||||
stop_loss: float | None = None
|
||||
if stop_loss_limit_ratio > 0:
|
||||
if normalized_side == "long":
|
||||
stop_loss = entry * (1.0 - stop_loss_limit_ratio)
|
||||
else:
|
||||
stop_loss = entry * (1.0 + stop_loss_limit_ratio)
|
||||
|
||||
take_profit: float | None = None
|
||||
if take_profit_limit_ratio > 0:
|
||||
if normalized_side == "long":
|
||||
take_profit = entry * (1.0 + take_profit_limit_ratio)
|
||||
else:
|
||||
take_profit = entry * (1.0 - take_profit_limit_ratio)
|
||||
|
||||
return {
|
||||
"entry": entry,
|
||||
"stop_loss": stop_loss,
|
||||
"take_profit": take_profit,
|
||||
}
|
||||
|
||||
|
||||
@contextmanager
|
||||
def mt5_trading_session(
|
||||
config: Mt5Config | None = None,
|
||||
retry_count: int = 0,
|
||||
) -> Iterator[Mt5TradingClient]:
|
||||
"""Open a trading-capable MT5 session and always shut down safely.
|
||||
|
||||
Launches the MetaTrader 5 terminal using ``Mt5Config.path`` when set,
|
||||
initializes and logs in via ``initialize_and_login_mt5()``, yields a
|
||||
connected :class:`~pdmt5.Mt5TradingClient`, and calls ``shutdown()`` on
|
||||
exit even when an error is raised inside the context.
|
||||
|
||||
Args:
|
||||
config: MT5 connection configuration. Defaults to an empty config that
|
||||
attaches to a running terminal.
|
||||
retry_count: Number of initialization retries passed to
|
||||
``Mt5TradingClient``.
|
||||
|
||||
Yields:
|
||||
Connected ``Mt5TradingClient`` bound to the session.
|
||||
"""
|
||||
mt5_config = config or build_config()
|
||||
client = Mt5TradingClient(config=mt5_config, retry_count=retry_count)
|
||||
try:
|
||||
client.initialize_and_login_mt5()
|
||||
yield client
|
||||
finally:
|
||||
client.shutdown()
|
||||
+31
-50
@@ -10,6 +10,9 @@ from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any, TypeGuard
|
||||
|
||||
import click
|
||||
from pdmt5 import COPY_TICKS_MAP, TIMEFRAME_MAP
|
||||
from pdmt5 import parse_copy_ticks as _parse_copy_ticks
|
||||
from pdmt5 import parse_timeframe as _parse_timeframe
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Sequence
|
||||
@@ -20,35 +23,15 @@ if TYPE_CHECKING:
|
||||
# Constants
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
TIMEFRAME_MAP: dict[str, int] = {
|
||||
"M1": 1,
|
||||
"M2": 2,
|
||||
"M3": 3,
|
||||
"M4": 4,
|
||||
"M5": 5,
|
||||
"M6": 6,
|
||||
"M10": 10,
|
||||
"M12": 12,
|
||||
"M15": 15,
|
||||
"M20": 20,
|
||||
"M30": 30,
|
||||
"H1": 16385,
|
||||
"H2": 16386,
|
||||
"H3": 16387,
|
||||
"H4": 16388,
|
||||
"H6": 16390,
|
||||
"H8": 16392,
|
||||
"H12": 16396,
|
||||
"D1": 16408,
|
||||
"W1": 32769,
|
||||
"MN1": 49153,
|
||||
}
|
||||
# Backward-compatible snapshot; prefer ``COPY_TICKS_MAP`` from pdmt5 directly.
|
||||
TICK_FLAG_MAP: dict[str, int] = dict(COPY_TICKS_MAP)
|
||||
|
||||
TICK_FLAG_MAP: dict[str, int] = {
|
||||
"ALL": 1,
|
||||
"INFO": 2,
|
||||
"TRADE": 4,
|
||||
}
|
||||
TIMEFRAME_NAMES: tuple[str, ...] = tuple(
|
||||
name for name in TIMEFRAME_MAP if not name.startswith("TIMEFRAME_")
|
||||
)
|
||||
_TICK_FLAG_NAMES: tuple[str, ...] = tuple(
|
||||
name for name in COPY_TICKS_MAP if not name.startswith("COPY_TICKS_")
|
||||
)
|
||||
|
||||
_FORMAT_EXTENSIONS: dict[str, str] = {
|
||||
".csv": "csv",
|
||||
@@ -160,10 +143,8 @@ class _TimeframeType(click.ParamType):
|
||||
Returns:
|
||||
Integer timeframe value.
|
||||
"""
|
||||
if isinstance(value, int):
|
||||
return value
|
||||
try:
|
||||
return parse_timeframe(str(value))
|
||||
return parse_timeframe(value)
|
||||
except ValueError as exc:
|
||||
self.fail(str(exc), param, ctx)
|
||||
|
||||
@@ -189,10 +170,8 @@ class _TickFlagsType(click.ParamType):
|
||||
Returns:
|
||||
Integer tick flag value.
|
||||
"""
|
||||
if isinstance(value, int):
|
||||
return value
|
||||
try:
|
||||
return parse_tick_flags(str(value))
|
||||
return parse_tick_flags(value)
|
||||
except ValueError as exc:
|
||||
self.fail(str(exc), param, ctx)
|
||||
|
||||
@@ -370,7 +349,7 @@ def parse_datetime(value: str) -> datetime:
|
||||
return dt
|
||||
|
||||
|
||||
def parse_timeframe(value: str) -> int:
|
||||
def parse_timeframe(value: object) -> int:
|
||||
"""Parse a timeframe string or integer value.
|
||||
|
||||
Args:
|
||||
@@ -382,37 +361,39 @@ def parse_timeframe(value: str) -> int:
|
||||
Raises:
|
||||
ValueError: If the timeframe is invalid.
|
||||
"""
|
||||
upper = value.upper()
|
||||
if upper in TIMEFRAME_MAP:
|
||||
return TIMEFRAME_MAP[upper]
|
||||
try:
|
||||
return int(value)
|
||||
return _parse_timeframe(value)
|
||||
except ValueError:
|
||||
valid = ", ".join(TIMEFRAME_MAP)
|
||||
msg = f"Invalid timeframe: '{value}'. Use one of: {valid}, or an integer."
|
||||
display = value if isinstance(value, str) else repr(value)
|
||||
valid = ", ".join(TIMEFRAME_NAMES)
|
||||
msg = (
|
||||
f"Invalid timeframe: '{display}'. "
|
||||
f"Use one of: {valid}, or a supported integer."
|
||||
)
|
||||
raise ValueError(msg) from None
|
||||
|
||||
|
||||
def parse_tick_flags(value: str) -> int:
|
||||
def parse_tick_flags(value: object) -> int:
|
||||
"""Parse tick flags string or integer value.
|
||||
|
||||
Args:
|
||||
value: Tick flag name (ALL, INFO, TRADE) or integer value.
|
||||
value: Tick flag name (ALL, INFO, TRADE, COPY_TICKS_*) or integer value.
|
||||
|
||||
Returns:
|
||||
Integer tick flag value.
|
||||
Integer tick flag value compatible with MetaTrader 5 ``COPY_TICKS_*``.
|
||||
|
||||
Raises:
|
||||
ValueError: If the flag is invalid.
|
||||
"""
|
||||
upper = value.upper()
|
||||
if upper in TICK_FLAG_MAP:
|
||||
return TICK_FLAG_MAP[upper]
|
||||
try:
|
||||
return int(value)
|
||||
return _parse_copy_ticks(value)
|
||||
except ValueError:
|
||||
valid = ", ".join(TICK_FLAG_MAP)
|
||||
msg = f"Invalid tick flags: '{value}'. Use one of: {valid}, or an integer."
|
||||
display = value if isinstance(value, str) else repr(value)
|
||||
valid = ", ".join(_TICK_FLAG_NAMES)
|
||||
msg = (
|
||||
f"Invalid tick flags: '{display}'. "
|
||||
f"Use one of: {valid}, or a supported integer."
|
||||
)
|
||||
raise ValueError(msg) from None
|
||||
|
||||
|
||||
|
||||
+2
-2
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "mt5cli"
|
||||
version = "0.5.2"
|
||||
version = "0.7.0"
|
||||
description = "Command-line tool for MetaTrader 5"
|
||||
authors = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
|
||||
maintainers = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
|
||||
@@ -9,7 +9,7 @@ license-files = ["LICENSE"]
|
||||
readme = "README.md"
|
||||
requires-python = ">= 3.11, < 3.14"
|
||||
dependencies = [
|
||||
"pdmt5 >= 0.2.3",
|
||||
"pdmt5>=0.3.0",
|
||||
"click >= 8.1.0",
|
||||
"pyarrow >= 19.0.0",
|
||||
"typer >= 0.15.0",
|
||||
|
||||
+5
-5
@@ -317,7 +317,7 @@ class TestCommands:
|
||||
symbol="EURUSD",
|
||||
date_from=datetime(2024, 1, 1, tzinfo=UTC),
|
||||
count=100,
|
||||
flags=1,
|
||||
flags=-1,
|
||||
)
|
||||
|
||||
def test_ticks_range(
|
||||
@@ -348,7 +348,7 @@ class TestCommands:
|
||||
symbol="EURUSD",
|
||||
date_from=datetime(2024, 1, 1, tzinfo=UTC),
|
||||
date_to=datetime(2024, 2, 1, tzinfo=UTC),
|
||||
flags=2,
|
||||
flags=1,
|
||||
)
|
||||
|
||||
def test_ticks_recent(
|
||||
@@ -381,7 +381,7 @@ class TestCommands:
|
||||
symbol="EURUSD",
|
||||
date_from=datetime(2024, 1, 2, tzinfo=UTC) - timedelta(seconds=120),
|
||||
count=500,
|
||||
flags=1,
|
||||
flags=-1,
|
||||
)
|
||||
mock_client.copy_ticks_range_as_df.assert_not_called()
|
||||
|
||||
@@ -1000,7 +1000,7 @@ class TestCollectHistory:
|
||||
symbol="EURUSD",
|
||||
date_from=datetime(2024, 1, 1, tzinfo=UTC),
|
||||
date_to=datetime(2024, 2, 1, tzinfo=UTC),
|
||||
flags=1,
|
||||
flags=-1,
|
||||
)
|
||||
with sqlite3.connect(output) as conn:
|
||||
tables = {
|
||||
@@ -1213,7 +1213,7 @@ class TestCollectHistory:
|
||||
symbol="EURUSD",
|
||||
date_from=datetime(2024, 1, 1, tzinfo=UTC),
|
||||
date_to=datetime(2024, 2, 1, tzinfo=UTC),
|
||||
flags=1,
|
||||
flags=-1,
|
||||
)
|
||||
|
||||
def test_collect_history_with_views(
|
||||
|
||||
+109
-1
@@ -30,6 +30,7 @@ from mt5cli.history import (
|
||||
create_rate_compatibility_views,
|
||||
deduplicate_history_tables,
|
||||
drop_duplicates_in_table,
|
||||
drop_forming_rate_bar,
|
||||
filter_incremental_history_deals_frame,
|
||||
filter_trade_history_frame,
|
||||
get_history_deals_account_event_start_datetime,
|
||||
@@ -38,6 +39,7 @@ from mt5cli.history import (
|
||||
load_incremental_start_datetimes,
|
||||
load_rate_data,
|
||||
load_rate_data_from_connection,
|
||||
load_rate_series_by_granularity,
|
||||
load_rate_series_from_sqlite,
|
||||
parse_sqlite_timestamp,
|
||||
quote_sqlite_identifier,
|
||||
@@ -515,6 +517,9 @@ class TestResolveHistorySettings:
|
||||
"""Test default timeframes include all fixed MT5 values."""
|
||||
resolved = resolve_history_timeframes(None)
|
||||
assert len(resolved) == len(DEFAULT_HISTORY_TIMEFRAMES)
|
||||
assert not any(
|
||||
name.startswith("TIMEFRAME_") for name in DEFAULT_HISTORY_TIMEFRAMES
|
||||
)
|
||||
assert 1 in resolved
|
||||
assert TIMEFRAME_MAP["H1"] in resolved
|
||||
|
||||
@@ -524,7 +529,7 @@ class TestResolveHistorySettings:
|
||||
|
||||
def test_resolve_history_tick_flags(self) -> None:
|
||||
"""Test tick flag resolution."""
|
||||
assert resolve_history_tick_flags("ALL") == 1
|
||||
assert resolve_history_tick_flags("ALL") == -1
|
||||
assert resolve_history_tick_flags(2) == 2
|
||||
|
||||
def test_resolve_granularity_name_falls_back_to_integer(self) -> None:
|
||||
@@ -532,6 +537,57 @@ class TestResolveHistorySettings:
|
||||
assert resolve_granularity_name(999) == "999"
|
||||
assert resolve_granularity_name(1) == "M1"
|
||||
|
||||
def test_resolve_granularity_name_strips_official_prefix(
|
||||
self,
|
||||
mocker: MockerFixture,
|
||||
) -> None:
|
||||
"""Test official pdmt5 timeframe names are normalized to short aliases."""
|
||||
mocker.patch(
|
||||
"mt5cli.history._get_timeframe_name",
|
||||
return_value="TIMEFRAME_H1",
|
||||
)
|
||||
assert resolve_granularity_name(16385) == "H1"
|
||||
|
||||
|
||||
class TestDropFormingRateBar:
|
||||
"""Tests for drop_forming_rate_bar."""
|
||||
|
||||
def test_drops_still_forming_last_bar(self) -> None:
|
||||
"""Test the still-forming last bar is removed."""
|
||||
df_rate = pd.DataFrame(
|
||||
{"time": [1, 2, 3], "close": [1.1, 1.2, 1.3]},
|
||||
index=pd.Index(["a", "b", "c"], name="idx"),
|
||||
)
|
||||
|
||||
result = drop_forming_rate_bar(df_rate)
|
||||
|
||||
pd.testing.assert_frame_equal(
|
||||
result,
|
||||
pd.DataFrame(
|
||||
{"time": [1, 2], "close": [1.1, 1.2]},
|
||||
index=pd.Index(["a", "b"], name="idx"),
|
||||
),
|
||||
)
|
||||
assert df_rate.shape == (3, 2)
|
||||
|
||||
def test_returns_empty_frame_when_input_empty(self) -> None:
|
||||
"""Test empty frames stay empty."""
|
||||
df_rate = pd.DataFrame(columns=["time", "close"])
|
||||
|
||||
result = drop_forming_rate_bar(df_rate)
|
||||
|
||||
assert result.empty
|
||||
assert list(result.columns) == ["time", "close"]
|
||||
|
||||
def test_returns_empty_frame_when_only_forming_bar_present(self) -> None:
|
||||
"""Test a single-bar frame becomes empty after dropping the forming bar."""
|
||||
df_rate = pd.DataFrame({"time": [1], "close": [1.1]})
|
||||
|
||||
result = drop_forming_rate_bar(df_rate)
|
||||
|
||||
assert result.empty
|
||||
assert list(result.columns) == ["time", "close"]
|
||||
|
||||
|
||||
class TestParseSqliteTimestamp:
|
||||
"""Tests for parse_sqlite_timestamp."""
|
||||
@@ -1712,6 +1768,8 @@ class TestIncrementalIntegration:
|
||||
"""Test invalid tick flags raise ValueError."""
|
||||
with pytest.raises(ValueError, match="Invalid tick flags"):
|
||||
resolve_history_tick_flags("BAD")
|
||||
with pytest.raises(ValueError, match="Invalid tick flags"):
|
||||
resolve_history_tick_flags(7)
|
||||
|
||||
def test_resolve_history_timeframes_invalid(self) -> None:
|
||||
"""Test invalid timeframes raise ValueError."""
|
||||
@@ -2257,6 +2315,56 @@ class TestRateSourceHelpers:
|
||||
assert set(result) == {("EURUSD", 1)}
|
||||
assert len(result["EURUSD", 1]) == 2
|
||||
|
||||
def test_load_rate_series_by_granularity(self, tmp_path: Path) -> None:
|
||||
"""Test loading rate series keyed by symbol and granularity name."""
|
||||
db_path = tmp_path / "granularity.db"
|
||||
with sqlite3.connect(db_path) as conn:
|
||||
conn.execute(
|
||||
"CREATE TABLE rates("
|
||||
" symbol TEXT, timeframe INTEGER, time TEXT, close REAL)",
|
||||
)
|
||||
conn.executemany(
|
||||
"INSERT INTO rates(symbol, timeframe, time, close) VALUES (?, ?, ?, ?)",
|
||||
[
|
||||
("EURUSD", 1, "2024-01-01T00:00:00+00:00", 1.0),
|
||||
("EURUSD", 16385, "2024-01-01T00:00:00+00:00", 1.1),
|
||||
],
|
||||
)
|
||||
create_rate_compatibility_views(conn)
|
||||
|
||||
result = load_rate_series_by_granularity(
|
||||
db_path,
|
||||
["EURUSD"],
|
||||
["M1", "H1"],
|
||||
count=1,
|
||||
)
|
||||
|
||||
assert set(result) == {("EURUSD", "M1"), ("EURUSD", "H1")}
|
||||
|
||||
def test_load_rate_series_by_granularity_explicit_tables(
|
||||
self,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""Test explicit tables with None-symbol targets key by granularity."""
|
||||
db_path = tmp_path / "granularity-explicit.db"
|
||||
with sqlite3.connect(db_path) as conn:
|
||||
conn.execute("CREATE TABLE custom_view(time TEXT, close REAL)")
|
||||
conn.execute(
|
||||
"INSERT INTO custom_view(time, close) VALUES (?, ?)",
|
||||
("2024-01-01T00:00:00+00:00", 1.0),
|
||||
)
|
||||
|
||||
result = load_rate_series_by_granularity(
|
||||
db_path,
|
||||
[],
|
||||
["M1"],
|
||||
count=1,
|
||||
explicit_tables=["custom_view"],
|
||||
allow_missing_symbol=True,
|
||||
)
|
||||
|
||||
assert set(result) == {(None, "M1")}
|
||||
|
||||
def test_load_rate_series_reuses_path_connection(
|
||||
self,
|
||||
tmp_path: Path,
|
||||
|
||||
+651
-7
@@ -10,6 +10,7 @@ from unittest.mock import MagicMock, call
|
||||
|
||||
import pandas as pd
|
||||
import pytest
|
||||
from pdmt5 import Mt5RuntimeError, Mt5TradingError
|
||||
from pytest_mock import MockerFixture # noqa: TC002
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -18,15 +19,19 @@ if TYPE_CHECKING:
|
||||
from pdmt5 import Mt5Config, Mt5DataClient
|
||||
|
||||
from mt5cli import sdk
|
||||
from mt5cli.history import DEFAULT_HISTORY_TIMEFRAMES
|
||||
from mt5cli.history import DEFAULT_HISTORY_TIMEFRAMES, write_rates_dataset
|
||||
from mt5cli.sdk import (
|
||||
AccountSpec,
|
||||
Mt5CliClient,
|
||||
ThrottledHistoryUpdater,
|
||||
account_info,
|
||||
build_config,
|
||||
collect_history,
|
||||
collect_latest_closed_rates_by_granularity,
|
||||
collect_latest_closed_rates_for_accounts,
|
||||
collect_latest_rates,
|
||||
collect_latest_rates_for_accounts,
|
||||
collect_latest_rates_for_accounts_with_retries,
|
||||
copy_rates_from,
|
||||
copy_rates_from_pos,
|
||||
copy_rates_range,
|
||||
@@ -45,6 +50,9 @@ from mt5cli.sdk import (
|
||||
positions,
|
||||
recent_history_deals,
|
||||
recent_ticks,
|
||||
resolve_account_spec,
|
||||
resolve_account_specs,
|
||||
substitute_env_placeholders,
|
||||
symbol_info,
|
||||
symbol_info_tick,
|
||||
symbols,
|
||||
@@ -53,7 +61,7 @@ from mt5cli.sdk import (
|
||||
update_history_with_config,
|
||||
version,
|
||||
)
|
||||
from mt5cli.utils import Dataset
|
||||
from mt5cli.utils import Dataset, IfExists
|
||||
|
||||
|
||||
class _TerminalInfo(NamedTuple):
|
||||
@@ -382,7 +390,7 @@ class TestMt5CliClient:
|
||||
symbol="EURUSD",
|
||||
date_from=datetime(2024, 1, 1, tzinfo=UTC),
|
||||
count=100,
|
||||
flags=2,
|
||||
flags=1,
|
||||
)
|
||||
|
||||
def test_history_orders_accepts_string_dates(
|
||||
@@ -943,7 +951,7 @@ class TestUpdateHistory:
|
||||
assert kwargs["symbol"] == "EURUSD"
|
||||
assert kwargs["date_from"] == expected_start
|
||||
assert kwargs["date_to"] == date_to
|
||||
assert kwargs["flags"] == 1
|
||||
assert kwargs["flags"] == -1
|
||||
return pd.DataFrame({
|
||||
"time": ["2024-01-01T12:00:00+00:00"],
|
||||
"time_msc": [1_704_110_400_000],
|
||||
@@ -1111,7 +1119,7 @@ class TestRecentTicks:
|
||||
symbol="EURUSD",
|
||||
date_from=end - timedelta(seconds=60),
|
||||
count=100,
|
||||
flags=2,
|
||||
flags=1,
|
||||
)
|
||||
client.copy_ticks_range_as_df.assert_not_called()
|
||||
|
||||
@@ -1141,7 +1149,7 @@ class TestRecentTicks:
|
||||
assert kwargs["symbol"] == "EURUSD"
|
||||
assert kwargs["date_to"] == tick.time
|
||||
assert kwargs["date_from"] == tick.time - timedelta(seconds=30)
|
||||
assert kwargs["flags"] == 1
|
||||
assert kwargs["flags"] == -1
|
||||
|
||||
def test_recent_ticks_rejects_unsupported_tick_time(
|
||||
self,
|
||||
@@ -1211,7 +1219,7 @@ class TestRecentTicks:
|
||||
symbol="EURUSD",
|
||||
date_from=end - timedelta(seconds=60),
|
||||
date_to=end,
|
||||
flags=1,
|
||||
flags=-1,
|
||||
)
|
||||
|
||||
|
||||
@@ -1425,3 +1433,639 @@ class TestCollectLatestRatesForAccounts:
|
||||
collect_latest_rates_for_accounts(accounts, ["M1"], count=1)
|
||||
|
||||
mt5_data_client.assert_not_called()
|
||||
|
||||
|
||||
class TestCollectLatestRatesForAccountsWithRetries:
|
||||
"""Tests for collect_latest_rates_for_accounts_with_retries."""
|
||||
|
||||
def test_returns_result_on_first_success(self, mocker: MockerFixture) -> None:
|
||||
"""Test no retry happens when the first attempt succeeds."""
|
||||
expected = {("EURUSD", 1): pd.DataFrame()}
|
||||
wrapped = mocker.patch(
|
||||
"mt5cli.sdk.collect_latest_rates_for_accounts",
|
||||
return_value=expected,
|
||||
)
|
||||
sleep = mocker.patch("mt5cli.sdk.time.sleep")
|
||||
accounts = [AccountSpec(symbols=["EURUSD"])]
|
||||
|
||||
result = collect_latest_rates_for_accounts_with_retries(
|
||||
accounts,
|
||||
["M1"],
|
||||
count=1,
|
||||
retry_count=3,
|
||||
)
|
||||
|
||||
assert result is expected
|
||||
assert wrapped.call_count == 1
|
||||
sleep.assert_not_called()
|
||||
|
||||
def test_retries_then_succeeds(self, mocker: MockerFixture) -> None:
|
||||
"""Test transient MT5 errors are retried with exponential backoff."""
|
||||
expected = {("EURUSD", 1): pd.DataFrame()}
|
||||
wrapped = mocker.patch(
|
||||
"mt5cli.sdk.collect_latest_rates_for_accounts",
|
||||
side_effect=[
|
||||
Mt5TradingError("boom"),
|
||||
Mt5RuntimeError("boom"),
|
||||
expected,
|
||||
],
|
||||
)
|
||||
sleep = mocker.patch("mt5cli.sdk.time.sleep")
|
||||
accounts = [AccountSpec(symbols=["EURUSD"])]
|
||||
|
||||
result = collect_latest_rates_for_accounts_with_retries(
|
||||
accounts,
|
||||
["M1"],
|
||||
count=1,
|
||||
retry_count=2,
|
||||
backoff_base=2,
|
||||
)
|
||||
|
||||
assert result is expected
|
||||
assert wrapped.call_count == 3
|
||||
assert sleep.call_args_list == [call(2), call(4)]
|
||||
|
||||
def test_reraises_after_exhausting_retries(self, mocker: MockerFixture) -> None:
|
||||
"""Test the final error is re-raised once retries are exhausted."""
|
||||
wrapped = mocker.patch(
|
||||
"mt5cli.sdk.collect_latest_rates_for_accounts",
|
||||
side_effect=Mt5RuntimeError("boom"),
|
||||
)
|
||||
sleep = mocker.patch("mt5cli.sdk.time.sleep")
|
||||
accounts = [AccountSpec(symbols=["EURUSD"])]
|
||||
|
||||
with pytest.raises(Mt5RuntimeError, match="boom"):
|
||||
collect_latest_rates_for_accounts_with_retries(
|
||||
accounts,
|
||||
["M1"],
|
||||
count=1,
|
||||
retry_count=2,
|
||||
)
|
||||
|
||||
assert wrapped.call_count == 3
|
||||
assert sleep.call_count == 2
|
||||
|
||||
def test_does_not_retry_unrelated_errors(self, mocker: MockerFixture) -> None:
|
||||
"""Test non-MT5 errors propagate without retrying."""
|
||||
wrapped = mocker.patch(
|
||||
"mt5cli.sdk.collect_latest_rates_for_accounts",
|
||||
side_effect=ValueError("bad input"),
|
||||
)
|
||||
sleep = mocker.patch("mt5cli.sdk.time.sleep")
|
||||
|
||||
with pytest.raises(ValueError, match="bad input"):
|
||||
collect_latest_rates_for_accounts_with_retries(
|
||||
[AccountSpec(symbols=["EURUSD"])],
|
||||
["M1"],
|
||||
count=1,
|
||||
retry_count=3,
|
||||
)
|
||||
|
||||
assert wrapped.call_count == 1
|
||||
sleep.assert_not_called()
|
||||
|
||||
|
||||
class TestCollectLatestClosedRatesForAccounts:
|
||||
"""Tests for collect_latest_closed_rates_for_accounts."""
|
||||
|
||||
def test_fetches_count_plus_one_and_drops_forming_bar(
|
||||
self,
|
||||
mocker: MockerFixture,
|
||||
) -> None:
|
||||
"""Test closed-bar collection requests one extra bar at start_pos=0."""
|
||||
df_rate = pd.DataFrame({"time": [1, 2, 3], "close": [1.1, 1.2, 1.3]})
|
||||
wrapped = mocker.patch(
|
||||
"mt5cli.sdk.collect_latest_rates_for_accounts_with_retries",
|
||||
return_value={("EURUSD", 1): df_rate},
|
||||
)
|
||||
accounts = [AccountSpec(symbols=["EURUSD"])]
|
||||
|
||||
result = collect_latest_closed_rates_for_accounts(
|
||||
accounts,
|
||||
["M1"],
|
||||
count=2,
|
||||
retry_count=1,
|
||||
backoff_base=3,
|
||||
)
|
||||
|
||||
wrapped.assert_called_once_with(
|
||||
accounts,
|
||||
["M1"],
|
||||
3,
|
||||
start_pos=0,
|
||||
base_config=None,
|
||||
retry_count=1,
|
||||
backoff_base=3,
|
||||
)
|
||||
pd.testing.assert_frame_equal(
|
||||
result["EURUSD", 1],
|
||||
pd.DataFrame({"time": [1, 2], "close": [1.1, 1.2]}),
|
||||
)
|
||||
|
||||
def test_rejects_forming_bar_only_frames(self, mocker: MockerFixture) -> None:
|
||||
"""Test empty results after dropping the forming bar raise ValueError."""
|
||||
mocker.patch(
|
||||
"mt5cli.sdk.collect_latest_rates_for_accounts_with_retries",
|
||||
return_value={("EURUSD", 1): pd.DataFrame({"time": [1], "close": [1.1]})},
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="Rate data is empty"):
|
||||
collect_latest_closed_rates_for_accounts(
|
||||
[AccountSpec(symbols=["EURUSD"])],
|
||||
["M1"],
|
||||
count=1,
|
||||
)
|
||||
|
||||
def test_skips_extra_fetch_when_start_pos_nonzero(
|
||||
self,
|
||||
mocker: MockerFixture,
|
||||
) -> None:
|
||||
"""Test start_pos > 0 fetches count bars without dropping the last row."""
|
||||
df_rate = pd.DataFrame({"time": [1, 2], "close": [1.1, 1.2]})
|
||||
wrapped = mocker.patch(
|
||||
"mt5cli.sdk.collect_latest_rates_for_accounts_with_retries",
|
||||
return_value={("EURUSD", 1): df_rate},
|
||||
)
|
||||
|
||||
result = collect_latest_closed_rates_for_accounts(
|
||||
[AccountSpec(symbols=["EURUSD"])],
|
||||
["M1"],
|
||||
count=2,
|
||||
start_pos=1,
|
||||
)
|
||||
|
||||
wrapped.assert_called_once_with(
|
||||
[AccountSpec(symbols=["EURUSD"])],
|
||||
["M1"],
|
||||
2,
|
||||
start_pos=1,
|
||||
base_config=None,
|
||||
retry_count=0,
|
||||
backoff_base=2.0,
|
||||
)
|
||||
pd.testing.assert_frame_equal(result["EURUSD", 1], df_rate)
|
||||
|
||||
def test_rejects_zero_count_before_fetching(self, mocker: MockerFixture) -> None:
|
||||
"""Test count=0 is rejected before any MT5 collection attempt."""
|
||||
wrapped = mocker.patch(
|
||||
"mt5cli.sdk.collect_latest_rates_for_accounts_with_retries",
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="count must be positive"):
|
||||
collect_latest_closed_rates_for_accounts(
|
||||
[AccountSpec(symbols=["EURUSD"])],
|
||||
["M1"],
|
||||
count=0,
|
||||
)
|
||||
|
||||
wrapped.assert_not_called()
|
||||
|
||||
def test_rejects_negative_start_pos(self, mocker: MockerFixture) -> None:
|
||||
"""Test negative start_pos is rejected before any MT5 collection attempt."""
|
||||
wrapped = mocker.patch(
|
||||
"mt5cli.sdk.collect_latest_rates_for_accounts_with_retries",
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="start_pos must be non-negative"):
|
||||
collect_latest_closed_rates_for_accounts(
|
||||
[AccountSpec(symbols=["EURUSD"])],
|
||||
["M1"],
|
||||
count=1,
|
||||
start_pos=-1,
|
||||
)
|
||||
|
||||
wrapped.assert_not_called()
|
||||
|
||||
def test_rejects_empty_frames_with_start_pos_nonzero(
|
||||
self,
|
||||
mocker: MockerFixture,
|
||||
) -> None:
|
||||
"""Test empty upstream frames raise ValueError when start_pos > 0."""
|
||||
mocker.patch(
|
||||
"mt5cli.sdk.collect_latest_rates_for_accounts_with_retries",
|
||||
return_value={("EURUSD", 1): pd.DataFrame(columns=["time", "close"])},
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="Rate data is empty"):
|
||||
collect_latest_closed_rates_for_accounts(
|
||||
[AccountSpec(symbols=["EURUSD"])],
|
||||
["M1"],
|
||||
count=1,
|
||||
start_pos=1,
|
||||
)
|
||||
|
||||
def test_processes_multiple_symbol_timeframe_pairs(
|
||||
self,
|
||||
mocker: MockerFixture,
|
||||
) -> None:
|
||||
"""Test each returned series is trimmed and validated independently."""
|
||||
mocker.patch(
|
||||
"mt5cli.sdk.collect_latest_rates_for_accounts_with_retries",
|
||||
return_value={
|
||||
("EURUSD", 1): pd.DataFrame(
|
||||
{"time": [1, 2, 3], "close": [1.1, 1.2, 1.3]},
|
||||
),
|
||||
("GBPUSD", 16385): pd.DataFrame(
|
||||
{"time": [4, 5, 6], "close": [2.1, 2.2, 2.3]},
|
||||
),
|
||||
},
|
||||
)
|
||||
|
||||
result = collect_latest_closed_rates_for_accounts(
|
||||
[AccountSpec(symbols=["EURUSD", "GBPUSD"])],
|
||||
["M1", "H1"],
|
||||
count=2,
|
||||
)
|
||||
|
||||
assert set(result) == {("EURUSD", 1), ("GBPUSD", 16385)}
|
||||
pd.testing.assert_frame_equal(
|
||||
result["EURUSD", 1],
|
||||
pd.DataFrame({"time": [1, 2], "close": [1.1, 1.2]}),
|
||||
)
|
||||
pd.testing.assert_frame_equal(
|
||||
result["GBPUSD", 16385],
|
||||
pd.DataFrame({"time": [4, 5], "close": [2.1, 2.2]}),
|
||||
)
|
||||
|
||||
|
||||
class TestCollectLatestClosedRatesByGranularity:
|
||||
"""Tests for collect_latest_closed_rates_by_granularity."""
|
||||
|
||||
def test_rekeys_by_granularity_name(self, mocker: MockerFixture) -> None:
|
||||
"""Test closed rates are keyed by symbol and granularity name."""
|
||||
df_rate = pd.DataFrame({"time": [1, 2], "close": [1.1, 1.2]})
|
||||
wrapped = mocker.patch(
|
||||
"mt5cli.sdk.collect_latest_closed_rates_for_accounts",
|
||||
return_value={("EURUSD", 1): df_rate},
|
||||
)
|
||||
|
||||
result = collect_latest_closed_rates_by_granularity(
|
||||
[AccountSpec(symbols=["EURUSD"])],
|
||||
["M1"],
|
||||
count=2,
|
||||
)
|
||||
|
||||
wrapped.assert_called_once_with(
|
||||
[AccountSpec(symbols=["EURUSD"])],
|
||||
["M1"],
|
||||
2,
|
||||
start_pos=0,
|
||||
base_config=None,
|
||||
retry_count=0,
|
||||
backoff_base=2.0,
|
||||
)
|
||||
assert ("EURUSD", "M1") in result
|
||||
pd.testing.assert_frame_equal(result["EURUSD", "M1"], df_rate)
|
||||
|
||||
|
||||
class TestSubstituteEnvPlaceholders:
|
||||
"""Tests for ${ENV_VAR} substitution."""
|
||||
|
||||
def test_substitutes_known_variables(
|
||||
self,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Test placeholders are replaced with environment values."""
|
||||
monkeypatch.setenv("MT5_LOGIN", "12345")
|
||||
monkeypatch.setenv("MT5_SERVER", "Broker-Demo")
|
||||
|
||||
assert substitute_env_placeholders("${MT5_LOGIN}") == "12345"
|
||||
assert substitute_env_placeholders("srv=${MT5_SERVER}!") == "srv=Broker-Demo!"
|
||||
|
||||
def test_returns_plain_strings_unchanged(self) -> None:
|
||||
"""Test strings without placeholders are returned as-is."""
|
||||
assert substitute_env_placeholders("plain") == "plain"
|
||||
|
||||
def test_raises_on_missing_variable(
|
||||
self,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Test a missing environment variable raises a clear error."""
|
||||
monkeypatch.delenv("MT5_MISSING", raising=False)
|
||||
|
||||
with pytest.raises(ValueError, match="'MT5_MISSING' is not set"):
|
||||
substitute_env_placeholders("${MT5_MISSING}")
|
||||
|
||||
|
||||
class TestResolveAccountSpec:
|
||||
"""Tests for resolve_account_spec and resolve_account_specs."""
|
||||
|
||||
def test_substitutes_env_placeholders_in_account(
|
||||
self,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Test account string fields resolve ${ENV_VAR} placeholders."""
|
||||
monkeypatch.setenv("MT5_PASSWORD", "secret")
|
||||
account = AccountSpec(
|
||||
symbols=["EURUSD"],
|
||||
login="${MT5_LOGIN}",
|
||||
password="${MT5_PASSWORD}",
|
||||
)
|
||||
monkeypatch.setenv("MT5_LOGIN", "999")
|
||||
|
||||
resolved = resolve_account_spec(account)
|
||||
|
||||
assert resolved.login == "999"
|
||||
assert resolved.password == "secret" # noqa: S105
|
||||
assert resolved.symbols == ["EURUSD"]
|
||||
|
||||
def test_explicit_overrides_take_precedence(self) -> None:
|
||||
"""Test explicit override values win over account fields."""
|
||||
account = AccountSpec(symbols=["EURUSD"], login=111, server="Acct")
|
||||
|
||||
resolved = resolve_account_spec(
|
||||
account,
|
||||
login=222,
|
||||
server="Override",
|
||||
timeout=5000,
|
||||
)
|
||||
|
||||
assert resolved.login == 222
|
||||
assert resolved.server == "Override"
|
||||
assert resolved.timeout == 5000
|
||||
|
||||
def test_resolves_string_login_override(
|
||||
self,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Test string login overrides expand ${ENV_VAR} placeholders."""
|
||||
monkeypatch.setenv("MT5_LOGIN", "777")
|
||||
account = AccountSpec(symbols=["EURUSD"], login=111)
|
||||
|
||||
resolved = resolve_account_spec(account, login="${MT5_LOGIN}")
|
||||
|
||||
assert resolved.login == "777"
|
||||
|
||||
def test_preserves_integer_login_without_coercion(self) -> None:
|
||||
"""Test integer logins remain integers after resolution."""
|
||||
account = AccountSpec(symbols=["EURUSD"], login=111)
|
||||
|
||||
resolved = resolve_account_spec(account)
|
||||
|
||||
assert resolved.login == 111
|
||||
assert isinstance(resolved.login, int)
|
||||
|
||||
def test_raises_on_missing_env_variable(
|
||||
self,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Test missing environment variables raise ValueError."""
|
||||
monkeypatch.delenv("MT5_NOPE", raising=False)
|
||||
account = AccountSpec(symbols=["EURUSD"], server="${MT5_NOPE}")
|
||||
|
||||
with pytest.raises(ValueError, match="'MT5_NOPE' is not set"):
|
||||
resolve_account_spec(account)
|
||||
|
||||
def test_resolve_account_specs_applies_to_all(
|
||||
self,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Test resolve_account_specs resolves every account in order."""
|
||||
monkeypatch.setenv("MT5_SERVER", "Shared")
|
||||
accounts = [
|
||||
AccountSpec(symbols=["EURUSD"], server="${MT5_SERVER}"),
|
||||
AccountSpec(symbols=["GBPUSD"], server="Fixed"),
|
||||
]
|
||||
|
||||
resolved = resolve_account_specs(accounts, timeout=1000)
|
||||
|
||||
assert [a.server for a in resolved] == ["Shared", "Fixed"]
|
||||
assert all(a.timeout == 1000 for a in resolved)
|
||||
|
||||
|
||||
class TestThrottledHistoryUpdater:
|
||||
"""Tests for the throttled incremental history updater."""
|
||||
|
||||
def test_updates_every_call_when_interval_non_positive(
|
||||
self,
|
||||
mocker: MockerFixture,
|
||||
) -> None:
|
||||
"""Test interval_seconds <= 0 updates on every call."""
|
||||
update = mocker.patch("mt5cli.sdk.update_history")
|
||||
client = MagicMock()
|
||||
updater = ThrottledHistoryUpdater(output="history.db", interval_seconds=0)
|
||||
|
||||
assert updater.update(client, ["EURUSD"]) is True
|
||||
assert updater.update(client, ["EURUSD"]) is True
|
||||
assert update.call_count == 2
|
||||
|
||||
def test_throttles_within_interval(self, mocker: MockerFixture) -> None:
|
||||
"""Test updates are skipped until the interval elapses."""
|
||||
update = mocker.patch("mt5cli.sdk.update_history")
|
||||
monotonic = mocker.patch("mt5cli.sdk.time.monotonic")
|
||||
# Calls: set(t=100), check(t=105), check(t=200), set(t=200).
|
||||
monotonic.side_effect = [100.0, 105.0, 200.0, 200.0]
|
||||
client = MagicMock()
|
||||
updater = ThrottledHistoryUpdater(output="history.db", interval_seconds=60)
|
||||
|
||||
assert updater.update(client, ["EURUSD"]) is True # first update at t=100
|
||||
assert updater.update(client, ["EURUSD"]) is False # t=105, throttled
|
||||
assert updater.update(client, ["EURUSD"]) is True # t=200, elapsed
|
||||
assert update.call_count == 2
|
||||
|
||||
def test_update_passes_expected_arguments(
|
||||
self,
|
||||
mocker: MockerFixture,
|
||||
) -> None:
|
||||
"""Test update_history is called with the configured arguments."""
|
||||
update = mocker.patch("mt5cli.sdk.update_history")
|
||||
client = MagicMock()
|
||||
updater = ThrottledHistoryUpdater(
|
||||
output="history.db",
|
||||
datasets={Dataset.rates},
|
||||
timeframes=["M1", "H1"],
|
||||
flags="INFO",
|
||||
lookback_hours=12.0,
|
||||
with_views=True,
|
||||
include_account_events=False,
|
||||
)
|
||||
|
||||
updater.update(client, ["EURUSD", "GBPUSD"])
|
||||
|
||||
update.assert_called_once_with(
|
||||
client=client,
|
||||
output="history.db",
|
||||
symbols=["EURUSD", "GBPUSD"],
|
||||
datasets={Dataset.rates},
|
||||
timeframes=["M1", "H1"],
|
||||
flags="INFO",
|
||||
lookback_hours=12.0,
|
||||
with_views=True,
|
||||
include_account_events=False,
|
||||
)
|
||||
|
||||
def test_propagates_errors_by_default(self, mocker: MockerFixture) -> None:
|
||||
"""Test MT5/SQLite errors propagate and do not advance the throttle."""
|
||||
mocker.patch(
|
||||
"mt5cli.sdk.update_history",
|
||||
side_effect=Mt5RuntimeError("boom"),
|
||||
)
|
||||
updater = ThrottledHistoryUpdater(output="history.db")
|
||||
|
||||
with pytest.raises(Mt5RuntimeError, match="boom"):
|
||||
updater.update(MagicMock(), ["EURUSD"])
|
||||
|
||||
assert updater.last_update_monotonic is None
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"error",
|
||||
[
|
||||
Mt5RuntimeError("boom"),
|
||||
Mt5TradingError("trade failed"),
|
||||
sqlite3.OperationalError("locked"),
|
||||
ValueError("invalid symbols"),
|
||||
OSError("disk full"),
|
||||
AttributeError(
|
||||
"'StubClient' object has no attribute 'copy_rates_range_as_df'",
|
||||
name="copy_rates_range_as_df",
|
||||
),
|
||||
AttributeError(
|
||||
"MT5 client is missing required method: copy_ticks_range_as_df"
|
||||
),
|
||||
TypeError("MT5 client attribute is not callable: history_orders_get_as_df"),
|
||||
],
|
||||
)
|
||||
def test_suppresses_errors_when_requested(
|
||||
self,
|
||||
mocker: MockerFixture,
|
||||
error: Exception,
|
||||
) -> None:
|
||||
"""Test suppress_errors swallows recoverable errors and returns False."""
|
||||
mocker.patch(
|
||||
"mt5cli.sdk.update_history",
|
||||
side_effect=error,
|
||||
)
|
||||
updater = ThrottledHistoryUpdater(
|
||||
output="history.db",
|
||||
suppress_errors=True,
|
||||
)
|
||||
|
||||
assert updater.update(MagicMock(), ["EURUSD"]) is False
|
||||
assert updater.last_update_monotonic is None
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"error",
|
||||
[
|
||||
AttributeError("'dict' object has no attribute 'typo'"),
|
||||
TypeError("unsupported operand types"),
|
||||
],
|
||||
)
|
||||
def test_suppress_errors_does_not_hide_programming_errors(
|
||||
self,
|
||||
mocker: MockerFixture,
|
||||
error: Exception,
|
||||
) -> None:
|
||||
"""Test generic AttributeError/TypeError still propagate when suppressed."""
|
||||
mocker.patch(
|
||||
"mt5cli.sdk.update_history",
|
||||
side_effect=error,
|
||||
)
|
||||
updater = ThrottledHistoryUpdater(
|
||||
output="history.db",
|
||||
suppress_errors=True,
|
||||
)
|
||||
|
||||
with pytest.raises(type(error)):
|
||||
updater.update(MagicMock(), ["EURUSD"])
|
||||
|
||||
assert updater.last_update_monotonic is None
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("error", "expected"),
|
||||
[
|
||||
(AttributeError("MT5 client is missing required method: version"), True),
|
||||
(
|
||||
AttributeError(
|
||||
"'Stub' object has no attribute 'copy_rates_range_as_df'",
|
||||
name="copy_rates_range_as_df",
|
||||
),
|
||||
True,
|
||||
),
|
||||
(AttributeError("'dict' object has no attribute 'typo'"), False),
|
||||
(TypeError("MT5 client attribute is not callable: version"), True),
|
||||
(TypeError("unsupported operand types"), False),
|
||||
(TypeError("'NoneType' object is not callable"), False),
|
||||
(ValueError("invalid"), False),
|
||||
],
|
||||
)
|
||||
def test_is_mt5_client_capability_error(
|
||||
self,
|
||||
error: BaseException,
|
||||
expected: bool,
|
||||
) -> None:
|
||||
"""Test MT5 client capability error detection."""
|
||||
assert sdk._is_mt5_client_capability_error(error) is expected # type: ignore[reportPrivateUsage]
|
||||
|
||||
def test_is_mt5_client_capability_error_for_non_callable_history_client(
|
||||
self,
|
||||
) -> None:
|
||||
"""Test non-callable history client attributes are capability errors."""
|
||||
client = MagicMock()
|
||||
client.copy_rates_range_as_df = None
|
||||
with (
|
||||
sqlite3.connect(":memory:") as conn,
|
||||
pytest.raises(TypeError, match="not callable") as exc_info,
|
||||
):
|
||||
write_rates_dataset(
|
||||
conn,
|
||||
client,
|
||||
["EURUSD"],
|
||||
1,
|
||||
datetime.now(UTC),
|
||||
datetime.now(UTC),
|
||||
IfExists.APPEND,
|
||||
{},
|
||||
)
|
||||
|
||||
assert sdk._is_mt5_client_capability_error(exc_info.value) is True # type: ignore[reportPrivateUsage]
|
||||
|
||||
def test_suppresses_non_callable_history_client_method(
|
||||
self,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""Test suppress_errors swallows non-callable history client API attributes."""
|
||||
client = MagicMock()
|
||||
client.copy_rates_range_as_df = None
|
||||
updater = ThrottledHistoryUpdater(
|
||||
output=tmp_path / "history.db",
|
||||
datasets={Dataset.rates},
|
||||
timeframes=["M1"],
|
||||
suppress_errors=True,
|
||||
)
|
||||
|
||||
assert updater.update(client, ["EURUSD"]) is False
|
||||
assert updater.last_update_monotonic is None
|
||||
|
||||
def test_suppress_errors_does_not_hide_internal_client_type_error(
|
||||
self,
|
||||
mocker: MockerFixture,
|
||||
) -> None:
|
||||
"""Test TypeError raised inside a callable client method still propagates."""
|
||||
mocker.patch(
|
||||
"mt5cli.sdk.update_history",
|
||||
side_effect=TypeError("'int' object is not callable"),
|
||||
)
|
||||
updater = ThrottledHistoryUpdater(
|
||||
output="history.db",
|
||||
suppress_errors=True,
|
||||
)
|
||||
|
||||
with pytest.raises(TypeError, match="not callable"):
|
||||
updater.update(MagicMock(), ["EURUSD"])
|
||||
|
||||
assert updater.last_update_monotonic is None
|
||||
|
||||
def test_suppresses_validation_errors_before_update(
|
||||
self,
|
||||
mocker: MockerFixture,
|
||||
) -> None:
|
||||
"""Test validation failures are suppressed without calling update_history."""
|
||||
update = mocker.patch("mt5cli.sdk.update_history")
|
||||
updater = ThrottledHistoryUpdater(
|
||||
output="history.db",
|
||||
suppress_errors=True,
|
||||
)
|
||||
|
||||
assert updater.update(MagicMock(), []) is False
|
||||
update.assert_not_called()
|
||||
assert updater.last_update_monotonic is None
|
||||
|
||||
@@ -0,0 +1,356 @@
|
||||
"""Tests for trading session helpers and operational utilities."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pandas as pd
|
||||
import pytest
|
||||
from pdmt5 import Mt5RuntimeError
|
||||
from pytest_mock import MockerFixture # noqa: TC002
|
||||
|
||||
from mt5cli.sdk import build_config
|
||||
from mt5cli.trading import (
|
||||
calculate_margin_and_volume,
|
||||
detect_position_side,
|
||||
determine_order_limits,
|
||||
mt5_trading_session,
|
||||
)
|
||||
|
||||
|
||||
class TestDetectPositionSide:
|
||||
"""Tests for detect_position_side."""
|
||||
|
||||
def test_returns_none_when_no_positions(self) -> None:
|
||||
"""Test None is returned when no open positions exist."""
|
||||
client = MagicMock()
|
||||
client.positions_get_as_df.return_value = pd.DataFrame()
|
||||
|
||||
assert detect_position_side(client, "EURUSD") is None
|
||||
|
||||
def test_returns_long_for_net_buy_volume(self) -> None:
|
||||
"""Test long is returned when buy volume exceeds sell volume."""
|
||||
client = MagicMock()
|
||||
client.mt5.POSITION_TYPE_BUY = 0
|
||||
client.mt5.POSITION_TYPE_SELL = 1
|
||||
client.positions_get_as_df.return_value = pd.DataFrame(
|
||||
{
|
||||
"type": [0, 0, 1],
|
||||
"volume": [0.2, 0.1, 0.05],
|
||||
},
|
||||
)
|
||||
|
||||
assert detect_position_side(client, "EURUSD") == "long"
|
||||
|
||||
def test_returns_short_for_net_sell_volume(self) -> None:
|
||||
"""Test short is returned when sell volume exceeds buy volume."""
|
||||
client = MagicMock()
|
||||
client.mt5.POSITION_TYPE_BUY = 0
|
||||
client.mt5.POSITION_TYPE_SELL = 1
|
||||
client.positions_get_as_df.return_value = pd.DataFrame(
|
||||
{
|
||||
"type": [1, 1],
|
||||
"volume": [0.3, 0.1],
|
||||
},
|
||||
)
|
||||
|
||||
assert detect_position_side(client, "EURUSD") == "short"
|
||||
|
||||
def test_returns_none_for_balanced_hedged_positions(self) -> None:
|
||||
"""Test None is returned when buy and sell volumes net to zero."""
|
||||
client = MagicMock()
|
||||
client.mt5.POSITION_TYPE_BUY = 0
|
||||
client.mt5.POSITION_TYPE_SELL = 1
|
||||
client.positions_get_as_df.return_value = pd.DataFrame(
|
||||
{
|
||||
"type": [0, 1],
|
||||
"volume": [0.2, 0.2],
|
||||
},
|
||||
)
|
||||
|
||||
assert detect_position_side(client, "EURUSD") is None
|
||||
|
||||
|
||||
class TestCalculateMarginAndVolume:
|
||||
"""Tests for calculate_margin_and_volume."""
|
||||
|
||||
def test_calculates_margin_budget_and_volumes(self) -> None:
|
||||
"""Test margin budget and buy/sell volumes are derived from ratios."""
|
||||
client = MagicMock()
|
||||
client.account_info_as_dict.return_value = {"margin_free": 1000.0}
|
||||
client.calculate_volume_by_margin.side_effect = [0.3, 0.2]
|
||||
|
||||
result = calculate_margin_and_volume(
|
||||
client,
|
||||
"EURUSD",
|
||||
unit_margin_ratio=0.5,
|
||||
preserved_margin_ratio=0.2,
|
||||
)
|
||||
|
||||
assert result == {
|
||||
"margin_free": 1000.0,
|
||||
"available_margin": 800.0,
|
||||
"trade_margin": 400.0,
|
||||
"buy_volume": 0.3,
|
||||
"sell_volume": 0.2,
|
||||
}
|
||||
client.calculate_volume_by_margin.assert_any_call("EURUSD", 400.0, "BUY")
|
||||
client.calculate_volume_by_margin.assert_any_call("EURUSD", 400.0, "SELL")
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("account_dict", "expected_margin_free"),
|
||||
[
|
||||
({"margin_free": 0.0}, 0.0),
|
||||
({}, 0.0),
|
||||
({"margin_free": None}, 0.0),
|
||||
],
|
||||
)
|
||||
def test_zero_or_missing_margin_free(
|
||||
self,
|
||||
account_dict: dict[str, float | None],
|
||||
expected_margin_free: float,
|
||||
) -> None:
|
||||
"""Test missing or zero margin_free yields zero trade margin."""
|
||||
client = MagicMock()
|
||||
client.account_info_as_dict.return_value = account_dict
|
||||
client.calculate_volume_by_margin.return_value = 0.0
|
||||
|
||||
result = calculate_margin_and_volume(
|
||||
client,
|
||||
"EURUSD",
|
||||
unit_margin_ratio=0.5,
|
||||
preserved_margin_ratio=0.2,
|
||||
)
|
||||
|
||||
assert result["margin_free"] == expected_margin_free
|
||||
client.calculate_volume_by_margin.assert_any_call("EURUSD", 0.0, "BUY")
|
||||
client.calculate_volume_by_margin.assert_any_call("EURUSD", 0.0, "SELL")
|
||||
|
||||
def test_clamps_negative_margin_free_to_zero(self) -> None:
|
||||
"""Test negative margin_free is clamped to zero before sizing."""
|
||||
client = MagicMock()
|
||||
client.account_info_as_dict.return_value = {"margin_free": -500.0}
|
||||
client.calculate_volume_by_margin.return_value = 0.0
|
||||
|
||||
result = calculate_margin_and_volume(
|
||||
client,
|
||||
"EURUSD",
|
||||
unit_margin_ratio=0.5,
|
||||
preserved_margin_ratio=0.2,
|
||||
)
|
||||
|
||||
expected_margin_free = 0.0
|
||||
assert result["margin_free"] == expected_margin_free
|
||||
client.calculate_volume_by_margin.assert_any_call("EURUSD", 0.0, "BUY")
|
||||
client.calculate_volume_by_margin.assert_any_call("EURUSD", 0.0, "SELL")
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("unit_ratio", "preserved_ratio"),
|
||||
[
|
||||
(-0.1, 0.0),
|
||||
(1.1, 0.0),
|
||||
(0.5, -0.1),
|
||||
(0.5, 1.1),
|
||||
],
|
||||
)
|
||||
def test_rejects_invalid_ratios(
|
||||
self,
|
||||
unit_ratio: float,
|
||||
preserved_ratio: float,
|
||||
) -> None:
|
||||
"""Test invalid ratio values raise ValueError."""
|
||||
with pytest.raises(ValueError, match="must be between 0 and 1"):
|
||||
calculate_margin_and_volume(
|
||||
MagicMock(),
|
||||
"EURUSD",
|
||||
unit_margin_ratio=unit_ratio,
|
||||
preserved_margin_ratio=preserved_ratio,
|
||||
)
|
||||
|
||||
|
||||
class TestDetermineOrderLimits:
|
||||
"""Tests for determine_order_limits."""
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("side", "expected_entry_key"),
|
||||
[
|
||||
("long", "ask"),
|
||||
("short", "bid"),
|
||||
("buy", "ask"),
|
||||
("sell", "bid"),
|
||||
],
|
||||
)
|
||||
def test_uses_expected_quote_for_entry(
|
||||
self,
|
||||
side: str,
|
||||
expected_entry_key: str,
|
||||
) -> None:
|
||||
"""Test entry price is taken from ask for long/buy and bid for short/sell."""
|
||||
client = MagicMock()
|
||||
client.symbol_info_tick_as_dict.return_value = {"ask": 1.1010, "bid": 1.1000}
|
||||
|
||||
result = determine_order_limits(
|
||||
client,
|
||||
"EURUSD",
|
||||
side,
|
||||
stop_loss_limit_ratio=0.0,
|
||||
take_profit_limit_ratio=0.0,
|
||||
)
|
||||
|
||||
assert (
|
||||
result["entry"]
|
||||
== client.symbol_info_tick_as_dict.return_value[expected_entry_key]
|
||||
)
|
||||
assert result["stop_loss"] is None
|
||||
assert result["take_profit"] is None
|
||||
|
||||
def test_calculates_long_protective_levels(self) -> None:
|
||||
"""Test long stop loss and take profit are placed below/above entry."""
|
||||
client = MagicMock()
|
||||
client.symbol_info_tick_as_dict.return_value = {"ask": 100.0, "bid": 99.0}
|
||||
|
||||
result = determine_order_limits(
|
||||
client,
|
||||
"EURUSD",
|
||||
"long",
|
||||
stop_loss_limit_ratio=0.02,
|
||||
take_profit_limit_ratio=0.03,
|
||||
)
|
||||
|
||||
assert result == {
|
||||
"entry": 100.0,
|
||||
"stop_loss": 98.0,
|
||||
"take_profit": 103.0,
|
||||
}
|
||||
|
||||
def test_calculates_short_protective_levels(self) -> None:
|
||||
"""Test short stop loss and take profit are placed above/below entry."""
|
||||
client = MagicMock()
|
||||
client.symbol_info_tick_as_dict.return_value = {"ask": 100.0, "bid": 99.0}
|
||||
|
||||
result = determine_order_limits(
|
||||
client,
|
||||
"EURUSD",
|
||||
"short",
|
||||
stop_loss_limit_ratio=0.02,
|
||||
take_profit_limit_ratio=0.03,
|
||||
)
|
||||
|
||||
assert result == {
|
||||
"entry": 99.0,
|
||||
"stop_loss": 100.98,
|
||||
"take_profit": 96.03,
|
||||
}
|
||||
|
||||
def test_rejects_unknown_side(self) -> None:
|
||||
"""Test unsupported side values raise ValueError."""
|
||||
with pytest.raises(ValueError, match="Unsupported order side"):
|
||||
determine_order_limits(
|
||||
MagicMock(),
|
||||
"EURUSD",
|
||||
"flat",
|
||||
stop_loss_limit_ratio=0.01,
|
||||
take_profit_limit_ratio=0.01,
|
||||
)
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("stop_loss_ratio", "take_profit_ratio"),
|
||||
[
|
||||
(-0.05, 0.01),
|
||||
(0.01, 2.0),
|
||||
],
|
||||
)
|
||||
def test_rejects_invalid_protective_ratios(
|
||||
self,
|
||||
stop_loss_ratio: float,
|
||||
take_profit_ratio: float,
|
||||
) -> None:
|
||||
"""Test out-of-range protective ratios raise ValueError."""
|
||||
with pytest.raises(ValueError, match="must be at least 0 and less than 1"):
|
||||
determine_order_limits(
|
||||
MagicMock(),
|
||||
"EURUSD",
|
||||
"long",
|
||||
stop_loss_limit_ratio=stop_loss_ratio,
|
||||
take_profit_limit_ratio=take_profit_ratio,
|
||||
)
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("field", "ratio"),
|
||||
[
|
||||
("stop_loss_limit_ratio", 1.0),
|
||||
("take_profit_limit_ratio", 1.0),
|
||||
],
|
||||
)
|
||||
def test_rejects_unit_boundary_protective_ratios(
|
||||
self,
|
||||
field: str,
|
||||
ratio: float,
|
||||
) -> None:
|
||||
"""Test protective ratios of exactly 1.0 are rejected."""
|
||||
kwargs = {
|
||||
"stop_loss_limit_ratio": 0.01,
|
||||
"take_profit_limit_ratio": 0.01,
|
||||
field: ratio,
|
||||
}
|
||||
with pytest.raises(ValueError, match="must be at least 0 and less than 1"):
|
||||
determine_order_limits(
|
||||
MagicMock(),
|
||||
"EURUSD",
|
||||
"long",
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
|
||||
class TestMt5TradingSession:
|
||||
"""Tests for the mt5_trading_session context manager."""
|
||||
|
||||
def test_yields_connected_client_and_shuts_down(
|
||||
self,
|
||||
mocker: MockerFixture,
|
||||
) -> None:
|
||||
"""Test mt5_trading_session connects, yields a client, and shuts down."""
|
||||
mock_client = MagicMock()
|
||||
trading_client = mocker.patch(
|
||||
"mt5cli.trading.Mt5TradingClient",
|
||||
return_value=mock_client,
|
||||
)
|
||||
|
||||
with mt5_trading_session(
|
||||
build_config(path="/opt/mt5/terminal64.exe"),
|
||||
retry_count=2,
|
||||
) as client:
|
||||
mock_client.initialize_and_login_mt5.assert_called_once()
|
||||
assert client is mock_client
|
||||
|
||||
trading_client.assert_called_once()
|
||||
assert trading_client.call_args.kwargs["retry_count"] == 2
|
||||
assert (
|
||||
trading_client.call_args.kwargs["config"].path == "/opt/mt5/terminal64.exe"
|
||||
)
|
||||
mock_client.shutdown.assert_called_once()
|
||||
|
||||
def test_shuts_down_when_initialize_raises(
|
||||
self,
|
||||
mocker: MockerFixture,
|
||||
) -> None:
|
||||
"""Test shutdown is called when initialization fails."""
|
||||
mock_client = MagicMock()
|
||||
mock_client.initialize_and_login_mt5.side_effect = Mt5RuntimeError("boom")
|
||||
mocker.patch("mt5cli.trading.Mt5TradingClient", return_value=mock_client)
|
||||
|
||||
with pytest.raises(Mt5RuntimeError, match="boom"), mt5_trading_session():
|
||||
pass
|
||||
|
||||
mock_client.shutdown.assert_called_once()
|
||||
|
||||
def test_shuts_down_when_body_raises(self, mocker: MockerFixture) -> None:
|
||||
"""Test shutdown is called when the context body raises."""
|
||||
mock_client = MagicMock()
|
||||
mocker.patch("mt5cli.trading.Mt5TradingClient", return_value=mock_client)
|
||||
|
||||
body_error = "body error"
|
||||
with pytest.raises(RuntimeError, match=body_error), mt5_trading_session():
|
||||
raise RuntimeError(body_error)
|
||||
|
||||
mock_client.shutdown.assert_called_once()
|
||||
+51
-14
@@ -274,8 +274,14 @@ class TestParseTimeframe:
|
||||
assert parse_timeframe(value) == expected
|
||||
|
||||
def test_integer_timeframe(self) -> None:
|
||||
"""Test parsing integer timeframe."""
|
||||
assert parse_timeframe("42") == 42
|
||||
"""Test parsing supported integer timeframes."""
|
||||
assert parse_timeframe("1") == 1
|
||||
assert parse_timeframe(16385) == 16385
|
||||
|
||||
def test_unsupported_integer_timeframe_raises(self) -> None:
|
||||
"""Test that unsupported integer timeframes raise ValueError."""
|
||||
with pytest.raises(ValueError, match="Invalid timeframe"):
|
||||
parse_timeframe("42")
|
||||
|
||||
def test_invalid_timeframe_raises(self) -> None:
|
||||
"""Test that invalid timeframe raises ValueError."""
|
||||
@@ -288,15 +294,21 @@ class TestParseTickFlags:
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("value", "expected"),
|
||||
[("ALL", 1), ("info", 2), ("TRADE", 4)],
|
||||
[("ALL", -1), ("info", 1), ("TRADE", 2), ("COPY_TICKS_ALL", -1)],
|
||||
)
|
||||
def test_named_flag(self, value: str, expected: int) -> None:
|
||||
"""Test parsing named tick flags."""
|
||||
assert parse_tick_flags(value) == expected
|
||||
|
||||
def test_integer_flag(self) -> None:
|
||||
"""Test parsing integer tick flag."""
|
||||
assert parse_tick_flags("7") == 7
|
||||
"""Test parsing supported integer tick flags."""
|
||||
assert parse_tick_flags("-1") == -1
|
||||
assert parse_tick_flags(2) == 2
|
||||
|
||||
def test_unsupported_integer_flag_raises(self) -> None:
|
||||
"""Test that unsupported integer tick flags raise ValueError."""
|
||||
with pytest.raises(ValueError, match="Invalid tick flags"):
|
||||
parse_tick_flags("7")
|
||||
|
||||
def test_invalid_flag_raises(self) -> None:
|
||||
"""Test that invalid flag raises ValueError."""
|
||||
@@ -355,8 +367,11 @@ class TestConstants:
|
||||
assert key in TIMEFRAME_MAP
|
||||
|
||||
def test_tick_flag_map_has_expected_keys(self) -> None:
|
||||
"""Test that TICK_FLAG_MAP contains standard flags."""
|
||||
assert set(TICK_FLAG_MAP) == {"ALL", "INFO", "TRADE"}
|
||||
"""Test that TICK_FLAG_MAP contains standard flags with MT5 values."""
|
||||
assert {"ALL", "INFO", "TRADE"} <= set(TICK_FLAG_MAP)
|
||||
assert TICK_FLAG_MAP["ALL"] == -1
|
||||
assert TICK_FLAG_MAP["INFO"] == 1
|
||||
assert TICK_FLAG_MAP["TRADE"] == 2
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("dataset", "expected"),
|
||||
@@ -403,26 +418,48 @@ class TestTimeframeType:
|
||||
"""Test converting a string to timeframe integer."""
|
||||
assert TIMEFRAME_TYPE.convert("H1", None, None) == 16385
|
||||
|
||||
def test_convert_int_passthrough(self) -> None:
|
||||
"""Test that integer values pass through unchanged."""
|
||||
assert TIMEFRAME_TYPE.convert(42, None, None) == 42
|
||||
def test_convert_int(self) -> None:
|
||||
"""Test converting supported integer timeframe values."""
|
||||
assert TIMEFRAME_TYPE.convert(16385, None, None) == 16385
|
||||
|
||||
def test_convert_unsupported_int(self) -> None:
|
||||
"""Test that unsupported integer values raise BadParameter."""
|
||||
with pytest.raises(Exception, match="Invalid timeframe"):
|
||||
TIMEFRAME_TYPE.convert(42, None, None)
|
||||
|
||||
def test_convert_invalid(self) -> None:
|
||||
"""Test that invalid values raise BadParameter."""
|
||||
with pytest.raises(Exception, match="Invalid timeframe"):
|
||||
TIMEFRAME_TYPE.convert("bad", None, None)
|
||||
|
||||
@pytest.mark.parametrize("value", [True, False, None, 1.5])
|
||||
def test_convert_invalid_types(self, value: object) -> None:
|
||||
"""Test that bool, float, and None values raise BadParameter."""
|
||||
with pytest.raises(Exception, match="Invalid timeframe"):
|
||||
TIMEFRAME_TYPE.convert(value, None, None)
|
||||
|
||||
|
||||
class TestTickFlagsType:
|
||||
"""Tests for _TickFlagsType."""
|
||||
|
||||
def test_convert_string(self) -> None:
|
||||
"""Test converting a string to tick flags integer."""
|
||||
assert TICK_FLAGS_TYPE.convert("ALL", None, None) == 1
|
||||
assert TICK_FLAGS_TYPE.convert("ALL", None, None) == -1
|
||||
|
||||
def test_convert_int_passthrough(self) -> None:
|
||||
"""Test that integer values pass through unchanged."""
|
||||
assert TICK_FLAGS_TYPE.convert(7, None, None) == 7
|
||||
def test_convert_int(self) -> None:
|
||||
"""Test converting supported integer tick flag values."""
|
||||
assert TICK_FLAGS_TYPE.convert(2, None, None) == 2
|
||||
|
||||
def test_convert_unsupported_int(self) -> None:
|
||||
"""Test that unsupported integer values raise BadParameter."""
|
||||
with pytest.raises(Exception, match="Invalid tick flags"):
|
||||
TICK_FLAGS_TYPE.convert(7, None, None)
|
||||
|
||||
@pytest.mark.parametrize("value", [True, False, None, 1.5])
|
||||
def test_convert_invalid_types(self, value: object) -> None:
|
||||
"""Test that bool, float, and None values raise BadParameter."""
|
||||
with pytest.raises(Exception, match="Invalid tick flags"):
|
||||
TICK_FLAGS_TYPE.convert(value, None, None)
|
||||
|
||||
def test_convert_invalid(self) -> None:
|
||||
"""Test that invalid values raise BadParameter."""
|
||||
|
||||
@@ -487,7 +487,7 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "mt5cli"
|
||||
version = "0.5.2"
|
||||
version = "0.7.0"
|
||||
source = { editable = "." }
|
||||
dependencies = [
|
||||
{ name = "click" },
|
||||
@@ -513,7 +513,7 @@ dev = [
|
||||
[package.metadata]
|
||||
requires-dist = [
|
||||
{ name = "click", specifier = ">=8.1.0" },
|
||||
{ name = "pdmt5", specifier = ">=0.2.3" },
|
||||
{ name = "pdmt5", specifier = ">=0.3.0" },
|
||||
{ name = "pyarrow", specifier = ">=19.0.0" },
|
||||
{ name = "typer", specifier = ">=0.15.0" },
|
||||
]
|
||||
@@ -684,16 +684,16 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "pdmt5"
|
||||
version = "0.2.3"
|
||||
version = "0.3.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "metatrader5", marker = "sys_platform == 'win32'" },
|
||||
{ name = "pandas" },
|
||||
{ name = "pydantic" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/02/25/52d9d954504ccdd0fe91f715ab74c424d61234b237cc4160d3ebe20070f1/pdmt5-0.2.3.tar.gz", hash = "sha256:21384f5826fb0125fee3f93c90b108340f55ab53b1c819d229ceac162289d2ec", size = 226665, upload-time = "2026-02-05T13:28:21.071Z" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/bf/cc/c8fa3a01e0e34178fec8527992f7bb8eda5881477ce23aaacaa9b2ef7bec/pdmt5-0.3.0.tar.gz", hash = "sha256:bb612d5c2695eafac9b2a7b74756e13bd383d7e5517bd90c9a2efa92492c484c", size = 215100, upload-time = "2026-06-11T13:26:46.976Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/c1/75/c5e52a9cf459b85b2dd52f83e70857571b1b45805c9fe610b3959a26ac15/pdmt5-0.2.3-py3-none-any.whl", hash = "sha256:f92246a05cfc3b7feb3ab0cc5b48768a4d84aad6b02e7a68060948f5828718a1", size = 22967, upload-time = "2026-02-05T13:28:19.523Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f2/03/b12cc4c9db983d971c9172b3765161b6d91136d0624e6718a04dd815e7a1/pdmt5-0.3.0-py3-none-any.whl", hash = "sha256:5388b406cc583202600cfe22c9d781679b1d931b1ed5a2b5dcf37c566149b49f", size = 26250, upload-time = "2026-06-11T13:26:45.689Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -836,11 +836,11 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "pygments"
|
||||
version = "2.19.2"
|
||||
version = "2.20.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/b0/77/a5b8c569bf593b0140bde72ea885a803b82086995367bf2037de0159d924/pygments-2.19.2.tar.gz", hash = "sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887", size = 4968631, upload-time = "2025-06-21T13:39:12.283Z" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", size = 4955991, upload-time = "2026-03-29T13:29:33.898Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/c7/21/705964c7812476f378728bdf590ca4b771ec72385c533964653c68e86bdc/pygments-2.19.2-py3-none-any.whl", hash = "sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b", size = 1225217, upload-time = "2025-06-21T13:39:07.939Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
||||
Reference in New Issue
Block a user