Compare commits
13 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 254c159ad5 | |||
| 78c49238cf | |||
| 9356d5dcdf | |||
| 0fad55d609 | |||
| d654b82f9d | |||
| b5e82e71c7 | |||
| 18df96872b | |||
| 5b1d54bfe9 | |||
| ad9e513253 | |||
| 334f01b647 | |||
| 1b69e8f08e | |||
| 9957b0a1de | |||
| b2bb2ad0a0 |
@@ -2,10 +2,16 @@
|
||||
|
||||
[](https://github.com/dceoy/mt5cli/actions/workflows/ci.yml)
|
||||
|
||||
Command-line tool for exporting MetaTrader 5 data to CSV, JSON, Parquet, and SQLite3.
|
||||
Generic MT5 data and execution infrastructure for Python applications. Export from the CLI or import a small, stable Python API in downstream packages.
|
||||
|
||||
Built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data handler for MetaTrader 5.
|
||||
|
||||
## Architecture
|
||||
|
||||
- **pdmt5** — canonical MT5 client, DataFrame/trading primitives, and MT5 constant parsing (`TIMEFRAME_*`, `COPY_TICKS_*`, order types).
|
||||
- **mt5cli** — public `MT5Client` API, standardized dataset schemas, storage helpers, CLI commands, and SQLite history collection built on pdmt5.
|
||||
- **mt5api** — sibling HTTP adapter for remote MT5 access; not a dependency of mt5cli.
|
||||
|
||||
## Features
|
||||
|
||||
- **Multi-format export**: CSV, JSON, Parquet, and SQLite3 output formats
|
||||
@@ -13,6 +19,7 @@ Built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data han
|
||||
- **Comprehensive data access**: Rates, ticks, account info, symbols, orders, positions, and trading history
|
||||
- **Flexible timeframes**: Named timeframes (M1, H1, D1, etc.) and numeric values
|
||||
- **Connection management**: Optional credentials, server, and timeout configuration
|
||||
- **SQLite rate loading**: Load mt5cli-managed rate tables/views for offline workflows
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -20,7 +27,65 @@ Built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data han
|
||||
pip install -U mt5cli MetaTrader5
|
||||
```
|
||||
|
||||
## Usage
|
||||
## Python API (downstream packages)
|
||||
|
||||
Import `MT5Client` for generic MT5 data access, schema normalization, and optional order primitives. `Mt5CliClient` remains available as a backward-compatible alias.
|
||||
|
||||
```python
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
|
||||
from mt5cli import (
|
||||
DataKind,
|
||||
Dataset,
|
||||
MT5Client,
|
||||
build_config,
|
||||
collect_history,
|
||||
export_dataframe,
|
||||
mt5_session,
|
||||
normalize_dataframe,
|
||||
update_history_with_config,
|
||||
)
|
||||
|
||||
# Persistent session for multiple calls
|
||||
with mt5_session(build_config(login=12345, server="Broker-Demo")) as client:
|
||||
rates = client.copy_rates_range(
|
||||
"EURUSD",
|
||||
timeframe="H1",
|
||||
date_from="2024-01-01",
|
||||
date_to="2024-02-01",
|
||||
)
|
||||
positions = client.positions()
|
||||
check = client.order_check({"action": 1, "symbol": "EURUSD", "volume": 0.1})
|
||||
|
||||
# Normalize MT5 frames to the public schema contract before storage
|
||||
closed_rates = normalize_dataframe(
|
||||
rates, DataKind.rates, symbol="EURUSD", timeframe="H1"
|
||||
)
|
||||
export_dataframe(closed_rates, Path("rates.csv"), "csv")
|
||||
|
||||
# Bulk SQLite history (same behavior as collect-history CLI command)
|
||||
collect_history(
|
||||
Path("history.db"),
|
||||
symbols=["EURUSD"],
|
||||
date_from=datetime(2024, 1, 1, tzinfo=UTC),
|
||||
date_to=datetime(2024, 2, 1, tzinfo=UTC),
|
||||
datasets={Dataset.rates, Dataset.history_deals},
|
||||
)
|
||||
|
||||
# Incremental append for automated pipelines
|
||||
update_history_with_config(
|
||||
output="history.db",
|
||||
symbols=["EURUSD"],
|
||||
config=build_config(login=12345),
|
||||
)
|
||||
```
|
||||
|
||||
Schema contracts live in `mt5cli.schemas` (`DataKind`, `validate_schema`, `normalize_dataframe`). Storage helpers are re-exported from `mt5cli.storage` and the package root.
|
||||
|
||||
`MT5Client.order_send()` is a live execution primitive: it can place real trades on the connected account. mt5cli does not implement strategy logic, signal generation, backtesting, or optimization — downstream applications must gate live execution explicitly.
|
||||
|
||||
## CLI usage
|
||||
|
||||
```bash
|
||||
# Export account information to CSV
|
||||
@@ -50,28 +115,33 @@ python -m mt5cli -o account.csv account-info
|
||||
|
||||
## Commands
|
||||
|
||||
| Command | Description |
|
||||
| ------------------ | ------------------------------------------------------------------------------------------------------------ |
|
||||
| `rates-from` | Export rates from a start date |
|
||||
| `rates-from-pos` | Export rates from a start position |
|
||||
| `rates-range` | Export rates for a date range |
|
||||
| `ticks-from` | Export ticks from a start date |
|
||||
| `ticks-range` | Export ticks for a date range |
|
||||
| `account-info` | Export account information |
|
||||
| `terminal-info` | Export terminal information |
|
||||
| `version` | Export MetaTrader 5 version information |
|
||||
| `last-error` | Export the last error information |
|
||||
| `symbols` | Export symbol list |
|
||||
| `symbol-info` | Export symbol details |
|
||||
| `symbol-info-tick` | Export the last tick for a symbol |
|
||||
| `market-book` | Export market depth (order book) |
|
||||
| `orders` | Export active orders |
|
||||
| `positions` | Export open positions |
|
||||
| `history-orders` | Export historical orders |
|
||||
| `history-deals` | Export historical deals |
|
||||
| `order-check` | Check funds sufficiency for a trade request |
|
||||
| `order-send` | Send a trade request to the trade server (`--yes` required) |
|
||||
| `collect-history` | Bundle rates, ticks, history-orders, and history-deals for one or more symbols into a single SQLite database |
|
||||
| Command | Description |
|
||||
| ---------------------- | ------------------------------------------------------------------------------------------------------------ |
|
||||
| `rates-from` | Export rates from a start date |
|
||||
| `rates-from-pos` | Export rates from a start position |
|
||||
| `latest-rates` | Export latest rates from a start position |
|
||||
| `rates-range` | Export rates for a date range |
|
||||
| `ticks-from` | Export ticks from a start date |
|
||||
| `ticks-range` | Export ticks for a date range |
|
||||
| `ticks-recent` | Export ticks from a recent trailing window |
|
||||
| `account-info` | Export account information |
|
||||
| `terminal-info` | Export terminal information |
|
||||
| `version` | Export MetaTrader 5 version information |
|
||||
| `last-error` | Export the last error information |
|
||||
| `symbols` | Export symbol list |
|
||||
| `symbol-info` | Export symbol details |
|
||||
| `symbol-info-tick` | Export the last tick for a symbol |
|
||||
| `minimum-margins` | Export minimum-volume buy and sell margin requirements |
|
||||
| `market-book` | Export market depth (order book) |
|
||||
| `orders` | Export active orders |
|
||||
| `positions` | Export open positions |
|
||||
| `history-orders` | Export historical orders |
|
||||
| `history-deals` | Export historical deals |
|
||||
| `recent-history-deals` | Export historical deals from a recent trailing window |
|
||||
| `mt5-summary` | Export terminal/account status summary |
|
||||
| `order-check` | Check funds sufficiency for a trade request |
|
||||
| `order-send` | Send a trade request to the trade server (`--yes` required) |
|
||||
| `collect-history` | Bundle rates, ticks, history-orders, and history-deals for one or more symbols into a single SQLite database |
|
||||
|
||||
Use `order-check` to validate a request payload before running `order-send --yes`.
|
||||
|
||||
@@ -127,6 +197,31 @@ update_history_with_config(
|
||||
- **`update_history`**: incremental append based on existing SQLite `MAX(time)` per symbol (and timeframe for rates); account-level deals use a separate cursor when `include_account_events=True`.
|
||||
- **`rates` table**: normalized storage with `symbol` and `timeframe` columns.
|
||||
- **Rate compatibility views**: mt5cli manages all `rate_*` views. Naming is `rate_<symbol>__<timeframe>` when a symbol has one timeframe, otherwise `rate_<symbol>__<granularity>_<timeframe>` (for example `rate_EURUSD__M1_1`). Stale `rate_*` views are dropped and recreated when rates change for offline tools such as mteor optimize.
|
||||
- **Rate view resolution**: use `resolve_rate_view_name()` / `resolve_rate_view_names()` to map symbols and granularities to existing SQLite compatibility views without creating databases. Both accept `None` (or a missing path) and return deterministic default names unless `require_existing=True`.
|
||||
- **Rate view loading**: use `load_rate_data()` / `load_rate_data_from_connection()` to load a SQLite rate table or view into a `DatetimeIndex` DataFrame.
|
||||
- **Multi-series rate loading**: use `build_rate_targets()` to build neutral `RateTarget(symbol, timeframe)` pairs, `resolve_rate_tables()` to map them to table/view names (pass `require_existing=True` for strict resolution), and `load_rate_series_from_sqlite()` to load them into a mapping keyed by `(symbol, integer timeframe)`. The loader requires existing managed views unless `explicit_tables` is supplied, and rejects duplicate `(symbol, timeframe)` targets.
|
||||
- **Multi-account latest rates**: use `collect_latest_rates_for_accounts()` with `AccountSpec` to read the latest bars for several account groups, merged into a `(symbol, integer timeframe)` mapping. For long-running pollers, `collect_latest_rates_for_accounts_with_retries()` adds bounded exponential backoff that retries only `pdmt5.Mt5TradingError` / `pdmt5.Mt5RuntimeError` and re-raises once `retry_count` is exhausted.
|
||||
- **Latest closed bars**: use `collect_latest_closed_rates_for_accounts()` when downstream logic must exclude the still-forming current bar. It fetches `count + 1` bars at `start_pos=0`, drops the last row with `drop_forming_rate_bar()`, and validates each series is non-empty. `collect_latest_closed_rates_by_granularity()` returns the same data keyed by `(symbol, granularity_name)` such as `("EURUSD", "M1")`.
|
||||
|
||||
```python
|
||||
from mt5cli import AccountSpec, collect_latest_closed_rates_by_granularity
|
||||
|
||||
rates = collect_latest_closed_rates_by_granularity(
|
||||
[AccountSpec(symbols=["EURUSD", "GBPUSD"], login=12345)],
|
||||
["M1", "H1"],
|
||||
count=500,
|
||||
retry_count=3,
|
||||
)
|
||||
eurusd_m1 = rates["EURUSD", "M1"] # closed bars only
|
||||
```
|
||||
|
||||
- **Credential resolution**: use `resolve_account_spec()` / `resolve_account_specs()` to merge explicit override values over `AccountSpec` fields and expand `${ENV_VAR}` placeholders (via `substitute_env_placeholders()`), raising `ValueError` for missing variables. This keeps secrets out of plan/config files without coupling to any strategy code.
|
||||
- **Throttled history updates**: use `ThrottledHistoryUpdater` to wrap `update_history()` with a minimum `interval_seconds` between successful runs (monotonic clock). Call `should_update()` / `update(client, symbols)` from an application loop; errors propagate by default, or pass `suppress_errors=True` to swallow recoverable `Mt5*Error`, `sqlite3.Error`, `ValueError`, `OSError`, and MT5 client capability errors for history API methods without advancing the throttle (other `AttributeError` / `TypeError` values always propagate).
|
||||
- **Trading session helpers**: use `mt5_trading_session()` for a trading-capable `pdmt5.Mt5TradingClient` that initializes/logs in via `Mt5Config.path` and always shuts down safely. Pair with `detect_position_side()`, `calculate_margin_and_volume()`, and `determine_order_limits()` for generic position and sizing utilities. The read-only `mt5_session()` / `Mt5CliClient` SDK is unchanged.
|
||||
- **Granularity-keyed rate loading**: `load_rate_series_by_granularity()` builds targets with `build_rate_targets()`, loads them with `load_rate_series_from_sqlite()`, and returns a mapping keyed by `(symbol | None, granularity_name)` such as `("EURUSD", "M1")` to reduce downstream boilerplate.
|
||||
- **MT5 session helper**: use the `mt5_session()` context manager to attach to (or, when `Mt5Config.path` is set, launch) an MT5 terminal, log in, and yield a connected `MT5Client` that shuts down on exit.
|
||||
- **SQLite export helpers**: use `export_dataframe_to_sqlite()` for append mode, optional index export, and post-write deduplication by key columns.
|
||||
- **Recent ticks and margins**: `recent_ticks()` and `minimum_margins()` SDK helpers (and matching CLI commands) cover common downstream read-only queries.
|
||||
|
||||
## Requirements
|
||||
|
||||
@@ -134,6 +229,63 @@ update_history_with_config(
|
||||
- Windows OS (MetaTrader 5 requirement)
|
||||
- MetaTrader 5 platform installed
|
||||
|
||||
### Migration note for mteor
|
||||
|
||||
Replace local MT5 lifecycle and trading helper code with mt5cli imports:
|
||||
|
||||
```python
|
||||
# Before (local mteor helpers)
|
||||
# with local_mt5_trading_session(config) as client:
|
||||
# side = local_detect_position_side(client, symbol)
|
||||
# sizing = local_calculate_margin_and_volume(client, symbol, unit_ratio, preserved_ratio)
|
||||
# limits = local_determine_order_limits(client, symbol, side, sl_ratio, tp_ratio)
|
||||
|
||||
# After (mt5cli shared layer)
|
||||
from pdmt5 import Mt5Config
|
||||
from mt5cli import (
|
||||
calculate_margin_and_volume,
|
||||
detect_position_side,
|
||||
determine_order_limits,
|
||||
mt5_trading_session,
|
||||
)
|
||||
|
||||
with mt5_trading_session(
|
||||
Mt5Config(path=terminal_path, login=login), retry_count=2
|
||||
) as client:
|
||||
side = detect_position_side(client, symbol)
|
||||
sizing = calculate_margin_and_volume(
|
||||
client, symbol, unit_margin_ratio=0.5, preserved_margin_ratio=0.2
|
||||
)
|
||||
if side is not None:
|
||||
limits = determine_order_limits(
|
||||
client,
|
||||
symbol,
|
||||
side,
|
||||
stop_loss_limit_ratio=0.01,
|
||||
take_profit_limit_ratio=0.02,
|
||||
)
|
||||
```
|
||||
|
||||
Throttled history updates use a separate read-only session:
|
||||
|
||||
```python
|
||||
from pdmt5 import Mt5Config, Mt5DataClient
|
||||
|
||||
from mt5cli import ThrottledHistoryUpdater
|
||||
|
||||
updater = ThrottledHistoryUpdater(
|
||||
output="history.db", interval_seconds=60, suppress_errors=True
|
||||
)
|
||||
client = Mt5DataClient(config=Mt5Config(login=login))
|
||||
client.initialize_and_login_mt5()
|
||||
try:
|
||||
updater.update(client, ["EURUSD"])
|
||||
finally:
|
||||
client.shutdown()
|
||||
```
|
||||
|
||||
Read-only collectors can keep using `mt5_session()` and `MT5Client` (or the `Mt5CliClient` alias) without changes.
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
# Client
|
||||
|
||||
::: mt5cli.client
|
||||
@@ -0,0 +1,3 @@
|
||||
# Converters
|
||||
|
||||
::: mt5cli.converters
|
||||
@@ -0,0 +1,3 @@
|
||||
# Exceptions
|
||||
|
||||
::: mt5cli.exceptions
|
||||
@@ -129,3 +129,101 @@ when required columns are missing.
|
||||
The `update_history` SDK path uses the same base tables and optional
|
||||
`cash_events` / `positions_reconstructed` views. It additionally maintains
|
||||
`rate_<symbol>__<timeframe>` compatibility views when `create_rate_views=True`.
|
||||
|
||||
### Rate view resolution
|
||||
|
||||
Downstream tools can resolve mt5cli-managed compatibility view names from an
|
||||
existing SQLite history database without creating files or guessing naming
|
||||
schemes:
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
|
||||
from mt5cli.history import resolve_rate_view_name, resolve_rate_view_names
|
||||
|
||||
# Single symbol and granularity
|
||||
view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1")
|
||||
|
||||
# Batch resolution in row-major order
|
||||
views = resolve_rate_view_names(
|
||||
Path("history.db"),
|
||||
["EURUSD", "GBPUSD"],
|
||||
["M1", "H1"],
|
||||
)
|
||||
```
|
||||
|
||||
Resolution rules:
|
||||
|
||||
- Returns `rate_<symbol>__<timeframe>` when a symbol stores one timeframe.
|
||||
- Returns `rate_<symbol>__<granularity>_<timeframe>` when multiple timeframes
|
||||
are stored for the same symbol.
|
||||
- When multiple naming candidates apply, prefers an existing managed
|
||||
`rate_*__*` view from the candidate list.
|
||||
- Falls back to single-timeframe naming when the database path is missing or
|
||||
`rates` metadata is unavailable.
|
||||
- Pass `require_existing=True` to raise `ValueError` instead of returning a
|
||||
best-guess name when the database or view is missing.
|
||||
- Accepts either a SQLite path or an open `sqlite3.Connection`.
|
||||
|
||||
### Rate data loading
|
||||
|
||||
Use `load_rate_data()` to load a table or view from a SQLite path, or
|
||||
`load_rate_data_from_connection()` when you already have a connection:
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
|
||||
from mt5cli import load_rate_data
|
||||
from mt5cli.history import resolve_rate_view_name
|
||||
|
||||
view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1", require_existing=True)
|
||||
rates = load_rate_data(Path("history.db"), view, count=1000)
|
||||
```
|
||||
|
||||
The loader accepts close-based OHLC rate data or tick-like bid/ask data. It
|
||||
validates that `time` exists, parses timestamps with pandas, and returns a
|
||||
DataFrame indexed by ascending `DatetimeIndex` named `time`.
|
||||
|
||||
### Multi-series rate loading
|
||||
|
||||
For loading many rate series at once, build neutral `RateTarget` pairs and load
|
||||
them from SQLite in one call. View names are resolved via the same
|
||||
compatibility-view rules, or you can pass `explicit_tables` to bypass resolution:
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
|
||||
from mt5cli import build_rate_targets, load_rate_series_from_sqlite
|
||||
|
||||
targets = build_rate_targets(["EURUSD", "GBPUSD"], ["M1", "H1"])
|
||||
series = load_rate_series_from_sqlite(Path("history.db"), targets, count=1000)
|
||||
frame = series["EURUSD", 1] # keyed by (symbol, integer timeframe)
|
||||
```
|
||||
|
||||
- `build_rate_targets()` returns `RateTarget(symbol, timeframe)` pairs in
|
||||
row-major order, normalizing timeframe names such as `"M1"` to their integer
|
||||
values; set `allow_missing_symbol=True` to address series solely by
|
||||
`explicit_tables` (targets carry `symbol=None`).
|
||||
- `resolve_rate_tables()` maps targets to table or view names and validates that
|
||||
any `explicit_tables` count matches the target count. Pass
|
||||
`require_existing=True` to raise `ValueError` instead of returning a
|
||||
best-guess name when the database or managed view is missing. When
|
||||
`explicit_tables` is provided, names are returned as-is and
|
||||
`require_existing` is ignored.
|
||||
- `load_rate_series_from_sqlite()` returns a mapping keyed by
|
||||
`(symbol, integer timeframe)`. Unless `explicit_tables` is supplied, it
|
||||
requires existing managed `rate_*` compatibility views and raises
|
||||
`ValueError` when they are missing. Duplicate `(symbol, timeframe)` targets
|
||||
are rejected.
|
||||
- `load_rate_series_by_granularity()` is a thin wrapper that builds the targets,
|
||||
loads the series, and rekeys the result by granularity name to avoid
|
||||
converting integer timeframes downstream:
|
||||
|
||||
```python
|
||||
from mt5cli import load_rate_series_by_granularity
|
||||
|
||||
series = load_rate_series_by_granularity(
|
||||
"history.db", ["EURUSD"], ["M1", "H1"], count=1000
|
||||
)
|
||||
frame = series["EURUSD", "M1"] # keyed by (symbol | None, granularity_name)
|
||||
```
|
||||
|
||||
+39
-86
@@ -1,100 +1,53 @@
|
||||
# API Reference
|
||||
|
||||
This section contains the complete API documentation for mt5cli.
|
||||
This section documents the mt5cli public Python API and CLI modules.
|
||||
|
||||
## Modules
|
||||
## Public API layers
|
||||
|
||||
The mt5cli package consists of the following modules:
|
||||
| Module | Purpose |
|
||||
| ----------------------------------------- | ------------------------------------------------------------------------- |
|
||||
| [Client](client.md) | `MT5Client` session abstraction for data access and order primitives |
|
||||
| [Schemas](schemas.md) | Canonical DataFrame contracts and normalization helpers |
|
||||
| [Storage](storage.md) | CSV/JSON/Parquet/SQLite export and history collection helpers |
|
||||
| [Converters](converters.md) | Symbol, timeframe, timezone, and date-range utilities |
|
||||
| [Exceptions](exceptions.md) | Stable mt5cli exception types and MT5 error normalization |
|
||||
| [SDK](sdk.md) | Module-level fetch helpers, multi-account collectors, incremental history |
|
||||
| [Trading](trading.md) | Trading-capable sessions and operational helpers |
|
||||
| [History Collection (SQLite)](history.md) | SQLite schema, incremental writes, dedup, and rate views |
|
||||
| [CLI](cli.md) | Typer commands that delegate to the Python API |
|
||||
| [Utils](utils.md) | Parsing helpers and Click parameter types |
|
||||
|
||||
### [CLI](cli.md)
|
||||
## Architecture overview
|
||||
|
||||
Command-line interface module providing typer-based commands for exporting MetaTrader 5 data to CSV, JSON, Parquet, and SQLite3 formats.
|
||||
|
||||
### [Utils](utils.md)
|
||||
|
||||
Utility module providing constants, enums, Click parameter types, and helper functions for parsing and exporting data.
|
||||
|
||||
### [SDK](sdk.md)
|
||||
|
||||
Programmatic SDK for read-only MetaTrader 5 data collection. Returns pandas DataFrames and provides `collect_history` for SQLite bulk collection.
|
||||
|
||||
### [History Collection (SQLite)](history.md)
|
||||
|
||||
SQLite storage helpers for the `collect-history` command schema, incremental updates, deduplication, indexes, and optional views.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
The package follows a simple architecture built on top of pdmt5:
|
||||
|
||||
1. **CLI Layer** (`cli.py`): Typer application with subcommands that delegate to the SDK and export results.
|
||||
2. **SDK Layer** (`sdk.py`): Read-only data access functions, `Mt5CliClient`, and `collect_history` orchestration.
|
||||
3. **Utils Layer** (`utils.py`): Constants, enums, custom Click parameter types, parsing helpers, and format detection/export utilities.
|
||||
4. **Data Layer** (via `pdmt5`): Uses `Mt5DataClient` and `Mt5Config` from the pdmt5 package for all MetaTrader 5 data access.
|
||||
|
||||
## Usage Guidelines
|
||||
|
||||
All modules follow these conventions:
|
||||
|
||||
- **Type Safety**: All functions include comprehensive type hints
|
||||
- **Error Handling**: User-friendly error messages via typer
|
||||
- **Documentation**: Google-style docstrings with examples
|
||||
- **Validation**: Custom Click parameter types for input validation
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Export account information to CSV
|
||||
mt5cli -o account.csv account-info
|
||||
|
||||
# Export EURUSD H1 rates to Parquet
|
||||
mt5cli -o rates.parquet rates-from --symbol EURUSD --timeframe H1 \
|
||||
--date-from 2024-01-01 --count 1000
|
||||
|
||||
# Export ticks to JSON
|
||||
mt5cli -o ticks.json ticks-from --symbol EURUSD \
|
||||
--date-from 2024-01-01 --count 500 --flags ALL
|
||||
|
||||
# Export to SQLite3 with custom table name
|
||||
mt5cli -o data.db --table symbols symbols --group "*USD*"
|
||||
```mermaid
|
||||
flowchart TD
|
||||
App["Downstream application"] --> Client["MT5Client"]
|
||||
CLI["mt5cli CLI"] --> Client
|
||||
Client --> SDK["sdk / pdmt5"]
|
||||
Client --> Schemas["schemas"]
|
||||
Storage["storage"] --> History["history SQLite"]
|
||||
Storage --> Utils["utils export"]
|
||||
SDK --> PDMT5["pdmt5.Mt5DataClient"]
|
||||
```
|
||||
|
||||
## Python API
|
||||
Downstream packages should depend on the package root exports (`MT5Client`, `DataKind`, `normalize_dataframe`, `export_dataframe`, `collect_history`, etc.) rather than private modules.
|
||||
|
||||
`MT5Client.order_send()` is a live execution primitive that can place real trades. mt5cli exposes minimal execution helpers only; strategy logic, signals, backtests, and optimization remain out of scope and must be implemented downstream with explicit execution gating.
|
||||
|
||||
## Quick start
|
||||
|
||||
```python
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from mt5cli import MT5Client, build_config, mt5_session
|
||||
|
||||
from mt5cli import (
|
||||
Mt5CliClient,
|
||||
collect_history,
|
||||
copy_rates_range,
|
||||
detect_format,
|
||||
export_dataframe,
|
||||
)
|
||||
|
||||
# Fetch rates programmatically
|
||||
rates = copy_rates_range(
|
||||
"EURUSD",
|
||||
timeframe="H1",
|
||||
date_from="2024-01-01",
|
||||
date_to="2024-02-01",
|
||||
)
|
||||
|
||||
# Detect output format from file extension
|
||||
fmt = detect_format(Path("output.parquet")) # Returns "parquet"
|
||||
|
||||
# Export a DataFrame
|
||||
export_dataframe(rates, Path("output.csv"), "csv")
|
||||
|
||||
# Collect history into SQLite
|
||||
collect_history(
|
||||
Path("history.db"),
|
||||
symbols=["EURUSD"],
|
||||
date_from=datetime(2024, 1, 1, tzinfo=UTC),
|
||||
date_to=datetime(2024, 2, 1, tzinfo=UTC),
|
||||
)
|
||||
with mt5_session(build_config(login=12345)) as client:
|
||||
rates = client.copy_rates_range("EURUSD", "H1", "2024-01-01", "2024-02-01")
|
||||
positions = client.positions()
|
||||
```
|
||||
|
||||
## Examples
|
||||
```bash
|
||||
mt5cli -o account.csv account-info
|
||||
mt5cli -o rates.parquet rates-range --symbol EURUSD --timeframe H1 \
|
||||
--date-from 2024-01-01 --date-to 2024-02-01
|
||||
```
|
||||
|
||||
See individual module pages for detailed usage examples and code samples.
|
||||
See individual module pages for detailed usage examples.
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
# Schemas
|
||||
|
||||
::: mt5cli.schemas
|
||||
+111
@@ -1,3 +1,114 @@
|
||||
# SDK Module
|
||||
|
||||
::: mt5cli.sdk
|
||||
|
||||
## Resilient multi-account orchestration
|
||||
|
||||
The SDK ships strategy-agnostic helpers for building long-running collectors on
|
||||
top of the read-only client. None of them depend on a particular trading
|
||||
application.
|
||||
|
||||
### Retrying transient rate collection
|
||||
|
||||
`collect_latest_rates_for_accounts_with_retries()` wraps
|
||||
`collect_latest_rates_for_accounts()` with bounded exponential backoff. Only
|
||||
`pdmt5.Mt5TradingError` and `pdmt5.Mt5RuntimeError` are retried; the final
|
||||
failure is re-raised once `retry_count` is exhausted.
|
||||
|
||||
```python
|
||||
from mt5cli import AccountSpec, collect_latest_rates_for_accounts_with_retries
|
||||
|
||||
accounts = [AccountSpec(symbols=["EURUSD"], login=12345)]
|
||||
rates = collect_latest_rates_for_accounts_with_retries(
|
||||
accounts,
|
||||
["M1", "H1"],
|
||||
count=500,
|
||||
retry_count=3,
|
||||
backoff_base=2, # sleeps 2s, 4s, 8s between attempts
|
||||
)
|
||||
```
|
||||
|
||||
### Latest closed rate bars
|
||||
|
||||
MetaTrader 5 `start_pos=0` includes the still-forming current bar as the last
|
||||
row. `collect_latest_closed_rates_for_accounts()` fetches `count + 1` bars,
|
||||
drops that row with `drop_forming_rate_bar()`, and validates each series is
|
||||
non-empty. Use `collect_latest_closed_rates_by_granularity()` when callers
|
||||
prefer keys such as `("EURUSD", "M1")` instead of integer timeframes.
|
||||
|
||||
```python
|
||||
from mt5cli import AccountSpec, collect_latest_closed_rates_by_granularity
|
||||
|
||||
rates = collect_latest_closed_rates_by_granularity(
|
||||
[AccountSpec(symbols=["EURUSD"], login=12345)],
|
||||
["M1", "H1"],
|
||||
count=500,
|
||||
retry_count=3,
|
||||
)
|
||||
closed_m1 = rates["EURUSD", "M1"]
|
||||
```
|
||||
|
||||
### Resolving credentials and `${ENV_VAR}` placeholders
|
||||
|
||||
`resolve_account_spec()` / `resolve_account_specs()` merge explicit override
|
||||
values over `AccountSpec` fields and expand `${ENV_VAR}` placeholders, keeping
|
||||
secrets out of plan/config files. A missing environment variable raises
|
||||
`ValueError`.
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
from mt5cli import AccountSpec, resolve_account_specs
|
||||
|
||||
os.environ["MT5_LOGIN"] = "12345"
|
||||
os.environ["MT5_PASSWORD"] = "secret"
|
||||
accounts = [
|
||||
AccountSpec(symbols=["EURUSD"], login="${MT5_LOGIN}", password="${MT5_PASSWORD}")
|
||||
]
|
||||
|
||||
resolved = resolve_account_specs(accounts, server="Broker-Demo")
|
||||
# resolved[0].login == "12345", resolved[0].server == "Broker-Demo"
|
||||
```
|
||||
|
||||
### Throttled incremental history updates
|
||||
|
||||
`ThrottledHistoryUpdater` wraps `update_history()` with a minimum interval
|
||||
between successful runs (using a monotonic clock), so an application loop can
|
||||
call it every iteration without over-fetching.
|
||||
|
||||
```python
|
||||
from pdmt5 import Mt5Config, Mt5DataClient
|
||||
|
||||
from mt5cli import Dataset, ThrottledHistoryUpdater
|
||||
|
||||
updater = ThrottledHistoryUpdater(
|
||||
output="history.db",
|
||||
datasets={Dataset.rates},
|
||||
timeframes=["M1"],
|
||||
interval_seconds=60, # <= 0 updates on every call
|
||||
)
|
||||
|
||||
client = Mt5DataClient(config=Mt5Config(login=12345))
|
||||
client.initialize_and_login_mt5()
|
||||
try:
|
||||
while True:
|
||||
updater.update(client, ["EURUSD", "GBPUSD"]) # no-op until 60s elapse
|
||||
# ... do other work; break when shutting down ...
|
||||
finally:
|
||||
client.shutdown()
|
||||
```
|
||||
|
||||
By default recoverable errors (`Mt5TradingError`, `Mt5RuntimeError`,
|
||||
`sqlite3.Error`, `ValueError`, `OSError`, and MT5 client capability
|
||||
`AttributeError` / `TypeError` for history API methods) propagate so the caller
|
||||
controls logging; pass `suppress_errors=True` to swallow them and return
|
||||
`False` without advancing the throttle. Other `AttributeError` / `TypeError`
|
||||
values always propagate. Input validation (`_resolve_update_history_request`)
|
||||
runs before any MT5 or SQLite calls, but when `suppress_errors=True` the
|
||||
resulting `ValueError` is suppressed along with other recoverable errors.
|
||||
|
||||
## Trading-capable sessions
|
||||
|
||||
For order placement and trading calculations, use the dedicated
|
||||
[Trading module](trading.md). The read-only `Mt5CliClient` and `mt5_session()`
|
||||
helpers in this module are unchanged.
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
# Storage
|
||||
|
||||
::: mt5cli.storage
|
||||
@@ -0,0 +1,70 @@
|
||||
# Trading Module
|
||||
|
||||
::: mt5cli.trading
|
||||
|
||||
## Trading-capable MT5 sessions
|
||||
|
||||
`mt5_trading_session()` complements the read-only `mt5_session()` helper in
|
||||
`sdk.py`. It yields a connected `pdmt5.Mt5TradingClient`, uses
|
||||
`Mt5Config.path` to launch the terminal when configured, and always calls
|
||||
`shutdown()` on exit.
|
||||
|
||||
```python
|
||||
from pdmt5 import Mt5Config
|
||||
|
||||
from mt5cli import mt5_trading_session
|
||||
|
||||
with mt5_trading_session(
|
||||
Mt5Config(path=r"C:\Program Files\MetaTrader 5\terminal64.exe", login=12345),
|
||||
retry_count=2,
|
||||
) as client:
|
||||
positions = client.positions_get_as_df(symbol="EURUSD")
|
||||
```
|
||||
|
||||
The read-only `Mt5CliClient` / `mt5_session()` API is unchanged.
|
||||
|
||||
## Operational trading helpers
|
||||
|
||||
These helpers are strategy-agnostic and do not depend on signal detection,
|
||||
betting logic, or scheduling code in downstream applications.
|
||||
|
||||
```python
|
||||
from mt5cli import (
|
||||
calculate_margin_and_volume,
|
||||
detect_position_side,
|
||||
determine_order_limits,
|
||||
)
|
||||
|
||||
side = detect_position_side(client, "EURUSD")
|
||||
sizing = calculate_margin_and_volume(
|
||||
client,
|
||||
"EURUSD",
|
||||
unit_margin_ratio=0.5,
|
||||
preserved_margin_ratio=0.2,
|
||||
)
|
||||
limits = determine_order_limits(
|
||||
client,
|
||||
"EURUSD",
|
||||
side="long",
|
||||
stop_loss_limit_ratio=0.01,
|
||||
take_profit_limit_ratio=0.02,
|
||||
)
|
||||
```
|
||||
|
||||
Protective ratios must satisfy `0 <= ratio < 1`; `0` omits that level.
|
||||
`calculate_margin_and_volume()` clamps negative `margin_free` to `0.0`
|
||||
before sizing.
|
||||
|
||||
## Migration from mteor-local helpers
|
||||
|
||||
| mteor-local concern | mt5cli replacement |
|
||||
| -------------------------------------------------------- | ----------------------------------------------- |
|
||||
| Manual terminal spawn/kill around trading code | `mt5_trading_session()` |
|
||||
| Local position-side detection | `detect_position_side()` |
|
||||
| Local margin/volume sizing | `calculate_margin_and_volume()` |
|
||||
| Local SL/TP price derivation | `determine_order_limits()` |
|
||||
| Throttled SQLite history loop with ad-hoc error handling | `ThrottledHistoryUpdater(suppress_errors=True)` |
|
||||
|
||||
Keep read-only data collection on `mt5_session()` / `Mt5CliClient`; use
|
||||
`mt5_trading_session()` only where order placement or trading calculations are
|
||||
required.
|
||||
+70
-33
@@ -1,10 +1,16 @@
|
||||
# mt5cli
|
||||
|
||||
Command-line tool for MetaTrader 5 data export.
|
||||
Generic MT5 data and execution infrastructure for Python applications.
|
||||
|
||||
## Overview
|
||||
|
||||
mt5cli is a CLI application that exports MetaTrader 5 trading data to multiple file formats. It is built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data handler for MetaTrader 5.
|
||||
mt5cli provides a stable `MT5Client` Python API, standardized dataset schemas, storage helpers, and a CLI for exporting MetaTrader 5 data. It is built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data handler for MetaTrader 5.
|
||||
|
||||
## Architecture
|
||||
|
||||
- **pdmt5** — canonical MT5 client, DataFrame/trading primitives, and MT5 constant parsing (`TIMEFRAME_*`, `COPY_TICKS_*`, order types).
|
||||
- **mt5cli** — public `MT5Client` API, schema contracts, storage helpers, CLI commands, and SQLite history collection built on pdmt5.
|
||||
- **mt5api** — sibling HTTP adapter for remote MT5 access; not a dependency of mt5cli.
|
||||
|
||||
## Features
|
||||
|
||||
@@ -13,6 +19,7 @@ mt5cli is a CLI application that exports MetaTrader 5 trading data to multiple f
|
||||
- **Comprehensive data access**: Rates, ticks, account info, symbols, orders, positions, and trading history
|
||||
- **Flexible timeframes**: Named timeframes (M1, H1, D1, etc.) and numeric values
|
||||
- **Connection management**: Optional credentials, server, and timeout configuration
|
||||
- **SQLite rate loading**: Load mt5cli-managed rate tables/views for offline workflows
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -20,43 +27,68 @@ mt5cli is a CLI application that exports MetaTrader 5 trading data to multiple f
|
||||
pip install mt5cli
|
||||
```
|
||||
|
||||
## Programmatic usage / SDK usage
|
||||
## Python API for downstream packages
|
||||
|
||||
mt5cli can be used as a small Python SDK for read-only MetaTrader 5 data collection. SDK functions return pandas DataFrames without writing files. Use `export_dataframe` when you need to persist results.
|
||||
Import `MT5Client` for generic MT5 data access, schema normalization, and optional order primitives. `Mt5CliClient` remains available as a backward-compatible alias.
|
||||
|
||||
```python
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
|
||||
from mt5cli import Mt5CliClient, collect_history, copy_rates_range, export_dataframe
|
||||
|
||||
# One-off fetch with module-level helpers
|
||||
rates = copy_rates_range(
|
||||
"EURUSD",
|
||||
timeframe="H1",
|
||||
date_from="2024-01-01",
|
||||
date_to="2024-02-01",
|
||||
from mt5cli import (
|
||||
DataKind,
|
||||
Dataset,
|
||||
MT5Client,
|
||||
build_config,
|
||||
collect_history,
|
||||
export_dataframe,
|
||||
load_rate_data,
|
||||
minimum_margins,
|
||||
mt5_session,
|
||||
normalize_dataframe,
|
||||
recent_ticks,
|
||||
resolve_rate_view_name,
|
||||
)
|
||||
export_dataframe(rates, Path("rates.csv"), "csv")
|
||||
|
||||
# Reuse one MT5 connection for multiple calls
|
||||
with Mt5CliClient(login=12345, password="secret", server="Broker-Demo") as client:
|
||||
account = client.account_info()
|
||||
# Persistent session for multiple calls
|
||||
with mt5_session(build_config(login=12345, server="Broker-Demo")) as client:
|
||||
rates = client.copy_rates_range(
|
||||
"EURUSD",
|
||||
timeframe="H1",
|
||||
date_from="2024-01-01",
|
||||
date_to="2024-02-01",
|
||||
)
|
||||
positions = client.positions()
|
||||
check = client.order_check({"action": 1, "symbol": "EURUSD", "volume": 0.1})
|
||||
|
||||
# Normalize MT5 frames to the public schema contract before storage
|
||||
closed_rates = normalize_dataframe(
|
||||
rates, DataKind.rates, symbol="EURUSD", timeframe="H1"
|
||||
)
|
||||
export_dataframe(closed_rates, Path("rates.csv"), "csv")
|
||||
|
||||
# Offline rate loading from mt5cli-managed SQLite history
|
||||
view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1", require_existing=True)
|
||||
offline_rates = load_rate_data(Path("history.db"), view, count=1000)
|
||||
|
||||
# One-off helpers still work without instantiating a client
|
||||
ticks = recent_ticks("EURUSD", seconds=300)
|
||||
margins = minimum_margins("EURUSD")
|
||||
|
||||
# Bulk SQLite collection (same behavior as the collect-history CLI command)
|
||||
collect_history(
|
||||
Path("history.db"),
|
||||
symbols=["EURUSD", "GBPUSD"],
|
||||
date_from=datetime(2024, 1, 1, tzinfo=UTC),
|
||||
date_to=datetime(2024, 2, 1, tzinfo=UTC),
|
||||
timeframe="M1",
|
||||
flags="ALL",
|
||||
with_views=True,
|
||||
datasets={Dataset.rates, Dataset.history_deals},
|
||||
)
|
||||
```
|
||||
|
||||
Timeframes, tick flags, and ISO 8601 date strings are accepted wherever noted in the SDK API.
|
||||
Schema contracts live in `mt5cli.schemas` (`DataKind`, `validate_schema`, `normalize_dataframe`). Storage helpers are re-exported from `mt5cli.storage` and the package root.
|
||||
|
||||
`MT5Client.order_send()` is a live execution primitive: it can place real trades on the connected account. mt5cli does not implement strategy logic, signal generation, backtesting, or optimization — downstream applications must gate live execution explicitly (the CLI requires `--yes` for `order-send`).
|
||||
|
||||
`MT5Client.mt5_summary()` returns structured nested Python values. Use `MT5Client.mt5_summary_as_df()` when you need a one-row DataFrame for export.
|
||||
|
||||
## Quick Start
|
||||
|
||||
@@ -88,14 +120,16 @@ mt5cli --login 12345 --password mypass --server MyBroker-Demo \
|
||||
| ---------------- | ---------------------------------- |
|
||||
| `rates-from` | Export rates from a start date |
|
||||
| `rates-from-pos` | Export rates from a start position |
|
||||
| `latest-rates` | Export latest rates |
|
||||
| `rates-range` | Export rates for a date range |
|
||||
|
||||
### Ticks
|
||||
|
||||
| Command | Description |
|
||||
| ------------- | ------------------------------ |
|
||||
| `ticks-from` | Export ticks from a start date |
|
||||
| `ticks-range` | Export ticks for a date range |
|
||||
| Command | Description |
|
||||
| -------------- | ----------------------------------- |
|
||||
| `ticks-from` | Export ticks from a start date |
|
||||
| `ticks-range` | Export ticks for a date range |
|
||||
| `ticks-recent` | Export ticks from a trailing window |
|
||||
|
||||
### Information
|
||||
|
||||
@@ -108,18 +142,21 @@ mt5cli --login 12345 --password mypass --server MyBroker-Demo \
|
||||
| `symbols` | Export symbol list |
|
||||
| `symbol-info` | Export symbol details |
|
||||
| `symbol-info-tick` | Export the last tick for a symbol |
|
||||
| `minimum-margins` | Export minimum-volume margin summary |
|
||||
| `market-book` | Export market depth (order book) |
|
||||
|
||||
### Trading
|
||||
|
||||
| Command | Description |
|
||||
| ---------------- | ----------------------------------------------------------- |
|
||||
| `orders` | Export active orders |
|
||||
| `positions` | Export open positions |
|
||||
| `history-orders` | Export historical orders |
|
||||
| `history-deals` | Export historical deals |
|
||||
| `order-check` | Check funds sufficiency for a trade request |
|
||||
| `order-send` | Send a trade request to the trade server (`--yes` required) |
|
||||
| Command | Description |
|
||||
| ---------------------- | ----------------------------------------------------------- |
|
||||
| `orders` | Export active orders |
|
||||
| `positions` | Export open positions |
|
||||
| `history-orders` | Export historical orders |
|
||||
| `history-deals` | Export historical deals |
|
||||
| `recent-history-deals` | Export historical deals from a trailing window |
|
||||
| `mt5-summary` | Export terminal/account status summary |
|
||||
| `order-check` | Check funds sufficiency for a trade request |
|
||||
| `order-send` | Send a trade request to the trade server (`--yes` required) |
|
||||
|
||||
Use `order-check` to validate a request payload before running `order-send --yes`.
|
||||
|
||||
|
||||
+7
-1
@@ -1,5 +1,5 @@
|
||||
site_name: mt5cli API Documentation
|
||||
site_description: Command-line tool for MetaTrader 5
|
||||
site_description: Generic MT5 data and execution infrastructure for Python
|
||||
site_author: dceoy
|
||||
site_url: https://github.com/dceoy/mt5cli
|
||||
|
||||
@@ -56,8 +56,14 @@ nav:
|
||||
- Home: index.md
|
||||
- API Reference:
|
||||
- Overview: api/index.md
|
||||
- Client: api/client.md
|
||||
- Schemas: api/schemas.md
|
||||
- Storage: api/storage.md
|
||||
- Converters: api/converters.md
|
||||
- Exceptions: api/exceptions.md
|
||||
- CLI: api/cli.md
|
||||
- SDK: api/sdk.md
|
||||
- Trading: api/trading.md
|
||||
- History Collection (SQLite): api/history.md
|
||||
- Utils: api/utils.md
|
||||
|
||||
|
||||
+146
-3
@@ -1,12 +1,63 @@
|
||||
"""mt5cli: Command-line tool and SDK for MetaTrader 5."""
|
||||
"""mt5cli: Generic MT5 data and execution infrastructure for Python applications."""
|
||||
|
||||
from importlib.metadata import version
|
||||
|
||||
from .client import MT5Client, build_config, mt5_session
|
||||
from .converters import (
|
||||
ensure_utc,
|
||||
granularity_name,
|
||||
normalize_symbol,
|
||||
normalize_symbols,
|
||||
parse_date_range,
|
||||
recent_window,
|
||||
)
|
||||
from .exceptions import (
|
||||
Mt5CliError,
|
||||
Mt5ConnectionError,
|
||||
Mt5OperationError,
|
||||
Mt5SchemaError,
|
||||
call_with_normalized_errors,
|
||||
is_recoverable_mt5_error,
|
||||
normalize_mt5_exception,
|
||||
)
|
||||
from .history import (
|
||||
RateTarget,
|
||||
build_rate_targets,
|
||||
build_rate_view_name,
|
||||
drop_forming_rate_bar,
|
||||
load_rate_data,
|
||||
load_rate_data_from_connection,
|
||||
load_rate_series_by_granularity,
|
||||
load_rate_series_from_sqlite,
|
||||
resolve_history_datasets,
|
||||
resolve_history_tick_flags,
|
||||
resolve_history_timeframes,
|
||||
resolve_rate_tables,
|
||||
resolve_rate_view_name,
|
||||
resolve_rate_view_names,
|
||||
)
|
||||
from .schemas import (
|
||||
DEDUP_KEYS,
|
||||
KNOWN_MT5_TIME_COLUMNS,
|
||||
REQUIRED_COLUMNS,
|
||||
TIME_COLUMNS,
|
||||
DataKind,
|
||||
normalize_dataframe,
|
||||
normalize_time_columns,
|
||||
schema_columns,
|
||||
validate_schema,
|
||||
)
|
||||
from .sdk import (
|
||||
AccountSpec,
|
||||
Mt5CliClient,
|
||||
ThrottledHistoryUpdater,
|
||||
account_info,
|
||||
build_config,
|
||||
collect_history,
|
||||
collect_latest_closed_rates_by_granularity,
|
||||
collect_latest_closed_rates_for_accounts,
|
||||
collect_latest_rates,
|
||||
collect_latest_rates_for_accounts,
|
||||
collect_latest_rates_for_accounts_with_retries,
|
||||
copy_rates_from,
|
||||
copy_rates_from_pos,
|
||||
copy_rates_range,
|
||||
@@ -15,9 +66,18 @@ from .sdk import (
|
||||
history_deals,
|
||||
history_orders,
|
||||
last_error,
|
||||
latest_rates,
|
||||
market_book,
|
||||
minimum_margins,
|
||||
mt5_summary,
|
||||
mt5_summary_as_df,
|
||||
orders,
|
||||
positions,
|
||||
recent_history_deals,
|
||||
recent_ticks,
|
||||
resolve_account_spec,
|
||||
resolve_account_specs,
|
||||
substitute_env_placeholders,
|
||||
symbol_info,
|
||||
symbol_info_tick,
|
||||
symbols,
|
||||
@@ -28,35 +88,118 @@ from .sdk import (
|
||||
from .sdk import (
|
||||
version as mt5_version,
|
||||
)
|
||||
from .utils import Dataset, IfExists, detect_format, export_dataframe
|
||||
from .storage import (
|
||||
Dataset,
|
||||
IfExists,
|
||||
detect_format,
|
||||
export_dataframe,
|
||||
export_dataframe_to_sqlite,
|
||||
)
|
||||
from .trading import (
|
||||
calculate_margin_and_volume,
|
||||
detect_position_side,
|
||||
determine_order_limits,
|
||||
mt5_trading_session,
|
||||
)
|
||||
from .utils import (
|
||||
TICK_FLAG_MAP,
|
||||
TIMEFRAME_MAP,
|
||||
parse_datetime,
|
||||
parse_tick_flags,
|
||||
parse_timeframe,
|
||||
)
|
||||
|
||||
__version__ = version(__package__) if __package__ else None
|
||||
|
||||
__all__ = [
|
||||
"DEDUP_KEYS",
|
||||
"KNOWN_MT5_TIME_COLUMNS",
|
||||
"REQUIRED_COLUMNS",
|
||||
"TICK_FLAG_MAP",
|
||||
"TIMEFRAME_MAP",
|
||||
"TIME_COLUMNS",
|
||||
"AccountSpec",
|
||||
"DataKind",
|
||||
"Dataset",
|
||||
"IfExists",
|
||||
"MT5Client",
|
||||
"Mt5CliClient",
|
||||
"Mt5CliError",
|
||||
"Mt5ConnectionError",
|
||||
"Mt5OperationError",
|
||||
"Mt5SchemaError",
|
||||
"RateTarget",
|
||||
"ThrottledHistoryUpdater",
|
||||
"account_info",
|
||||
"build_config",
|
||||
"build_rate_targets",
|
||||
"build_rate_view_name",
|
||||
"calculate_margin_and_volume",
|
||||
"call_with_normalized_errors",
|
||||
"collect_history",
|
||||
"collect_latest_closed_rates_by_granularity",
|
||||
"collect_latest_closed_rates_for_accounts",
|
||||
"collect_latest_rates",
|
||||
"collect_latest_rates_for_accounts",
|
||||
"collect_latest_rates_for_accounts_with_retries",
|
||||
"copy_rates_from",
|
||||
"copy_rates_from_pos",
|
||||
"copy_rates_range",
|
||||
"copy_ticks_from",
|
||||
"copy_ticks_range",
|
||||
"detect_format",
|
||||
"detect_position_side",
|
||||
"determine_order_limits",
|
||||
"drop_forming_rate_bar",
|
||||
"ensure_utc",
|
||||
"export_dataframe",
|
||||
"export_dataframe_to_sqlite",
|
||||
"granularity_name",
|
||||
"history_deals",
|
||||
"history_orders",
|
||||
"is_recoverable_mt5_error",
|
||||
"last_error",
|
||||
"latest_rates",
|
||||
"load_rate_data",
|
||||
"load_rate_data_from_connection",
|
||||
"load_rate_series_by_granularity",
|
||||
"load_rate_series_from_sqlite",
|
||||
"market_book",
|
||||
"minimum_margins",
|
||||
"mt5_session",
|
||||
"mt5_summary",
|
||||
"mt5_summary_as_df",
|
||||
"mt5_trading_session",
|
||||
"mt5_version",
|
||||
"normalize_dataframe",
|
||||
"normalize_mt5_exception",
|
||||
"normalize_symbol",
|
||||
"normalize_symbols",
|
||||
"normalize_time_columns",
|
||||
"orders",
|
||||
"parse_date_range",
|
||||
"parse_datetime",
|
||||
"parse_tick_flags",
|
||||
"parse_timeframe",
|
||||
"positions",
|
||||
"recent_history_deals",
|
||||
"recent_ticks",
|
||||
"recent_window",
|
||||
"resolve_account_spec",
|
||||
"resolve_account_specs",
|
||||
"resolve_history_datasets",
|
||||
"resolve_history_tick_flags",
|
||||
"resolve_history_timeframes",
|
||||
"resolve_rate_tables",
|
||||
"resolve_rate_view_name",
|
||||
"resolve_rate_view_names",
|
||||
"schema_columns",
|
||||
"substitute_env_placeholders",
|
||||
"symbol_info",
|
||||
"symbol_info_tick",
|
||||
"symbols",
|
||||
"terminal_info",
|
||||
"update_history",
|
||||
"update_history_with_config",
|
||||
"validate_schema",
|
||||
]
|
||||
|
||||
+150
-60
@@ -12,6 +12,7 @@ import typer
|
||||
from pdmt5 import Mt5Config
|
||||
|
||||
from . import sdk
|
||||
from .client import MT5Client
|
||||
from .utils import (
|
||||
DATETIME_TYPE,
|
||||
REQUEST_TYPE,
|
||||
@@ -91,9 +92,18 @@ def _execute_export(
|
||||
)
|
||||
|
||||
|
||||
def _sdk_client(ctx: typer.Context) -> sdk.Mt5CliClient:
|
||||
def _sdk_client(ctx: typer.Context) -> MT5Client:
|
||||
export_ctx = _get_export_context(ctx)
|
||||
return sdk.Mt5CliClient(config=export_ctx.config)
|
||||
return MT5Client(config=export_ctx.config)
|
||||
|
||||
|
||||
def _export_command(
|
||||
ctx: typer.Context,
|
||||
fetch_fn: Callable[[MT5Client], pd.DataFrame],
|
||||
) -> None:
|
||||
"""Create an SDK client, fetch a DataFrame, and export it."""
|
||||
client = _sdk_client(ctx)
|
||||
_execute_export(ctx, lambda: fetch_fn(client))
|
||||
|
||||
|
||||
@app.callback()
|
||||
@@ -193,10 +203,9 @@ def rates_from(
|
||||
count: Annotated[int, typer.Option(help="Number of records.")],
|
||||
) -> None:
|
||||
"""Export rates from a start date."""
|
||||
client = _sdk_client(ctx)
|
||||
_execute_export(
|
||||
_export_command(
|
||||
ctx,
|
||||
lambda: client.copy_rates_from(symbol, timeframe, date_from, count),
|
||||
lambda client: client.copy_rates_from(symbol, timeframe, date_from, count),
|
||||
)
|
||||
|
||||
|
||||
@@ -215,10 +224,43 @@ def rates_from_pos(
|
||||
count: Annotated[int, typer.Option(help="Number of records.")],
|
||||
) -> None:
|
||||
"""Export rates from a start position."""
|
||||
client = _sdk_client(ctx)
|
||||
_execute_export(
|
||||
_export_command(
|
||||
ctx,
|
||||
lambda: client.copy_rates_from_pos(symbol, timeframe, start_pos, count),
|
||||
lambda client: client.copy_rates_from_pos(
|
||||
symbol,
|
||||
timeframe,
|
||||
start_pos,
|
||||
count,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@app.command()
|
||||
def latest_rates(
|
||||
ctx: typer.Context,
|
||||
symbol: Annotated[str, typer.Option(help="Symbol name.")],
|
||||
timeframe: Annotated[
|
||||
int,
|
||||
typer.Option(
|
||||
click_type=TIMEFRAME_TYPE,
|
||||
help="Timeframe.",
|
||||
),
|
||||
],
|
||||
count: Annotated[int, typer.Option(help="Number of records.")],
|
||||
start_pos: Annotated[
|
||||
int,
|
||||
typer.Option(help="Start position (0 = current bar)."),
|
||||
] = 0,
|
||||
) -> None:
|
||||
"""Export latest rates from a start position."""
|
||||
_export_command(
|
||||
ctx,
|
||||
lambda client: client.latest_rates(
|
||||
symbol,
|
||||
timeframe,
|
||||
count,
|
||||
start_pos=start_pos,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@@ -243,10 +285,9 @@ def rates_range(
|
||||
],
|
||||
) -> None:
|
||||
"""Export rates for a date range."""
|
||||
client = _sdk_client(ctx)
|
||||
_execute_export(
|
||||
_export_command(
|
||||
ctx,
|
||||
lambda: client.copy_rates_range(symbol, timeframe, date_from, date_to),
|
||||
lambda client: client.copy_rates_range(symbol, timeframe, date_from, date_to),
|
||||
)
|
||||
|
||||
|
||||
@@ -268,10 +309,9 @@ def ticks_from(
|
||||
],
|
||||
) -> None:
|
||||
"""Export ticks from a start date."""
|
||||
client = _sdk_client(ctx)
|
||||
_execute_export(
|
||||
_export_command(
|
||||
ctx,
|
||||
lambda: client.copy_ticks_from(symbol, date_from, count, flags),
|
||||
lambda client: client.copy_ticks_from(symbol, date_from, count, flags),
|
||||
)
|
||||
|
||||
|
||||
@@ -293,23 +333,59 @@ def ticks_range(
|
||||
],
|
||||
) -> None:
|
||||
"""Export ticks for a date range."""
|
||||
client = _sdk_client(ctx)
|
||||
_execute_export(
|
||||
_export_command(
|
||||
ctx,
|
||||
lambda: client.copy_ticks_range(symbol, date_from, date_to, flags),
|
||||
lambda client: client.copy_ticks_range(symbol, date_from, date_to, flags),
|
||||
)
|
||||
|
||||
|
||||
@app.command()
|
||||
def ticks_recent(
|
||||
ctx: typer.Context,
|
||||
symbol: Annotated[str, typer.Option(help="Symbol name.")],
|
||||
seconds: Annotated[
|
||||
float,
|
||||
typer.Option(help="Lookback window in seconds."),
|
||||
],
|
||||
date_to: Annotated[
|
||||
datetime | None,
|
||||
typer.Option(click_type=DATETIME_TYPE, help="Window end date."),
|
||||
] = None,
|
||||
count: Annotated[
|
||||
int,
|
||||
typer.Option(help="Maximum number of ticks to return."),
|
||||
] = 10000,
|
||||
flags: Annotated[
|
||||
int,
|
||||
typer.Option(
|
||||
click_type=TICK_FLAGS_TYPE,
|
||||
help="Tick flags (ALL, INFO, TRADE, or integer).",
|
||||
),
|
||||
] = "ALL", # pyright: ignore[reportArgumentType]
|
||||
) -> None:
|
||||
"""Export ticks from a recent time window."""
|
||||
_export_command(
|
||||
ctx,
|
||||
lambda client: client.recent_ticks(
|
||||
symbol,
|
||||
seconds,
|
||||
date_to=date_to,
|
||||
count=count,
|
||||
flags=flags,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@app.command()
|
||||
def account_info(ctx: typer.Context) -> None:
|
||||
"""Export account information."""
|
||||
_execute_export(ctx, _sdk_client(ctx).account_info)
|
||||
_export_command(ctx, lambda client: client.account_info())
|
||||
|
||||
|
||||
@app.command()
|
||||
def terminal_info(ctx: typer.Context) -> None:
|
||||
"""Export terminal information."""
|
||||
_execute_export(ctx, _sdk_client(ctx).terminal_info)
|
||||
_export_command(ctx, lambda client: client.terminal_info())
|
||||
|
||||
|
||||
@app.command()
|
||||
@@ -321,8 +397,7 @@ def symbols(
|
||||
] = None,
|
||||
) -> None:
|
||||
"""Export symbol list."""
|
||||
client = _sdk_client(ctx)
|
||||
_execute_export(ctx, lambda: client.symbols(group=group))
|
||||
_export_command(ctx, lambda client: client.symbols(group=group))
|
||||
|
||||
|
||||
@app.command()
|
||||
@@ -331,8 +406,16 @@ def symbol_info(
|
||||
symbol: Annotated[str, typer.Option(help="Symbol name.")],
|
||||
) -> None:
|
||||
"""Export symbol details."""
|
||||
client = _sdk_client(ctx)
|
||||
_execute_export(ctx, lambda: client.symbol_info(symbol))
|
||||
_export_command(ctx, lambda client: client.symbol_info(symbol))
|
||||
|
||||
|
||||
@app.command()
|
||||
def minimum_margins(
|
||||
ctx: typer.Context,
|
||||
symbol: Annotated[str, typer.Option(help="Symbol name.")],
|
||||
) -> None:
|
||||
"""Export minimum-volume buy and sell margin requirements."""
|
||||
_export_command(ctx, lambda client: client.minimum_margins(symbol))
|
||||
|
||||
|
||||
@app.command()
|
||||
@@ -343,10 +426,9 @@ def orders(
|
||||
ticket: Annotated[int | None, typer.Option(help="Ticket filter.")] = None,
|
||||
) -> None:
|
||||
"""Export active orders."""
|
||||
client = _sdk_client(ctx)
|
||||
_execute_export(
|
||||
_export_command(
|
||||
ctx,
|
||||
lambda: client.orders(symbol=symbol, group=group, ticket=ticket),
|
||||
lambda client: client.orders(symbol=symbol, group=group, ticket=ticket),
|
||||
)
|
||||
|
||||
|
||||
@@ -358,10 +440,9 @@ def positions(
|
||||
ticket: Annotated[int | None, typer.Option(help="Ticket filter.")] = None,
|
||||
) -> None:
|
||||
"""Export open positions."""
|
||||
client = _sdk_client(ctx)
|
||||
_execute_export(
|
||||
_export_command(
|
||||
ctx,
|
||||
lambda: client.positions(symbol=symbol, group=group, ticket=ticket),
|
||||
lambda client: client.positions(symbol=symbol, group=group, ticket=ticket),
|
||||
)
|
||||
|
||||
|
||||
@@ -382,10 +463,9 @@ def history_orders(
|
||||
position: Annotated[int | None, typer.Option(help="Position ticket.")] = None,
|
||||
) -> None:
|
||||
"""Export historical orders."""
|
||||
client = _sdk_client(ctx)
|
||||
_execute_export(
|
||||
_export_command(
|
||||
ctx,
|
||||
lambda: client.history_orders(
|
||||
lambda client: client.history_orders(
|
||||
date_from=date_from,
|
||||
date_to=date_to,
|
||||
group=group,
|
||||
@@ -413,10 +493,9 @@ def history_deals(
|
||||
position: Annotated[int | None, typer.Option(help="Position ticket.")] = None,
|
||||
) -> None:
|
||||
"""Export historical deals."""
|
||||
client = _sdk_client(ctx)
|
||||
_execute_export(
|
||||
_export_command(
|
||||
ctx,
|
||||
lambda: client.history_deals(
|
||||
lambda client: client.history_deals(
|
||||
date_from=date_from,
|
||||
date_to=date_to,
|
||||
group=group,
|
||||
@@ -427,16 +506,45 @@ def history_deals(
|
||||
)
|
||||
|
||||
|
||||
@app.command()
|
||||
def recent_history_deals(
|
||||
ctx: typer.Context,
|
||||
hours: Annotated[float, typer.Option(help="Lookback window in hours.")],
|
||||
date_to: Annotated[
|
||||
datetime | None,
|
||||
typer.Option(click_type=DATETIME_TYPE, help="Window end date."),
|
||||
] = None,
|
||||
group: Annotated[str | None, typer.Option(help="Group filter.")] = None,
|
||||
symbol: Annotated[str | None, typer.Option(help="Symbol filter.")] = None,
|
||||
) -> None:
|
||||
"""Export historical deals from a recent trailing window."""
|
||||
_export_command(
|
||||
ctx,
|
||||
lambda client: client.recent_history_deals(
|
||||
hours,
|
||||
date_to=date_to,
|
||||
group=group,
|
||||
symbol=symbol,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@app.command()
|
||||
def mt5_summary(ctx: typer.Context) -> None:
|
||||
"""Export a compact terminal/account status summary."""
|
||||
_export_command(ctx, lambda client: client.mt5_summary_as_df())
|
||||
|
||||
|
||||
@app.command()
|
||||
def version(ctx: typer.Context) -> None:
|
||||
"""Export MetaTrader5 version information."""
|
||||
_execute_export(ctx, _sdk_client(ctx).version)
|
||||
_export_command(ctx, lambda client: client.version())
|
||||
|
||||
|
||||
@app.command()
|
||||
def last_error(ctx: typer.Context) -> None:
|
||||
"""Export the last error information."""
|
||||
_execute_export(ctx, _sdk_client(ctx).last_error)
|
||||
_export_command(ctx, lambda client: client.last_error())
|
||||
|
||||
|
||||
@app.command()
|
||||
@@ -445,8 +553,7 @@ def symbol_info_tick(
|
||||
symbol: Annotated[str, typer.Option(help="Symbol name.")],
|
||||
) -> None:
|
||||
"""Export the last tick for a symbol."""
|
||||
client = _sdk_client(ctx)
|
||||
_execute_export(ctx, lambda: client.symbol_info_tick(symbol))
|
||||
_export_command(ctx, lambda client: client.symbol_info_tick(symbol))
|
||||
|
||||
|
||||
@app.command()
|
||||
@@ -455,8 +562,7 @@ def market_book(
|
||||
symbol: Annotated[str, typer.Option(help="Symbol name.")],
|
||||
) -> None:
|
||||
"""Export market depth (order book) for a symbol."""
|
||||
client = _sdk_client(ctx)
|
||||
_execute_export(ctx, lambda: client.market_book(symbol))
|
||||
_export_command(ctx, lambda client: client.market_book(symbol))
|
||||
|
||||
|
||||
@app.command()
|
||||
@@ -468,15 +574,7 @@ def order_check(
|
||||
],
|
||||
) -> None:
|
||||
"""Check funds sufficiency for a trading operation."""
|
||||
export_ctx = _get_export_context(ctx)
|
||||
|
||||
def _fetch() -> pd.DataFrame:
|
||||
return sdk._run_with_client( # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
|
||||
export_ctx.config,
|
||||
lambda c: c.order_check_as_df(request=request),
|
||||
)
|
||||
|
||||
_execute_export(ctx, _fetch)
|
||||
_export_command(ctx, lambda client: client.order_check(request))
|
||||
|
||||
|
||||
@app.command()
|
||||
@@ -499,15 +597,7 @@ def order_send(
|
||||
if not yes:
|
||||
msg = "Pass --yes to send a live trade request."
|
||||
raise typer.BadParameter(msg, param_hint="--yes")
|
||||
export_ctx = _get_export_context(ctx)
|
||||
|
||||
def _fetch() -> pd.DataFrame:
|
||||
return sdk._run_with_client( # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
|
||||
export_ctx.config,
|
||||
lambda c: c.order_send_as_df(request=request),
|
||||
)
|
||||
|
||||
_execute_export(ctx, _fetch)
|
||||
_export_command(ctx, lambda client: client.order_send(request))
|
||||
|
||||
|
||||
@app.command()
|
||||
@@ -552,7 +642,7 @@ def collect_history(
|
||||
click_type=TICK_FLAGS_TYPE,
|
||||
help="Tick copy flags (ALL, INFO, TRADE, or integer).",
|
||||
),
|
||||
] = 1,
|
||||
] = "ALL", # pyright: ignore[reportArgumentType]
|
||||
if_exists: Annotated[
|
||||
IfExists,
|
||||
typer.Option(
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
"""Stable public client abstraction for MT5 data and execution operations."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from contextlib import contextmanager
|
||||
from typing import TYPE_CHECKING, Any, Self
|
||||
|
||||
from .sdk import Mt5CliClient, build_config, connected_client
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterator
|
||||
|
||||
import pandas as pd
|
||||
from pdmt5 import Mt5Config, Mt5DataClient
|
||||
|
||||
__all__ = [
|
||||
"MT5Client",
|
||||
"build_config",
|
||||
"mt5_session",
|
||||
]
|
||||
|
||||
|
||||
class MT5Client(Mt5CliClient):
|
||||
"""Public client for generic MT5 data access and order primitives.
|
||||
|
||||
Extends the read-only SDK client with optional order check/send helpers and
|
||||
exposes the same connection lifecycle as :class:`~mt5cli.sdk.Mt5CliClient`.
|
||||
Downstream applications such as private trading packages should prefer this
|
||||
type over the legacy ``Mt5CliClient`` name.
|
||||
|
||||
mt5cli intentionally exposes minimal execution primitives only. Trading
|
||||
decisions, signals, strategies, backtests, and optimization remain the
|
||||
responsibility of downstream applications.
|
||||
"""
|
||||
|
||||
def order_check(self, request: dict[str, Any]) -> pd.DataFrame:
|
||||
"""Check funds sufficiency for a trade request.
|
||||
|
||||
Args:
|
||||
request: MT5 order request dictionary.
|
||||
|
||||
Returns:
|
||||
One-row DataFrame with the order-check result.
|
||||
"""
|
||||
return self._fetch(lambda client: client.order_check_as_df(request=request))
|
||||
|
||||
def order_send(self, request: dict[str, Any]) -> pd.DataFrame:
|
||||
"""Send a live trade request to the MT5 trade server.
|
||||
|
||||
Warning:
|
||||
This is a live execution primitive. A successful call can place,
|
||||
modify, or close real trades on the connected account. Downstream
|
||||
applications must gate usage explicitly (for example behind manual
|
||||
confirmation or application-specific risk controls). mt5cli does
|
||||
not implement strategy logic, signal generation, or trade sizing.
|
||||
|
||||
Args:
|
||||
request: MT5 order request dictionary.
|
||||
|
||||
Returns:
|
||||
One-row DataFrame with the order-send result.
|
||||
"""
|
||||
return self._fetch(lambda client: client.order_send_as_df(request=request))
|
||||
|
||||
@classmethod
|
||||
def from_connected_client(cls, client: Mt5DataClient) -> Self:
|
||||
"""Bind to an already-connected ``Mt5DataClient`` without owning it.
|
||||
|
||||
Returns:
|
||||
Client wrapper bound to the injected connection.
|
||||
"""
|
||||
return cls(client=client)
|
||||
|
||||
|
||||
@contextmanager
|
||||
def mt5_session(config: Mt5Config | None = None) -> Iterator[MT5Client]:
|
||||
"""Open an MT5 terminal session and yield a connected :class:`MT5Client`.
|
||||
|
||||
Args:
|
||||
config: MT5 connection configuration. Defaults to an empty config that
|
||||
attaches to a running terminal.
|
||||
|
||||
Yields:
|
||||
Connected :class:`MT5Client` bound to the session.
|
||||
"""
|
||||
mt5_config = config or build_config()
|
||||
with connected_client(mt5_config) as client:
|
||||
yield MT5Client.from_connected_client(client)
|
||||
@@ -0,0 +1,162 @@
|
||||
"""Shared conversion helpers for MT5 symbols, timeframes, and date ranges."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from pdmt5 import get_timeframe_name as _get_timeframe_name
|
||||
|
||||
from .utils import parse_datetime, parse_tick_flags, parse_timeframe
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Sequence
|
||||
|
||||
__all__ = [
|
||||
"ensure_utc",
|
||||
"granularity_name",
|
||||
"normalize_symbol",
|
||||
"normalize_symbols",
|
||||
"parse_date_range",
|
||||
"parse_datetime",
|
||||
"parse_tick_flags",
|
||||
"parse_timeframe",
|
||||
"recent_window",
|
||||
]
|
||||
|
||||
|
||||
def normalize_symbol(symbol: str) -> str:
|
||||
"""Normalize a broker symbol name for MT5 API calls.
|
||||
|
||||
Strips surrounding whitespace while preserving broker-specific casing and
|
||||
suffixes (for example ``XAUUSDm``, ``US500.cash``, or ``EURUSD.r``).
|
||||
|
||||
Args:
|
||||
symbol: Raw symbol name.
|
||||
|
||||
Returns:
|
||||
Normalized symbol string.
|
||||
|
||||
Raises:
|
||||
ValueError: If the symbol is empty after normalization.
|
||||
"""
|
||||
normalized = symbol.strip()
|
||||
if not normalized:
|
||||
msg = "Symbol must not be empty."
|
||||
raise ValueError(msg)
|
||||
return normalized
|
||||
|
||||
|
||||
def normalize_symbols(symbols: Sequence[str]) -> list[str]:
|
||||
"""Normalize a sequence of broker symbol names.
|
||||
|
||||
Args:
|
||||
symbols: Raw symbol names.
|
||||
|
||||
Returns:
|
||||
List of normalized, de-duplicated symbols preserving first-seen order.
|
||||
"""
|
||||
seen: set[str] = set()
|
||||
resolved: list[str] = []
|
||||
for symbol in symbols:
|
||||
normalized = normalize_symbol(symbol)
|
||||
if normalized not in seen:
|
||||
seen.add(normalized)
|
||||
resolved.append(normalized)
|
||||
return resolved
|
||||
|
||||
|
||||
def ensure_utc(value: datetime | str) -> datetime:
|
||||
"""Return a timezone-aware UTC datetime.
|
||||
|
||||
Args:
|
||||
value: Datetime instance or ISO 8601 string.
|
||||
|
||||
Returns:
|
||||
UTC-aware datetime.
|
||||
"""
|
||||
if isinstance(value, str):
|
||||
return parse_datetime(value)
|
||||
if value.tzinfo is None:
|
||||
return value.replace(tzinfo=UTC)
|
||||
return value.astimezone(UTC)
|
||||
|
||||
|
||||
def parse_date_range(
|
||||
date_from: datetime | str,
|
||||
date_to: datetime | str,
|
||||
) -> tuple[datetime, datetime]:
|
||||
"""Parse and validate an inclusive UTC date range.
|
||||
|
||||
Args:
|
||||
date_from: Range start as datetime or ISO 8601 string.
|
||||
date_to: Range end as datetime or ISO 8601 string.
|
||||
|
||||
Returns:
|
||||
Tuple of UTC-aware ``(start, end)`` datetimes.
|
||||
|
||||
Raises:
|
||||
ValueError: If ``date_from`` is after ``date_to``.
|
||||
"""
|
||||
start = ensure_utc(date_from)
|
||||
end = ensure_utc(date_to)
|
||||
if start > end:
|
||||
msg = (
|
||||
f"date_from ({start.isoformat()}) must not be after "
|
||||
f"date_to ({end.isoformat()})."
|
||||
)
|
||||
raise ValueError(msg)
|
||||
return start, end
|
||||
|
||||
|
||||
def recent_window(
|
||||
*,
|
||||
hours: float | None = None,
|
||||
seconds: float | None = None,
|
||||
date_to: datetime | str | None = None,
|
||||
) -> tuple[datetime, datetime]:
|
||||
"""Build a trailing UTC window ending at ``date_to`` or now.
|
||||
|
||||
Exactly one of ``hours`` or ``seconds`` must be provided.
|
||||
|
||||
Args:
|
||||
hours: Trailing window length in hours.
|
||||
seconds: Trailing window length in seconds.
|
||||
date_to: Window end. Defaults to current UTC time.
|
||||
|
||||
Returns:
|
||||
Tuple of UTC-aware ``(start, end)`` datetimes.
|
||||
|
||||
Raises:
|
||||
ValueError: If neither or both window lengths are provided, or if a
|
||||
length is not positive.
|
||||
"""
|
||||
if (hours is None) == (seconds is None):
|
||||
msg = "Provide exactly one of hours or seconds."
|
||||
raise ValueError(msg)
|
||||
if hours is not None:
|
||||
length = timedelta(hours=hours)
|
||||
else:
|
||||
length = timedelta(seconds=seconds if seconds is not None else 0)
|
||||
if length.total_seconds() <= 0:
|
||||
msg = "Window length must be positive."
|
||||
raise ValueError(msg)
|
||||
end = ensure_utc(date_to) if date_to is not None else datetime.now(UTC)
|
||||
return end - length, end
|
||||
|
||||
|
||||
def granularity_name(timeframe: int | str) -> str:
|
||||
"""Return a short granularity label for a timeframe integer or name.
|
||||
|
||||
Args:
|
||||
timeframe: MT5 timeframe as integer or name (for example ``M1``).
|
||||
|
||||
Returns:
|
||||
Short name such as ``M1`` or the stringified integer when unknown.
|
||||
"""
|
||||
tf = parse_timeframe(timeframe)
|
||||
try:
|
||||
name = _get_timeframe_name(tf)
|
||||
except ValueError:
|
||||
return str(tf)
|
||||
return name.removeprefix("TIMEFRAME_")
|
||||
@@ -0,0 +1,90 @@
|
||||
"""Normalized exception types for MT5 and mt5cli operations."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING, TypeVar
|
||||
|
||||
from pdmt5 import Mt5RuntimeError, Mt5TradingError
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
|
||||
T = TypeVar("T")
|
||||
|
||||
__all__ = [
|
||||
"Mt5CliError",
|
||||
"Mt5ConnectionError",
|
||||
"Mt5OperationError",
|
||||
"Mt5SchemaError",
|
||||
"call_with_normalized_errors",
|
||||
"is_recoverable_mt5_error",
|
||||
"normalize_mt5_exception",
|
||||
]
|
||||
|
||||
_RECOVERABLE_MT5_ERRORS: tuple[type[BaseException], ...] = (
|
||||
Mt5TradingError,
|
||||
Mt5RuntimeError,
|
||||
)
|
||||
|
||||
|
||||
class Mt5CliError(Exception):
|
||||
"""Base exception for mt5cli public API errors."""
|
||||
|
||||
|
||||
class Mt5ConnectionError(Mt5CliError):
|
||||
"""Raised when MT5 initialization, login, or shutdown fails."""
|
||||
|
||||
|
||||
class Mt5OperationError(Mt5CliError):
|
||||
"""Raised when an MT5 data or trading operation fails."""
|
||||
|
||||
|
||||
class Mt5SchemaError(Mt5CliError):
|
||||
"""Raised when a DataFrame does not match an expected dataset schema."""
|
||||
|
||||
|
||||
def is_recoverable_mt5_error(exc: BaseException) -> bool:
|
||||
"""Return whether an exception is a transient MT5 failure worth retrying.
|
||||
|
||||
Args:
|
||||
exc: Exception raised by MT5 or pdmt5.
|
||||
|
||||
Returns:
|
||||
True for ``Mt5RuntimeError`` and ``Mt5TradingError``.
|
||||
"""
|
||||
return isinstance(exc, _RECOVERABLE_MT5_ERRORS)
|
||||
|
||||
|
||||
def normalize_mt5_exception(exc: BaseException) -> Mt5CliError:
|
||||
"""Map pdmt5/MT5 exceptions to stable mt5cli exception types.
|
||||
|
||||
Args:
|
||||
exc: Original exception from MT5 or pdmt5.
|
||||
|
||||
Returns:
|
||||
``Mt5ConnectionError`` for runtime failures, ``Mt5OperationError`` for
|
||||
trading failures, or the original exception when it is not recognized.
|
||||
"""
|
||||
if isinstance(exc, Mt5TradingError):
|
||||
return Mt5OperationError(str(exc))
|
||||
if isinstance(exc, Mt5RuntimeError):
|
||||
return Mt5ConnectionError(str(exc))
|
||||
if isinstance(exc, Mt5CliError):
|
||||
return exc
|
||||
return Mt5CliError(str(exc))
|
||||
|
||||
|
||||
def call_with_normalized_errors(fn: Callable[[], T]) -> T:
|
||||
"""Run ``fn`` and map recoverable MT5 errors to mt5cli types.
|
||||
|
||||
Args:
|
||||
fn: Callable performing MT5 work.
|
||||
|
||||
Returns:
|
||||
Value returned by ``fn``.
|
||||
"""
|
||||
try:
|
||||
return fn()
|
||||
except _RECOVERABLE_MT5_ERRORS as exc:
|
||||
normalized = normalize_mt5_exception(exc)
|
||||
raise normalized from exc
|
||||
+797
-71
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,64 @@
|
||||
"""Retry and reconnect helpers for transient MT5 failures."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import time
|
||||
from typing import TYPE_CHECKING, TypeVar
|
||||
|
||||
from .exceptions import is_recoverable_mt5_error
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Callable
|
||||
|
||||
T = TypeVar("T")
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
__all__ = [
|
||||
"retry_with_backoff",
|
||||
]
|
||||
|
||||
|
||||
def retry_with_backoff(
|
||||
fn: Callable[[], T],
|
||||
*,
|
||||
retry_count: int = 0,
|
||||
backoff_base: float = 2.0,
|
||||
operation: str = "MT5 operation",
|
||||
) -> T:
|
||||
"""Call ``fn`` with bounded exponential backoff on recoverable MT5 errors.
|
||||
|
||||
Only ``pdmt5.Mt5RuntimeError`` and ``pdmt5.Mt5TradingError`` are retried.
|
||||
Other exceptions propagate immediately. The final failure is re-raised once
|
||||
retries are exhausted.
|
||||
|
||||
Args:
|
||||
fn: Callable performing MT5 work.
|
||||
retry_count: Maximum number of retries after the first attempt. ``0``
|
||||
disables retries.
|
||||
backoff_base: Base for exponential backoff. The delay before retry
|
||||
attempt ``n`` (1-indexed) is ``backoff_base ** n`` seconds.
|
||||
operation: Label used in warning logs.
|
||||
|
||||
Returns:
|
||||
Value returned by ``fn`` on success.
|
||||
"""
|
||||
attempts = max(retry_count, 0) + 1
|
||||
for attempt in range(attempts - 1):
|
||||
try:
|
||||
return fn()
|
||||
except Exception as exc:
|
||||
if not is_recoverable_mt5_error(exc):
|
||||
raise
|
||||
delay = backoff_base ** (attempt + 1)
|
||||
logger.warning(
|
||||
"%s failed (attempt %d/%d): %s; retrying in %.1fs",
|
||||
operation,
|
||||
attempt + 1,
|
||||
attempts,
|
||||
exc,
|
||||
delay,
|
||||
)
|
||||
time.sleep(delay)
|
||||
return fn()
|
||||
@@ -0,0 +1,291 @@
|
||||
"""Canonical DataFrame schemas for MT5 market and account datasets."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from enum import StrEnum
|
||||
from typing import TYPE_CHECKING, Final
|
||||
|
||||
import pandas as pd
|
||||
|
||||
from .converters import normalize_symbol, parse_timeframe
|
||||
from .exceptions import Mt5SchemaError
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterable
|
||||
|
||||
__all__ = [
|
||||
"DEDUP_KEYS",
|
||||
"KNOWN_MT5_TIME_COLUMNS",
|
||||
"REQUIRED_COLUMNS",
|
||||
"TIME_COLUMNS",
|
||||
"DataKind",
|
||||
"normalize_dataframe",
|
||||
"normalize_time_columns",
|
||||
"schema_columns",
|
||||
"validate_schema",
|
||||
]
|
||||
|
||||
KNOWN_MT5_TIME_COLUMNS: Final[frozenset[str]] = frozenset({
|
||||
"time",
|
||||
"time_setup",
|
||||
"time_setup_msc",
|
||||
"time_done",
|
||||
"time_done_msc",
|
||||
"time_msc",
|
||||
})
|
||||
|
||||
_TIME_COLUMN_NAMES = KNOWN_MT5_TIME_COLUMNS
|
||||
|
||||
|
||||
class DataKind(StrEnum):
|
||||
"""Supported MT5 dataset kinds with canonical column contracts."""
|
||||
|
||||
rates = "rates"
|
||||
ticks = "ticks"
|
||||
orders = "orders"
|
||||
positions = "positions"
|
||||
history_orders = "history_orders"
|
||||
history_deals = "history_deals"
|
||||
|
||||
|
||||
REQUIRED_COLUMNS: dict[DataKind, frozenset[str]] = {
|
||||
DataKind.rates: frozenset({
|
||||
"time",
|
||||
"open",
|
||||
"high",
|
||||
"low",
|
||||
"close",
|
||||
"tick_volume",
|
||||
"spread",
|
||||
"real_volume",
|
||||
}),
|
||||
DataKind.ticks: frozenset({
|
||||
"time",
|
||||
"bid",
|
||||
"ask",
|
||||
"last",
|
||||
"volume",
|
||||
"time_msc",
|
||||
"flags",
|
||||
"volume_real",
|
||||
}),
|
||||
DataKind.orders: frozenset({
|
||||
"ticket",
|
||||
"time_setup",
|
||||
"type",
|
||||
"state",
|
||||
"symbol",
|
||||
"volume_current",
|
||||
"price_open",
|
||||
}),
|
||||
DataKind.positions: frozenset({
|
||||
"ticket",
|
||||
"time",
|
||||
"type",
|
||||
"symbol",
|
||||
"volume",
|
||||
"price_open",
|
||||
"price_current",
|
||||
"profit",
|
||||
}),
|
||||
DataKind.history_orders: frozenset({
|
||||
"ticket",
|
||||
"time_setup",
|
||||
"type",
|
||||
"state",
|
||||
"symbol",
|
||||
"volume_initial",
|
||||
"price_open",
|
||||
}),
|
||||
DataKind.history_deals: frozenset({
|
||||
"ticket",
|
||||
"order",
|
||||
"time",
|
||||
"type",
|
||||
"entry",
|
||||
"symbol",
|
||||
"volume",
|
||||
"price",
|
||||
"profit",
|
||||
}),
|
||||
}
|
||||
|
||||
_OPTIONAL_TIME_COLUMNS_BY_KIND: dict[DataKind, frozenset[str]] = {
|
||||
DataKind.orders: frozenset({
|
||||
"time_setup_msc",
|
||||
"time_done",
|
||||
"time_done_msc",
|
||||
}),
|
||||
DataKind.history_orders: frozenset({
|
||||
"time_setup_msc",
|
||||
"time_done",
|
||||
"time_done_msc",
|
||||
}),
|
||||
DataKind.positions: frozenset({"time_msc"}),
|
||||
}
|
||||
|
||||
TIME_COLUMNS: dict[DataKind, frozenset[str]] = {
|
||||
kind: (REQUIRED_COLUMNS[kind] & _TIME_COLUMN_NAMES)
|
||||
| _OPTIONAL_TIME_COLUMNS_BY_KIND.get(kind, frozenset())
|
||||
for kind in DataKind
|
||||
}
|
||||
|
||||
DEDUP_KEYS: dict[DataKind, tuple[tuple[str, ...], ...]] = {
|
||||
DataKind.rates: (("symbol", "timeframe", "time"), ("symbol", "time")),
|
||||
DataKind.ticks: (("symbol", "time_msc"), ("symbol", "time")),
|
||||
DataKind.history_orders: (("ticket",), ("symbol", "time", "type")),
|
||||
DataKind.history_deals: (("ticket",), ("symbol", "time", "type", "entry")),
|
||||
}
|
||||
|
||||
|
||||
def schema_columns(kind: DataKind) -> frozenset[str]:
|
||||
"""Return required column names for a dataset kind.
|
||||
|
||||
Args:
|
||||
kind: Dataset kind.
|
||||
|
||||
Returns:
|
||||
Required column names for ``kind``.
|
||||
"""
|
||||
return REQUIRED_COLUMNS[kind]
|
||||
|
||||
|
||||
def validate_schema(
|
||||
frame: pd.DataFrame,
|
||||
kind: DataKind,
|
||||
*,
|
||||
extra_required: Iterable[str] | None = None,
|
||||
) -> None:
|
||||
"""Validate that a DataFrame includes required columns for a dataset kind.
|
||||
|
||||
Args:
|
||||
frame: DataFrame to validate.
|
||||
kind: Expected dataset kind.
|
||||
extra_required: Additional columns that must be present (for example
|
||||
``symbol`` and ``timeframe`` on stored rate history).
|
||||
|
||||
Raises:
|
||||
Mt5SchemaError: If required columns are missing.
|
||||
"""
|
||||
if frame.empty and len(frame.columns) == 0:
|
||||
return
|
||||
required = set(REQUIRED_COLUMNS[kind])
|
||||
if extra_required is not None:
|
||||
required.update(extra_required)
|
||||
missing = required - set(frame.columns)
|
||||
if missing:
|
||||
msg = (
|
||||
f"{kind.value} schema is missing required columns: "
|
||||
f"{', '.join(sorted(missing))}."
|
||||
)
|
||||
raise Mt5SchemaError(msg)
|
||||
|
||||
|
||||
def _coerce_mt5_time_column(series: pd.Series, column: str) -> pd.Series:
|
||||
"""Coerce one MT5 time column to UTC-aware datetimes.
|
||||
|
||||
Returns:
|
||||
Series with UTC-aware datetime values.
|
||||
"""
|
||||
if pd.api.types.is_datetime64_any_dtype(series):
|
||||
return pd.to_datetime(series, utc=True, errors="coerce")
|
||||
if pd.api.types.is_numeric_dtype(series):
|
||||
unit = "ms" if column.endswith("_msc") else "s"
|
||||
return pd.to_datetime(series, unit=unit, utc=True, errors="coerce")
|
||||
return pd.to_datetime(series, utc=True, errors="coerce")
|
||||
|
||||
|
||||
def normalize_time_columns(frame: pd.DataFrame, kind: DataKind) -> pd.DataFrame:
|
||||
"""Coerce dataset time columns to UTC-aware datetimes when present.
|
||||
|
||||
Any column in :data:`KNOWN_MT5_TIME_COLUMNS` that is present in ``frame``
|
||||
is normalized. Numeric MT5 epoch values use seconds for ``time``,
|
||||
``time_setup``, and ``time_done``, and milliseconds for ``*_msc`` columns.
|
||||
|
||||
Args:
|
||||
frame: Source DataFrame from MT5 or pdmt5.
|
||||
kind: Dataset kind (retained for API compatibility).
|
||||
|
||||
Returns:
|
||||
DataFrame copy with normalized time columns.
|
||||
"""
|
||||
del kind
|
||||
normalized = frame.copy()
|
||||
for column in normalized.columns:
|
||||
if column not in _TIME_COLUMN_NAMES:
|
||||
continue
|
||||
normalized[column] = _coerce_mt5_time_column(normalized[column], column)
|
||||
return normalized
|
||||
|
||||
|
||||
def normalize_dataframe(
|
||||
frame: pd.DataFrame,
|
||||
kind: DataKind,
|
||||
*,
|
||||
symbol: str | None = None,
|
||||
timeframe: int | str | None = None,
|
||||
sort: bool = True,
|
||||
) -> pd.DataFrame:
|
||||
"""Normalize MT5 DataFrame columns, timestamps, and storage metadata.
|
||||
|
||||
Ensures UTC timestamps, optionally injects ``symbol`` / ``timeframe`` for
|
||||
storage-oriented datasets, and sorts chronologically when a ``time`` column
|
||||
exists.
|
||||
|
||||
Args:
|
||||
frame: Source DataFrame from MT5 or pdmt5.
|
||||
kind: Dataset kind guiding normalization rules.
|
||||
symbol: Optional symbol to inject when missing.
|
||||
timeframe: Optional timeframe integer or name to inject for rates.
|
||||
sort: Whether to sort by ``time`` or ``time_msc`` when present.
|
||||
|
||||
Returns:
|
||||
Normalized DataFrame copy.
|
||||
"""
|
||||
if frame.empty and len(frame.columns) == 0:
|
||||
return frame.copy()
|
||||
|
||||
normalized = normalize_time_columns(frame, kind)
|
||||
|
||||
if symbol is not None and "symbol" not in normalized.columns:
|
||||
normalized.insert(0, "symbol", normalize_symbol(symbol))
|
||||
|
||||
if timeframe is not None and kind is DataKind.rates:
|
||||
tf = parse_timeframe(timeframe)
|
||||
if "timeframe" not in normalized.columns:
|
||||
insert_at = 1 if "symbol" in normalized.columns else 0
|
||||
normalized.insert(insert_at, "timeframe", tf)
|
||||
|
||||
validate_schema(normalized, kind)
|
||||
|
||||
if sort:
|
||||
if "time" in normalized.columns:
|
||||
normalized = normalized.sort_values("time", kind="stable")
|
||||
elif "time_msc" in normalized.columns:
|
||||
normalized = normalized.sort_values("time_msc", kind="stable")
|
||||
normalized = normalized.reset_index(drop=True)
|
||||
|
||||
return normalized
|
||||
|
||||
|
||||
def ensure_utc_columns(frame: pd.DataFrame, columns: Iterable[str]) -> pd.DataFrame:
|
||||
"""Return a copy with selected columns coerced to UTC datetimes.
|
||||
|
||||
Args:
|
||||
frame: Source DataFrame.
|
||||
columns: Column names to coerce.
|
||||
|
||||
Returns:
|
||||
DataFrame copy with UTC-aware datetime columns.
|
||||
"""
|
||||
normalized = frame.copy()
|
||||
for column in columns:
|
||||
if column not in normalized.columns:
|
||||
continue
|
||||
if column in _TIME_COLUMN_NAMES:
|
||||
normalized[column] = _coerce_mt5_time_column(normalized[column], column)
|
||||
else:
|
||||
normalized[column] = pd.to_datetime(
|
||||
normalized[column], utc=True, errors="coerce"
|
||||
)
|
||||
return normalized
|
||||
+1065
-18
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,49 @@
|
||||
"""Generic storage helpers for MT5 market and account history."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from .history import (
|
||||
RateTarget,
|
||||
build_rate_targets,
|
||||
build_rate_view_name,
|
||||
drop_forming_rate_bar,
|
||||
load_rate_data,
|
||||
load_rate_data_from_connection,
|
||||
load_rate_series_by_granularity,
|
||||
load_rate_series_from_sqlite,
|
||||
resolve_rate_tables,
|
||||
resolve_rate_view_name,
|
||||
resolve_rate_view_names,
|
||||
)
|
||||
from .sdk import collect_history, update_history, update_history_with_config
|
||||
from .utils import (
|
||||
Dataset,
|
||||
IfExists,
|
||||
OutputFormat,
|
||||
detect_format,
|
||||
export_dataframe,
|
||||
export_dataframe_to_sqlite,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"Dataset",
|
||||
"IfExists",
|
||||
"OutputFormat",
|
||||
"RateTarget",
|
||||
"build_rate_targets",
|
||||
"build_rate_view_name",
|
||||
"collect_history",
|
||||
"detect_format",
|
||||
"drop_forming_rate_bar",
|
||||
"export_dataframe",
|
||||
"export_dataframe_to_sqlite",
|
||||
"load_rate_data",
|
||||
"load_rate_data_from_connection",
|
||||
"load_rate_series_by_granularity",
|
||||
"load_rate_series_from_sqlite",
|
||||
"resolve_rate_tables",
|
||||
"resolve_rate_view_name",
|
||||
"resolve_rate_view_names",
|
||||
"update_history",
|
||||
"update_history_with_config",
|
||||
]
|
||||
@@ -0,0 +1,210 @@
|
||||
"""Trading-capable MetaTrader 5 session helpers and operational utilities."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from contextlib import contextmanager
|
||||
from typing import TYPE_CHECKING, Literal
|
||||
|
||||
from pdmt5 import Mt5Config, Mt5TradingClient
|
||||
|
||||
from .sdk import build_config
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Iterator
|
||||
|
||||
import pandas as pd
|
||||
|
||||
PositionSide = Literal["long", "short"]
|
||||
OrderSide = Literal["long", "short"]
|
||||
|
||||
__all__ = [
|
||||
"OrderSide",
|
||||
"PositionSide",
|
||||
"calculate_margin_and_volume",
|
||||
"detect_position_side",
|
||||
"determine_order_limits",
|
||||
"mt5_trading_session",
|
||||
]
|
||||
|
||||
|
||||
def _require_unit_ratio(value: float, name: str) -> None:
|
||||
if not 0.0 <= value <= 1.0:
|
||||
msg = f"{name} must be between 0 and 1 inclusive."
|
||||
raise ValueError(msg)
|
||||
|
||||
|
||||
def _require_protective_ratio(value: float, name: str) -> None:
|
||||
if not 0.0 <= value < 1.0:
|
||||
msg = f"{name} must be at least 0 and less than 1."
|
||||
raise ValueError(msg)
|
||||
|
||||
|
||||
def _sum_position_volume(positions: pd.DataFrame, position_type: object) -> float:
|
||||
matched = positions.loc[positions["type"] == position_type, "volume"]
|
||||
if matched.empty:
|
||||
return 0.0
|
||||
return float(matched.to_numpy(dtype=float).sum())
|
||||
|
||||
|
||||
def _normalize_order_side(side: str) -> OrderSide:
|
||||
normalized = side.lower()
|
||||
if normalized in {"long", "buy"}:
|
||||
return "long"
|
||||
if normalized in {"short", "sell"}:
|
||||
return "short"
|
||||
msg = (
|
||||
f"Unsupported order side: {side!r}. Expected 'long', 'short', 'buy', or 'sell'."
|
||||
)
|
||||
raise ValueError(msg)
|
||||
|
||||
|
||||
def detect_position_side(
|
||||
client: Mt5TradingClient,
|
||||
symbol: str,
|
||||
) -> PositionSide | None:
|
||||
"""Detect the net open position side for a symbol.
|
||||
|
||||
Args:
|
||||
client: Connected ``Mt5TradingClient`` instance.
|
||||
symbol: Symbol to inspect.
|
||||
|
||||
Returns:
|
||||
``"long"`` when net buy volume exceeds sell volume, ``"short"`` when
|
||||
net sell volume exceeds buy volume, or ``None`` when no positions exist
|
||||
or buy/sell volumes are exactly balanced.
|
||||
"""
|
||||
positions = client.positions_get_as_df(symbol=symbol)
|
||||
if positions.empty:
|
||||
return None
|
||||
|
||||
buy_type = client.mt5.POSITION_TYPE_BUY
|
||||
sell_type = client.mt5.POSITION_TYPE_SELL
|
||||
buy_volume = _sum_position_volume(positions, buy_type)
|
||||
sell_volume = _sum_position_volume(positions, sell_type)
|
||||
net_volume = buy_volume - sell_volume
|
||||
if net_volume > 0:
|
||||
return "long"
|
||||
if net_volume < 0:
|
||||
return "short"
|
||||
return None
|
||||
|
||||
|
||||
def calculate_margin_and_volume(
|
||||
client: Mt5TradingClient,
|
||||
symbol: str,
|
||||
unit_margin_ratio: float,
|
||||
preserved_margin_ratio: float,
|
||||
) -> dict[str, float]:
|
||||
"""Calculate tradable margin and volumes from account free margin.
|
||||
|
||||
Applies ``preserved_margin_ratio`` to keep a reserve off ``margin_free``,
|
||||
then allocates ``unit_margin_ratio`` of the remainder as the margin budget
|
||||
for volume sizing on both buy and sell sides.
|
||||
|
||||
Args:
|
||||
client: Connected ``Mt5TradingClient`` instance.
|
||||
symbol: Symbol used for minimum-lot margin and volume calculations.
|
||||
unit_margin_ratio: Fraction of post-reserve margin to allocate per unit.
|
||||
preserved_margin_ratio: Fraction of ``margin_free`` to preserve.
|
||||
|
||||
Returns:
|
||||
Dictionary with ``margin_free``, ``available_margin``, ``trade_margin``,
|
||||
``buy_volume``, and ``sell_volume``. Negative ``margin_free`` values are
|
||||
clamped to ``0.0`` before sizing.
|
||||
"""
|
||||
_require_unit_ratio(unit_margin_ratio, "unit_margin_ratio")
|
||||
_require_unit_ratio(preserved_margin_ratio, "preserved_margin_ratio")
|
||||
|
||||
account = client.account_info_as_dict()
|
||||
margin_free = max(0.0, float(account.get("margin_free") or 0.0))
|
||||
available_margin = margin_free * (1.0 - preserved_margin_ratio)
|
||||
trade_margin = available_margin * unit_margin_ratio
|
||||
buy_volume = client.calculate_volume_by_margin(symbol, trade_margin, "BUY")
|
||||
sell_volume = client.calculate_volume_by_margin(symbol, trade_margin, "SELL")
|
||||
return {
|
||||
"margin_free": margin_free,
|
||||
"available_margin": available_margin,
|
||||
"trade_margin": trade_margin,
|
||||
"buy_volume": buy_volume,
|
||||
"sell_volume": sell_volume,
|
||||
}
|
||||
|
||||
|
||||
def determine_order_limits(
|
||||
client: Mt5TradingClient,
|
||||
symbol: str,
|
||||
side: OrderSide | str,
|
||||
stop_loss_limit_ratio: float,
|
||||
take_profit_limit_ratio: float,
|
||||
) -> dict[str, float | None]:
|
||||
"""Derive entry and protective order prices from current market quotes.
|
||||
|
||||
Args:
|
||||
client: Connected ``Mt5TradingClient`` instance.
|
||||
symbol: Symbol used for the quote lookup.
|
||||
side: Position side as ``"long"``/``"short"`` (``"buy"``/``"sell"``
|
||||
aliases are accepted).
|
||||
stop_loss_limit_ratio: Relative distance from entry for stop loss in
|
||||
``[0, 1)``. A value of ``0`` omits the stop loss.
|
||||
take_profit_limit_ratio: Relative distance from entry for take profit in
|
||||
``[0, 1)``. A value of ``0`` omits the take profit.
|
||||
|
||||
Returns:
|
||||
Dictionary with ``entry``, ``stop_loss``, and ``take_profit`` keys.
|
||||
Omitted protective levels are returned as ``None``.
|
||||
"""
|
||||
_require_protective_ratio(stop_loss_limit_ratio, "stop_loss_limit_ratio")
|
||||
_require_protective_ratio(take_profit_limit_ratio, "take_profit_limit_ratio")
|
||||
normalized_side = _normalize_order_side(side)
|
||||
tick = client.symbol_info_tick_as_dict(symbol=symbol)
|
||||
entry = float(tick["ask"] if normalized_side == "long" else tick["bid"])
|
||||
|
||||
stop_loss: float | None = None
|
||||
if stop_loss_limit_ratio > 0:
|
||||
if normalized_side == "long":
|
||||
stop_loss = entry * (1.0 - stop_loss_limit_ratio)
|
||||
else:
|
||||
stop_loss = entry * (1.0 + stop_loss_limit_ratio)
|
||||
|
||||
take_profit: float | None = None
|
||||
if take_profit_limit_ratio > 0:
|
||||
if normalized_side == "long":
|
||||
take_profit = entry * (1.0 + take_profit_limit_ratio)
|
||||
else:
|
||||
take_profit = entry * (1.0 - take_profit_limit_ratio)
|
||||
|
||||
return {
|
||||
"entry": entry,
|
||||
"stop_loss": stop_loss,
|
||||
"take_profit": take_profit,
|
||||
}
|
||||
|
||||
|
||||
@contextmanager
|
||||
def mt5_trading_session(
|
||||
config: Mt5Config | None = None,
|
||||
retry_count: int = 0,
|
||||
) -> Iterator[Mt5TradingClient]:
|
||||
"""Open a trading-capable MT5 session and always shut down safely.
|
||||
|
||||
Launches the MetaTrader 5 terminal using ``Mt5Config.path`` when set,
|
||||
initializes and logs in via ``initialize_and_login_mt5()``, yields a
|
||||
connected :class:`~pdmt5.Mt5TradingClient`, and calls ``shutdown()`` on
|
||||
exit even when an error is raised inside the context.
|
||||
|
||||
Args:
|
||||
config: MT5 connection configuration. Defaults to an empty config that
|
||||
attaches to a running terminal.
|
||||
retry_count: Number of initialization retries passed to
|
||||
``Mt5TradingClient``.
|
||||
|
||||
Yields:
|
||||
Connected ``Mt5TradingClient`` bound to the session.
|
||||
"""
|
||||
mt5_config = config or build_config()
|
||||
client = Mt5TradingClient(config=mt5_config, retry_count=retry_count)
|
||||
try:
|
||||
client.initialize_and_login_mt5()
|
||||
yield client
|
||||
finally:
|
||||
client.shutdown()
|
||||
+86
-60
@@ -2,51 +2,36 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import json
|
||||
import sqlite3
|
||||
from datetime import UTC, datetime
|
||||
from enum import StrEnum
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any, TypeGuard, cast
|
||||
from typing import TYPE_CHECKING, Any, TypeGuard
|
||||
|
||||
import click
|
||||
from pdmt5 import COPY_TICKS_MAP, TIMEFRAME_MAP
|
||||
from pdmt5 import parse_copy_ticks as _parse_copy_ticks
|
||||
from pdmt5 import parse_timeframe as _parse_timeframe
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Sequence
|
||||
|
||||
import pandas as pd
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Constants
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
TIMEFRAME_MAP: dict[str, int] = {
|
||||
"M1": 1,
|
||||
"M2": 2,
|
||||
"M3": 3,
|
||||
"M4": 4,
|
||||
"M5": 5,
|
||||
"M6": 6,
|
||||
"M10": 10,
|
||||
"M12": 12,
|
||||
"M15": 15,
|
||||
"M20": 20,
|
||||
"M30": 30,
|
||||
"H1": 16385,
|
||||
"H2": 16386,
|
||||
"H3": 16387,
|
||||
"H4": 16388,
|
||||
"H6": 16390,
|
||||
"H8": 16392,
|
||||
"H12": 16396,
|
||||
"D1": 16408,
|
||||
"W1": 32769,
|
||||
"MN1": 49153,
|
||||
}
|
||||
# Backward-compatible snapshot; prefer ``COPY_TICKS_MAP`` from pdmt5 directly.
|
||||
TICK_FLAG_MAP: dict[str, int] = dict(COPY_TICKS_MAP)
|
||||
|
||||
TICK_FLAG_MAP: dict[str, int] = {
|
||||
"ALL": 1,
|
||||
"INFO": 2,
|
||||
"TRADE": 4,
|
||||
}
|
||||
TIMEFRAME_NAMES: tuple[str, ...] = tuple(
|
||||
name for name in TIMEFRAME_MAP if not name.startswith("TIMEFRAME_")
|
||||
)
|
||||
_TICK_FLAG_NAMES: tuple[str, ...] = tuple(
|
||||
name for name in COPY_TICKS_MAP if not name.startswith("COPY_TICKS_")
|
||||
)
|
||||
|
||||
_FORMAT_EXTENSIONS: dict[str, str] = {
|
||||
".csv": "csv",
|
||||
@@ -158,10 +143,8 @@ class _TimeframeType(click.ParamType):
|
||||
Returns:
|
||||
Integer timeframe value.
|
||||
"""
|
||||
if isinstance(value, int):
|
||||
return value
|
||||
try:
|
||||
return parse_timeframe(str(value))
|
||||
return parse_timeframe(value)
|
||||
except ValueError as exc:
|
||||
self.fail(str(exc), param, ctx)
|
||||
|
||||
@@ -187,10 +170,8 @@ class _TickFlagsType(click.ParamType):
|
||||
Returns:
|
||||
Integer tick flag value.
|
||||
"""
|
||||
if isinstance(value, int):
|
||||
return value
|
||||
try:
|
||||
return parse_tick_flags(str(value))
|
||||
return parse_tick_flags(value)
|
||||
except ValueError as exc:
|
||||
self.fail(str(exc), param, ctx)
|
||||
|
||||
@@ -260,6 +241,50 @@ def detect_format(
|
||||
raise ValueError(msg)
|
||||
|
||||
|
||||
def export_dataframe_to_sqlite(
|
||||
df: pd.DataFrame,
|
||||
output_path: Path,
|
||||
table_name: str = "data",
|
||||
*,
|
||||
if_exists: IfExists = IfExists.APPEND,
|
||||
index: bool = False,
|
||||
index_label: str | None = None,
|
||||
deduplicate_on: Sequence[str] | None = None,
|
||||
) -> None:
|
||||
"""Write a DataFrame to SQLite with configurable append and deduplication.
|
||||
|
||||
Args:
|
||||
df: DataFrame to export.
|
||||
output_path: SQLite database path.
|
||||
table_name: Target table name.
|
||||
if_exists: Conflict behavior when the table already exists.
|
||||
index: Whether to write the DataFrame index as a column.
|
||||
index_label: Column name for the index when ``index=True``.
|
||||
deduplicate_on: Optional key columns to deduplicate after writing,
|
||||
keeping the latest ``ROWID`` per key group. Deduplication scans the
|
||||
full table, so repeated appends cost O(table size); index the key
|
||||
columns when appending frequently.
|
||||
"""
|
||||
with sqlite3.connect(output_path) as conn:
|
||||
df.to_sql( # type: ignore[reportUnknownMemberType]
|
||||
table_name,
|
||||
conn,
|
||||
if_exists=if_exists.value,
|
||||
index=index,
|
||||
index_label=index_label,
|
||||
)
|
||||
if deduplicate_on:
|
||||
from .history import drop_duplicates_in_table # noqa: PLC0415
|
||||
|
||||
drop_duplicates_in_table(
|
||||
conn.cursor(),
|
||||
table_name,
|
||||
list(deduplicate_on),
|
||||
keep="last",
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
|
||||
def export_dataframe(
|
||||
df: pd.DataFrame,
|
||||
output_path: Path,
|
||||
@@ -289,14 +314,13 @@ def export_dataframe(
|
||||
elif output_format == "parquet":
|
||||
df.to_parquet(output_path, index=False)
|
||||
elif output_format == "sqlite3":
|
||||
sqlite3 = cast("Any", importlib.import_module("sqlite3"))
|
||||
with sqlite3.connect(output_path) as conn:
|
||||
df.to_sql( # type: ignore[reportUnknownMemberType]
|
||||
table_name,
|
||||
conn,
|
||||
if_exists="replace",
|
||||
index=False,
|
||||
)
|
||||
export_dataframe_to_sqlite(
|
||||
df,
|
||||
output_path,
|
||||
table_name,
|
||||
if_exists=IfExists.REPLACE,
|
||||
index=False,
|
||||
)
|
||||
else:
|
||||
msg = f"Unsupported output format: {output_format}"
|
||||
raise ValueError(msg)
|
||||
@@ -325,7 +349,7 @@ def parse_datetime(value: str) -> datetime:
|
||||
return dt
|
||||
|
||||
|
||||
def parse_timeframe(value: str) -> int:
|
||||
def parse_timeframe(value: object) -> int:
|
||||
"""Parse a timeframe string or integer value.
|
||||
|
||||
Args:
|
||||
@@ -337,37 +361,39 @@ def parse_timeframe(value: str) -> int:
|
||||
Raises:
|
||||
ValueError: If the timeframe is invalid.
|
||||
"""
|
||||
upper = value.upper()
|
||||
if upper in TIMEFRAME_MAP:
|
||||
return TIMEFRAME_MAP[upper]
|
||||
try:
|
||||
return int(value)
|
||||
return _parse_timeframe(value)
|
||||
except ValueError:
|
||||
valid = ", ".join(TIMEFRAME_MAP)
|
||||
msg = f"Invalid timeframe: '{value}'. Use one of: {valid}, or an integer."
|
||||
display = value if isinstance(value, str) else repr(value)
|
||||
valid = ", ".join(TIMEFRAME_NAMES)
|
||||
msg = (
|
||||
f"Invalid timeframe: '{display}'. "
|
||||
f"Use one of: {valid}, or a supported integer."
|
||||
)
|
||||
raise ValueError(msg) from None
|
||||
|
||||
|
||||
def parse_tick_flags(value: str) -> int:
|
||||
def parse_tick_flags(value: object) -> int:
|
||||
"""Parse tick flags string or integer value.
|
||||
|
||||
Args:
|
||||
value: Tick flag name (ALL, INFO, TRADE) or integer value.
|
||||
value: Tick flag name (ALL, INFO, TRADE, COPY_TICKS_*) or integer value.
|
||||
|
||||
Returns:
|
||||
Integer tick flag value.
|
||||
Integer tick flag value compatible with MetaTrader 5 ``COPY_TICKS_*``.
|
||||
|
||||
Raises:
|
||||
ValueError: If the flag is invalid.
|
||||
"""
|
||||
upper = value.upper()
|
||||
if upper in TICK_FLAG_MAP:
|
||||
return TICK_FLAG_MAP[upper]
|
||||
try:
|
||||
return int(value)
|
||||
return _parse_copy_ticks(value)
|
||||
except ValueError:
|
||||
valid = ", ".join(TICK_FLAG_MAP)
|
||||
msg = f"Invalid tick flags: '{value}'. Use one of: {valid}, or an integer."
|
||||
display = value if isinstance(value, str) else repr(value)
|
||||
valid = ", ".join(_TICK_FLAG_NAMES)
|
||||
msg = (
|
||||
f"Invalid tick flags: '{display}'. "
|
||||
f"Use one of: {valid}, or a supported integer."
|
||||
)
|
||||
raise ValueError(msg) from None
|
||||
|
||||
|
||||
|
||||
+3
-3
@@ -1,7 +1,7 @@
|
||||
[project]
|
||||
name = "mt5cli"
|
||||
version = "0.4.2"
|
||||
description = "Command-line tool for MetaTrader 5"
|
||||
version = "0.7.2"
|
||||
description = "Generic MT5 data and execution infrastructure for Python applications"
|
||||
authors = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
|
||||
maintainers = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
|
||||
license = "MIT"
|
||||
@@ -9,7 +9,7 @@ license-files = ["LICENSE"]
|
||||
readme = "README.md"
|
||||
requires-python = ">= 3.11, < 3.14"
|
||||
dependencies = [
|
||||
"pdmt5 >= 0.2.3",
|
||||
"pdmt5>=0.3.0",
|
||||
"click >= 8.1.0",
|
||||
"pyarrow >= 19.0.0",
|
||||
"typer >= 0.15.0",
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
"""Shared pytest fixtures for mt5cli tests."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pandas as pd
|
||||
import pytest
|
||||
from pytest_mock import MockerFixture # noqa: TC002
|
||||
|
||||
_DATAFRAME_METHODS = (
|
||||
"copy_rates_from_as_df",
|
||||
"copy_rates_from_pos_as_df",
|
||||
"copy_rates_range_as_df",
|
||||
"copy_ticks_from_as_df",
|
||||
"copy_ticks_range_as_df",
|
||||
"account_info_as_df",
|
||||
"terminal_info_as_df",
|
||||
"symbols_get_as_df",
|
||||
"symbol_info_as_df",
|
||||
"orders_get_as_df",
|
||||
"positions_get_as_df",
|
||||
"history_orders_get_as_df",
|
||||
"history_deals_get_as_df",
|
||||
"version_as_df",
|
||||
"last_error_as_df",
|
||||
"symbol_info_tick_as_df",
|
||||
"market_book_get_as_df",
|
||||
"order_check_as_df",
|
||||
"order_send_as_df",
|
||||
)
|
||||
|
||||
|
||||
def build_mock_mt5_data_client() -> MagicMock:
|
||||
"""Return a MagicMock Mt5DataClient with common DataFrame stubs."""
|
||||
client = MagicMock()
|
||||
sample_df = pd.DataFrame({"col": [1]})
|
||||
for method_name in _DATAFRAME_METHODS:
|
||||
getattr(client, method_name).return_value = sample_df
|
||||
client.version.return_value = (5, 0, 1)
|
||||
client.terminal_info.return_value = {"connected": True, "paths": ["terminal.exe"]}
|
||||
client.account_info.return_value = {"login": 123, "limits": {"modes": ["demo"]}}
|
||||
client.symbols_total.return_value = 42
|
||||
return client
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_client(mocker: MockerFixture) -> MagicMock:
|
||||
"""Create and patch a mock Mt5DataClient for CLI and SDK tests."""
|
||||
client = build_mock_mt5_data_client()
|
||||
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=client)
|
||||
return client
|
||||
+173
-33
@@ -6,7 +6,7 @@ import json
|
||||
import logging
|
||||
import re
|
||||
import sqlite3
|
||||
from datetime import UTC, datetime
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from typing import TYPE_CHECKING
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
@@ -69,34 +69,6 @@ class TestExecuteExport:
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_client(mocker: MockerFixture) -> MagicMock:
|
||||
"""Create and patch a mock Mt5DataClient for CLI tests."""
|
||||
client = MagicMock()
|
||||
sample_df = pd.DataFrame({"col": [1]})
|
||||
client.copy_rates_from_as_df.return_value = sample_df
|
||||
client.copy_rates_from_pos_as_df.return_value = sample_df
|
||||
client.copy_rates_range_as_df.return_value = sample_df
|
||||
client.copy_ticks_from_as_df.return_value = sample_df
|
||||
client.copy_ticks_range_as_df.return_value = sample_df
|
||||
client.account_info_as_df.return_value = sample_df
|
||||
client.terminal_info_as_df.return_value = sample_df
|
||||
client.symbols_get_as_df.return_value = sample_df
|
||||
client.symbol_info_as_df.return_value = sample_df
|
||||
client.orders_get_as_df.return_value = sample_df
|
||||
client.positions_get_as_df.return_value = sample_df
|
||||
client.history_orders_get_as_df.return_value = sample_df
|
||||
client.history_deals_get_as_df.return_value = sample_df
|
||||
client.version_as_df.return_value = sample_df
|
||||
client.last_error_as_df.return_value = sample_df
|
||||
client.symbol_info_tick_as_df.return_value = sample_df
|
||||
client.market_book_get_as_df.return_value = sample_df
|
||||
client.order_check_as_df.return_value = sample_df
|
||||
client.order_send_as_df.return_value = sample_df
|
||||
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=client)
|
||||
return client
|
||||
|
||||
|
||||
class TestCommands:
|
||||
"""Tests for all CLI subcommands via CliRunner."""
|
||||
|
||||
@@ -223,6 +195,37 @@ class TestCommands:
|
||||
count=50,
|
||||
)
|
||||
|
||||
def test_latest_rates(
|
||||
self,
|
||||
tmp_path: Path,
|
||||
mock_client: MagicMock,
|
||||
) -> None:
|
||||
"""Test latest-rates command."""
|
||||
output = tmp_path / "out.csv"
|
||||
result = runner.invoke(
|
||||
app,
|
||||
[
|
||||
"-o",
|
||||
str(output),
|
||||
"latest-rates",
|
||||
"--symbol",
|
||||
"GBPUSD",
|
||||
"--timeframe",
|
||||
"H1",
|
||||
"--count",
|
||||
"50",
|
||||
"--start-pos",
|
||||
"2",
|
||||
],
|
||||
)
|
||||
assert result.exit_code == 0, result.output
|
||||
mock_client.copy_rates_from_pos_as_df.assert_called_once_with(
|
||||
symbol="GBPUSD",
|
||||
timeframe=16385,
|
||||
start_pos=2,
|
||||
count=50,
|
||||
)
|
||||
|
||||
def test_rates_range(
|
||||
self,
|
||||
tmp_path: Path,
|
||||
@@ -282,7 +285,7 @@ class TestCommands:
|
||||
symbol="EURUSD",
|
||||
date_from=datetime(2024, 1, 1, tzinfo=UTC),
|
||||
count=100,
|
||||
flags=1,
|
||||
flags=-1,
|
||||
)
|
||||
|
||||
def test_ticks_range(
|
||||
@@ -313,9 +316,68 @@ class TestCommands:
|
||||
symbol="EURUSD",
|
||||
date_from=datetime(2024, 1, 1, tzinfo=UTC),
|
||||
date_to=datetime(2024, 2, 1, tzinfo=UTC),
|
||||
flags=2,
|
||||
flags=1,
|
||||
)
|
||||
|
||||
def test_ticks_recent(
|
||||
self,
|
||||
tmp_path: Path,
|
||||
mock_client: MagicMock,
|
||||
) -> None:
|
||||
"""Test ticks-recent command."""
|
||||
output = tmp_path / "out.csv"
|
||||
result = runner.invoke(
|
||||
app,
|
||||
[
|
||||
"-o",
|
||||
str(output),
|
||||
"ticks-recent",
|
||||
"--symbol",
|
||||
"EURUSD",
|
||||
"--seconds",
|
||||
"120",
|
||||
"--date-to",
|
||||
"2024-01-02",
|
||||
"--count",
|
||||
"500",
|
||||
"--flags",
|
||||
"ALL",
|
||||
],
|
||||
)
|
||||
assert result.exit_code == 0, result.output
|
||||
mock_client.copy_ticks_from_as_df.assert_called_once_with(
|
||||
symbol="EURUSD",
|
||||
date_from=datetime(2024, 1, 2, tzinfo=UTC) - timedelta(seconds=120),
|
||||
count=500,
|
||||
flags=-1,
|
||||
)
|
||||
mock_client.copy_ticks_range_as_df.assert_not_called()
|
||||
|
||||
def test_minimum_margins(
|
||||
self,
|
||||
tmp_path: Path,
|
||||
mock_client: MagicMock,
|
||||
) -> None:
|
||||
"""Test minimum-margins command."""
|
||||
sym = MagicMock(volume_min=0.01)
|
||||
account = MagicMock(currency="USD")
|
||||
tick = MagicMock(ask=1.1010, bid=1.1000)
|
||||
mock_client.symbol_info.return_value = sym
|
||||
mock_client.account_info.return_value = account
|
||||
mock_client.symbol_info_tick.return_value = tick
|
||||
mock_client.order_calc_margin.side_effect = [12.5, 12.4]
|
||||
mock_client.mt5.ORDER_TYPE_BUY = 0
|
||||
mock_client.mt5.ORDER_TYPE_SELL = 1
|
||||
output = tmp_path / "out.csv"
|
||||
result = runner.invoke(
|
||||
app,
|
||||
["-o", str(output), "minimum-margins", "--symbol", "EURUSD"],
|
||||
)
|
||||
assert result.exit_code == 0, result.output
|
||||
mock_client.symbol_info.assert_called_once_with("EURUSD")
|
||||
mock_client.order_calc_margin.assert_any_call(0, "EURUSD", 0.01, 1.1010)
|
||||
mock_client.order_calc_margin.assert_any_call(1, "EURUSD", 0.01, 1.1000)
|
||||
|
||||
def test_orders(
|
||||
self,
|
||||
tmp_path: Path,
|
||||
@@ -392,6 +454,84 @@ class TestCommands:
|
||||
assert result.exit_code == 0, result.output
|
||||
mock_client.history_deals_get_as_df.assert_called_once()
|
||||
|
||||
def test_recent_history_deals(
|
||||
self,
|
||||
tmp_path: Path,
|
||||
mock_client: MagicMock,
|
||||
) -> None:
|
||||
"""Test recent-history-deals command."""
|
||||
output = tmp_path / "out.csv"
|
||||
result = runner.invoke(
|
||||
app,
|
||||
[
|
||||
"-o",
|
||||
str(output),
|
||||
"recent-history-deals",
|
||||
"--hours",
|
||||
"6",
|
||||
"--date-to",
|
||||
"2024-01-02",
|
||||
"--symbol",
|
||||
"EURUSD",
|
||||
],
|
||||
)
|
||||
assert result.exit_code == 0, result.output
|
||||
mock_client.history_deals_get_as_df.assert_called_once_with(
|
||||
date_from=datetime(2024, 1, 1, 18, tzinfo=UTC),
|
||||
date_to=datetime(2024, 1, 2, tzinfo=UTC),
|
||||
group=None,
|
||||
symbol="EURUSD",
|
||||
ticket=None,
|
||||
position=None,
|
||||
)
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("filename", "reader"),
|
||||
[
|
||||
("summary.csv", "csv"),
|
||||
("summary.json", "json"),
|
||||
("summary.db", "sqlite3"),
|
||||
("summary.parquet", "parquet"),
|
||||
],
|
||||
)
|
||||
def test_mt5_summary_export_formats(
|
||||
self,
|
||||
tmp_path: Path,
|
||||
mock_client: MagicMock,
|
||||
filename: str,
|
||||
reader: str,
|
||||
) -> None:
|
||||
"""Test mt5-summary writes export-safe files for supported formats."""
|
||||
output = tmp_path / filename
|
||||
result = runner.invoke(app, ["-o", str(output), "mt5-summary"])
|
||||
assert result.exit_code == 0, result.output
|
||||
assert output.exists()
|
||||
mock_client.version.assert_called_once()
|
||||
mock_client.terminal_info.assert_called_once()
|
||||
mock_client.account_info.assert_called_once()
|
||||
mock_client.symbols_total.assert_called_once()
|
||||
if reader == "csv":
|
||||
frame = pd.read_csv(output)
|
||||
elif reader == "json":
|
||||
with output.open() as f:
|
||||
records = json.load(f)
|
||||
frame = pd.DataFrame(records)
|
||||
elif reader == "sqlite3":
|
||||
with sqlite3.connect(output) as conn:
|
||||
frame = pd.read_sql( # type: ignore[reportUnknownMemberType]
|
||||
"SELECT * FROM data",
|
||||
conn,
|
||||
)
|
||||
else:
|
||||
frame = pd.read_parquet(output)
|
||||
assert len(frame) == 1
|
||||
assert frame.iloc[0].to_dict() == {
|
||||
"version": "[5,0,1]",
|
||||
"terminal_info": '{"connected":true,"paths":["terminal.exe"]}',
|
||||
"account_info": '{"limits":{"modes":["demo"]},"login":123}',
|
||||
"symbols_total": 42,
|
||||
}
|
||||
|
||||
def test_version(
|
||||
self,
|
||||
tmp_path: Path,
|
||||
@@ -828,7 +968,7 @@ class TestCollectHistory:
|
||||
symbol="EURUSD",
|
||||
date_from=datetime(2024, 1, 1, tzinfo=UTC),
|
||||
date_to=datetime(2024, 2, 1, tzinfo=UTC),
|
||||
flags=1,
|
||||
flags=-1,
|
||||
)
|
||||
with sqlite3.connect(output) as conn:
|
||||
tables = {
|
||||
@@ -1041,7 +1181,7 @@ class TestCollectHistory:
|
||||
symbol="EURUSD",
|
||||
date_from=datetime(2024, 1, 1, tzinfo=UTC),
|
||||
date_to=datetime(2024, 2, 1, tzinfo=UTC),
|
||||
flags=1,
|
||||
flags=-1,
|
||||
)
|
||||
|
||||
def test_collect_history_with_views(
|
||||
|
||||
@@ -0,0 +1,512 @@
|
||||
"""Contract tests for the mt5cli public API and dataset schemas."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import pandas as pd
|
||||
import pytest
|
||||
from pdmt5 import Mt5RuntimeError, Mt5TradingError
|
||||
from pytest_mock import MockerFixture # noqa: TC002
|
||||
|
||||
from mt5cli import (
|
||||
DEDUP_KEYS,
|
||||
REQUIRED_COLUMNS,
|
||||
TIME_COLUMNS,
|
||||
DataKind,
|
||||
Dataset,
|
||||
MT5Client,
|
||||
Mt5CliError,
|
||||
Mt5ConnectionError,
|
||||
Mt5OperationError,
|
||||
Mt5SchemaError,
|
||||
build_config,
|
||||
call_with_normalized_errors,
|
||||
detect_format,
|
||||
ensure_utc,
|
||||
export_dataframe,
|
||||
export_dataframe_to_sqlite,
|
||||
granularity_name,
|
||||
is_recoverable_mt5_error,
|
||||
mt5_session,
|
||||
normalize_dataframe,
|
||||
normalize_mt5_exception,
|
||||
normalize_symbol,
|
||||
normalize_symbols,
|
||||
parse_date_range,
|
||||
recent_window,
|
||||
schema_columns,
|
||||
validate_schema,
|
||||
)
|
||||
from mt5cli.retry import retry_with_backoff
|
||||
from mt5cli.schemas import ensure_utc_columns, normalize_time_columns
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def _sample_frame(kind: DataKind) -> pd.DataFrame:
|
||||
if kind is DataKind.rates:
|
||||
return pd.DataFrame({
|
||||
"time": [datetime(2024, 1, 1, tzinfo=UTC)],
|
||||
"open": [1.1],
|
||||
"high": [1.2],
|
||||
"low": [1.0],
|
||||
"close": [1.15],
|
||||
"tick_volume": [10],
|
||||
"spread": [1],
|
||||
"real_volume": [0],
|
||||
})
|
||||
if kind is DataKind.ticks:
|
||||
return pd.DataFrame({
|
||||
"time": [datetime(2024, 1, 1, tzinfo=UTC)],
|
||||
"bid": [1.1],
|
||||
"ask": [1.11],
|
||||
"last": [1.105],
|
||||
"volume": [1],
|
||||
"time_msc": [datetime(2024, 1, 1, tzinfo=UTC)],
|
||||
"flags": [2],
|
||||
"volume_real": [0.0],
|
||||
})
|
||||
if kind is DataKind.orders:
|
||||
return pd.DataFrame({
|
||||
"ticket": [1],
|
||||
"time_setup": [datetime(2024, 1, 1, tzinfo=UTC)],
|
||||
"type": [0],
|
||||
"state": [1],
|
||||
"symbol": ["EURUSD"],
|
||||
"volume_current": [0.1],
|
||||
"price_open": [1.1],
|
||||
})
|
||||
if kind is DataKind.positions:
|
||||
return pd.DataFrame({
|
||||
"ticket": [1],
|
||||
"time": [datetime(2024, 1, 1, tzinfo=UTC)],
|
||||
"type": [0],
|
||||
"symbol": ["EURUSD"],
|
||||
"volume": [0.1],
|
||||
"price_open": [1.1],
|
||||
"price_current": [1.11],
|
||||
"profit": [1.0],
|
||||
})
|
||||
if kind is DataKind.history_orders:
|
||||
return pd.DataFrame({
|
||||
"ticket": [1],
|
||||
"time_setup": [datetime(2024, 1, 1, tzinfo=UTC)],
|
||||
"type": [0],
|
||||
"state": [3],
|
||||
"symbol": ["EURUSD"],
|
||||
"volume_initial": [0.1],
|
||||
"price_open": [1.1],
|
||||
})
|
||||
return pd.DataFrame({
|
||||
"ticket": [1],
|
||||
"order": [2],
|
||||
"time": [datetime(2024, 1, 1, tzinfo=UTC)],
|
||||
"type": [0],
|
||||
"entry": [0],
|
||||
"symbol": ["EURUSD"],
|
||||
"volume": [0.1],
|
||||
"price": [1.1],
|
||||
"profit": [0.0],
|
||||
})
|
||||
|
||||
|
||||
@pytest.mark.parametrize("kind", list(DataKind))
|
||||
def test_required_columns_contract(kind: DataKind) -> None:
|
||||
"""Each dataset kind exposes a non-empty required column contract."""
|
||||
assert REQUIRED_COLUMNS[kind]
|
||||
validate_schema(_sample_frame(kind), kind)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("kind", list(DataKind))
|
||||
def test_normalize_dataframe_injects_storage_metadata(kind: DataKind) -> None:
|
||||
"""Normalization accepts MT5 frames and optional storage metadata."""
|
||||
frame = _sample_frame(kind)
|
||||
normalized = normalize_dataframe(
|
||||
frame,
|
||||
kind,
|
||||
symbol="eurusd",
|
||||
timeframe="M1" if kind is DataKind.rates else None,
|
||||
)
|
||||
if kind is DataKind.rates:
|
||||
assert normalized.loc[0, "symbol"] == "eurusd"
|
||||
assert normalized.loc[0, "timeframe"] == 1
|
||||
validate_schema(normalized, kind)
|
||||
|
||||
|
||||
def test_validate_schema_raises_for_missing_columns() -> None:
|
||||
"""Schema validation fails fast on missing required columns."""
|
||||
with pytest.raises(Mt5SchemaError, match="missing required columns"):
|
||||
validate_schema(pd.DataFrame({"time": [1]}), DataKind.rates)
|
||||
|
||||
|
||||
def test_history_dedup_keys_match_schema_contract() -> None:
|
||||
"""SQLite history dedup keys stay aligned with schema contracts."""
|
||||
assert DEDUP_KEYS[DataKind.rates][0] == ("symbol", "timeframe", "time")
|
||||
assert DEDUP_KEYS[DataKind.ticks][0] == ("symbol", "time_msc")
|
||||
assert Dataset.rates.table_name == "rates"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("raw", "expected"),
|
||||
[
|
||||
(" eurusd ", "eurusd"),
|
||||
("GbpJpy", "GbpJpy"),
|
||||
("XAUUSDm", "XAUUSDm"),
|
||||
("US500.cash", "US500.cash"),
|
||||
("EURUSD.r", "EURUSD.r"),
|
||||
],
|
||||
)
|
||||
def test_normalize_symbol(raw: str, expected: str) -> None:
|
||||
"""Symbol normalization trims whitespace and preserves broker casing."""
|
||||
assert normalize_symbol(raw) == expected
|
||||
|
||||
|
||||
def test_normalize_symbols_deduplicates() -> None:
|
||||
"""Symbol lists are normalized and de-duplicated in order."""
|
||||
assert normalize_symbols(["XAUUSDm", " XAUUSDm ", "EURUSD.r", "eurusd"]) == [
|
||||
"XAUUSDm",
|
||||
"EURUSD.r",
|
||||
"eurusd",
|
||||
]
|
||||
|
||||
|
||||
def test_parse_date_range_rejects_inverted_bounds() -> None:
|
||||
"""Date ranges must not be inverted."""
|
||||
with pytest.raises(ValueError, match="must not be after"):
|
||||
parse_date_range("2024-02-01", "2024-01-01")
|
||||
|
||||
|
||||
def test_recent_window_builds_trailing_bounds() -> None:
|
||||
"""Recent windows end at the provided timestamp."""
|
||||
end = datetime(2024, 1, 2, tzinfo=UTC)
|
||||
start, resolved_end = recent_window(hours=24, date_to=end)
|
||||
assert resolved_end == end
|
||||
assert start < end
|
||||
|
||||
|
||||
def test_granularity_name_maps_timeframe_alias() -> None:
|
||||
"""Granularity labels resolve MT5 timeframe aliases."""
|
||||
assert granularity_name("M1") == "M1"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"exc",
|
||||
[Mt5RuntimeError("init failed"), Mt5TradingError("trade failed")],
|
||||
)
|
||||
def test_is_recoverable_mt5_error(exc: Exception) -> None:
|
||||
"""Recoverable MT5 errors are classified consistently."""
|
||||
assert is_recoverable_mt5_error(exc)
|
||||
|
||||
|
||||
def test_normalize_mt5_exception_maps_types() -> None:
|
||||
"""MT5 exceptions map to stable mt5cli types."""
|
||||
assert isinstance(
|
||||
normalize_mt5_exception(Mt5RuntimeError("x")),
|
||||
Mt5ConnectionError,
|
||||
)
|
||||
assert isinstance(
|
||||
normalize_mt5_exception(Mt5TradingError("x")),
|
||||
Mt5OperationError,
|
||||
)
|
||||
|
||||
|
||||
def test_call_with_normalized_errors_reraises_mapped_type() -> None:
|
||||
"""Normalized error helper re-raises mapped mt5cli exceptions."""
|
||||
|
||||
def _raise() -> None:
|
||||
message = "boom"
|
||||
raise Mt5RuntimeError(message)
|
||||
|
||||
with pytest.raises(Mt5ConnectionError):
|
||||
call_with_normalized_errors(_raise)
|
||||
|
||||
|
||||
def test_retry_with_backoff_retries_recoverable_errors(
|
||||
mocker: MockerFixture,
|
||||
) -> None:
|
||||
"""Retry helper retries recoverable MT5 failures."""
|
||||
calls = {"count": 0}
|
||||
|
||||
def _flaky() -> str:
|
||||
calls["count"] += 1
|
||||
if calls["count"] == 1:
|
||||
message = "transient"
|
||||
raise Mt5RuntimeError(message)
|
||||
return "ok"
|
||||
|
||||
mocker.patch("mt5cli.retry.time.sleep")
|
||||
assert retry_with_backoff(_flaky, retry_count=1) == "ok"
|
||||
assert calls["count"] == 2
|
||||
|
||||
|
||||
def test_public_api_exports_mt5_client() -> None:
|
||||
"""MT5Client is the primary importable client abstraction."""
|
||||
client = MT5Client(config=build_config())
|
||||
assert isinstance(client, MT5Client)
|
||||
assert isinstance(client, MT5Client.__mro__[1])
|
||||
|
||||
|
||||
def test_mt5_client_order_primitives_use_connected_client(
|
||||
mock_client: object,
|
||||
) -> None:
|
||||
"""Order check/send route through the same client fetch path as exports."""
|
||||
request = {"action": 1}
|
||||
client = MT5Client()
|
||||
client.order_check(request)
|
||||
client.order_send(request)
|
||||
assert mock_client.order_check_as_df.call_count == 1 # type: ignore[attr-defined]
|
||||
assert mock_client.order_send_as_df.call_count == 1 # type: ignore[attr-defined]
|
||||
|
||||
|
||||
def test_storage_export_round_trip_csv(tmp_path: Path) -> None:
|
||||
"""Storage helpers export normalized rate frames to CSV."""
|
||||
frame = normalize_dataframe(
|
||||
_sample_frame(DataKind.rates),
|
||||
DataKind.rates,
|
||||
symbol="EURUSD",
|
||||
timeframe="M1",
|
||||
)
|
||||
output = tmp_path / "rates.csv"
|
||||
export_dataframe(frame, output, detect_format(output))
|
||||
loaded = pd.read_csv(output)
|
||||
assert len(loaded) == 1
|
||||
assert "close" in loaded.columns
|
||||
|
||||
|
||||
def test_normalize_symbol_rejects_empty_value() -> None:
|
||||
"""Empty symbols are rejected after trimming."""
|
||||
with pytest.raises(ValueError, match="must not be empty"):
|
||||
normalize_symbol(" ")
|
||||
|
||||
|
||||
def test_ensure_utc_handles_naive_and_aware_datetimes() -> None:
|
||||
"""UTC coercion accepts naive and timezone-aware datetimes."""
|
||||
naive = datetime(2024, 1, 1, tzinfo=UTC).replace(tzinfo=None)
|
||||
aware = datetime(2024, 1, 1, tzinfo=UTC)
|
||||
assert ensure_utc(naive).tzinfo == UTC
|
||||
assert ensure_utc(aware).tzinfo == UTC
|
||||
assert ensure_utc("2024-01-01T00:00:00+00:00").tzinfo == UTC
|
||||
|
||||
|
||||
def test_recent_window_validation_errors() -> None:
|
||||
"""Recent window helpers validate mutually exclusive length arguments."""
|
||||
with pytest.raises(ValueError, match="exactly one"):
|
||||
recent_window()
|
||||
with pytest.raises(ValueError, match="exactly one"):
|
||||
recent_window(hours=1, seconds=1)
|
||||
with pytest.raises(ValueError, match="positive"):
|
||||
recent_window(hours=0)
|
||||
|
||||
|
||||
def test_recent_window_supports_seconds_argument() -> None:
|
||||
"""Recent windows can be built from a seconds-based length."""
|
||||
end = datetime(2024, 1, 2, tzinfo=UTC)
|
||||
start, resolved_end = recent_window(seconds=3600, date_to=end)
|
||||
assert resolved_end == end
|
||||
assert start < end
|
||||
|
||||
|
||||
def test_parse_date_range_returns_ordered_bounds() -> None:
|
||||
"""Valid date ranges return UTC-aware bounds."""
|
||||
start, end = parse_date_range("2024-01-01", "2024-02-01")
|
||||
assert start < end
|
||||
|
||||
|
||||
def test_granularity_name_falls_back_for_unknown_timeframe(
|
||||
mocker: MockerFixture,
|
||||
) -> None:
|
||||
"""Unknown timeframe integers stringify as granularity labels."""
|
||||
mocker.patch(
|
||||
"mt5cli.converters._get_timeframe_name",
|
||||
side_effect=ValueError("unknown"),
|
||||
)
|
||||
assert granularity_name(1) == "1"
|
||||
|
||||
|
||||
def test_normalize_mt5_exception_passthrough_and_generic() -> None:
|
||||
"""Normalization preserves mt5cli errors and wraps unknown exceptions."""
|
||||
original = Mt5CliError("known")
|
||||
assert normalize_mt5_exception(original) is original
|
||||
assert isinstance(normalize_mt5_exception(ValueError("x")), Mt5CliError)
|
||||
|
||||
|
||||
def test_schema_columns_and_extra_required_validation() -> None:
|
||||
"""Schema helpers expose contracts and honor extra required columns."""
|
||||
assert schema_columns(DataKind.rates) == REQUIRED_COLUMNS[DataKind.rates]
|
||||
validate_schema(pd.DataFrame(), DataKind.rates)
|
||||
frame = _sample_frame(DataKind.rates)
|
||||
with pytest.raises(Mt5SchemaError, match="storage_symbol"):
|
||||
validate_schema(frame, DataKind.rates, extra_required=["storage_symbol"])
|
||||
|
||||
|
||||
def test_normalize_dataframe_empty_and_tick_sort_paths() -> None:
|
||||
"""Normalization handles empty frames and tick time_msc sorting."""
|
||||
empty = pd.DataFrame()
|
||||
assert normalize_dataframe(empty, DataKind.rates).empty
|
||||
|
||||
ticks = _sample_frame(DataKind.ticks)
|
||||
ticks = pd.concat([ticks, ticks], ignore_index=True)
|
||||
sorted_ticks = normalize_dataframe(ticks, DataKind.ticks, sort=True)
|
||||
assert len(sorted_ticks) == 2
|
||||
unsorted_ticks = normalize_dataframe(ticks, DataKind.ticks, sort=False)
|
||||
assert len(unsorted_ticks) == 2
|
||||
|
||||
|
||||
def test_normalize_dataframe_rate_timeframe_without_symbol() -> None:
|
||||
"""Rate normalization can inject timeframe without symbol metadata."""
|
||||
frame = _sample_frame(DataKind.rates)
|
||||
normalized = normalize_dataframe(frame, DataKind.rates, timeframe="M1")
|
||||
assert "timeframe" in normalized.columns
|
||||
|
||||
|
||||
def test_normalize_dataframe_keeps_existing_symbol_and_timeframe() -> None:
|
||||
"""Normalization does not duplicate existing storage metadata columns."""
|
||||
frame = normalize_dataframe(
|
||||
_sample_frame(DataKind.rates),
|
||||
DataKind.rates,
|
||||
symbol="EURUSD",
|
||||
timeframe="M1",
|
||||
)
|
||||
normalized = normalize_dataframe(
|
||||
frame,
|
||||
DataKind.rates,
|
||||
symbol="GBPUSD",
|
||||
timeframe="H1",
|
||||
)
|
||||
assert normalized.loc[0, "symbol"] == "EURUSD"
|
||||
assert normalized.loc[0, "timeframe"] == 1
|
||||
|
||||
|
||||
def test_normalize_time_columns_skips_absent_time_fields() -> None:
|
||||
"""Time normalization ignores absent optional time columns."""
|
||||
frame = pd.DataFrame({"open": [1.0]})
|
||||
result = normalize_time_columns(frame, DataKind.rates)
|
||||
assert list(result.columns) == ["open"]
|
||||
|
||||
|
||||
def test_normalize_time_columns_converts_unix_seconds() -> None:
|
||||
"""Numeric MT5 ``time`` values are interpreted as Unix seconds."""
|
||||
frame = pd.DataFrame({"time": [1704067200]})
|
||||
result = normalize_time_columns(frame, DataKind.rates)
|
||||
assert result.loc[0, "time"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
|
||||
|
||||
|
||||
def test_normalize_time_columns_converts_unix_milliseconds() -> None:
|
||||
"""Numeric MT5 ``time_msc`` values are interpreted as Unix milliseconds."""
|
||||
frame = pd.DataFrame({"time_msc": [1704067200000]})
|
||||
result = normalize_time_columns(frame, DataKind.ticks)
|
||||
assert result.loc[0, "time_msc"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
|
||||
|
||||
|
||||
def test_normalize_time_columns_preserves_utc_datetimes() -> None:
|
||||
"""Already-converted datetime values remain UTC-normalized."""
|
||||
aware = datetime(2024, 1, 1, tzinfo=UTC)
|
||||
frame = pd.DataFrame({"time": [aware]})
|
||||
result = normalize_time_columns(frame, DataKind.rates)
|
||||
assert result.loc[0, "time"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
|
||||
|
||||
|
||||
def test_normalize_time_columns_handles_optional_order_times() -> None:
|
||||
"""Optional order/history time columns are normalized when present."""
|
||||
frame = pd.DataFrame({
|
||||
"time_setup": [1704067200],
|
||||
"time_setup_msc": [1704067200000],
|
||||
"time_done": [1704153600],
|
||||
"time_done_msc": [1704153600000],
|
||||
})
|
||||
result = normalize_time_columns(frame, DataKind.orders)
|
||||
assert result.loc[0, "time_setup"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
|
||||
assert result.loc[0, "time_setup_msc"] == pd.Timestamp(
|
||||
"2024-01-01T00:00:00+00:00",
|
||||
)
|
||||
assert result.loc[0, "time_done"] == pd.Timestamp("2024-01-02T00:00:00+00:00")
|
||||
assert result.loc[0, "time_done_msc"] == pd.Timestamp(
|
||||
"2024-01-02T00:00:00+00:00",
|
||||
)
|
||||
|
||||
|
||||
def test_time_columns_include_optional_order_fields() -> None:
|
||||
"""Schema contracts document optional MT5 time columns per dataset kind."""
|
||||
assert "time_done" in TIME_COLUMNS[DataKind.orders]
|
||||
assert "time_setup_msc" in TIME_COLUMNS[DataKind.history_orders]
|
||||
|
||||
|
||||
def test_normalize_dataframe_sorts_ticks_by_time_msc(
|
||||
mocker: MockerFixture,
|
||||
) -> None:
|
||||
"""Tick frames without ``time`` can still sort on ``time_msc``."""
|
||||
mocker.patch("mt5cli.schemas.validate_schema")
|
||||
ticks = pd.concat([_sample_frame(DataKind.ticks)] * 2, ignore_index=True).drop(
|
||||
columns=["time"],
|
||||
)
|
||||
ticks.loc[0, "time_msc"] = datetime(2024, 1, 1, tzinfo=UTC)
|
||||
ticks.loc[1, "time_msc"] = datetime(2024, 1, 2, tzinfo=UTC)
|
||||
ticks = pd.concat([ticks.iloc[[1]], ticks.iloc[[0]]], ignore_index=True)
|
||||
normalized = normalize_dataframe(ticks, DataKind.ticks, sort=True)
|
||||
assert normalized.iloc[0]["time_msc"] <= normalized.iloc[1]["time_msc"]
|
||||
|
||||
|
||||
def test_ensure_utc_columns_skips_missing_columns() -> None:
|
||||
"""UTC column coercion ignores absent columns."""
|
||||
frame = _sample_frame(DataKind.rates)
|
||||
result = ensure_utc_columns(frame, ["time", "missing"])
|
||||
assert "time" in result.columns
|
||||
|
||||
|
||||
def test_normalize_time_columns_coerces_string_timestamps() -> None:
|
||||
"""String timestamps are parsed with timezone-aware datetime coercion."""
|
||||
frame = pd.DataFrame({"time": ["2024-01-01T00:00:00+00:00"]})
|
||||
result = normalize_time_columns(frame, DataKind.rates)
|
||||
assert result.loc[0, "time"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
|
||||
|
||||
|
||||
def test_ensure_utc_columns_coerces_non_mt5_columns() -> None:
|
||||
"""Non-MT5 columns still coerce to UTC datetimes."""
|
||||
frame = pd.DataFrame({"created_at": ["2024-01-01T00:00:00+00:00"]})
|
||||
result = ensure_utc_columns(frame, ["created_at"])
|
||||
assert result.loc[0, "created_at"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
|
||||
|
||||
|
||||
def test_mt5_session_yields_connected_client(mocker: MockerFixture) -> None:
|
||||
"""Public mt5_session yields an MT5Client bound to a connected session."""
|
||||
connected = mocker.MagicMock()
|
||||
context = mocker.MagicMock()
|
||||
context.__enter__.return_value = connected
|
||||
context.__exit__.return_value = False
|
||||
mocker.patch("mt5cli.client.connected_client", return_value=context)
|
||||
with mt5_session(build_config()) as client:
|
||||
assert isinstance(client, MT5Client)
|
||||
|
||||
|
||||
def test_retry_with_backoff_reraises_non_recoverable_errors() -> None:
|
||||
"""Non-MT5 errors are not retried."""
|
||||
|
||||
def _raise() -> None:
|
||||
message = "fatal"
|
||||
raise ValueError(message)
|
||||
|
||||
with pytest.raises(ValueError, match="fatal"):
|
||||
retry_with_backoff(_raise, retry_count=2)
|
||||
|
||||
|
||||
def test_storage_export_round_trip_sqlite(tmp_path: Path) -> None:
|
||||
"""Storage helpers append deduplicated frames to SQLite."""
|
||||
frame = normalize_dataframe(
|
||||
_sample_frame(DataKind.rates),
|
||||
DataKind.rates,
|
||||
symbol="EURUSD",
|
||||
timeframe="M1",
|
||||
)
|
||||
output = tmp_path / "rates.db"
|
||||
export_dataframe_to_sqlite(
|
||||
frame,
|
||||
output,
|
||||
"rates",
|
||||
deduplicate_on=DEDUP_KEYS[DataKind.rates][0],
|
||||
)
|
||||
with __import__("sqlite3").connect(output) as conn:
|
||||
count = conn.execute("SELECT COUNT(*) FROM rates").fetchone()[0]
|
||||
assert count == 1
|
||||
+988
-4
File diff suppressed because it is too large
Load Diff
+1270
-35
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,356 @@
|
||||
"""Tests for trading session helpers and operational utilities."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pandas as pd
|
||||
import pytest
|
||||
from pdmt5 import Mt5RuntimeError
|
||||
from pytest_mock import MockerFixture # noqa: TC002
|
||||
|
||||
from mt5cli.sdk import build_config
|
||||
from mt5cli.trading import (
|
||||
calculate_margin_and_volume,
|
||||
detect_position_side,
|
||||
determine_order_limits,
|
||||
mt5_trading_session,
|
||||
)
|
||||
|
||||
|
||||
class TestDetectPositionSide:
|
||||
"""Tests for detect_position_side."""
|
||||
|
||||
def test_returns_none_when_no_positions(self) -> None:
|
||||
"""Test None is returned when no open positions exist."""
|
||||
client = MagicMock()
|
||||
client.positions_get_as_df.return_value = pd.DataFrame()
|
||||
|
||||
assert detect_position_side(client, "EURUSD") is None
|
||||
|
||||
def test_returns_long_for_net_buy_volume(self) -> None:
|
||||
"""Test long is returned when buy volume exceeds sell volume."""
|
||||
client = MagicMock()
|
||||
client.mt5.POSITION_TYPE_BUY = 0
|
||||
client.mt5.POSITION_TYPE_SELL = 1
|
||||
client.positions_get_as_df.return_value = pd.DataFrame(
|
||||
{
|
||||
"type": [0, 0, 1],
|
||||
"volume": [0.2, 0.1, 0.05],
|
||||
},
|
||||
)
|
||||
|
||||
assert detect_position_side(client, "EURUSD") == "long"
|
||||
|
||||
def test_returns_short_for_net_sell_volume(self) -> None:
|
||||
"""Test short is returned when sell volume exceeds buy volume."""
|
||||
client = MagicMock()
|
||||
client.mt5.POSITION_TYPE_BUY = 0
|
||||
client.mt5.POSITION_TYPE_SELL = 1
|
||||
client.positions_get_as_df.return_value = pd.DataFrame(
|
||||
{
|
||||
"type": [1, 1],
|
||||
"volume": [0.3, 0.1],
|
||||
},
|
||||
)
|
||||
|
||||
assert detect_position_side(client, "EURUSD") == "short"
|
||||
|
||||
def test_returns_none_for_balanced_hedged_positions(self) -> None:
|
||||
"""Test None is returned when buy and sell volumes net to zero."""
|
||||
client = MagicMock()
|
||||
client.mt5.POSITION_TYPE_BUY = 0
|
||||
client.mt5.POSITION_TYPE_SELL = 1
|
||||
client.positions_get_as_df.return_value = pd.DataFrame(
|
||||
{
|
||||
"type": [0, 1],
|
||||
"volume": [0.2, 0.2],
|
||||
},
|
||||
)
|
||||
|
||||
assert detect_position_side(client, "EURUSD") is None
|
||||
|
||||
|
||||
class TestCalculateMarginAndVolume:
|
||||
"""Tests for calculate_margin_and_volume."""
|
||||
|
||||
def test_calculates_margin_budget_and_volumes(self) -> None:
|
||||
"""Test margin budget and buy/sell volumes are derived from ratios."""
|
||||
client = MagicMock()
|
||||
client.account_info_as_dict.return_value = {"margin_free": 1000.0}
|
||||
client.calculate_volume_by_margin.side_effect = [0.3, 0.2]
|
||||
|
||||
result = calculate_margin_and_volume(
|
||||
client,
|
||||
"EURUSD",
|
||||
unit_margin_ratio=0.5,
|
||||
preserved_margin_ratio=0.2,
|
||||
)
|
||||
|
||||
assert result == {
|
||||
"margin_free": 1000.0,
|
||||
"available_margin": 800.0,
|
||||
"trade_margin": 400.0,
|
||||
"buy_volume": 0.3,
|
||||
"sell_volume": 0.2,
|
||||
}
|
||||
client.calculate_volume_by_margin.assert_any_call("EURUSD", 400.0, "BUY")
|
||||
client.calculate_volume_by_margin.assert_any_call("EURUSD", 400.0, "SELL")
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("account_dict", "expected_margin_free"),
|
||||
[
|
||||
({"margin_free": 0.0}, 0.0),
|
||||
({}, 0.0),
|
||||
({"margin_free": None}, 0.0),
|
||||
],
|
||||
)
|
||||
def test_zero_or_missing_margin_free(
|
||||
self,
|
||||
account_dict: dict[str, float | None],
|
||||
expected_margin_free: float,
|
||||
) -> None:
|
||||
"""Test missing or zero margin_free yields zero trade margin."""
|
||||
client = MagicMock()
|
||||
client.account_info_as_dict.return_value = account_dict
|
||||
client.calculate_volume_by_margin.return_value = 0.0
|
||||
|
||||
result = calculate_margin_and_volume(
|
||||
client,
|
||||
"EURUSD",
|
||||
unit_margin_ratio=0.5,
|
||||
preserved_margin_ratio=0.2,
|
||||
)
|
||||
|
||||
assert result["margin_free"] == expected_margin_free
|
||||
client.calculate_volume_by_margin.assert_any_call("EURUSD", 0.0, "BUY")
|
||||
client.calculate_volume_by_margin.assert_any_call("EURUSD", 0.0, "SELL")
|
||||
|
||||
def test_clamps_negative_margin_free_to_zero(self) -> None:
|
||||
"""Test negative margin_free is clamped to zero before sizing."""
|
||||
client = MagicMock()
|
||||
client.account_info_as_dict.return_value = {"margin_free": -500.0}
|
||||
client.calculate_volume_by_margin.return_value = 0.0
|
||||
|
||||
result = calculate_margin_and_volume(
|
||||
client,
|
||||
"EURUSD",
|
||||
unit_margin_ratio=0.5,
|
||||
preserved_margin_ratio=0.2,
|
||||
)
|
||||
|
||||
expected_margin_free = 0.0
|
||||
assert result["margin_free"] == expected_margin_free
|
||||
client.calculate_volume_by_margin.assert_any_call("EURUSD", 0.0, "BUY")
|
||||
client.calculate_volume_by_margin.assert_any_call("EURUSD", 0.0, "SELL")
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("unit_ratio", "preserved_ratio"),
|
||||
[
|
||||
(-0.1, 0.0),
|
||||
(1.1, 0.0),
|
||||
(0.5, -0.1),
|
||||
(0.5, 1.1),
|
||||
],
|
||||
)
|
||||
def test_rejects_invalid_ratios(
|
||||
self,
|
||||
unit_ratio: float,
|
||||
preserved_ratio: float,
|
||||
) -> None:
|
||||
"""Test invalid ratio values raise ValueError."""
|
||||
with pytest.raises(ValueError, match="must be between 0 and 1"):
|
||||
calculate_margin_and_volume(
|
||||
MagicMock(),
|
||||
"EURUSD",
|
||||
unit_margin_ratio=unit_ratio,
|
||||
preserved_margin_ratio=preserved_ratio,
|
||||
)
|
||||
|
||||
|
||||
class TestDetermineOrderLimits:
|
||||
"""Tests for determine_order_limits."""
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("side", "expected_entry_key"),
|
||||
[
|
||||
("long", "ask"),
|
||||
("short", "bid"),
|
||||
("buy", "ask"),
|
||||
("sell", "bid"),
|
||||
],
|
||||
)
|
||||
def test_uses_expected_quote_for_entry(
|
||||
self,
|
||||
side: str,
|
||||
expected_entry_key: str,
|
||||
) -> None:
|
||||
"""Test entry price is taken from ask for long/buy and bid for short/sell."""
|
||||
client = MagicMock()
|
||||
client.symbol_info_tick_as_dict.return_value = {"ask": 1.1010, "bid": 1.1000}
|
||||
|
||||
result = determine_order_limits(
|
||||
client,
|
||||
"EURUSD",
|
||||
side,
|
||||
stop_loss_limit_ratio=0.0,
|
||||
take_profit_limit_ratio=0.0,
|
||||
)
|
||||
|
||||
assert (
|
||||
result["entry"]
|
||||
== client.symbol_info_tick_as_dict.return_value[expected_entry_key]
|
||||
)
|
||||
assert result["stop_loss"] is None
|
||||
assert result["take_profit"] is None
|
||||
|
||||
def test_calculates_long_protective_levels(self) -> None:
|
||||
"""Test long stop loss and take profit are placed below/above entry."""
|
||||
client = MagicMock()
|
||||
client.symbol_info_tick_as_dict.return_value = {"ask": 100.0, "bid": 99.0}
|
||||
|
||||
result = determine_order_limits(
|
||||
client,
|
||||
"EURUSD",
|
||||
"long",
|
||||
stop_loss_limit_ratio=0.02,
|
||||
take_profit_limit_ratio=0.03,
|
||||
)
|
||||
|
||||
assert result == {
|
||||
"entry": 100.0,
|
||||
"stop_loss": 98.0,
|
||||
"take_profit": 103.0,
|
||||
}
|
||||
|
||||
def test_calculates_short_protective_levels(self) -> None:
|
||||
"""Test short stop loss and take profit are placed above/below entry."""
|
||||
client = MagicMock()
|
||||
client.symbol_info_tick_as_dict.return_value = {"ask": 100.0, "bid": 99.0}
|
||||
|
||||
result = determine_order_limits(
|
||||
client,
|
||||
"EURUSD",
|
||||
"short",
|
||||
stop_loss_limit_ratio=0.02,
|
||||
take_profit_limit_ratio=0.03,
|
||||
)
|
||||
|
||||
assert result == {
|
||||
"entry": 99.0,
|
||||
"stop_loss": 100.98,
|
||||
"take_profit": 96.03,
|
||||
}
|
||||
|
||||
def test_rejects_unknown_side(self) -> None:
|
||||
"""Test unsupported side values raise ValueError."""
|
||||
with pytest.raises(ValueError, match="Unsupported order side"):
|
||||
determine_order_limits(
|
||||
MagicMock(),
|
||||
"EURUSD",
|
||||
"flat",
|
||||
stop_loss_limit_ratio=0.01,
|
||||
take_profit_limit_ratio=0.01,
|
||||
)
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("stop_loss_ratio", "take_profit_ratio"),
|
||||
[
|
||||
(-0.05, 0.01),
|
||||
(0.01, 2.0),
|
||||
],
|
||||
)
|
||||
def test_rejects_invalid_protective_ratios(
|
||||
self,
|
||||
stop_loss_ratio: float,
|
||||
take_profit_ratio: float,
|
||||
) -> None:
|
||||
"""Test out-of-range protective ratios raise ValueError."""
|
||||
with pytest.raises(ValueError, match="must be at least 0 and less than 1"):
|
||||
determine_order_limits(
|
||||
MagicMock(),
|
||||
"EURUSD",
|
||||
"long",
|
||||
stop_loss_limit_ratio=stop_loss_ratio,
|
||||
take_profit_limit_ratio=take_profit_ratio,
|
||||
)
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("field", "ratio"),
|
||||
[
|
||||
("stop_loss_limit_ratio", 1.0),
|
||||
("take_profit_limit_ratio", 1.0),
|
||||
],
|
||||
)
|
||||
def test_rejects_unit_boundary_protective_ratios(
|
||||
self,
|
||||
field: str,
|
||||
ratio: float,
|
||||
) -> None:
|
||||
"""Test protective ratios of exactly 1.0 are rejected."""
|
||||
kwargs = {
|
||||
"stop_loss_limit_ratio": 0.01,
|
||||
"take_profit_limit_ratio": 0.01,
|
||||
field: ratio,
|
||||
}
|
||||
with pytest.raises(ValueError, match="must be at least 0 and less than 1"):
|
||||
determine_order_limits(
|
||||
MagicMock(),
|
||||
"EURUSD",
|
||||
"long",
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
|
||||
class TestMt5TradingSession:
|
||||
"""Tests for the mt5_trading_session context manager."""
|
||||
|
||||
def test_yields_connected_client_and_shuts_down(
|
||||
self,
|
||||
mocker: MockerFixture,
|
||||
) -> None:
|
||||
"""Test mt5_trading_session connects, yields a client, and shuts down."""
|
||||
mock_client = MagicMock()
|
||||
trading_client = mocker.patch(
|
||||
"mt5cli.trading.Mt5TradingClient",
|
||||
return_value=mock_client,
|
||||
)
|
||||
|
||||
with mt5_trading_session(
|
||||
build_config(path="/opt/mt5/terminal64.exe"),
|
||||
retry_count=2,
|
||||
) as client:
|
||||
mock_client.initialize_and_login_mt5.assert_called_once()
|
||||
assert client is mock_client
|
||||
|
||||
trading_client.assert_called_once()
|
||||
assert trading_client.call_args.kwargs["retry_count"] == 2
|
||||
assert (
|
||||
trading_client.call_args.kwargs["config"].path == "/opt/mt5/terminal64.exe"
|
||||
)
|
||||
mock_client.shutdown.assert_called_once()
|
||||
|
||||
def test_shuts_down_when_initialize_raises(
|
||||
self,
|
||||
mocker: MockerFixture,
|
||||
) -> None:
|
||||
"""Test shutdown is called when initialization fails."""
|
||||
mock_client = MagicMock()
|
||||
mock_client.initialize_and_login_mt5.side_effect = Mt5RuntimeError("boom")
|
||||
mocker.patch("mt5cli.trading.Mt5TradingClient", return_value=mock_client)
|
||||
|
||||
with pytest.raises(Mt5RuntimeError, match="boom"), mt5_trading_session():
|
||||
pass
|
||||
|
||||
mock_client.shutdown.assert_called_once()
|
||||
|
||||
def test_shuts_down_when_body_raises(self, mocker: MockerFixture) -> None:
|
||||
"""Test shutdown is called when the context body raises."""
|
||||
mock_client = MagicMock()
|
||||
mocker.patch("mt5cli.trading.Mt5TradingClient", return_value=mock_client)
|
||||
|
||||
body_error = "body error"
|
||||
with pytest.raises(RuntimeError, match=body_error), mt5_trading_session():
|
||||
raise RuntimeError(body_error)
|
||||
|
||||
mock_client.shutdown.assert_called_once()
|
||||
+159
-14
@@ -21,8 +21,10 @@ from mt5cli.utils import (
|
||||
TIMEFRAME_MAP,
|
||||
TIMEFRAME_TYPE,
|
||||
Dataset,
|
||||
IfExists,
|
||||
detect_format,
|
||||
export_dataframe,
|
||||
export_dataframe_to_sqlite,
|
||||
parse_datetime,
|
||||
parse_request,
|
||||
parse_tick_flags,
|
||||
@@ -130,6 +132,112 @@ class TestExportDataframe:
|
||||
export_dataframe(sample_df, tmp_path / "out.txt", "xml")
|
||||
|
||||
|
||||
class TestExportDataframeToSqlite:
|
||||
"""Tests for export_dataframe_to_sqlite."""
|
||||
|
||||
def test_append_preserves_existing_rows(self, tmp_path: Path) -> None:
|
||||
"""Test append mode keeps prior rows in the SQLite table."""
|
||||
output = tmp_path / "append.db"
|
||||
first = pd.DataFrame({"id": [1], "value": ["a"]})
|
||||
second = pd.DataFrame({"id": [2], "value": ["b"]})
|
||||
export_dataframe_to_sqlite(first, output, "items", if_exists=IfExists.REPLACE)
|
||||
export_dataframe_to_sqlite(second, output, "items", if_exists=IfExists.APPEND)
|
||||
with sqlite3.connect(output) as conn:
|
||||
result = pd.read_sql( # type: ignore[reportUnknownMemberType]
|
||||
"SELECT id, value FROM items ORDER BY id",
|
||||
conn,
|
||||
)
|
||||
pd.testing.assert_frame_equal(
|
||||
result,
|
||||
pd.DataFrame({"id": [1, 2], "value": ["a", "b"]}),
|
||||
)
|
||||
|
||||
def test_deduplicate_keeps_latest_row(self, tmp_path: Path) -> None:
|
||||
"""Test deduplication keeps the latest ROWID for key columns."""
|
||||
output = tmp_path / "dedup.db"
|
||||
first = pd.DataFrame({
|
||||
"symbol": ["EURUSD", "EURUSD"],
|
||||
"time": ["2024-01-01", "2024-01-01"],
|
||||
"bid": [1.0, 1.1],
|
||||
})
|
||||
second = pd.DataFrame({
|
||||
"symbol": ["EURUSD"],
|
||||
"time": ["2024-01-01"],
|
||||
"bid": [1.2],
|
||||
})
|
||||
export_dataframe_to_sqlite(
|
||||
first,
|
||||
output,
|
||||
"ticks",
|
||||
if_exists=IfExists.REPLACE,
|
||||
deduplicate_on=("symbol", "time"),
|
||||
)
|
||||
export_dataframe_to_sqlite(
|
||||
second,
|
||||
output,
|
||||
"ticks",
|
||||
if_exists=IfExists.APPEND,
|
||||
deduplicate_on=("symbol", "time"),
|
||||
)
|
||||
with sqlite3.connect(output) as conn:
|
||||
result = pd.read_sql( # type: ignore[reportUnknownMemberType]
|
||||
"SELECT symbol, time, bid FROM ticks",
|
||||
conn,
|
||||
)
|
||||
pd.testing.assert_frame_equal(
|
||||
result.reset_index(drop=True),
|
||||
pd.DataFrame({
|
||||
"symbol": ["EURUSD"],
|
||||
"time": ["2024-01-01"],
|
||||
"bid": [1.2],
|
||||
}),
|
||||
)
|
||||
|
||||
def test_default_if_exists_appends_without_dropping_rows(
|
||||
self,
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""Test the default append mode keeps prior rows."""
|
||||
output = tmp_path / "default-append.db"
|
||||
first = pd.DataFrame({"id": [1], "value": ["a"]})
|
||||
second = pd.DataFrame({"id": [2], "value": ["b"]})
|
||||
export_dataframe_to_sqlite(first, output, "items")
|
||||
export_dataframe_to_sqlite(second, output, "items")
|
||||
with sqlite3.connect(output) as conn:
|
||||
result = pd.read_sql( # type: ignore[reportUnknownMemberType]
|
||||
"SELECT id, value FROM items ORDER BY id",
|
||||
conn,
|
||||
)
|
||||
pd.testing.assert_frame_equal(
|
||||
result,
|
||||
pd.DataFrame({"id": [1, 2], "value": ["a", "b"]}),
|
||||
)
|
||||
|
||||
def test_writes_index_with_label(self, tmp_path: Path) -> None:
|
||||
"""Test optional index export with a custom label."""
|
||||
output = tmp_path / "index.db"
|
||||
frame = pd.DataFrame(
|
||||
{"value": [1.0]}, index=pd.Index(["EURUSD"], name="symbol")
|
||||
)
|
||||
export_dataframe_to_sqlite(
|
||||
frame,
|
||||
output,
|
||||
"margins",
|
||||
if_exists=IfExists.REPLACE,
|
||||
index=True,
|
||||
index_label="symbol",
|
||||
)
|
||||
with sqlite3.connect(output) as conn:
|
||||
result = pd.read_sql( # type: ignore[reportUnknownMemberType]
|
||||
"SELECT symbol, value FROM margins",
|
||||
conn,
|
||||
)
|
||||
pd.testing.assert_frame_equal(
|
||||
result,
|
||||
pd.DataFrame({"symbol": ["EURUSD"], "value": [1.0]}),
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Parse helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -166,8 +274,14 @@ class TestParseTimeframe:
|
||||
assert parse_timeframe(value) == expected
|
||||
|
||||
def test_integer_timeframe(self) -> None:
|
||||
"""Test parsing integer timeframe."""
|
||||
assert parse_timeframe("42") == 42
|
||||
"""Test parsing supported integer timeframes."""
|
||||
assert parse_timeframe("1") == 1
|
||||
assert parse_timeframe(16385) == 16385
|
||||
|
||||
def test_unsupported_integer_timeframe_raises(self) -> None:
|
||||
"""Test that unsupported integer timeframes raise ValueError."""
|
||||
with pytest.raises(ValueError, match="Invalid timeframe"):
|
||||
parse_timeframe("42")
|
||||
|
||||
def test_invalid_timeframe_raises(self) -> None:
|
||||
"""Test that invalid timeframe raises ValueError."""
|
||||
@@ -180,15 +294,21 @@ class TestParseTickFlags:
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("value", "expected"),
|
||||
[("ALL", 1), ("info", 2), ("TRADE", 4)],
|
||||
[("ALL", -1), ("info", 1), ("TRADE", 2), ("COPY_TICKS_ALL", -1)],
|
||||
)
|
||||
def test_named_flag(self, value: str, expected: int) -> None:
|
||||
"""Test parsing named tick flags."""
|
||||
assert parse_tick_flags(value) == expected
|
||||
|
||||
def test_integer_flag(self) -> None:
|
||||
"""Test parsing integer tick flag."""
|
||||
assert parse_tick_flags("7") == 7
|
||||
"""Test parsing supported integer tick flags."""
|
||||
assert parse_tick_flags("-1") == -1
|
||||
assert parse_tick_flags(2) == 2
|
||||
|
||||
def test_unsupported_integer_flag_raises(self) -> None:
|
||||
"""Test that unsupported integer tick flags raise ValueError."""
|
||||
with pytest.raises(ValueError, match="Invalid tick flags"):
|
||||
parse_tick_flags("7")
|
||||
|
||||
def test_invalid_flag_raises(self) -> None:
|
||||
"""Test that invalid flag raises ValueError."""
|
||||
@@ -247,8 +367,11 @@ class TestConstants:
|
||||
assert key in TIMEFRAME_MAP
|
||||
|
||||
def test_tick_flag_map_has_expected_keys(self) -> None:
|
||||
"""Test that TICK_FLAG_MAP contains standard flags."""
|
||||
assert set(TICK_FLAG_MAP) == {"ALL", "INFO", "TRADE"}
|
||||
"""Test that TICK_FLAG_MAP contains standard flags with MT5 values."""
|
||||
assert {"ALL", "INFO", "TRADE"} <= set(TICK_FLAG_MAP)
|
||||
assert TICK_FLAG_MAP["ALL"] == -1
|
||||
assert TICK_FLAG_MAP["INFO"] == 1
|
||||
assert TICK_FLAG_MAP["TRADE"] == 2
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("dataset", "expected"),
|
||||
@@ -295,26 +418,48 @@ class TestTimeframeType:
|
||||
"""Test converting a string to timeframe integer."""
|
||||
assert TIMEFRAME_TYPE.convert("H1", None, None) == 16385
|
||||
|
||||
def test_convert_int_passthrough(self) -> None:
|
||||
"""Test that integer values pass through unchanged."""
|
||||
assert TIMEFRAME_TYPE.convert(42, None, None) == 42
|
||||
def test_convert_int(self) -> None:
|
||||
"""Test converting supported integer timeframe values."""
|
||||
assert TIMEFRAME_TYPE.convert(16385, None, None) == 16385
|
||||
|
||||
def test_convert_unsupported_int(self) -> None:
|
||||
"""Test that unsupported integer values raise BadParameter."""
|
||||
with pytest.raises(Exception, match="Invalid timeframe"):
|
||||
TIMEFRAME_TYPE.convert(42, None, None)
|
||||
|
||||
def test_convert_invalid(self) -> None:
|
||||
"""Test that invalid values raise BadParameter."""
|
||||
with pytest.raises(Exception, match="Invalid timeframe"):
|
||||
TIMEFRAME_TYPE.convert("bad", None, None)
|
||||
|
||||
@pytest.mark.parametrize("value", [True, False, None, 1.5])
|
||||
def test_convert_invalid_types(self, value: object) -> None:
|
||||
"""Test that bool, float, and None values raise BadParameter."""
|
||||
with pytest.raises(Exception, match="Invalid timeframe"):
|
||||
TIMEFRAME_TYPE.convert(value, None, None)
|
||||
|
||||
|
||||
class TestTickFlagsType:
|
||||
"""Tests for _TickFlagsType."""
|
||||
|
||||
def test_convert_string(self) -> None:
|
||||
"""Test converting a string to tick flags integer."""
|
||||
assert TICK_FLAGS_TYPE.convert("ALL", None, None) == 1
|
||||
assert TICK_FLAGS_TYPE.convert("ALL", None, None) == -1
|
||||
|
||||
def test_convert_int_passthrough(self) -> None:
|
||||
"""Test that integer values pass through unchanged."""
|
||||
assert TICK_FLAGS_TYPE.convert(7, None, None) == 7
|
||||
def test_convert_int(self) -> None:
|
||||
"""Test converting supported integer tick flag values."""
|
||||
assert TICK_FLAGS_TYPE.convert(2, None, None) == 2
|
||||
|
||||
def test_convert_unsupported_int(self) -> None:
|
||||
"""Test that unsupported integer values raise BadParameter."""
|
||||
with pytest.raises(Exception, match="Invalid tick flags"):
|
||||
TICK_FLAGS_TYPE.convert(7, None, None)
|
||||
|
||||
@pytest.mark.parametrize("value", [True, False, None, 1.5])
|
||||
def test_convert_invalid_types(self, value: object) -> None:
|
||||
"""Test that bool, float, and None values raise BadParameter."""
|
||||
with pytest.raises(Exception, match="Invalid tick flags"):
|
||||
TICK_FLAGS_TYPE.convert(value, None, None)
|
||||
|
||||
def test_convert_invalid(self) -> None:
|
||||
"""Test that invalid values raise BadParameter."""
|
||||
|
||||
@@ -487,7 +487,7 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "mt5cli"
|
||||
version = "0.4.2"
|
||||
version = "0.7.2"
|
||||
source = { editable = "." }
|
||||
dependencies = [
|
||||
{ name = "click" },
|
||||
@@ -513,7 +513,7 @@ dev = [
|
||||
[package.metadata]
|
||||
requires-dist = [
|
||||
{ name = "click", specifier = ">=8.1.0" },
|
||||
{ name = "pdmt5", specifier = ">=0.2.3" },
|
||||
{ name = "pdmt5", specifier = ">=0.3.0" },
|
||||
{ name = "pyarrow", specifier = ">=19.0.0" },
|
||||
{ name = "typer", specifier = ">=0.15.0" },
|
||||
]
|
||||
@@ -684,16 +684,16 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "pdmt5"
|
||||
version = "0.2.3"
|
||||
version = "0.3.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "metatrader5", marker = "sys_platform == 'win32'" },
|
||||
{ name = "pandas" },
|
||||
{ name = "pydantic" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/02/25/52d9d954504ccdd0fe91f715ab74c424d61234b237cc4160d3ebe20070f1/pdmt5-0.2.3.tar.gz", hash = "sha256:21384f5826fb0125fee3f93c90b108340f55ab53b1c819d229ceac162289d2ec", size = 226665, upload-time = "2026-02-05T13:28:21.071Z" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/bf/cc/c8fa3a01e0e34178fec8527992f7bb8eda5881477ce23aaacaa9b2ef7bec/pdmt5-0.3.0.tar.gz", hash = "sha256:bb612d5c2695eafac9b2a7b74756e13bd383d7e5517bd90c9a2efa92492c484c", size = 215100, upload-time = "2026-06-11T13:26:46.976Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/c1/75/c5e52a9cf459b85b2dd52f83e70857571b1b45805c9fe610b3959a26ac15/pdmt5-0.2.3-py3-none-any.whl", hash = "sha256:f92246a05cfc3b7feb3ab0cc5b48768a4d84aad6b02e7a68060948f5828718a1", size = 22967, upload-time = "2026-02-05T13:28:19.523Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f2/03/b12cc4c9db983d971c9172b3765161b6d91136d0624e6718a04dd815e7a1/pdmt5-0.3.0-py3-none-any.whl", hash = "sha256:5388b406cc583202600cfe22c9d781679b1d931b1ed5a2b5dcf37c566149b49f", size = 26250, upload-time = "2026-06-11T13:26:45.689Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -836,11 +836,11 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "pygments"
|
||||
version = "2.19.2"
|
||||
version = "2.20.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/b0/77/a5b8c569bf593b0140bde72ea885a803b82086995367bf2037de0159d924/pygments-2.19.2.tar.gz", hash = "sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887", size = 4968631, upload-time = "2025-06-21T13:39:12.283Z" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", size = 4955991, upload-time = "2026-03-29T13:29:33.898Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/c7/21/705964c7812476f378728bdf590ca4b771ec72385c533964653c68e86bdc/pygments-2.19.2-py3-none-any.whl", hash = "sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b", size = 1225217, upload-time = "2025-06-21T13:39:07.939Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
||||
Reference in New Issue
Block a user