gavindiaz/mt5cli
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[codex] Add generic MT5 SDK and SQLite rate loader (#19)

Browse Source 
* Add generic MT5 SDK and SQLite rate loader

* Fix MT5 latest rates connection reuse

* Make MT5 summary export safe

* Address PR review feedback for SDK and SQLite rate loader.

Reuse parse_sqlite_timestamp for rate time parsing, document empty-table
errors, tighten tests, and align docs with require_existing=True.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Daichi Narushima
2026-06-09 11:27:29 +09:00
committed by GitHub
parent b2bb2ad0a0
commit 9957b0a1de
12 changed files with 1036 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+29 -24
View File
@@ -13,6 +13,7 @@ Built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data han
- **Comprehensive data access**: Rates, ticks, account info, symbols, orders, positions, and trading history
- **Flexible timeframes**: Named timeframes (M1, H1, D1, etc.) and numeric values
- **Connection management**: Optional credentials, server, and timeout configuration
- **SQLite rate loading**: Load mt5cli-managed rate tables/views for offline workflows
## Installation
@@ -50,30 +51,33 @@ python -m mt5cli -o account.csv account-info
## Commands
| Command | Description |
| ------------------ | ------------------------------------------------------------------------------------------------------------ |
| `rates-from` | Export rates from a start date |
| `rates-from-pos` | Export rates from a start position |
| `rates-range` | Export rates for a date range |
| `ticks-from` | Export ticks from a start date |
| `ticks-range` | Export ticks for a date range |
| `ticks-recent` | Export ticks from a recent trailing window |
| `account-info` | Export account information |
| `terminal-info` | Export terminal information |
| `version` | Export MetaTrader 5 version information |
| `last-error` | Export the last error information |
| `symbols` | Export symbol list |
| `symbol-info` | Export symbol details |
| `symbol-info-tick` | Export the last tick for a symbol |
| `minimum-margins` | Export minimum-volume buy and sell margin requirements |
| `market-book` | Export market depth (order book) |
| `orders` | Export active orders |
| `positions` | Export open positions |
| `history-orders` | Export historical orders |
| `history-deals` | Export historical deals |
| `order-check` | Check funds sufficiency for a trade request |
| `order-send` | Send a trade request to the trade server (`--yes` required) |
| `collect-history` | Bundle rates, ticks, history-orders, and history-deals for one or more symbols into a single SQLite database |
| Command | Description |
| ---------------------- | ------------------------------------------------------------------------------------------------------------ |
| `rates-from` | Export rates from a start date |
| `rates-from-pos` | Export rates from a start position |
| `latest-rates` | Export latest rates from a start position |
| `rates-range` | Export rates for a date range |
| `ticks-from` | Export ticks from a start date |
| `ticks-range` | Export ticks for a date range |
| `ticks-recent` | Export ticks from a recent trailing window |
| `account-info` | Export account information |
| `terminal-info` | Export terminal information |
| `version` | Export MetaTrader 5 version information |
| `last-error` | Export the last error information |
| `symbols` | Export symbol list |
| `symbol-info` | Export symbol details |
| `symbol-info-tick` | Export the last tick for a symbol |
| `minimum-margins` | Export minimum-volume buy and sell margin requirements |
| `market-book` | Export market depth (order book) |
| `orders` | Export active orders |
| `positions` | Export open positions |
| `history-orders` | Export historical orders |
| `history-deals` | Export historical deals |
| `recent-history-deals` | Export historical deals from a recent trailing window |
| `mt5-summary` | Export terminal/account status summary |
| `order-check` | Check funds sufficiency for a trade request |
| `order-send` | Send a trade request to the trade server (`--yes` required) |
| `collect-history` | Bundle rates, ticks, history-orders, and history-deals for one or more symbols into a single SQLite database |
Use `order-check` to validate a request payload before running `order-send --yes`.
@@ -130,6 +134,7 @@ update_history_with_config(
- **`rates` table**: normalized storage with `symbol` and `timeframe` columns.
- **Rate compatibility views**: mt5cli manages all `rate_*` views. Naming is `rate_<symbol>__<timeframe>` when a symbol has one timeframe, otherwise `rate_<symbol>__<granularity>_<timeframe>` (for example `rate_EURUSD__M1_1`). Stale `rate_*` views are dropped and recreated when rates change for offline tools such as mteor optimize.
- **Rate view resolution**: use `mt5cli.history.resolve_rate_view_name()` / `resolve_rate_view_names()` to map symbols and granularities to existing SQLite compatibility views without creating databases.
- **Rate view loading**: use `load_rate_data()` / `load_rate_data_from_connection()` to load a SQLite rate table or view into a `DatetimeIndex` DataFrame.
- **SQLite export helpers**: use `export_dataframe_to_sqlite()` for append mode, optional index export, and post-write deduplication by key columns.
- **Recent ticks and margins**: `recent_ticks()` and `minimum_margins()` SDK helpers (and matching CLI commands) cover common downstream read-only queries.
docs/api/history.md
+19
View File
@@ -164,3 +164,22 @@ Resolution rules:
- Pass `require_existing=True` to raise `ValueError` instead of returning a
best-guess name when the database or view is missing.
- Accepts either a SQLite path or an open `sqlite3.Connection`.
### Rate data loading
Use `load_rate_data()` to load a table or view from a SQLite path, or
`load_rate_data_from_connection()` when you already have a connection:
```python
from pathlib import Path
from mt5cli import load_rate_data
from mt5cli.history import resolve_rate_view_name
view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1", require_existing=True)
rates = load_rate_data(Path("history.db"), view, count=1000)
```
The loader accepts close-based OHLC rate data or tick-like bid/ask data. It
validates that `time` exists, parses timestamps with pandas, and returns a
DataFrame indexed by ascending `DatetimeIndex` named `time`.
docs/index.md
+20 -9
View File
@@ -13,6 +13,7 @@ mt5cli is a CLI application that exports MetaTrader 5 trading data to multiple f
- **Comprehensive data access**: Rates, ticks, account info, symbols, orders, positions, and trading history
- **Flexible timeframes**: Named timeframes (M1, H1, D1, etc.) and numeric values
- **Connection management**: Optional credentials, server, and timeout configuration
- **SQLite rate loading**: Load mt5cli-managed rate tables/views for offline workflows
## Installation
@@ -34,6 +35,7 @@ from mt5cli import (
copy_rates_range,
export_dataframe,
export_dataframe_to_sqlite,
load_rate_data,
minimum_margins,
recent_ticks,
)
@@ -49,7 +51,8 @@ rates = copy_rates_range(
export_dataframe(rates, Path("rates.csv"), "csv")
# Resolve SQLite rate compatibility views for downstream tools
view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1")
view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1", require_existing=True)
offline_rates = load_rate_data(Path("history.db"), view, count=1000)
# Recent tick window and minimum margin summary
ticks = recent_ticks("EURUSD", seconds=300)
@@ -59,6 +62,9 @@ margins = minimum_margins("EURUSD")
with Mt5CliClient(login=12345, password="secret", server="Broker-Demo") as client:
account = client.account_info()
positions = client.positions()
latest = client.latest_rates("EURUSD", "M1", count=100)
summary = client.mt5_summary()
summary_table = client.mt5_summary_as_df()
# Bulk SQLite collection (same behavior as the collect-history CLI command)
collect_history(
@@ -74,6 +80,8 @@ collect_history(
Timeframes, tick flags, and ISO 8601 date strings are accepted wherever noted in the SDK API.
`Mt5CliClient.mt5_summary()` returns the SDK structured form as plain nested Python values. Use `Mt5CliClient.mt5_summary_as_df()` when you need a one-row DataFrame for export. The `mt5-summary` CLI command uses this tabular form, so nested terminal/account fields are JSON-encoded strings that are safe for CSV, JSON, Parquet, and SQLite output.
## Quick Start
```bash
@@ -104,6 +112,7 @@ mt5cli --login 12345 --password mypass --server MyBroker-Demo \
| ---------------- | ---------------------------------- |
| `rates-from` | Export rates from a start date |
| `rates-from-pos` | Export rates from a start position |
| `latest-rates` | Export latest rates |
| `rates-range` | Export rates for a date range |
### Ticks
@@ -130,14 +139,16 @@ mt5cli --login 12345 --password mypass --server MyBroker-Demo \
### Trading
| Command | Description |
| ---------------- | ----------------------------------------------------------- |
| `orders` | Export active orders |
| `positions` | Export open positions |
| `history-orders` | Export historical orders |
| `history-deals` | Export historical deals |
| `order-check` | Check funds sufficiency for a trade request |
| `order-send` | Send a trade request to the trade server (`--yes` required) |
| Command | Description |
| ---------------------- | ----------------------------------------------------------- |
| `orders` | Export active orders |
| `positions` | Export open positions |
| `history-orders` | Export historical orders |
| `history-deals` | Export historical deals |
| `recent-history-deals` | Export historical deals from a trailing window |
| `mt5-summary` | Export terminal/account status summary |
| `order-check` | Check funds sufficiency for a trade request |
| `order-send` | Send a trade request to the trade server (`--yes` required) |
Use `order-check` to validate a request payload before running `order-send --yes`.
mt5cli/__init__.py
+13
View File
@@ -2,11 +2,13 @@
from importlib.metadata import version
from .history import load_rate_data, load_rate_data_from_connection
from .sdk import (
Mt5CliClient,
account_info,
build_config,
collect_history,
collect_latest_rates,
copy_rates_from,
copy_rates_from_pos,
copy_rates_range,
@@ -15,10 +17,14 @@ from .sdk import (
history_deals,
history_orders,
last_error,
latest_rates,
market_book,
minimum_margins,
mt5_summary,
mt5_summary_as_df,
orders,
positions,
recent_history_deals,
recent_ticks,
symbol_info,
symbol_info_tick,
@@ -47,6 +53,7 @@ __all__ = [
"account_info",
"build_config",
"collect_history",
"collect_latest_rates",
"copy_rates_from",
"copy_rates_from_pos",
"copy_rates_range",
@@ -58,11 +65,17 @@ __all__ = [
"history_deals",
"history_orders",
"last_error",
"latest_rates",
"load_rate_data",
"load_rate_data_from_connection",
"market_book",
"minimum_margins",
"mt5_summary",
"mt5_summary_as_df",
"mt5_version",
"orders",
"positions",
"recent_history_deals",
"recent_ticks",
"symbol_info",
"symbol_info_tick",
mt5cli/cli.py
+56
View File
@@ -222,6 +222,31 @@ def rates_from_pos(
)
@app.command()
def latest_rates(
ctx: typer.Context,
symbol: Annotated[str, typer.Option(help="Symbol name.")],
timeframe: Annotated[
int,
typer.Option(
click_type=TIMEFRAME_TYPE,
help="Timeframe.",
),
],
count: Annotated[int, typer.Option(help="Number of records.")],
start_pos: Annotated[
int,
typer.Option(help="Start position (0 = current bar)."),
] = 0,
) -> None:
"""Export latest rates from a start position."""
client = _sdk_client(ctx)
_execute_export(
ctx,
lambda: client.latest_rates(symbol, timeframe, count, start_pos=start_pos),
)
@app.command()
def rates_range(
ctx: typer.Context,
@@ -475,6 +500,37 @@ def history_deals(
)
@app.command()
def recent_history_deals(
ctx: typer.Context,
hours: Annotated[float, typer.Option(help="Lookback window in hours.")],
date_to: Annotated[
datetime | None,
typer.Option(click_type=DATETIME_TYPE, help="Window end date."),
] = None,
group: Annotated[str | None, typer.Option(help="Group filter.")] = None,
symbol: Annotated[str | None, typer.Option(help="Symbol filter.")] = None,
) -> None:
"""Export historical deals from a recent trailing window."""
client = _sdk_client(ctx)
_execute_export(
ctx,
lambda: client.recent_history_deals(
hours,
date_to=date_to,
group=group,
symbol=symbol,
),
)
@app.command()
def mt5_summary(ctx: typer.Context) -> None:
"""Export a compact terminal/account status summary."""
client = _sdk_client(ctx)
_execute_export(ctx, client.mt5_summary_as_df)
@app.command()
def version(ctx: typer.Context) -> None:
"""Export MetaTrader5 version information."""
mt5cli/history.py
+138 -2
View File
@@ -6,7 +6,7 @@ import logging
import sqlite3
from datetime import UTC, datetime
from pathlib import Path
from typing import TYPE_CHECKING, Literal
from typing import TYPE_CHECKING, Literal, cast
import pandas as pd
@@ -126,6 +126,14 @@ def build_rate_view_name(
SqliteConnOrPath = sqlite3.Connection | Path | str
def _require_non_empty_identifier(identifier: str, kind: str) -> str:
value = identifier.strip()
if not value:
msg = f"SQLite {kind} name must not be empty."
raise ValueError(msg)
return value
def _open_history_connection(
conn_or_path: SqliteConnOrPath,
) -> tuple[sqlite3.Connection | None, bool]:
@@ -144,6 +152,133 @@ def _open_history_connection(
return conn, True
def _open_existing_sqlite_database(
conn_or_path: SqliteConnOrPath,
) -> tuple[sqlite3.Connection, bool]:
"""Open a read-only SQLite database or reuse an existing connection.
Returns:
Tuple of connection and whether the caller should close it.
Raises:
ValueError: If the database path does not exist or is not a file.
"""
if isinstance(conn_or_path, sqlite3.Connection):
return conn_or_path, False
path = Path(conn_or_path)
if not path.exists():
msg = f"SQLite database not found: {path}"
raise ValueError(msg)
if not path.is_file():
msg = f"SQLite database path is not a file: {path}"
raise ValueError(msg)
conn = sqlite3.connect(f"{path.resolve().as_uri()}?mode=ro", uri=True)
return conn, True
def _validate_rate_load_request(table: str, count: int | None) -> str:
table_name = _require_non_empty_identifier(table, "table or view")
if count is not None and count <= 0:
msg = "count must be positive when provided."
raise ValueError(msg)
return table_name
def _ensure_rate_columns(columns: set[str], table: str) -> None:
if not columns:
msg = f"SQLite table or view not found: {table}"
raise ValueError(msg)
if "time" not in columns:
msg = f"SQLite table or view {table!r} must include a time column."
raise ValueError(msg)
if "close" not in columns and not {"ask", "bid"}.issubset(columns):
msg = (
f"SQLite table or view {table!r} must include close, "
"or both ask and bid columns."
)
raise ValueError(msg)
def _parse_rate_time_index(frame: pd.DataFrame, table: str) -> pd.DataFrame:
parsed = frame["time"].map(parse_sqlite_timestamp)
if parsed.isna().any():
msg = f"SQLite table or view {table!r} contains unparsable time values."
raise ValueError(msg)
result = frame.drop(columns=["time"])
result.index = pd.DatetimeIndex(parsed, name="time")
return result.sort_index(kind="stable")
def load_rate_data_from_connection(
connection: sqlite3.Connection,
table: str,
count: int | None = None,
) -> pd.DataFrame:
"""Load rate-like data from a SQLite table or view.
Args:
connection: Open SQLite connection.
table: Source table or view name.
count: Optional number of most recent rows to load.
Returns:
DataFrame indexed by ascending ``time``.
Raises:
ValueError: If inputs, schema, timestamps are invalid, or the table
or view contains no rows.
"""
table_name = _validate_rate_load_request(table, count)
columns = get_table_columns(connection, table_name)
_ensure_rate_columns(columns, table_name)
quoted_table = quote_sqlite_identifier(table_name)
if count is None:
frame = cast(
"pd.DataFrame",
pd.read_sql_query( # type: ignore[reportUnknownMemberType]
f"SELECT * FROM {quoted_table} ORDER BY time ASC", # noqa: S608
connection,
),
)
else:
frame = cast(
"pd.DataFrame",
pd.read_sql_query( # type: ignore[reportUnknownMemberType]
f"SELECT * FROM {quoted_table} ORDER BY time DESC LIMIT ?", # noqa: S608
connection,
params=(count,),
),
)
if frame.empty:
msg = f"SQLite table or view {table_name!r} contains no rows."
raise ValueError(msg)
return _parse_rate_time_index(frame, table_name)
def load_rate_data(
conn_or_path: SqliteConnOrPath,
table: str,
count: int | None = None,
) -> pd.DataFrame:
"""Load rate-like data from a SQLite database path or connection.
Args:
conn_or_path: SQLite database path or open connection.
table: Source table or view name.
count: Optional number of most recent rows to load.
Returns:
DataFrame indexed by ascending ``time``.
"""
conn, should_close = _open_existing_sqlite_database(conn_or_path)
try:
return load_rate_data_from_connection(conn, table, count=count)
finally:
if should_close:
conn.close()
def _load_rates_timeframe_counts(conn: sqlite3.Connection) -> dict[str, int] | None:
"""Return distinct timeframe counts per symbol from the normalized rates table."""
columns = get_table_columns(conn, Dataset.rates.table_name)
@@ -349,7 +484,8 @@ def resolve_rate_view_names(
def get_table_columns(conn: sqlite3.Connection, table: str) -> set[str]:
"""Return existing SQLite columns for a table."""
rows = conn.execute(f"PRAGMA table_info({table})").fetchall()
quoted_table = quote_sqlite_identifier(table)
rows = conn.execute(f"PRAGMA table_info({quoted_table})").fetchall()
return {str(row[1]) for row in rows}
mt5cli/sdk.py
+233 -4
View File
@@ -2,13 +2,14 @@
from __future__ import annotations
import json
import logging
import sqlite3
from contextlib import contextmanager
from dataclasses import dataclass
from datetime import UTC, datetime, timedelta
from pathlib import Path
from typing import TYPE_CHECKING, Self, TypeVar
from typing import TYPE_CHECKING, Self, TypeVar, cast
import pandas as pd
from pdmt5 import Mt5Config, Mt5DataClient
@@ -43,6 +44,7 @@ __all__ = [
"account_info",
"build_config",
"collect_history",
"collect_latest_rates",
"copy_rates_from",
"copy_rates_from_pos",
"copy_rates_range",
@@ -51,10 +53,14 @@ __all__ = [
"history_deals",
"history_orders",
"last_error",
"latest_rates",
"market_book",
"minimum_margins",
"mt5_summary",
"mt5_summary_as_df",
"orders",
"positions",
"recent_history_deals",
"recent_ticks",
"symbol_info",
"symbol_info_tick",
@@ -78,6 +84,22 @@ def _coerce_tick_flags(flags: int | str) -> int:
return parse_tick_flags(flags)
def _plain_mt5_value(value: object) -> object:
asdict = getattr(value, "_asdict", None)
if callable(asdict):
return _plain_mt5_value(asdict())
if isinstance(value, dict):
typed_value = cast("dict[object, object]", value)
return {key: _plain_mt5_value(item) for key, item in typed_value.items()}
if isinstance(value, tuple):
typed_value = cast("tuple[object, ...]", value)
return [_plain_mt5_value(item) for item in typed_value]
if isinstance(value, list):
typed_value = cast("list[object]", value)
return [_plain_mt5_value(item) for item in typed_value]
return value
def _require_datetime(value: datetime | str) -> datetime:
if isinstance(value, datetime):
return value
@@ -90,6 +112,31 @@ def _coerce_datetime(value: datetime | str | None) -> datetime | None:
return parse_datetime(value)
def _require_positive(value: float, name: str) -> None:
if value <= 0:
msg = f"{name} must be positive."
raise ValueError(msg)
def _call_required_client_method(client: Mt5DataClient, name: str) -> object:
try:
method = getattr(client, name)
except AttributeError as exc:
msg = f"MT5 client is missing required method: {name}"
raise AttributeError(msg) from exc
if not callable(method):
msg = f"MT5 client attribute is not callable: {name}"
raise TypeError(msg)
return method()
def _mt5_summary_export_value(value: object) -> object:
plain_value = _plain_mt5_value(value)
if isinstance(plain_value, dict | list):
return json.dumps(plain_value, sort_keys=True, separators=(",", ":"))
return plain_value
def _coerce_tick_time(value: object) -> datetime:
if isinstance(value, datetime):
return value
@@ -242,6 +289,7 @@ class Mt5CliClient:
server: str | None = None,
timeout: int | None = None,
config: Mt5Config | None = None,
client: Mt5DataClient | None = None,
) -> None:
"""Initialize the SDK client.
@@ -252,6 +300,8 @@ class Mt5CliClient:
server: Trading server name.
timeout: Connection timeout in milliseconds.
config: Optional pre-built ``Mt5Config`` (overrides other args).
client: Optional already-connected ``Mt5DataClient``. Injected
clients are reused as-is and are not initialized or shut down.
"""
self._config = config or build_config(
path=path,
@@ -260,7 +310,20 @@ class Mt5CliClient:
server=server,
timeout=timeout,
)
self._client: Mt5DataClient | None = None
self._client = client
self._owns_client = client is None
@classmethod
def from_connected_client(cls, client: Mt5DataClient) -> Self:
"""Bind to an already-connected ``Mt5DataClient`` without owning it.
The returned ``Mt5CliClient`` never initializes or shuts down the
injected client, including when used as a context manager.
Returns:
Client wrapper bound to the injected connection.
"""
return cls(client=client)
@property
def config(self) -> Mt5Config:
@@ -273,6 +336,8 @@ class Mt5CliClient:
Returns:
This client instance.
"""
if self._client is not None:
return self
client = Mt5DataClient(config=self._config)
try:
client.initialize_and_login_mt5()
@@ -280,6 +345,7 @@ class Mt5CliClient:
client.shutdown()
raise
self._client = client
self._owns_client = True # only set when this method created the client
return self
def __exit__(
@@ -289,15 +355,18 @@ class Mt5CliClient:
tb: object,
) -> None:
"""Shut down the persistent MT5 connection."""
if self._client is not None:
if self._client is not None and self._owns_client:
self._client.shutdown()
self._client = None
def _fetch(self, fetch_fn: Callable[[Mt5DataClient], pd.DataFrame]) -> pd.DataFrame:
def _fetch_value(self, fetch_fn: Callable[[Mt5DataClient], T]) -> T:
if self._client is not None:
return fetch_fn(self._client)
return _run_with_client(self._config, fetch_fn)
def _fetch(self, fetch_fn: Callable[[Mt5DataClient], pd.DataFrame]) -> pd.DataFrame:
return self._fetch_value(fetch_fn)
def copy_rates_from(
self,
symbol: str,
@@ -335,6 +404,54 @@ class Mt5CliClient:
),
)
def latest_rates(
self,
symbol: str,
timeframe: int | str,
count: int,
start_pos: int = 0,
) -> pd.DataFrame:
"""Return the latest rates from a bar position."""
_require_positive(count, "count")
return self.copy_rates_from_pos(symbol, timeframe, start_pos, count)
def collect_latest_rates(
self,
symbols: Sequence[str],
timeframes: Sequence[int | str],
*,
count: int,
start_pos: int = 0,
) -> dict[tuple[str, int], pd.DataFrame]:
"""Return latest rates for each symbol/timeframe pair.
Returns:
Mapping keyed by ``(symbol, timeframe_int)``.
Raises:
ValueError: If ``count`` is not positive or inputs are empty.
"""
_require_positive(count, "count")
if not symbols:
msg = "At least one symbol is required."
raise ValueError(msg)
if not timeframes:
msg = "At least one timeframe is required."
raise ValueError(msg)
resolved_timeframes = [_coerce_timeframe(timeframe) for timeframe in timeframes]
return self._fetch_value(
lambda c: {
(symbol, timeframe): c.copy_rates_from_pos_as_df(
symbol=symbol,
timeframe=timeframe,
start_pos=start_pos,
count=count,
)
for symbol in symbols
for timeframe in resolved_timeframes
},
)
def copy_rates_range(
self,
symbol: str,
@@ -486,6 +603,24 @@ class Mt5CliClient:
),
)
def recent_history_deals(
self,
hours: float,
date_to: datetime | str | None = None,
group: str | None = None,
symbol: str | None = None,
) -> pd.DataFrame:
"""Return historical deals from a recent trailing window."""
_require_positive(hours, "hours")
end = _require_datetime(date_to) if date_to is not None else datetime.now(UTC)
start = end - timedelta(hours=hours)
return self.history_deals(
date_from=start,
date_to=end,
group=group,
symbol=symbol,
)
def version(self) -> pd.DataFrame:
"""Return MetaTrader5 version information."""
return self._fetch(lambda c: c.version_as_df())
@@ -553,6 +688,39 @@ class Mt5CliClient:
"""
return self._fetch(lambda c: _fetch_minimum_margins(c, symbol))
def mt5_summary(self) -> dict[str, object]:
"""Return a compact terminal/account status summary."""
def _summary(client: Mt5DataClient) -> dict[str, object]:
return {
"version": _plain_mt5_value(
_call_required_client_method(client, "version"),
),
"terminal_info": _plain_mt5_value(
_call_required_client_method(client, "terminal_info"),
),
"account_info": _plain_mt5_value(
_call_required_client_method(client, "account_info"),
),
"symbols_total": _plain_mt5_value(
_call_required_client_method(client, "symbols_total"),
),
}
return self._fetch_value(_summary)
def mt5_summary_as_df(self) -> pd.DataFrame:
"""Return an export-safe one-row terminal/account summary DataFrame."""
summary = self.mt5_summary()
return pd.DataFrame(
[
{
key: _mt5_summary_export_value(value)
for key, value in summary.items()
},
],
)
def _resolve_incremental_settings(
selected_datasets: set[Dataset],
@@ -873,6 +1041,40 @@ def copy_rates_from_pos(
)
def latest_rates(
symbol: str,
timeframe: int | str,
count: int,
start_pos: int = 0,
*,
config: Mt5Config | None = None,
) -> pd.DataFrame:
"""Return the latest rates from a bar position."""
return _make_client(config=config).latest_rates(
symbol,
timeframe,
count,
start_pos=start_pos,
)
def collect_latest_rates(
symbols: Sequence[str],
timeframes: Sequence[int | str],
*,
count: int,
start_pos: int = 0,
config: Mt5Config | None = None,
) -> dict[tuple[str, int], pd.DataFrame]:
"""Return latest rates for each symbol/timeframe pair."""
return _make_client(config=config).collect_latest_rates(
symbols,
timeframes,
count=count,
start_pos=start_pos,
)
def copy_rates_range(
symbol: str,
timeframe: int | str,
@@ -1024,6 +1226,23 @@ def history_deals(
)
def recent_history_deals(
hours: float,
date_to: datetime | str | None = None,
group: str | None = None,
symbol: str | None = None,
*,
config: Mt5Config | None = None,
) -> pd.DataFrame:
"""Return historical deals from a recent trailing window."""
return _make_client(config=config).recent_history_deals(
hours,
date_to=date_to,
group=group,
symbol=symbol,
)
def version(*, config: Mt5Config | None = None) -> pd.DataFrame:
"""Return MetaTrader5 version information."""
return _make_client(config=config).version()
@@ -1084,3 +1303,13 @@ def minimum_margins(
See ``Mt5CliClient.minimum_margins`` for return details.
"""
return _make_client(config=config).minimum_margins(symbol)
def mt5_summary(*, config: Mt5Config | None = None) -> dict[str, object]:
"""Return a compact terminal/account status summary."""
return _make_client(config=config).mt5_summary()
def mt5_summary_as_df(*, config: Mt5Config | None = None) -> pd.DataFrame:
"""Return an export-safe terminal/account status summary DataFrame."""
return _make_client(config=config).mt5_summary_as_df()
pyproject.toml
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "mt5cli"
version = "0.4.3"
version = "0.5.0"
description = "Command-line tool for MetaTrader 5"
authors = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
maintainers = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
tests/test_cli.py
+113
View File
@@ -93,6 +93,10 @@ def mock_client(mocker: MockerFixture) -> MagicMock:
client.market_book_get_as_df.return_value = sample_df
client.order_check_as_df.return_value = sample_df
client.order_send_as_df.return_value = sample_df
client.version.return_value = (5, 0, 1)
client.terminal_info.return_value = {"connected": True, "paths": ["terminal.exe"]}
client.account_info.return_value = {"login": 123, "limits": {"modes": ["demo"]}}
client.symbols_total.return_value = 42
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=client)
return client
@@ -223,6 +227,37 @@ class TestCommands:
count=50,
)
def test_latest_rates(
self,
tmp_path: Path,
mock_client: MagicMock,
) -> None:
"""Test latest-rates command."""
output = tmp_path / "out.csv"
result = runner.invoke(
app,
[
"-o",
str(output),
"latest-rates",
"--symbol",
"GBPUSD",
"--timeframe",
"H1",
"--count",
"50",
"--start-pos",
"2",
],
)
assert result.exit_code == 0, result.output
mock_client.copy_rates_from_pos_as_df.assert_called_once_with(
symbol="GBPUSD",
timeframe=16385,
start_pos=2,
count=50,
)
def test_rates_range(
self,
tmp_path: Path,
@@ -451,6 +486,84 @@ class TestCommands:
assert result.exit_code == 0, result.output
mock_client.history_deals_get_as_df.assert_called_once()
def test_recent_history_deals(
self,
tmp_path: Path,
mock_client: MagicMock,
) -> None:
"""Test recent-history-deals command."""
output = tmp_path / "out.csv"
result = runner.invoke(
app,
[
"-o",
str(output),
"recent-history-deals",
"--hours",
"6",
"--date-to",
"2024-01-02",
"--symbol",
"EURUSD",
],
)
assert result.exit_code == 0, result.output
mock_client.history_deals_get_as_df.assert_called_once_with(
date_from=datetime(2024, 1, 1, 18, tzinfo=UTC),
date_to=datetime(2024, 1, 2, tzinfo=UTC),
group=None,
symbol="EURUSD",
ticket=None,
position=None,
)
@pytest.mark.parametrize(
("filename", "reader"),
[
("summary.csv", "csv"),
("summary.json", "json"),
("summary.db", "sqlite3"),
("summary.parquet", "parquet"),
],
)
def test_mt5_summary_export_formats(
self,
tmp_path: Path,
mock_client: MagicMock,
filename: str,
reader: str,
) -> None:
"""Test mt5-summary writes export-safe files for supported formats."""
output = tmp_path / filename
result = runner.invoke(app, ["-o", str(output), "mt5-summary"])
assert result.exit_code == 0, result.output
assert output.exists()
mock_client.version.assert_called_once()
mock_client.terminal_info.assert_called_once()
mock_client.account_info.assert_called_once()
mock_client.symbols_total.assert_called_once()
if reader == "csv":
frame = pd.read_csv(output)
elif reader == "json":
with output.open() as f:
records = json.load(f)
frame = pd.DataFrame(records)
elif reader == "sqlite3":
with sqlite3.connect(output) as conn:
frame = pd.read_sql( # type: ignore[reportUnknownMemberType]
"SELECT * FROM data",
conn,
)
else:
frame = pd.read_parquet(output)
assert len(frame) == 1
assert frame.iloc[0].to_dict() == {
"version": "[5,0,1]",
"terminal_info": '{"connected":true,"paths":["terminal.exe"]}',
"account_info": '{"limits":{"modes":["demo"]},"login":123}',
"symbols_total": 42,
}
def test_version(
self,
tmp_path: Path,
tests/test_history.py
+143
View File
@@ -31,6 +31,8 @@ from mt5cli.history import (
get_incremental_start_datetime,
get_table_columns,
load_incremental_start_datetimes,
load_rate_data,
load_rate_data_from_connection,
parse_sqlite_timestamp,
quote_sqlite_identifier,
record_written_columns,
@@ -338,6 +340,147 @@ class TestQuoteSqliteIdentifier:
assert quoted.endswith('"')
class TestLoadRateData:
"""Tests for SQLite rate-like table and view loading."""
def test_loads_close_rates_from_path_with_count(self, tmp_path: Path) -> None:
"""Test loading the latest close-based rates in ascending time order."""
db_path = tmp_path / "rates.db"
with sqlite3.connect(db_path) as conn:
conn.execute("CREATE TABLE rates(time TEXT, close REAL)")
conn.executemany(
"INSERT INTO rates(time, close) VALUES (?, ?)",
[
("2024-01-01T00:00:00+00:00", 1.0),
("2024-01-01T00:02:00+00:00", 1.2),
("2024-01-01T00:01:00+00:00", 1.1),
],
)
frame = load_rate_data(db_path, "rates", count=2)
assert list(frame["close"]) == [1.1, 1.2]
assert isinstance(frame.index, pd.DatetimeIndex)
assert frame.index.name == "time"
assert frame.index.is_monotonic_increasing
def test_loads_ask_bid_tick_like_rates_from_connection(
self,
tmp_path: Path,
) -> None:
"""Test loading tick-like tables with bid and ask columns."""
db_path = tmp_path / "ticks.db"
with sqlite3.connect(db_path) as conn:
conn.execute("CREATE TABLE ticks(time TEXT, bid REAL, ask REAL)")
conn.execute(
"INSERT INTO ticks(time, bid, ask) VALUES (?, ?, ?)",
("2024-01-01T00:00:00+00:00", 1.0, 1.1),
)
frame = load_rate_data_from_connection(conn, "ticks")
path_frame = load_rate_data(conn, "ticks")
assert frame.iloc[0].to_dict() == {"bid": 1.0, "ask": 1.1}
assert path_frame.iloc[0].to_dict() == {"bid": 1.0, "ask": 1.1}
def test_loads_from_view(self, tmp_path: Path) -> None:
"""Test loading from a SQLite view."""
db_path = tmp_path / "view.db"
with sqlite3.connect(db_path) as conn:
conn.execute("CREATE TABLE rates(time TEXT, close REAL)")
conn.execute(
"INSERT INTO rates(time, close) VALUES (?, ?)",
("2024-01-01T00:00:00+00:00", 1.0),
)
conn.execute("CREATE VIEW rate_view AS SELECT time, close FROM rates")
frame = load_rate_data_from_connection(conn, "rate_view")
assert list(frame["close"]) == [1.0]
def test_loads_quoted_identifier(self, tmp_path: Path) -> None:
"""Test table names are quoted safely."""
db_path = tmp_path / "quoted.db"
table = 'rate "quoted"'
quoted = quote_sqlite_identifier(table)
with sqlite3.connect(db_path) as conn:
conn.execute(f"CREATE TABLE {quoted}(time TEXT, close REAL)")
conn.execute(
f"INSERT INTO {quoted}(time, close) VALUES (?, ?)", # noqa: S608
("2024-01-01T00:00:00+00:00", 1.0),
)
frame = load_rate_data_from_connection(conn, table)
assert list(frame["close"]) == [1.0]
def test_rejects_missing_database_and_non_file(self, tmp_path: Path) -> None:
"""Test path validation for SQLite database inputs."""
with pytest.raises(ValueError, match="SQLite database not found"):
load_rate_data(tmp_path / "missing.db", "rates")
with pytest.raises(ValueError, match="not a file"):
load_rate_data(tmp_path, "rates")
@pytest.mark.parametrize(
("table", "count", "match"),
[
("", None, "must not be empty"),
("rates", 0, "count must be positive"),
("rates", -1, "count must be positive"),
],
)
def test_rejects_invalid_inputs(
self,
tmp_path: Path,
table: str,
count: int | None,
match: str,
) -> None:
"""Test request validation."""
db_path = tmp_path / "invalid-inputs.db"
with sqlite3.connect(db_path) as conn:
conn.execute("CREATE TABLE rates(time TEXT, close REAL)")
with pytest.raises(ValueError, match=match):
load_rate_data_from_connection(conn, table, count=count)
@pytest.mark.parametrize(
("ddl", "match"),
[
("CREATE TABLE rates(time TEXT, close REAL)", "contains no rows"),
("CREATE TABLE rates(close REAL)", "time column"),
("CREATE TABLE rates(time TEXT, open REAL)", "close, or both ask and bid"),
],
)
def test_rejects_invalid_tables(
self,
tmp_path: Path,
ddl: str,
match: str,
) -> None:
"""Test missing table, empty table, and invalid schemas."""
db_path = tmp_path / "invalid-tables.db"
with sqlite3.connect(db_path) as conn:
conn.execute(ddl)
with pytest.raises(ValueError, match=match):
load_rate_data_from_connection(conn, "rates")
with pytest.raises(ValueError, match="not found"):
load_rate_data_from_connection(conn, "missing")
def test_rejects_invalid_timestamp(self, tmp_path: Path) -> None:
"""Test unparsable timestamps fail clearly."""
db_path = tmp_path / "invalid-time.db"
with sqlite3.connect(db_path) as conn:
conn.execute("CREATE TABLE rates(time TEXT, close REAL)")
conn.execute("INSERT INTO rates(time, close) VALUES (?, ?)", ("bad", 1.0))
with pytest.raises(ValueError, match="unparsable time"):
load_rate_data_from_connection(conn, "rates")
def test_loads_numeric_mt5_epoch_seconds(self, tmp_path: Path) -> None:
"""Test MT5-native integer timestamps are parsed as epoch seconds."""
db_path = tmp_path / "epoch-rates.db"
with sqlite3.connect(db_path) as conn:
conn.execute("CREATE TABLE rates(time INTEGER, close REAL)")
conn.execute(
"INSERT INTO rates(time, close) VALUES (?, ?)",
(1_704_067_200, 1.0),
)
frame = load_rate_data_from_connection(conn, "rates")
assert frame.index[0] == pd.Timestamp("2024-01-01", tz="UTC")
assert list(frame["close"]) == [1.0]
class TestResolveHistorySettings:
"""Tests for history dataset and timeframe resolution."""
tests/test_sdk.py
+270 -2
View File
@@ -5,8 +5,8 @@ from __future__ import annotations
import logging
import sqlite3
from datetime import UTC, datetime, timedelta
from typing import TYPE_CHECKING
from unittest.mock import MagicMock
from typing import TYPE_CHECKING, NamedTuple, cast
from unittest.mock import MagicMock, call
import pandas as pd
import pytest
@@ -15,6 +15,8 @@ from pytest_mock import MockerFixture # noqa: TC002
if TYPE_CHECKING:
from pathlib import Path
from pdmt5 import Mt5DataClient
from mt5cli import sdk
from mt5cli.history import DEFAULT_HISTORY_TIMEFRAMES
from mt5cli.sdk import (
@@ -22,6 +24,7 @@ from mt5cli.sdk import (
account_info,
build_config,
collect_history,
collect_latest_rates,
copy_rates_from,
copy_rates_from_pos,
copy_rates_range,
@@ -30,10 +33,14 @@ from mt5cli.sdk import (
history_deals,
history_orders,
last_error,
latest_rates,
market_book,
minimum_margins,
mt5_summary,
mt5_summary_as_df,
orders,
positions,
recent_history_deals,
recent_ticks,
symbol_info,
symbol_info_tick,
@@ -45,6 +52,32 @@ from mt5cli.sdk import (
)
from mt5cli.utils import Dataset
class _TerminalInfo(NamedTuple):
connected: bool
path: str
class _AccountInfo(NamedTuple):
login: int
limits: dict[str, object]
class _MissingSummaryMethodClient:
def version(self) -> tuple[int, int, int]:
return (5, 0, 1)
def terminal_info(self) -> dict[str, bool]:
return {"connected": True}
def symbols_total(self) -> int:
return 42
class _NonCallableSummaryMethodClient:
version = (5, 0, 1)
_DEALS_FIXTURE: dict[str, list[object]] = {
"ticket": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14],
"position_id": [100, 100, 100, 0, 200, 200, 300, 400, 400, 500, 500, 600, 600, 600],
@@ -232,6 +265,32 @@ class TestConnectionLifecycle:
client = Mt5CliClient()
client.__exit__(None, None, None)
def test_injected_client_is_reused_and_not_shutdown(self) -> None:
"""Test injected connected clients are not initialized or shut down."""
connected = MagicMock()
connected.account_info_as_df.return_value = pd.DataFrame({"a": [1]})
connected.terminal_info_as_df.return_value = pd.DataFrame({"b": [2]})
with Mt5CliClient.from_connected_client(connected) as client:
result = client.account_info()
assert result.to_dict("list") == {"a": [1]}
connected.initialize_and_login_mt5.assert_not_called()
connected.shutdown.assert_not_called()
connected.account_info_as_df.assert_called_once()
after_exit = client.terminal_info()
assert after_exit.to_dict("list") == {"b": [2]}
connected.terminal_info_as_df.assert_called_once()
def test_constructor_injected_client_is_reused_and_not_shutdown(self) -> None:
"""Test constructor injection has the same non-owning lifecycle."""
connected = MagicMock()
connected.terminal_info_as_df.return_value = pd.DataFrame({"b": [2]})
client = Mt5CliClient(client=connected)
with client:
result = client.terminal_info()
assert result.to_dict("list") == {"b": [2]}
connected.initialize_and_login_mt5.assert_not_called()
connected.shutdown.assert_not_called()
class TestModuleFunctions:
"""Tests for module-level SDK wrappers."""
@@ -271,6 +330,7 @@ class TestModuleFunctions:
(last_error, (), "last_error_as_df"),
(symbol_info_tick, ("EURUSD",), "symbol_info_tick_as_df"),
(market_book, ("EURUSD",), "market_book_get_as_df"),
(latest_rates, ("EURUSD", "M1", 10), "copy_rates_from_pos_as_df"),
],
)
def test_module_functions_delegate(
@@ -354,6 +414,214 @@ class TestMt5CliClient:
assert isinstance(df, pd.DataFrame)
mock_client.copy_rates_range_as_df.assert_called_once()
def test_latest_rates_delegates_to_copy_rates_from_pos(
self,
mock_client: MagicMock,
) -> None:
"""Test latest_rates is a convenience wrapper for positional rates."""
Mt5CliClient().latest_rates("EURUSD", "M1", 5, start_pos=2)
mock_client.copy_rates_from_pos_as_df.assert_called_once_with(
symbol="EURUSD",
timeframe=1,
start_pos=2,
count=5,
)
def test_latest_rates_rejects_non_positive_count(self) -> None:
"""Test latest_rates validates count."""
with pytest.raises(ValueError, match="count must be positive"):
Mt5CliClient().latest_rates("EURUSD", "M1", 0)
def test_collect_latest_rates_returns_mapping(
self,
mock_client: MagicMock,
) -> None:
"""Test multi-target latest rate collection."""
result = collect_latest_rates(["EURUSD", "GBPUSD"], ["M1", "H1"], count=3)
assert set(result) == {
("EURUSD", 1),
("EURUSD", 16385),
("GBPUSD", 1),
("GBPUSD", 16385),
}
assert mock_client.copy_rates_from_pos_as_df.call_count == 4
def test_collect_latest_rates_uses_single_transient_connection(
self,
mock_client: MagicMock,
mocker: MockerFixture,
) -> None:
"""Test module helper opens one connection for all target pairs."""
mt5_data_client = mocker.patch(
"mt5cli.sdk.Mt5DataClient",
return_value=mock_client,
)
collect_latest_rates(["EURUSD", "GBPUSD"], ["M1", "H1"], count=3)
mt5_data_client.assert_called_once()
mock_client.initialize_and_login_mt5.assert_called_once()
mock_client.shutdown.assert_called_once()
assert mock_client.copy_rates_from_pos_as_df.call_count == 4
mock_client.copy_rates_from_pos_as_df.assert_has_calls(
[
call(symbol="EURUSD", timeframe=1, start_pos=0, count=3),
call(symbol="EURUSD", timeframe=16385, start_pos=0, count=3),
call(symbol="GBPUSD", timeframe=1, start_pos=0, count=3),
call(symbol="GBPUSD", timeframe=16385, start_pos=0, count=3),
],
)
@pytest.mark.parametrize(
("symbols", "timeframes", "match"),
[
([], ["M1"], "At least one symbol"),
(["EURUSD"], [], "At least one timeframe"),
],
)
def test_collect_latest_rates_rejects_empty_inputs(
self,
symbols: list[str],
timeframes: list[str],
match: str,
) -> None:
"""Test multi-target latest rate input validation."""
with pytest.raises(ValueError, match=match):
Mt5CliClient().collect_latest_rates(symbols, timeframes, count=1)
def test_recent_history_deals_uses_trailing_window(
self,
mock_client: MagicMock,
) -> None:
"""Test recent_history_deals calculates date_from from hours."""
result = recent_history_deals(
6,
date_to="2024-01-02T00:00:00+00:00",
group="*",
symbol="EURUSD",
)
assert isinstance(result, pd.DataFrame)
mock_client.history_deals_get_as_df.assert_called_once_with(
date_from=datetime(2024, 1, 1, 18, tzinfo=UTC),
date_to=datetime(2024, 1, 2, tzinfo=UTC),
group="*",
symbol="EURUSD",
ticket=None,
position=None,
)
def test_recent_history_deals_defaults_date_to_now(
self,
mock_client: MagicMock,
) -> None:
"""Test recent_history_deals uses current UTC time when date_to is omitted."""
before = datetime.now(UTC)
recent_history_deals(1.0)
after = datetime.now(UTC)
call_kwargs = mock_client.history_deals_get_as_df.call_args.kwargs
assert before <= call_kwargs["date_to"] <= after
assert call_kwargs["date_from"] == call_kwargs["date_to"] - timedelta(hours=1)
def test_recent_history_deals_rejects_non_positive_hours(self) -> None:
"""Test recent_history_deals validates hours."""
with pytest.raises(ValueError, match="hours must be positive"):
Mt5CliClient().recent_history_deals(0)
def test_mt5_summary_returns_status_mapping(
self,
mock_client: MagicMock,
) -> None:
"""Test mt5_summary calls raw terminal/account status methods."""
mock_client.version.return_value = (5, 0, 1)
mock_client.terminal_info.return_value = {"connected": True}
mock_client.account_info.return_value = {"login": 123}
mock_client.symbols_total.return_value = 42
assert mt5_summary() == {
"version": [5, 0, 1],
"terminal_info": {"connected": True},
"account_info": {"login": 123},
"symbols_total": 42,
}
def test_mt5_summary_normalizes_namedtuple_values(
self,
mock_client: MagicMock,
) -> None:
"""Test mt5_summary returns structured plain Python values."""
mock_client.version.return_value = (5, 0, 1)
mock_client.terminal_info.return_value = _TerminalInfo(
connected=True,
path="terminal.exe",
)
mock_client.account_info.return_value = _AccountInfo(
login=123,
limits={"modes": ("netting", "hedging"), "servers": ["demo"]},
)
mock_client.symbols_total.return_value = 42
assert mt5_summary() == {
"version": [5, 0, 1],
"terminal_info": {"connected": True, "path": "terminal.exe"},
"account_info": {
"login": 123,
"limits": {"modes": ["netting", "hedging"], "servers": ["demo"]},
},
"symbols_total": 42,
}
def test_mt5_summary_as_df_stringifies_nested_values(
self,
mock_client: MagicMock,
) -> None:
"""Test mt5_summary_as_df returns export-safe tabular values."""
mock_client.version.return_value = (5, 0, 1)
mock_client.terminal_info.return_value = _TerminalInfo(
connected=True,
path="terminal.exe",
)
mock_client.account_info.return_value = _AccountInfo(
login=123,
limits={"modes": ("netting", "hedging"), "servers": ["demo"]},
)
mock_client.symbols_total.return_value = 42
result = mt5_summary_as_df()
assert len(result) == 1
assert result.iloc[0].to_dict() == {
"version": "[5,0,1]",
"terminal_info": '{"connected":true,"path":"terminal.exe"}',
"account_info": (
'{"limits":{"modes":["netting","hedging"],'
'"servers":["demo"]},"login":123}'
),
"symbols_total": 42,
}
def test_mt5_summary_missing_method_raises_clear_error(self) -> None:
"""Test mt5_summary fails clearly when a required method is missing."""
client = Mt5CliClient(
client=cast("Mt5DataClient", _MissingSummaryMethodClient()),
)
with pytest.raises(
AttributeError,
match="MT5 client is missing required method: account_info",
):
client.mt5_summary()
def test_mt5_summary_non_callable_method_raises_clear_error(self) -> None:
"""Test mt5_summary fails clearly when a required method is not callable."""
client = Mt5CliClient(
client=cast("Mt5DataClient", _NonCallableSummaryMethodClient()),
)
with pytest.raises(
TypeError,
match="MT5 client attribute is not callable: version",
):
client.mt5_summary()
class TestCollectHistory:
"""Tests for collect_history SDK function."""
uv.lock
Generated
+1 -1
View File
@@ -487,7 +487,7 @@ wheels = [
[[package]]
name = "mt5cli"
version = "0.4.3"
version = "0.5.0"
source = { editable = "." }
dependencies = [
{ name = "click" },