Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 9957b0a1de | |||
| b2bb2ad0a0 |
@@ -13,6 +13,7 @@ Built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data han
|
|||||||
- **Comprehensive data access**: Rates, ticks, account info, symbols, orders, positions, and trading history
|
- **Comprehensive data access**: Rates, ticks, account info, symbols, orders, positions, and trading history
|
||||||
- **Flexible timeframes**: Named timeframes (M1, H1, D1, etc.) and numeric values
|
- **Flexible timeframes**: Named timeframes (M1, H1, D1, etc.) and numeric values
|
||||||
- **Connection management**: Optional credentials, server, and timeout configuration
|
- **Connection management**: Optional credentials, server, and timeout configuration
|
||||||
|
- **SQLite rate loading**: Load mt5cli-managed rate tables/views for offline workflows
|
||||||
|
|
||||||
## Installation
|
## Installation
|
||||||
|
|
||||||
@@ -50,28 +51,33 @@ python -m mt5cli -o account.csv account-info
|
|||||||
|
|
||||||
## Commands
|
## Commands
|
||||||
|
|
||||||
| Command | Description |
|
| Command | Description |
|
||||||
| ------------------ | ------------------------------------------------------------------------------------------------------------ |
|
| ---------------------- | ------------------------------------------------------------------------------------------------------------ |
|
||||||
| `rates-from` | Export rates from a start date |
|
| `rates-from` | Export rates from a start date |
|
||||||
| `rates-from-pos` | Export rates from a start position |
|
| `rates-from-pos` | Export rates from a start position |
|
||||||
| `rates-range` | Export rates for a date range |
|
| `latest-rates` | Export latest rates from a start position |
|
||||||
| `ticks-from` | Export ticks from a start date |
|
| `rates-range` | Export rates for a date range |
|
||||||
| `ticks-range` | Export ticks for a date range |
|
| `ticks-from` | Export ticks from a start date |
|
||||||
| `account-info` | Export account information |
|
| `ticks-range` | Export ticks for a date range |
|
||||||
| `terminal-info` | Export terminal information |
|
| `ticks-recent` | Export ticks from a recent trailing window |
|
||||||
| `version` | Export MetaTrader 5 version information |
|
| `account-info` | Export account information |
|
||||||
| `last-error` | Export the last error information |
|
| `terminal-info` | Export terminal information |
|
||||||
| `symbols` | Export symbol list |
|
| `version` | Export MetaTrader 5 version information |
|
||||||
| `symbol-info` | Export symbol details |
|
| `last-error` | Export the last error information |
|
||||||
| `symbol-info-tick` | Export the last tick for a symbol |
|
| `symbols` | Export symbol list |
|
||||||
| `market-book` | Export market depth (order book) |
|
| `symbol-info` | Export symbol details |
|
||||||
| `orders` | Export active orders |
|
| `symbol-info-tick` | Export the last tick for a symbol |
|
||||||
| `positions` | Export open positions |
|
| `minimum-margins` | Export minimum-volume buy and sell margin requirements |
|
||||||
| `history-orders` | Export historical orders |
|
| `market-book` | Export market depth (order book) |
|
||||||
| `history-deals` | Export historical deals |
|
| `orders` | Export active orders |
|
||||||
| `order-check` | Check funds sufficiency for a trade request |
|
| `positions` | Export open positions |
|
||||||
| `order-send` | Send a trade request to the trade server (`--yes` required) |
|
| `history-orders` | Export historical orders |
|
||||||
| `collect-history` | Bundle rates, ticks, history-orders, and history-deals for one or more symbols into a single SQLite database |
|
| `history-deals` | Export historical deals |
|
||||||
|
| `recent-history-deals` | Export historical deals from a recent trailing window |
|
||||||
|
| `mt5-summary` | Export terminal/account status summary |
|
||||||
|
| `order-check` | Check funds sufficiency for a trade request |
|
||||||
|
| `order-send` | Send a trade request to the trade server (`--yes` required) |
|
||||||
|
| `collect-history` | Bundle rates, ticks, history-orders, and history-deals for one or more symbols into a single SQLite database |
|
||||||
|
|
||||||
Use `order-check` to validate a request payload before running `order-send --yes`.
|
Use `order-check` to validate a request payload before running `order-send --yes`.
|
||||||
|
|
||||||
@@ -127,6 +133,10 @@ update_history_with_config(
|
|||||||
- **`update_history`**: incremental append based on existing SQLite `MAX(time)` per symbol (and timeframe for rates); account-level deals use a separate cursor when `include_account_events=True`.
|
- **`update_history`**: incremental append based on existing SQLite `MAX(time)` per symbol (and timeframe for rates); account-level deals use a separate cursor when `include_account_events=True`.
|
||||||
- **`rates` table**: normalized storage with `symbol` and `timeframe` columns.
|
- **`rates` table**: normalized storage with `symbol` and `timeframe` columns.
|
||||||
- **Rate compatibility views**: mt5cli manages all `rate_*` views. Naming is `rate_<symbol>__<timeframe>` when a symbol has one timeframe, otherwise `rate_<symbol>__<granularity>_<timeframe>` (for example `rate_EURUSD__M1_1`). Stale `rate_*` views are dropped and recreated when rates change for offline tools such as mteor optimize.
|
- **Rate compatibility views**: mt5cli manages all `rate_*` views. Naming is `rate_<symbol>__<timeframe>` when a symbol has one timeframe, otherwise `rate_<symbol>__<granularity>_<timeframe>` (for example `rate_EURUSD__M1_1`). Stale `rate_*` views are dropped and recreated when rates change for offline tools such as mteor optimize.
|
||||||
|
- **Rate view resolution**: use `mt5cli.history.resolve_rate_view_name()` / `resolve_rate_view_names()` to map symbols and granularities to existing SQLite compatibility views without creating databases.
|
||||||
|
- **Rate view loading**: use `load_rate_data()` / `load_rate_data_from_connection()` to load a SQLite rate table or view into a `DatetimeIndex` DataFrame.
|
||||||
|
- **SQLite export helpers**: use `export_dataframe_to_sqlite()` for append mode, optional index export, and post-write deduplication by key columns.
|
||||||
|
- **Recent ticks and margins**: `recent_ticks()` and `minimum_margins()` SDK helpers (and matching CLI commands) cover common downstream read-only queries.
|
||||||
|
|
||||||
## Requirements
|
## Requirements
|
||||||
|
|
||||||
|
|||||||
@@ -129,3 +129,57 @@ when required columns are missing.
|
|||||||
The `update_history` SDK path uses the same base tables and optional
|
The `update_history` SDK path uses the same base tables and optional
|
||||||
`cash_events` / `positions_reconstructed` views. It additionally maintains
|
`cash_events` / `positions_reconstructed` views. It additionally maintains
|
||||||
`rate_<symbol>__<timeframe>` compatibility views when `create_rate_views=True`.
|
`rate_<symbol>__<timeframe>` compatibility views when `create_rate_views=True`.
|
||||||
|
|
||||||
|
### Rate view resolution
|
||||||
|
|
||||||
|
Downstream tools can resolve mt5cli-managed compatibility view names from an
|
||||||
|
existing SQLite history database without creating files or guessing legacy
|
||||||
|
naming schemes:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from mt5cli.history import resolve_rate_view_name, resolve_rate_view_names
|
||||||
|
|
||||||
|
# Single symbol and granularity
|
||||||
|
view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1")
|
||||||
|
|
||||||
|
# Batch resolution in row-major order
|
||||||
|
views = resolve_rate_view_names(
|
||||||
|
Path("history.db"),
|
||||||
|
["EURUSD", "GBPUSD"],
|
||||||
|
["M1", "H1"],
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
Resolution rules:
|
||||||
|
|
||||||
|
- Returns `rate_<symbol>__<timeframe>` when a symbol stores one timeframe.
|
||||||
|
- Returns `rate_<symbol>__<granularity>_<timeframe>` when multiple timeframes
|
||||||
|
are stored for the same symbol.
|
||||||
|
- When multiple naming candidates apply, prefers an existing managed
|
||||||
|
`rate_*__*` view from the candidate list.
|
||||||
|
- Falls back to single-timeframe naming when the database path is missing or
|
||||||
|
`rates` metadata is unavailable.
|
||||||
|
- Pass `require_existing=True` to raise `ValueError` instead of returning a
|
||||||
|
best-guess name when the database or view is missing.
|
||||||
|
- Accepts either a SQLite path or an open `sqlite3.Connection`.
|
||||||
|
|
||||||
|
### Rate data loading
|
||||||
|
|
||||||
|
Use `load_rate_data()` to load a table or view from a SQLite path, or
|
||||||
|
`load_rate_data_from_connection()` when you already have a connection:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from mt5cli import load_rate_data
|
||||||
|
from mt5cli.history import resolve_rate_view_name
|
||||||
|
|
||||||
|
view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1", require_existing=True)
|
||||||
|
rates = load_rate_data(Path("history.db"), view, count=1000)
|
||||||
|
```
|
||||||
|
|
||||||
|
The loader accepts close-based OHLC rate data or tick-like bid/ask data. It
|
||||||
|
validates that `time` exists, parses timestamps with pandas, and returns a
|
||||||
|
DataFrame indexed by ascending `DatetimeIndex` named `time`.
|
||||||
|
|||||||
@@ -65,12 +65,18 @@ from datetime import UTC, datetime
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from mt5cli import (
|
from mt5cli import (
|
||||||
|
Dataset,
|
||||||
|
IfExists,
|
||||||
Mt5CliClient,
|
Mt5CliClient,
|
||||||
collect_history,
|
collect_history,
|
||||||
copy_rates_range,
|
copy_rates_range,
|
||||||
detect_format,
|
detect_format,
|
||||||
export_dataframe,
|
export_dataframe,
|
||||||
|
export_dataframe_to_sqlite,
|
||||||
|
minimum_margins,
|
||||||
|
recent_ticks,
|
||||||
)
|
)
|
||||||
|
from mt5cli.history import resolve_rate_view_name
|
||||||
|
|
||||||
# Fetch rates programmatically
|
# Fetch rates programmatically
|
||||||
rates = copy_rates_range(
|
rates = copy_rates_range(
|
||||||
@@ -86,6 +92,20 @@ fmt = detect_format(Path("output.parquet")) # Returns "parquet"
|
|||||||
# Export a DataFrame
|
# Export a DataFrame
|
||||||
export_dataframe(rates, Path("output.csv"), "csv")
|
export_dataframe(rates, Path("output.csv"), "csv")
|
||||||
|
|
||||||
|
# Append to SQLite with deduplication
|
||||||
|
export_dataframe_to_sqlite(
|
||||||
|
rates,
|
||||||
|
Path("history.db"),
|
||||||
|
"rates",
|
||||||
|
if_exists=IfExists.APPEND,
|
||||||
|
deduplicate_on=("symbol", "timeframe", "time"),
|
||||||
|
)
|
||||||
|
|
||||||
|
# Resolve rate compatibility views and fetch recent ticks
|
||||||
|
view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1")
|
||||||
|
ticks = recent_ticks("EURUSD", seconds=300)
|
||||||
|
margins = minimum_margins("EURUSD")
|
||||||
|
|
||||||
# Collect history into SQLite
|
# Collect history into SQLite
|
||||||
collect_history(
|
collect_history(
|
||||||
Path("history.db"),
|
Path("history.db"),
|
||||||
|
|||||||
+43
-14
@@ -13,6 +13,7 @@ mt5cli is a CLI application that exports MetaTrader 5 trading data to multiple f
|
|||||||
- **Comprehensive data access**: Rates, ticks, account info, symbols, orders, positions, and trading history
|
- **Comprehensive data access**: Rates, ticks, account info, symbols, orders, positions, and trading history
|
||||||
- **Flexible timeframes**: Named timeframes (M1, H1, D1, etc.) and numeric values
|
- **Flexible timeframes**: Named timeframes (M1, H1, D1, etc.) and numeric values
|
||||||
- **Connection management**: Optional credentials, server, and timeout configuration
|
- **Connection management**: Optional credentials, server, and timeout configuration
|
||||||
|
- **SQLite rate loading**: Load mt5cli-managed rate tables/views for offline workflows
|
||||||
|
|
||||||
## Installation
|
## Installation
|
||||||
|
|
||||||
@@ -22,13 +23,23 @@ pip install mt5cli
|
|||||||
|
|
||||||
## Programmatic usage / SDK usage
|
## Programmatic usage / SDK usage
|
||||||
|
|
||||||
mt5cli can be used as a small Python SDK for read-only MetaTrader 5 data collection. SDK functions return pandas DataFrames without writing files. Use `export_dataframe` when you need to persist results.
|
mt5cli can be used as a small Python SDK for read-only MetaTrader 5 data collection. SDK functions return pandas DataFrames without writing files. Use `export_dataframe` or `export_dataframe_to_sqlite` when you need to persist results.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from mt5cli import Mt5CliClient, collect_history, copy_rates_range, export_dataframe
|
from mt5cli import (
|
||||||
|
Mt5CliClient,
|
||||||
|
collect_history,
|
||||||
|
copy_rates_range,
|
||||||
|
export_dataframe,
|
||||||
|
export_dataframe_to_sqlite,
|
||||||
|
load_rate_data,
|
||||||
|
minimum_margins,
|
||||||
|
recent_ticks,
|
||||||
|
)
|
||||||
|
from mt5cli.history import resolve_rate_view_name
|
||||||
|
|
||||||
# One-off fetch with module-level helpers
|
# One-off fetch with module-level helpers
|
||||||
rates = copy_rates_range(
|
rates = copy_rates_range(
|
||||||
@@ -39,10 +50,21 @@ rates = copy_rates_range(
|
|||||||
)
|
)
|
||||||
export_dataframe(rates, Path("rates.csv"), "csv")
|
export_dataframe(rates, Path("rates.csv"), "csv")
|
||||||
|
|
||||||
|
# Resolve SQLite rate compatibility views for downstream tools
|
||||||
|
view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1", require_existing=True)
|
||||||
|
offline_rates = load_rate_data(Path("history.db"), view, count=1000)
|
||||||
|
|
||||||
|
# Recent tick window and minimum margin summary
|
||||||
|
ticks = recent_ticks("EURUSD", seconds=300)
|
||||||
|
margins = minimum_margins("EURUSD")
|
||||||
|
|
||||||
# Reuse one MT5 connection for multiple calls
|
# Reuse one MT5 connection for multiple calls
|
||||||
with Mt5CliClient(login=12345, password="secret", server="Broker-Demo") as client:
|
with Mt5CliClient(login=12345, password="secret", server="Broker-Demo") as client:
|
||||||
account = client.account_info()
|
account = client.account_info()
|
||||||
positions = client.positions()
|
positions = client.positions()
|
||||||
|
latest = client.latest_rates("EURUSD", "M1", count=100)
|
||||||
|
summary = client.mt5_summary()
|
||||||
|
summary_table = client.mt5_summary_as_df()
|
||||||
|
|
||||||
# Bulk SQLite collection (same behavior as the collect-history CLI command)
|
# Bulk SQLite collection (same behavior as the collect-history CLI command)
|
||||||
collect_history(
|
collect_history(
|
||||||
@@ -58,6 +80,8 @@ collect_history(
|
|||||||
|
|
||||||
Timeframes, tick flags, and ISO 8601 date strings are accepted wherever noted in the SDK API.
|
Timeframes, tick flags, and ISO 8601 date strings are accepted wherever noted in the SDK API.
|
||||||
|
|
||||||
|
`Mt5CliClient.mt5_summary()` returns the SDK structured form as plain nested Python values. Use `Mt5CliClient.mt5_summary_as_df()` when you need a one-row DataFrame for export. The `mt5-summary` CLI command uses this tabular form, so nested terminal/account fields are JSON-encoded strings that are safe for CSV, JSON, Parquet, and SQLite output.
|
||||||
|
|
||||||
## Quick Start
|
## Quick Start
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -88,14 +112,16 @@ mt5cli --login 12345 --password mypass --server MyBroker-Demo \
|
|||||||
| ---------------- | ---------------------------------- |
|
| ---------------- | ---------------------------------- |
|
||||||
| `rates-from` | Export rates from a start date |
|
| `rates-from` | Export rates from a start date |
|
||||||
| `rates-from-pos` | Export rates from a start position |
|
| `rates-from-pos` | Export rates from a start position |
|
||||||
|
| `latest-rates` | Export latest rates |
|
||||||
| `rates-range` | Export rates for a date range |
|
| `rates-range` | Export rates for a date range |
|
||||||
|
|
||||||
### Ticks
|
### Ticks
|
||||||
|
|
||||||
| Command | Description |
|
| Command | Description |
|
||||||
| ------------- | ------------------------------ |
|
| -------------- | ----------------------------------- |
|
||||||
| `ticks-from` | Export ticks from a start date |
|
| `ticks-from` | Export ticks from a start date |
|
||||||
| `ticks-range` | Export ticks for a date range |
|
| `ticks-range` | Export ticks for a date range |
|
||||||
|
| `ticks-recent` | Export ticks from a trailing window |
|
||||||
|
|
||||||
### Information
|
### Information
|
||||||
|
|
||||||
@@ -108,18 +134,21 @@ mt5cli --login 12345 --password mypass --server MyBroker-Demo \
|
|||||||
| `symbols` | Export symbol list |
|
| `symbols` | Export symbol list |
|
||||||
| `symbol-info` | Export symbol details |
|
| `symbol-info` | Export symbol details |
|
||||||
| `symbol-info-tick` | Export the last tick for a symbol |
|
| `symbol-info-tick` | Export the last tick for a symbol |
|
||||||
|
| `minimum-margins` | Export minimum-volume margin summary |
|
||||||
| `market-book` | Export market depth (order book) |
|
| `market-book` | Export market depth (order book) |
|
||||||
|
|
||||||
### Trading
|
### Trading
|
||||||
|
|
||||||
| Command | Description |
|
| Command | Description |
|
||||||
| ---------------- | ----------------------------------------------------------- |
|
| ---------------------- | ----------------------------------------------------------- |
|
||||||
| `orders` | Export active orders |
|
| `orders` | Export active orders |
|
||||||
| `positions` | Export open positions |
|
| `positions` | Export open positions |
|
||||||
| `history-orders` | Export historical orders |
|
| `history-orders` | Export historical orders |
|
||||||
| `history-deals` | Export historical deals |
|
| `history-deals` | Export historical deals |
|
||||||
| `order-check` | Check funds sufficiency for a trade request |
|
| `recent-history-deals` | Export historical deals from a trailing window |
|
||||||
| `order-send` | Send a trade request to the trade server (`--yes` required) |
|
| `mt5-summary` | Export terminal/account status summary |
|
||||||
|
| `order-check` | Check funds sufficiency for a trade request |
|
||||||
|
| `order-send` | Send a trade request to the trade server (`--yes` required) |
|
||||||
|
|
||||||
Use `order-check` to validate a request payload before running `order-send --yes`.
|
Use `order-check` to validate a request payload before running `order-send --yes`.
|
||||||
|
|
||||||
|
|||||||
+25
-1
@@ -2,11 +2,13 @@
|
|||||||
|
|
||||||
from importlib.metadata import version
|
from importlib.metadata import version
|
||||||
|
|
||||||
|
from .history import load_rate_data, load_rate_data_from_connection
|
||||||
from .sdk import (
|
from .sdk import (
|
||||||
Mt5CliClient,
|
Mt5CliClient,
|
||||||
account_info,
|
account_info,
|
||||||
build_config,
|
build_config,
|
||||||
collect_history,
|
collect_history,
|
||||||
|
collect_latest_rates,
|
||||||
copy_rates_from,
|
copy_rates_from,
|
||||||
copy_rates_from_pos,
|
copy_rates_from_pos,
|
||||||
copy_rates_range,
|
copy_rates_range,
|
||||||
@@ -15,9 +17,15 @@ from .sdk import (
|
|||||||
history_deals,
|
history_deals,
|
||||||
history_orders,
|
history_orders,
|
||||||
last_error,
|
last_error,
|
||||||
|
latest_rates,
|
||||||
market_book,
|
market_book,
|
||||||
|
minimum_margins,
|
||||||
|
mt5_summary,
|
||||||
|
mt5_summary_as_df,
|
||||||
orders,
|
orders,
|
||||||
positions,
|
positions,
|
||||||
|
recent_history_deals,
|
||||||
|
recent_ticks,
|
||||||
symbol_info,
|
symbol_info,
|
||||||
symbol_info_tick,
|
symbol_info_tick,
|
||||||
symbols,
|
symbols,
|
||||||
@@ -28,7 +36,13 @@ from .sdk import (
|
|||||||
from .sdk import (
|
from .sdk import (
|
||||||
version as mt5_version,
|
version as mt5_version,
|
||||||
)
|
)
|
||||||
from .utils import Dataset, IfExists, detect_format, export_dataframe
|
from .utils import (
|
||||||
|
Dataset,
|
||||||
|
IfExists,
|
||||||
|
detect_format,
|
||||||
|
export_dataframe,
|
||||||
|
export_dataframe_to_sqlite,
|
||||||
|
)
|
||||||
|
|
||||||
__version__ = version(__package__) if __package__ else None
|
__version__ = version(__package__) if __package__ else None
|
||||||
|
|
||||||
@@ -39,6 +53,7 @@ __all__ = [
|
|||||||
"account_info",
|
"account_info",
|
||||||
"build_config",
|
"build_config",
|
||||||
"collect_history",
|
"collect_history",
|
||||||
|
"collect_latest_rates",
|
||||||
"copy_rates_from",
|
"copy_rates_from",
|
||||||
"copy_rates_from_pos",
|
"copy_rates_from_pos",
|
||||||
"copy_rates_range",
|
"copy_rates_range",
|
||||||
@@ -46,13 +61,22 @@ __all__ = [
|
|||||||
"copy_ticks_range",
|
"copy_ticks_range",
|
||||||
"detect_format",
|
"detect_format",
|
||||||
"export_dataframe",
|
"export_dataframe",
|
||||||
|
"export_dataframe_to_sqlite",
|
||||||
"history_deals",
|
"history_deals",
|
||||||
"history_orders",
|
"history_orders",
|
||||||
"last_error",
|
"last_error",
|
||||||
|
"latest_rates",
|
||||||
|
"load_rate_data",
|
||||||
|
"load_rate_data_from_connection",
|
||||||
"market_book",
|
"market_book",
|
||||||
|
"minimum_margins",
|
||||||
|
"mt5_summary",
|
||||||
|
"mt5_summary_as_df",
|
||||||
"mt5_version",
|
"mt5_version",
|
||||||
"orders",
|
"orders",
|
||||||
"positions",
|
"positions",
|
||||||
|
"recent_history_deals",
|
||||||
|
"recent_ticks",
|
||||||
"symbol_info",
|
"symbol_info",
|
||||||
"symbol_info_tick",
|
"symbol_info_tick",
|
||||||
"symbols",
|
"symbols",
|
||||||
|
|||||||
+104
@@ -222,6 +222,31 @@ def rates_from_pos(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@app.command()
|
||||||
|
def latest_rates(
|
||||||
|
ctx: typer.Context,
|
||||||
|
symbol: Annotated[str, typer.Option(help="Symbol name.")],
|
||||||
|
timeframe: Annotated[
|
||||||
|
int,
|
||||||
|
typer.Option(
|
||||||
|
click_type=TIMEFRAME_TYPE,
|
||||||
|
help="Timeframe.",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
count: Annotated[int, typer.Option(help="Number of records.")],
|
||||||
|
start_pos: Annotated[
|
||||||
|
int,
|
||||||
|
typer.Option(help="Start position (0 = current bar)."),
|
||||||
|
] = 0,
|
||||||
|
) -> None:
|
||||||
|
"""Export latest rates from a start position."""
|
||||||
|
client = _sdk_client(ctx)
|
||||||
|
_execute_export(
|
||||||
|
ctx,
|
||||||
|
lambda: client.latest_rates(symbol, timeframe, count, start_pos=start_pos),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
@app.command()
|
@app.command()
|
||||||
def rates_range(
|
def rates_range(
|
||||||
ctx: typer.Context,
|
ctx: typer.Context,
|
||||||
@@ -300,6 +325,44 @@ def ticks_range(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@app.command()
|
||||||
|
def ticks_recent(
|
||||||
|
ctx: typer.Context,
|
||||||
|
symbol: Annotated[str, typer.Option(help="Symbol name.")],
|
||||||
|
seconds: Annotated[
|
||||||
|
float,
|
||||||
|
typer.Option(help="Lookback window in seconds."),
|
||||||
|
],
|
||||||
|
date_to: Annotated[
|
||||||
|
datetime | None,
|
||||||
|
typer.Option(click_type=DATETIME_TYPE, help="Window end date."),
|
||||||
|
] = None,
|
||||||
|
count: Annotated[
|
||||||
|
int,
|
||||||
|
typer.Option(help="Maximum number of ticks to return."),
|
||||||
|
] = 10000,
|
||||||
|
flags: Annotated[
|
||||||
|
int,
|
||||||
|
typer.Option(
|
||||||
|
click_type=TICK_FLAGS_TYPE,
|
||||||
|
help="Tick flags (ALL, INFO, TRADE, or integer).",
|
||||||
|
),
|
||||||
|
] = 1,
|
||||||
|
) -> None:
|
||||||
|
"""Export ticks from a recent time window."""
|
||||||
|
client = _sdk_client(ctx)
|
||||||
|
_execute_export(
|
||||||
|
ctx,
|
||||||
|
lambda: client.recent_ticks(
|
||||||
|
symbol,
|
||||||
|
seconds,
|
||||||
|
date_to=date_to,
|
||||||
|
count=count,
|
||||||
|
flags=flags,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
@app.command()
|
@app.command()
|
||||||
def account_info(ctx: typer.Context) -> None:
|
def account_info(ctx: typer.Context) -> None:
|
||||||
"""Export account information."""
|
"""Export account information."""
|
||||||
@@ -335,6 +398,16 @@ def symbol_info(
|
|||||||
_execute_export(ctx, lambda: client.symbol_info(symbol))
|
_execute_export(ctx, lambda: client.symbol_info(symbol))
|
||||||
|
|
||||||
|
|
||||||
|
@app.command()
|
||||||
|
def minimum_margins(
|
||||||
|
ctx: typer.Context,
|
||||||
|
symbol: Annotated[str, typer.Option(help="Symbol name.")],
|
||||||
|
) -> None:
|
||||||
|
"""Export minimum-volume buy and sell margin requirements."""
|
||||||
|
client = _sdk_client(ctx)
|
||||||
|
_execute_export(ctx, lambda: client.minimum_margins(symbol))
|
||||||
|
|
||||||
|
|
||||||
@app.command()
|
@app.command()
|
||||||
def orders(
|
def orders(
|
||||||
ctx: typer.Context,
|
ctx: typer.Context,
|
||||||
@@ -427,6 +500,37 @@ def history_deals(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@app.command()
|
||||||
|
def recent_history_deals(
|
||||||
|
ctx: typer.Context,
|
||||||
|
hours: Annotated[float, typer.Option(help="Lookback window in hours.")],
|
||||||
|
date_to: Annotated[
|
||||||
|
datetime | None,
|
||||||
|
typer.Option(click_type=DATETIME_TYPE, help="Window end date."),
|
||||||
|
] = None,
|
||||||
|
group: Annotated[str | None, typer.Option(help="Group filter.")] = None,
|
||||||
|
symbol: Annotated[str | None, typer.Option(help="Symbol filter.")] = None,
|
||||||
|
) -> None:
|
||||||
|
"""Export historical deals from a recent trailing window."""
|
||||||
|
client = _sdk_client(ctx)
|
||||||
|
_execute_export(
|
||||||
|
ctx,
|
||||||
|
lambda: client.recent_history_deals(
|
||||||
|
hours,
|
||||||
|
date_to=date_to,
|
||||||
|
group=group,
|
||||||
|
symbol=symbol,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@app.command()
|
||||||
|
def mt5_summary(ctx: typer.Context) -> None:
|
||||||
|
"""Export a compact terminal/account status summary."""
|
||||||
|
client = _sdk_client(ctx)
|
||||||
|
_execute_export(ctx, client.mt5_summary_as_df)
|
||||||
|
|
||||||
|
|
||||||
@app.command()
|
@app.command()
|
||||||
def version(ctx: typer.Context) -> None:
|
def version(ctx: typer.Context) -> None:
|
||||||
"""Export MetaTrader5 version information."""
|
"""Export MetaTrader5 version information."""
|
||||||
|
|||||||
+363
-2
@@ -5,7 +5,8 @@ from __future__ import annotations
|
|||||||
import logging
|
import logging
|
||||||
import sqlite3
|
import sqlite3
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
from typing import TYPE_CHECKING, Literal
|
from pathlib import Path
|
||||||
|
from typing import TYPE_CHECKING, Literal, cast
|
||||||
|
|
||||||
import pandas as pd
|
import pandas as pd
|
||||||
|
|
||||||
@@ -122,9 +123,369 @@ def build_rate_view_name(
|
|||||||
return f"rate_{symbol}__{granularity}_{timeframe}"
|
return f"rate_{symbol}__{granularity}_{timeframe}"
|
||||||
|
|
||||||
|
|
||||||
|
SqliteConnOrPath = sqlite3.Connection | Path | str
|
||||||
|
|
||||||
|
|
||||||
|
def _require_non_empty_identifier(identifier: str, kind: str) -> str:
|
||||||
|
value = identifier.strip()
|
||||||
|
if not value:
|
||||||
|
msg = f"SQLite {kind} name must not be empty."
|
||||||
|
raise ValueError(msg)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _open_history_connection(
|
||||||
|
conn_or_path: SqliteConnOrPath,
|
||||||
|
) -> tuple[sqlite3.Connection | None, bool]:
|
||||||
|
"""Open a read-only SQLite connection when given a path.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
A connection and whether the caller should close it. When the path does
|
||||||
|
not exist, returns ``(None, False)`` without creating a database file.
|
||||||
|
"""
|
||||||
|
if isinstance(conn_or_path, sqlite3.Connection):
|
||||||
|
return conn_or_path, False
|
||||||
|
path = Path(conn_or_path)
|
||||||
|
if not path.exists():
|
||||||
|
return None, False
|
||||||
|
conn = sqlite3.connect(f"{path.resolve().as_uri()}?mode=ro", uri=True)
|
||||||
|
return conn, True
|
||||||
|
|
||||||
|
|
||||||
|
def _open_existing_sqlite_database(
|
||||||
|
conn_or_path: SqliteConnOrPath,
|
||||||
|
) -> tuple[sqlite3.Connection, bool]:
|
||||||
|
"""Open a read-only SQLite database or reuse an existing connection.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Tuple of connection and whether the caller should close it.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
ValueError: If the database path does not exist or is not a file.
|
||||||
|
"""
|
||||||
|
if isinstance(conn_or_path, sqlite3.Connection):
|
||||||
|
return conn_or_path, False
|
||||||
|
path = Path(conn_or_path)
|
||||||
|
if not path.exists():
|
||||||
|
msg = f"SQLite database not found: {path}"
|
||||||
|
raise ValueError(msg)
|
||||||
|
if not path.is_file():
|
||||||
|
msg = f"SQLite database path is not a file: {path}"
|
||||||
|
raise ValueError(msg)
|
||||||
|
conn = sqlite3.connect(f"{path.resolve().as_uri()}?mode=ro", uri=True)
|
||||||
|
return conn, True
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_rate_load_request(table: str, count: int | None) -> str:
|
||||||
|
table_name = _require_non_empty_identifier(table, "table or view")
|
||||||
|
if count is not None and count <= 0:
|
||||||
|
msg = "count must be positive when provided."
|
||||||
|
raise ValueError(msg)
|
||||||
|
return table_name
|
||||||
|
|
||||||
|
|
||||||
|
def _ensure_rate_columns(columns: set[str], table: str) -> None:
|
||||||
|
if not columns:
|
||||||
|
msg = f"SQLite table or view not found: {table}"
|
||||||
|
raise ValueError(msg)
|
||||||
|
if "time" not in columns:
|
||||||
|
msg = f"SQLite table or view {table!r} must include a time column."
|
||||||
|
raise ValueError(msg)
|
||||||
|
if "close" not in columns and not {"ask", "bid"}.issubset(columns):
|
||||||
|
msg = (
|
||||||
|
f"SQLite table or view {table!r} must include close, "
|
||||||
|
"or both ask and bid columns."
|
||||||
|
)
|
||||||
|
raise ValueError(msg)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_rate_time_index(frame: pd.DataFrame, table: str) -> pd.DataFrame:
|
||||||
|
parsed = frame["time"].map(parse_sqlite_timestamp)
|
||||||
|
if parsed.isna().any():
|
||||||
|
msg = f"SQLite table or view {table!r} contains unparsable time values."
|
||||||
|
raise ValueError(msg)
|
||||||
|
result = frame.drop(columns=["time"])
|
||||||
|
result.index = pd.DatetimeIndex(parsed, name="time")
|
||||||
|
return result.sort_index(kind="stable")
|
||||||
|
|
||||||
|
|
||||||
|
def load_rate_data_from_connection(
|
||||||
|
connection: sqlite3.Connection,
|
||||||
|
table: str,
|
||||||
|
count: int | None = None,
|
||||||
|
) -> pd.DataFrame:
|
||||||
|
"""Load rate-like data from a SQLite table or view.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
connection: Open SQLite connection.
|
||||||
|
table: Source table or view name.
|
||||||
|
count: Optional number of most recent rows to load.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
DataFrame indexed by ascending ``time``.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
ValueError: If inputs, schema, timestamps are invalid, or the table
|
||||||
|
or view contains no rows.
|
||||||
|
"""
|
||||||
|
table_name = _validate_rate_load_request(table, count)
|
||||||
|
columns = get_table_columns(connection, table_name)
|
||||||
|
_ensure_rate_columns(columns, table_name)
|
||||||
|
quoted_table = quote_sqlite_identifier(table_name)
|
||||||
|
if count is None:
|
||||||
|
frame = cast(
|
||||||
|
"pd.DataFrame",
|
||||||
|
pd.read_sql_query( # type: ignore[reportUnknownMemberType]
|
||||||
|
f"SELECT * FROM {quoted_table} ORDER BY time ASC", # noqa: S608
|
||||||
|
connection,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
frame = cast(
|
||||||
|
"pd.DataFrame",
|
||||||
|
pd.read_sql_query( # type: ignore[reportUnknownMemberType]
|
||||||
|
f"SELECT * FROM {quoted_table} ORDER BY time DESC LIMIT ?", # noqa: S608
|
||||||
|
connection,
|
||||||
|
params=(count,),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
if frame.empty:
|
||||||
|
msg = f"SQLite table or view {table_name!r} contains no rows."
|
||||||
|
raise ValueError(msg)
|
||||||
|
return _parse_rate_time_index(frame, table_name)
|
||||||
|
|
||||||
|
|
||||||
|
def load_rate_data(
|
||||||
|
conn_or_path: SqliteConnOrPath,
|
||||||
|
table: str,
|
||||||
|
count: int | None = None,
|
||||||
|
) -> pd.DataFrame:
|
||||||
|
"""Load rate-like data from a SQLite database path or connection.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
conn_or_path: SQLite database path or open connection.
|
||||||
|
table: Source table or view name.
|
||||||
|
count: Optional number of most recent rows to load.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
DataFrame indexed by ascending ``time``.
|
||||||
|
|
||||||
|
"""
|
||||||
|
conn, should_close = _open_existing_sqlite_database(conn_or_path)
|
||||||
|
try:
|
||||||
|
return load_rate_data_from_connection(conn, table, count=count)
|
||||||
|
finally:
|
||||||
|
if should_close:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _load_rates_timeframe_counts(conn: sqlite3.Connection) -> dict[str, int] | None:
|
||||||
|
"""Return distinct timeframe counts per symbol from the normalized rates table."""
|
||||||
|
columns = get_table_columns(conn, Dataset.rates.table_name)
|
||||||
|
if not {"symbol", "timeframe"}.issubset(columns):
|
||||||
|
return None
|
||||||
|
rows = conn.execute(
|
||||||
|
"SELECT symbol, COUNT(DISTINCT timeframe) FROM rates GROUP BY symbol",
|
||||||
|
).fetchall()
|
||||||
|
return {str(symbol): int(count) for symbol, count in rows}
|
||||||
|
|
||||||
|
|
||||||
|
def _load_existing_rate_views(conn: sqlite3.Connection) -> set[str]:
|
||||||
|
"""Return mt5cli-managed ``rate_*__*`` compatibility view names."""
|
||||||
|
rows = conn.execute(
|
||||||
|
"SELECT name FROM sqlite_master WHERE type = 'view' AND name GLOB 'rate_*__*'",
|
||||||
|
).fetchall()
|
||||||
|
return {str(row[0]) for row in rows}
|
||||||
|
|
||||||
|
|
||||||
|
def _rate_view_name_candidates(
|
||||||
|
*,
|
||||||
|
symbol: str,
|
||||||
|
granularity: str,
|
||||||
|
granularity_count: int,
|
||||||
|
timeframe: int,
|
||||||
|
) -> list[str]:
|
||||||
|
"""Return candidate view names in preference order."""
|
||||||
|
single = build_rate_view_name(
|
||||||
|
symbol=symbol,
|
||||||
|
granularity=granularity,
|
||||||
|
granularity_count=1,
|
||||||
|
timeframe=timeframe,
|
||||||
|
)
|
||||||
|
if granularity_count <= 1:
|
||||||
|
return [single]
|
||||||
|
multi = build_rate_view_name(
|
||||||
|
symbol=symbol,
|
||||||
|
granularity=granularity,
|
||||||
|
granularity_count=granularity_count,
|
||||||
|
timeframe=timeframe,
|
||||||
|
)
|
||||||
|
return [multi, single]
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_rate_view_name_from_context(
|
||||||
|
*,
|
||||||
|
symbol: str,
|
||||||
|
timeframe: int,
|
||||||
|
granularity_name: str,
|
||||||
|
timeframe_counts: dict[str, int] | None,
|
||||||
|
existing_views: set[str],
|
||||||
|
require_existing: bool = False,
|
||||||
|
) -> str:
|
||||||
|
"""Resolve one rate view name using preloaded SQLite metadata.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Preferred mt5cli-managed rate compatibility view name.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
ValueError: If ``require_existing`` is True and no managed view exists.
|
||||||
|
"""
|
||||||
|
if timeframe_counts is None or symbol not in timeframe_counts:
|
||||||
|
candidates = [
|
||||||
|
build_rate_view_name(
|
||||||
|
symbol=symbol,
|
||||||
|
granularity=granularity_name,
|
||||||
|
granularity_count=1,
|
||||||
|
timeframe=timeframe,
|
||||||
|
),
|
||||||
|
build_rate_view_name(
|
||||||
|
symbol=symbol,
|
||||||
|
granularity=granularity_name,
|
||||||
|
granularity_count=2,
|
||||||
|
timeframe=timeframe,
|
||||||
|
),
|
||||||
|
]
|
||||||
|
else:
|
||||||
|
candidates = _rate_view_name_candidates(
|
||||||
|
symbol=symbol,
|
||||||
|
granularity=granularity_name,
|
||||||
|
granularity_count=timeframe_counts[symbol],
|
||||||
|
timeframe=timeframe,
|
||||||
|
)
|
||||||
|
for candidate in candidates:
|
||||||
|
if candidate in existing_views:
|
||||||
|
return candidate
|
||||||
|
if require_existing:
|
||||||
|
msg = (
|
||||||
|
f"No rate compatibility view exists for symbol {symbol!r} "
|
||||||
|
f"and granularity {granularity_name!r}; "
|
||||||
|
f"candidates: {', '.join(candidates)}."
|
||||||
|
)
|
||||||
|
raise ValueError(msg)
|
||||||
|
return candidates[0]
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_rate_view_name(
|
||||||
|
conn_or_path: SqliteConnOrPath,
|
||||||
|
symbol: str,
|
||||||
|
granularity: str,
|
||||||
|
*,
|
||||||
|
require_existing: bool = False,
|
||||||
|
) -> str:
|
||||||
|
"""Resolve the mt5cli-managed rate compatibility view name.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
conn_or_path: SQLite database path or open connection.
|
||||||
|
symbol: Symbol stored in the normalized ``rates`` table.
|
||||||
|
granularity: Timeframe name (for example ``M1``) or integer string.
|
||||||
|
require_existing: When True, require the database and a managed view to exist.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
View name such as ``rate_EURUSD__1`` or ``rate_EURUSD__M1_1``.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
ValueError: If ``require_existing`` is True and the database or view is missing.
|
||||||
|
"""
|
||||||
|
timeframe = parse_timeframe(granularity)
|
||||||
|
granularity_name = resolve_granularity_name(timeframe)
|
||||||
|
conn, should_close = _open_history_connection(conn_or_path)
|
||||||
|
try:
|
||||||
|
if conn is None:
|
||||||
|
if require_existing:
|
||||||
|
path = (
|
||||||
|
conn_or_path
|
||||||
|
if isinstance(conn_or_path, (Path, str))
|
||||||
|
else "database"
|
||||||
|
)
|
||||||
|
msg = f"SQLite database not found: {path}"
|
||||||
|
raise ValueError(msg)
|
||||||
|
return build_rate_view_name(
|
||||||
|
symbol=symbol,
|
||||||
|
granularity=granularity_name,
|
||||||
|
granularity_count=1,
|
||||||
|
timeframe=timeframe,
|
||||||
|
)
|
||||||
|
return _resolve_rate_view_name_from_context(
|
||||||
|
symbol=symbol,
|
||||||
|
timeframe=timeframe,
|
||||||
|
granularity_name=granularity_name,
|
||||||
|
timeframe_counts=_load_rates_timeframe_counts(conn),
|
||||||
|
existing_views=_load_existing_rate_views(conn),
|
||||||
|
require_existing=require_existing,
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
if should_close and conn is not None:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_rate_view_names(
|
||||||
|
conn_or_path: SqliteConnOrPath,
|
||||||
|
symbols: Sequence[str],
|
||||||
|
granularities: Sequence[str],
|
||||||
|
*,
|
||||||
|
require_existing: bool = False,
|
||||||
|
) -> list[str]:
|
||||||
|
"""Resolve rate compatibility view names for symbol and granularity pairs.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
conn_or_path: SQLite database path or open connection.
|
||||||
|
symbols: Symbols stored in the normalized ``rates`` table.
|
||||||
|
granularities: Timeframe names (for example ``M1``) or integer strings.
|
||||||
|
require_existing: When True, require the database and managed views to exist.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
View names in row-major order: every ``granularity`` for the first
|
||||||
|
symbol, then every granularity for the next symbol, and so on.
|
||||||
|
"""
|
||||||
|
conn, should_close = _open_history_connection(conn_or_path)
|
||||||
|
try:
|
||||||
|
if conn is None:
|
||||||
|
return [
|
||||||
|
resolve_rate_view_name(
|
||||||
|
conn_or_path,
|
||||||
|
symbol,
|
||||||
|
granularity,
|
||||||
|
require_existing=require_existing,
|
||||||
|
)
|
||||||
|
for symbol in symbols
|
||||||
|
for granularity in granularities
|
||||||
|
]
|
||||||
|
timeframe_counts = _load_rates_timeframe_counts(conn)
|
||||||
|
existing_views = _load_existing_rate_views(conn)
|
||||||
|
resolved: list[str] = []
|
||||||
|
for symbol in symbols:
|
||||||
|
for granularity in granularities:
|
||||||
|
timeframe = parse_timeframe(granularity)
|
||||||
|
resolved.append(
|
||||||
|
_resolve_rate_view_name_from_context(
|
||||||
|
symbol=symbol,
|
||||||
|
timeframe=timeframe,
|
||||||
|
granularity_name=resolve_granularity_name(timeframe),
|
||||||
|
timeframe_counts=timeframe_counts,
|
||||||
|
existing_views=existing_views,
|
||||||
|
require_existing=require_existing,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
return resolved
|
||||||
|
finally:
|
||||||
|
if should_close and conn is not None:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
def get_table_columns(conn: sqlite3.Connection, table: str) -> set[str]:
|
def get_table_columns(conn: sqlite3.Connection, table: str) -> set[str]:
|
||||||
"""Return existing SQLite columns for a table."""
|
"""Return existing SQLite columns for a table."""
|
||||||
rows = conn.execute(f"PRAGMA table_info({table})").fetchall()
|
quoted_table = quote_sqlite_identifier(table)
|
||||||
|
rows = conn.execute(f"PRAGMA table_info({quoted_table})").fetchall()
|
||||||
return {str(row[1]) for row in rows}
|
return {str(row[1]) for row in rows}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
+404
-6
@@ -2,14 +2,16 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
import logging
|
import logging
|
||||||
import sqlite3
|
import sqlite3
|
||||||
from contextlib import contextmanager
|
from contextlib import contextmanager
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from datetime import UTC, datetime, timedelta
|
from datetime import UTC, datetime, timedelta
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import TYPE_CHECKING, Self, TypeVar
|
from typing import TYPE_CHECKING, Self, TypeVar, cast
|
||||||
|
|
||||||
|
import pandas as pd
|
||||||
from pdmt5 import Mt5Config, Mt5DataClient
|
from pdmt5 import Mt5Config, Mt5DataClient
|
||||||
|
|
||||||
from .history import (
|
from .history import (
|
||||||
@@ -33,8 +35,6 @@ from .utils import (
|
|||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from collections.abc import Callable, Iterator, Sequence
|
from collections.abc import Callable, Iterator, Sequence
|
||||||
|
|
||||||
import pandas as pd
|
|
||||||
|
|
||||||
T = TypeVar("T")
|
T = TypeVar("T")
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
@@ -44,6 +44,7 @@ __all__ = [
|
|||||||
"account_info",
|
"account_info",
|
||||||
"build_config",
|
"build_config",
|
||||||
"collect_history",
|
"collect_history",
|
||||||
|
"collect_latest_rates",
|
||||||
"copy_rates_from",
|
"copy_rates_from",
|
||||||
"copy_rates_from_pos",
|
"copy_rates_from_pos",
|
||||||
"copy_rates_range",
|
"copy_rates_range",
|
||||||
@@ -52,9 +53,15 @@ __all__ = [
|
|||||||
"history_deals",
|
"history_deals",
|
||||||
"history_orders",
|
"history_orders",
|
||||||
"last_error",
|
"last_error",
|
||||||
|
"latest_rates",
|
||||||
"market_book",
|
"market_book",
|
||||||
|
"minimum_margins",
|
||||||
|
"mt5_summary",
|
||||||
|
"mt5_summary_as_df",
|
||||||
"orders",
|
"orders",
|
||||||
"positions",
|
"positions",
|
||||||
|
"recent_history_deals",
|
||||||
|
"recent_ticks",
|
||||||
"symbol_info",
|
"symbol_info",
|
||||||
"symbol_info_tick",
|
"symbol_info_tick",
|
||||||
"symbols",
|
"symbols",
|
||||||
@@ -77,6 +84,22 @@ def _coerce_tick_flags(flags: int | str) -> int:
|
|||||||
return parse_tick_flags(flags)
|
return parse_tick_flags(flags)
|
||||||
|
|
||||||
|
|
||||||
|
def _plain_mt5_value(value: object) -> object:
|
||||||
|
asdict = getattr(value, "_asdict", None)
|
||||||
|
if callable(asdict):
|
||||||
|
return _plain_mt5_value(asdict())
|
||||||
|
if isinstance(value, dict):
|
||||||
|
typed_value = cast("dict[object, object]", value)
|
||||||
|
return {key: _plain_mt5_value(item) for key, item in typed_value.items()}
|
||||||
|
if isinstance(value, tuple):
|
||||||
|
typed_value = cast("tuple[object, ...]", value)
|
||||||
|
return [_plain_mt5_value(item) for item in typed_value]
|
||||||
|
if isinstance(value, list):
|
||||||
|
typed_value = cast("list[object]", value)
|
||||||
|
return [_plain_mt5_value(item) for item in typed_value]
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
def _require_datetime(value: datetime | str) -> datetime:
|
def _require_datetime(value: datetime | str) -> datetime:
|
||||||
if isinstance(value, datetime):
|
if isinstance(value, datetime):
|
||||||
return value
|
return value
|
||||||
@@ -89,6 +112,114 @@ def _coerce_datetime(value: datetime | str | None) -> datetime | None:
|
|||||||
return parse_datetime(value)
|
return parse_datetime(value)
|
||||||
|
|
||||||
|
|
||||||
|
def _require_positive(value: float, name: str) -> None:
|
||||||
|
if value <= 0:
|
||||||
|
msg = f"{name} must be positive."
|
||||||
|
raise ValueError(msg)
|
||||||
|
|
||||||
|
|
||||||
|
def _call_required_client_method(client: Mt5DataClient, name: str) -> object:
|
||||||
|
try:
|
||||||
|
method = getattr(client, name)
|
||||||
|
except AttributeError as exc:
|
||||||
|
msg = f"MT5 client is missing required method: {name}"
|
||||||
|
raise AttributeError(msg) from exc
|
||||||
|
if not callable(method):
|
||||||
|
msg = f"MT5 client attribute is not callable: {name}"
|
||||||
|
raise TypeError(msg)
|
||||||
|
return method()
|
||||||
|
|
||||||
|
|
||||||
|
def _mt5_summary_export_value(value: object) -> object:
|
||||||
|
plain_value = _plain_mt5_value(value)
|
||||||
|
if isinstance(plain_value, dict | list):
|
||||||
|
return json.dumps(plain_value, sort_keys=True, separators=(",", ":"))
|
||||||
|
return plain_value
|
||||||
|
|
||||||
|
|
||||||
|
def _coerce_tick_time(value: object) -> datetime:
|
||||||
|
if isinstance(value, datetime):
|
||||||
|
return value
|
||||||
|
if isinstance(value, str):
|
||||||
|
return parse_datetime(value)
|
||||||
|
if isinstance(value, (int, float)):
|
||||||
|
return datetime.fromtimestamp(value, tz=UTC)
|
||||||
|
msg = f"Unsupported tick time value: {value!r}"
|
||||||
|
raise TypeError(msg)
|
||||||
|
|
||||||
|
|
||||||
|
def _filter_ticks_to_end(frame: pd.DataFrame, end: datetime) -> pd.DataFrame:
|
||||||
|
if frame.empty or "time" not in frame.columns:
|
||||||
|
return frame
|
||||||
|
times = pd.to_datetime(frame["time"], utc=True)
|
||||||
|
return frame.loc[times <= end].reset_index(drop=True)
|
||||||
|
|
||||||
|
|
||||||
|
def _fetch_recent_ticks(
|
||||||
|
client: Mt5DataClient,
|
||||||
|
symbol: str,
|
||||||
|
seconds: float,
|
||||||
|
date_to: datetime | None,
|
||||||
|
count: int,
|
||||||
|
flags: int,
|
||||||
|
) -> pd.DataFrame:
|
||||||
|
if date_to is not None:
|
||||||
|
end = date_to
|
||||||
|
else:
|
||||||
|
tick = client.symbol_info_tick(symbol)
|
||||||
|
end = _coerce_tick_time(tick.time)
|
||||||
|
start = end - timedelta(seconds=seconds)
|
||||||
|
if count > 0:
|
||||||
|
from_frame = _filter_ticks_to_end(
|
||||||
|
client.copy_ticks_from_as_df(
|
||||||
|
symbol=symbol,
|
||||||
|
date_from=start,
|
||||||
|
count=count,
|
||||||
|
flags=flags,
|
||||||
|
),
|
||||||
|
end,
|
||||||
|
)
|
||||||
|
if len(from_frame) < count:
|
||||||
|
return from_frame
|
||||||
|
frame = client.copy_ticks_range_as_df(
|
||||||
|
symbol=symbol,
|
||||||
|
date_from=start,
|
||||||
|
date_to=end,
|
||||||
|
flags=flags,
|
||||||
|
)
|
||||||
|
if count > 0 and len(frame) > count:
|
||||||
|
return frame.tail(count).reset_index(drop=True)
|
||||||
|
return frame
|
||||||
|
|
||||||
|
|
||||||
|
def _fetch_minimum_margins(client: Mt5DataClient, symbol: str) -> pd.DataFrame:
|
||||||
|
sym = client.symbol_info(symbol)
|
||||||
|
account = client.account_info()
|
||||||
|
tick = client.symbol_info_tick(symbol)
|
||||||
|
volume_min = sym.volume_min
|
||||||
|
buy_margin = client.order_calc_margin(
|
||||||
|
client.mt5.ORDER_TYPE_BUY,
|
||||||
|
symbol,
|
||||||
|
volume_min,
|
||||||
|
tick.ask,
|
||||||
|
)
|
||||||
|
sell_margin = client.order_calc_margin(
|
||||||
|
client.mt5.ORDER_TYPE_SELL,
|
||||||
|
symbol,
|
||||||
|
volume_min,
|
||||||
|
tick.bid,
|
||||||
|
)
|
||||||
|
return pd.DataFrame([
|
||||||
|
{
|
||||||
|
"symbol": symbol,
|
||||||
|
"account_currency": account.currency,
|
||||||
|
"volume_min": volume_min,
|
||||||
|
"buy_margin": buy_margin,
|
||||||
|
"sell_margin": sell_margin,
|
||||||
|
}
|
||||||
|
])
|
||||||
|
|
||||||
|
|
||||||
def build_config(
|
def build_config(
|
||||||
*,
|
*,
|
||||||
path: str | None = None,
|
path: str | None = None,
|
||||||
@@ -158,6 +289,7 @@ class Mt5CliClient:
|
|||||||
server: str | None = None,
|
server: str | None = None,
|
||||||
timeout: int | None = None,
|
timeout: int | None = None,
|
||||||
config: Mt5Config | None = None,
|
config: Mt5Config | None = None,
|
||||||
|
client: Mt5DataClient | None = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Initialize the SDK client.
|
"""Initialize the SDK client.
|
||||||
|
|
||||||
@@ -168,6 +300,8 @@ class Mt5CliClient:
|
|||||||
server: Trading server name.
|
server: Trading server name.
|
||||||
timeout: Connection timeout in milliseconds.
|
timeout: Connection timeout in milliseconds.
|
||||||
config: Optional pre-built ``Mt5Config`` (overrides other args).
|
config: Optional pre-built ``Mt5Config`` (overrides other args).
|
||||||
|
client: Optional already-connected ``Mt5DataClient``. Injected
|
||||||
|
clients are reused as-is and are not initialized or shut down.
|
||||||
"""
|
"""
|
||||||
self._config = config or build_config(
|
self._config = config or build_config(
|
||||||
path=path,
|
path=path,
|
||||||
@@ -176,7 +310,20 @@ class Mt5CliClient:
|
|||||||
server=server,
|
server=server,
|
||||||
timeout=timeout,
|
timeout=timeout,
|
||||||
)
|
)
|
||||||
self._client: Mt5DataClient | None = None
|
self._client = client
|
||||||
|
self._owns_client = client is None
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_connected_client(cls, client: Mt5DataClient) -> Self:
|
||||||
|
"""Bind to an already-connected ``Mt5DataClient`` without owning it.
|
||||||
|
|
||||||
|
The returned ``Mt5CliClient`` never initializes or shuts down the
|
||||||
|
injected client, including when used as a context manager.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Client wrapper bound to the injected connection.
|
||||||
|
"""
|
||||||
|
return cls(client=client)
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def config(self) -> Mt5Config:
|
def config(self) -> Mt5Config:
|
||||||
@@ -189,6 +336,8 @@ class Mt5CliClient:
|
|||||||
Returns:
|
Returns:
|
||||||
This client instance.
|
This client instance.
|
||||||
"""
|
"""
|
||||||
|
if self._client is not None:
|
||||||
|
return self
|
||||||
client = Mt5DataClient(config=self._config)
|
client = Mt5DataClient(config=self._config)
|
||||||
try:
|
try:
|
||||||
client.initialize_and_login_mt5()
|
client.initialize_and_login_mt5()
|
||||||
@@ -196,6 +345,7 @@ class Mt5CliClient:
|
|||||||
client.shutdown()
|
client.shutdown()
|
||||||
raise
|
raise
|
||||||
self._client = client
|
self._client = client
|
||||||
|
self._owns_client = True # only set when this method created the client
|
||||||
return self
|
return self
|
||||||
|
|
||||||
def __exit__(
|
def __exit__(
|
||||||
@@ -205,15 +355,18 @@ class Mt5CliClient:
|
|||||||
tb: object,
|
tb: object,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Shut down the persistent MT5 connection."""
|
"""Shut down the persistent MT5 connection."""
|
||||||
if self._client is not None:
|
if self._client is not None and self._owns_client:
|
||||||
self._client.shutdown()
|
self._client.shutdown()
|
||||||
self._client = None
|
self._client = None
|
||||||
|
|
||||||
def _fetch(self, fetch_fn: Callable[[Mt5DataClient], pd.DataFrame]) -> pd.DataFrame:
|
def _fetch_value(self, fetch_fn: Callable[[Mt5DataClient], T]) -> T:
|
||||||
if self._client is not None:
|
if self._client is not None:
|
||||||
return fetch_fn(self._client)
|
return fetch_fn(self._client)
|
||||||
return _run_with_client(self._config, fetch_fn)
|
return _run_with_client(self._config, fetch_fn)
|
||||||
|
|
||||||
|
def _fetch(self, fetch_fn: Callable[[Mt5DataClient], pd.DataFrame]) -> pd.DataFrame:
|
||||||
|
return self._fetch_value(fetch_fn)
|
||||||
|
|
||||||
def copy_rates_from(
|
def copy_rates_from(
|
||||||
self,
|
self,
|
||||||
symbol: str,
|
symbol: str,
|
||||||
@@ -251,6 +404,54 @@ class Mt5CliClient:
|
|||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def latest_rates(
|
||||||
|
self,
|
||||||
|
symbol: str,
|
||||||
|
timeframe: int | str,
|
||||||
|
count: int,
|
||||||
|
start_pos: int = 0,
|
||||||
|
) -> pd.DataFrame:
|
||||||
|
"""Return the latest rates from a bar position."""
|
||||||
|
_require_positive(count, "count")
|
||||||
|
return self.copy_rates_from_pos(symbol, timeframe, start_pos, count)
|
||||||
|
|
||||||
|
def collect_latest_rates(
|
||||||
|
self,
|
||||||
|
symbols: Sequence[str],
|
||||||
|
timeframes: Sequence[int | str],
|
||||||
|
*,
|
||||||
|
count: int,
|
||||||
|
start_pos: int = 0,
|
||||||
|
) -> dict[tuple[str, int], pd.DataFrame]:
|
||||||
|
"""Return latest rates for each symbol/timeframe pair.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Mapping keyed by ``(symbol, timeframe_int)``.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
ValueError: If ``count`` is not positive or inputs are empty.
|
||||||
|
"""
|
||||||
|
_require_positive(count, "count")
|
||||||
|
if not symbols:
|
||||||
|
msg = "At least one symbol is required."
|
||||||
|
raise ValueError(msg)
|
||||||
|
if not timeframes:
|
||||||
|
msg = "At least one timeframe is required."
|
||||||
|
raise ValueError(msg)
|
||||||
|
resolved_timeframes = [_coerce_timeframe(timeframe) for timeframe in timeframes]
|
||||||
|
return self._fetch_value(
|
||||||
|
lambda c: {
|
||||||
|
(symbol, timeframe): c.copy_rates_from_pos_as_df(
|
||||||
|
symbol=symbol,
|
||||||
|
timeframe=timeframe,
|
||||||
|
start_pos=start_pos,
|
||||||
|
count=count,
|
||||||
|
)
|
||||||
|
for symbol in symbols
|
||||||
|
for timeframe in resolved_timeframes
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
def copy_rates_range(
|
def copy_rates_range(
|
||||||
self,
|
self,
|
||||||
symbol: str,
|
symbol: str,
|
||||||
@@ -402,6 +603,24 @@ class Mt5CliClient:
|
|||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def recent_history_deals(
|
||||||
|
self,
|
||||||
|
hours: float,
|
||||||
|
date_to: datetime | str | None = None,
|
||||||
|
group: str | None = None,
|
||||||
|
symbol: str | None = None,
|
||||||
|
) -> pd.DataFrame:
|
||||||
|
"""Return historical deals from a recent trailing window."""
|
||||||
|
_require_positive(hours, "hours")
|
||||||
|
end = _require_datetime(date_to) if date_to is not None else datetime.now(UTC)
|
||||||
|
start = end - timedelta(hours=hours)
|
||||||
|
return self.history_deals(
|
||||||
|
date_from=start,
|
||||||
|
date_to=end,
|
||||||
|
group=group,
|
||||||
|
symbol=symbol,
|
||||||
|
)
|
||||||
|
|
||||||
def version(self) -> pd.DataFrame:
|
def version(self) -> pd.DataFrame:
|
||||||
"""Return MetaTrader5 version information."""
|
"""Return MetaTrader5 version information."""
|
||||||
return self._fetch(lambda c: c.version_as_df())
|
return self._fetch(lambda c: c.version_as_df())
|
||||||
@@ -418,6 +637,90 @@ class Mt5CliClient:
|
|||||||
"""Return market depth for a symbol."""
|
"""Return market depth for a symbol."""
|
||||||
return self._fetch(lambda c: c.market_book_get_as_df(symbol=symbol))
|
return self._fetch(lambda c: c.market_book_get_as_df(symbol=symbol))
|
||||||
|
|
||||||
|
def recent_ticks(
|
||||||
|
self,
|
||||||
|
symbol: str,
|
||||||
|
seconds: float,
|
||||||
|
*,
|
||||||
|
date_to: datetime | str | None = None,
|
||||||
|
count: int = 10000,
|
||||||
|
flags: int | str = "ALL",
|
||||||
|
) -> pd.DataFrame:
|
||||||
|
"""Return ticks from a recent time window.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
symbol: Symbol name.
|
||||||
|
seconds: Lookback window in seconds ending at ``date_to``.
|
||||||
|
date_to: Window end time. When ``None``, uses the latest
|
||||||
|
``symbol_info_tick().time`` rather than wall-clock now.
|
||||||
|
count: Maximum ticks to return. Values ``<= 0`` return the full
|
||||||
|
window without trimming. Positive values keep the most recent
|
||||||
|
ticks; when the window is sparse, ``copy_ticks_from`` avoids
|
||||||
|
fetching the entire range.
|
||||||
|
flags: Tick flags as ``ALL``, ``INFO``, ``TRADE``, or an integer.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Tick DataFrame with MT5 tick columns such as ``time``, ``bid``,
|
||||||
|
``ask``, ``last``, and ``volume``.
|
||||||
|
"""
|
||||||
|
tick_flags = _coerce_tick_flags(flags)
|
||||||
|
end = _coerce_datetime(date_to)
|
||||||
|
return self._fetch(
|
||||||
|
lambda c: _fetch_recent_ticks(
|
||||||
|
c,
|
||||||
|
symbol,
|
||||||
|
seconds,
|
||||||
|
end,
|
||||||
|
count,
|
||||||
|
tick_flags,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
def minimum_margins(self, symbol: str) -> pd.DataFrame:
|
||||||
|
"""Return minimum-volume buy and sell margin requirements.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
symbol: Symbol name.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
One-row DataFrame with columns ``symbol``, ``account_currency``,
|
||||||
|
``volume_min``, ``buy_margin``, and ``sell_margin``.
|
||||||
|
"""
|
||||||
|
return self._fetch(lambda c: _fetch_minimum_margins(c, symbol))
|
||||||
|
|
||||||
|
def mt5_summary(self) -> dict[str, object]:
|
||||||
|
"""Return a compact terminal/account status summary."""
|
||||||
|
|
||||||
|
def _summary(client: Mt5DataClient) -> dict[str, object]:
|
||||||
|
return {
|
||||||
|
"version": _plain_mt5_value(
|
||||||
|
_call_required_client_method(client, "version"),
|
||||||
|
),
|
||||||
|
"terminal_info": _plain_mt5_value(
|
||||||
|
_call_required_client_method(client, "terminal_info"),
|
||||||
|
),
|
||||||
|
"account_info": _plain_mt5_value(
|
||||||
|
_call_required_client_method(client, "account_info"),
|
||||||
|
),
|
||||||
|
"symbols_total": _plain_mt5_value(
|
||||||
|
_call_required_client_method(client, "symbols_total"),
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
return self._fetch_value(_summary)
|
||||||
|
|
||||||
|
def mt5_summary_as_df(self) -> pd.DataFrame:
|
||||||
|
"""Return an export-safe one-row terminal/account summary DataFrame."""
|
||||||
|
summary = self.mt5_summary()
|
||||||
|
return pd.DataFrame(
|
||||||
|
[
|
||||||
|
{
|
||||||
|
key: _mt5_summary_export_value(value)
|
||||||
|
for key, value in summary.items()
|
||||||
|
},
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _resolve_incremental_settings(
|
def _resolve_incremental_settings(
|
||||||
selected_datasets: set[Dataset],
|
selected_datasets: set[Dataset],
|
||||||
@@ -738,6 +1041,40 @@ def copy_rates_from_pos(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def latest_rates(
|
||||||
|
symbol: str,
|
||||||
|
timeframe: int | str,
|
||||||
|
count: int,
|
||||||
|
start_pos: int = 0,
|
||||||
|
*,
|
||||||
|
config: Mt5Config | None = None,
|
||||||
|
) -> pd.DataFrame:
|
||||||
|
"""Return the latest rates from a bar position."""
|
||||||
|
return _make_client(config=config).latest_rates(
|
||||||
|
symbol,
|
||||||
|
timeframe,
|
||||||
|
count,
|
||||||
|
start_pos=start_pos,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def collect_latest_rates(
|
||||||
|
symbols: Sequence[str],
|
||||||
|
timeframes: Sequence[int | str],
|
||||||
|
*,
|
||||||
|
count: int,
|
||||||
|
start_pos: int = 0,
|
||||||
|
config: Mt5Config | None = None,
|
||||||
|
) -> dict[tuple[str, int], pd.DataFrame]:
|
||||||
|
"""Return latest rates for each symbol/timeframe pair."""
|
||||||
|
return _make_client(config=config).collect_latest_rates(
|
||||||
|
symbols,
|
||||||
|
timeframes,
|
||||||
|
count=count,
|
||||||
|
start_pos=start_pos,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def copy_rates_range(
|
def copy_rates_range(
|
||||||
symbol: str,
|
symbol: str,
|
||||||
timeframe: int | str,
|
timeframe: int | str,
|
||||||
@@ -889,6 +1226,23 @@ def history_deals(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def recent_history_deals(
|
||||||
|
hours: float,
|
||||||
|
date_to: datetime | str | None = None,
|
||||||
|
group: str | None = None,
|
||||||
|
symbol: str | None = None,
|
||||||
|
*,
|
||||||
|
config: Mt5Config | None = None,
|
||||||
|
) -> pd.DataFrame:
|
||||||
|
"""Return historical deals from a recent trailing window."""
|
||||||
|
return _make_client(config=config).recent_history_deals(
|
||||||
|
hours,
|
||||||
|
date_to=date_to,
|
||||||
|
group=group,
|
||||||
|
symbol=symbol,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def version(*, config: Mt5Config | None = None) -> pd.DataFrame:
|
def version(*, config: Mt5Config | None = None) -> pd.DataFrame:
|
||||||
"""Return MetaTrader5 version information."""
|
"""Return MetaTrader5 version information."""
|
||||||
return _make_client(config=config).version()
|
return _make_client(config=config).version()
|
||||||
@@ -915,3 +1269,47 @@ def market_book(
|
|||||||
) -> pd.DataFrame:
|
) -> pd.DataFrame:
|
||||||
"""Return market depth for a symbol."""
|
"""Return market depth for a symbol."""
|
||||||
return _make_client(config=config).market_book(symbol)
|
return _make_client(config=config).market_book(symbol)
|
||||||
|
|
||||||
|
|
||||||
|
def recent_ticks(
|
||||||
|
symbol: str,
|
||||||
|
seconds: float,
|
||||||
|
*,
|
||||||
|
date_to: datetime | str | None = None,
|
||||||
|
count: int = 10000,
|
||||||
|
flags: int | str = "ALL",
|
||||||
|
config: Mt5Config | None = None,
|
||||||
|
) -> pd.DataFrame:
|
||||||
|
"""Return ticks from a recent time window ending at ``date_to`` or now.
|
||||||
|
|
||||||
|
See ``Mt5CliClient.recent_ticks`` for parameter and return details.
|
||||||
|
"""
|
||||||
|
return _make_client(config=config).recent_ticks(
|
||||||
|
symbol,
|
||||||
|
seconds,
|
||||||
|
date_to=date_to,
|
||||||
|
count=count,
|
||||||
|
flags=flags,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def minimum_margins(
|
||||||
|
symbol: str,
|
||||||
|
*,
|
||||||
|
config: Mt5Config | None = None,
|
||||||
|
) -> pd.DataFrame:
|
||||||
|
"""Return minimum-volume buy and sell margin requirements.
|
||||||
|
|
||||||
|
See ``Mt5CliClient.minimum_margins`` for return details.
|
||||||
|
"""
|
||||||
|
return _make_client(config=config).minimum_margins(symbol)
|
||||||
|
|
||||||
|
|
||||||
|
def mt5_summary(*, config: Mt5Config | None = None) -> dict[str, object]:
|
||||||
|
"""Return a compact terminal/account status summary."""
|
||||||
|
return _make_client(config=config).mt5_summary()
|
||||||
|
|
||||||
|
|
||||||
|
def mt5_summary_as_df(*, config: Mt5Config | None = None) -> pd.DataFrame:
|
||||||
|
"""Return an export-safe terminal/account status summary DataFrame."""
|
||||||
|
return _make_client(config=config).mt5_summary_as_df()
|
||||||
|
|||||||
+55
-10
@@ -2,16 +2,18 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import importlib
|
|
||||||
import json
|
import json
|
||||||
|
import sqlite3
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
from enum import StrEnum
|
from enum import StrEnum
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import TYPE_CHECKING, Any, TypeGuard, cast
|
from typing import TYPE_CHECKING, Any, TypeGuard
|
||||||
|
|
||||||
import click
|
import click
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
|
from collections.abc import Sequence
|
||||||
|
|
||||||
import pandas as pd
|
import pandas as pd
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -260,6 +262,50 @@ def detect_format(
|
|||||||
raise ValueError(msg)
|
raise ValueError(msg)
|
||||||
|
|
||||||
|
|
||||||
|
def export_dataframe_to_sqlite(
|
||||||
|
df: pd.DataFrame,
|
||||||
|
output_path: Path,
|
||||||
|
table_name: str = "data",
|
||||||
|
*,
|
||||||
|
if_exists: IfExists = IfExists.APPEND,
|
||||||
|
index: bool = False,
|
||||||
|
index_label: str | None = None,
|
||||||
|
deduplicate_on: Sequence[str] | None = None,
|
||||||
|
) -> None:
|
||||||
|
"""Write a DataFrame to SQLite with configurable append and deduplication.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
df: DataFrame to export.
|
||||||
|
output_path: SQLite database path.
|
||||||
|
table_name: Target table name.
|
||||||
|
if_exists: Conflict behavior when the table already exists.
|
||||||
|
index: Whether to write the DataFrame index as a column.
|
||||||
|
index_label: Column name for the index when ``index=True``.
|
||||||
|
deduplicate_on: Optional key columns to deduplicate after writing,
|
||||||
|
keeping the latest ``ROWID`` per key group. Deduplication scans the
|
||||||
|
full table, so repeated appends cost O(table size); index the key
|
||||||
|
columns when appending frequently.
|
||||||
|
"""
|
||||||
|
with sqlite3.connect(output_path) as conn:
|
||||||
|
df.to_sql( # type: ignore[reportUnknownMemberType]
|
||||||
|
table_name,
|
||||||
|
conn,
|
||||||
|
if_exists=if_exists.value,
|
||||||
|
index=index,
|
||||||
|
index_label=index_label,
|
||||||
|
)
|
||||||
|
if deduplicate_on:
|
||||||
|
from .history import drop_duplicates_in_table # noqa: PLC0415
|
||||||
|
|
||||||
|
drop_duplicates_in_table(
|
||||||
|
conn.cursor(),
|
||||||
|
table_name,
|
||||||
|
list(deduplicate_on),
|
||||||
|
keep="last",
|
||||||
|
)
|
||||||
|
conn.commit()
|
||||||
|
|
||||||
|
|
||||||
def export_dataframe(
|
def export_dataframe(
|
||||||
df: pd.DataFrame,
|
df: pd.DataFrame,
|
||||||
output_path: Path,
|
output_path: Path,
|
||||||
@@ -289,14 +335,13 @@ def export_dataframe(
|
|||||||
elif output_format == "parquet":
|
elif output_format == "parquet":
|
||||||
df.to_parquet(output_path, index=False)
|
df.to_parquet(output_path, index=False)
|
||||||
elif output_format == "sqlite3":
|
elif output_format == "sqlite3":
|
||||||
sqlite3 = cast("Any", importlib.import_module("sqlite3"))
|
export_dataframe_to_sqlite(
|
||||||
with sqlite3.connect(output_path) as conn:
|
df,
|
||||||
df.to_sql( # type: ignore[reportUnknownMemberType]
|
output_path,
|
||||||
table_name,
|
table_name,
|
||||||
conn,
|
if_exists=IfExists.REPLACE,
|
||||||
if_exists="replace",
|
index=False,
|
||||||
index=False,
|
)
|
||||||
)
|
|
||||||
else:
|
else:
|
||||||
msg = f"Unsupported output format: {output_format}"
|
msg = f"Unsupported output format: {output_format}"
|
||||||
raise ValueError(msg)
|
raise ValueError(msg)
|
||||||
|
|||||||
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
[project]
|
[project]
|
||||||
name = "mt5cli"
|
name = "mt5cli"
|
||||||
version = "0.4.2"
|
version = "0.5.0"
|
||||||
description = "Command-line tool for MetaTrader 5"
|
description = "Command-line tool for MetaTrader 5"
|
||||||
authors = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
|
authors = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
|
||||||
maintainers = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
|
maintainers = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
|
||||||
|
|||||||
+173
-1
@@ -6,7 +6,7 @@ import json
|
|||||||
import logging
|
import logging
|
||||||
import re
|
import re
|
||||||
import sqlite3
|
import sqlite3
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime, timedelta
|
||||||
from typing import TYPE_CHECKING
|
from typing import TYPE_CHECKING
|
||||||
from unittest.mock import MagicMock
|
from unittest.mock import MagicMock
|
||||||
|
|
||||||
@@ -93,6 +93,10 @@ def mock_client(mocker: MockerFixture) -> MagicMock:
|
|||||||
client.market_book_get_as_df.return_value = sample_df
|
client.market_book_get_as_df.return_value = sample_df
|
||||||
client.order_check_as_df.return_value = sample_df
|
client.order_check_as_df.return_value = sample_df
|
||||||
client.order_send_as_df.return_value = sample_df
|
client.order_send_as_df.return_value = sample_df
|
||||||
|
client.version.return_value = (5, 0, 1)
|
||||||
|
client.terminal_info.return_value = {"connected": True, "paths": ["terminal.exe"]}
|
||||||
|
client.account_info.return_value = {"login": 123, "limits": {"modes": ["demo"]}}
|
||||||
|
client.symbols_total.return_value = 42
|
||||||
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=client)
|
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=client)
|
||||||
return client
|
return client
|
||||||
|
|
||||||
@@ -223,6 +227,37 @@ class TestCommands:
|
|||||||
count=50,
|
count=50,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def test_latest_rates(
|
||||||
|
self,
|
||||||
|
tmp_path: Path,
|
||||||
|
mock_client: MagicMock,
|
||||||
|
) -> None:
|
||||||
|
"""Test latest-rates command."""
|
||||||
|
output = tmp_path / "out.csv"
|
||||||
|
result = runner.invoke(
|
||||||
|
app,
|
||||||
|
[
|
||||||
|
"-o",
|
||||||
|
str(output),
|
||||||
|
"latest-rates",
|
||||||
|
"--symbol",
|
||||||
|
"GBPUSD",
|
||||||
|
"--timeframe",
|
||||||
|
"H1",
|
||||||
|
"--count",
|
||||||
|
"50",
|
||||||
|
"--start-pos",
|
||||||
|
"2",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
mock_client.copy_rates_from_pos_as_df.assert_called_once_with(
|
||||||
|
symbol="GBPUSD",
|
||||||
|
timeframe=16385,
|
||||||
|
start_pos=2,
|
||||||
|
count=50,
|
||||||
|
)
|
||||||
|
|
||||||
def test_rates_range(
|
def test_rates_range(
|
||||||
self,
|
self,
|
||||||
tmp_path: Path,
|
tmp_path: Path,
|
||||||
@@ -316,6 +351,65 @@ class TestCommands:
|
|||||||
flags=2,
|
flags=2,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def test_ticks_recent(
|
||||||
|
self,
|
||||||
|
tmp_path: Path,
|
||||||
|
mock_client: MagicMock,
|
||||||
|
) -> None:
|
||||||
|
"""Test ticks-recent command."""
|
||||||
|
output = tmp_path / "out.csv"
|
||||||
|
result = runner.invoke(
|
||||||
|
app,
|
||||||
|
[
|
||||||
|
"-o",
|
||||||
|
str(output),
|
||||||
|
"ticks-recent",
|
||||||
|
"--symbol",
|
||||||
|
"EURUSD",
|
||||||
|
"--seconds",
|
||||||
|
"120",
|
||||||
|
"--date-to",
|
||||||
|
"2024-01-02",
|
||||||
|
"--count",
|
||||||
|
"500",
|
||||||
|
"--flags",
|
||||||
|
"ALL",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
mock_client.copy_ticks_from_as_df.assert_called_once_with(
|
||||||
|
symbol="EURUSD",
|
||||||
|
date_from=datetime(2024, 1, 2, tzinfo=UTC) - timedelta(seconds=120),
|
||||||
|
count=500,
|
||||||
|
flags=1,
|
||||||
|
)
|
||||||
|
mock_client.copy_ticks_range_as_df.assert_not_called()
|
||||||
|
|
||||||
|
def test_minimum_margins(
|
||||||
|
self,
|
||||||
|
tmp_path: Path,
|
||||||
|
mock_client: MagicMock,
|
||||||
|
) -> None:
|
||||||
|
"""Test minimum-margins command."""
|
||||||
|
sym = MagicMock(volume_min=0.01)
|
||||||
|
account = MagicMock(currency="USD")
|
||||||
|
tick = MagicMock(ask=1.1010, bid=1.1000)
|
||||||
|
mock_client.symbol_info.return_value = sym
|
||||||
|
mock_client.account_info.return_value = account
|
||||||
|
mock_client.symbol_info_tick.return_value = tick
|
||||||
|
mock_client.order_calc_margin.side_effect = [12.5, 12.4]
|
||||||
|
mock_client.mt5.ORDER_TYPE_BUY = 0
|
||||||
|
mock_client.mt5.ORDER_TYPE_SELL = 1
|
||||||
|
output = tmp_path / "out.csv"
|
||||||
|
result = runner.invoke(
|
||||||
|
app,
|
||||||
|
["-o", str(output), "minimum-margins", "--symbol", "EURUSD"],
|
||||||
|
)
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
mock_client.symbol_info.assert_called_once_with("EURUSD")
|
||||||
|
mock_client.order_calc_margin.assert_any_call(0, "EURUSD", 0.01, 1.1010)
|
||||||
|
mock_client.order_calc_margin.assert_any_call(1, "EURUSD", 0.01, 1.1000)
|
||||||
|
|
||||||
def test_orders(
|
def test_orders(
|
||||||
self,
|
self,
|
||||||
tmp_path: Path,
|
tmp_path: Path,
|
||||||
@@ -392,6 +486,84 @@ class TestCommands:
|
|||||||
assert result.exit_code == 0, result.output
|
assert result.exit_code == 0, result.output
|
||||||
mock_client.history_deals_get_as_df.assert_called_once()
|
mock_client.history_deals_get_as_df.assert_called_once()
|
||||||
|
|
||||||
|
def test_recent_history_deals(
|
||||||
|
self,
|
||||||
|
tmp_path: Path,
|
||||||
|
mock_client: MagicMock,
|
||||||
|
) -> None:
|
||||||
|
"""Test recent-history-deals command."""
|
||||||
|
output = tmp_path / "out.csv"
|
||||||
|
result = runner.invoke(
|
||||||
|
app,
|
||||||
|
[
|
||||||
|
"-o",
|
||||||
|
str(output),
|
||||||
|
"recent-history-deals",
|
||||||
|
"--hours",
|
||||||
|
"6",
|
||||||
|
"--date-to",
|
||||||
|
"2024-01-02",
|
||||||
|
"--symbol",
|
||||||
|
"EURUSD",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
mock_client.history_deals_get_as_df.assert_called_once_with(
|
||||||
|
date_from=datetime(2024, 1, 1, 18, tzinfo=UTC),
|
||||||
|
date_to=datetime(2024, 1, 2, tzinfo=UTC),
|
||||||
|
group=None,
|
||||||
|
symbol="EURUSD",
|
||||||
|
ticket=None,
|
||||||
|
position=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("filename", "reader"),
|
||||||
|
[
|
||||||
|
("summary.csv", "csv"),
|
||||||
|
("summary.json", "json"),
|
||||||
|
("summary.db", "sqlite3"),
|
||||||
|
("summary.parquet", "parquet"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_mt5_summary_export_formats(
|
||||||
|
self,
|
||||||
|
tmp_path: Path,
|
||||||
|
mock_client: MagicMock,
|
||||||
|
filename: str,
|
||||||
|
reader: str,
|
||||||
|
) -> None:
|
||||||
|
"""Test mt5-summary writes export-safe files for supported formats."""
|
||||||
|
output = tmp_path / filename
|
||||||
|
result = runner.invoke(app, ["-o", str(output), "mt5-summary"])
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
assert output.exists()
|
||||||
|
mock_client.version.assert_called_once()
|
||||||
|
mock_client.terminal_info.assert_called_once()
|
||||||
|
mock_client.account_info.assert_called_once()
|
||||||
|
mock_client.symbols_total.assert_called_once()
|
||||||
|
if reader == "csv":
|
||||||
|
frame = pd.read_csv(output)
|
||||||
|
elif reader == "json":
|
||||||
|
with output.open() as f:
|
||||||
|
records = json.load(f)
|
||||||
|
frame = pd.DataFrame(records)
|
||||||
|
elif reader == "sqlite3":
|
||||||
|
with sqlite3.connect(output) as conn:
|
||||||
|
frame = pd.read_sql( # type: ignore[reportUnknownMemberType]
|
||||||
|
"SELECT * FROM data",
|
||||||
|
conn,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
frame = pd.read_parquet(output)
|
||||||
|
assert len(frame) == 1
|
||||||
|
assert frame.iloc[0].to_dict() == {
|
||||||
|
"version": "[5,0,1]",
|
||||||
|
"terminal_info": '{"connected":true,"paths":["terminal.exe"]}',
|
||||||
|
"account_info": '{"limits":{"modes":["demo"]},"login":123}',
|
||||||
|
"symbols_total": 42,
|
||||||
|
}
|
||||||
|
|
||||||
def test_version(
|
def test_version(
|
||||||
self,
|
self,
|
||||||
tmp_path: Path,
|
tmp_path: Path,
|
||||||
|
|||||||
@@ -31,6 +31,8 @@ from mt5cli.history import (
|
|||||||
get_incremental_start_datetime,
|
get_incremental_start_datetime,
|
||||||
get_table_columns,
|
get_table_columns,
|
||||||
load_incremental_start_datetimes,
|
load_incremental_start_datetimes,
|
||||||
|
load_rate_data,
|
||||||
|
load_rate_data_from_connection,
|
||||||
parse_sqlite_timestamp,
|
parse_sqlite_timestamp,
|
||||||
quote_sqlite_identifier,
|
quote_sqlite_identifier,
|
||||||
record_written_columns,
|
record_written_columns,
|
||||||
@@ -38,6 +40,8 @@ from mt5cli.history import (
|
|||||||
resolve_history_datasets,
|
resolve_history_datasets,
|
||||||
resolve_history_tick_flags,
|
resolve_history_tick_flags,
|
||||||
resolve_history_timeframes,
|
resolve_history_timeframes,
|
||||||
|
resolve_rate_view_name,
|
||||||
|
resolve_rate_view_names,
|
||||||
write_collected_datasets,
|
write_collected_datasets,
|
||||||
write_history_dataset,
|
write_history_dataset,
|
||||||
write_incremental_datasets,
|
write_incremental_datasets,
|
||||||
@@ -47,6 +51,281 @@ from mt5cli.history import (
|
|||||||
from mt5cli.utils import TIMEFRAME_MAP, Dataset, IfExists
|
from mt5cli.utils import TIMEFRAME_MAP, Dataset, IfExists
|
||||||
|
|
||||||
|
|
||||||
|
class TestResolveRateViewName:
|
||||||
|
"""Tests for resolve_rate_view_name and resolve_rate_view_names."""
|
||||||
|
|
||||||
|
def test_missing_database_path_does_not_create_file(self, tmp_path: Path) -> None:
|
||||||
|
"""Test resolving against a missing path does not create a database."""
|
||||||
|
db_path = tmp_path / "missing.db"
|
||||||
|
assert resolve_rate_view_name(db_path, "EURUSD", "M1") == "rate_EURUSD__1"
|
||||||
|
assert not db_path.exists()
|
||||||
|
|
||||||
|
def test_no_rates_table_falls_back_to_single_timeframe_name(
|
||||||
|
self,
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
"""Test databases without a rates table use single-timeframe naming."""
|
||||||
|
db_path = tmp_path / "no-rates.db"
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute("CREATE TABLE ticks(symbol TEXT, time TEXT)")
|
||||||
|
assert resolve_rate_view_name(db_path, "EURUSD", "M1") == "rate_EURUSD__1"
|
||||||
|
|
||||||
|
def test_single_timeframe_for_one_symbol(self, tmp_path: Path) -> None:
|
||||||
|
"""Test one stored timeframe resolves to the short view name."""
|
||||||
|
db_path = tmp_path / "single-timeframe.db"
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute(
|
||||||
|
"CREATE TABLE rates("
|
||||||
|
" symbol TEXT, timeframe INTEGER, time TEXT, close REAL)",
|
||||||
|
)
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO rates(symbol, timeframe, time, close) VALUES (?, ?, ?, ?)",
|
||||||
|
("EURUSD", 1, "2024-01-01T00:00:00+00:00", 1.0),
|
||||||
|
)
|
||||||
|
create_rate_compatibility_views(conn)
|
||||||
|
assert resolve_rate_view_name(db_path, "EURUSD", "M1") == "rate_EURUSD__1"
|
||||||
|
|
||||||
|
def test_multiple_timeframes_for_one_symbol(self, tmp_path: Path) -> None:
|
||||||
|
"""Test multiple stored timeframes resolve to disambiguated view names."""
|
||||||
|
db_path = tmp_path / "multi-timeframe.db"
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute(
|
||||||
|
"CREATE TABLE rates("
|
||||||
|
" symbol TEXT, timeframe INTEGER, time TEXT, close REAL)",
|
||||||
|
)
|
||||||
|
conn.executemany(
|
||||||
|
"INSERT INTO rates(symbol, timeframe, time, close) VALUES (?, ?, ?, ?)",
|
||||||
|
[
|
||||||
|
("EURUSD", 1, "2024-01-01T00:00:00+00:00", 1.0),
|
||||||
|
("EURUSD", TIMEFRAME_MAP["H1"], "2024-01-01T01:00:00+00:00", 1.1),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
create_rate_compatibility_views(conn)
|
||||||
|
assert resolve_rate_view_name(db_path, "EURUSD", "M1") == "rate_EURUSD__M1_1"
|
||||||
|
assert (
|
||||||
|
resolve_rate_view_name(db_path, "EURUSD", "H1") == "rate_EURUSD__H1_16385"
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_prefers_multi_name_when_both_candidate_views_exist(
|
||||||
|
self,
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
"""Test multi-timeframe metadata wins over stale single-timeframe views."""
|
||||||
|
db_path = tmp_path / "stale-and-current-views.db"
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute(
|
||||||
|
"CREATE TABLE rates("
|
||||||
|
" symbol TEXT, timeframe INTEGER, time TEXT, close REAL)",
|
||||||
|
)
|
||||||
|
conn.executemany(
|
||||||
|
"INSERT INTO rates(symbol, timeframe, time, close) VALUES (?, ?, ?, ?)",
|
||||||
|
[
|
||||||
|
("EURUSD", 1, "2024-01-01T00:00:00+00:00", 1.0),
|
||||||
|
("EURUSD", TIMEFRAME_MAP["H1"], "2024-01-01T01:00:00+00:00", 1.1),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
conn.execute(
|
||||||
|
'CREATE VIEW "rate_EURUSD__1" AS'
|
||||||
|
" SELECT time, close FROM rates"
|
||||||
|
" WHERE symbol = 'EURUSD' AND timeframe = 1",
|
||||||
|
)
|
||||||
|
conn.execute(
|
||||||
|
'CREATE VIEW "rate_EURUSD__M1_1" AS'
|
||||||
|
" SELECT time, close FROM rates"
|
||||||
|
" WHERE symbol = 'EURUSD' AND timeframe = 1",
|
||||||
|
)
|
||||||
|
assert resolve_rate_view_name(db_path, "EURUSD", "M1") == "rate_EURUSD__M1_1"
|
||||||
|
|
||||||
|
def test_prefers_existing_view_when_metadata_unavailable(
|
||||||
|
self,
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
"""Test an existing managed view is preferred without rates metadata."""
|
||||||
|
db_path = tmp_path / "view-only.db"
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute("CREATE TABLE ticks(symbol TEXT, time TEXT)")
|
||||||
|
conn.execute('CREATE VIEW "rate_EURUSD__M1_1" AS SELECT 1 AS close')
|
||||||
|
assert resolve_rate_view_name(db_path, "EURUSD", "M1") == "rate_EURUSD__M1_1"
|
||||||
|
|
||||||
|
def test_symbol_absent_from_rates_metadata_uses_candidate_pair(
|
||||||
|
self,
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
"""Test symbols missing from rates metadata still resolve known views."""
|
||||||
|
db_path = tmp_path / "other-symbol-only.db"
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute(
|
||||||
|
"CREATE TABLE rates("
|
||||||
|
" symbol TEXT, timeframe INTEGER, time TEXT, close REAL)",
|
||||||
|
)
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO rates(symbol, timeframe, time, close) VALUES (?, ?, ?, ?)",
|
||||||
|
("GBPUSD", 1, "2024-01-01T00:00:00+00:00", 1.0),
|
||||||
|
)
|
||||||
|
conn.execute(
|
||||||
|
'CREATE VIEW "rate_EURUSD__1" AS'
|
||||||
|
" SELECT time, close FROM rates"
|
||||||
|
" WHERE symbol = 'EURUSD' AND timeframe = 1",
|
||||||
|
)
|
||||||
|
assert resolve_rate_view_name(db_path, "EURUSD", "M1") == "rate_EURUSD__1"
|
||||||
|
|
||||||
|
def test_ignores_non_compatibility_rate_views(self, tmp_path: Path) -> None:
|
||||||
|
"""Test unrelated rate_* views without the __ separator are ignored."""
|
||||||
|
db_path = tmp_path / "summary-view.db"
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute("CREATE TABLE ticks(symbol TEXT, time TEXT)")
|
||||||
|
conn.execute('CREATE VIEW "rate_summary" AS SELECT 1 AS close')
|
||||||
|
assert resolve_rate_view_name(db_path, "EURUSD", "M1") == "rate_EURUSD__1"
|
||||||
|
|
||||||
|
def test_invalid_granularity_propagates_value_error(self, tmp_path: Path) -> None:
|
||||||
|
"""Test invalid granularities raise ValueError from parse_timeframe."""
|
||||||
|
with pytest.raises(ValueError, match="Invalid timeframe"):
|
||||||
|
resolve_rate_view_name(tmp_path / "unused.db", "EURUSD", "BAD")
|
||||||
|
with pytest.raises(ValueError, match="Invalid timeframe"):
|
||||||
|
resolve_rate_view_names(tmp_path / "unused.db", ["EURUSD"], ["BAD"])
|
||||||
|
|
||||||
|
def test_resolve_rate_view_names_for_multiple_pairs(self, tmp_path: Path) -> None:
|
||||||
|
"""Test batch resolution returns row-major symbol/granularity pairs."""
|
||||||
|
db_path = tmp_path / "batch-resolve.db"
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute(
|
||||||
|
"CREATE TABLE rates("
|
||||||
|
" symbol TEXT, timeframe INTEGER, time TEXT, close REAL)",
|
||||||
|
)
|
||||||
|
conn.executemany(
|
||||||
|
"INSERT INTO rates(symbol, timeframe, time, close) VALUES (?, ?, ?, ?)",
|
||||||
|
[
|
||||||
|
("EURUSD", 1, "2024-01-01T00:00:00+00:00", 1.0),
|
||||||
|
("EURUSD", TIMEFRAME_MAP["H1"], "2024-01-01T01:00:00+00:00", 1.1),
|
||||||
|
("GBPUSD", 1, "2024-01-01T00:00:00+00:00", 1.2),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
create_rate_compatibility_views(conn)
|
||||||
|
assert resolve_rate_view_names(
|
||||||
|
db_path,
|
||||||
|
["EURUSD", "GBPUSD"],
|
||||||
|
["M1", "H1"],
|
||||||
|
) == [
|
||||||
|
"rate_EURUSD__M1_1",
|
||||||
|
"rate_EURUSD__H1_16385",
|
||||||
|
"rate_GBPUSD__1",
|
||||||
|
"rate_GBPUSD__16385",
|
||||||
|
]
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"symbol",
|
||||||
|
["EUR/USD", "US500.cash", "#US500"],
|
||||||
|
)
|
||||||
|
def test_supports_broker_specific_symbols(
|
||||||
|
self,
|
||||||
|
tmp_path: Path,
|
||||||
|
symbol: str,
|
||||||
|
) -> None:
|
||||||
|
"""Test broker-specific symbols resolve to safely created view names."""
|
||||||
|
db_path = tmp_path / "broker-symbol-resolve.db"
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute(
|
||||||
|
"CREATE TABLE rates("
|
||||||
|
" symbol TEXT, timeframe INTEGER, time TEXT, close REAL)",
|
||||||
|
)
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO rates(symbol, timeframe, time, close) VALUES (?, ?, ?, ?)",
|
||||||
|
(symbol, 1, "2024-01-01T00:00:00+00:00", 1.0),
|
||||||
|
)
|
||||||
|
create_rate_compatibility_views(conn)
|
||||||
|
assert resolve_rate_view_name(db_path, symbol, "M1") == build_rate_view_name(
|
||||||
|
symbol=symbol,
|
||||||
|
granularity="M1",
|
||||||
|
granularity_count=1,
|
||||||
|
timeframe=1,
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_accepts_open_sqlite_connection(self, tmp_path: Path) -> None:
|
||||||
|
"""Test resolver accepts an already-open SQLite connection."""
|
||||||
|
db_path = tmp_path / "open-connection.db"
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute(
|
||||||
|
"CREATE TABLE rates("
|
||||||
|
" symbol TEXT, timeframe INTEGER, time TEXT, close REAL)",
|
||||||
|
)
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO rates(symbol, timeframe, time, close) VALUES (?, ?, ?, ?)",
|
||||||
|
("EURUSD", 1, "2024-01-01T00:00:00+00:00", 1.0),
|
||||||
|
)
|
||||||
|
create_rate_compatibility_views(conn)
|
||||||
|
assert resolve_rate_view_name(conn, "EURUSD", "M1") == "rate_EURUSD__1"
|
||||||
|
|
||||||
|
def test_require_existing_raises_when_database_missing(
|
||||||
|
self,
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
"""Test strict mode rejects missing database paths."""
|
||||||
|
db_path = tmp_path / "missing.db"
|
||||||
|
with pytest.raises(ValueError, match="SQLite database not found"):
|
||||||
|
resolve_rate_view_name(
|
||||||
|
db_path,
|
||||||
|
"EURUSD",
|
||||||
|
"M1",
|
||||||
|
require_existing=True,
|
||||||
|
)
|
||||||
|
with pytest.raises(ValueError, match="SQLite database not found"):
|
||||||
|
resolve_rate_view_names(
|
||||||
|
db_path,
|
||||||
|
["EURUSD"],
|
||||||
|
["M1"],
|
||||||
|
require_existing=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_require_existing_raises_when_view_missing(self, tmp_path: Path) -> None:
|
||||||
|
"""Test strict mode rejects databases without matching rate views."""
|
||||||
|
db_path = tmp_path / "no-view.db"
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute("CREATE TABLE ticks(symbol TEXT, time TEXT)")
|
||||||
|
with pytest.raises(ValueError, match="No rate compatibility view exists"):
|
||||||
|
resolve_rate_view_name(
|
||||||
|
db_path,
|
||||||
|
"EURUSD",
|
||||||
|
"M1",
|
||||||
|
require_existing=True,
|
||||||
|
)
|
||||||
|
with pytest.raises(ValueError, match="No rate compatibility view exists"):
|
||||||
|
resolve_rate_view_names(
|
||||||
|
db_path,
|
||||||
|
["EURUSD"],
|
||||||
|
["M1"],
|
||||||
|
require_existing=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_require_existing_returns_existing_view(self, tmp_path: Path) -> None:
|
||||||
|
"""Test strict mode returns a view when one exists."""
|
||||||
|
db_path = tmp_path / "existing-view.db"
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute(
|
||||||
|
"CREATE TABLE rates("
|
||||||
|
" symbol TEXT, timeframe INTEGER, time TEXT, close REAL)",
|
||||||
|
)
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO rates(symbol, timeframe, time, close) VALUES (?, ?, ?, ?)",
|
||||||
|
("EURUSD", 1, "2024-01-01T00:00:00+00:00", 1.0),
|
||||||
|
)
|
||||||
|
create_rate_compatibility_views(conn)
|
||||||
|
assert (
|
||||||
|
resolve_rate_view_name(
|
||||||
|
db_path,
|
||||||
|
"EURUSD",
|
||||||
|
"M1",
|
||||||
|
require_existing=True,
|
||||||
|
)
|
||||||
|
== "rate_EURUSD__1"
|
||||||
|
)
|
||||||
|
assert resolve_rate_view_names(
|
||||||
|
db_path,
|
||||||
|
["EURUSD"],
|
||||||
|
["M1"],
|
||||||
|
require_existing=True,
|
||||||
|
) == ["rate_EURUSD__1"]
|
||||||
|
|
||||||
|
|
||||||
class TestQuoteSqliteIdentifier:
|
class TestQuoteSqliteIdentifier:
|
||||||
"""Tests for quote_sqlite_identifier."""
|
"""Tests for quote_sqlite_identifier."""
|
||||||
|
|
||||||
@@ -61,6 +340,147 @@ class TestQuoteSqliteIdentifier:
|
|||||||
assert quoted.endswith('"')
|
assert quoted.endswith('"')
|
||||||
|
|
||||||
|
|
||||||
|
class TestLoadRateData:
|
||||||
|
"""Tests for SQLite rate-like table and view loading."""
|
||||||
|
|
||||||
|
def test_loads_close_rates_from_path_with_count(self, tmp_path: Path) -> None:
|
||||||
|
"""Test loading the latest close-based rates in ascending time order."""
|
||||||
|
db_path = tmp_path / "rates.db"
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute("CREATE TABLE rates(time TEXT, close REAL)")
|
||||||
|
conn.executemany(
|
||||||
|
"INSERT INTO rates(time, close) VALUES (?, ?)",
|
||||||
|
[
|
||||||
|
("2024-01-01T00:00:00+00:00", 1.0),
|
||||||
|
("2024-01-01T00:02:00+00:00", 1.2),
|
||||||
|
("2024-01-01T00:01:00+00:00", 1.1),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
frame = load_rate_data(db_path, "rates", count=2)
|
||||||
|
assert list(frame["close"]) == [1.1, 1.2]
|
||||||
|
assert isinstance(frame.index, pd.DatetimeIndex)
|
||||||
|
assert frame.index.name == "time"
|
||||||
|
assert frame.index.is_monotonic_increasing
|
||||||
|
|
||||||
|
def test_loads_ask_bid_tick_like_rates_from_connection(
|
||||||
|
self,
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
"""Test loading tick-like tables with bid and ask columns."""
|
||||||
|
db_path = tmp_path / "ticks.db"
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute("CREATE TABLE ticks(time TEXT, bid REAL, ask REAL)")
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO ticks(time, bid, ask) VALUES (?, ?, ?)",
|
||||||
|
("2024-01-01T00:00:00+00:00", 1.0, 1.1),
|
||||||
|
)
|
||||||
|
frame = load_rate_data_from_connection(conn, "ticks")
|
||||||
|
path_frame = load_rate_data(conn, "ticks")
|
||||||
|
assert frame.iloc[0].to_dict() == {"bid": 1.0, "ask": 1.1}
|
||||||
|
assert path_frame.iloc[0].to_dict() == {"bid": 1.0, "ask": 1.1}
|
||||||
|
|
||||||
|
def test_loads_from_view(self, tmp_path: Path) -> None:
|
||||||
|
"""Test loading from a SQLite view."""
|
||||||
|
db_path = tmp_path / "view.db"
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute("CREATE TABLE rates(time TEXT, close REAL)")
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO rates(time, close) VALUES (?, ?)",
|
||||||
|
("2024-01-01T00:00:00+00:00", 1.0),
|
||||||
|
)
|
||||||
|
conn.execute("CREATE VIEW rate_view AS SELECT time, close FROM rates")
|
||||||
|
frame = load_rate_data_from_connection(conn, "rate_view")
|
||||||
|
assert list(frame["close"]) == [1.0]
|
||||||
|
|
||||||
|
def test_loads_quoted_identifier(self, tmp_path: Path) -> None:
|
||||||
|
"""Test table names are quoted safely."""
|
||||||
|
db_path = tmp_path / "quoted.db"
|
||||||
|
table = 'rate "quoted"'
|
||||||
|
quoted = quote_sqlite_identifier(table)
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute(f"CREATE TABLE {quoted}(time TEXT, close REAL)")
|
||||||
|
conn.execute(
|
||||||
|
f"INSERT INTO {quoted}(time, close) VALUES (?, ?)", # noqa: S608
|
||||||
|
("2024-01-01T00:00:00+00:00", 1.0),
|
||||||
|
)
|
||||||
|
frame = load_rate_data_from_connection(conn, table)
|
||||||
|
assert list(frame["close"]) == [1.0]
|
||||||
|
|
||||||
|
def test_rejects_missing_database_and_non_file(self, tmp_path: Path) -> None:
|
||||||
|
"""Test path validation for SQLite database inputs."""
|
||||||
|
with pytest.raises(ValueError, match="SQLite database not found"):
|
||||||
|
load_rate_data(tmp_path / "missing.db", "rates")
|
||||||
|
with pytest.raises(ValueError, match="not a file"):
|
||||||
|
load_rate_data(tmp_path, "rates")
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("table", "count", "match"),
|
||||||
|
[
|
||||||
|
("", None, "must not be empty"),
|
||||||
|
("rates", 0, "count must be positive"),
|
||||||
|
("rates", -1, "count must be positive"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_rejects_invalid_inputs(
|
||||||
|
self,
|
||||||
|
tmp_path: Path,
|
||||||
|
table: str,
|
||||||
|
count: int | None,
|
||||||
|
match: str,
|
||||||
|
) -> None:
|
||||||
|
"""Test request validation."""
|
||||||
|
db_path = tmp_path / "invalid-inputs.db"
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute("CREATE TABLE rates(time TEXT, close REAL)")
|
||||||
|
with pytest.raises(ValueError, match=match):
|
||||||
|
load_rate_data_from_connection(conn, table, count=count)
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("ddl", "match"),
|
||||||
|
[
|
||||||
|
("CREATE TABLE rates(time TEXT, close REAL)", "contains no rows"),
|
||||||
|
("CREATE TABLE rates(close REAL)", "time column"),
|
||||||
|
("CREATE TABLE rates(time TEXT, open REAL)", "close, or both ask and bid"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_rejects_invalid_tables(
|
||||||
|
self,
|
||||||
|
tmp_path: Path,
|
||||||
|
ddl: str,
|
||||||
|
match: str,
|
||||||
|
) -> None:
|
||||||
|
"""Test missing table, empty table, and invalid schemas."""
|
||||||
|
db_path = tmp_path / "invalid-tables.db"
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute(ddl)
|
||||||
|
with pytest.raises(ValueError, match=match):
|
||||||
|
load_rate_data_from_connection(conn, "rates")
|
||||||
|
with pytest.raises(ValueError, match="not found"):
|
||||||
|
load_rate_data_from_connection(conn, "missing")
|
||||||
|
|
||||||
|
def test_rejects_invalid_timestamp(self, tmp_path: Path) -> None:
|
||||||
|
"""Test unparsable timestamps fail clearly."""
|
||||||
|
db_path = tmp_path / "invalid-time.db"
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute("CREATE TABLE rates(time TEXT, close REAL)")
|
||||||
|
conn.execute("INSERT INTO rates(time, close) VALUES (?, ?)", ("bad", 1.0))
|
||||||
|
with pytest.raises(ValueError, match="unparsable time"):
|
||||||
|
load_rate_data_from_connection(conn, "rates")
|
||||||
|
|
||||||
|
def test_loads_numeric_mt5_epoch_seconds(self, tmp_path: Path) -> None:
|
||||||
|
"""Test MT5-native integer timestamps are parsed as epoch seconds."""
|
||||||
|
db_path = tmp_path / "epoch-rates.db"
|
||||||
|
with sqlite3.connect(db_path) as conn:
|
||||||
|
conn.execute("CREATE TABLE rates(time INTEGER, close REAL)")
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO rates(time, close) VALUES (?, ?)",
|
||||||
|
(1_704_067_200, 1.0),
|
||||||
|
)
|
||||||
|
frame = load_rate_data_from_connection(conn, "rates")
|
||||||
|
assert frame.index[0] == pd.Timestamp("2024-01-01", tz="UTC")
|
||||||
|
assert list(frame["close"]) == [1.0]
|
||||||
|
|
||||||
|
|
||||||
class TestResolveHistorySettings:
|
class TestResolveHistorySettings:
|
||||||
"""Tests for history dataset and timeframe resolution."""
|
"""Tests for history dataset and timeframe resolution."""
|
||||||
|
|
||||||
|
|||||||
+443
-3
@@ -4,9 +4,9 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import logging
|
import logging
|
||||||
import sqlite3
|
import sqlite3
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime, timedelta
|
||||||
from typing import TYPE_CHECKING
|
from typing import TYPE_CHECKING, NamedTuple, cast
|
||||||
from unittest.mock import MagicMock
|
from unittest.mock import MagicMock, call
|
||||||
|
|
||||||
import pandas as pd
|
import pandas as pd
|
||||||
import pytest
|
import pytest
|
||||||
@@ -15,6 +15,8 @@ from pytest_mock import MockerFixture # noqa: TC002
|
|||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
|
from pdmt5 import Mt5DataClient
|
||||||
|
|
||||||
from mt5cli import sdk
|
from mt5cli import sdk
|
||||||
from mt5cli.history import DEFAULT_HISTORY_TIMEFRAMES
|
from mt5cli.history import DEFAULT_HISTORY_TIMEFRAMES
|
||||||
from mt5cli.sdk import (
|
from mt5cli.sdk import (
|
||||||
@@ -22,6 +24,7 @@ from mt5cli.sdk import (
|
|||||||
account_info,
|
account_info,
|
||||||
build_config,
|
build_config,
|
||||||
collect_history,
|
collect_history,
|
||||||
|
collect_latest_rates,
|
||||||
copy_rates_from,
|
copy_rates_from,
|
||||||
copy_rates_from_pos,
|
copy_rates_from_pos,
|
||||||
copy_rates_range,
|
copy_rates_range,
|
||||||
@@ -30,9 +33,15 @@ from mt5cli.sdk import (
|
|||||||
history_deals,
|
history_deals,
|
||||||
history_orders,
|
history_orders,
|
||||||
last_error,
|
last_error,
|
||||||
|
latest_rates,
|
||||||
market_book,
|
market_book,
|
||||||
|
minimum_margins,
|
||||||
|
mt5_summary,
|
||||||
|
mt5_summary_as_df,
|
||||||
orders,
|
orders,
|
||||||
positions,
|
positions,
|
||||||
|
recent_history_deals,
|
||||||
|
recent_ticks,
|
||||||
symbol_info,
|
symbol_info,
|
||||||
symbol_info_tick,
|
symbol_info_tick,
|
||||||
symbols,
|
symbols,
|
||||||
@@ -43,6 +52,32 @@ from mt5cli.sdk import (
|
|||||||
)
|
)
|
||||||
from mt5cli.utils import Dataset
|
from mt5cli.utils import Dataset
|
||||||
|
|
||||||
|
|
||||||
|
class _TerminalInfo(NamedTuple):
|
||||||
|
connected: bool
|
||||||
|
path: str
|
||||||
|
|
||||||
|
|
||||||
|
class _AccountInfo(NamedTuple):
|
||||||
|
login: int
|
||||||
|
limits: dict[str, object]
|
||||||
|
|
||||||
|
|
||||||
|
class _MissingSummaryMethodClient:
|
||||||
|
def version(self) -> tuple[int, int, int]:
|
||||||
|
return (5, 0, 1)
|
||||||
|
|
||||||
|
def terminal_info(self) -> dict[str, bool]:
|
||||||
|
return {"connected": True}
|
||||||
|
|
||||||
|
def symbols_total(self) -> int:
|
||||||
|
return 42
|
||||||
|
|
||||||
|
|
||||||
|
class _NonCallableSummaryMethodClient:
|
||||||
|
version = (5, 0, 1)
|
||||||
|
|
||||||
|
|
||||||
_DEALS_FIXTURE: dict[str, list[object]] = {
|
_DEALS_FIXTURE: dict[str, list[object]] = {
|
||||||
"ticket": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14],
|
"ticket": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14],
|
||||||
"position_id": [100, 100, 100, 0, 200, 200, 300, 400, 400, 500, 500, 600, 600, 600],
|
"position_id": [100, 100, 100, 0, 200, 200, 300, 400, 400, 500, 500, 600, 600, 600],
|
||||||
@@ -230,6 +265,32 @@ class TestConnectionLifecycle:
|
|||||||
client = Mt5CliClient()
|
client = Mt5CliClient()
|
||||||
client.__exit__(None, None, None)
|
client.__exit__(None, None, None)
|
||||||
|
|
||||||
|
def test_injected_client_is_reused_and_not_shutdown(self) -> None:
|
||||||
|
"""Test injected connected clients are not initialized or shut down."""
|
||||||
|
connected = MagicMock()
|
||||||
|
connected.account_info_as_df.return_value = pd.DataFrame({"a": [1]})
|
||||||
|
connected.terminal_info_as_df.return_value = pd.DataFrame({"b": [2]})
|
||||||
|
with Mt5CliClient.from_connected_client(connected) as client:
|
||||||
|
result = client.account_info()
|
||||||
|
assert result.to_dict("list") == {"a": [1]}
|
||||||
|
connected.initialize_and_login_mt5.assert_not_called()
|
||||||
|
connected.shutdown.assert_not_called()
|
||||||
|
connected.account_info_as_df.assert_called_once()
|
||||||
|
after_exit = client.terminal_info()
|
||||||
|
assert after_exit.to_dict("list") == {"b": [2]}
|
||||||
|
connected.terminal_info_as_df.assert_called_once()
|
||||||
|
|
||||||
|
def test_constructor_injected_client_is_reused_and_not_shutdown(self) -> None:
|
||||||
|
"""Test constructor injection has the same non-owning lifecycle."""
|
||||||
|
connected = MagicMock()
|
||||||
|
connected.terminal_info_as_df.return_value = pd.DataFrame({"b": [2]})
|
||||||
|
client = Mt5CliClient(client=connected)
|
||||||
|
with client:
|
||||||
|
result = client.terminal_info()
|
||||||
|
assert result.to_dict("list") == {"b": [2]}
|
||||||
|
connected.initialize_and_login_mt5.assert_not_called()
|
||||||
|
connected.shutdown.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
class TestModuleFunctions:
|
class TestModuleFunctions:
|
||||||
"""Tests for module-level SDK wrappers."""
|
"""Tests for module-level SDK wrappers."""
|
||||||
@@ -269,6 +330,7 @@ class TestModuleFunctions:
|
|||||||
(last_error, (), "last_error_as_df"),
|
(last_error, (), "last_error_as_df"),
|
||||||
(symbol_info_tick, ("EURUSD",), "symbol_info_tick_as_df"),
|
(symbol_info_tick, ("EURUSD",), "symbol_info_tick_as_df"),
|
||||||
(market_book, ("EURUSD",), "market_book_get_as_df"),
|
(market_book, ("EURUSD",), "market_book_get_as_df"),
|
||||||
|
(latest_rates, ("EURUSD", "M1", 10), "copy_rates_from_pos_as_df"),
|
||||||
],
|
],
|
||||||
)
|
)
|
||||||
def test_module_functions_delegate(
|
def test_module_functions_delegate(
|
||||||
@@ -352,6 +414,214 @@ class TestMt5CliClient:
|
|||||||
assert isinstance(df, pd.DataFrame)
|
assert isinstance(df, pd.DataFrame)
|
||||||
mock_client.copy_rates_range_as_df.assert_called_once()
|
mock_client.copy_rates_range_as_df.assert_called_once()
|
||||||
|
|
||||||
|
def test_latest_rates_delegates_to_copy_rates_from_pos(
|
||||||
|
self,
|
||||||
|
mock_client: MagicMock,
|
||||||
|
) -> None:
|
||||||
|
"""Test latest_rates is a convenience wrapper for positional rates."""
|
||||||
|
Mt5CliClient().latest_rates("EURUSD", "M1", 5, start_pos=2)
|
||||||
|
mock_client.copy_rates_from_pos_as_df.assert_called_once_with(
|
||||||
|
symbol="EURUSD",
|
||||||
|
timeframe=1,
|
||||||
|
start_pos=2,
|
||||||
|
count=5,
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_latest_rates_rejects_non_positive_count(self) -> None:
|
||||||
|
"""Test latest_rates validates count."""
|
||||||
|
with pytest.raises(ValueError, match="count must be positive"):
|
||||||
|
Mt5CliClient().latest_rates("EURUSD", "M1", 0)
|
||||||
|
|
||||||
|
def test_collect_latest_rates_returns_mapping(
|
||||||
|
self,
|
||||||
|
mock_client: MagicMock,
|
||||||
|
) -> None:
|
||||||
|
"""Test multi-target latest rate collection."""
|
||||||
|
result = collect_latest_rates(["EURUSD", "GBPUSD"], ["M1", "H1"], count=3)
|
||||||
|
assert set(result) == {
|
||||||
|
("EURUSD", 1),
|
||||||
|
("EURUSD", 16385),
|
||||||
|
("GBPUSD", 1),
|
||||||
|
("GBPUSD", 16385),
|
||||||
|
}
|
||||||
|
assert mock_client.copy_rates_from_pos_as_df.call_count == 4
|
||||||
|
|
||||||
|
def test_collect_latest_rates_uses_single_transient_connection(
|
||||||
|
self,
|
||||||
|
mock_client: MagicMock,
|
||||||
|
mocker: MockerFixture,
|
||||||
|
) -> None:
|
||||||
|
"""Test module helper opens one connection for all target pairs."""
|
||||||
|
mt5_data_client = mocker.patch(
|
||||||
|
"mt5cli.sdk.Mt5DataClient",
|
||||||
|
return_value=mock_client,
|
||||||
|
)
|
||||||
|
|
||||||
|
collect_latest_rates(["EURUSD", "GBPUSD"], ["M1", "H1"], count=3)
|
||||||
|
|
||||||
|
mt5_data_client.assert_called_once()
|
||||||
|
mock_client.initialize_and_login_mt5.assert_called_once()
|
||||||
|
mock_client.shutdown.assert_called_once()
|
||||||
|
assert mock_client.copy_rates_from_pos_as_df.call_count == 4
|
||||||
|
mock_client.copy_rates_from_pos_as_df.assert_has_calls(
|
||||||
|
[
|
||||||
|
call(symbol="EURUSD", timeframe=1, start_pos=0, count=3),
|
||||||
|
call(symbol="EURUSD", timeframe=16385, start_pos=0, count=3),
|
||||||
|
call(symbol="GBPUSD", timeframe=1, start_pos=0, count=3),
|
||||||
|
call(symbol="GBPUSD", timeframe=16385, start_pos=0, count=3),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("symbols", "timeframes", "match"),
|
||||||
|
[
|
||||||
|
([], ["M1"], "At least one symbol"),
|
||||||
|
(["EURUSD"], [], "At least one timeframe"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_collect_latest_rates_rejects_empty_inputs(
|
||||||
|
self,
|
||||||
|
symbols: list[str],
|
||||||
|
timeframes: list[str],
|
||||||
|
match: str,
|
||||||
|
) -> None:
|
||||||
|
"""Test multi-target latest rate input validation."""
|
||||||
|
with pytest.raises(ValueError, match=match):
|
||||||
|
Mt5CliClient().collect_latest_rates(symbols, timeframes, count=1)
|
||||||
|
|
||||||
|
def test_recent_history_deals_uses_trailing_window(
|
||||||
|
self,
|
||||||
|
mock_client: MagicMock,
|
||||||
|
) -> None:
|
||||||
|
"""Test recent_history_deals calculates date_from from hours."""
|
||||||
|
result = recent_history_deals(
|
||||||
|
6,
|
||||||
|
date_to="2024-01-02T00:00:00+00:00",
|
||||||
|
group="*",
|
||||||
|
symbol="EURUSD",
|
||||||
|
)
|
||||||
|
assert isinstance(result, pd.DataFrame)
|
||||||
|
mock_client.history_deals_get_as_df.assert_called_once_with(
|
||||||
|
date_from=datetime(2024, 1, 1, 18, tzinfo=UTC),
|
||||||
|
date_to=datetime(2024, 1, 2, tzinfo=UTC),
|
||||||
|
group="*",
|
||||||
|
symbol="EURUSD",
|
||||||
|
ticket=None,
|
||||||
|
position=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_recent_history_deals_defaults_date_to_now(
|
||||||
|
self,
|
||||||
|
mock_client: MagicMock,
|
||||||
|
) -> None:
|
||||||
|
"""Test recent_history_deals uses current UTC time when date_to is omitted."""
|
||||||
|
before = datetime.now(UTC)
|
||||||
|
recent_history_deals(1.0)
|
||||||
|
after = datetime.now(UTC)
|
||||||
|
call_kwargs = mock_client.history_deals_get_as_df.call_args.kwargs
|
||||||
|
assert before <= call_kwargs["date_to"] <= after
|
||||||
|
assert call_kwargs["date_from"] == call_kwargs["date_to"] - timedelta(hours=1)
|
||||||
|
|
||||||
|
def test_recent_history_deals_rejects_non_positive_hours(self) -> None:
|
||||||
|
"""Test recent_history_deals validates hours."""
|
||||||
|
with pytest.raises(ValueError, match="hours must be positive"):
|
||||||
|
Mt5CliClient().recent_history_deals(0)
|
||||||
|
|
||||||
|
def test_mt5_summary_returns_status_mapping(
|
||||||
|
self,
|
||||||
|
mock_client: MagicMock,
|
||||||
|
) -> None:
|
||||||
|
"""Test mt5_summary calls raw terminal/account status methods."""
|
||||||
|
mock_client.version.return_value = (5, 0, 1)
|
||||||
|
mock_client.terminal_info.return_value = {"connected": True}
|
||||||
|
mock_client.account_info.return_value = {"login": 123}
|
||||||
|
mock_client.symbols_total.return_value = 42
|
||||||
|
assert mt5_summary() == {
|
||||||
|
"version": [5, 0, 1],
|
||||||
|
"terminal_info": {"connected": True},
|
||||||
|
"account_info": {"login": 123},
|
||||||
|
"symbols_total": 42,
|
||||||
|
}
|
||||||
|
|
||||||
|
def test_mt5_summary_normalizes_namedtuple_values(
|
||||||
|
self,
|
||||||
|
mock_client: MagicMock,
|
||||||
|
) -> None:
|
||||||
|
"""Test mt5_summary returns structured plain Python values."""
|
||||||
|
mock_client.version.return_value = (5, 0, 1)
|
||||||
|
mock_client.terminal_info.return_value = _TerminalInfo(
|
||||||
|
connected=True,
|
||||||
|
path="terminal.exe",
|
||||||
|
)
|
||||||
|
mock_client.account_info.return_value = _AccountInfo(
|
||||||
|
login=123,
|
||||||
|
limits={"modes": ("netting", "hedging"), "servers": ["demo"]},
|
||||||
|
)
|
||||||
|
mock_client.symbols_total.return_value = 42
|
||||||
|
|
||||||
|
assert mt5_summary() == {
|
||||||
|
"version": [5, 0, 1],
|
||||||
|
"terminal_info": {"connected": True, "path": "terminal.exe"},
|
||||||
|
"account_info": {
|
||||||
|
"login": 123,
|
||||||
|
"limits": {"modes": ["netting", "hedging"], "servers": ["demo"]},
|
||||||
|
},
|
||||||
|
"symbols_total": 42,
|
||||||
|
}
|
||||||
|
|
||||||
|
def test_mt5_summary_as_df_stringifies_nested_values(
|
||||||
|
self,
|
||||||
|
mock_client: MagicMock,
|
||||||
|
) -> None:
|
||||||
|
"""Test mt5_summary_as_df returns export-safe tabular values."""
|
||||||
|
mock_client.version.return_value = (5, 0, 1)
|
||||||
|
mock_client.terminal_info.return_value = _TerminalInfo(
|
||||||
|
connected=True,
|
||||||
|
path="terminal.exe",
|
||||||
|
)
|
||||||
|
mock_client.account_info.return_value = _AccountInfo(
|
||||||
|
login=123,
|
||||||
|
limits={"modes": ("netting", "hedging"), "servers": ["demo"]},
|
||||||
|
)
|
||||||
|
mock_client.symbols_total.return_value = 42
|
||||||
|
|
||||||
|
result = mt5_summary_as_df()
|
||||||
|
|
||||||
|
assert len(result) == 1
|
||||||
|
assert result.iloc[0].to_dict() == {
|
||||||
|
"version": "[5,0,1]",
|
||||||
|
"terminal_info": '{"connected":true,"path":"terminal.exe"}',
|
||||||
|
"account_info": (
|
||||||
|
'{"limits":{"modes":["netting","hedging"],'
|
||||||
|
'"servers":["demo"]},"login":123}'
|
||||||
|
),
|
||||||
|
"symbols_total": 42,
|
||||||
|
}
|
||||||
|
|
||||||
|
def test_mt5_summary_missing_method_raises_clear_error(self) -> None:
|
||||||
|
"""Test mt5_summary fails clearly when a required method is missing."""
|
||||||
|
client = Mt5CliClient(
|
||||||
|
client=cast("Mt5DataClient", _MissingSummaryMethodClient()),
|
||||||
|
)
|
||||||
|
|
||||||
|
with pytest.raises(
|
||||||
|
AttributeError,
|
||||||
|
match="MT5 client is missing required method: account_info",
|
||||||
|
):
|
||||||
|
client.mt5_summary()
|
||||||
|
|
||||||
|
def test_mt5_summary_non_callable_method_raises_clear_error(self) -> None:
|
||||||
|
"""Test mt5_summary fails clearly when a required method is not callable."""
|
||||||
|
client = Mt5CliClient(
|
||||||
|
client=cast("Mt5DataClient", _NonCallableSummaryMethodClient()),
|
||||||
|
)
|
||||||
|
|
||||||
|
with pytest.raises(
|
||||||
|
TypeError,
|
||||||
|
match="MT5 client attribute is not callable: version",
|
||||||
|
):
|
||||||
|
client.mt5_summary()
|
||||||
|
|
||||||
|
|
||||||
class TestCollectHistory:
|
class TestCollectHistory:
|
||||||
"""Tests for collect_history SDK function."""
|
"""Tests for collect_history SDK function."""
|
||||||
@@ -808,3 +1078,173 @@ class TestUpdateHistory:
|
|||||||
)
|
)
|
||||||
after = datetime.now(UTC)
|
after = datetime.now(UTC)
|
||||||
assert before <= captured["end"] <= after
|
assert before <= captured["end"] <= after
|
||||||
|
|
||||||
|
|
||||||
|
class TestRecentTicks:
|
||||||
|
"""Tests for recent_ticks helper."""
|
||||||
|
|
||||||
|
def test_recent_ticks_uses_explicit_date_to_window(
|
||||||
|
self,
|
||||||
|
mocker: MockerFixture,
|
||||||
|
) -> None:
|
||||||
|
"""Test recent_ticks fetches the requested trailing window."""
|
||||||
|
client = MagicMock()
|
||||||
|
end = datetime(2024, 1, 2, 12, 0, 0, tzinfo=UTC)
|
||||||
|
client.copy_ticks_from_as_df.return_value = pd.DataFrame({
|
||||||
|
"time": [end],
|
||||||
|
"bid": [1.0],
|
||||||
|
})
|
||||||
|
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=client)
|
||||||
|
result = recent_ticks(
|
||||||
|
"EURUSD",
|
||||||
|
60,
|
||||||
|
date_to=end,
|
||||||
|
count=100,
|
||||||
|
flags="INFO",
|
||||||
|
config=build_config(login=123),
|
||||||
|
)
|
||||||
|
assert isinstance(result, pd.DataFrame)
|
||||||
|
client.copy_ticks_from_as_df.assert_called_once_with(
|
||||||
|
symbol="EURUSD",
|
||||||
|
date_from=end - timedelta(seconds=60),
|
||||||
|
count=100,
|
||||||
|
flags=2,
|
||||||
|
)
|
||||||
|
client.copy_ticks_range_as_df.assert_not_called()
|
||||||
|
|
||||||
|
def test_recent_ticks_uses_latest_tick_when_date_to_omitted(
|
||||||
|
self,
|
||||||
|
mocker: MockerFixture,
|
||||||
|
) -> None:
|
||||||
|
"""Test recent_ticks anchors the window on the latest tick time."""
|
||||||
|
client = MagicMock()
|
||||||
|
tick = MagicMock()
|
||||||
|
tick.time = datetime(2024, 1, 2, 12, 0, 0, tzinfo=UTC)
|
||||||
|
client.symbol_info_tick.return_value = tick
|
||||||
|
client.copy_ticks_from_as_df.return_value = pd.DataFrame({
|
||||||
|
"time": [1, 2],
|
||||||
|
"bid": [1.0, 1.1],
|
||||||
|
})
|
||||||
|
client.copy_ticks_range_as_df.return_value = pd.DataFrame({
|
||||||
|
"time": [1, 2, 3],
|
||||||
|
"bid": [1.0, 1.1, 1.2],
|
||||||
|
})
|
||||||
|
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=client)
|
||||||
|
result = Mt5CliClient().recent_ticks("EURUSD", 30, count=2, flags="ALL")
|
||||||
|
assert len(result) == 2
|
||||||
|
client.symbol_info_tick.assert_called_once_with("EURUSD")
|
||||||
|
client.copy_ticks_from_as_df.assert_called_once()
|
||||||
|
_, kwargs = client.copy_ticks_range_as_df.call_args
|
||||||
|
assert kwargs["symbol"] == "EURUSD"
|
||||||
|
assert kwargs["date_to"] == tick.time
|
||||||
|
assert kwargs["date_from"] == tick.time - timedelta(seconds=30)
|
||||||
|
assert kwargs["flags"] == 1
|
||||||
|
|
||||||
|
def test_recent_ticks_rejects_unsupported_tick_time(
|
||||||
|
self,
|
||||||
|
mocker: MockerFixture,
|
||||||
|
) -> None:
|
||||||
|
"""Test recent_ticks raises when the latest tick time is unsupported."""
|
||||||
|
client = MagicMock()
|
||||||
|
tick = MagicMock()
|
||||||
|
tick.time = object()
|
||||||
|
client.symbol_info_tick.return_value = tick
|
||||||
|
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=client)
|
||||||
|
with pytest.raises(TypeError, match="Unsupported tick time value"):
|
||||||
|
Mt5CliClient().recent_ticks("EURUSD", 30)
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"tick_time",
|
||||||
|
[
|
||||||
|
"2024-01-02T12:00:00+00:00",
|
||||||
|
1704196800,
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_recent_ticks_coerces_string_and_unix_tick_times(
|
||||||
|
self,
|
||||||
|
mocker: MockerFixture,
|
||||||
|
tick_time: str | int,
|
||||||
|
) -> None:
|
||||||
|
"""Test recent_ticks accepts string and unix tick timestamps."""
|
||||||
|
client = MagicMock()
|
||||||
|
tick = MagicMock()
|
||||||
|
tick.time = tick_time
|
||||||
|
client.symbol_info_tick.return_value = tick
|
||||||
|
expected_end = (
|
||||||
|
datetime(2024, 1, 2, 12, 0, 0, tzinfo=UTC)
|
||||||
|
if isinstance(tick_time, str)
|
||||||
|
else datetime.fromtimestamp(tick_time, tz=UTC)
|
||||||
|
)
|
||||||
|
client.copy_ticks_from_as_df.return_value = pd.DataFrame({
|
||||||
|
"time": [expected_end],
|
||||||
|
})
|
||||||
|
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=client)
|
||||||
|
Mt5CliClient().recent_ticks("EURUSD", 30)
|
||||||
|
_, kwargs = client.copy_ticks_from_as_df.call_args
|
||||||
|
assert kwargs["date_from"] == expected_end - timedelta(seconds=30)
|
||||||
|
|
||||||
|
def test_recent_ticks_returns_full_frame_when_count_not_positive(
|
||||||
|
self,
|
||||||
|
mocker: MockerFixture,
|
||||||
|
) -> None:
|
||||||
|
"""Test non-positive count returns the full range without trimming."""
|
||||||
|
client = MagicMock()
|
||||||
|
end = datetime(2024, 1, 2, 12, 0, 0, tzinfo=UTC)
|
||||||
|
client.copy_ticks_range_as_df.return_value = pd.DataFrame({
|
||||||
|
"time": [1, 2, 3],
|
||||||
|
"bid": [1.0, 1.1, 1.2],
|
||||||
|
})
|
||||||
|
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=client)
|
||||||
|
result = recent_ticks(
|
||||||
|
"EURUSD",
|
||||||
|
60,
|
||||||
|
date_to=end,
|
||||||
|
count=0,
|
||||||
|
config=build_config(login=123),
|
||||||
|
)
|
||||||
|
assert len(result) == 3
|
||||||
|
client.copy_ticks_from_as_df.assert_not_called()
|
||||||
|
client.copy_ticks_range_as_df.assert_called_once_with(
|
||||||
|
symbol="EURUSD",
|
||||||
|
date_from=end - timedelta(seconds=60),
|
||||||
|
date_to=end,
|
||||||
|
flags=1,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class TestMinimumMargins:
|
||||||
|
"""Tests for minimum_margins helper."""
|
||||||
|
|
||||||
|
def test_minimum_margins_shape(
|
||||||
|
self,
|
||||||
|
mocker: MockerFixture,
|
||||||
|
) -> None:
|
||||||
|
"""Test minimum_margins returns the expected summary columns."""
|
||||||
|
client = MagicMock()
|
||||||
|
sym = MagicMock(volume_min=0.01)
|
||||||
|
account = MagicMock(currency="USD")
|
||||||
|
tick = MagicMock(ask=1.1010, bid=1.1000)
|
||||||
|
client.symbol_info.return_value = sym
|
||||||
|
client.account_info.return_value = account
|
||||||
|
client.symbol_info_tick.return_value = tick
|
||||||
|
client.order_calc_margin.side_effect = [12.5, 12.4]
|
||||||
|
client.mt5.ORDER_TYPE_BUY = 0
|
||||||
|
client.mt5.ORDER_TYPE_SELL = 1
|
||||||
|
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=client)
|
||||||
|
|
||||||
|
result = minimum_margins("EURUSD", config=build_config(login=123))
|
||||||
|
|
||||||
|
pd.testing.assert_frame_equal(
|
||||||
|
result,
|
||||||
|
pd.DataFrame([
|
||||||
|
{
|
||||||
|
"symbol": "EURUSD",
|
||||||
|
"account_currency": "USD",
|
||||||
|
"volume_min": 0.01,
|
||||||
|
"buy_margin": 12.5,
|
||||||
|
"sell_margin": 12.4,
|
||||||
|
}
|
||||||
|
]),
|
||||||
|
)
|
||||||
|
client.order_calc_margin.assert_any_call(0, "EURUSD", 0.01, 1.1010)
|
||||||
|
client.order_calc_margin.assert_any_call(1, "EURUSD", 0.01, 1.1000)
|
||||||
|
|||||||
@@ -21,8 +21,10 @@ from mt5cli.utils import (
|
|||||||
TIMEFRAME_MAP,
|
TIMEFRAME_MAP,
|
||||||
TIMEFRAME_TYPE,
|
TIMEFRAME_TYPE,
|
||||||
Dataset,
|
Dataset,
|
||||||
|
IfExists,
|
||||||
detect_format,
|
detect_format,
|
||||||
export_dataframe,
|
export_dataframe,
|
||||||
|
export_dataframe_to_sqlite,
|
||||||
parse_datetime,
|
parse_datetime,
|
||||||
parse_request,
|
parse_request,
|
||||||
parse_tick_flags,
|
parse_tick_flags,
|
||||||
@@ -130,6 +132,112 @@ class TestExportDataframe:
|
|||||||
export_dataframe(sample_df, tmp_path / "out.txt", "xml")
|
export_dataframe(sample_df, tmp_path / "out.txt", "xml")
|
||||||
|
|
||||||
|
|
||||||
|
class TestExportDataframeToSqlite:
|
||||||
|
"""Tests for export_dataframe_to_sqlite."""
|
||||||
|
|
||||||
|
def test_append_preserves_existing_rows(self, tmp_path: Path) -> None:
|
||||||
|
"""Test append mode keeps prior rows in the SQLite table."""
|
||||||
|
output = tmp_path / "append.db"
|
||||||
|
first = pd.DataFrame({"id": [1], "value": ["a"]})
|
||||||
|
second = pd.DataFrame({"id": [2], "value": ["b"]})
|
||||||
|
export_dataframe_to_sqlite(first, output, "items", if_exists=IfExists.REPLACE)
|
||||||
|
export_dataframe_to_sqlite(second, output, "items", if_exists=IfExists.APPEND)
|
||||||
|
with sqlite3.connect(output) as conn:
|
||||||
|
result = pd.read_sql( # type: ignore[reportUnknownMemberType]
|
||||||
|
"SELECT id, value FROM items ORDER BY id",
|
||||||
|
conn,
|
||||||
|
)
|
||||||
|
pd.testing.assert_frame_equal(
|
||||||
|
result,
|
||||||
|
pd.DataFrame({"id": [1, 2], "value": ["a", "b"]}),
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_deduplicate_keeps_latest_row(self, tmp_path: Path) -> None:
|
||||||
|
"""Test deduplication keeps the latest ROWID for key columns."""
|
||||||
|
output = tmp_path / "dedup.db"
|
||||||
|
first = pd.DataFrame({
|
||||||
|
"symbol": ["EURUSD", "EURUSD"],
|
||||||
|
"time": ["2024-01-01", "2024-01-01"],
|
||||||
|
"bid": [1.0, 1.1],
|
||||||
|
})
|
||||||
|
second = pd.DataFrame({
|
||||||
|
"symbol": ["EURUSD"],
|
||||||
|
"time": ["2024-01-01"],
|
||||||
|
"bid": [1.2],
|
||||||
|
})
|
||||||
|
export_dataframe_to_sqlite(
|
||||||
|
first,
|
||||||
|
output,
|
||||||
|
"ticks",
|
||||||
|
if_exists=IfExists.REPLACE,
|
||||||
|
deduplicate_on=("symbol", "time"),
|
||||||
|
)
|
||||||
|
export_dataframe_to_sqlite(
|
||||||
|
second,
|
||||||
|
output,
|
||||||
|
"ticks",
|
||||||
|
if_exists=IfExists.APPEND,
|
||||||
|
deduplicate_on=("symbol", "time"),
|
||||||
|
)
|
||||||
|
with sqlite3.connect(output) as conn:
|
||||||
|
result = pd.read_sql( # type: ignore[reportUnknownMemberType]
|
||||||
|
"SELECT symbol, time, bid FROM ticks",
|
||||||
|
conn,
|
||||||
|
)
|
||||||
|
pd.testing.assert_frame_equal(
|
||||||
|
result.reset_index(drop=True),
|
||||||
|
pd.DataFrame({
|
||||||
|
"symbol": ["EURUSD"],
|
||||||
|
"time": ["2024-01-01"],
|
||||||
|
"bid": [1.2],
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_default_if_exists_appends_without_dropping_rows(
|
||||||
|
self,
|
||||||
|
tmp_path: Path,
|
||||||
|
) -> None:
|
||||||
|
"""Test the default append mode keeps prior rows."""
|
||||||
|
output = tmp_path / "default-append.db"
|
||||||
|
first = pd.DataFrame({"id": [1], "value": ["a"]})
|
||||||
|
second = pd.DataFrame({"id": [2], "value": ["b"]})
|
||||||
|
export_dataframe_to_sqlite(first, output, "items")
|
||||||
|
export_dataframe_to_sqlite(second, output, "items")
|
||||||
|
with sqlite3.connect(output) as conn:
|
||||||
|
result = pd.read_sql( # type: ignore[reportUnknownMemberType]
|
||||||
|
"SELECT id, value FROM items ORDER BY id",
|
||||||
|
conn,
|
||||||
|
)
|
||||||
|
pd.testing.assert_frame_equal(
|
||||||
|
result,
|
||||||
|
pd.DataFrame({"id": [1, 2], "value": ["a", "b"]}),
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_writes_index_with_label(self, tmp_path: Path) -> None:
|
||||||
|
"""Test optional index export with a custom label."""
|
||||||
|
output = tmp_path / "index.db"
|
||||||
|
frame = pd.DataFrame(
|
||||||
|
{"value": [1.0]}, index=pd.Index(["EURUSD"], name="symbol")
|
||||||
|
)
|
||||||
|
export_dataframe_to_sqlite(
|
||||||
|
frame,
|
||||||
|
output,
|
||||||
|
"margins",
|
||||||
|
if_exists=IfExists.REPLACE,
|
||||||
|
index=True,
|
||||||
|
index_label="symbol",
|
||||||
|
)
|
||||||
|
with sqlite3.connect(output) as conn:
|
||||||
|
result = pd.read_sql( # type: ignore[reportUnknownMemberType]
|
||||||
|
"SELECT symbol, value FROM margins",
|
||||||
|
conn,
|
||||||
|
)
|
||||||
|
pd.testing.assert_frame_equal(
|
||||||
|
result,
|
||||||
|
pd.DataFrame({"symbol": ["EURUSD"], "value": [1.0]}),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Parse helpers
|
# Parse helpers
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|||||||
Reference in New Issue
Block a user