Compare commits

...

7 Commits

Author SHA1 Message Date
Daichi Narushima d156dd7176 [codex] fix mt5 adapter APIs (#36)
* fix mt5 adapter APIs

* address PR feedback

* fix zero ratio minimum volume sizing

* Bump version to v0.8.0
2026-06-15 02:47:05 +09:00
dceoy 307d6f5320 docs: restructure AGENTS.md with concise repository guidance
Align agent instructions with the streamlined project structure, QA workflow, and security notes used elsewhere in the repo.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-14 23:07:13 +09:00
Daichi Narushima 8031389a67 Add GitHub CodeQL analysis to CI workflow (#35)
* chore: add GitHub CodeQL analysis to CI workflow

Enable automated security scanning with GitHub CodeQL to detect potential vulnerabilities in Python code.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

* chore: run CodeQL analysis on pull requests

Co-authored-by: Cursor <cursoragent@cursor.com>

* Add checks and statuses read permissions for dependabot auto-merge.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-14 22:54:28 +09:00
dceoy fdf5e08d31 Add .agents/skills/pr-feedback-triage/SKILL.md 2026-06-14 21:16:52 +09:00
dceoy 254c159ad5 Bump version to v0.7.2 2026-06-13 01:34:03 +09:00
Daichi Narushima 78c49238cf feat: stable MT5Client public API and infrastructure layer (#30)
* feat: add stable MT5Client public API and infrastructure layer

Introduce a reusable public API for downstream trading applications:

- MT5Client as the primary client abstraction with order_check/order_send
- schemas module with DataKind contracts, validation, and normalization
- converters, exceptions, retry, and storage facade modules
- CLI order commands now route through MT5Client
- connected_client made public; retry logic centralized
- Contract tests for API surface, schemas, and storage round-trips
- README and docs updated with Python API usage examples

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* fix: correct time coercion, broker-safe symbols, and execution docs

- Normalize MT5 time columns with correct second/millisecond units
- Coerce all present known MT5 time fields, including optional order times
- Preserve broker symbol casing in normalize_symbol()
- Document order_send() as a live execution primitive with clear scope boundaries
- Add contract tests for timestamp and symbol normalization behavior

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
2026-06-13 01:32:03 +09:00
Daichi Narushima 9356d5dcdf Consolidate duplicated export and history streaming helpers (#29)
* Consolidate duplicated export and history streaming helpers.

Reduce repeated CLI export plumbing, shared per-symbol SQLite writes, and test mock setup without changing public behavior.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Bump version from 0.7.0 to 0.7.1.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-12 23:14:34 +09:00
35 changed files with 3896 additions and 529 deletions
+149
View File
@@ -0,0 +1,149 @@
---
name: pr-feedback-triage
description: Triage pull request review comments into fixes, replies, clarification requests, or open follow-ups while respecting safe execution modes.
---
# PR Feedback Triage
Triage pull request review feedback, decide what action each thread needs, make focused fixes when allowed, and report or resolve only what is actually handled.
## When to Use
- A PR has review comments, requested changes, unresolved review threads, or bot review findings.
- The user asks to address, respond to, or resolve PR feedback.
- The user provides a PR URL/number, a branch with an associated PR, or copied comments.
Do not use this skill for a first-pass code review with no existing feedback; use a code review skill instead.
## Inputs
- Pull request URL or number, or a current branch that has an associated pull request.
- Repository checkout or platform access sufficient to inspect the PR diff and review feedback.
- Optional reviewer priorities from the user, such as "only address blocking comments" or "do not reply on the PR platform".
- Optional operating mode flags: `dry_run`, `no_push`, and `no_reply`.
If no PR or review comments are identifiable, ask for the target PR or the copied comments before proceeding.
## Modes
- `dry_run`: inspect review feedback and report the triage only. Do not edit files, run write-mode formatters, commit, push, post replies, or resolve review threads.
- `no_push`: local edits and verification are allowed, but do not push commits or otherwise update the remote branch. Report the local diff or local commits that still need to be pushed.
- `no_reply`: do not post replies, submit reviews, or resolve review threads. Provide suggested replies and resolution actions in the final report instead.
When a mode disables an action, skip that destructive or externally visible action even if normal workflow text would otherwise allow it.
## Preflight
1. Identify the current branch and target PR.
2. Check tracked local changes with `git diff --name-only` and `git diff --cached --name-only`. Ignore untracked files unless the review feedback explicitly concerns them.
3. Check unpushed commits before relying on remote review feedback.
4. If tracked local changes or unpushed commits exist, warn that existing PR comments may not cover the latest local state. In `normal` mode, push only when the user request or repository workflow allows it; otherwise continue with a clearly reported limitation.
## Feedback Collection
Gather the complete feedback set before editing:
- Fetch unresolved review threads, requested-change reviews, PR-level summary comments, and copied comments.
- Use platform-native APIs/CLI when available. Paginate results; do not inspect only the first page of threads or comments.
- For bot reviewers that post both summary comments and inline comments, collect both. Summary comments often contain severity, rationale, and fix instructions; inline comments contain the exact file and line context.
- Preserve each thread/comment identifier needed to reply or resolve later.
- Compare each comment with the current diff and file contents because review lines can become outdated.
## Deduplication and Ordering
Build one triage record per distinct finding:
- Prefer exact review-thread identity when available.
- For duplicate bot findings appearing in both summary and inline comments, merge by exact issue title first, then by file path plus line range as a fallback.
- Prefer inline comments for location and current code context.
- Prefer summary comments for severity, category, rationale, and detailed agent prompts.
- Preserve the reviewers exact issue title and original wording where practical. Do not rename findings in a way that would make replies hard to map back to comments.
- Preserve the reviewers original ordering unless the user asks for priority reordering. Many review bots already order findings by severity.
Each triage record should track: original title, reviewer, source IDs, location, current applicability, severity/priority if available, disposition, planned action, verification, reply text if any, and resolution decision.
## Resolution Policy
In normal mode, `Resolve conversation` is the default action for any review thread that has been fully handled. A thread is handled when the requested change is implemented and verified, the current code already satisfies the comment, the comment is outdated and no longer applies, or a deliberate deferral/won't-fix response has been posted with a clear reason.
Keep a thread open only when it still needs reviewer, maintainer, or product input, the fix is local-only and not pushed, verification is missing for a material change, or the user explicitly requested `dry_run`, `no_push`, or `no_reply` behavior that prevents resolution.
When resolving a thread, add a concise reply first only if it provides useful context, such as what changed, why no code change was needed, why a finding was intentionally deferred, or why the original comment is now outdated. Do not add noisy replies for self-evident fixes unless project norms require them.
## Flow
```mermaid
flowchart TD
A[Identify PR and branch state] --> B[Collect all review feedback]
B --> C[Deduplicate and preserve source IDs]
C --> D[Inspect current diff and code]
D --> E{Classify each triage record}
E -->|Fix| F[Implement minimal change]
E -->|Answer| G[Prepare concise reply]
E -->|Clarify| H[Leave open with question]
E -->|Already addressed or Outdated| I[Prepare evidence]
E -->|Defer or Won't fix| J[Document reason]
F --> K[Verify]
G --> L{Mode}
H --> L
I --> L
J --> L
K --> L
L -->|dry_run| M[Report triage only]
L -->|no_push| N[Report local diff or commits]
L -->|no_reply| O[Report suggested replies/actions]
L -->|normal| P[Commit/push if changed, then batch reply and resolve handled threads]
M --> Q[Final summary]
N --> Q
O --> Q
P --> Q
```
## Compact Workflow
1. **Collect all relevant feedback**
- Identify the PR and gather unresolved review threads, requested-change reviews, PR-level summaries, inline comments, and copied comments.
- Paginate all platform calls and keep comment/thread IDs for later replies and resolution.
- For bot reviews, collect both summary and inline comments, then merge duplicates rather than fixing the same finding twice.
2. **Classify each triage record**
- **Fix**: Valid requested change; make the smallest focused edit when not in `dry_run`.
- **Answer**: No code change needed; prepare a concise explanation.
- **Clarify**: Ambiguous, conflicting, or missing context; leave open with a question.
- **Already addressed**: Current code already satisfies it; prepare evidence.
- **Outdated**: Commented code or issue no longer exists; prepare evidence.
- **Defer / Won't fix**: Valid concern intentionally not changed now; document a specific reason.
3. **Act according to the classification and mode**
- Keep edits scoped to the review feedback.
- Follow reviewer-provided fix instructions literally when they are still applicable; deviate only when the current code proves the instruction is stale or unsafe.
- In `dry_run`, stop at triage, proposed fixes, suggested replies, and verification plan.
- In `no_push`, local edits are allowed, but do not push or resolve threads for local-only fixes.
- In `no_reply`, do not post replies or resolve threads; report suggested replies/actions instead.
- In normal mode, batch replies where practical, then resolve every handled thread by default after the fix, explanation, or deferral reason is available on the PR.
4. **Verify before claiming completion**
- For fixes, run appropriate checks or explain why they could not run.
- Re-inspect the updated diff and comment context to confirm the concern is resolved.
- Do not mark a thread resolved if it still needs reviewer, maintainer, or product input.
5. **Finish**
- Normal mode: commit/push changes when appropriate, post useful replies or a summary, and resolve all handled threads by default.
- Safe modes: report the local state and the exact replies/resolution actions a human could take.
## Reply Guidance
- Keep inline replies short and tied to the original title or concern.
- For fixed findings, mention the concrete change or commit if useful.
- For already-addressed or outdated findings, cite the current code path or behavior that makes the finding no longer applicable.
- For deferred or won't-fix findings, provide the reason and any follow-up issue or owner if known.
- If a reply or resolve operation fails, continue with the remaining threads and report the failure in the final summary.
## Final Summary Checklist
- Mode used: `normal`, `dry_run`, `no_push`, or `no_reply`
- Counts by disposition: fixed, answered, clarified/left open, already addressed, outdated, deferred/won't-fix
- Threads resolved, intentionally left open, or resolution actions skipped by mode
- Verification run or planned
- Commits pushed, local diff/commits, or "none"
- Remaining open items and who needs to respond
+15
View File
@@ -62,6 +62,19 @@ jobs:
runs-on: ubuntu-slim
secrets:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
github-codeql-analysis:
if: >
github.event_name == 'push'
|| github.event_name == 'pull_request'
|| (github.event_name == 'workflow_dispatch' && inputs.workflow == 'lint-and-test')
permissions:
contents: read
security-events: write
actions: read
uses: dceoy/gh-actions-for-devops/.github/workflows/github-codeql-analysis.yml@main # zizmor: ignore[unpinned-uses]
with:
language: >
["python"]
dependabot-auto-merge:
if: >
github.event_name == 'pull_request' && github.actor == 'dependabot[bot]'
@@ -73,5 +86,7 @@ jobs:
contents: write
pull-requests: write
actions: read
checks: read
statuses: read
with:
unconditional: true
+20 -64
View File
@@ -1,81 +1,37 @@
# Repository Guidelines
## Commands
## Project Structure & Module Organization
### Development Setup
`mt5cli/` contains the package source. Important modules include `cli.py` for the Typer command-line app, `client.py` and `sdk.py` for public MT5 client/session APIs, `history.py` for SQLite history collection, `storage.py` and `converters.py` for export behavior, and `schemas.py` for normalized dataset contracts. `tests/` holds pytest coverage for CLI behavior, SDK contracts, trading helpers, history, and utilities. `docs/` and `mkdocs.yml` define the MkDocs site and API reference. `skills/mt5cli/SKILL.md` documents the mt5cli agent skill.
```bash
uv sync
```
## Build, Test, and Development Commands
### Code Quality and Documentation
- `uv sync` installs runtime and development dependencies from `pyproject.toml` and `uv.lock`.
- `uv run mt5cli --help` runs the local CLI entry point.
- `uv run ruff format .` formats Python files.
- `uv run ruff check --fix .` lints and applies safe fixes.
- `uv run pyright .` runs strict type checking.
- `uv run pytest` runs doctests, branch coverage, and the test suite.
- `uv run mkdocs serve` previews documentation locally; `uv run mkdocs build` validates the docs build.
**Important**: Run these before committing or creating a PR.
Use `.agents/skills/local-qa/SKILL.md` for pre-handoff QA. It runs `.agents/skills/local-qa/scripts/qa.sh`, which formats, lints, type-checks, tests, formats Markdown, and checks GitHub workflows.
1. **format, lint, and test**: Use `local-qa` skill.
2. **Documentation build** (if any public API changes): `uv run mkdocs build`
## Coding Style & Naming Conventions
## Architecture
Target Python `>=3.11,<3.14`. Use Ruffs configured 88-character line length and Google-style docstrings. Pyright is strict, so prefer explicit public type annotations and narrow exception handling. Keep module, function, and variable names in `snake_case`; classes and enums use `PascalCase`. Preserve the packages small, typed helper style rather than adding broad abstractions.
### Key Dependencies
## Design Principles
- **pdmt5**: Pandas-based data handler for MetaTrader 5 (core library)
- **typer**: CLI framework for building command-line interfaces
- **click**: Parameter type customization for CLI options
- **pandas**: Core data manipulation and analysis
Apply KISS, DRY, and YAGNI when changing code. Prefer the simplest implementation that satisfies the current CLI/API contract. Remove duplication when shared behavior is already proven by at least two concrete call sites, but avoid generic helpers for speculative reuse. Do not add configuration flags, extension hooks, or alternate backends until a real repository use case requires them.
### Package Structure
## Testing Guidelines
- `mt5cli/`: Main package directory
- `__init__.py`: Package initialization and exports (`detect_format`, `export_dataframe`)
- `cli.py`: CLI application with typer-based commands for data export
- `utils.py`: Constants, enums, parameter types, parsers, and export utilities
- `__main__.py`: Entry point for `python -m mt5cli`
- `tests/`: Comprehensive test suite (pytest-based)
- `test_cli.py`: Tests for CLI commands and collect-history behavior
- `test_utils.py`: Tests for utility constants, parameter types, parsers, and export functions
- `docs/`: MkDocs documentation with API reference
- `docs/index.md`: Main documentation
- `docs/api/`: Auto-generated API documentation for all modules
- Modern Python packaging with `pyproject.toml` and uv dependency management
### Quality Standards
- Type hints required (pyright strict mode)
- Comprehensive linting with 35+ rule categories (ruff)
- Test coverage tracking with 100% (pytest-cov)
- Parametrized tests for input/result matrices using `pytest.mark.parametrize` (pytest)
- Test doubles (mocks, stubs) using `pytest_mock` for external dependencies (pytest-mock)
- Pydantic models for data validation and configuration
### Documentation workflow
1. Add Google-style docstrings to functions/classes
2. Local preview: `uv run mkdocs serve`
3. Build: `uv run mkdocs build`
4. Deploy: `uv run mkdocs gh-deploy`
Tests use pytest, pytest-mock, doctests, and pytest-cov. Test files should match `tests/test_*.py`, classes `Test*`, and functions `test_*`. Coverage is configured with `fail_under = 100`, so add focused tests for every behavior change. Mock MT5/pdmt5 boundaries; do not require a live MetaTrader terminal in unit tests.
## Commit & Pull Request Guidelines
- Run QA checks using `local-qa` skill before committing or creating a PR.
- Branch names use appropriate prefixes on creation (e.g., `feature/...`, `bugfix/...`, `refactor/...`, `docs/...`, `chore/...`).
- When instructed to create a PR, create it as a draft with appropriate labels by default.
Recent history uses concise imperative commits, sometimes with conventional prefixes such as `feat:` or `chore:` and PR numbers appended by GitHub. Keep commits scoped to one logical change. Pull requests should describe behavior changes, note tests run, link related issues, and call out MT5/live-trading risk where relevant.
## Code Design Principles
## Security & Configuration Tips
Always prefer the simplest design that works.
- **KISS**: Choose straightforward solutions and avoid unnecessary abstraction.
- **DRY**: Remove duplication when it improves clarity and maintainability.
- **YAGNI**: Do not add features, hooks, or flexibility until they are needed.
- **SOLID/Clean Code**: Apply these as tools, only when they keep the design simpler and easier to change.
## Development Methodology
Keep delivery incremental, test-backed, and easy to review.
- Make small, safe, reversible changes.
- Prefer `Red -> Green -> Refactor`.
- Do not mix feature work and refactoring in the same commit.
- Refactor when it improves clarity or removes real duplication (Rule of Three).
- Keep tests fast, focused, and self-validating.
Never commit account credentials, broker passwords, exported private data, or local `.venv` contents. Treat `order_send` and CLI `order-send --yes` as live execution paths; gate examples and tests so they cannot place real trades accidentally.
+94 -5
View File
@@ -2,14 +2,14 @@
[![CI/CD](https://github.com/dceoy/mt5cli/actions/workflows/ci.yml/badge.svg)](https://github.com/dceoy/mt5cli/actions/workflows/ci.yml)
Command-line tool for exporting MetaTrader 5 data to CSV, JSON, Parquet, and SQLite3.
Generic MT5 data and execution infrastructure for Python applications. Export from the CLI or import a small, stable Python API in downstream packages.
Built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data handler for MetaTrader 5.
## Architecture
- **pdmt5** — canonical MT5 client, DataFrame/trading primitives, and MT5 constant parsing (`TIMEFRAME_*`, `COPY_TICKS_*`, order types).
- **mt5cli** — CLI commands, CSV/JSON/Parquet/SQLite export, SQLite history collection, rate views, and local batch/automation SDK helpers built on pdmt5.
- **mt5cli** — public `MT5Client` API, standardized dataset schemas, storage helpers, CLI commands, and SQLite history collection built on pdmt5.
- **mt5api** — sibling HTTP adapter for remote MT5 access; not a dependency of mt5cli.
## Features
@@ -27,7 +27,96 @@ Built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data han
pip install -U mt5cli MetaTrader5
```
## Usage
## Python API (downstream packages)
Import `MT5Client` for generic MT5 data access, schema normalization, and optional order primitives. `Mt5CliClient` remains available as a backward-compatible alias.
```python
from datetime import UTC, datetime
from pathlib import Path
from mt5cli import (
DataKind,
Dataset,
MT5Client,
build_config,
collect_history,
export_dataframe,
mt5_session,
normalize_dataframe,
update_history_with_config,
)
# Persistent session for multiple calls
with mt5_session(build_config(login=12345, server="Broker-Demo")) as client:
rates = client.copy_rates_range(
"EURUSD",
timeframe="H1",
date_from="2024-01-01",
date_to="2024-02-01",
)
positions = client.positions()
check = client.order_check({"action": 1, "symbol": "EURUSD", "volume": 0.1})
# Normalize MT5 frames to the public schema contract before storage
closed_rates = normalize_dataframe(
rates, DataKind.rates, symbol="EURUSD", timeframe="H1"
)
export_dataframe(closed_rates, Path("rates.csv"), "csv")
# Bulk SQLite history (same behavior as collect-history CLI command)
collect_history(
Path("history.db"),
symbols=["EURUSD"],
date_from=datetime(2024, 1, 1, tzinfo=UTC),
date_to=datetime(2024, 2, 1, tzinfo=UTC),
datasets={Dataset.rates, Dataset.history_deals},
)
# Incremental append for automated pipelines
update_history_with_config(
output="history.db",
symbols=["EURUSD"],
config=build_config(login=12345),
)
```
Schema contracts live in `mt5cli.schemas` (`DataKind`, `validate_schema`, `normalize_dataframe`). Storage helpers are re-exported from `mt5cli.storage` and the package root.
`MT5Client.order_send()` is a live execution primitive: it can place real trades on the connected account. mt5cli does not implement strategy logic, signal generation, backtesting, or optimization — downstream applications must gate live execution explicitly.
### Trading lifecycle and state helpers
Trading applications can depend on `mt5cli` imports only; terminal path,
credentials, server, and timeout are forwarded to `pdmt5.Mt5Config`, numeric
login strings are coerced to integers, and empty login strings are treated as
unset.
```python
from mt5cli import (
calculate_spread_ratio,
create_trading_client,
get_account_snapshot,
mt5_trading_session,
)
with mt5_trading_session(
path=r"C:\Program Files\MetaTrader 5\terminal64.exe",
login="12345",
password="from-env-or-secret-store",
server="Broker-Demo",
) as client:
account = get_account_snapshot(client)
spread = calculate_spread_ratio(client, "EURUSD")
client = create_trading_client(login=12345, server="Broker-Demo")
try:
positions = client.positions_get_as_df(symbol="EURUSD")
finally:
client.shutdown()
```
## CLI usage
```bash
# Export account information to CSV
@@ -161,7 +250,7 @@ eurusd_m1 = rates["EURUSD", "M1"] # closed bars only
- **Throttled history updates**: use `ThrottledHistoryUpdater` to wrap `update_history()` with a minimum `interval_seconds` between successful runs (monotonic clock). Call `should_update()` / `update(client, symbols)` from an application loop; errors propagate by default, or pass `suppress_errors=True` to swallow recoverable `Mt5*Error`, `sqlite3.Error`, `ValueError`, `OSError`, and MT5 client capability errors for history API methods without advancing the throttle (other `AttributeError` / `TypeError` values always propagate).
- **Trading session helpers**: use `mt5_trading_session()` for a trading-capable `pdmt5.Mt5TradingClient` that initializes/logs in via `Mt5Config.path` and always shuts down safely. Pair with `detect_position_side()`, `calculate_margin_and_volume()`, and `determine_order_limits()` for generic position and sizing utilities. The read-only `mt5_session()` / `Mt5CliClient` SDK is unchanged.
- **Granularity-keyed rate loading**: `load_rate_series_by_granularity()` builds targets with `build_rate_targets()`, loads them with `load_rate_series_from_sqlite()`, and returns a mapping keyed by `(symbol | None, granularity_name)` such as `("EURUSD", "M1")` to reduce downstream boilerplate.
- **MT5 session helper**: use the `mt5_session()` context manager to attach to (or, when `Mt5Config.path` is set, launch) an MT5 terminal, log in, and yield a connected `Mt5CliClient` that shuts down on exit.
- **MT5 session helper**: use the `mt5_session()` context manager to attach to (or, when `Mt5Config.path` is set, launch) an MT5 terminal, log in, and yield a connected `MT5Client` that shuts down on exit.
- **SQLite export helpers**: use `export_dataframe_to_sqlite()` for append mode, optional index export, and post-write deduplication by key columns.
- **Recent ticks and margins**: `recent_ticks()` and `minimum_margins()` SDK helpers (and matching CLI commands) cover common downstream read-only queries.
@@ -226,7 +315,7 @@ finally:
client.shutdown()
```
Read-only collectors can keep using `mt5_session()` and `Mt5CliClient` without changes.
Read-only collectors can keep using `mt5_session()` and `MT5Client` (or the `Mt5CliClient` alias) without changes.
## Development
+3
View File
@@ -0,0 +1,3 @@
# Client
::: mt5cli.client
+3
View File
@@ -0,0 +1,3 @@
# Converters
::: mt5cli.converters
+3
View File
@@ -0,0 +1,3 @@
# Exceptions
::: mt5cli.exceptions
+29 -3
View File
@@ -167,19 +167,45 @@ Resolution rules:
### Rate data loading
Use `load_rate_data()` to load a table or view from a SQLite path, or
`load_rate_data_from_connection()` when you already have a connection:
The canonical normalized rate table is `rates`; compatibility views are named
with `rate_<symbol>__<timeframe>` for single-timeframe symbols or
`rate_<symbol>__<granularity>_<timeframe>` when a symbol has multiple stored
timeframes. `resolve_rate_table_name()` returns `rates`, while
`resolve_rate_view_name()` returns the per-symbol compatibility view name.
Use `load_rate_data()` or `load_rate_series_from_sqlite(..., table=...)` to load
a single table or view from a SQLite path. Use
`load_rate_series_by_granularity()` to load multiple instrument/granularity
targets without hard-coding view names:
```python
from pathlib import Path
from mt5cli import load_rate_data
from mt5cli import (
load_rate_data,
load_rate_series_by_granularity,
load_rate_series_from_sqlite,
resolve_rate_table_name,
)
from mt5cli.history import resolve_rate_view_name
view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1", require_existing=True)
rates = load_rate_data(Path("history.db"), view, count=1000)
same_rates = load_rate_series_from_sqlite(Path("history.db"), table=view, count=1000)
table = resolve_rate_table_name("EURUSD", "M1") # "rates"
series = load_rate_series_by_granularity(
Path("history.db"),
symbols=["EURUSD", "GBPUSD"],
granularities=["M1", "H1"],
count=500,
)
```
`count` returns the latest rows while preserving chronological order. Missing
tables/views and mismatched `explicit_tables` lengths raise `ValueError` with
the requested database target in the message.
The loader accepts close-based OHLC rate data or tick-like bid/ask data. It
validates that `time` exists, parses timestamps with pandas, and returns a
DataFrame indexed by ascending `DatetimeIndex` named `time`.
+39 -111
View File
@@ -1,125 +1,53 @@
# API Reference
This section contains the complete API documentation for mt5cli.
This section documents the mt5cli public Python API and CLI modules.
## Modules
## Public API layers
The mt5cli package consists of the following modules:
| Module | Purpose |
| ----------------------------------------- | ------------------------------------------------------------------------- |
| [Client](client.md) | `MT5Client` session abstraction for data access and order primitives |
| [Schemas](schemas.md) | Canonical DataFrame contracts and normalization helpers |
| [Storage](storage.md) | CSV/JSON/Parquet/SQLite export and history collection helpers |
| [Converters](converters.md) | Symbol, timeframe, timezone, and date-range utilities |
| [Exceptions](exceptions.md) | Stable mt5cli exception types and MT5 error normalization |
| [SDK](sdk.md) | Module-level fetch helpers, multi-account collectors, incremental history |
| [Trading](trading.md) | Trading-capable sessions and operational helpers |
| [History Collection (SQLite)](history.md) | SQLite schema, incremental writes, dedup, and rate views |
| [CLI](cli.md) | Typer commands that delegate to the Python API |
| [Utils](utils.md) | Parsing helpers and Click parameter types |
### [CLI](cli.md)
## Architecture overview
Command-line interface module providing typer-based commands for exporting MetaTrader 5 data to CSV, JSON, Parquet, and SQLite3 formats.
### [Utils](utils.md)
Utility module providing constants, enums, Click parameter types, and helper functions for parsing and exporting data.
### [SDK](sdk.md)
Programmatic SDK for read-only MetaTrader 5 data collection. Returns pandas DataFrames and provides `collect_history` for SQLite bulk collection.
### [Trading](trading.md)
Trading-capable session management and operational helpers built on `pdmt5.Mt5TradingClient`. Complements the read-only SDK without changing existing `Mt5CliClient` behavior.
### [History Collection (SQLite)](history.md)
SQLite storage helpers for the `collect-history` command schema, incremental updates, deduplication, indexes, and optional views.
## Architecture Overview
The package follows a simple architecture built on top of pdmt5:
1. **CLI Layer** (`cli.py`): Typer application with subcommands that delegate to the SDK and export results.
2. **SDK Layer** (`sdk.py`): Read-only data access functions, `Mt5CliClient`, and `collect_history` orchestration.
3. **Trading Layer** (`trading.py`): Trading-capable sessions and operational helpers on `Mt5TradingClient`.
4. **Utils Layer** (`utils.py`): Constants, enums, custom Click parameter types, parsing helpers, and format detection/export utilities.
5. **Data Layer** (via `pdmt5`): Uses `Mt5DataClient`, `Mt5TradingClient`, and `Mt5Config` from the pdmt5 package for MetaTrader 5 access.
## Usage Guidelines
All modules follow these conventions:
- **Type Safety**: All functions include comprehensive type hints
- **Error Handling**: User-friendly error messages via typer
- **Documentation**: Google-style docstrings with examples
- **Validation**: Custom Click parameter types for input validation
## Quick Start
```bash
# Export account information to CSV
mt5cli -o account.csv account-info
# Export EURUSD H1 rates to Parquet
mt5cli -o rates.parquet rates-from --symbol EURUSD --timeframe H1 \
--date-from 2024-01-01 --count 1000
# Export ticks to JSON
mt5cli -o ticks.json ticks-from --symbol EURUSD \
--date-from 2024-01-01 --count 500 --flags ALL
# Export to SQLite3 with custom table name
mt5cli -o data.db --table symbols symbols --group "*USD*"
```mermaid
flowchart TD
App["Downstream application"] --> Client["MT5Client"]
CLI["mt5cli CLI"] --> Client
Client --> SDK["sdk / pdmt5"]
Client --> Schemas["schemas"]
Storage["storage"] --> History["history SQLite"]
Storage --> Utils["utils export"]
SDK --> PDMT5["pdmt5.Mt5DataClient"]
```
## Python API
Downstream packages should depend on the package root exports (`MT5Client`, `DataKind`, `normalize_dataframe`, `export_dataframe`, `collect_history`, etc.) rather than private modules.
`MT5Client.order_send()` is a live execution primitive that can place real trades. mt5cli exposes minimal execution helpers only; strategy logic, signals, backtests, and optimization remain out of scope and must be implemented downstream with explicit execution gating.
## Quick start
```python
from datetime import UTC, datetime
from pathlib import Path
from mt5cli import MT5Client, build_config, mt5_session
from mt5cli import (
Dataset,
IfExists,
Mt5CliClient,
collect_history,
copy_rates_range,
detect_format,
export_dataframe,
export_dataframe_to_sqlite,
minimum_margins,
recent_ticks,
)
from mt5cli.history import resolve_rate_view_name
# Fetch rates programmatically
rates = copy_rates_range(
"EURUSD",
timeframe="H1",
date_from="2024-01-01",
date_to="2024-02-01",
)
# Detect output format from file extension
fmt = detect_format(Path("output.parquet")) # Returns "parquet"
# Export a DataFrame
export_dataframe(rates, Path("output.csv"), "csv")
# Append to SQLite with deduplication
export_dataframe_to_sqlite(
rates,
Path("history.db"),
"rates",
if_exists=IfExists.APPEND,
deduplicate_on=("symbol", "timeframe", "time"),
)
# Resolve rate compatibility views and fetch recent ticks
view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1")
ticks = recent_ticks("EURUSD", seconds=300)
margins = minimum_margins("EURUSD")
# Collect history into SQLite
collect_history(
Path("history.db"),
symbols=["EURUSD"],
date_from=datetime(2024, 1, 1, tzinfo=UTC),
date_to=datetime(2024, 2, 1, tzinfo=UTC),
)
with mt5_session(build_config(login=12345)) as client:
rates = client.copy_rates_range("EURUSD", "H1", "2024-01-01", "2024-02-01")
positions = client.positions()
```
## Examples
```bash
mt5cli -o account.csv account-info
mt5cli -o rates.parquet rates-range --symbol EURUSD --timeframe H1 \
--date-from 2024-01-01 --date-to 2024-02-01
```
See individual module pages for detailed usage examples and code samples.
See individual module pages for detailed usage examples.
+3
View File
@@ -0,0 +1,3 @@
# Schemas
::: mt5cli.schemas
+20 -5
View File
@@ -31,13 +31,25 @@ rates = collect_latest_rates_for_accounts_with_retries(
### Latest closed rate bars
MetaTrader 5 `start_pos=0` includes the still-forming current bar as the last
row. `collect_latest_closed_rates_for_accounts()` fetches `count + 1` bars,
drops that row with `drop_forming_rate_bar()`, and validates each series is
non-empty. Use `collect_latest_closed_rates_by_granularity()` when callers
prefer keys such as `("EURUSD", "M1")` instead of integer timeframes.
row. `fetch_latest_closed_rates()` handles one connected client; multi-account
helpers fetch `count + 1` bars, drop that row with `drop_forming_rate_bar()`,
and validate each series is non-empty. Returned frames are ordered
oldest-to-newest and may contain fewer than `count` rows only when MT5 returns
fewer closed bars.
```python
from mt5cli import AccountSpec, collect_latest_closed_rates_by_granularity
from mt5cli import (
AccountSpec,
collect_latest_closed_rates_by_granularity,
fetch_latest_closed_rates,
)
closed = fetch_latest_closed_rates(
client,
symbol="EURUSD",
granularity="M1",
count=500,
)
rates = collect_latest_closed_rates_by_granularity(
[AccountSpec(symbols=["EURUSD"], login=12345)],
@@ -48,6 +60,9 @@ rates = collect_latest_closed_rates_by_granularity(
closed_m1 = rates["EURUSD", "M1"]
```
Use `collect_latest_closed_rates_by_granularity()` when callers prefer keys such
as `("EURUSD", "M1")` instead of integer timeframes.
### Resolving credentials and `${ENV_VAR}` placeholders
`resolve_account_spec()` / `resolve_account_specs()` merge explicit override
+3
View File
@@ -0,0 +1,3 @@
# Storage
::: mt5cli.storage
+56 -12
View File
@@ -4,38 +4,60 @@
## Trading-capable MT5 sessions
`mt5_trading_session()` complements the read-only `mt5_session()` helper in
`sdk.py`. It yields a connected `pdmt5.Mt5TradingClient`, uses
`Mt5Config.path` to launch the terminal when configured, and always calls
`shutdown()` on exit.
`create_trading_client()` and `mt5_trading_session()` complement the read-only
`mt5_session()` helper in `sdk.py`. They return or yield an initialized
`pdmt5.Mt5TradingClient`, use `Mt5Config.path` to launch the terminal when
configured, and `mt5_trading_session()` always calls `shutdown()` on exit.
```python
from pdmt5 import Mt5Config
from mt5cli import mt5_trading_session
from mt5cli import create_trading_client, mt5_trading_session
with mt5_trading_session(
Mt5Config(path=r"C:\Program Files\MetaTrader 5\terminal64.exe", login=12345),
path=r"C:\Program Files\MetaTrader 5\terminal64.exe",
login="12345",
password="secret",
server="Broker-Demo",
retry_count=2,
) as client:
positions = client.positions_get_as_df(symbol="EURUSD")
client = create_trading_client(login=12345, server="Broker-Demo")
try:
account = client.account_info_as_dict()
finally:
client.shutdown()
```
`login` accepts `int`, numeric `str`, or an empty string; empty strings are
treated as unset. `path`, `password`, `server`, and `timeout` are forwarded to
`pdmt5.Mt5Config`, and omitted `timeout` values keep the lower-level default.
The read-only `Mt5CliClient` / `mt5_session()` API is unchanged.
## Operational trading helpers
## State and order helpers
These helpers are strategy-agnostic and do not depend on signal detection,
betting logic, or scheduling code in downstream applications.
```python
from mt5cli import (
calculate_spread_ratio,
calculate_margin_and_volume,
close_open_positions,
detect_position_side,
determine_order_limits,
get_account_snapshot,
get_positions_frame,
get_symbol_snapshot,
get_tick_snapshot,
place_market_order,
)
account = get_account_snapshot(client)
symbol = get_symbol_snapshot(client, "EURUSD")
tick = get_tick_snapshot(client, "EURUSD")
positions = get_positions_frame(client, "EURUSD")
side = detect_position_side(client, "EURUSD")
spread_ratio = calculate_spread_ratio(client, "EURUSD")
sizing = calculate_margin_and_volume(
client,
"EURUSD",
@@ -49,11 +71,33 @@ limits = determine_order_limits(
stop_loss_limit_ratio=0.01,
take_profit_limit_ratio=0.02,
)
preview = place_market_order(
client,
symbol="EURUSD",
volume=sizing["buy_volume"],
order_side="BUY",
sl=limits["stop_loss"],
tp=limits["take_profit"],
dry_run=True,
)
closed = close_open_positions(client, symbols="EURUSD", dry_run=True)
```
Protective ratios must satisfy `0 <= ratio < 1`; `0` omits that level.
`calculate_margin_and_volume()` clamps negative `margin_free` to `0.0`
before sizing.
`detect_position_side()` returns `long` for buy-only exposure, `short` for
sell-only exposure, and `None` for no positions or mixed long/short exposure.
`calculate_spread_ratio()` uses `(ask - bid) / ((ask + bid) / 2)` and raises
`Mt5TradingError` when bid or ask is missing or non-positive.
SL/TP ratios for `determine_order_limits()` must satisfy `0 <= ratio < 1`; `0`
omits that level. SL/TP prices are rounded with symbol `digits` metadata when
available. `unit_margin_ratio` and `preserved_margin_ratio` for
`calculate_margin_and_volume()` accept `0 <= ratio <= 1`; `unit_margin_ratio=0`
requests one minimum valid unit when the post-reserve margin can afford it.
Negative `margin_free` is clamped to `0.0` before sizing. Execution helpers
return normalized dictionaries containing the request, response, status,
retcode, and `dry_run` flag; `dry_run=True` never sends an order. Market order
helpers mark known non-success MT5 retcodes as `status="failed"` while keeping
the normalized response for inspection.
## Migration from mteor-local helpers
+34 -32
View File
@@ -1,15 +1,15 @@
# mt5cli
Command-line tool for MetaTrader 5 data export.
Generic MT5 data and execution infrastructure for Python applications.
## Overview
mt5cli is a CLI application that exports MetaTrader 5 trading data to multiple file formats. It is built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data handler for MetaTrader 5.
mt5cli provides a stable `MT5Client` Python API, standardized dataset schemas, storage helpers, and a CLI for exporting MetaTrader 5 data. It is built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data handler for MetaTrader 5.
## Architecture
- **pdmt5** — canonical MT5 client, DataFrame/trading primitives, and MT5 constant parsing (`TIMEFRAME_*`, `COPY_TICKS_*`, order types).
- **mt5cli**CLI commands, CSV/JSON/Parquet/SQLite export, SQLite history collection, rate views, and local batch/automation SDK helpers built on pdmt5.
- **mt5cli**public `MT5Client` API, schema contracts, storage helpers, CLI commands, and SQLite history collection built on pdmt5.
- **mt5api** — sibling HTTP adapter for remote MT5 access; not a dependency of mt5cli.
## Features
@@ -27,66 +27,68 @@ mt5cli is a CLI application that exports MetaTrader 5 trading data to multiple f
pip install mt5cli
```
## Programmatic usage / SDK usage
## Python API for downstream packages
mt5cli can be used as a small Python SDK for read-only MetaTrader 5 data collection. SDK functions return pandas DataFrames without writing files. Use `export_dataframe` or `export_dataframe_to_sqlite` when you need to persist results.
Import `MT5Client` for generic MT5 data access, schema normalization, and optional order primitives. `Mt5CliClient` remains available as a backward-compatible alias.
```python
from datetime import UTC, datetime
from pathlib import Path
from mt5cli import (
Mt5CliClient,
DataKind,
Dataset,
MT5Client,
build_config,
collect_history,
copy_rates_range,
export_dataframe,
export_dataframe_to_sqlite,
load_rate_data,
minimum_margins,
mt5_session,
normalize_dataframe,
recent_ticks,
resolve_rate_view_name,
)
from mt5cli.history import resolve_rate_view_name
# One-off fetch with module-level helpers
rates = copy_rates_range(
"EURUSD",
timeframe="H1",
date_from="2024-01-01",
date_to="2024-02-01",
# Persistent session for multiple calls
with mt5_session(build_config(login=12345, server="Broker-Demo")) as client:
rates = client.copy_rates_range(
"EURUSD",
timeframe="H1",
date_from="2024-01-01",
date_to="2024-02-01",
)
positions = client.positions()
check = client.order_check({"action": 1, "symbol": "EURUSD", "volume": 0.1})
# Normalize MT5 frames to the public schema contract before storage
closed_rates = normalize_dataframe(
rates, DataKind.rates, symbol="EURUSD", timeframe="H1"
)
export_dataframe(rates, Path("rates.csv"), "csv")
export_dataframe(closed_rates, Path("rates.csv"), "csv")
# Resolve SQLite rate compatibility views for downstream tools
# Offline rate loading from mt5cli-managed SQLite history
view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1", require_existing=True)
offline_rates = load_rate_data(Path("history.db"), view, count=1000)
# Recent tick window and minimum margin summary
# One-off helpers still work without instantiating a client
ticks = recent_ticks("EURUSD", seconds=300)
margins = minimum_margins("EURUSD")
# Reuse one MT5 connection for multiple calls
with Mt5CliClient(login=12345, password="secret", server="Broker-Demo") as client:
account = client.account_info()
positions = client.positions()
latest = client.latest_rates("EURUSD", "M1", count=100)
summary = client.mt5_summary()
summary_table = client.mt5_summary_as_df()
# Bulk SQLite collection (same behavior as the collect-history CLI command)
collect_history(
Path("history.db"),
symbols=["EURUSD", "GBPUSD"],
date_from=datetime(2024, 1, 1, tzinfo=UTC),
date_to=datetime(2024, 2, 1, tzinfo=UTC),
timeframe="M1",
flags="ALL",
with_views=True,
datasets={Dataset.rates, Dataset.history_deals},
)
```
Timeframes, tick flags, and ISO 8601 date strings are accepted wherever noted in the SDK API.
Schema contracts live in `mt5cli.schemas` (`DataKind`, `validate_schema`, `normalize_dataframe`). Storage helpers are re-exported from `mt5cli.storage` and the package root.
`Mt5CliClient.mt5_summary()` returns the SDK structured form as plain nested Python values. Use `Mt5CliClient.mt5_summary_as_df()` when you need a one-row DataFrame for export. The `mt5-summary` CLI command uses this tabular form, so nested terminal/account fields are JSON-encoded strings that are safe for CSV, JSON, Parquet, and SQLite output.
`MT5Client.order_send()` is a live execution primitive: it can place real trades on the connected account. mt5cli does not implement strategy logic, signal generation, backtesting, or optimization — downstream applications must gate live execution explicitly (the CLI requires `--yes` for `order-send`).
`MT5Client.mt5_summary()` returns structured nested Python values. Use `MT5Client.mt5_summary_as_df()` when you need a one-row DataFrame for export.
## Quick Start
+6 -1
View File
@@ -1,5 +1,5 @@
site_name: mt5cli API Documentation
site_description: Command-line tool for MetaTrader 5
site_description: Generic MT5 data and execution infrastructure for Python
site_author: dceoy
site_url: https://github.com/dceoy/mt5cli
@@ -56,6 +56,11 @@ nav:
- Home: index.md
- API Reference:
- Overview: api/index.md
- Client: api/client.md
- Schemas: api/schemas.md
- Storage: api/storage.md
- Converters: api/converters.md
- Exceptions: api/exceptions.md
- CLI: api/cli.md
- SDK: api/sdk.md
- Trading: api/trading.md
+106 -12
View File
@@ -1,7 +1,27 @@
"""mt5cli: Command-line tool and SDK for MetaTrader 5."""
"""mt5cli: Generic MT5 data and execution infrastructure for Python applications."""
from importlib.metadata import version
from pdmt5 import Mt5Config, Mt5RuntimeError, Mt5TradingClient, Mt5TradingError
from .client import MT5Client, build_config, mt5_session
from .converters import (
ensure_utc,
granularity_name,
normalize_symbol,
normalize_symbols,
parse_date_range,
recent_window,
)
from .exceptions import (
Mt5CliError,
Mt5ConnectionError,
Mt5OperationError,
Mt5SchemaError,
call_with_normalized_errors,
is_recoverable_mt5_error,
normalize_mt5_exception,
)
from .history import (
RateTarget,
build_rate_targets,
@@ -14,16 +34,27 @@ from .history import (
resolve_history_datasets,
resolve_history_tick_flags,
resolve_history_timeframes,
resolve_rate_table_name,
resolve_rate_tables,
resolve_rate_view_name,
resolve_rate_view_names,
)
from .schemas import (
DEDUP_KEYS,
KNOWN_MT5_TIME_COLUMNS,
REQUIRED_COLUMNS,
TIME_COLUMNS,
DataKind,
normalize_dataframe,
normalize_time_columns,
schema_columns,
validate_schema,
)
from .sdk import (
AccountSpec,
Mt5CliClient,
ThrottledHistoryUpdater,
account_info,
build_config,
collect_history,
collect_latest_closed_rates_by_granularity,
collect_latest_closed_rates_for_accounts,
@@ -35,13 +66,13 @@ from .sdk import (
copy_rates_range,
copy_ticks_from,
copy_ticks_range,
fetch_latest_closed_rates,
history_deals,
history_orders,
last_error,
latest_rates,
market_book,
minimum_margins,
mt5_session,
mt5_summary,
mt5_summary_as_df,
orders,
@@ -61,20 +92,38 @@ from .sdk import (
from .sdk import (
version as mt5_version,
)
from .trading import (
calculate_margin_and_volume,
detect_position_side,
determine_order_limits,
mt5_trading_session,
)
from .utils import (
TICK_FLAG_MAP,
TIMEFRAME_MAP,
from .storage import (
Dataset,
IfExists,
detect_format,
export_dataframe,
export_dataframe_to_sqlite,
)
from .trading import (
POSITION_COLUMNS,
OrderFillingMode,
OrderSide,
OrderTimeMode,
PositionSide,
calculate_margin_and_volume,
calculate_new_position_margin_ratio,
calculate_spread_ratio,
calculate_volume_by_margin,
close_open_positions,
create_trading_client,
detect_position_side,
determine_order_limits,
get_account_snapshot,
get_positions_frame,
get_symbol_snapshot,
get_tick_snapshot,
mt5_trading_session,
place_market_order,
update_sltp_for_open_positions,
)
from .utils import (
TICK_FLAG_MAP,
TIMEFRAME_MAP,
parse_datetime,
parse_tick_flags,
parse_timeframe,
@@ -83,12 +132,31 @@ from .utils import (
__version__ = version(__package__) if __package__ else None
__all__ = [
"DEDUP_KEYS",
"KNOWN_MT5_TIME_COLUMNS",
"POSITION_COLUMNS",
"REQUIRED_COLUMNS",
"TICK_FLAG_MAP",
"TIMEFRAME_MAP",
"TIME_COLUMNS",
"AccountSpec",
"DataKind",
"Dataset",
"IfExists",
"MT5Client",
"Mt5CliClient",
"Mt5CliError",
"Mt5Config",
"Mt5ConnectionError",
"Mt5OperationError",
"Mt5RuntimeError",
"Mt5SchemaError",
"Mt5TradingClient",
"Mt5TradingError",
"OrderFillingMode",
"OrderSide",
"OrderTimeMode",
"PositionSide",
"RateTarget",
"ThrottledHistoryUpdater",
"account_info",
@@ -96,6 +164,11 @@ __all__ = [
"build_rate_targets",
"build_rate_view_name",
"calculate_margin_and_volume",
"calculate_new_position_margin_ratio",
"calculate_spread_ratio",
"calculate_volume_by_margin",
"call_with_normalized_errors",
"close_open_positions",
"collect_history",
"collect_latest_closed_rates_by_granularity",
"collect_latest_closed_rates_for_accounts",
@@ -107,14 +180,23 @@ __all__ = [
"copy_rates_range",
"copy_ticks_from",
"copy_ticks_range",
"create_trading_client",
"detect_format",
"detect_position_side",
"determine_order_limits",
"drop_forming_rate_bar",
"ensure_utc",
"export_dataframe",
"export_dataframe_to_sqlite",
"fetch_latest_closed_rates",
"get_account_snapshot",
"get_positions_frame",
"get_symbol_snapshot",
"get_tick_snapshot",
"granularity_name",
"history_deals",
"history_orders",
"is_recoverable_mt5_error",
"last_error",
"latest_rates",
"load_rate_data",
@@ -128,21 +210,31 @@ __all__ = [
"mt5_summary_as_df",
"mt5_trading_session",
"mt5_version",
"normalize_dataframe",
"normalize_mt5_exception",
"normalize_symbol",
"normalize_symbols",
"normalize_time_columns",
"orders",
"parse_date_range",
"parse_datetime",
"parse_tick_flags",
"parse_timeframe",
"place_market_order",
"positions",
"recent_history_deals",
"recent_ticks",
"recent_window",
"resolve_account_spec",
"resolve_account_specs",
"resolve_history_datasets",
"resolve_history_tick_flags",
"resolve_history_timeframes",
"resolve_rate_table_name",
"resolve_rate_tables",
"resolve_rate_view_name",
"resolve_rate_view_names",
"schema_columns",
"substitute_env_placeholders",
"symbol_info",
"symbol_info_tick",
@@ -150,4 +242,6 @@ __all__ = [
"terminal_info",
"update_history",
"update_history_with_config",
"update_sltp_for_open_positions",
"validate_schema",
]
+58 -72
View File
@@ -12,6 +12,7 @@ import typer
from pdmt5 import Mt5Config
from . import sdk
from .client import MT5Client
from .utils import (
DATETIME_TYPE,
REQUEST_TYPE,
@@ -91,9 +92,18 @@ def _execute_export(
)
def _sdk_client(ctx: typer.Context) -> sdk.Mt5CliClient:
def _sdk_client(ctx: typer.Context) -> MT5Client:
export_ctx = _get_export_context(ctx)
return sdk.Mt5CliClient(config=export_ctx.config)
return MT5Client(config=export_ctx.config)
def _export_command(
ctx: typer.Context,
fetch_fn: Callable[[MT5Client], pd.DataFrame],
) -> None:
"""Create an SDK client, fetch a DataFrame, and export it."""
client = _sdk_client(ctx)
_execute_export(ctx, lambda: fetch_fn(client))
@app.callback()
@@ -193,10 +203,9 @@ def rates_from(
count: Annotated[int, typer.Option(help="Number of records.")],
) -> None:
"""Export rates from a start date."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.copy_rates_from(symbol, timeframe, date_from, count),
lambda client: client.copy_rates_from(symbol, timeframe, date_from, count),
)
@@ -215,10 +224,14 @@ def rates_from_pos(
count: Annotated[int, typer.Option(help="Number of records.")],
) -> None:
"""Export rates from a start position."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.copy_rates_from_pos(symbol, timeframe, start_pos, count),
lambda client: client.copy_rates_from_pos(
symbol,
timeframe,
start_pos,
count,
),
)
@@ -240,10 +253,14 @@ def latest_rates(
] = 0,
) -> None:
"""Export latest rates from a start position."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.latest_rates(symbol, timeframe, count, start_pos=start_pos),
lambda client: client.latest_rates(
symbol,
timeframe,
count,
start_pos=start_pos,
),
)
@@ -268,10 +285,9 @@ def rates_range(
],
) -> None:
"""Export rates for a date range."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.copy_rates_range(symbol, timeframe, date_from, date_to),
lambda client: client.copy_rates_range(symbol, timeframe, date_from, date_to),
)
@@ -293,10 +309,9 @@ def ticks_from(
],
) -> None:
"""Export ticks from a start date."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.copy_ticks_from(symbol, date_from, count, flags),
lambda client: client.copy_ticks_from(symbol, date_from, count, flags),
)
@@ -318,10 +333,9 @@ def ticks_range(
],
) -> None:
"""Export ticks for a date range."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.copy_ticks_range(symbol, date_from, date_to, flags),
lambda client: client.copy_ticks_range(symbol, date_from, date_to, flags),
)
@@ -350,10 +364,9 @@ def ticks_recent(
] = "ALL", # pyright: ignore[reportArgumentType]
) -> None:
"""Export ticks from a recent time window."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.recent_ticks(
lambda client: client.recent_ticks(
symbol,
seconds,
date_to=date_to,
@@ -366,13 +379,13 @@ def ticks_recent(
@app.command()
def account_info(ctx: typer.Context) -> None:
"""Export account information."""
_execute_export(ctx, _sdk_client(ctx).account_info)
_export_command(ctx, lambda client: client.account_info())
@app.command()
def terminal_info(ctx: typer.Context) -> None:
"""Export terminal information."""
_execute_export(ctx, _sdk_client(ctx).terminal_info)
_export_command(ctx, lambda client: client.terminal_info())
@app.command()
@@ -384,8 +397,7 @@ def symbols(
] = None,
) -> None:
"""Export symbol list."""
client = _sdk_client(ctx)
_execute_export(ctx, lambda: client.symbols(group=group))
_export_command(ctx, lambda client: client.symbols(group=group))
@app.command()
@@ -394,8 +406,7 @@ def symbol_info(
symbol: Annotated[str, typer.Option(help="Symbol name.")],
) -> None:
"""Export symbol details."""
client = _sdk_client(ctx)
_execute_export(ctx, lambda: client.symbol_info(symbol))
_export_command(ctx, lambda client: client.symbol_info(symbol))
@app.command()
@@ -404,8 +415,7 @@ def minimum_margins(
symbol: Annotated[str, typer.Option(help="Symbol name.")],
) -> None:
"""Export minimum-volume buy and sell margin requirements."""
client = _sdk_client(ctx)
_execute_export(ctx, lambda: client.minimum_margins(symbol))
_export_command(ctx, lambda client: client.minimum_margins(symbol))
@app.command()
@@ -416,10 +426,9 @@ def orders(
ticket: Annotated[int | None, typer.Option(help="Ticket filter.")] = None,
) -> None:
"""Export active orders."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.orders(symbol=symbol, group=group, ticket=ticket),
lambda client: client.orders(symbol=symbol, group=group, ticket=ticket),
)
@@ -431,10 +440,9 @@ def positions(
ticket: Annotated[int | None, typer.Option(help="Ticket filter.")] = None,
) -> None:
"""Export open positions."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.positions(symbol=symbol, group=group, ticket=ticket),
lambda client: client.positions(symbol=symbol, group=group, ticket=ticket),
)
@@ -455,10 +463,9 @@ def history_orders(
position: Annotated[int | None, typer.Option(help="Position ticket.")] = None,
) -> None:
"""Export historical orders."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.history_orders(
lambda client: client.history_orders(
date_from=date_from,
date_to=date_to,
group=group,
@@ -486,10 +493,9 @@ def history_deals(
position: Annotated[int | None, typer.Option(help="Position ticket.")] = None,
) -> None:
"""Export historical deals."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.history_deals(
lambda client: client.history_deals(
date_from=date_from,
date_to=date_to,
group=group,
@@ -512,10 +518,9 @@ def recent_history_deals(
symbol: Annotated[str | None, typer.Option(help="Symbol filter.")] = None,
) -> None:
"""Export historical deals from a recent trailing window."""
client = _sdk_client(ctx)
_execute_export(
_export_command(
ctx,
lambda: client.recent_history_deals(
lambda client: client.recent_history_deals(
hours,
date_to=date_to,
group=group,
@@ -527,20 +532,19 @@ def recent_history_deals(
@app.command()
def mt5_summary(ctx: typer.Context) -> None:
"""Export a compact terminal/account status summary."""
client = _sdk_client(ctx)
_execute_export(ctx, client.mt5_summary_as_df)
_export_command(ctx, lambda client: client.mt5_summary_as_df())
@app.command()
def version(ctx: typer.Context) -> None:
"""Export MetaTrader5 version information."""
_execute_export(ctx, _sdk_client(ctx).version)
_export_command(ctx, lambda client: client.version())
@app.command()
def last_error(ctx: typer.Context) -> None:
"""Export the last error information."""
_execute_export(ctx, _sdk_client(ctx).last_error)
_export_command(ctx, lambda client: client.last_error())
@app.command()
@@ -549,8 +553,7 @@ def symbol_info_tick(
symbol: Annotated[str, typer.Option(help="Symbol name.")],
) -> None:
"""Export the last tick for a symbol."""
client = _sdk_client(ctx)
_execute_export(ctx, lambda: client.symbol_info_tick(symbol))
_export_command(ctx, lambda client: client.symbol_info_tick(symbol))
@app.command()
@@ -559,8 +562,7 @@ def market_book(
symbol: Annotated[str, typer.Option(help="Symbol name.")],
) -> None:
"""Export market depth (order book) for a symbol."""
client = _sdk_client(ctx)
_execute_export(ctx, lambda: client.market_book(symbol))
_export_command(ctx, lambda client: client.market_book(symbol))
@app.command()
@@ -572,15 +574,7 @@ def order_check(
],
) -> None:
"""Check funds sufficiency for a trading operation."""
export_ctx = _get_export_context(ctx)
def _fetch() -> pd.DataFrame:
return sdk._run_with_client( # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
export_ctx.config,
lambda c: c.order_check_as_df(request=request),
)
_execute_export(ctx, _fetch)
_export_command(ctx, lambda client: client.order_check(request))
@app.command()
@@ -603,15 +597,7 @@ def order_send(
if not yes:
msg = "Pass --yes to send a live trade request."
raise typer.BadParameter(msg, param_hint="--yes")
export_ctx = _get_export_context(ctx)
def _fetch() -> pd.DataFrame:
return sdk._run_with_client( # noqa: SLF001 # pyright: ignore[reportPrivateUsage]
export_ctx.config,
lambda c: c.order_send_as_df(request=request),
)
_execute_export(ctx, _fetch)
_export_command(ctx, lambda client: client.order_send(request))
@app.command()
+88
View File
@@ -0,0 +1,88 @@
"""Stable public client abstraction for MT5 data and execution operations."""
from __future__ import annotations
from contextlib import contextmanager
from typing import TYPE_CHECKING, Any, Self
from .sdk import Mt5CliClient, build_config, connected_client
if TYPE_CHECKING:
from collections.abc import Iterator
import pandas as pd
from pdmt5 import Mt5Config, Mt5DataClient
__all__ = [
"MT5Client",
"build_config",
"mt5_session",
]
class MT5Client(Mt5CliClient):
"""Public client for generic MT5 data access and order primitives.
Extends the read-only SDK client with optional order check/send helpers and
exposes the same connection lifecycle as :class:`~mt5cli.sdk.Mt5CliClient`.
Downstream applications such as private trading packages should prefer this
type over the legacy ``Mt5CliClient`` name.
mt5cli intentionally exposes minimal execution primitives only. Trading
decisions, signals, strategies, backtests, and optimization remain the
responsibility of downstream applications.
"""
def order_check(self, request: dict[str, Any]) -> pd.DataFrame:
"""Check funds sufficiency for a trade request.
Args:
request: MT5 order request dictionary.
Returns:
One-row DataFrame with the order-check result.
"""
return self._fetch(lambda client: client.order_check_as_df(request=request))
def order_send(self, request: dict[str, Any]) -> pd.DataFrame:
"""Send a live trade request to the MT5 trade server.
Warning:
This is a live execution primitive. A successful call can place,
modify, or close real trades on the connected account. Downstream
applications must gate usage explicitly (for example behind manual
confirmation or application-specific risk controls). mt5cli does
not implement strategy logic, signal generation, or trade sizing.
Args:
request: MT5 order request dictionary.
Returns:
One-row DataFrame with the order-send result.
"""
return self._fetch(lambda client: client.order_send_as_df(request=request))
@classmethod
def from_connected_client(cls, client: Mt5DataClient) -> Self:
"""Bind to an already-connected ``Mt5DataClient`` without owning it.
Returns:
Client wrapper bound to the injected connection.
"""
return cls(client=client)
@contextmanager
def mt5_session(config: Mt5Config | None = None) -> Iterator[MT5Client]:
"""Open an MT5 terminal session and yield a connected :class:`MT5Client`.
Args:
config: MT5 connection configuration. Defaults to an empty config that
attaches to a running terminal.
Yields:
Connected :class:`MT5Client` bound to the session.
"""
mt5_config = config or build_config()
with connected_client(mt5_config) as client:
yield MT5Client.from_connected_client(client)
+162
View File
@@ -0,0 +1,162 @@
"""Shared conversion helpers for MT5 symbols, timeframes, and date ranges."""
from __future__ import annotations
from datetime import UTC, datetime, timedelta
from typing import TYPE_CHECKING
from pdmt5 import get_timeframe_name as _get_timeframe_name
from .utils import parse_datetime, parse_tick_flags, parse_timeframe
if TYPE_CHECKING:
from collections.abc import Sequence
__all__ = [
"ensure_utc",
"granularity_name",
"normalize_symbol",
"normalize_symbols",
"parse_date_range",
"parse_datetime",
"parse_tick_flags",
"parse_timeframe",
"recent_window",
]
def normalize_symbol(symbol: str) -> str:
"""Normalize a broker symbol name for MT5 API calls.
Strips surrounding whitespace while preserving broker-specific casing and
suffixes (for example ``XAUUSDm``, ``US500.cash``, or ``EURUSD.r``).
Args:
symbol: Raw symbol name.
Returns:
Normalized symbol string.
Raises:
ValueError: If the symbol is empty after normalization.
"""
normalized = symbol.strip()
if not normalized:
msg = "Symbol must not be empty."
raise ValueError(msg)
return normalized
def normalize_symbols(symbols: Sequence[str]) -> list[str]:
"""Normalize a sequence of broker symbol names.
Args:
symbols: Raw symbol names.
Returns:
List of normalized, de-duplicated symbols preserving first-seen order.
"""
seen: set[str] = set()
resolved: list[str] = []
for symbol in symbols:
normalized = normalize_symbol(symbol)
if normalized not in seen:
seen.add(normalized)
resolved.append(normalized)
return resolved
def ensure_utc(value: datetime | str) -> datetime:
"""Return a timezone-aware UTC datetime.
Args:
value: Datetime instance or ISO 8601 string.
Returns:
UTC-aware datetime.
"""
if isinstance(value, str):
return parse_datetime(value)
if value.tzinfo is None:
return value.replace(tzinfo=UTC)
return value.astimezone(UTC)
def parse_date_range(
date_from: datetime | str,
date_to: datetime | str,
) -> tuple[datetime, datetime]:
"""Parse and validate an inclusive UTC date range.
Args:
date_from: Range start as datetime or ISO 8601 string.
date_to: Range end as datetime or ISO 8601 string.
Returns:
Tuple of UTC-aware ``(start, end)`` datetimes.
Raises:
ValueError: If ``date_from`` is after ``date_to``.
"""
start = ensure_utc(date_from)
end = ensure_utc(date_to)
if start > end:
msg = (
f"date_from ({start.isoformat()}) must not be after "
f"date_to ({end.isoformat()})."
)
raise ValueError(msg)
return start, end
def recent_window(
*,
hours: float | None = None,
seconds: float | None = None,
date_to: datetime | str | None = None,
) -> tuple[datetime, datetime]:
"""Build a trailing UTC window ending at ``date_to`` or now.
Exactly one of ``hours`` or ``seconds`` must be provided.
Args:
hours: Trailing window length in hours.
seconds: Trailing window length in seconds.
date_to: Window end. Defaults to current UTC time.
Returns:
Tuple of UTC-aware ``(start, end)`` datetimes.
Raises:
ValueError: If neither or both window lengths are provided, or if a
length is not positive.
"""
if (hours is None) == (seconds is None):
msg = "Provide exactly one of hours or seconds."
raise ValueError(msg)
if hours is not None:
length = timedelta(hours=hours)
else:
length = timedelta(seconds=seconds if seconds is not None else 0)
if length.total_seconds() <= 0:
msg = "Window length must be positive."
raise ValueError(msg)
end = ensure_utc(date_to) if date_to is not None else datetime.now(UTC)
return end - length, end
def granularity_name(timeframe: int | str) -> str:
"""Return a short granularity label for a timeframe integer or name.
Args:
timeframe: MT5 timeframe as integer or name (for example ``M1``).
Returns:
Short name such as ``M1`` or the stringified integer when unknown.
"""
tf = parse_timeframe(timeframe)
try:
name = _get_timeframe_name(tf)
except ValueError:
return str(tf)
return name.removeprefix("TIMEFRAME_")
+90
View File
@@ -0,0 +1,90 @@
"""Normalized exception types for MT5 and mt5cli operations."""
from __future__ import annotations
from typing import TYPE_CHECKING, TypeVar
from pdmt5 import Mt5RuntimeError, Mt5TradingError
if TYPE_CHECKING:
from collections.abc import Callable
T = TypeVar("T")
__all__ = [
"Mt5CliError",
"Mt5ConnectionError",
"Mt5OperationError",
"Mt5SchemaError",
"call_with_normalized_errors",
"is_recoverable_mt5_error",
"normalize_mt5_exception",
]
_RECOVERABLE_MT5_ERRORS: tuple[type[BaseException], ...] = (
Mt5TradingError,
Mt5RuntimeError,
)
class Mt5CliError(Exception):
"""Base exception for mt5cli public API errors."""
class Mt5ConnectionError(Mt5CliError):
"""Raised when MT5 initialization, login, or shutdown fails."""
class Mt5OperationError(Mt5CliError):
"""Raised when an MT5 data or trading operation fails."""
class Mt5SchemaError(Mt5CliError):
"""Raised when a DataFrame does not match an expected dataset schema."""
def is_recoverable_mt5_error(exc: BaseException) -> bool:
"""Return whether an exception is a transient MT5 failure worth retrying.
Args:
exc: Exception raised by MT5 or pdmt5.
Returns:
True for ``Mt5RuntimeError`` and ``Mt5TradingError``.
"""
return isinstance(exc, _RECOVERABLE_MT5_ERRORS)
def normalize_mt5_exception(exc: BaseException) -> Mt5CliError:
"""Map pdmt5/MT5 exceptions to stable mt5cli exception types.
Args:
exc: Original exception from MT5 or pdmt5.
Returns:
``Mt5ConnectionError`` for runtime failures, ``Mt5OperationError`` for
trading failures, or the original exception when it is not recognized.
"""
if isinstance(exc, Mt5TradingError):
return Mt5OperationError(str(exc))
if isinstance(exc, Mt5RuntimeError):
return Mt5ConnectionError(str(exc))
if isinstance(exc, Mt5CliError):
return exc
return Mt5CliError(str(exc))
def call_with_normalized_errors(fn: Callable[[], T]) -> T:
"""Run ``fn`` and map recoverable MT5 errors to mt5cli types.
Args:
fn: Callable performing MT5 work.
Returns:
Value returned by ``fn``.
"""
try:
return fn()
except _RECOVERABLE_MT5_ERRORS as exc:
normalized = normalize_mt5_exception(exc)
raise normalized from exc
+170 -64
View File
@@ -7,11 +7,12 @@ import sqlite3
from dataclasses import dataclass
from datetime import UTC, datetime
from pathlib import Path
from typing import TYPE_CHECKING, Literal, cast
from typing import TYPE_CHECKING, Literal, cast, overload
import pandas as pd
from pdmt5 import get_timeframe_name as _get_timeframe_name
from .schemas import DEDUP_KEYS, DataKind
from .utils import (
TIMEFRAME_NAMES,
Dataset,
@@ -31,10 +32,10 @@ logger = logging.getLogger(__name__)
DEFAULT_HISTORY_TIMEFRAMES: tuple[str, ...] = TIMEFRAME_NAMES
_HISTORY_DEDUP_KEYS: dict[Dataset, tuple[tuple[str, ...], ...]] = {
Dataset.rates: (("symbol", "timeframe", "time"), ("symbol", "time")),
Dataset.ticks: (("symbol", "time_msc"), ("symbol", "time")),
Dataset.history_orders: (("ticket",), ("symbol", "time", "type")),
Dataset.history_deals: (("ticket",), ("symbol", "time", "type", "entry")),
Dataset.rates: DEDUP_KEYS[DataKind.rates],
Dataset.ticks: DEDUP_KEYS[DataKind.ticks],
Dataset.history_orders: DEDUP_KEYS[DataKind.history_orders],
Dataset.history_deals: DEDUP_KEYS[DataKind.history_deals],
}
_TRADE_DEAL_TYPES: tuple[int, int] = (0, 1)
@@ -141,6 +142,26 @@ def build_rate_view_name(
return f"rate_{symbol}__{granularity}_{timeframe}"
def resolve_rate_table_name(symbol: str, granularity: str) -> str:
"""Return the canonical normalized SQLite rate table name.
The normalized history table stores all symbols and timeframes in
``rates``; use :func:`resolve_rate_view_name` for per-symbol compatibility
view names.
Returns:
Canonical normalized rates table name.
Raises:
ValueError: If ``symbol`` or ``granularity`` is invalid.
"""
parse_timeframe(granularity)
if not symbol.strip():
msg = "symbol must not be empty."
raise ValueError(msg)
return Dataset.rates.table_name
SqliteConnOrPath = sqlite3.Connection | Path | str
@@ -652,34 +673,76 @@ def resolve_rate_tables(
conn.close()
if TYPE_CHECKING:
@overload
def load_rate_series_from_sqlite(
conn_or_path: SqliteConnOrPath,
targets: None = None,
count: int | None = None,
explicit_tables: None = None,
*,
table: str,
) -> pd.DataFrame: ...
@overload
def load_rate_series_from_sqlite(
conn_or_path: SqliteConnOrPath,
targets: None = None,
count: int | None = None,
explicit_tables: Sequence[str] | None = None,
*,
table: None = None,
) -> dict[tuple[str | None, int], pd.DataFrame]: ...
@overload
def load_rate_series_from_sqlite(
conn_or_path: SqliteConnOrPath,
targets: Sequence[RateTarget],
count: int,
explicit_tables: Sequence[str] | None = None,
*,
table: None = None,
) -> dict[tuple[str | None, int], pd.DataFrame]: ...
def load_rate_series_from_sqlite(
conn_or_path: SqliteConnOrPath,
targets: Sequence[RateTarget],
count: int,
targets: Sequence[RateTarget] | None = None,
count: int | None = None,
explicit_tables: Sequence[str] | None = None,
) -> dict[tuple[str | None, int], pd.DataFrame]:
"""Load multiple rate series from a SQLite database.
*,
table: str | None = None,
) -> dict[tuple[str | None, int], pd.DataFrame] | pd.DataFrame:
"""Load one table/view or multiple rate series from a SQLite database.
Args:
conn_or_path: SQLite database path or open connection.
targets: Rate targets to load. Each ``(symbol, timeframe_int)`` pair
must be unique.
count: Number of most recent rows to load per series.
targets: Rate targets to load. Each ``(symbol, timeframe_int)`` pair must
be unique. Omit when loading a single explicit ``table``.
count: Optional number of most recent rows to load per series.
explicit_tables: Optional explicit table or view names matching targets.
When omitted, managed ``rate_*`` compatibility views must already
exist in the database.
table: Optional single table or view name to load directly.
Returns:
Mapping keyed by ``(symbol, timeframe_int)`` to each rate DataFrame.
A DataFrame when ``table`` is provided, otherwise a mapping keyed by
``(symbol, timeframe_int)`` to each rate DataFrame.
Raises:
ValueError: If ``count`` is not positive, targets are empty, duplicate
``(symbol, timeframe_int)`` pairs are present, or table resolution
fails.
"""
if count <= 0:
if table is not None:
return load_rate_data(conn_or_path, table, count=count)
if count is None or count <= 0:
msg = "count must be positive."
raise ValueError(msg)
if targets is None:
msg = "targets are required when table is not provided."
raise ValueError(msg)
target_list = list(targets)
if not target_list:
msg = "At least one rate target is required."
@@ -1332,6 +1395,50 @@ def create_rate_compatibility_views(conn: sqlite3.Connection) -> None:
)
def _stream_symbol_frames(
conn: sqlite3.Connection,
symbols: Sequence[str],
dataset: Dataset,
if_exists: IfExists,
written_columns: dict[Dataset, set[str]],
fetch_frame: Callable[[str], pd.DataFrame],
) -> bool:
"""Stream per-symbol frames into SQLite.
Returns:
True if the dataset table was written.
"""
table_exists = False
for sym in symbols:
table_exists = write_streamed_frame(
conn,
fetch_frame(sym),
dataset,
table_exists,
if_exists,
written_columns,
)
return table_exists
def _record_symbol_time_dedup(
dedup_scopes: dict[Dataset, list[DedupScope]],
written_tables: set[Dataset],
dataset: Dataset,
symbol: str,
start_date: datetime,
) -> None:
"""Record a symbol-scoped deduplication window after an incremental write."""
written_tables.add(dataset)
_record_dedup_scope(
dedup_scopes,
dataset,
"symbol = ? AND time >= ?",
(symbol, start_date),
frozenset({"symbol", "time"}),
)
def write_rates_dataset(
conn: sqlite3.Connection,
client: Mt5DataClient,
@@ -1347,8 +1454,8 @@ def write_rates_dataset(
Returns:
True if the rates table was written.
"""
table_exists = False
for sym in symbols:
def _fetch_rates_frame(sym: str) -> pd.DataFrame:
frame = client.copy_rates_range_as_df(
symbol=sym,
timeframe=timeframe,
@@ -1358,15 +1465,16 @@ def write_rates_dataset(
if len(frame.columns) != 0:
frame.insert(0, "symbol", sym)
frame.insert(1, "timeframe", timeframe)
table_exists = write_streamed_frame(
conn,
frame,
Dataset.rates,
table_exists,
if_exists,
written_columns,
)
return table_exists
return frame
return _stream_symbol_frames(
conn,
symbols,
Dataset.rates,
if_exists,
written_columns,
_fetch_rates_frame,
)
def write_ticks_dataset(
@@ -1384,8 +1492,8 @@ def write_ticks_dataset(
Returns:
True if the ticks table was written.
"""
table_exists = False
for sym in symbols:
def _fetch_ticks_frame(sym: str) -> pd.DataFrame:
frame = client.copy_ticks_range_as_df(
symbol=sym,
date_from=date_from,
@@ -1394,15 +1502,16 @@ def write_ticks_dataset(
).drop(columns=["symbol"], errors="ignore")
if len(frame.columns) != 0:
frame.insert(0, "symbol", sym)
table_exists = write_streamed_frame(
conn,
frame,
Dataset.ticks,
table_exists,
if_exists,
written_columns,
)
return table_exists
return frame
return _stream_symbol_frames(
conn,
symbols,
Dataset.ticks,
if_exists,
written_columns,
_fetch_ticks_frame,
)
def write_history_dataset(
@@ -1437,22 +1546,22 @@ def write_history_dataset(
if_exists,
written_columns,
)
for sym in symbols:
frame = fetch(date_from=date_from, date_to=date_to, symbol=sym)
frame = filter_trade_history_frame(
frame,
def _fetch_history_frame(sym: str) -> pd.DataFrame:
return filter_trade_history_frame(
fetch(date_from=date_from, date_to=date_to, symbol=sym),
[sym],
include_account_events=False,
)
table_exists = write_streamed_frame(
conn,
frame,
dataset,
table_exists,
if_exists,
written_columns,
)
return table_exists
return _stream_symbol_frames(
conn,
symbols,
dataset,
if_exists,
written_columns,
_fetch_history_frame,
)
def _write_incremental_rates(
@@ -1525,13 +1634,12 @@ def _write_incremental_ticks(
IfExists.APPEND,
written_columns,
):
written_tables.add(Dataset.ticks)
_record_dedup_scope(
_record_symbol_time_dedup(
dedup_scopes,
written_tables,
Dataset.ticks,
"symbol = ? AND time >= ?",
(symbol, start_date),
frozenset({"symbol", "time"}),
symbol,
start_date,
)
@@ -1564,13 +1672,12 @@ def _write_incremental_history_orders(
written_columns,
include_account_events=False,
):
written_tables.add(Dataset.history_orders)
_record_dedup_scope(
_record_symbol_time_dedup(
dedup_scopes,
written_tables,
Dataset.history_orders,
"symbol = ? AND time >= ?",
(symbol, start_date),
frozenset({"symbol", "time"}),
symbol,
start_date,
)
@@ -1662,13 +1769,12 @@ def _write_incremental_history_deals(
written_columns,
include_account_events=False,
):
written_tables.add(Dataset.history_deals)
_record_dedup_scope(
_record_symbol_time_dedup(
dedup_scopes,
written_tables,
Dataset.history_deals,
"symbol = ? AND time >= ?",
(symbol, start_date),
frozenset({"symbol", "time"}),
symbol,
start_date,
)
+64
View File
@@ -0,0 +1,64 @@
"""Retry and reconnect helpers for transient MT5 failures."""
from __future__ import annotations
import logging
import time
from typing import TYPE_CHECKING, TypeVar
from .exceptions import is_recoverable_mt5_error
if TYPE_CHECKING:
from collections.abc import Callable
T = TypeVar("T")
logger = logging.getLogger(__name__)
__all__ = [
"retry_with_backoff",
]
def retry_with_backoff(
fn: Callable[[], T],
*,
retry_count: int = 0,
backoff_base: float = 2.0,
operation: str = "MT5 operation",
) -> T:
"""Call ``fn`` with bounded exponential backoff on recoverable MT5 errors.
Only ``pdmt5.Mt5RuntimeError`` and ``pdmt5.Mt5TradingError`` are retried.
Other exceptions propagate immediately. The final failure is re-raised once
retries are exhausted.
Args:
fn: Callable performing MT5 work.
retry_count: Maximum number of retries after the first attempt. ``0``
disables retries.
backoff_base: Base for exponential backoff. The delay before retry
attempt ``n`` (1-indexed) is ``backoff_base ** n`` seconds.
operation: Label used in warning logs.
Returns:
Value returned by ``fn`` on success.
"""
attempts = max(retry_count, 0) + 1
for attempt in range(attempts - 1):
try:
return fn()
except Exception as exc:
if not is_recoverable_mt5_error(exc):
raise
delay = backoff_base ** (attempt + 1)
logger.warning(
"%s failed (attempt %d/%d): %s; retrying in %.1fs",
operation,
attempt + 1,
attempts,
exc,
delay,
)
time.sleep(delay)
return fn()
+291
View File
@@ -0,0 +1,291 @@
"""Canonical DataFrame schemas for MT5 market and account datasets."""
from __future__ import annotations
from enum import StrEnum
from typing import TYPE_CHECKING, Final
import pandas as pd
from .converters import normalize_symbol, parse_timeframe
from .exceptions import Mt5SchemaError
if TYPE_CHECKING:
from collections.abc import Iterable
__all__ = [
"DEDUP_KEYS",
"KNOWN_MT5_TIME_COLUMNS",
"REQUIRED_COLUMNS",
"TIME_COLUMNS",
"DataKind",
"normalize_dataframe",
"normalize_time_columns",
"schema_columns",
"validate_schema",
]
KNOWN_MT5_TIME_COLUMNS: Final[frozenset[str]] = frozenset({
"time",
"time_setup",
"time_setup_msc",
"time_done",
"time_done_msc",
"time_msc",
})
_TIME_COLUMN_NAMES = KNOWN_MT5_TIME_COLUMNS
class DataKind(StrEnum):
"""Supported MT5 dataset kinds with canonical column contracts."""
rates = "rates"
ticks = "ticks"
orders = "orders"
positions = "positions"
history_orders = "history_orders"
history_deals = "history_deals"
REQUIRED_COLUMNS: dict[DataKind, frozenset[str]] = {
DataKind.rates: frozenset({
"time",
"open",
"high",
"low",
"close",
"tick_volume",
"spread",
"real_volume",
}),
DataKind.ticks: frozenset({
"time",
"bid",
"ask",
"last",
"volume",
"time_msc",
"flags",
"volume_real",
}),
DataKind.orders: frozenset({
"ticket",
"time_setup",
"type",
"state",
"symbol",
"volume_current",
"price_open",
}),
DataKind.positions: frozenset({
"ticket",
"time",
"type",
"symbol",
"volume",
"price_open",
"price_current",
"profit",
}),
DataKind.history_orders: frozenset({
"ticket",
"time_setup",
"type",
"state",
"symbol",
"volume_initial",
"price_open",
}),
DataKind.history_deals: frozenset({
"ticket",
"order",
"time",
"type",
"entry",
"symbol",
"volume",
"price",
"profit",
}),
}
_OPTIONAL_TIME_COLUMNS_BY_KIND: dict[DataKind, frozenset[str]] = {
DataKind.orders: frozenset({
"time_setup_msc",
"time_done",
"time_done_msc",
}),
DataKind.history_orders: frozenset({
"time_setup_msc",
"time_done",
"time_done_msc",
}),
DataKind.positions: frozenset({"time_msc"}),
}
TIME_COLUMNS: dict[DataKind, frozenset[str]] = {
kind: (REQUIRED_COLUMNS[kind] & _TIME_COLUMN_NAMES)
| _OPTIONAL_TIME_COLUMNS_BY_KIND.get(kind, frozenset())
for kind in DataKind
}
DEDUP_KEYS: dict[DataKind, tuple[tuple[str, ...], ...]] = {
DataKind.rates: (("symbol", "timeframe", "time"), ("symbol", "time")),
DataKind.ticks: (("symbol", "time_msc"), ("symbol", "time")),
DataKind.history_orders: (("ticket",), ("symbol", "time", "type")),
DataKind.history_deals: (("ticket",), ("symbol", "time", "type", "entry")),
}
def schema_columns(kind: DataKind) -> frozenset[str]:
"""Return required column names for a dataset kind.
Args:
kind: Dataset kind.
Returns:
Required column names for ``kind``.
"""
return REQUIRED_COLUMNS[kind]
def validate_schema(
frame: pd.DataFrame,
kind: DataKind,
*,
extra_required: Iterable[str] | None = None,
) -> None:
"""Validate that a DataFrame includes required columns for a dataset kind.
Args:
frame: DataFrame to validate.
kind: Expected dataset kind.
extra_required: Additional columns that must be present (for example
``symbol`` and ``timeframe`` on stored rate history).
Raises:
Mt5SchemaError: If required columns are missing.
"""
if frame.empty and len(frame.columns) == 0:
return
required = set(REQUIRED_COLUMNS[kind])
if extra_required is not None:
required.update(extra_required)
missing = required - set(frame.columns)
if missing:
msg = (
f"{kind.value} schema is missing required columns: "
f"{', '.join(sorted(missing))}."
)
raise Mt5SchemaError(msg)
def _coerce_mt5_time_column(series: pd.Series, column: str) -> pd.Series:
"""Coerce one MT5 time column to UTC-aware datetimes.
Returns:
Series with UTC-aware datetime values.
"""
if pd.api.types.is_datetime64_any_dtype(series):
return pd.to_datetime(series, utc=True, errors="coerce")
if pd.api.types.is_numeric_dtype(series):
unit = "ms" if column.endswith("_msc") else "s"
return pd.to_datetime(series, unit=unit, utc=True, errors="coerce")
return pd.to_datetime(series, utc=True, errors="coerce")
def normalize_time_columns(frame: pd.DataFrame, kind: DataKind) -> pd.DataFrame:
"""Coerce dataset time columns to UTC-aware datetimes when present.
Any column in :data:`KNOWN_MT5_TIME_COLUMNS` that is present in ``frame``
is normalized. Numeric MT5 epoch values use seconds for ``time``,
``time_setup``, and ``time_done``, and milliseconds for ``*_msc`` columns.
Args:
frame: Source DataFrame from MT5 or pdmt5.
kind: Dataset kind (retained for API compatibility).
Returns:
DataFrame copy with normalized time columns.
"""
del kind
normalized = frame.copy()
for column in normalized.columns:
if column not in _TIME_COLUMN_NAMES:
continue
normalized[column] = _coerce_mt5_time_column(normalized[column], column)
return normalized
def normalize_dataframe(
frame: pd.DataFrame,
kind: DataKind,
*,
symbol: str | None = None,
timeframe: int | str | None = None,
sort: bool = True,
) -> pd.DataFrame:
"""Normalize MT5 DataFrame columns, timestamps, and storage metadata.
Ensures UTC timestamps, optionally injects ``symbol`` / ``timeframe`` for
storage-oriented datasets, and sorts chronologically when a ``time`` column
exists.
Args:
frame: Source DataFrame from MT5 or pdmt5.
kind: Dataset kind guiding normalization rules.
symbol: Optional symbol to inject when missing.
timeframe: Optional timeframe integer or name to inject for rates.
sort: Whether to sort by ``time`` or ``time_msc`` when present.
Returns:
Normalized DataFrame copy.
"""
if frame.empty and len(frame.columns) == 0:
return frame.copy()
normalized = normalize_time_columns(frame, kind)
if symbol is not None and "symbol" not in normalized.columns:
normalized.insert(0, "symbol", normalize_symbol(symbol))
if timeframe is not None and kind is DataKind.rates:
tf = parse_timeframe(timeframe)
if "timeframe" not in normalized.columns:
insert_at = 1 if "symbol" in normalized.columns else 0
normalized.insert(insert_at, "timeframe", tf)
validate_schema(normalized, kind)
if sort:
if "time" in normalized.columns:
normalized = normalized.sort_values("time", kind="stable")
elif "time_msc" in normalized.columns:
normalized = normalized.sort_values("time_msc", kind="stable")
normalized = normalized.reset_index(drop=True)
return normalized
def ensure_utc_columns(frame: pd.DataFrame, columns: Iterable[str]) -> pd.DataFrame:
"""Return a copy with selected columns coerced to UTC datetimes.
Args:
frame: Source DataFrame.
columns: Column names to coerce.
Returns:
DataFrame copy with UTC-aware datetime columns.
"""
normalized = frame.copy()
for column in columns:
if column not in normalized.columns:
continue
if column in _TIME_COLUMN_NAMES:
normalized[column] = _coerce_mt5_time_column(normalized[column], column)
else:
normalized[column] = pd.to_datetime(
normalized[column], utc=True, errors="coerce"
)
return normalized
+50 -35
View File
@@ -29,6 +29,7 @@ from .history import (
write_collected_datasets,
write_incremental_datasets,
)
from .retry import retry_with_backoff
from .utils import (
Dataset,
IfExists,
@@ -36,6 +37,7 @@ from .utils import (
parse_tick_flags,
parse_timeframe,
)
from .utils import coerce_login as _coerce_login
if TYPE_CHECKING:
from collections.abc import Callable, Iterator, Sequence
@@ -64,6 +66,9 @@ _MT5_HISTORY_CLIENT_CALL_FUNCTIONS: frozenset[str] = frozenset({
"write_ticks_dataset",
"write_history_dataset",
"_write_incremental_history_deals",
"_fetch_rates_frame",
"_fetch_ticks_frame",
"_fetch_history_frame",
})
_NON_CALLABLE_TYPE_ERROR = re.compile(r"^'[^']+' object is not callable$")
@@ -112,11 +117,13 @@ __all__ = [
"collect_latest_rates",
"collect_latest_rates_for_accounts",
"collect_latest_rates_for_accounts_with_retries",
"connected_client",
"copy_rates_from",
"copy_rates_from_pos",
"copy_rates_range",
"copy_ticks_from",
"copy_ticks_range",
"fetch_latest_closed_rates",
"history_deals",
"history_orders",
"last_error",
@@ -316,7 +323,7 @@ def build_config(
@contextmanager
def _connected_client(config: Mt5Config) -> Iterator[Mt5DataClient]:
def connected_client(config: Mt5Config) -> Iterator[Mt5DataClient]:
"""Initialize MT5, yield a connected client, and always shut down.
Args:
@@ -346,7 +353,7 @@ def _run_with_client(
Returns:
Value returned by ``fetch_fn``.
"""
with _connected_client(config) as client:
with connected_client(config) as client:
return fetch_fn(client)
@@ -366,7 +373,7 @@ def mt5_session(config: Mt5Config | None = None) -> Iterator[Mt5CliClient]:
Connected ``Mt5CliClient`` bound to the session.
"""
mt5_config = config or build_config()
with _connected_client(mt5_config) as client:
with connected_client(mt5_config) as client:
yield Mt5CliClient.from_connected_client(client)
@@ -381,6 +388,7 @@ class Mt5CliClient:
password: str | None = None,
server: str | None = None,
timeout: int | None = None,
retry_count: int = 3,
config: Mt5Config | None = None,
client: Mt5DataClient | None = None,
) -> None:
@@ -392,6 +400,8 @@ class Mt5CliClient:
password: Trading account password.
server: Trading server name.
timeout: Connection timeout in milliseconds.
retry_count: Number of MT5 initialization retries for sessions
opened by this client.
config: Optional pre-built ``Mt5Config`` (overrides other args).
client: Optional already-connected ``Mt5DataClient``. Injected
clients are reused as-is and are not initialized or shut down.
@@ -403,6 +413,7 @@ class Mt5CliClient:
server=server,
timeout=timeout,
)
self._retry_count = retry_count
self._client = client
self._owns_client = client is None
@@ -431,7 +442,7 @@ class Mt5CliClient:
"""
if self._client is not None:
return self
client = Mt5DataClient(config=self._config)
client = Mt5DataClient(config=self._config, retry_count=self._retry_count)
try:
client.initialize_and_login_mt5()
except Exception:
@@ -1013,7 +1024,7 @@ def update_history_with_config( # noqa: PLR0913
if request is None:
return
mt5_config = config or build_config()
with _connected_client(mt5_config) as client:
with connected_client(mt5_config) as client:
update_history(
client=client,
output=output,
@@ -1192,7 +1203,7 @@ def collect_history(
tf = _coerce_timeframe(timeframe)
tick_flags = _coerce_tick_flags(flags)
mt5_config = config or build_config()
with _connected_client(mt5_config) as client, sqlite3.connect(output) as conn:
with connected_client(mt5_config) as client, sqlite3.connect(output) as conn:
conn.execute("PRAGMA journal_mode=WAL")
conn.execute("PRAGMA synchronous=NORMAL")
written_tables, written_columns = write_collected_datasets(
@@ -1280,6 +1291,33 @@ def latest_rates(
)
def fetch_latest_closed_rates(
client: Mt5CliClient,
*,
symbol: str,
granularity: str,
count: int,
) -> pd.DataFrame:
"""Fetch up to ``count`` most recent closed bars, oldest to newest.
Returns:
Closed rate bars ordered oldest to newest.
Raises:
ValueError: If ``count`` is not positive or no closed bars are returned.
"""
_require_positive(count, "count")
frame = client.latest_rates(symbol, granularity, count + 1, start_pos=0)
closed = drop_forming_rate_bar(frame)
if closed.empty:
msg = (
f"Rate data is empty for {symbol!r} at granularity {granularity!r} "
f"with count {count}."
)
raise ValueError(msg)
return closed.tail(count).reset_index(drop=True)
def collect_latest_rates(
symbols: Sequence[str],
timeframes: Sequence[int | str],
@@ -1461,20 +1499,6 @@ def resolve_account_specs(
]
def _coerce_login(login: int | str | None) -> int | None:
"""Coerce a login value to int, treating empty strings as unset.
Returns:
Integer login, or None when unset or an empty string.
"""
if login is None or isinstance(login, int):
return login
text = login.strip()
if not text:
return None
return int(text)
def _build_account_config(
account: AccountSpec,
base_config: Mt5Config | None,
@@ -1588,7 +1612,6 @@ def collect_latest_rates_for_accounts_with_retries(
re-raises the last ``pdmt5.Mt5TradingError`` or ``pdmt5.Mt5RuntimeError``
once retries are exhausted.
"""
attempts = max(retry_count, 0) + 1
def _collect() -> dict[tuple[str, int], pd.DataFrame]:
return collect_latest_rates_for_accounts(
@@ -1599,20 +1622,12 @@ def collect_latest_rates_for_accounts_with_retries(
base_config=base_config,
)
for attempt in range(attempts - 1):
try:
return _collect()
except (Mt5TradingError, Mt5RuntimeError) as exc:
delay = backoff_base ** (attempt + 1)
logger.warning(
"Rate collection failed (attempt %d/%d): %s; retrying in %.1fs",
attempt + 1,
attempts,
exc,
delay,
)
time.sleep(delay)
return _collect()
return retry_with_backoff(
_collect,
retry_count=retry_count,
backoff_base=backoff_base,
operation="Rate collection",
)
def collect_latest_closed_rates_for_accounts(
+49
View File
@@ -0,0 +1,49 @@
"""Generic storage helpers for MT5 market and account history."""
from __future__ import annotations
from .history import (
RateTarget,
build_rate_targets,
build_rate_view_name,
drop_forming_rate_bar,
load_rate_data,
load_rate_data_from_connection,
load_rate_series_by_granularity,
load_rate_series_from_sqlite,
resolve_rate_tables,
resolve_rate_view_name,
resolve_rate_view_names,
)
from .sdk import collect_history, update_history, update_history_with_config
from .utils import (
Dataset,
IfExists,
OutputFormat,
detect_format,
export_dataframe,
export_dataframe_to_sqlite,
)
__all__ = [
"Dataset",
"IfExists",
"OutputFormat",
"RateTarget",
"build_rate_targets",
"build_rate_view_name",
"collect_history",
"detect_format",
"drop_forming_rate_bar",
"export_dataframe",
"export_dataframe_to_sqlite",
"load_rate_data",
"load_rate_data_from_connection",
"load_rate_series_by_granularity",
"load_rate_series_from_sqlite",
"resolve_rate_tables",
"resolve_rate_view_name",
"resolve_rate_view_names",
"update_history",
"update_history_with_config",
]
+688 -37
View File
@@ -3,27 +3,98 @@
from __future__ import annotations
from contextlib import contextmanager
from typing import TYPE_CHECKING, Literal
from math import floor, isfinite
from typing import TYPE_CHECKING, Literal, cast
from pdmt5 import Mt5Config, Mt5TradingClient
import pandas as pd
from pdmt5 import Mt5Config, Mt5TradingClient, Mt5TradingError
from .sdk import build_config
from .utils import coerce_login as _coerce_login
if TYPE_CHECKING:
from collections.abc import Iterator
import pandas as pd
PositionSide = Literal["long", "short"]
OrderSide = Literal["long", "short"]
OrderSide = Literal["BUY", "SELL"]
OrderFillingMode = Literal["IOC", "FOK", "RETURN"]
OrderTimeMode = Literal["GTC", "DAY", "SPECIFIED", "SPECIFIED_DAY"]
_ORDER_FILLING_MODES: frozenset[str] = frozenset({"IOC", "FOK", "RETURN"})
_ORDER_TIME_MODES: frozenset[str] = frozenset({
"GTC",
"DAY",
"SPECIFIED",
"SPECIFIED_DAY",
})
_SUCCESS_RETCODE_NAMES: tuple[str, ...] = (
"TRADE_RETCODE_DONE",
"TRADE_RETCODE_DONE_PARTIAL",
"TRADE_RETCODE_PLACED",
)
_SUCCESS_RETCODE_FALLBACKS: frozenset[int] = frozenset({10008, 10009, 10010})
_ACCOUNT_SNAPSHOT_FIELDS = (
"login",
"balance",
"equity",
"margin",
"margin_free",
"margin_level",
"leverage",
"currency",
)
_SYMBOL_SNAPSHOT_FIELDS = (
"symbol",
"visible",
"trade_mode",
"digits",
"point",
"volume_min",
"volume_max",
"volume_step",
"trade_contract_size",
"trade_tick_size",
"trade_tick_value",
"trade_stops_level",
"filling_mode",
)
_TICK_SNAPSHOT_FIELDS = ("symbol", "time", "bid", "ask", "last", "volume")
POSITION_COLUMNS = (
"ticket",
"time",
"symbol",
"type",
"volume",
"price_open",
"sl",
"tp",
"price_current",
"profit",
"swap",
"comment",
)
__all__ = [
"POSITION_COLUMNS",
"OrderFillingMode",
"OrderSide",
"OrderTimeMode",
"PositionSide",
"calculate_margin_and_volume",
"calculate_new_position_margin_ratio",
"calculate_spread_ratio",
"calculate_volume_by_margin",
"close_open_positions",
"create_trading_client",
"detect_position_side",
"determine_order_limits",
"get_account_snapshot",
"get_positions_frame",
"get_symbol_snapshot",
"get_tick_snapshot",
"mt5_trading_session",
"place_market_order",
"update_sltp_for_open_positions",
]
@@ -46,18 +117,184 @@ def _sum_position_volume(positions: pd.DataFrame, position_type: object) -> floa
return float(matched.to_numpy(dtype=float).sum())
def _resolve_config(
*,
config: Mt5Config | None,
login: int | str | None,
password: str | None,
server: str | None,
path: str | None,
timeout: int | None,
) -> Mt5Config:
if config is not None:
return config
return build_config(
path=path,
login=_coerce_login(login),
password=password,
server=server,
timeout=timeout,
)
def _normalize_order_side(side: str) -> OrderSide:
normalized = side.upper()
if normalized in {"BUY", "LONG"}:
return "BUY"
if normalized in {"SELL", "SHORT"}:
return "SELL"
msg = f"Unsupported order side: {side!r}. Expected 'BUY' or 'SELL'."
raise ValueError(msg)
def _position_side_from_order_side(side: str) -> PositionSide:
normalized = side.lower()
if normalized in {"long", "buy"}:
return "long"
if normalized in {"short", "sell"}:
return "short"
msg = (
f"Unsupported order side: {side!r}. Expected 'long', 'short', 'buy', or 'sell'."
)
msg = f"Unsupported position side: {side!r}. Expected 'long' or 'short'."
raise ValueError(msg)
def _snapshot_from_value(value: object, fields: tuple[str, ...]) -> dict[str, object]:
if isinstance(value, pd.DataFrame):
row: dict[str, object] = (
{} if value.empty else cast("dict[str, object]", value.iloc[0].to_dict())
)
else:
asdict = getattr(value, "_asdict", None)
if callable(asdict):
row = cast("dict[str, object]", asdict())
elif isinstance(value, dict):
typed_value = cast("dict[object, object]", value)
row = {str(key): item for key, item in typed_value.items()}
else:
row = {
field: getattr(value, field)
for field in fields
if hasattr(value, field)
}
if not fields:
return row
return {field: row.get(field) for field in fields}
def _call_snapshot_method(client: Mt5TradingClient, *names: str) -> object:
for name in names:
method = getattr(client, name, None)
if callable(method):
return method()
msg = f"MT5 client is missing required method: {' or '.join(names)}"
raise AttributeError(msg)
def _resolve_mt5_constant(
mt5: object,
prefix: str,
value: str,
allowed: frozenset[str],
) -> int:
normalized = value.upper()
if normalized not in allowed:
msg = f"Unsupported {prefix.lower()} mode: {value!r}."
raise ValueError(msg)
name = f"{prefix}_{normalized}"
try:
return cast("int", getattr(mt5, name))
except AttributeError as exc:
msg = f"MT5 module is missing required constant: {name}"
raise Mt5TradingError(msg) from exc
def _optional_price(value: object) -> float | None:
if value is None:
return None
if not isinstance(value, int | float):
return None
price = float(value)
if price <= 0 or not isfinite(price):
return None
return price
def _success_retcodes(mt5: object) -> frozenset[int]:
values = {
value
for name in _SUCCESS_RETCODE_NAMES
if isinstance(value := getattr(mt5, name, None), int)
}
return frozenset(values) or _SUCCESS_RETCODE_FALLBACKS
def _order_status_from_retcode(mt5: object, retcode: object) -> str:
if retcode is None:
return "executed"
if isinstance(retcode, int) and retcode not in _success_retcodes(mt5):
return "failed"
return "executed"
def _calculate_min_volume_if_affordable(
client: Mt5TradingClient,
symbol: str,
available_margin: float,
order_side: OrderSide,
) -> float:
if available_margin <= 0:
return 0.0
symbol_info = get_symbol_snapshot(client, symbol)
volume_min = float(symbol_info.get("volume_min") or 0.0)
volume_max = float(symbol_info.get("volume_max") or 0.0)
volume_step = float(symbol_info.get("volume_step") or volume_min or 0.0)
if (
volume_min <= 0
or volume_step <= 0
or (volume_max > 0 and volume_min > volume_max)
):
msg = f"Invalid volume constraints for {symbol!r}."
raise Mt5TradingError(msg)
side = _normalize_order_side(order_side)
tick = get_tick_snapshot(client, symbol)
price = tick["ask"] if side == "BUY" else tick["bid"]
if not isinstance(price, int | float) or price <= 0:
msg = f"Tick price is unavailable for {symbol!r}."
raise Mt5TradingError(msg)
order_type = (
client.mt5.ORDER_TYPE_BUY if side == "BUY" else client.mt5.ORDER_TYPE_SELL
)
min_margin = float(client.order_calc_margin(order_type, symbol, volume_min, price))
return round(volume_min, 10) if 0 < min_margin <= available_margin else 0.0
def create_trading_client(
*,
config: Mt5Config | None = None,
login: int | str | None = None,
password: str | None = None,
server: str | None = None,
path: str | None = None,
timeout: int | None = None,
retry_count: int = 0,
) -> Mt5TradingClient:
"""Return an initialized and logged-in trading client."""
mt5_config = _resolve_config(
config=config,
login=login,
password=password,
server=server,
path=path,
timeout=timeout,
)
client = Mt5TradingClient(config=mt5_config, retry_count=retry_count)
try:
client.initialize_and_login_mt5()
except Exception:
client.shutdown()
raise
return client
def detect_position_side(
client: Mt5TradingClient,
symbol: str,
@@ -69,11 +306,11 @@ def detect_position_side(
symbol: Symbol to inspect.
Returns:
``"long"`` when net buy volume exceeds sell volume, ``"short"`` when
net sell volume exceeds buy volume, or ``None`` when no positions exist
or buy/sell volumes are exactly balanced.
``"long"`` when there are buy positions and no sell positions,
``"short"`` when there are sell positions and no buy positions, or
``None`` when no positions or mixed exposure exists.
"""
positions = client.positions_get_as_df(symbol=symbol)
positions = get_positions_frame(client, symbol=symbol)
if positions.empty:
return None
@@ -81,14 +318,114 @@ def detect_position_side(
sell_type = client.mt5.POSITION_TYPE_SELL
buy_volume = _sum_position_volume(positions, buy_type)
sell_volume = _sum_position_volume(positions, sell_type)
net_volume = buy_volume - sell_volume
if net_volume > 0:
if buy_volume > 0 and sell_volume == 0:
return "long"
if net_volume < 0:
if sell_volume > 0 and buy_volume == 0:
return "short"
return None
def get_account_snapshot(
client: Mt5TradingClient,
) -> dict[str, float | int | str | None]:
"""Return normalized account state with stable keys."""
value = _call_snapshot_method(client, "account_info_as_dict", "account_info")
return cast(
"dict[str, float | int | str | None]",
_snapshot_from_value(value, _ACCOUNT_SNAPSHOT_FIELDS),
)
def get_symbol_snapshot(
client: Mt5TradingClient,
symbol: str,
) -> dict[str, float | int | str | bool | None]:
"""Return normalized symbol metadata required for trading decisions."""
method = getattr(client, "symbol_info_as_dict", None)
value = method(symbol=symbol) if callable(method) else client.symbol_info(symbol)
snapshot = _snapshot_from_value(value, _SYMBOL_SNAPSHOT_FIELDS)
snapshot["symbol"] = snapshot.get("symbol") or symbol
return cast("dict[str, float | int | str | bool | None]", snapshot)
def get_tick_snapshot(
client: Mt5TradingClient,
symbol: str,
) -> dict[str, float | int | None]:
"""Return normalized latest tick data, including bid, ask, and timestamp."""
method = getattr(client, "symbol_info_tick_as_dict", None)
value = (
method(symbol=symbol) if callable(method) else client.symbol_info_tick(symbol)
)
snapshot = _snapshot_from_value(value, _TICK_SNAPSHOT_FIELDS)
snapshot["symbol"] = snapshot.get("symbol") or symbol
return cast("dict[str, float | int | None]", snapshot)
def get_positions_frame(
client: Mt5TradingClient,
symbol: str | None = None,
) -> pd.DataFrame:
"""Return open positions as a DataFrame with stable baseline columns."""
frame = client.positions_get_as_df(symbol=symbol)
for column in POSITION_COLUMNS:
if column not in frame.columns:
frame[column] = pd.Series(dtype="object")
return frame
def calculate_spread_ratio(client: Mt5TradingClient, symbol: str) -> float:
"""Return ``(ask - bid) / ((ask + bid) / 2)`` for the latest tick.
Raises:
Mt5TradingError: If bid or ask is unavailable or non-positive.
"""
tick = get_tick_snapshot(client, symbol)
bid = tick.get("bid")
ask = tick.get("ask")
if not isinstance(bid, int | float) or not isinstance(ask, int | float):
msg = f"Tick bid/ask is unavailable for {symbol!r}."
raise Mt5TradingError(msg)
if bid <= 0 or ask <= 0:
msg = f"Tick bid/ask must be positive for {symbol!r}."
raise Mt5TradingError(msg)
return (float(ask) - float(bid)) / ((float(ask) + float(bid)) / 2.0)
def calculate_new_position_margin_ratio(
client: Mt5TradingClient,
*,
symbol: str,
new_position_side: OrderSide | None = None,
new_position_volume: float = 0.0,
) -> float:
"""Return total margin/equity ratio after an optional hypothetical position.
Raises:
Mt5TradingError: If equity or required tick data is invalid.
"""
account = get_account_snapshot(client)
equity = float(account.get("equity") or 0.0)
if equity <= 0:
msg = "Account equity must be positive to calculate margin ratio."
raise Mt5TradingError(msg)
margin = float(account.get("margin") or 0.0)
if new_position_side is not None and new_position_volume > 0:
side = _normalize_order_side(new_position_side)
tick = get_tick_snapshot(client, symbol)
price = tick["ask"] if side == "BUY" else tick["bid"]
if not isinstance(price, int | float) or price <= 0:
msg = f"Tick price is unavailable for {symbol!r}."
raise Mt5TradingError(msg)
order_type = (
client.mt5.ORDER_TYPE_BUY if side == "BUY" else client.mt5.ORDER_TYPE_SELL
)
margin += float(
client.order_calc_margin(order_type, symbol, new_position_volume, price),
)
return margin / equity
def calculate_margin_and_volume(
client: Mt5TradingClient,
symbol: str,
@@ -99,7 +436,9 @@ def calculate_margin_and_volume(
Applies ``preserved_margin_ratio`` to keep a reserve off ``margin_free``,
then allocates ``unit_margin_ratio`` of the remainder as the margin budget
for volume sizing on both buy and sell sides.
for proportional volume sizing on both buy and sell sides. A
``unit_margin_ratio`` of ``0`` requests exactly one minimum valid unit per
side when the post-reserve margin can afford it.
Args:
client: Connected ``Mt5TradingClient`` instance.
@@ -119,23 +458,109 @@ def calculate_margin_and_volume(
margin_free = max(0.0, float(account.get("margin_free") or 0.0))
available_margin = margin_free * (1.0 - preserved_margin_ratio)
trade_margin = available_margin * unit_margin_ratio
buy_volume = client.calculate_volume_by_margin(symbol, trade_margin, "BUY")
sell_volume = client.calculate_volume_by_margin(symbol, trade_margin, "SELL")
if unit_margin_ratio == 0:
buy_volume = _calculate_min_volume_if_affordable(
client,
symbol,
available_margin,
"BUY",
)
sell_volume = _calculate_min_volume_if_affordable(
client,
symbol,
available_margin,
"SELL",
)
else:
native_calculate_volume = getattr(client, "calculate_volume_by_margin", None)
if callable(native_calculate_volume):
buy_volume = float(
cast(
"float | int | str",
native_calculate_volume(symbol, trade_margin, "BUY"),
),
)
sell_volume = float(
cast(
"float | int | str",
native_calculate_volume(symbol, trade_margin, "SELL"),
),
)
else:
buy_volume = calculate_volume_by_margin(client, symbol, trade_margin, "BUY")
sell_volume = calculate_volume_by_margin(
client,
symbol,
trade_margin,
"SELL",
)
try:
symbol_info = get_symbol_snapshot(client, symbol)
volume_min = float(symbol_info.get("volume_min") or 0.0)
volume_max = float(symbol_info.get("volume_max") or 0.0)
volume_step = float(symbol_info.get("volume_step") or 0.0)
except AttributeError:
volume_min = volume_max = volume_step = 0.0
return {
"margin_free": margin_free,
"available_margin": available_margin,
"trade_margin": trade_margin,
"buy_volume": buy_volume,
"sell_volume": sell_volume,
"buy_volume": float(buy_volume),
"sell_volume": float(sell_volume),
"volume_min": volume_min,
"volume_max": volume_max,
"volume_step": volume_step,
}
def calculate_volume_by_margin(
client: Mt5TradingClient,
symbol: str,
available_margin: float,
order_side: OrderSide,
) -> float:
"""Calculate max normalized volume affordable for one side.
Returns:
Affordable volume rounded down to symbol volume constraints.
Raises:
Mt5TradingError: If symbol volume constraints or tick data are invalid.
"""
if available_margin <= 0:
return 0.0
symbol_info = get_symbol_snapshot(client, symbol)
volume_min = float(symbol_info.get("volume_min") or 0.0)
volume_max = float(symbol_info.get("volume_max") or 0.0)
volume_step = float(symbol_info.get("volume_step") or volume_min or 0.0)
if volume_min <= 0 or volume_step <= 0:
msg = f"Invalid volume constraints for {symbol!r}."
raise Mt5TradingError(msg)
side = _normalize_order_side(order_side)
tick = get_tick_snapshot(client, symbol)
price = tick["ask"] if side == "BUY" else tick["bid"]
if not isinstance(price, int | float) or price <= 0:
msg = f"Tick price is unavailable for {symbol!r}."
raise Mt5TradingError(msg)
order_type = (
client.mt5.ORDER_TYPE_BUY if side == "BUY" else client.mt5.ORDER_TYPE_SELL
)
min_margin = float(client.order_calc_margin(order_type, symbol, volume_min, price))
if min_margin <= 0 or min_margin > available_margin:
return 0.0
raw_volume = available_margin / min_margin * volume_min
capped = min(raw_volume, volume_max) if volume_max > 0 else raw_volume
steps = floor(((capped - volume_min) / volume_step) + 1e-12)
normalized = volume_min + max(0, steps) * volume_step
return round(normalized, 10) if normalized >= volume_min else 0.0
def determine_order_limits(
client: Mt5TradingClient,
symbol: str,
side: OrderSide | str,
stop_loss_limit_ratio: float,
take_profit_limit_ratio: float,
side: PositionSide | str,
stop_loss_limit_ratio: float | None = None,
take_profit_limit_ratio: float | None = None,
) -> dict[str, float | None]:
"""Derive entry and protective order prices from current market quotes.
@@ -152,26 +577,41 @@ def determine_order_limits(
Returns:
Dictionary with ``entry``, ``stop_loss``, and ``take_profit`` keys.
Omitted protective levels are returned as ``None``.
Raises:
Mt5TradingError: If required tick data is invalid.
"""
_require_protective_ratio(stop_loss_limit_ratio, "stop_loss_limit_ratio")
_require_protective_ratio(take_profit_limit_ratio, "take_profit_limit_ratio")
normalized_side = _normalize_order_side(side)
tick = client.symbol_info_tick_as_dict(symbol=symbol)
entry = float(tick["ask"] if normalized_side == "long" else tick["bid"])
stop_loss_ratio = stop_loss_limit_ratio or 0.0
take_profit_ratio = take_profit_limit_ratio or 0.0
_require_protective_ratio(stop_loss_ratio, "stop_loss_limit_ratio")
_require_protective_ratio(take_profit_ratio, "take_profit_limit_ratio")
normalized_side = _position_side_from_order_side(side)
tick = get_tick_snapshot(client, symbol)
entry_value = tick["ask"] if normalized_side == "long" else tick["bid"]
if not isinstance(entry_value, int | float):
msg = f"Tick price is unavailable for {symbol!r}."
raise Mt5TradingError(msg)
entry = float(entry_value)
try:
digits = int(get_symbol_snapshot(client, symbol).get("digits") or 8)
except AttributeError:
digits = 8
stop_loss: float | None = None
if stop_loss_limit_ratio > 0:
if stop_loss_ratio > 0:
if normalized_side == "long":
stop_loss = entry * (1.0 - stop_loss_limit_ratio)
stop_loss = entry * (1.0 - stop_loss_ratio)
else:
stop_loss = entry * (1.0 + stop_loss_limit_ratio)
stop_loss = entry * (1.0 + stop_loss_ratio)
stop_loss = round(stop_loss, digits)
take_profit: float | None = None
if take_profit_limit_ratio > 0:
if take_profit_ratio > 0:
if normalized_side == "long":
take_profit = entry * (1.0 + take_profit_limit_ratio)
take_profit = entry * (1.0 + take_profit_ratio)
else:
take_profit = entry * (1.0 - take_profit_limit_ratio)
take_profit = entry * (1.0 - take_profit_ratio)
take_profit = round(take_profit, digits)
return {
"entry": entry,
@@ -180,9 +620,209 @@ def determine_order_limits(
}
def place_market_order(
client: Mt5TradingClient,
*,
symbol: str,
volume: float,
order_side: OrderSide,
order_filling_mode: OrderFillingMode = "IOC",
order_time_mode: OrderTimeMode = "GTC",
sl: float | None = None,
tp: float | None = None,
position: int | None = None,
dry_run: bool = False,
) -> dict[str, object]:
"""Place one normalized market order or return a dry-run result.
``pdmt5.Mt5TradingClient.order_send()`` raises only when MT5 returns no
response. When MT5 returns a response with a known non-success retcode, this
helper returns ``status="failed"`` and keeps the normalized response
details for callers to inspect.
Returns:
Normalized execution result containing request and response details.
Raises:
Mt5TradingError: If volume or required tick data is invalid.
"""
if volume <= 0:
msg = "volume must be positive."
raise Mt5TradingError(msg)
side = _normalize_order_side(order_side)
tick = get_tick_snapshot(client, symbol)
price = tick["ask"] if side == "BUY" else tick["bid"]
if not isinstance(price, int | float) or price <= 0:
msg = f"Tick price is unavailable for {symbol!r}."
raise Mt5TradingError(msg)
request = {
"action": client.mt5.TRADE_ACTION_DEAL,
"symbol": symbol,
"volume": volume,
"type": (
client.mt5.ORDER_TYPE_BUY if side == "BUY" else client.mt5.ORDER_TYPE_SELL
),
"price": float(price),
"type_filling": _resolve_mt5_constant(
client.mt5,
"ORDER_FILLING",
order_filling_mode,
_ORDER_FILLING_MODES,
),
"type_time": _resolve_mt5_constant(
client.mt5,
"ORDER_TIME",
order_time_mode,
_ORDER_TIME_MODES,
),
}
if sl is not None:
request["sl"] = sl
if tp is not None:
request["tp"] = tp
if position is not None:
request["position"] = position
if dry_run:
return {
"status": "dry_run",
"symbol": symbol,
"order_side": side,
"volume": volume,
"retcode": None,
"comment": None,
"request": request,
"response": None,
"dry_run": True,
}
response = client.order_send(request)
response_dict = _snapshot_from_value(response, ())
retcode = response_dict.get("retcode")
return {
"status": _order_status_from_retcode(client.mt5, retcode),
"symbol": symbol,
"order_side": side,
"volume": volume,
"retcode": retcode,
"comment": response_dict.get("comment"),
"request": request,
"response": response_dict,
"dry_run": False,
}
def _filter_positions(
positions: pd.DataFrame,
*,
symbols: str | list[str] | None = None,
tickets: list[int] | None = None,
) -> pd.DataFrame:
frame = positions
if symbols is not None:
symbol_set = {symbols} if isinstance(symbols, str) else set(symbols)
frame = frame.loc[frame["symbol"].isin(symbol_set)]
if tickets is not None:
frame = frame.loc[frame["ticket"].isin(tickets)]
return frame
def close_open_positions(
client: Mt5TradingClient,
*,
symbols: str | list[str] | None = None,
tickets: list[int] | None = None,
dry_run: bool = False,
) -> list[dict[str, object]]:
"""Close matching open positions.
Returns:
Normalized execution results for matching positions.
"""
positions = _filter_positions(
get_positions_frame(client),
symbols=symbols,
tickets=tickets,
)
results: list[dict[str, object]] = []
for row in positions.to_dict("records"):
pos_type = row["type"]
side: OrderSide = "SELL" if pos_type == client.mt5.POSITION_TYPE_BUY else "BUY"
result = place_market_order(
client,
symbol=str(row["symbol"]),
volume=float(row["volume"]),
order_side=side,
position=int(row["ticket"]),
dry_run=dry_run,
)
results.append(result)
return results
def update_sltp_for_open_positions(
client: Mt5TradingClient,
*,
symbol: str | None = None,
tickets: list[int] | None = None,
stop_loss: float | None = None,
take_profit: float | None = None,
dry_run: bool = False,
) -> list[dict[str, object]]:
"""Update SL/TP for matching open positions.
Returns:
Normalized execution results for matching positions.
"""
positions = _filter_positions(
get_positions_frame(client),
symbols=symbol,
tickets=tickets,
)
results: list[dict[str, object]] = []
for row in positions.to_dict("records"):
request = {
"action": client.mt5.TRADE_ACTION_SLTP,
"symbol": row["symbol"],
"position": row["ticket"],
}
sl = _optional_price(row.get("sl") if stop_loss is None else stop_loss)
tp = _optional_price(row.get("tp") if take_profit is None else take_profit)
if sl is not None:
request["sl"] = sl
if tp is not None:
request["tp"] = tp
if dry_run:
response = None
status = "dry_run"
else:
response = _snapshot_from_value(client.order_send(request), ())
status = "executed"
results.append(
{
"status": status,
"symbol": row["symbol"],
"order_side": "BUY"
if row["type"] == client.mt5.POSITION_TYPE_BUY
else "SELL",
"volume": row["volume"],
"retcode": None if response is None else response.get("retcode"),
"comment": None if response is None else response.get("comment"),
"request": request,
"response": response,
"dry_run": dry_run,
},
)
return results
@contextmanager
def mt5_trading_session(
config: Mt5Config | None = None,
*,
login: int | str | None = None,
password: str | None = None,
server: str | None = None,
path: str | None = None,
timeout: int | None = None,
retry_count: int = 0,
) -> Iterator[Mt5TradingClient]:
"""Open a trading-capable MT5 session and always shut down safely.
@@ -195,16 +835,27 @@ def mt5_trading_session(
Args:
config: MT5 connection configuration. Defaults to an empty config that
attaches to a running terminal.
login: Optional trading account login.
password: Optional trading account password.
server: Optional trading server name.
path: Optional terminal executable path.
timeout: Optional connection timeout in milliseconds.
retry_count: Number of initialization retries passed to
``Mt5TradingClient``.
Yields:
Connected ``Mt5TradingClient`` bound to the session.
"""
mt5_config = config or build_config()
client = Mt5TradingClient(config=mt5_config, retry_count=retry_count)
client = create_trading_client(
config=config,
login=login,
password=password,
server=server,
path=path,
timeout=timeout,
retry_count=retry_count,
)
try:
client.initialize_and_login_mt5()
yield client
finally:
client.shutdown()
+14
View File
@@ -241,6 +241,20 @@ def detect_format(
raise ValueError(msg)
def coerce_login(login: int | str | None) -> int | None:
"""Coerce a login value to int, treating empty strings as unset.
Returns:
Integer login, or None when unset or an empty string.
"""
if login is None or isinstance(login, int):
return login
text = login.strip()
if not text:
return None
return int(text)
def export_dataframe_to_sqlite(
df: pd.DataFrame,
output_path: Path,
+2 -2
View File
@@ -1,7 +1,7 @@
[project]
name = "mt5cli"
version = "0.7.0"
description = "Command-line tool for MetaTrader 5"
version = "0.8.0"
description = "Generic MT5 data and execution infrastructure for Python applications"
authors = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
maintainers = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
license = "MIT"
+52
View File
@@ -0,0 +1,52 @@
"""Shared pytest fixtures for mt5cli tests."""
from __future__ import annotations
from unittest.mock import MagicMock
import pandas as pd
import pytest
from pytest_mock import MockerFixture # noqa: TC002
_DATAFRAME_METHODS = (
"copy_rates_from_as_df",
"copy_rates_from_pos_as_df",
"copy_rates_range_as_df",
"copy_ticks_from_as_df",
"copy_ticks_range_as_df",
"account_info_as_df",
"terminal_info_as_df",
"symbols_get_as_df",
"symbol_info_as_df",
"orders_get_as_df",
"positions_get_as_df",
"history_orders_get_as_df",
"history_deals_get_as_df",
"version_as_df",
"last_error_as_df",
"symbol_info_tick_as_df",
"market_book_get_as_df",
"order_check_as_df",
"order_send_as_df",
)
def build_mock_mt5_data_client() -> MagicMock:
"""Return a MagicMock Mt5DataClient with common DataFrame stubs."""
client = MagicMock()
sample_df = pd.DataFrame({"col": [1]})
for method_name in _DATAFRAME_METHODS:
getattr(client, method_name).return_value = sample_df
client.version.return_value = (5, 0, 1)
client.terminal_info.return_value = {"connected": True, "paths": ["terminal.exe"]}
client.account_info.return_value = {"login": 123, "limits": {"modes": ["demo"]}}
client.symbols_total.return_value = 42
return client
@pytest.fixture
def mock_client(mocker: MockerFixture) -> MagicMock:
"""Create and patch a mock Mt5DataClient for CLI and SDK tests."""
client = build_mock_mt5_data_client()
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=client)
return client
-32
View File
@@ -69,38 +69,6 @@ class TestExecuteExport:
# ---------------------------------------------------------------------------
@pytest.fixture
def mock_client(mocker: MockerFixture) -> MagicMock:
"""Create and patch a mock Mt5DataClient for CLI tests."""
client = MagicMock()
sample_df = pd.DataFrame({"col": [1]})
client.copy_rates_from_as_df.return_value = sample_df
client.copy_rates_from_pos_as_df.return_value = sample_df
client.copy_rates_range_as_df.return_value = sample_df
client.copy_ticks_from_as_df.return_value = sample_df
client.copy_ticks_range_as_df.return_value = sample_df
client.account_info_as_df.return_value = sample_df
client.terminal_info_as_df.return_value = sample_df
client.symbols_get_as_df.return_value = sample_df
client.symbol_info_as_df.return_value = sample_df
client.orders_get_as_df.return_value = sample_df
client.positions_get_as_df.return_value = sample_df
client.history_orders_get_as_df.return_value = sample_df
client.history_deals_get_as_df.return_value = sample_df
client.version_as_df.return_value = sample_df
client.last_error_as_df.return_value = sample_df
client.symbol_info_tick_as_df.return_value = sample_df
client.market_book_get_as_df.return_value = sample_df
client.order_check_as_df.return_value = sample_df
client.order_send_as_df.return_value = sample_df
client.version.return_value = (5, 0, 1)
client.terminal_info.return_value = {"connected": True, "paths": ["terminal.exe"]}
client.account_info.return_value = {"login": 123, "limits": {"modes": ["demo"]}}
client.symbols_total.return_value = 42
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=client)
return client
class TestCommands:
"""Tests for all CLI subcommands via CliRunner."""
+512
View File
@@ -0,0 +1,512 @@
"""Contract tests for the mt5cli public API and dataset schemas."""
from __future__ import annotations
from datetime import UTC, datetime
from typing import TYPE_CHECKING
import pandas as pd
import pytest
from pdmt5 import Mt5RuntimeError, Mt5TradingError
from pytest_mock import MockerFixture # noqa: TC002
from mt5cli import (
DEDUP_KEYS,
REQUIRED_COLUMNS,
TIME_COLUMNS,
DataKind,
Dataset,
MT5Client,
Mt5CliError,
Mt5ConnectionError,
Mt5OperationError,
Mt5SchemaError,
build_config,
call_with_normalized_errors,
detect_format,
ensure_utc,
export_dataframe,
export_dataframe_to_sqlite,
granularity_name,
is_recoverable_mt5_error,
mt5_session,
normalize_dataframe,
normalize_mt5_exception,
normalize_symbol,
normalize_symbols,
parse_date_range,
recent_window,
schema_columns,
validate_schema,
)
from mt5cli.retry import retry_with_backoff
from mt5cli.schemas import ensure_utc_columns, normalize_time_columns
if TYPE_CHECKING:
from pathlib import Path
def _sample_frame(kind: DataKind) -> pd.DataFrame:
if kind is DataKind.rates:
return pd.DataFrame({
"time": [datetime(2024, 1, 1, tzinfo=UTC)],
"open": [1.1],
"high": [1.2],
"low": [1.0],
"close": [1.15],
"tick_volume": [10],
"spread": [1],
"real_volume": [0],
})
if kind is DataKind.ticks:
return pd.DataFrame({
"time": [datetime(2024, 1, 1, tzinfo=UTC)],
"bid": [1.1],
"ask": [1.11],
"last": [1.105],
"volume": [1],
"time_msc": [datetime(2024, 1, 1, tzinfo=UTC)],
"flags": [2],
"volume_real": [0.0],
})
if kind is DataKind.orders:
return pd.DataFrame({
"ticket": [1],
"time_setup": [datetime(2024, 1, 1, tzinfo=UTC)],
"type": [0],
"state": [1],
"symbol": ["EURUSD"],
"volume_current": [0.1],
"price_open": [1.1],
})
if kind is DataKind.positions:
return pd.DataFrame({
"ticket": [1],
"time": [datetime(2024, 1, 1, tzinfo=UTC)],
"type": [0],
"symbol": ["EURUSD"],
"volume": [0.1],
"price_open": [1.1],
"price_current": [1.11],
"profit": [1.0],
})
if kind is DataKind.history_orders:
return pd.DataFrame({
"ticket": [1],
"time_setup": [datetime(2024, 1, 1, tzinfo=UTC)],
"type": [0],
"state": [3],
"symbol": ["EURUSD"],
"volume_initial": [0.1],
"price_open": [1.1],
})
return pd.DataFrame({
"ticket": [1],
"order": [2],
"time": [datetime(2024, 1, 1, tzinfo=UTC)],
"type": [0],
"entry": [0],
"symbol": ["EURUSD"],
"volume": [0.1],
"price": [1.1],
"profit": [0.0],
})
@pytest.mark.parametrize("kind", list(DataKind))
def test_required_columns_contract(kind: DataKind) -> None:
"""Each dataset kind exposes a non-empty required column contract."""
assert REQUIRED_COLUMNS[kind]
validate_schema(_sample_frame(kind), kind)
@pytest.mark.parametrize("kind", list(DataKind))
def test_normalize_dataframe_injects_storage_metadata(kind: DataKind) -> None:
"""Normalization accepts MT5 frames and optional storage metadata."""
frame = _sample_frame(kind)
normalized = normalize_dataframe(
frame,
kind,
symbol="eurusd",
timeframe="M1" if kind is DataKind.rates else None,
)
if kind is DataKind.rates:
assert normalized.loc[0, "symbol"] == "eurusd"
assert normalized.loc[0, "timeframe"] == 1
validate_schema(normalized, kind)
def test_validate_schema_raises_for_missing_columns() -> None:
"""Schema validation fails fast on missing required columns."""
with pytest.raises(Mt5SchemaError, match="missing required columns"):
validate_schema(pd.DataFrame({"time": [1]}), DataKind.rates)
def test_history_dedup_keys_match_schema_contract() -> None:
"""SQLite history dedup keys stay aligned with schema contracts."""
assert DEDUP_KEYS[DataKind.rates][0] == ("symbol", "timeframe", "time")
assert DEDUP_KEYS[DataKind.ticks][0] == ("symbol", "time_msc")
assert Dataset.rates.table_name == "rates"
@pytest.mark.parametrize(
("raw", "expected"),
[
(" eurusd ", "eurusd"),
("GbpJpy", "GbpJpy"),
("XAUUSDm", "XAUUSDm"),
("US500.cash", "US500.cash"),
("EURUSD.r", "EURUSD.r"),
],
)
def test_normalize_symbol(raw: str, expected: str) -> None:
"""Symbol normalization trims whitespace and preserves broker casing."""
assert normalize_symbol(raw) == expected
def test_normalize_symbols_deduplicates() -> None:
"""Symbol lists are normalized and de-duplicated in order."""
assert normalize_symbols(["XAUUSDm", " XAUUSDm ", "EURUSD.r", "eurusd"]) == [
"XAUUSDm",
"EURUSD.r",
"eurusd",
]
def test_parse_date_range_rejects_inverted_bounds() -> None:
"""Date ranges must not be inverted."""
with pytest.raises(ValueError, match="must not be after"):
parse_date_range("2024-02-01", "2024-01-01")
def test_recent_window_builds_trailing_bounds() -> None:
"""Recent windows end at the provided timestamp."""
end = datetime(2024, 1, 2, tzinfo=UTC)
start, resolved_end = recent_window(hours=24, date_to=end)
assert resolved_end == end
assert start < end
def test_granularity_name_maps_timeframe_alias() -> None:
"""Granularity labels resolve MT5 timeframe aliases."""
assert granularity_name("M1") == "M1"
@pytest.mark.parametrize(
"exc",
[Mt5RuntimeError("init failed"), Mt5TradingError("trade failed")],
)
def test_is_recoverable_mt5_error(exc: Exception) -> None:
"""Recoverable MT5 errors are classified consistently."""
assert is_recoverable_mt5_error(exc)
def test_normalize_mt5_exception_maps_types() -> None:
"""MT5 exceptions map to stable mt5cli types."""
assert isinstance(
normalize_mt5_exception(Mt5RuntimeError("x")),
Mt5ConnectionError,
)
assert isinstance(
normalize_mt5_exception(Mt5TradingError("x")),
Mt5OperationError,
)
def test_call_with_normalized_errors_reraises_mapped_type() -> None:
"""Normalized error helper re-raises mapped mt5cli exceptions."""
def _raise() -> None:
message = "boom"
raise Mt5RuntimeError(message)
with pytest.raises(Mt5ConnectionError):
call_with_normalized_errors(_raise)
def test_retry_with_backoff_retries_recoverable_errors(
mocker: MockerFixture,
) -> None:
"""Retry helper retries recoverable MT5 failures."""
calls = {"count": 0}
def _flaky() -> str:
calls["count"] += 1
if calls["count"] == 1:
message = "transient"
raise Mt5RuntimeError(message)
return "ok"
mocker.patch("mt5cli.retry.time.sleep")
assert retry_with_backoff(_flaky, retry_count=1) == "ok"
assert calls["count"] == 2
def test_public_api_exports_mt5_client() -> None:
"""MT5Client is the primary importable client abstraction."""
client = MT5Client(config=build_config())
assert isinstance(client, MT5Client)
assert isinstance(client, MT5Client.__mro__[1])
def test_mt5_client_order_primitives_use_connected_client(
mock_client: object,
) -> None:
"""Order check/send route through the same client fetch path as exports."""
request = {"action": 1}
client = MT5Client()
client.order_check(request)
client.order_send(request)
assert mock_client.order_check_as_df.call_count == 1 # type: ignore[attr-defined]
assert mock_client.order_send_as_df.call_count == 1 # type: ignore[attr-defined]
def test_storage_export_round_trip_csv(tmp_path: Path) -> None:
"""Storage helpers export normalized rate frames to CSV."""
frame = normalize_dataframe(
_sample_frame(DataKind.rates),
DataKind.rates,
symbol="EURUSD",
timeframe="M1",
)
output = tmp_path / "rates.csv"
export_dataframe(frame, output, detect_format(output))
loaded = pd.read_csv(output)
assert len(loaded) == 1
assert "close" in loaded.columns
def test_normalize_symbol_rejects_empty_value() -> None:
"""Empty symbols are rejected after trimming."""
with pytest.raises(ValueError, match="must not be empty"):
normalize_symbol(" ")
def test_ensure_utc_handles_naive_and_aware_datetimes() -> None:
"""UTC coercion accepts naive and timezone-aware datetimes."""
naive = datetime(2024, 1, 1, tzinfo=UTC).replace(tzinfo=None)
aware = datetime(2024, 1, 1, tzinfo=UTC)
assert ensure_utc(naive).tzinfo == UTC
assert ensure_utc(aware).tzinfo == UTC
assert ensure_utc("2024-01-01T00:00:00+00:00").tzinfo == UTC
def test_recent_window_validation_errors() -> None:
"""Recent window helpers validate mutually exclusive length arguments."""
with pytest.raises(ValueError, match="exactly one"):
recent_window()
with pytest.raises(ValueError, match="exactly one"):
recent_window(hours=1, seconds=1)
with pytest.raises(ValueError, match="positive"):
recent_window(hours=0)
def test_recent_window_supports_seconds_argument() -> None:
"""Recent windows can be built from a seconds-based length."""
end = datetime(2024, 1, 2, tzinfo=UTC)
start, resolved_end = recent_window(seconds=3600, date_to=end)
assert resolved_end == end
assert start < end
def test_parse_date_range_returns_ordered_bounds() -> None:
"""Valid date ranges return UTC-aware bounds."""
start, end = parse_date_range("2024-01-01", "2024-02-01")
assert start < end
def test_granularity_name_falls_back_for_unknown_timeframe(
mocker: MockerFixture,
) -> None:
"""Unknown timeframe integers stringify as granularity labels."""
mocker.patch(
"mt5cli.converters._get_timeframe_name",
side_effect=ValueError("unknown"),
)
assert granularity_name(1) == "1"
def test_normalize_mt5_exception_passthrough_and_generic() -> None:
"""Normalization preserves mt5cli errors and wraps unknown exceptions."""
original = Mt5CliError("known")
assert normalize_mt5_exception(original) is original
assert isinstance(normalize_mt5_exception(ValueError("x")), Mt5CliError)
def test_schema_columns_and_extra_required_validation() -> None:
"""Schema helpers expose contracts and honor extra required columns."""
assert schema_columns(DataKind.rates) == REQUIRED_COLUMNS[DataKind.rates]
validate_schema(pd.DataFrame(), DataKind.rates)
frame = _sample_frame(DataKind.rates)
with pytest.raises(Mt5SchemaError, match="storage_symbol"):
validate_schema(frame, DataKind.rates, extra_required=["storage_symbol"])
def test_normalize_dataframe_empty_and_tick_sort_paths() -> None:
"""Normalization handles empty frames and tick time_msc sorting."""
empty = pd.DataFrame()
assert normalize_dataframe(empty, DataKind.rates).empty
ticks = _sample_frame(DataKind.ticks)
ticks = pd.concat([ticks, ticks], ignore_index=True)
sorted_ticks = normalize_dataframe(ticks, DataKind.ticks, sort=True)
assert len(sorted_ticks) == 2
unsorted_ticks = normalize_dataframe(ticks, DataKind.ticks, sort=False)
assert len(unsorted_ticks) == 2
def test_normalize_dataframe_rate_timeframe_without_symbol() -> None:
"""Rate normalization can inject timeframe without symbol metadata."""
frame = _sample_frame(DataKind.rates)
normalized = normalize_dataframe(frame, DataKind.rates, timeframe="M1")
assert "timeframe" in normalized.columns
def test_normalize_dataframe_keeps_existing_symbol_and_timeframe() -> None:
"""Normalization does not duplicate existing storage metadata columns."""
frame = normalize_dataframe(
_sample_frame(DataKind.rates),
DataKind.rates,
symbol="EURUSD",
timeframe="M1",
)
normalized = normalize_dataframe(
frame,
DataKind.rates,
symbol="GBPUSD",
timeframe="H1",
)
assert normalized.loc[0, "symbol"] == "EURUSD"
assert normalized.loc[0, "timeframe"] == 1
def test_normalize_time_columns_skips_absent_time_fields() -> None:
"""Time normalization ignores absent optional time columns."""
frame = pd.DataFrame({"open": [1.0]})
result = normalize_time_columns(frame, DataKind.rates)
assert list(result.columns) == ["open"]
def test_normalize_time_columns_converts_unix_seconds() -> None:
"""Numeric MT5 ``time`` values are interpreted as Unix seconds."""
frame = pd.DataFrame({"time": [1704067200]})
result = normalize_time_columns(frame, DataKind.rates)
assert result.loc[0, "time"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
def test_normalize_time_columns_converts_unix_milliseconds() -> None:
"""Numeric MT5 ``time_msc`` values are interpreted as Unix milliseconds."""
frame = pd.DataFrame({"time_msc": [1704067200000]})
result = normalize_time_columns(frame, DataKind.ticks)
assert result.loc[0, "time_msc"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
def test_normalize_time_columns_preserves_utc_datetimes() -> None:
"""Already-converted datetime values remain UTC-normalized."""
aware = datetime(2024, 1, 1, tzinfo=UTC)
frame = pd.DataFrame({"time": [aware]})
result = normalize_time_columns(frame, DataKind.rates)
assert result.loc[0, "time"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
def test_normalize_time_columns_handles_optional_order_times() -> None:
"""Optional order/history time columns are normalized when present."""
frame = pd.DataFrame({
"time_setup": [1704067200],
"time_setup_msc": [1704067200000],
"time_done": [1704153600],
"time_done_msc": [1704153600000],
})
result = normalize_time_columns(frame, DataKind.orders)
assert result.loc[0, "time_setup"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
assert result.loc[0, "time_setup_msc"] == pd.Timestamp(
"2024-01-01T00:00:00+00:00",
)
assert result.loc[0, "time_done"] == pd.Timestamp("2024-01-02T00:00:00+00:00")
assert result.loc[0, "time_done_msc"] == pd.Timestamp(
"2024-01-02T00:00:00+00:00",
)
def test_time_columns_include_optional_order_fields() -> None:
"""Schema contracts document optional MT5 time columns per dataset kind."""
assert "time_done" in TIME_COLUMNS[DataKind.orders]
assert "time_setup_msc" in TIME_COLUMNS[DataKind.history_orders]
def test_normalize_dataframe_sorts_ticks_by_time_msc(
mocker: MockerFixture,
) -> None:
"""Tick frames without ``time`` can still sort on ``time_msc``."""
mocker.patch("mt5cli.schemas.validate_schema")
ticks = pd.concat([_sample_frame(DataKind.ticks)] * 2, ignore_index=True).drop(
columns=["time"],
)
ticks.loc[0, "time_msc"] = datetime(2024, 1, 1, tzinfo=UTC)
ticks.loc[1, "time_msc"] = datetime(2024, 1, 2, tzinfo=UTC)
ticks = pd.concat([ticks.iloc[[1]], ticks.iloc[[0]]], ignore_index=True)
normalized = normalize_dataframe(ticks, DataKind.ticks, sort=True)
assert normalized.iloc[0]["time_msc"] <= normalized.iloc[1]["time_msc"]
def test_ensure_utc_columns_skips_missing_columns() -> None:
"""UTC column coercion ignores absent columns."""
frame = _sample_frame(DataKind.rates)
result = ensure_utc_columns(frame, ["time", "missing"])
assert "time" in result.columns
def test_normalize_time_columns_coerces_string_timestamps() -> None:
"""String timestamps are parsed with timezone-aware datetime coercion."""
frame = pd.DataFrame({"time": ["2024-01-01T00:00:00+00:00"]})
result = normalize_time_columns(frame, DataKind.rates)
assert result.loc[0, "time"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
def test_ensure_utc_columns_coerces_non_mt5_columns() -> None:
"""Non-MT5 columns still coerce to UTC datetimes."""
frame = pd.DataFrame({"created_at": ["2024-01-01T00:00:00+00:00"]})
result = ensure_utc_columns(frame, ["created_at"])
assert result.loc[0, "created_at"] == pd.Timestamp("2024-01-01T00:00:00+00:00")
def test_mt5_session_yields_connected_client(mocker: MockerFixture) -> None:
"""Public mt5_session yields an MT5Client bound to a connected session."""
connected = mocker.MagicMock()
context = mocker.MagicMock()
context.__enter__.return_value = connected
context.__exit__.return_value = False
mocker.patch("mt5cli.client.connected_client", return_value=context)
with mt5_session(build_config()) as client:
assert isinstance(client, MT5Client)
def test_retry_with_backoff_reraises_non_recoverable_errors() -> None:
"""Non-MT5 errors are not retried."""
def _raise() -> None:
message = "fatal"
raise ValueError(message)
with pytest.raises(ValueError, match="fatal"):
retry_with_backoff(_raise, retry_count=2)
def test_storage_export_round_trip_sqlite(tmp_path: Path) -> None:
"""Storage helpers append deduplicated frames to SQLite."""
frame = normalize_dataframe(
_sample_frame(DataKind.rates),
DataKind.rates,
symbol="EURUSD",
timeframe="M1",
)
output = tmp_path / "rates.db"
export_dataframe_to_sqlite(
frame,
output,
"rates",
deduplicate_on=DEDUP_KEYS[DataKind.rates][0],
)
with __import__("sqlite3").connect(output) as conn:
count = conn.execute("SELECT COUNT(*) FROM rates").fetchone()[0]
assert count == 1
+36
View File
@@ -48,6 +48,7 @@ from mt5cli.history import (
resolve_history_datasets,
resolve_history_tick_flags,
resolve_history_timeframes,
resolve_rate_table_name,
resolve_rate_tables,
resolve_rate_view_name,
resolve_rate_view_names,
@@ -63,6 +64,15 @@ from mt5cli.utils import TIMEFRAME_MAP, Dataset, IfExists
class TestResolveRateViewName:
"""Tests for resolve_rate_view_name and resolve_rate_view_names."""
def test_resolve_rate_table_name_returns_normalized_table(self) -> None:
"""Test canonical normalized rates table name is stable."""
assert resolve_rate_table_name("EURUSD", "M1") == "rates"
def test_resolve_rate_table_name_rejects_empty_symbol(self) -> None:
"""Test canonical rate table resolution validates symbols."""
with pytest.raises(ValueError, match="symbol must not be empty"):
resolve_rate_table_name(" ", "M1")
def test_missing_database_path_does_not_create_file(self, tmp_path: Path) -> None:
"""Test resolving against a missing path does not create a database."""
db_path = tmp_path / "missing.db"
@@ -416,6 +426,32 @@ class TestLoadRateData:
frame = load_rate_data_from_connection(conn, "rate_view")
assert list(frame["close"]) == [1.0]
def test_load_rate_series_from_sqlite_table_style(
self,
tmp_path: Path,
) -> None:
"""Test public table-style loader returns one rate DataFrame."""
db_path = tmp_path / "table-style.db"
with sqlite3.connect(db_path) as conn:
conn.execute("CREATE TABLE rates(time TEXT, close REAL)")
conn.executemany(
"INSERT INTO rates(time, close) VALUES (?, ?)",
[
("2024-01-01T00:00:00+00:00", 1.0),
("2024-01-01T00:01:00+00:00", 1.1),
],
)
frame = load_rate_series_from_sqlite(db_path, table="rates", count=1)
assert isinstance(frame, pd.DataFrame)
assert list(frame["close"]) == [1.1]
def test_load_rate_series_from_sqlite_requires_targets_without_table(self) -> None:
"""Test multi-series loading requires targets when table is omitted."""
with pytest.raises(ValueError, match="targets are required"):
load_rate_series_from_sqlite("unused.db", count=1)
def test_loads_quoted_identifier(self, tmp_path: Path) -> None:
"""Test table names are quoted safely."""
db_path = tmp_path / "quoted.db"
+63 -32
View File
@@ -37,6 +37,7 @@ from mt5cli.sdk import (
copy_rates_range,
copy_ticks_from,
copy_ticks_range,
fetch_latest_closed_rates,
history_deals,
history_orders,
last_error,
@@ -61,7 +62,7 @@ from mt5cli.sdk import (
update_history_with_config,
version,
)
from mt5cli.utils import Dataset, IfExists
from mt5cli.utils import Dataset, IfExists, coerce_login
class _TerminalInfo(NamedTuple):
@@ -132,32 +133,6 @@ _DEALS_FIXTURE: dict[str, list[object]] = {
}
@pytest.fixture
def mock_client(mocker: MockerFixture) -> MagicMock:
"""Create and patch a mock Mt5DataClient for SDK tests."""
client = MagicMock()
sample_df = pd.DataFrame({"col": [1]})
client.copy_rates_from_as_df.return_value = sample_df
client.copy_rates_from_pos_as_df.return_value = sample_df
client.copy_rates_range_as_df.return_value = sample_df
client.copy_ticks_from_as_df.return_value = sample_df
client.copy_ticks_range_as_df.return_value = sample_df
client.account_info_as_df.return_value = sample_df
client.terminal_info_as_df.return_value = sample_df
client.symbols_get_as_df.return_value = sample_df
client.symbol_info_as_df.return_value = sample_df
client.orders_get_as_df.return_value = sample_df
client.positions_get_as_df.return_value = sample_df
client.history_orders_get_as_df.return_value = sample_df
client.history_deals_get_as_df.return_value = sample_df
client.version_as_df.return_value = sample_df
client.last_error_as_df.return_value = sample_df
client.symbol_info_tick_as_df.return_value = sample_df
client.market_book_get_as_df.return_value = sample_df
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=client)
return client
def _build_history_client(mocker: MockerFixture) -> MagicMock:
"""Build a mocked Mt5DataClient with per-symbol history results."""
client = MagicMock()
@@ -201,7 +176,7 @@ class TestConnectionLifecycle:
mock_client = MagicMock()
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=mock_client)
config = MagicMock()
with sdk._connected_client(config): # type: ignore[reportPrivateUsage]
with sdk.connected_client(config): # type: ignore[reportPrivateUsage]
mock_client.initialize_and_login_mt5.assert_called_once()
mock_client.shutdown.assert_called_once()
@@ -217,7 +192,7 @@ class TestConnectionLifecycle:
mocker.patch("mt5cli.sdk.Mt5DataClient", return_value=mock_client)
with (
pytest.raises(RuntimeError, match="login failed"),
sdk._connected_client(MagicMock()), # type: ignore[reportPrivateUsage]
sdk.connected_client(MagicMock()), # type: ignore[reportPrivateUsage]
):
pass
mock_client.shutdown.assert_called_once()
@@ -1327,12 +1302,12 @@ class TestAccountSpec:
expected: int | None,
) -> None:
"""Test login values are normalized for account configs."""
assert sdk._coerce_login(login) == expected # type: ignore[reportPrivateUsage]
assert coerce_login(login) == expected
def test_coerce_login_rejects_non_numeric_string(self) -> None:
"""Test non-numeric login strings raise ValueError."""
with pytest.raises(ValueError, match="invalid literal"):
sdk._coerce_login("abc") # type: ignore[reportPrivateUsage]
coerce_login("abc")
class TestCollectLatestRatesForAccounts:
@@ -1368,7 +1343,7 @@ class TestCollectLatestRatesForAccounts:
"""Test account fields override base_config, empty login falls back."""
configs: list[object] = []
def _record_config(*, config: object) -> MagicMock:
def _record_config(*, config: object, **_: object) -> MagicMock:
configs.append(config)
return mock_client
@@ -1688,6 +1663,62 @@ class TestCollectLatestClosedRatesForAccounts:
)
class TestFetchLatestClosedRates:
"""Tests for fetch_latest_closed_rates."""
def test_fetches_extra_bar_and_drops_forming_row(self) -> None:
"""Test single-symbol closed-bar helper hides the forming bar."""
client = MagicMock()
client.latest_rates.return_value = pd.DataFrame(
{
"time": [1, 2, 3],
"close": [1.0, 1.1, 1.2],
},
)
result = fetch_latest_closed_rates(
client,
symbol="EURUSD",
granularity="M1",
count=2,
)
client.latest_rates.assert_called_once_with(
"EURUSD",
"M1",
3,
start_pos=0,
)
assert list(result["close"]) == [1.0, 1.1]
def test_raises_when_no_closed_bars_are_available(self) -> None:
"""Test empty closed-bar results raise an actionable ValueError."""
client = MagicMock()
client.latest_rates.return_value = pd.DataFrame({"close": [1.0]})
with pytest.raises(ValueError, match="Rate data is empty"):
fetch_latest_closed_rates(
client,
symbol="EURUSD",
granularity="M1",
count=1,
)
def test_rejects_non_positive_count_before_fetching(self) -> None:
"""Test invalid count values fail before calling MT5."""
client = MagicMock()
with pytest.raises(ValueError, match="count must be positive"):
fetch_latest_closed_rates(
client,
symbol="EURUSD",
granularity="M1",
count=0,
)
client.latest_rates.assert_not_called()
class TestCollectLatestClosedRatesByGranularity:
"""Tests for collect_latest_closed_rates_by_granularity."""
+923 -9
View File
File diff suppressed because it is too large Load Diff
Generated
+1 -1
View File
@@ -487,7 +487,7 @@ wheels = [
[[package]]
name = "mt5cli"
version = "0.7.0"
version = "0.8.0"
source = { editable = "." }
dependencies = [
{ name = "click" },