Compare commits

...

10 Commits

Author SHA1 Message Date
dceoy 0610ea732c fix: handle NumPy object rate timestamps 2026-06-22 23:01:31 +09:00
Daichi Narushima 82a39731ed feat: add fetch_latest_closed_rates_indexed and allow_whole_dollar_env opt-in (#45)
* feat: add fetch_latest_closed_rates_indexed and allow_whole_dollar_env opt-in (#43, #44)

Closes #43: add fetch_latest_closed_rates_indexed(client, *, symbol,
granularity, count) -> pd.DataFrame to mt5cli/trading.py. Internally
reuses fetch_latest_closed_rates_for_trading_client(), converts the
"time" column to a UTC-aware DatetimeIndex named "time", and drops the
original column. Exported from trading.__all__, mt5cli.__init__, and
STABLE_SDK_EXPORTS.

Closes #44: extend substitute_env_placeholders() with opt-in
allow_whole_dollar_env=False that expands whole-value $ENV_NAME strings
(entire string must be exactly $IDENTIFIER). Threaded through
build_config(), resolve_account_spec(), and resolve_account_specs() with
the same default=False. Partial strings like "plan$pass", "abc$ENV", or
"$ENV-suffix" are never expanded.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: align Markdown table columns in docs and skill file

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: treat numeric (float64) epoch seconds as UTC in _rate_time_to_utc

After DataFrame concat or NA upcast the time column becomes float64, which
is still epoch seconds. Using is_numeric_dtype instead of is_integer_dtype
fixes the silent misalignment. Using series.to_numpy() before passing to
pd.to_datetime avoids the redundant pd.DatetimeIndex() wrapper and aligns
with how existing rate-time normalization in schemas.py handles numeric
timestamps.

Add test_converts_float_epoch_seconds_to_utc_datetime_index to cover the
regression. Add a doc note clarifying that build_config cannot expand
login since that parameter is int | None.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: reject NaT values after rate timestamp conversion in _rate_time_to_utc

pd.to_datetime() silently produces NaT for None/NaN inputs rather than
raising, so the function could return a DatetimeIndex containing NaT
despite documenting invalid timestamps as a ValueError. Check any(idx.isna())
after conversion and raise with a clear message.

Add test_raises_on_nat_time_column to cover the regression.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Bump version to v0.9.0

* fix: handle object numeric rate timestamps

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-22 22:52:19 +09:00
Daichi Narushima c4a4253fbc feat: stable SDK helpers for volume, margin, and closed bars (#39–#41) (#42)
* feat: add stable SDK helpers for volume, margin, and closed bars (#39, #40, #41)

Expose generic trading utilities in the stable downstream SDK so applications
like mteor can drop local MT5 adapter code:

- normalize_order_volume() for broker step/min/max sizing
- estimate_order_margin() and calculate_positions_margin() for margin totals
- fetch_latest_closed_rates_for_trading_client() for closed bars from Mt5TradingClient

Update STABLE_SDK_EXPORTS, package-root exports, docs, and unit tests.

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* chore: bump version to 0.8.3

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* fix: address PR review feedback on volume cap, rate time, and margin grouping

- Re-apply volume_max after step normalization in normalize_order_volume()
- Drop misleading non-time index reset branch in _ensure_rate_time_column()
- Group positions by (symbol, side) before margin estimation
- Add branch-coverage tests for tick price validation and volume cap edge case

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* fix: address remaining PR review threads on docs and DatetimeIndex

- Rename unnamed DatetimeIndex column to time after reset_index()
- Guard estimate_order_margin example on positive normalized volume
- Document calculate_positions_margin skip vs error propagation behavior
- Add test for unnamed DatetimeIndex branch coverage

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* fix: harden stable SDK margin, rate fetch, and volume normalization

- Wrap order_calc_margin conversion and reject None/non-numeric results
- Validate fetched rate objects are DataFrames before time normalization
- Return 0.0 for non-finite volume inputs and constraints in normalize_order_volume

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* fix: reject non-finite volumes in margin estimation helpers

Use _is_positive_finite_number() in estimate_order_margin() and
calculate_positions_margin() so NaN/inf volumes never reach broker calls.
Add focused tests and document non-finite volume skipping in trading.md.

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* fix: guard symbol filter in calculate_positions_margin for empty frames

Return 0.0 before filtering when positions are empty or lack a symbol column.
Add regression tests for filtered calls on malformed position frames.

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
2026-06-19 01:02:18 +09:00
dceoy 9f2968cc98 Update .agents/skills/pr-feedback-triage/SKILL.md 2026-06-19 00:21:33 +09:00
Daichi Narushima 7de3ce0b7a feat: add injectable update_backend to ThrottledHistoryUpdater (#38)
* feat: add injectable update_backend to ThrottledHistoryUpdater

Allow downstream applications to substitute the history update backend via
the ThrottledHistoryUpdater constructor without monkey-patching
mt5cli.sdk.update_history. Defaults to update_history for backward
compatibility.

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* chore: fix lint and format after QA

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* chore: bump version to 0.8.2

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

* fix: use explicit None check for ThrottledHistoryUpdater backend

Only None selects the default update_history backend so falsy callable
objects with __bool__ returning False are preserved as custom backends.

Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
2026-06-18 22:58:09 +09:00
Daichi Narushima 897f7f0a0d docs: stable SDK contract and strategy-neutral order helpers (#37) 2026-06-18 19:12:11 +09:00
Daichi Narushima d156dd7176 [codex] fix mt5 adapter APIs (#36)
* fix mt5 adapter APIs

* address PR feedback

* fix zero ratio minimum volume sizing

* Bump version to v0.8.0
2026-06-15 02:47:05 +09:00
dceoy 307d6f5320 docs: restructure AGENTS.md with concise repository guidance
Align agent instructions with the streamlined project structure, QA workflow, and security notes used elsewhere in the repo.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-14 23:07:13 +09:00
Daichi Narushima 8031389a67 Add GitHub CodeQL analysis to CI workflow (#35)
* chore: add GitHub CodeQL analysis to CI workflow

Enable automated security scanning with GitHub CodeQL to detect potential vulnerabilities in Python code.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

* chore: run CodeQL analysis on pull requests

Co-authored-by: Cursor <cursoragent@cursor.com>

* Add checks and statuses read permissions for dependabot auto-merge.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-14 22:54:28 +09:00
dceoy fdf5e08d31 Add .agents/skills/pr-feedback-triage/SKILL.md 2026-06-14 21:16:52 +09:00
22 changed files with 5517 additions and 191 deletions
+201
View File
@@ -0,0 +1,201 @@
---
name: pr-feedback-triage
description: Triage pull request review comments into fixes, replies, clarification requests, or open follow-ups while respecting safe execution modes.
---
# PR Feedback Triage
Triage pull request review feedback, decide what action each thread needs, make focused fixes when allowed, and report or resolve only what is actually handled.
## When to Use
- A PR has review comments, requested changes, unresolved review threads, or bot review findings.
- The user asks to address, respond to, or resolve PR feedback.
- The user provides a PR URL/number, a branch with an associated PR, or copied comments.
Do not use this skill for a first-pass code review with no existing feedback; use a code review skill instead.
## Inputs
- Pull request URL or number, or a current branch that has an associated pull request.
- Repository checkout or platform access sufficient to inspect the PR diff and review feedback.
- Optional reviewer priorities from the user, such as "only address blocking comments" or "do not reply on the PR platform".
- Optional operating mode flags: `dry_run`, `no_push`, and `no_reply`.
If no PR or review comments are identifiable, ask for the target PR or the copied comments before proceeding.
## Modes
- `dry_run`: inspect review feedback and report the triage only. Do not edit files, run write-mode formatters, commit, push, post replies, or resolve review threads.
- `no_push`: local edits and verification are allowed, but do not push commits or otherwise update the remote branch. Report the local diff or local commits that still need to be pushed. Do not resolve threads whose resolution depends on unpushed local edits.
- `no_reply`: do not post replies, submit reviews, or resolve review threads. Provide suggested replies and resolution actions in the final report instead.
When a mode disables an action, skip that destructive or externally visible action even if normal workflow text would otherwise allow it.
## Preflight
1. Identify the current branch and target PR.
2. Check tracked local changes with `git diff --name-only` and `git diff --cached --name-only`. Ignore untracked files unless the review feedback explicitly concerns them.
3. Check unpushed commits before relying on remote review feedback.
4. If tracked local changes or unpushed commits exist, warn that existing PR comments may not cover the latest local state. In `normal` mode, push only when the user request or repository workflow allows it; otherwise continue with a clearly reported limitation.
## Feedback Collection
Gather the complete feedback set before editing:
- Fetch unresolved review threads, requested-change reviews, PR-level summary comments, and copied comments.
- Use platform-native APIs/CLI when available. Paginate results; do not inspect only the first page of threads or comments.
- For bot reviewers that post both summary comments and inline comments, collect both. Summary comments often contain severity, rationale, and fix instructions; inline comments contain the exact file and line context.
- Preserve every thread/comment identifier needed to reply or resolve later.
- Compare each comment with the current diff and file contents because review lines can become outdated.
## Deduplication and Ordering
Build one triage record per distinct finding:
- Prefer exact review-thread identity when available.
- For duplicate bot findings appearing in both summary and inline comments, merge by exact issue title first, then by file path plus line range as a fallback.
- Prefer inline comments for location and current code context.
- Prefer summary comments for severity, category, rationale, and detailed agent prompts.
- Preserve the reviewers exact issue title and original wording where practical. Do not rename findings in a way that would make replies hard to map back to comments.
- Preserve the reviewers original ordering unless the user asks for priority reordering. Many review bots already order findings by severity.
Each triage record should track: original title, reviewer, source IDs, location, current applicability, severity/priority if available, disposition, planned action, verification, reply text if any, resolution decision, platform action attempted, and final platform state.
## Resolution Policy
In normal mode, `Resolve conversation` is the default action for any review thread that has been fully handled. A thread is handled when the requested change is implemented and verified, the current code already satisfies the comment, the comment is outdated and no longer applies, or a deliberate deferral/won't-fix response has been posted with a clear reason.
Keep a thread open only when it still needs reviewer, maintainer, or product input, the fix is local-only and not pushed, verification is missing for a material change, or the user explicitly requested `dry_run`, `no_push`, or `no_reply` behavior that prevents resolution.
When resolving a thread, add a concise reply first only if it provides useful context, such as what changed, why no code change was needed, why a finding was intentionally deferred, or why the original comment is now outdated. Do not add noisy replies for self-evident fixes unless project norms require them.
## Platform Action Contract
Do not treat triage as complete until every collected source ID reaches an explicit terminal state:
- `resolved`: a platform resolve action succeeded, or a re-check shows the thread is already resolved.
- `replied_left_open`: a reply or question was posted and the thread is intentionally left unresolved.
- `not_resolvable`: the source is a PR-level summary comment or copied comment that has no platform-level resolve action; reply or post a PR summary when useful.
- `skipped_by_mode`: `dry_run`, `no_push`, or `no_reply` prevented the external action.
- `failed_action`: a reply or resolve action was attempted and failed; include the attempted action and failure in the final summary.
In normal mode, build and execute a platform action queue after fixes are verified and pushed when needed:
- `reply_then_resolve`: use for handled threads where the reviewer needs context before resolution.
- `resolve_only`: use for self-evident fixes and already-addressed or outdated threads where an extra reply would add noise.
- `reply_leave_open`: use only for clarification requests, blocked work, or intentionally open follow-ups.
- `reply_only`: use for PR-level comments or summaries that cannot be resolved as review threads.
For duplicate findings, execute the terminal action for every source thread ID, not only the primary triage record. If one finding is represented by three unresolved inline threads, all three must be resolved or explicitly left open.
## GitHub Action Guidance
Prefer platform-native APIs or `gh` commands that expose review-thread resolution state. For GitHub inline review threads, use the thread node ID and the GraphQL `resolveReviewThread` mutation rather than assuming that a reply resolves the conversation.
A reliable pattern is:
1. Re-fetch review threads and comments immediately before acting.
2. Reply to the thread when the action queue says a reply is needed.
3. Resolve the review thread by node ID when the terminal state should be `resolved`.
4. Re-fetch unresolved review threads after the action queue completes.
5. Retry any expected-to-be-resolved thread that is still unresolved once; if it still remains unresolved, mark it `failed_action` instead of claiming completion.
Example GraphQL mutation shape:
```graphql
mutation ($threadId: ID!) {
resolveReviewThread(input: { threadId: $threadId }) {
thread {
id
isResolved
}
}
}
```
A posted reply alone is sufficient only for `reply_leave_open`, `reply_only`, or `not_resolvable` sources. For handled inline review threads, reply and resolve are separate actions.
## Flow
```mermaid
flowchart TD
A[Identify PR and branch state] --> B[Collect all review feedback]
B --> C[Deduplicate and preserve source IDs]
C --> D[Inspect current diff and code]
D --> E{Classify each triage record}
E -->|Fix| F[Implement minimal change]
E -->|Answer| G[Prepare concise reply]
E -->|Clarify| H[Prepare question and leave open]
E -->|Already addressed or Outdated| I[Prepare evidence]
E -->|Defer or Won't fix| J[Document reason]
F --> K[Verify]
G --> L{Mode}
H --> L
I --> L
J --> L
K --> L
L -->|dry_run| M[Report triage only]
L -->|no_push| N[Report local diff or commits]
L -->|no_reply| O[Report suggested replies/actions]
L -->|normal| P[Commit/push if changed]
P --> R[Execute reply/resolve action queue]
R --> S[Re-fetch threads and retry unresolved handled threads once]
M --> Q[Final summary]
N --> Q
O --> Q
S --> Q
```
## Compact Workflow
1. **Collect all relevant feedback**
- Identify the PR and gather unresolved review threads, requested-change reviews, PR-level summaries, inline comments, and copied comments.
- Paginate all platform calls and keep comment/thread IDs for later replies and resolution.
- For bot reviews, collect both summary and inline comments, then merge duplicates rather than fixing the same finding twice.
2. **Classify each triage record**
- **Fix**: Valid requested change; make the smallest focused edit when not in `dry_run`.
- **Answer**: No code change needed; prepare a concise explanation.
- **Clarify**: Ambiguous, conflicting, or missing context; reply with the question and leave unresolved.
- **Already addressed**: Current code already satisfies it; prepare evidence.
- **Outdated**: Commented code or issue no longer exists; prepare evidence.
- **Defer / Won't fix**: Valid concern intentionally not changed now; document a specific reason.
3. **Act according to the classification and mode**
- Keep edits scoped to the review feedback.
- Follow reviewer-provided fix instructions literally when they are still applicable; deviate only when the current code proves the instruction is stale or unsafe.
- In `dry_run`, stop at triage, proposed fixes, suggested replies, and verification plan.
- In `no_push`, local edits are allowed, but do not push or resolve threads whose fix is only local. Reply or resolve non-code, already-addressed, or outdated threads only when the action does not depend on unpushed work and `no_reply` is not set.
- In `no_reply`, do not post replies or resolve threads; report suggested replies/actions instead.
- In normal mode, commit and push changed code when appropriate, then execute the platform action queue for every collected source ID.
4. **Verify before claiming completion**
- For fixes, run appropriate checks or explain why they could not run.
- Re-inspect the updated diff and comment context to confirm the concern is resolved.
- Re-fetch review threads after reply/resolve actions and confirm all expected-to-be-resolved thread IDs are resolved.
- Do not mark a thread resolved if it still needs reviewer, maintainer, or product input.
- If a resolve or reply operation fails, retry once when safe; then report `failed_action` with the affected source ID and reason.
5. **Finish**
- Normal mode: commit/push changes when appropriate, post useful replies or a summary, resolve all handled threads by default, and reconcile the final unresolved set.
- Safe modes: report the local state and the exact replies/resolution actions a human could take.
## Reply Guidance
- Keep inline replies short and tied to the original title or concern.
- For fixed findings, mention the concrete change or commit if useful.
- For already-addressed or outdated findings, cite the current code path or behavior that makes the finding no longer applicable.
- For deferred or won't-fix findings, provide the reason and any follow-up issue or owner if known.
- If a reply or resolve operation fails, continue with the remaining threads and report the failure in the final summary.
## Final Summary Checklist
- Mode used: `normal`, `dry_run`, `no_push`, or `no_reply`
- Counts by disposition: fixed, answered, clarified/left open, already addressed, outdated, deferred/won't-fix
- Counts by platform terminal state: resolved, replied-left-open, not-resolvable, skipped-by-mode, failed-action
- Threads resolved, intentionally left open, already resolved, or resolution actions skipped by mode
- Any expected-to-be-resolved thread that remained unresolved after retry
- Verification run or planned
- Commits pushed, local diff/commits, or "none"
- Remaining open items and who needs to respond
+15
View File
@@ -62,6 +62,19 @@ jobs:
runs-on: ubuntu-slim
secrets:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
github-codeql-analysis:
if: >
github.event_name == 'push'
|| github.event_name == 'pull_request'
|| (github.event_name == 'workflow_dispatch' && inputs.workflow == 'lint-and-test')
permissions:
contents: read
security-events: write
actions: read
uses: dceoy/gh-actions-for-devops/.github/workflows/github-codeql-analysis.yml@main # zizmor: ignore[unpinned-uses]
with:
language: >
["python"]
dependabot-auto-merge:
if: >
github.event_name == 'pull_request' && github.actor == 'dependabot[bot]'
@@ -73,5 +86,7 @@ jobs:
contents: write
pull-requests: write
actions: read
checks: read
statuses: read
with:
unconditional: true
+20 -64
View File
@@ -1,81 +1,37 @@
# Repository Guidelines
## Commands
## Project Structure & Module Organization
### Development Setup
`mt5cli/` contains the package source. Important modules include `cli.py` for the Typer command-line app, `client.py` and `sdk.py` for public MT5 client/session APIs, `history.py` for SQLite history collection, `storage.py` and `converters.py` for export behavior, and `schemas.py` for normalized dataset contracts. `tests/` holds pytest coverage for CLI behavior, SDK contracts, trading helpers, history, and utilities. `docs/` and `mkdocs.yml` define the MkDocs site and API reference. `skills/mt5cli/SKILL.md` documents the mt5cli agent skill.
```bash
uv sync
```
## Build, Test, and Development Commands
### Code Quality and Documentation
- `uv sync` installs runtime and development dependencies from `pyproject.toml` and `uv.lock`.
- `uv run mt5cli --help` runs the local CLI entry point.
- `uv run ruff format .` formats Python files.
- `uv run ruff check --fix .` lints and applies safe fixes.
- `uv run pyright .` runs strict type checking.
- `uv run pytest` runs doctests, branch coverage, and the test suite.
- `uv run mkdocs serve` previews documentation locally; `uv run mkdocs build` validates the docs build.
**Important**: Run these before committing or creating a PR.
Use `.agents/skills/local-qa/SKILL.md` for pre-handoff QA. It runs `.agents/skills/local-qa/scripts/qa.sh`, which formats, lints, type-checks, tests, formats Markdown, and checks GitHub workflows.
1. **format, lint, and test**: Use `local-qa` skill.
2. **Documentation build** (if any public API changes): `uv run mkdocs build`
## Coding Style & Naming Conventions
## Architecture
Target Python `>=3.11,<3.14`. Use Ruffs configured 88-character line length and Google-style docstrings. Pyright is strict, so prefer explicit public type annotations and narrow exception handling. Keep module, function, and variable names in `snake_case`; classes and enums use `PascalCase`. Preserve the packages small, typed helper style rather than adding broad abstractions.
### Key Dependencies
## Design Principles
- **pdmt5**: Pandas-based data handler for MetaTrader 5 (core library)
- **typer**: CLI framework for building command-line interfaces
- **click**: Parameter type customization for CLI options
- **pandas**: Core data manipulation and analysis
Apply KISS, DRY, and YAGNI when changing code. Prefer the simplest implementation that satisfies the current CLI/API contract. Remove duplication when shared behavior is already proven by at least two concrete call sites, but avoid generic helpers for speculative reuse. Do not add configuration flags, extension hooks, or alternate backends until a real repository use case requires them.
### Package Structure
## Testing Guidelines
- `mt5cli/`: Main package directory
- `__init__.py`: Package initialization and exports (`detect_format`, `export_dataframe`)
- `cli.py`: CLI application with typer-based commands for data export
- `utils.py`: Constants, enums, parameter types, parsers, and export utilities
- `__main__.py`: Entry point for `python -m mt5cli`
- `tests/`: Comprehensive test suite (pytest-based)
- `test_cli.py`: Tests for CLI commands and collect-history behavior
- `test_utils.py`: Tests for utility constants, parameter types, parsers, and export functions
- `docs/`: MkDocs documentation with API reference
- `docs/index.md`: Main documentation
- `docs/api/`: Auto-generated API documentation for all modules
- Modern Python packaging with `pyproject.toml` and uv dependency management
### Quality Standards
- Type hints required (pyright strict mode)
- Comprehensive linting with 35+ rule categories (ruff)
- Test coverage tracking with 100% (pytest-cov)
- Parametrized tests for input/result matrices using `pytest.mark.parametrize` (pytest)
- Test doubles (mocks, stubs) using `pytest_mock` for external dependencies (pytest-mock)
- Pydantic models for data validation and configuration
### Documentation workflow
1. Add Google-style docstrings to functions/classes
2. Local preview: `uv run mkdocs serve`
3. Build: `uv run mkdocs build`
4. Deploy: `uv run mkdocs gh-deploy`
Tests use pytest, pytest-mock, doctests, and pytest-cov. Test files should match `tests/test_*.py`, classes `Test*`, and functions `test_*`. Coverage is configured with `fail_under = 100`, so add focused tests for every behavior change. Mock MT5/pdmt5 boundaries; do not require a live MetaTrader terminal in unit tests.
## Commit & Pull Request Guidelines
- Run QA checks using `local-qa` skill before committing or creating a PR.
- Branch names use appropriate prefixes on creation (e.g., `feature/...`, `bugfix/...`, `refactor/...`, `docs/...`, `chore/...`).
- When instructed to create a PR, create it as a draft with appropriate labels by default.
Recent history uses concise imperative commits, sometimes with conventional prefixes such as `feat:` or `chore:` and PR numbers appended by GitHub. Keep commits scoped to one logical change. Pull requests should describe behavior changes, note tests run, link related issues, and call out MT5/live-trading risk where relevant.
## Code Design Principles
## Security & Configuration Tips
Always prefer the simplest design that works.
- **KISS**: Choose straightforward solutions and avoid unnecessary abstraction.
- **DRY**: Remove duplication when it improves clarity and maintainability.
- **YAGNI**: Do not add features, hooks, or flexibility until they are needed.
- **SOLID/Clean Code**: Apply these as tools, only when they keep the design simpler and easier to change.
## Development Methodology
Keep delivery incremental, test-backed, and easy to review.
- Make small, safe, reversible changes.
- Prefer `Red -> Green -> Refactor`.
- Do not mix feature work and refactoring in the same commit.
- Refactor when it improves clarity or removes real duplication (Rule of Three).
- Keep tests fast, focused, and self-validating.
Never commit account credentials, broker passwords, exported private data, or local `.venv` contents. Treat `order_send` and CLI `order-send --yes` as live execution paths; gate examples and tests so they cannot place real trades accidentally.
+37 -4
View File
@@ -4,6 +4,8 @@
Generic MT5 data and execution infrastructure for Python applications. Export from the CLI or import a small, stable Python API in downstream packages.
The [Public API Contract](docs/api/public-contract.md) lists stable SDK exports (`mt5cli.STABLE_SDK_EXPORTS`), CLI commands, internal helpers, and responsibilities that remain out of scope (strategy logic, backtests, optimization).
Built on top of [pdmt5](https://github.com/dceoy/pdmt5), a pandas-based data handler for MetaTrader 5.
## Architecture
@@ -85,6 +87,37 @@ Schema contracts live in `mt5cli.schemas` (`DataKind`, `validate_schema`, `norma
`MT5Client.order_send()` is a live execution primitive: it can place real trades on the connected account. mt5cli does not implement strategy logic, signal generation, backtesting, or optimization — downstream applications must gate live execution explicitly.
### Trading lifecycle and state helpers
Trading applications can depend on `mt5cli` imports only; terminal path,
credentials, server, and timeout are forwarded to `pdmt5.Mt5Config`, numeric
login strings are coerced to integers, and empty login strings are treated as
unset.
```python
from mt5cli import (
calculate_spread_ratio,
create_trading_client,
get_account_snapshot,
mt5_trading_session,
)
with mt5_trading_session(
path=r"C:\Program Files\MetaTrader 5\terminal64.exe",
login="12345",
password="from-env-or-secret-store",
server="Broker-Demo",
) as client:
account = get_account_snapshot(client)
spread = calculate_spread_ratio(client, "EURUSD")
client = create_trading_client(login=12345, server="Broker-Demo")
try:
positions = client.positions_get_as_df(symbol="EURUSD")
finally:
client.shutdown()
```
## CLI usage
```bash
@@ -196,7 +229,7 @@ update_history_with_config(
- **`collect-history`**: explicit date-range export into SQLite.
- **`update_history`**: incremental append based on existing SQLite `MAX(time)` per symbol (and timeframe for rates); account-level deals use a separate cursor when `include_account_events=True`.
- **`rates` table**: normalized storage with `symbol` and `timeframe` columns.
- **Rate compatibility views**: mt5cli manages all `rate_*` views. Naming is `rate_<symbol>__<timeframe>` when a symbol has one timeframe, otherwise `rate_<symbol>__<granularity>_<timeframe>` (for example `rate_EURUSD__M1_1`). Stale `rate_*` views are dropped and recreated when rates change for offline tools such as mteor optimize.
- **Rate compatibility views**: mt5cli manages all `rate_*` views. Naming is `rate_<symbol>__<timeframe>` when a symbol has one timeframe, otherwise `rate_<symbol>__<granularity>_<timeframe>` (for example `rate_EURUSD__M1_1`). Stale `rate_*` views are dropped and recreated when rates change for offline downstream tools.
- **Rate view resolution**: use `resolve_rate_view_name()` / `resolve_rate_view_names()` to map symbols and granularities to existing SQLite compatibility views without creating databases. Both accept `None` (or a missing path) and return deterministic default names unless `require_existing=True`.
- **Rate view loading**: use `load_rate_data()` / `load_rate_data_from_connection()` to load a SQLite rate table or view into a `DatetimeIndex` DataFrame.
- **Multi-series rate loading**: use `build_rate_targets()` to build neutral `RateTarget(symbol, timeframe)` pairs, `resolve_rate_tables()` to map them to table/view names (pass `require_existing=True` for strict resolution), and `load_rate_series_from_sqlite()` to load them into a mapping keyed by `(symbol, integer timeframe)`. The loader requires existing managed views unless `explicit_tables` is supplied, and rejects duplicate `(symbol, timeframe)` targets.
@@ -216,7 +249,7 @@ eurusd_m1 = rates["EURUSD", "M1"] # closed bars only
```
- **Credential resolution**: use `resolve_account_spec()` / `resolve_account_specs()` to merge explicit override values over `AccountSpec` fields and expand `${ENV_VAR}` placeholders (via `substitute_env_placeholders()`), raising `ValueError` for missing variables. This keeps secrets out of plan/config files without coupling to any strategy code.
- **Throttled history updates**: use `ThrottledHistoryUpdater` to wrap `update_history()` with a minimum `interval_seconds` between successful runs (monotonic clock). Call `should_update()` / `update(client, symbols)` from an application loop; errors propagate by default, or pass `suppress_errors=True` to swallow recoverable `Mt5*Error`, `sqlite3.Error`, `ValueError`, `OSError`, and MT5 client capability errors for history API methods without advancing the throttle (other `AttributeError` / `TypeError` values always propagate).
- **Throttled history updates**: use `ThrottledHistoryUpdater` to wrap `update_history()` with a minimum `interval_seconds` between successful runs (monotonic clock). Call `should_update()` / `update(client, symbols)` from an application loop; errors propagate by default, or pass `suppress_errors=True` to swallow recoverable `Mt5*Error`, `sqlite3.Error`, `ValueError`, `OSError`, and MT5 client capability errors for history API methods without advancing the throttle (other `AttributeError` / `TypeError` values always propagate). Pass `update_backend` to inject a custom history update callable (same keyword arguments as `update_history`) instead of monkey-patching `mt5cli.sdk.update_history`.
- **Trading session helpers**: use `mt5_trading_session()` for a trading-capable `pdmt5.Mt5TradingClient` that initializes/logs in via `Mt5Config.path` and always shuts down safely. Pair with `detect_position_side()`, `calculate_margin_and_volume()`, and `determine_order_limits()` for generic position and sizing utilities. The read-only `mt5_session()` / `Mt5CliClient` SDK is unchanged.
- **Granularity-keyed rate loading**: `load_rate_series_by_granularity()` builds targets with `build_rate_targets()`, loads them with `load_rate_series_from_sqlite()`, and returns a mapping keyed by `(symbol | None, granularity_name)` such as `("EURUSD", "M1")` to reduce downstream boilerplate.
- **MT5 session helper**: use the `mt5_session()` context manager to attach to (or, when `Mt5Config.path` is set, launch) an MT5 terminal, log in, and yield a connected `MT5Client` that shuts down on exit.
@@ -229,12 +262,12 @@ eurusd_m1 = rates["EURUSD", "M1"] # closed bars only
- Windows OS (MetaTrader 5 requirement)
- MetaTrader 5 platform installed
### Migration note for mteor
### Migration note for downstream trading apps
Replace local MT5 lifecycle and trading helper code with mt5cli imports:
```python
# Before (local mteor helpers)
# Before (local application helpers)
# with local_mt5_trading_session(config) as client:
# side = local_detect_position_side(client, symbol)
# sizing = local_calculate_margin_and_volume(client, symbol, unit_ratio, preserved_ratio)
+29 -3
View File
@@ -167,19 +167,45 @@ Resolution rules:
### Rate data loading
Use `load_rate_data()` to load a table or view from a SQLite path, or
`load_rate_data_from_connection()` when you already have a connection:
The canonical normalized rate table is `rates`; compatibility views are named
with `rate_<symbol>__<timeframe>` for single-timeframe symbols or
`rate_<symbol>__<granularity>_<timeframe>` when a symbol has multiple stored
timeframes. `resolve_rate_table_name()` returns `rates`, while
`resolve_rate_view_name()` returns the per-symbol compatibility view name.
Use `load_rate_data()` or `load_rate_series_from_sqlite(..., table=...)` to load
a single table or view from a SQLite path. Use
`load_rate_series_by_granularity()` to load multiple instrument/granularity
targets without hard-coding view names:
```python
from pathlib import Path
from mt5cli import load_rate_data
from mt5cli import (
load_rate_data,
load_rate_series_by_granularity,
load_rate_series_from_sqlite,
resolve_rate_table_name,
)
from mt5cli.history import resolve_rate_view_name
view = resolve_rate_view_name(Path("history.db"), "EURUSD", "M1", require_existing=True)
rates = load_rate_data(Path("history.db"), view, count=1000)
same_rates = load_rate_series_from_sqlite(Path("history.db"), table=view, count=1000)
table = resolve_rate_table_name("EURUSD", "M1") # "rates"
series = load_rate_series_by_granularity(
Path("history.db"),
symbols=["EURUSD", "GBPUSD"],
granularities=["M1", "H1"],
count=500,
)
```
`count` returns the latest rows while preserving chronological order. Missing
tables/views and mismatched `explicit_tables` lengths raise `ValueError` with
the requested database target in the message.
The loader accepts close-based OHLC rate data or tick-like bid/ask data. It
validates that `time` exists, parses timestamps with pandas, and returns a
DataFrame indexed by ascending `DatetimeIndex` named `time`.
+9 -1
View File
@@ -2,10 +2,15 @@
This section documents the mt5cli public Python API and CLI modules.
Start with the [Public API Contract](public-contract.md) for the stable
downstream SDK surface, CLI boundary, internal modules, and out-of-scope strategy
responsibilities.
## Public API layers
| Module | Purpose |
| ----------------------------------------- | ------------------------------------------------------------------------- |
| [Public API Contract](public-contract.md) | Stable downstream SDK exports, CLI boundary, and out-of-scope items |
| [Client](client.md) | `MT5Client` session abstraction for data access and order primitives |
| [Schemas](schemas.md) | Canonical DataFrame contracts and normalization helpers |
| [Storage](storage.md) | CSV/JSON/Parquet/SQLite export and history collection helpers |
@@ -30,7 +35,10 @@ flowchart TD
SDK --> PDMT5["pdmt5.Mt5DataClient"]
```
Downstream packages should depend on the package root exports (`MT5Client`, `DataKind`, `normalize_dataframe`, `export_dataframe`, `collect_history`, etc.) rather than private modules.
Downstream packages should depend on the package root exports documented in the
[Public API Contract](public-contract.md) (`MT5Client`,
`DataKind`, `normalize_dataframe`, `collect_history`, `load_rate_data`,
`resolve_rate_view_name`, etc.) rather than private modules.
`MT5Client.order_send()` is a live execution primitive that can place real trades. mt5cli exposes minimal execution helpers only; strategy logic, signals, backtests, and optimization remain out of scope and must be implemented downstream with explicit execution gating.
+194
View File
@@ -0,0 +1,194 @@
# Public API Contract
mt5cli is the generic MT5 data and execution infrastructure layer for downstream
Python applications. The intended dependency direction is:
```text
downstream app -> mt5cli -> pdmt5 -> MetaTrader 5
```
Downstream packages should import from the package root (`from mt5cli import
...`) and treat the symbols listed below as the stable SDK contract. CLI
commands mirror the same behavior but are not importable Python APIs.
## Stable downstream SDK API
These names are exported from `mt5cli` and covered by the contract in
`mt5cli.STABLE_SDK_EXPORTS` (defined in `mt5cli.contract`). Prefer `MT5Client` over the legacy `Mt5CliClient`
alias for new code.
### Session lifecycle and configuration
| Symbol | Role |
| ----------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
| `MT5Client`, `Mt5CliClient` | Read-only data client with optional `order_check` / `order_send` |
| `build_config` | Build `pdmt5.Mt5Config` from connection fields |
| `mt5_session` | Context manager: initialize, login, yield client, shutdown |
| `create_trading_client`, `mt5_trading_session` | Trading-capable `pdmt5.Mt5TradingClient` lifecycle |
| `AccountSpec` | Generic account group: symbols plus optional credentials |
| `resolve_account_spec`, `resolve_account_specs` | Merge overrides and expand `${ENV_VAR}` placeholders; opt-in `allow_whole_dollar_env` for bare `$NAME` |
| `substitute_env_placeholders` | Replace `${NAME}` substrings from the environment; opt-in `allow_whole_dollar_env` for whole-value `$NAME` |
Credential resolution is generic: any environment variable name may appear inside
`${...}`. mt5cli does not hard-code application-specific keys such as
`mt5_login` or `mt5_exe`.
Pass `allow_whole_dollar_env=True` to `substitute_env_placeholders()`,
`resolve_account_spec()`, `resolve_account_specs()`, and `build_config()` to
additionally expand strings whose entire value is a bare `$ENV_NAME` identifier.
Partial strings such as `"plan$pass"`, `"abc$ENV"`, or `"$ENV-suffix"` are
**never** expanded — only an exact `$IDENTIFIER` whole-string match qualifies.
Default is `False` to preserve backward compatibility.
### Read-only MT5 data access
Module-level helpers open a transient connection per call. Prefer `mt5_session`
or `MT5Client` when making many requests in one process.
| Area | Symbols |
| -------------------- | ---------------------------------------------------------------------------------------------------- |
| Rates | `copy_rates_from`, `copy_rates_from_pos`, `copy_rates_range`, `latest_rates`, `collect_latest_rates` |
| Ticks | `copy_ticks_from`, `copy_ticks_range`, `recent_ticks` |
| Account / terminal | `account_info`, `terminal_info`, `mt5_version`, `last_error`, `mt5_summary`, `mt5_summary_as_df` |
| Symbols / market | `symbols`, `symbol_info`, `symbol_info_tick`, `market_book`, `minimum_margins` |
| Trading state (read) | `orders`, `positions`, `history_orders`, `history_deals`, `recent_history_deals` |
Use `mt5_version` for MetaTrader 5 terminal version data. The name `version` at
the package root refers to `importlib.metadata.version` (package metadata), not
the MT5 SDK helper.
### Closed-bar rate helpers
MetaTrader 5 returns the still-forming bar as the last row when
`start_pos=0`. Use these helpers instead of reimplementing bar trimming or
timestamp normalization in downstream apps.
| Symbol | Role |
| ------------------------------------------------ | ------------------------------------------------------------------------------- |
| `drop_forming_rate_bar` | Remove the last row from chronologically ordered rate data |
| `fetch_latest_closed_rates` | Single connected client: fetch `count + 1`, drop forming bar |
| `fetch_latest_closed_rates_for_trading_client` | Closed bars from an active `Mt5TradingClient` session; returns RangeIndex |
| `fetch_latest_closed_rates_indexed` | Same as above but returns a UTC `DatetimeIndex` named `"time"` (no time column) |
| `collect_latest_closed_rates_for_accounts` | Multi-account closed bars with optional retry wrapper |
| `collect_latest_closed_rates_by_granularity` | Same data keyed by `(symbol, granularity_name)` |
| `collect_latest_rates_for_accounts` | Latest bars including the forming bar when `start_pos=0` |
| `collect_latest_rates_for_accounts_with_retries` | Bounded exponential backoff for transient MT5 errors |
### SQLite history collection and rate loading
| Symbol | Role |
| ----------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| `collect_history` | One-shot date-range export into SQLite |
| `update_history`, `update_history_with_config` | Incremental append from `MAX(time)` cursors |
| `ThrottledHistoryUpdater` | Minimum interval between successful incremental updates; optional `update_backend` injection |
| `resolve_history_datasets`, `resolve_history_timeframes`, `resolve_history_tick_flags` | History pipeline configuration |
| `build_rate_view_name`, `resolve_rate_table_name`, `resolve_rate_view_name`, `resolve_rate_view_names`, `resolve_rate_tables` | Map symbols/timeframes to mt5cli-managed table or view names |
| `RateTarget`, `build_rate_targets` | Neutral `(symbol, timeframe)` series descriptors |
| `load_rate_data`, `load_rate_data_from_connection` | Load one table/view into a time-indexed DataFrame |
| `load_rate_series_from_sqlite`, `load_rate_series_by_granularity` | Load one or many series; fail clearly when managed views are missing |
Pass `require_existing=True` to rate view resolution helpers when downstream
code must fail instead of receiving a best-guess view name. Multi-series loaders
require existing managed `rate_*__*` views unless `explicit_tables` is supplied.
See [History Collection (SQLite)](history.md) for schema, view naming, and ER
diagrams.
### Trading and sizing primitives (generic)
These helpers implement broker-facing calculations only. They do not encode
strategy entries, exits, Kelly sizing, or signal logic.
| Symbol | Role |
| -------------------------------------------------------------------------------------------------- | --------------------------------------------- |
| `get_account_snapshot`, `get_symbol_snapshot`, `get_tick_snapshot`, `get_positions_frame` | Normalized account/symbol/tick/position views |
| `detect_position_side` | Net long / short / flat from open positions |
| `calculate_spread_ratio` | Relative bid-ask spread |
| `calculate_margin_and_volume`, `calculate_volume_by_margin`, `calculate_new_position_margin_ratio` | Margin budget and volume sizing |
| `normalize_order_volume`, `estimate_order_margin`, `calculate_positions_margin` | Broker volume normalization and margin totals |
| `determine_order_limits` | SL/TP price levels from ratios |
| `ensure_symbol_selected` | Select/verify Market Watch visibility |
| `place_market_order`, `close_open_positions`, `update_sltp_for_open_positions` | Order execution helpers (`dry_run` supported) |
| `MarginVolume`, `OrderLimits`, `OrderExecutionResult` | Typed return contracts for order helpers |
| `OrderSide`, `OrderFillingMode`, `OrderTimeMode`, `PositionSide`, `ExecutionStatus` | Typed enums for order helpers |
`MT5Client.order_send()` and CLI `order-send --yes` are live execution paths.
Order helpers validate broker stop-level distance in `determine_order_limits()` and
raise `Mt5TradingError` when computed SL/TP prices are too close to the entry
quote. Validation uses `trade_stops_level * point` from the current quote and
symbol metadata as a pre-check only; it does not guarantee live order acceptance
after price movement and does not inspect `trade_freeze_level`. Live
`place_market_order()` and SL/TP updates call
`ensure_symbol_selected()` so hidden symbols are added to Market Watch before
sending requests. Failed, malformed, or unknown broker retcodes are fail-closed
and returned as `status="failed"` with normalized `request` / `response` details;
`dry_run=True` never calls `ensure_symbol_selected()` or `order_send()`.
### Errors and MT5 type re-exports
| Symbol | Role |
| ------------------------------------------------------------------------------------ | ----------------------------------------------- |
| `Mt5CliError`, `Mt5ConnectionError`, `Mt5OperationError`, `Mt5SchemaError` | Stable mt5cli exception types |
| `normalize_mt5_exception`, `call_with_normalized_errors`, `is_recoverable_mt5_error` | Error normalization and retry classification |
| `Mt5Config`, `Mt5RuntimeError`, `Mt5TradingClient`, `Mt5TradingError` | Re-exported pdmt5 types for adapter convenience |
### Additional public exports (secondary)
The package root also exports schema, storage, and parsing helpers (for example
`DataKind`, `Dataset`, `normalize_dataframe`, `export_dataframe`,
`parse_timeframe`, `TIMEFRAME_MAP`). These are public but oriented toward export
pipelines and advanced integration. Prefer the stable symbols above for core
infrastructure.
## CLI commands
The Typer application in `mt5cli.cli` exposes file-export commands documented in
[CLI Module](cli.md) and the project README. CLI commands:
- Require `-o/--output` and write CSV, JSON, Parquet, or SQLite.
- Accept global MT5 connection options (`--login`, `--password`, `--server`,
`--path`, `--timeout`).
- Delegate to the same Python APIs described here; they are not duplicated
business logic.
`order-send` requires `--yes` before placing live trades.
## Internal helpers (not stable)
Do not import these for downstream contracts; they may change without a semver
notice:
| Module | Examples |
| ------------------------ | ------------------------------------------------------------------------- |
| `mt5cli.sdk` | `connected_client`, `_run_with_client`, private coercion helpers |
| `mt5cli.history` | `write_*_dataset`, `deduplicate_history_tables`, `parse_sqlite_timestamp` |
| `mt5cli.retry` | `retry_with_backoff` |
| `mt5cli.cli` | Typer command handlers and Click parameter types |
| Leading-underscore names | Any `_`-prefixed function or method |
Use the package-root stable exports instead of reaching into submodule
internals.
## Explicitly out of scope
mt5cli must **not** implement downstream strategy or research responsibilities.
The following belong in consuming applications, not in mt5cli:
- Signal detection (for example AR-GARCH or other model-specific triggers)
- Backtesting, walk-forward analysis, or parameter optimization
- Strategy-specific risk policy, position sizing systems, or Kelly fractions
- Entry/exit decision logic or YAML strategy semantics
- Application-specific credential schema keys wired into mt5cli internals
mt5cli provides connection lifecycle, normalized data access, SQLite history
machinery, closed-bar helpers, generic margin/volume/spread/SL/TP utilities, and
optional order primitives so downstream apps can focus on strategy code behind
their own adapter layer.
## Contract verification
`tests/test_contracts.py` asserts that every name in `STABLE_SDK_EXPORTS` is
importable from `mt5cli`, documents key closed-bar, rate-view, SQLite loading,
account-resolution, and trading-session behaviors, and keeps the contract set
aligned with `__all__`.
+65 -5
View File
@@ -31,13 +31,26 @@ rates = collect_latest_rates_for_accounts_with_retries(
### Latest closed rate bars
MetaTrader 5 `start_pos=0` includes the still-forming current bar as the last
row. `collect_latest_closed_rates_for_accounts()` fetches `count + 1` bars,
drops that row with `drop_forming_rate_bar()`, and validates each series is
non-empty. Use `collect_latest_closed_rates_by_granularity()` when callers
prefer keys such as `("EURUSD", "M1")` instead of integer timeframes.
row. `fetch_latest_closed_rates()` handles one connected `Mt5CliClient`; use
`fetch_latest_closed_rates_for_trading_client()` from an active
`Mt5TradingClient` session. Multi-account helpers fetch `count + 1` bars, drop
that row with `drop_forming_rate_bar()`, and validate each series is non-empty. Returned frames are ordered
oldest-to-newest and may contain fewer than `count` rows only when MT5 returns
fewer closed bars.
```python
from mt5cli import AccountSpec, collect_latest_closed_rates_by_granularity
from mt5cli import (
AccountSpec,
collect_latest_closed_rates_by_granularity,
fetch_latest_closed_rates,
)
closed = fetch_latest_closed_rates(
client,
symbol="EURUSD",
granularity="M1",
count=500,
)
rates = collect_latest_closed_rates_by_granularity(
[AccountSpec(symbols=["EURUSD"], login=12345)],
@@ -48,6 +61,9 @@ rates = collect_latest_closed_rates_by_granularity(
closed_m1 = rates["EURUSD", "M1"]
```
Use `collect_latest_closed_rates_by_granularity()` when callers prefer keys such
as `("EURUSD", "M1")` instead of integer timeframes.
### Resolving credentials and `${ENV_VAR}` placeholders
`resolve_account_spec()` / `resolve_account_specs()` merge explicit override
@@ -70,6 +86,28 @@ resolved = resolve_account_specs(accounts, server="Broker-Demo")
# resolved[0].login == "12345", resolved[0].server == "Broker-Demo"
```
Pass `allow_whole_dollar_env=True` to also expand strings whose **entire value**
is a bare `$ENV_NAME` identifier (no braces). This opt-in covers
`substitute_env_placeholders()`, `resolve_account_spec()`,
`resolve_account_specs()`, and `build_config()`. Note: `build_config` cannot
expand `login` because that parameter is `int | None`; use
`resolve_account_spec` for a string `login` placeholder. Partial strings such as
`"plan$pass"`, `"abc$ENV"`, or `"$ENV-suffix"` are never expanded — only an
exact `$IDENTIFIER` whole-string match qualifies. The default is `False` to
preserve backward compatibility.
```python
import os
from mt5cli import AccountSpec, resolve_account_specs
os.environ["MT5_PASSWORD"] = "secret"
accounts = [AccountSpec(symbols=["EURUSD"], password="$MT5_PASSWORD")]
resolved = resolve_account_specs(accounts, allow_whole_dollar_env=True)
# resolved[0].password == "secret"
```
### Throttled incremental history updates
`ThrottledHistoryUpdater` wraps `update_history()` with a minimum interval
@@ -98,6 +136,28 @@ finally:
client.shutdown()
```
Pass `update_backend` to substitute the default `update_history` implementation
without monkey-patching `mt5cli.sdk.update_history`. The callable receives the
same keyword arguments as `update_history` (`client`, `output`, `symbols`,
`datasets`, `timeframes`, `flags`, `lookback_hours`, `with_views`,
`include_account_events`). The resolved backend is stored on
`updater.update_backend` for inspection or subclassing.
```python
from mt5cli import ThrottledHistoryUpdater, update_history
def app_update_history(**kwargs) -> None:
update_history(**kwargs) # or delegate to application-specific logic
updater = ThrottledHistoryUpdater(
output="history.db",
interval_seconds=60,
update_backend=app_update_history,
)
```
By default recoverable errors (`Mt5TradingError`, `Mt5RuntimeError`,
`sqlite3.Error`, `ValueError`, `OSError`, and MT5 client capability
`AttributeError` / `TypeError` for history API methods) propagate so the caller
+149 -20
View File
@@ -4,38 +4,89 @@
## Trading-capable MT5 sessions
`mt5_trading_session()` complements the read-only `mt5_session()` helper in
`sdk.py`. It yields a connected `pdmt5.Mt5TradingClient`, uses
`Mt5Config.path` to launch the terminal when configured, and always calls
`shutdown()` on exit.
`create_trading_client()` and `mt5_trading_session()` complement the read-only
`mt5_session()` helper in `sdk.py`. They return or yield an initialized
`pdmt5.Mt5TradingClient`, use `Mt5Config.path` to launch the terminal when
configured, and `mt5_trading_session()` always calls `shutdown()` on exit.
```python
from pdmt5 import Mt5Config
from mt5cli import mt5_trading_session
from mt5cli import create_trading_client, mt5_trading_session
with mt5_trading_session(
Mt5Config(path=r"C:\Program Files\MetaTrader 5\terminal64.exe", login=12345),
path=r"C:\Program Files\MetaTrader 5\terminal64.exe",
login="12345",
password="secret",
server="Broker-Demo",
retry_count=2,
) as client:
positions = client.positions_get_as_df(symbol="EURUSD")
client = create_trading_client(login=12345, server="Broker-Demo")
try:
account = client.account_info_as_dict()
finally:
client.shutdown()
```
`login` accepts `int`, numeric `str`, or an empty string; empty strings are
treated as unset. `path`, `password`, `server`, and `timeout` are forwarded to
`pdmt5.Mt5Config`, and omitted `timeout` values keep the lower-level default.
The read-only `Mt5CliClient` / `mt5_session()` API is unchanged.
## Operational trading helpers
## State and order helpers
These helpers are strategy-agnostic and do not depend on signal detection,
betting logic, or scheduling code in downstream applications.
```python
from mt5cli import (
calculate_positions_margin,
calculate_spread_ratio,
calculate_margin_and_volume,
close_open_positions,
detect_position_side,
determine_order_limits,
estimate_order_margin,
fetch_latest_closed_rates_for_trading_client,
fetch_latest_closed_rates_indexed,
get_account_snapshot,
get_positions_frame,
get_symbol_snapshot,
get_tick_snapshot,
normalize_order_volume,
place_market_order,
)
account = get_account_snapshot(client)
symbol = get_symbol_snapshot(client, "EURUSD")
tick = get_tick_snapshot(client, "EURUSD")
positions = get_positions_frame(client, "EURUSD")
side = detect_position_side(client, "EURUSD")
spread_ratio = calculate_spread_ratio(client, "EURUSD")
volume = normalize_order_volume(
0.15,
volume_min=symbol["volume_min"],
volume_max=symbol["volume_max"],
volume_step=symbol["volume_step"],
)
buy_margin = (
estimate_order_margin(client, "EURUSD", "BUY", volume) if volume > 0 else 0.0
)
open_margin = calculate_positions_margin(client, symbols=["EURUSD"])
closed_bars = fetch_latest_closed_rates_for_trading_client(
client,
symbol="EURUSD",
granularity="M1",
count=100,
)
# Or fetch with a UTC DatetimeIndex instead of a "time" column:
indexed_bars = fetch_latest_closed_rates_indexed(
client,
symbol="EURUSD",
granularity="M1",
count=100,
)
# indexed_bars.index is a UTC-aware DatetimeIndex named "time"
sizing = calculate_margin_and_volume(
client,
"EURUSD",
@@ -49,21 +100,99 @@ limits = determine_order_limits(
stop_loss_limit_ratio=0.01,
take_profit_limit_ratio=0.02,
)
preview = place_market_order(
client,
symbol="EURUSD",
volume=sizing["buy_volume"],
order_side="BUY",
sl=limits["stop_loss"],
tp=limits["take_profit"],
dry_run=True,
)
closed = close_open_positions(client, symbols="EURUSD", dry_run=True)
```
Protective ratios must satisfy `0 <= ratio < 1`; `0` omits that level.
`calculate_margin_and_volume()` clamps negative `margin_free` to `0.0`
before sizing.
`detect_position_side()` returns `long` for buy-only exposure, `short` for
sell-only exposure, and `None` for no positions or mixed long/short exposure.
`calculate_spread_ratio()` uses `(ask - bid) / ((ask + bid) / 2)` and raises
`Mt5TradingError` when bid or ask is missing or non-positive.
`normalize_order_volume()` returns `0.0` for invalid constraints or
sub-minimum requests; check the result before calling `estimate_order_margin()`,
which requires a positive finite volume. `calculate_positions_margin()` silently
skips rows with missing symbols, non-positive volumes, non-finite volumes, or
unsupported position types, but propagates `Mt5TradingError` from `estimate_order_margin()` when a valid row
encounters invalid tick data or margin results from the broker.
## Migration from mteor-local helpers
SL/TP ratios for `determine_order_limits()` must satisfy `0 <= ratio < 1`; `0`
omits that level. SL/TP prices are rounded with symbol `digits` metadata when
available. `determine_order_limits()` pre-validates computed SL/TP prices against
available `trade_stops_level * point` metadata when present; violations raise
`Mt5TradingError`. This is a planning helper only: it does not guarantee broker
acceptance because live validation can still depend on price movement, bid/ask
side, freeze levels, and server-side rules, and it does not validate
`trade_freeze_level`. When symbol metadata cannot be loaded, protective prices
still round with `digits=8` and stop-level validation is skipped.
`unit_margin_ratio` and `preserved_margin_ratio` for `calculate_margin_and_volume()`
accept `0 <= ratio <= 1`; `unit_margin_ratio=0` requests one minimum valid unit
when the post-reserve margin can afford it. Negative `margin_free` is clamped to
`0.0` before sizing. Execution helpers return normalized `OrderExecutionResult`
dictionaries containing the request, response, status, retcode, and `dry_run`
flag; `dry_run=True` never sends an order or mutates Market Watch visibility.
`ensure_symbol_selected()` adds hidden symbols to Market Watch before live order
placement and SL/TP updates. Failed, malformed, or unknown broker retcodes are
fail-closed and returned as `status="failed"` while keeping the normalized
response for inspection.
| mteor-local concern | mt5cli replacement |
| -------------------------------------------------------- | ----------------------------------------------- |
| Manual terminal spawn/kill around trading code | `mt5_trading_session()` |
| Local position-side detection | `detect_position_side()` |
| Local margin/volume sizing | `calculate_margin_and_volume()` |
| Local SL/TP price derivation | `determine_order_limits()` |
| Throttled SQLite history loop with ad-hoc error handling | `ThrottledHistoryUpdater(suppress_errors=True)` |
## Order planning return contracts
```python
from mt5cli import MarginVolume, OrderLimits, OrderExecutionResult
sizing: MarginVolume = calculate_margin_and_volume(
client,
"EURUSD",
unit_margin_ratio=0.5,
preserved_margin_ratio=0.2,
)
limits: OrderLimits = determine_order_limits(
client,
"EURUSD",
side="long",
stop_loss_limit_ratio=0.01,
take_profit_limit_ratio=0.02,
)
preview: OrderExecutionResult = place_market_order(
client,
symbol="EURUSD",
volume=sizing["buy_volume"],
order_side="BUY",
sl=limits["stop_loss"],
tp=limits["take_profit"],
dry_run=True,
)
updates: list[OrderExecutionResult] = update_sltp_for_open_positions(
client,
symbol="EURUSD",
stop_loss=limits["stop_loss"],
dry_run=True,
)
```
Closes issue #33: strategy-neutral order planning and execution helpers exposed
through the stable package root without embedding entry/exit policy.
## Migration from application-local helpers
| Application-local concern | mt5cli replacement |
| -------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| Manual terminal spawn/kill around trading code | `mt5_trading_session()` |
| Local position-side detection | `detect_position_side()` |
| Local margin/volume sizing | `calculate_margin_and_volume()` |
| Local broker volume step normalization | `normalize_order_volume()` |
| Local order or position margin estimation | `estimate_order_margin()`, `calculate_positions_margin()` |
| Local closed-bar fetch from a trading session | `fetch_latest_closed_rates_for_trading_client()`, `fetch_latest_closed_rates_indexed()` |
| Local SL/TP price derivation | `determine_order_limits()` |
| Throttled SQLite history loop with ad-hoc error handling | `ThrottledHistoryUpdater(suppress_errors=True)` |
Keep read-only data collection on `mt5_session()` / `Mt5CliClient`; use
`mt5_trading_session()` only where order placement or trading calculations are
+1
View File
@@ -56,6 +56,7 @@ nav:
- Home: index.md
- API Reference:
- Overview: api/index.md
- Public API Contract: api/public-contract.md
- Client: api/client.md
- Schemas: api/schemas.md
- Storage: api/storage.md
+71 -1
View File
@@ -1,8 +1,17 @@
"""mt5cli: Generic MT5 data and execution infrastructure for Python applications."""
"""mt5cli: Generic MT5 data and execution infrastructure for Python applications.
Downstream packages should import from this module (``from mt5cli import ...``)
rather than private submodule helpers. See ``docs/api/public-contract.md`` for
the stable SDK contract, CLI surface, internal modules, and out-of-scope
strategy responsibilities.
"""
from importlib.metadata import version
from pdmt5 import Mt5Config, Mt5RuntimeError, Mt5TradingClient, Mt5TradingError
from .client import MT5Client, build_config, mt5_session
from .contract import STABLE_SDK_EXPORTS
from .converters import (
ensure_utc,
granularity_name,
@@ -32,6 +41,7 @@ from .history import (
resolve_history_datasets,
resolve_history_tick_flags,
resolve_history_timeframes,
resolve_rate_table_name,
resolve_rate_tables,
resolve_rate_view_name,
resolve_rate_view_names,
@@ -63,6 +73,7 @@ from .sdk import (
copy_rates_range,
copy_ticks_from,
copy_ticks_range,
fetch_latest_closed_rates,
history_deals,
history_orders,
last_error,
@@ -96,10 +107,36 @@ from .storage import (
export_dataframe_to_sqlite,
)
from .trading import (
POSITION_COLUMNS,
ExecutionStatus,
MarginVolume,
OrderExecutionResult,
OrderFillingMode,
OrderLimits,
OrderSide,
OrderTimeMode,
PositionSide,
calculate_margin_and_volume,
calculate_new_position_margin_ratio,
calculate_positions_margin,
calculate_spread_ratio,
calculate_volume_by_margin,
close_open_positions,
create_trading_client,
detect_position_side,
determine_order_limits,
ensure_symbol_selected,
estimate_order_margin,
fetch_latest_closed_rates_for_trading_client,
fetch_latest_closed_rates_indexed,
get_account_snapshot,
get_positions_frame,
get_symbol_snapshot,
get_tick_snapshot,
mt5_trading_session,
normalize_order_volume,
place_market_order,
update_sltp_for_open_positions,
)
from .utils import (
TICK_FLAG_MAP,
@@ -114,20 +151,34 @@ __version__ = version(__package__) if __package__ else None
__all__ = [
"DEDUP_KEYS",
"KNOWN_MT5_TIME_COLUMNS",
"POSITION_COLUMNS",
"REQUIRED_COLUMNS",
"STABLE_SDK_EXPORTS",
"TICK_FLAG_MAP",
"TIMEFRAME_MAP",
"TIME_COLUMNS",
"AccountSpec",
"DataKind",
"Dataset",
"ExecutionStatus",
"IfExists",
"MT5Client",
"MarginVolume",
"Mt5CliClient",
"Mt5CliError",
"Mt5Config",
"Mt5ConnectionError",
"Mt5OperationError",
"Mt5RuntimeError",
"Mt5SchemaError",
"Mt5TradingClient",
"Mt5TradingError",
"OrderExecutionResult",
"OrderFillingMode",
"OrderLimits",
"OrderSide",
"OrderTimeMode",
"PositionSide",
"RateTarget",
"ThrottledHistoryUpdater",
"account_info",
@@ -135,7 +186,12 @@ __all__ = [
"build_rate_targets",
"build_rate_view_name",
"calculate_margin_and_volume",
"calculate_new_position_margin_ratio",
"calculate_positions_margin",
"calculate_spread_ratio",
"calculate_volume_by_margin",
"call_with_normalized_errors",
"close_open_positions",
"collect_history",
"collect_latest_closed_rates_by_granularity",
"collect_latest_closed_rates_for_accounts",
@@ -147,13 +203,23 @@ __all__ = [
"copy_rates_range",
"copy_ticks_from",
"copy_ticks_range",
"create_trading_client",
"detect_format",
"detect_position_side",
"determine_order_limits",
"drop_forming_rate_bar",
"ensure_symbol_selected",
"ensure_utc",
"estimate_order_margin",
"export_dataframe",
"export_dataframe_to_sqlite",
"fetch_latest_closed_rates",
"fetch_latest_closed_rates_for_trading_client",
"fetch_latest_closed_rates_indexed",
"get_account_snapshot",
"get_positions_frame",
"get_symbol_snapshot",
"get_tick_snapshot",
"granularity_name",
"history_deals",
"history_orders",
@@ -173,6 +239,7 @@ __all__ = [
"mt5_version",
"normalize_dataframe",
"normalize_mt5_exception",
"normalize_order_volume",
"normalize_symbol",
"normalize_symbols",
"normalize_time_columns",
@@ -181,6 +248,7 @@ __all__ = [
"parse_datetime",
"parse_tick_flags",
"parse_timeframe",
"place_market_order",
"positions",
"recent_history_deals",
"recent_ticks",
@@ -190,6 +258,7 @@ __all__ = [
"resolve_history_datasets",
"resolve_history_tick_flags",
"resolve_history_timeframes",
"resolve_rate_table_name",
"resolve_rate_tables",
"resolve_rate_view_name",
"resolve_rate_view_names",
@@ -201,5 +270,6 @@ __all__ = [
"terminal_info",
"update_history",
"update_history_with_config",
"update_sltp_for_open_positions",
"validate_schema",
]
+106
View File
@@ -0,0 +1,106 @@
"""Stable downstream SDK export names for mt5cli."""
from __future__ import annotations
STABLE_SDK_EXPORTS: frozenset[str] = frozenset({
"AccountSpec",
"MT5Client",
"Mt5CliClient",
"Mt5CliError",
"Mt5Config",
"Mt5ConnectionError",
"Mt5OperationError",
"Mt5RuntimeError",
"Mt5SchemaError",
"Mt5TradingClient",
"Mt5TradingError",
"OrderFillingMode",
"OrderSide",
"OrderTimeMode",
"PositionSide",
"ExecutionStatus",
"MarginVolume",
"OrderExecutionResult",
"OrderLimits",
"RateTarget",
"ThrottledHistoryUpdater",
"account_info",
"build_config",
"build_rate_targets",
"build_rate_view_name",
"calculate_margin_and_volume",
"calculate_new_position_margin_ratio",
"calculate_positions_margin",
"calculate_spread_ratio",
"calculate_volume_by_margin",
"call_with_normalized_errors",
"close_open_positions",
"collect_history",
"collect_latest_closed_rates_by_granularity",
"collect_latest_closed_rates_for_accounts",
"collect_latest_rates",
"collect_latest_rates_for_accounts",
"collect_latest_rates_for_accounts_with_retries",
"copy_rates_from",
"copy_rates_from_pos",
"copy_rates_range",
"copy_ticks_from",
"copy_ticks_range",
"create_trading_client",
"detect_position_side",
"determine_order_limits",
"drop_forming_rate_bar",
"ensure_symbol_selected",
"estimate_order_margin",
"export_dataframe",
"export_dataframe_to_sqlite",
"fetch_latest_closed_rates",
"fetch_latest_closed_rates_for_trading_client",
"fetch_latest_closed_rates_indexed",
"get_account_snapshot",
"get_positions_frame",
"get_symbol_snapshot",
"get_tick_snapshot",
"history_deals",
"history_orders",
"is_recoverable_mt5_error",
"last_error",
"latest_rates",
"load_rate_data",
"load_rate_data_from_connection",
"load_rate_series_by_granularity",
"load_rate_series_from_sqlite",
"market_book",
"minimum_margins",
"mt5_session",
"mt5_summary",
"mt5_summary_as_df",
"mt5_trading_session",
"mt5_version",
"normalize_mt5_exception",
"normalize_order_volume",
"orders",
"place_market_order",
"positions",
"recent_history_deals",
"recent_ticks",
"resolve_account_spec",
"resolve_account_specs",
"resolve_history_datasets",
"resolve_history_tick_flags",
"resolve_history_timeframes",
"resolve_rate_table_name",
"resolve_rate_tables",
"resolve_rate_view_name",
"resolve_rate_view_names",
"substitute_env_placeholders",
"symbol_info",
"symbol_info_tick",
"symbols",
"terminal_info",
"update_history",
"update_history_with_config",
"update_sltp_for_open_positions",
})
__all__ = ["STABLE_SDK_EXPORTS"]
+72 -10
View File
@@ -7,7 +7,7 @@ import sqlite3
from dataclasses import dataclass
from datetime import UTC, datetime
from pathlib import Path
from typing import TYPE_CHECKING, Literal, cast
from typing import TYPE_CHECKING, Literal, cast, overload
import pandas as pd
from pdmt5 import get_timeframe_name as _get_timeframe_name
@@ -142,6 +142,26 @@ def build_rate_view_name(
return f"rate_{symbol}__{granularity}_{timeframe}"
def resolve_rate_table_name(symbol: str, granularity: str) -> str:
"""Return the canonical normalized SQLite rate table name.
The normalized history table stores all symbols and timeframes in
``rates``; use :func:`resolve_rate_view_name` for per-symbol compatibility
view names.
Returns:
Canonical normalized rates table name.
Raises:
ValueError: If ``symbol`` or ``granularity`` is invalid.
"""
parse_timeframe(granularity)
if not symbol.strip():
msg = "symbol must not be empty."
raise ValueError(msg)
return Dataset.rates.table_name
SqliteConnOrPath = sqlite3.Connection | Path | str
@@ -653,34 +673,76 @@ def resolve_rate_tables(
conn.close()
if TYPE_CHECKING:
@overload
def load_rate_series_from_sqlite(
conn_or_path: SqliteConnOrPath,
targets: None = None,
count: int | None = None,
explicit_tables: None = None,
*,
table: str,
) -> pd.DataFrame: ...
@overload
def load_rate_series_from_sqlite(
conn_or_path: SqliteConnOrPath,
targets: None = None,
count: int | None = None,
explicit_tables: Sequence[str] | None = None,
*,
table: None = None,
) -> dict[tuple[str | None, int], pd.DataFrame]: ...
@overload
def load_rate_series_from_sqlite(
conn_or_path: SqliteConnOrPath,
targets: Sequence[RateTarget],
count: int,
explicit_tables: Sequence[str] | None = None,
*,
table: None = None,
) -> dict[tuple[str | None, int], pd.DataFrame]: ...
def load_rate_series_from_sqlite(
conn_or_path: SqliteConnOrPath,
targets: Sequence[RateTarget],
count: int,
targets: Sequence[RateTarget] | None = None,
count: int | None = None,
explicit_tables: Sequence[str] | None = None,
) -> dict[tuple[str | None, int], pd.DataFrame]:
"""Load multiple rate series from a SQLite database.
*,
table: str | None = None,
) -> dict[tuple[str | None, int], pd.DataFrame] | pd.DataFrame:
"""Load one table/view or multiple rate series from a SQLite database.
Args:
conn_or_path: SQLite database path or open connection.
targets: Rate targets to load. Each ``(symbol, timeframe_int)`` pair
must be unique.
count: Number of most recent rows to load per series.
targets: Rate targets to load. Each ``(symbol, timeframe_int)`` pair must
be unique. Omit when loading a single explicit ``table``.
count: Optional number of most recent rows to load per series.
explicit_tables: Optional explicit table or view names matching targets.
When omitted, managed ``rate_*`` compatibility views must already
exist in the database.
table: Optional single table or view name to load directly.
Returns:
Mapping keyed by ``(symbol, timeframe_int)`` to each rate DataFrame.
A DataFrame when ``table`` is provided, otherwise a mapping keyed by
``(symbol, timeframe_int)`` to each rate DataFrame.
Raises:
ValueError: If ``count`` is not positive, targets are empty, duplicate
``(symbol, timeframe_int)`` pairs are present, or table resolution
fails.
"""
if count <= 0:
if table is not None:
return load_rate_data(conn_or_path, table, count=count)
if count is None or count <= 0:
msg = "count must be positive."
raise ValueError(msg)
if targets is None:
msg = "targets are required when table is not provided."
raise ValueError(msg)
target_list = list(targets)
if not target_list:
msg = "At least one rate target is required."
+129 -28
View File
@@ -37,10 +37,13 @@ from .utils import (
parse_tick_flags,
parse_timeframe,
)
from .utils import coerce_login as _coerce_login
if TYPE_CHECKING:
from collections.abc import Callable, Iterator, Sequence
UpdateHistoryBackend = Callable[..., None]
T = TypeVar("T")
logger = logging.getLogger(__name__)
@@ -122,6 +125,7 @@ __all__ = [
"copy_rates_range",
"copy_ticks_from",
"copy_ticks_range",
"fetch_latest_closed_rates",
"history_deals",
"history_orders",
"last_error",
@@ -305,12 +309,33 @@ def build_config(
password: str | None = None,
server: str | None = None,
timeout: int | None = None,
allow_whole_dollar_env: bool = False,
) -> Mt5Config:
"""Build an ``Mt5Config`` from optional connection parameters.
Args:
path: Optional terminal executable path.
login: Optional trading account login.
password: Optional trading account password.
server: Optional trading server name.
timeout: Optional connection timeout in milliseconds.
allow_whole_dollar_env: When ``True``, string parameters that are
exactly ``$ENV_NAME`` are expanded from the environment. Applies
to ``path``, ``password``, and ``server``. Default ``False``
preserves existing behavior.
Returns:
Configured ``Mt5Config`` instance.
"""
if allow_whole_dollar_env:
if path is not None:
path = substitute_env_placeholders(path, allow_whole_dollar_env=True)
if password is not None:
password = substitute_env_placeholders(
password, allow_whole_dollar_env=True
)
if server is not None:
server = substitute_env_placeholders(server, allow_whole_dollar_env=True)
return Mt5Config(
path=path,
login=login,
@@ -1042,10 +1067,16 @@ def update_history_with_config( # noqa: PLR0913
class ThrottledHistoryUpdater:
"""Throttled incremental SQLite history updater for long-running apps.
Wraps :func:`update_history` with a minimum interval between successful
updates, so a tight application loop can call :meth:`update` every
iteration without re-fetching MT5 history more often than desired. Timing
uses a monotonic clock, so it is unaffected by wall-clock changes.
Wraps :func:`update_history` (or a custom ``update_backend``) with a minimum
interval between successful updates, so a tight application loop can call
:meth:`update` every iteration without re-fetching MT5 history more often
than desired. Timing uses a monotonic clock, so it is unaffected by
wall-clock changes.
Downstream applications may pass ``update_backend`` to substitute the
default :func:`update_history` implementationfor example to add
application-specific logging, metrics, or test doubleswithout monkey-
patching ``mt5cli.sdk.update_history``.
"""
def __init__(
@@ -1060,6 +1091,7 @@ class ThrottledHistoryUpdater:
include_account_events: bool = True,
interval_seconds: float = 0.0,
suppress_errors: bool = False,
update_backend: UpdateHistoryBackend | None = None,
) -> None:
"""Initialize the throttled updater.
@@ -1083,6 +1115,12 @@ class ThrottledHistoryUpdater:
the throttle. Other ``AttributeError`` / ``TypeError`` values
always propagate. When False (default), recoverable errors
propagate so callers control logging.
update_backend: Callable invoked instead of :func:`update_history`
when :meth:`update` runs. Receives the same keyword arguments as
:func:`update_history` (``client``, ``output``, ``symbols``,
``datasets``, ``timeframes``, ``flags``, ``lookback_hours``,
``with_views``, ``include_account_events``). Defaults to
:func:`update_history`.
"""
self.output = output
self.datasets = datasets
@@ -1093,6 +1131,9 @@ class ThrottledHistoryUpdater:
self.include_account_events = include_account_events
self.interval_seconds = interval_seconds
self.suppress_errors = suppress_errors
self.update_backend = (
update_history if update_backend is None else update_backend
)
self._last_update_monotonic: float | None = None
@property
@@ -1143,7 +1184,7 @@ class ThrottledHistoryUpdater:
lookback_hours=self.lookback_hours,
date_to=None,
)
update_history(
self.update_backend(
client=client,
output=self.output,
symbols=symbols,
@@ -1289,6 +1330,33 @@ def latest_rates(
)
def fetch_latest_closed_rates(
client: Mt5CliClient,
*,
symbol: str,
granularity: str,
count: int,
) -> pd.DataFrame:
"""Fetch up to ``count`` most recent closed bars, oldest to newest.
Returns:
Closed rate bars ordered oldest to newest.
Raises:
ValueError: If ``count`` is not positive or no closed bars are returned.
"""
_require_positive(count, "count")
frame = client.latest_rates(symbol, granularity, count + 1, start_pos=0)
closed = drop_forming_rate_bar(frame)
if closed.empty:
msg = (
f"Rate data is empty for {symbol!r} at granularity {granularity!r} "
f"with count {count}."
)
raise ValueError(msg)
return closed.tail(count).reset_index(drop=True)
def collect_latest_rates(
symbols: Sequence[str],
timeframes: Sequence[int | str],
@@ -1329,13 +1397,22 @@ class AccountSpec:
_ENV_PLACEHOLDER_PATTERN = re.compile(r"\$\{(?P<name>[A-Za-z_][A-Za-z0-9_]*)\}")
_WHOLE_DOLLAR_PATTERN = re.compile(r"^\$(?P<name>[A-Za-z_][A-Za-z0-9_]*)$")
def substitute_env_placeholders(value: str) -> str:
def substitute_env_placeholders(
value: str,
*,
allow_whole_dollar_env: bool = False,
) -> str:
"""Replace ``${ENV_VAR}`` placeholders in a string with environment values.
Args:
value: String that may contain one or more ``${ENV_VAR}`` placeholders.
allow_whole_dollar_env: When ``True``, a string that is exactly
``$ENV_NAME`` (the whole value and nothing else) is also expanded
from the environment. Partial occurrences such as ``"plan$pass"``
or ``"$ENV-suffix"`` are left unchanged.
Returns:
The string with every placeholder replaced by its environment value.
@@ -1343,6 +1420,14 @@ def substitute_env_placeholders(value: str) -> str:
Raises:
ValueError: If a referenced environment variable is not set.
"""
if allow_whole_dollar_env:
m = _WHOLE_DOLLAR_PATTERN.match(value)
if m:
name = m.group("name")
if name not in os.environ:
msg = f"Environment variable {name!r} is not set."
raise ValueError(msg)
return os.environ[name]
parts: list[str] = []
last_end = 0
for match in _ENV_PLACEHOLDER_PATTERN.finditer(value):
@@ -1357,7 +1442,12 @@ def substitute_env_placeholders(value: str) -> str:
return "".join(parts)
def _resolve_field(override: str | None, account_value: str | None) -> str | None:
def _resolve_field(
override: str | None,
account_value: str | None,
*,
allow_whole_dollar_env: bool = False,
) -> str | None:
"""Resolve a string field from an override or account value with env subst.
Returns:
@@ -1367,12 +1457,16 @@ def _resolve_field(override: str | None, account_value: str | None) -> str | Non
value = override if override is not None else account_value
if value is None:
return None
return substitute_env_placeholders(value)
return substitute_env_placeholders(
value, allow_whole_dollar_env=allow_whole_dollar_env
)
def _resolve_login(
override: int | str | None,
account_login: int | str | None,
*,
allow_whole_dollar_env: bool = False,
) -> int | str | None:
"""Resolve a login from an override or account value with env substitution.
@@ -1384,10 +1478,14 @@ def _resolve_login(
if override is not None:
if isinstance(override, int):
return override
return substitute_env_placeholders(override)
return substitute_env_placeholders(
override, allow_whole_dollar_env=allow_whole_dollar_env
)
if account_login is None or isinstance(account_login, int):
return account_login
return substitute_env_placeholders(account_login)
return substitute_env_placeholders(
account_login, allow_whole_dollar_env=allow_whole_dollar_env
)
def resolve_account_spec(
@@ -1398,6 +1496,7 @@ def resolve_account_spec(
server: str | None = None,
path: str | None = None,
timeout: int | None = None,
allow_whole_dollar_env: bool = False,
) -> AccountSpec:
"""Resolve an account's credentials from overrides and ``${ENV_VAR}`` values.
@@ -1413,6 +1512,9 @@ def resolve_account_spec(
server: Optional explicit server override.
path: Optional explicit terminal path override.
timeout: Optional explicit connection timeout override.
allow_whole_dollar_env: When ``True``, string fields that are exactly
``$ENV_NAME`` are also expanded from the environment. Default
``False`` preserves existing behavior.
Returns:
A new :class:`AccountSpec` with resolved credentials and the original
@@ -1422,10 +1524,18 @@ def resolve_account_spec(
"""
return AccountSpec(
symbols=account.symbols,
login=_resolve_login(login, account.login),
password=_resolve_field(password, account.password),
server=_resolve_field(server, account.server),
path=_resolve_field(path, account.path),
login=_resolve_login(
login, account.login, allow_whole_dollar_env=allow_whole_dollar_env
),
password=_resolve_field(
password, account.password, allow_whole_dollar_env=allow_whole_dollar_env
),
server=_resolve_field(
server, account.server, allow_whole_dollar_env=allow_whole_dollar_env
),
path=_resolve_field(
path, account.path, allow_whole_dollar_env=allow_whole_dollar_env
),
timeout=timeout if timeout is not None else account.timeout,
)
@@ -1438,6 +1548,7 @@ def resolve_account_specs(
server: str | None = None,
path: str | None = None,
timeout: int | None = None,
allow_whole_dollar_env: bool = False,
) -> list[AccountSpec]:
"""Resolve credentials for multiple accounts.
@@ -1451,6 +1562,9 @@ def resolve_account_specs(
server: Optional explicit server override applied to each account.
path: Optional explicit terminal path override applied to each account.
timeout: Optional explicit timeout override applied to each account.
allow_whole_dollar_env: When ``True``, string fields that are exactly
``$ENV_NAME`` are also expanded from the environment. Default
``False`` preserves existing behavior.
Returns:
Resolved account specifications in the original order. Raises
@@ -1465,25 +1579,12 @@ def resolve_account_specs(
server=server,
path=path,
timeout=timeout,
allow_whole_dollar_env=allow_whole_dollar_env,
)
for account in accounts
]
def _coerce_login(login: int | str | None) -> int | None:
"""Coerce a login value to int, treating empty strings as unset.
Returns:
Integer login, or None when unset or an empty string.
"""
if login is None or isinstance(login, int):
return login
text = login.strip()
if not text:
return None
return int(text)
def _build_account_config(
account: AccountSpec,
base_config: Mt5Config | None,
+1166 -40
View File
File diff suppressed because it is too large Load Diff
+14
View File
@@ -241,6 +241,20 @@ def detect_format(
raise ValueError(msg)
def coerce_login(login: int | str | None) -> int | None:
"""Coerce a login value to int, treating empty strings as unset.
Returns:
Integer login, or None when unset or an empty string.
"""
if login is None or isinstance(login, int):
return login
text = login.strip()
if not text:
return None
return int(text)
def export_dataframe_to_sqlite(
df: pd.DataFrame,
output_path: Path,
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "mt5cli"
version = "0.7.2"
version = "0.9.0"
description = "Generic MT5 data and execution infrastructure for Python applications"
authors = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
maintainers = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
+255 -1
View File
@@ -2,43 +2,70 @@
from __future__ import annotations
import sqlite3
from datetime import UTC, datetime
from typing import TYPE_CHECKING
from typing import TYPE_CHECKING, get_type_hints
from unittest.mock import MagicMock
import pandas as pd
import pytest
from pdmt5 import Mt5RuntimeError, Mt5TradingError
from pytest_mock import MockerFixture # noqa: TC002
import mt5cli
from mt5cli import (
DEDUP_KEYS,
REQUIRED_COLUMNS,
STABLE_SDK_EXPORTS,
TIME_COLUMNS,
AccountSpec,
DataKind,
Dataset,
ExecutionStatus,
MarginVolume,
MT5Client,
Mt5CliError,
Mt5ConnectionError,
Mt5OperationError,
Mt5SchemaError,
OrderExecutionResult,
OrderLimits,
RateTarget,
build_config,
build_rate_targets,
calculate_margin_and_volume,
calculate_positions_margin,
call_with_normalized_errors,
detect_format,
drop_forming_rate_bar,
ensure_symbol_selected,
ensure_utc,
export_dataframe,
export_dataframe_to_sqlite,
fetch_latest_closed_rates,
fetch_latest_closed_rates_for_trading_client,
fetch_latest_closed_rates_indexed,
granularity_name,
is_recoverable_mt5_error,
load_rate_data,
load_rate_series_from_sqlite,
mt5_session,
mt5_trading_session,
normalize_dataframe,
normalize_mt5_exception,
normalize_order_volume,
normalize_symbol,
normalize_symbols,
parse_date_range,
place_market_order,
recent_window,
resolve_account_spec,
resolve_account_specs,
resolve_rate_view_name,
schema_columns,
validate_schema,
)
from mt5cli.history import create_rate_compatibility_views
from mt5cli.retry import retry_with_backoff
from mt5cli.schemas import ensure_utc_columns, normalize_time_columns
@@ -510,3 +537,230 @@ def test_storage_export_round_trip_sqlite(tmp_path: Path) -> None:
with __import__("sqlite3").connect(output) as conn:
count = conn.execute("SELECT COUNT(*) FROM rates").fetchone()[0]
assert count == 1
class TestStableSdkContract:
"""Tests for the documented stable downstream SDK contract."""
def test_stable_exports_are_subset_of_all(self) -> None:
"""Every stable export is also listed in the package __all__."""
missing = sorted(STABLE_SDK_EXPORTS - set(mt5cli.__all__))
assert not missing, f"STABLE_SDK_EXPORTS missing from __all__: {missing}"
@pytest.mark.parametrize("name", sorted(STABLE_SDK_EXPORTS))
def test_stable_exports_are_importable_from_package_root(self, name: str) -> None:
"""Stable SDK names resolve through ``from mt5cli import ...``."""
assert hasattr(mt5cli, name), f"{name!r} missing from mt5cli package root"
def test_drop_forming_rate_bar_from_package_root(self) -> None:
"""Closed-bar trimming is available from the stable package surface."""
frame = pd.DataFrame({"time": [1, 2, 3], "close": [1.0, 1.1, 1.2]})
closed = drop_forming_rate_bar(frame)
assert list(closed["close"]) == [1.0, 1.1]
assert len(closed) == 2
def test_fetch_latest_closed_rates_from_package_root(self) -> None:
"""Single-client closed-bar helper drops the forming row."""
client = MagicMock()
client.latest_rates.return_value = pd.DataFrame(
{"time": [1, 2, 3], "close": [1.0, 1.1, 1.2]},
)
result = fetch_latest_closed_rates(
client,
symbol="EURUSD",
granularity="M1",
count=2,
)
client.latest_rates.assert_called_once_with("EURUSD", "M1", 3, start_pos=0)
assert list(result["close"]) == [1.0, 1.1]
def test_fetch_latest_closed_rates_for_trading_client_from_package_root(
self,
) -> None:
"""Trading-client closed-bar helper is importable from the stable surface."""
client = MagicMock()
client.fetch_latest_rates_as_df.return_value = pd.DataFrame(
{"time": [1, 2, 3], "close": [1.0, 1.1, 1.2]},
)
result = fetch_latest_closed_rates_for_trading_client(
client,
symbol="EURUSD",
granularity="M1",
count=2,
)
assert list(result["close"]) == [1.0, 1.1]
def test_normalize_order_volume_from_package_root(self) -> None:
"""Volume normalization helper is importable from the stable surface."""
result = normalize_order_volume(
0.25,
volume_min=0.1,
volume_max=1.0,
volume_step=0.1,
)
assert abs(result - 0.2) < 1e-9
def test_calculate_positions_margin_from_package_root(self) -> None:
"""Position margin helper is importable from the stable surface."""
client = MagicMock()
client.mt5.POSITION_TYPE_BUY = 0
client.mt5.POSITION_TYPE_SELL = 1
client.mt5.ORDER_TYPE_BUY = 10
client.mt5.ORDER_TYPE_SELL = 11
client.positions_get_as_df.return_value = pd.DataFrame()
assert calculate_positions_margin(client) == 0
def test_resolve_rate_view_name_from_package_root(self, tmp_path: Path) -> None:
"""Rate view resolution is importable and honors require_existing."""
db_path = tmp_path / "rates.db"
with sqlite3.connect(db_path) as conn:
conn.execute(
"CREATE TABLE rates("
" symbol TEXT, timeframe INTEGER, time TEXT, close REAL)",
)
conn.execute(
"INSERT INTO rates(symbol, timeframe, time, close) VALUES (?, ?, ?, ?)",
("EURUSD", 1, "2024-01-01T00:00:00+00:00", 1.0),
)
create_rate_compatibility_views(conn)
assert resolve_rate_view_name(db_path, "EURUSD", "M1") == "rate_EURUSD__1"
missing = tmp_path / "missing.db"
with pytest.raises(ValueError, match="SQLite database not found"):
resolve_rate_view_name(missing, "EURUSD", "M1", require_existing=True)
def test_load_rate_data_from_package_root(self, tmp_path: Path) -> None:
"""SQLite rate loading normalizes timestamps through the stable API."""
db_path = tmp_path / "view.db"
with sqlite3.connect(db_path) as conn:
conn.execute(
'CREATE VIEW "rate_EURUSD__1" AS'
" SELECT '2024-01-01T00:00:00+00:00' AS time, 1.1 AS close",
)
frame = load_rate_data(db_path, "rate_EURUSD__1")
assert frame.index.name == "time"
assert abs(float(frame.iloc[0]["close"]) - 1.1) < 1e-9
def test_load_rate_series_from_sqlite_requires_managed_views(
self,
tmp_path: Path,
) -> None:
"""Multi-series loading fails clearly when managed views are absent."""
db_path = tmp_path / "empty-views.db"
with sqlite3.connect(db_path) as conn:
conn.execute(
"CREATE TABLE rates("
" symbol TEXT, timeframe INTEGER, time TEXT, close REAL)",
)
targets = build_rate_targets(["EURUSD"], ["M1"])
with pytest.raises(ValueError, match="No rate compatibility view exists"):
load_rate_series_from_sqlite(db_path, targets, count=10)
assert targets == [RateTarget(symbol="EURUSD", timeframe=1)]
def test_resolve_account_spec_from_package_root(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Account credential resolution uses generic ${ENV_VAR} placeholders."""
monkeypatch.setenv("APP_MT5_LOGIN", "555")
monkeypatch.setenv("APP_MT5_PASSWORD", "secret")
account = AccountSpec(
symbols=["EURUSD"],
login="${APP_MT5_LOGIN}",
password="${APP_MT5_PASSWORD}",
server="Broker-Demo",
)
resolved = resolve_account_spec(account, timeout=3000)
assert resolved.login == "555"
assert resolved.password == "secret" # noqa: S105
assert resolved.timeout == 3000
batch = resolve_account_specs([account], server="Override")
assert batch[0].server == "Override"
def test_mt5_trading_session_lifecycle_from_package_root(
self,
mocker: MockerFixture,
) -> None:
"""Trading session helper initializes and always shuts down."""
mock_client = MagicMock()
mocker.patch(
"mt5cli.trading.Mt5TradingClient",
return_value=mock_client,
)
with mt5_trading_session(login=12345, server="Broker-Demo") as client:
assert client is mock_client
mock_client.initialize_and_login_mt5.assert_called_once()
mock_client.shutdown.assert_called_once()
def test_trading_order_helpers_importable_from_package_root(self) -> None:
"""Order planning helpers resolve through the stable package surface."""
assert callable(calculate_margin_and_volume)
assert callable(ensure_symbol_selected)
assert callable(place_market_order)
margin_hints = get_type_hints(MarginVolume)
limits_hints = get_type_hints(OrderLimits)
execution_hints = get_type_hints(OrderExecutionResult)
assert margin_hints["buy_volume"] is float
assert limits_hints["stop_loss"] == float | None
assert execution_hints["status"] == ExecutionStatus
def test_mt5_trading_session_shuts_down_on_exception(
self,
mocker: MockerFixture,
) -> None:
"""Trading session helper shuts down even when the body raises."""
mock_client = MagicMock()
mocker.patch(
"mt5cli.trading.Mt5TradingClient",
return_value=mock_client,
)
message = "strategy error"
with (
pytest.raises(RuntimeError, match=message),
mt5_trading_session(login=12345, server="Broker-Demo"),
):
raise RuntimeError(message)
mock_client.shutdown.assert_called_once()
def test_fetch_latest_closed_rates_indexed_from_package_root(
self,
mocker: MockerFixture,
) -> None:
"""Indexed closed-bar helper returns a UTC DatetimeIndex named 'time'."""
client = MagicMock()
mocker.patch(
"mt5cli.trading.fetch_latest_closed_rates_for_trading_client",
return_value=pd.DataFrame(
{
"time": [1704067200, 1704153600, 1704240000],
"close": [1.0, 1.1, 1.2],
},
),
)
result = fetch_latest_closed_rates_indexed(
client,
symbol="EURUSD",
granularity="M1",
count=2,
)
assert isinstance(result.index, pd.DatetimeIndex)
assert result.index.name == "time"
assert result.index.tz is not None
assert "time" not in result.columns
assert "close" in result.columns
+36
View File
@@ -48,6 +48,7 @@ from mt5cli.history import (
resolve_history_datasets,
resolve_history_tick_flags,
resolve_history_timeframes,
resolve_rate_table_name,
resolve_rate_tables,
resolve_rate_view_name,
resolve_rate_view_names,
@@ -63,6 +64,15 @@ from mt5cli.utils import TIMEFRAME_MAP, Dataset, IfExists
class TestResolveRateViewName:
"""Tests for resolve_rate_view_name and resolve_rate_view_names."""
def test_resolve_rate_table_name_returns_normalized_table(self) -> None:
"""Test canonical normalized rates table name is stable."""
assert resolve_rate_table_name("EURUSD", "M1") == "rates"
def test_resolve_rate_table_name_rejects_empty_symbol(self) -> None:
"""Test canonical rate table resolution validates symbols."""
with pytest.raises(ValueError, match="symbol must not be empty"):
resolve_rate_table_name(" ", "M1")
def test_missing_database_path_does_not_create_file(self, tmp_path: Path) -> None:
"""Test resolving against a missing path does not create a database."""
db_path = tmp_path / "missing.db"
@@ -416,6 +426,32 @@ class TestLoadRateData:
frame = load_rate_data_from_connection(conn, "rate_view")
assert list(frame["close"]) == [1.0]
def test_load_rate_series_from_sqlite_table_style(
self,
tmp_path: Path,
) -> None:
"""Test public table-style loader returns one rate DataFrame."""
db_path = tmp_path / "table-style.db"
with sqlite3.connect(db_path) as conn:
conn.execute("CREATE TABLE rates(time TEXT, close REAL)")
conn.executemany(
"INSERT INTO rates(time, close) VALUES (?, ?)",
[
("2024-01-01T00:00:00+00:00", 1.0),
("2024-01-01T00:01:00+00:00", 1.1),
],
)
frame = load_rate_series_from_sqlite(db_path, table="rates", count=1)
assert isinstance(frame, pd.DataFrame)
assert list(frame["close"]) == [1.1]
def test_load_rate_series_from_sqlite_requires_targets_without_table(self) -> None:
"""Test multi-series loading requires targets when table is omitted."""
with pytest.raises(ValueError, match="targets are required"):
load_rate_series_from_sqlite("unused.db", count=1)
def test_loads_quoted_identifier(self, tmp_path: Path) -> None:
"""Test table names are quoted safely."""
db_path = tmp_path / "quoted.db"
+396 -3
View File
@@ -37,6 +37,7 @@ from mt5cli.sdk import (
copy_rates_range,
copy_ticks_from,
copy_ticks_range,
fetch_latest_closed_rates,
history_deals,
history_orders,
last_error,
@@ -61,7 +62,7 @@ from mt5cli.sdk import (
update_history_with_config,
version,
)
from mt5cli.utils import Dataset, IfExists
from mt5cli.utils import Dataset, IfExists, coerce_login
class _TerminalInfo(NamedTuple):
@@ -1301,12 +1302,12 @@ class TestAccountSpec:
expected: int | None,
) -> None:
"""Test login values are normalized for account configs."""
assert sdk._coerce_login(login) == expected # type: ignore[reportPrivateUsage]
assert coerce_login(login) == expected
def test_coerce_login_rejects_non_numeric_string(self) -> None:
"""Test non-numeric login strings raise ValueError."""
with pytest.raises(ValueError, match="invalid literal"):
sdk._coerce_login("abc") # type: ignore[reportPrivateUsage]
coerce_login("abc")
class TestCollectLatestRatesForAccounts:
@@ -1662,6 +1663,62 @@ class TestCollectLatestClosedRatesForAccounts:
)
class TestFetchLatestClosedRates:
"""Tests for fetch_latest_closed_rates."""
def test_fetches_extra_bar_and_drops_forming_row(self) -> None:
"""Test single-symbol closed-bar helper hides the forming bar."""
client = MagicMock()
client.latest_rates.return_value = pd.DataFrame(
{
"time": [1, 2, 3],
"close": [1.0, 1.1, 1.2],
},
)
result = fetch_latest_closed_rates(
client,
symbol="EURUSD",
granularity="M1",
count=2,
)
client.latest_rates.assert_called_once_with(
"EURUSD",
"M1",
3,
start_pos=0,
)
assert list(result["close"]) == [1.0, 1.1]
def test_raises_when_no_closed_bars_are_available(self) -> None:
"""Test empty closed-bar results raise an actionable ValueError."""
client = MagicMock()
client.latest_rates.return_value = pd.DataFrame({"close": [1.0]})
with pytest.raises(ValueError, match="Rate data is empty"):
fetch_latest_closed_rates(
client,
symbol="EURUSD",
granularity="M1",
count=1,
)
def test_rejects_non_positive_count_before_fetching(self) -> None:
"""Test invalid count values fail before calling MT5."""
client = MagicMock()
with pytest.raises(ValueError, match="count must be positive"):
fetch_latest_closed_rates(
client,
symbol="EURUSD",
granularity="M1",
count=0,
)
client.latest_rates.assert_not_called()
class TestCollectLatestClosedRatesByGranularity:
"""Tests for collect_latest_closed_rates_by_granularity."""
@@ -1720,6 +1777,80 @@ class TestSubstituteEnvPlaceholders:
with pytest.raises(ValueError, match="'MT5_MISSING' is not set"):
substitute_env_placeholders("${MT5_MISSING}")
def test_whole_dollar_not_substituted_by_default(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test $ENV_NAME is not expanded without allow_whole_dollar_env=True."""
monkeypatch.setenv("MT5_PASSWORD", "secret")
assert substitute_env_placeholders("$MT5_PASSWORD") == "$MT5_PASSWORD"
def test_whole_dollar_substituted_with_opt_in(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test $ENV_NAME is expanded when allow_whole_dollar_env=True."""
monkeypatch.setenv("MT5_PASSWORD", "secret")
result = substitute_env_placeholders(
"$MT5_PASSWORD", allow_whole_dollar_env=True
)
assert result == "secret"
def test_whole_dollar_missing_variable_raises_value_error(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test missing $ENV_NAME raises ValueError when opt-in is enabled."""
monkeypatch.delenv("MT5_MISSING", raising=False)
with pytest.raises(ValueError, match="'MT5_MISSING' is not set"):
substitute_env_placeholders("$MT5_MISSING", allow_whole_dollar_env=True)
def test_partial_dollar_not_expanded_with_opt_in(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test $ENV embedded in a larger string is not expanded."""
monkeypatch.setenv("pass", "secret")
monkeypatch.setenv("ENV", "val")
assert (
substitute_env_placeholders("plan$pass", allow_whole_dollar_env=True)
== "plan$pass"
)
assert (
substitute_env_placeholders("abc$ENV", allow_whole_dollar_env=True)
== "abc$ENV"
)
def test_dollar_with_suffix_not_expanded_with_opt_in(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test $ENV-suffix is not expanded (not a whole-value placeholder)."""
monkeypatch.setenv("ENV", "val")
assert (
substitute_env_placeholders("$ENV-suffix", allow_whole_dollar_env=True)
== "$ENV-suffix"
)
def test_brace_format_works_with_opt_in(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test ${ENV_VAR} substitution still works when allow_whole_dollar_env=True."""
monkeypatch.setenv("MT5_LOGIN", "12345")
result = substitute_env_placeholders(
"${MT5_LOGIN}", allow_whole_dollar_env=True
)
assert result == "12345"
class TestResolveAccountSpec:
"""Tests for resolve_account_spec and resolve_account_specs."""
@@ -1806,6 +1937,117 @@ class TestResolveAccountSpec:
assert [a.server for a in resolved] == ["Shared", "Fixed"]
assert all(a.timeout == 1000 for a in resolved)
def test_resolve_account_spec_with_whole_dollar_env(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Account spec expands $ENV_NAME when allow_whole_dollar_env=True."""
monkeypatch.setenv("MT5_PASSWORD", "secret")
account = AccountSpec(symbols=["EURUSD"], password="$MT5_PASSWORD")
resolved = resolve_account_spec(account, allow_whole_dollar_env=True)
assert resolved.password == "secret" # noqa: S105
def test_resolve_account_spec_whole_dollar_not_expanded_by_default(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test resolve_account_spec leaves $ENV_NAME literal by default."""
monkeypatch.setenv("MT5_PASSWORD", "secret")
account = AccountSpec(symbols=["EURUSD"], password="$MT5_PASSWORD")
resolved = resolve_account_spec(account)
assert resolved.password == "$MT5_PASSWORD" # noqa: S105
def test_resolve_account_specs_with_whole_dollar_env(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test resolve_account_specs threads allow_whole_dollar_env to each account."""
monkeypatch.setenv("MT5_SERVER", "Broker-Demo")
accounts = [
AccountSpec(symbols=["EURUSD"], server="$MT5_SERVER"),
AccountSpec(symbols=["GBPUSD"], server="Fixed"),
]
resolved = resolve_account_specs(accounts, allow_whole_dollar_env=True)
assert resolved[0].server == "Broker-Demo"
assert resolved[1].server == "Fixed"
def test_resolve_account_spec_whole_dollar_login(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test $ENV_NAME login string is expanded when allow_whole_dollar_env=True."""
monkeypatch.setenv("MT5_LOGIN", "12345")
account = AccountSpec(symbols=["EURUSD"], login="$MT5_LOGIN")
resolved = resolve_account_spec(account, allow_whole_dollar_env=True)
assert resolved.login == "12345"
class TestBuildConfigWholeDollarEnv:
"""Tests for build_config with allow_whole_dollar_env."""
def test_build_config_substitutes_server_with_opt_in(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""build_config expands $ENV_NAME server when allow_whole_dollar_env=True."""
monkeypatch.setenv("MT5_SERVER", "Broker-Demo")
config = build_config(server="$MT5_SERVER", allow_whole_dollar_env=True)
assert config.server == "Broker-Demo"
def test_build_config_substitutes_password_with_opt_in(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""build_config expands $ENV_NAME password when allow_whole_dollar_env=True."""
monkeypatch.setenv("MT5_PASSWORD", "secret")
config = build_config(password="$MT5_PASSWORD", allow_whole_dollar_env=True)
assert config.password == "secret" # noqa: S105
def test_build_config_substitutes_path_with_opt_in(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test build_config expands $ENV_NAME path when allow_whole_dollar_env=True."""
monkeypatch.setenv("MT5_PATH", "/opt/mt5/terminal64.exe")
config = build_config(path="$MT5_PATH", allow_whole_dollar_env=True)
assert config.path == "/opt/mt5/terminal64.exe"
def test_build_config_leaves_dollar_literal_by_default(
self,
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Test build_config does not substitute $ENV without opt-in."""
monkeypatch.setenv("MT5_SERVER", "Broker-Demo")
config = build_config(server="$MT5_SERVER")
assert config.server == "$MT5_SERVER"
def test_build_config_none_params_not_substituted(
self,
monkeypatch: pytest.MonkeyPatch, # noqa: ARG002
) -> None:
"""Test build_config with None params does not raise even with opt-in."""
config = build_config(allow_whole_dollar_env=True)
assert config.server is None
assert config.password is None
assert config.path is None
class TestThrottledHistoryUpdater:
"""Tests for the throttled incremental history updater."""
@@ -2043,3 +2285,154 @@ class TestThrottledHistoryUpdater:
assert updater.update(MagicMock(), []) is False
update.assert_not_called()
assert updater.last_update_monotonic is None
def test_default_update_backend_is_update_history(self) -> None:
"""Test the default backend resolves to update_history."""
updater = ThrottledHistoryUpdater(output="history.db")
assert updater.update_backend is update_history
def test_falsy_callable_update_backend_is_preserved(
self,
mocker: MockerFixture,
) -> None:
"""Test only None selects the default backend, not falsy callables."""
class FalsyCallable:
def __init__(self) -> None:
self.calls: list[dict[str, object]] = []
def __bool__(self) -> bool:
return False
def __call__(self, **kwargs: object) -> None:
self.calls.append(kwargs)
falsy_backend = FalsyCallable()
default_backend = mocker.patch("mt5cli.sdk.update_history")
updater = ThrottledHistoryUpdater(
output="history.db",
update_backend=falsy_backend,
)
assert updater.update_backend is falsy_backend
client = MagicMock()
assert updater.update(client, ["EURUSD"]) is True
assert len(falsy_backend.calls) == 1
assert falsy_backend.calls[0]["client"] is client
assert falsy_backend.calls[0]["symbols"] == ["EURUSD"]
default_backend.assert_not_called()
def test_custom_update_backend_receives_expected_kwargs(
self,
mocker: MockerFixture,
) -> None:
"""Test a custom backend receives update_history keyword arguments."""
backend = mocker.Mock()
client = MagicMock()
updater = ThrottledHistoryUpdater(
output="history.db",
datasets={Dataset.rates},
timeframes=["M1", "H1"],
flags="INFO",
lookback_hours=12.0,
with_views=True,
include_account_events=False,
update_backend=backend,
)
updater.update(client, ["EURUSD", "GBPUSD"])
backend.assert_called_once_with(
client=client,
output="history.db",
symbols=["EURUSD", "GBPUSD"],
datasets={Dataset.rates},
timeframes=["M1", "H1"],
flags="INFO",
lookback_hours=12.0,
with_views=True,
include_account_events=False,
)
def test_throttled_calls_do_not_invoke_custom_backend(
self,
mocker: MockerFixture,
) -> None:
"""Test throttled update cycles skip the injected backend."""
backend = mocker.Mock()
monotonic = mocker.patch("mt5cli.sdk.time.monotonic")
monotonic.side_effect = [100.0, 105.0, 200.0, 200.0]
client = MagicMock()
updater = ThrottledHistoryUpdater(
output="history.db",
interval_seconds=60,
update_backend=backend,
)
assert updater.update(client, ["EURUSD"]) is True
assert updater.update(client, ["EURUSD"]) is False
assert updater.update(client, ["EURUSD"]) is True
assert backend.call_count == 2
def test_successful_custom_backend_advances_throttle(
self,
mocker: MockerFixture,
) -> None:
"""Test a successful custom backend updates _last_update_monotonic."""
backend = mocker.Mock()
monotonic = mocker.patch("mt5cli.sdk.time.monotonic", return_value=42.0)
updater = ThrottledHistoryUpdater(
output="history.db",
update_backend=backend,
)
assert updater.update(MagicMock(), ["EURUSD"]) is True
assert updater.last_update_monotonic is monotonic.return_value
monotonic.assert_called_once()
def test_failed_custom_backend_does_not_advance_throttle(
self,
mocker: MockerFixture,
) -> None:
"""Test a failing custom backend leaves _last_update_monotonic unchanged."""
backend = mocker.Mock(side_effect=Mt5RuntimeError("boom"))
updater = ThrottledHistoryUpdater(
output="history.db",
update_backend=backend,
)
with pytest.raises(Mt5RuntimeError, match="boom"):
updater.update(MagicMock(), ["EURUSD"])
assert updater.last_update_monotonic is None
def test_custom_backend_suppresses_recoverable_errors_when_requested(
self,
mocker: MockerFixture,
) -> None:
"""Test suppress_errors swallows recoverable custom backend errors."""
backend = mocker.Mock(side_effect=Mt5RuntimeError("boom"))
updater = ThrottledHistoryUpdater(
output="history.db",
suppress_errors=True,
update_backend=backend,
)
assert updater.update(MagicMock(), ["EURUSD"]) is False
assert updater.last_update_monotonic is None
def test_custom_backend_propagates_errors_when_not_suppressed(
self,
mocker: MockerFixture,
) -> None:
"""Test recoverable custom backend errors propagate by default."""
backend = mocker.Mock(side_effect=Mt5RuntimeError("boom"))
updater = ThrottledHistoryUpdater(
output="history.db",
update_backend=backend,
)
with pytest.raises(Mt5RuntimeError, match="boom"):
updater.update(MagicMock(), ["EURUSD"])
assert updater.last_update_monotonic is None
+2550 -9
View File
File diff suppressed because it is too large Load Diff
Generated
+1 -1
View File
@@ -487,7 +487,7 @@ wheels = [
[[package]]
name = "mt5cli"
version = "0.7.2"
version = "0.9.0"
source = { editable = "." }
dependencies = [
{ name = "click" },