gavindiaz/mt5cli
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add fetch_latest_closed_rates_indexed and allow_whole_dollar_env opt-in (#45)

Browse Source 
* feat: add fetch_latest_closed_rates_indexed and allow_whole_dollar_env opt-in (#43, #44)

Closes #43: add fetch_latest_closed_rates_indexed(client, *, symbol,
granularity, count) -> pd.DataFrame to mt5cli/trading.py. Internally
reuses fetch_latest_closed_rates_for_trading_client(), converts the
"time" column to a UTC-aware DatetimeIndex named "time", and drops the
original column. Exported from trading.__all__, mt5cli.__init__, and
STABLE_SDK_EXPORTS.

Closes #44: extend substitute_env_placeholders() with opt-in
allow_whole_dollar_env=False that expands whole-value $ENV_NAME strings
(entire string must be exactly $IDENTIFIER). Threaded through
build_config(), resolve_account_spec(), and resolve_account_specs() with
the same default=False. Partial strings like "plan$pass", "abc$ENV", or
"$ENV-suffix" are never expanded.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: align Markdown table columns in docs and skill file

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: treat numeric (float64) epoch seconds as UTC in _rate_time_to_utc

After DataFrame concat or NA upcast the time column becomes float64, which
is still epoch seconds. Using is_numeric_dtype instead of is_integer_dtype
fixes the silent misalignment. Using series.to_numpy() before passing to
pd.to_datetime avoids the redundant pd.DatetimeIndex() wrapper and aligns
with how existing rate-time normalization in schemas.py handles numeric
timestamps.

Add test_converts_float_epoch_seconds_to_utc_datetime_index to cover the
regression. Add a doc note clarifying that build_config cannot expand
login since that parameter is int | None.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: reject NaT values after rate timestamp conversion in _rate_time_to_utc

pd.to_datetime() silently produces NaT for None/NaN inputs rather than
raising, so the function could return a DatetimeIndex containing NaT
despite documenting invalid timestamps as a ValueError. Check any(idx.isna())
after conversion and raise with a clear message.

Add test_raises_on_nat_time_column to cover the regression.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Bump version to v0.9.0

* fix: handle object numeric rate timestamps

---------

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Daichi Narushima
2026-06-22 22:52:19 +09:00
committed by GitHub
parent c4a4253fbc
commit 82a39731ed
13 changed files with 719 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/api/public-contract.md
+27 -19
View File
@@ -19,20 +19,27 @@ alias for new code.
### Session lifecycle and configuration
| Symbol | Role |
| ----------------------------------------------- | ---------------------------------------------------------------- |
| `MT5Client`, `Mt5CliClient` | Read-only data client with optional `order_check` / `order_send` |
| `build_config` | Build `pdmt5.Mt5Config` from connection fields |
| `mt5_session` | Context manager: initialize, login, yield client, shutdown |
| `create_trading_client`, `mt5_trading_session` | Trading-capable `pdmt5.Mt5TradingClient` lifecycle |
| `AccountSpec` | Generic account group: symbols plus optional credentials |
| `resolve_account_spec`, `resolve_account_specs` | Merge overrides and expand `${ENV_VAR}` placeholders |
| `substitute_env_placeholders` | Replace `${NAME}` substrings from the environment |
| Symbol | Role |
| ----------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
| `MT5Client`, `Mt5CliClient` | Read-only data client with optional `order_check` / `order_send` |
| `build_config` | Build `pdmt5.Mt5Config` from connection fields |
| `mt5_session` | Context manager: initialize, login, yield client, shutdown |
| `create_trading_client`, `mt5_trading_session` | Trading-capable `pdmt5.Mt5TradingClient` lifecycle |
| `AccountSpec` | Generic account group: symbols plus optional credentials |
| `resolve_account_spec`, `resolve_account_specs` | Merge overrides and expand `${ENV_VAR}` placeholders; opt-in `allow_whole_dollar_env` for bare `$NAME` |
| `substitute_env_placeholders` | Replace `${NAME}` substrings from the environment; opt-in `allow_whole_dollar_env` for whole-value `$NAME` |
Credential resolution is generic: any environment variable name may appear inside
`${...}`. mt5cli does not hard-code application-specific keys such as
`mt5_login` or `mt5_exe`.
Pass `allow_whole_dollar_env=True` to `substitute_env_placeholders()`,
`resolve_account_spec()`, `resolve_account_specs()`, and `build_config()` to
additionally expand strings whose entire value is a bare `$ENV_NAME` identifier.
Partial strings such as `"plan$pass"`, `"abc$ENV"`, or `"$ENV-suffix"` are
**never** expanded — only an exact `$IDENTIFIER` whole-string match qualifies.
Default is `False` to preserve backward compatibility.
### Read-only MT5 data access
Module-level helpers open a transient connection per call. Prefer `mt5_session`
@@ -56,15 +63,16 @@ MetaTrader 5 returns the still-forming bar as the last row when
`start_pos=0`. Use these helpers instead of reimplementing bar trimming or
timestamp normalization in downstream apps.
| Symbol | Role |
| ------------------------------------------------ | ------------------------------------------------------------ |
| `drop_forming_rate_bar` | Remove the last row from chronologically ordered rate data |
| `fetch_latest_closed_rates` | Single connected client: fetch `count + 1`, drop forming bar |
| `fetch_latest_closed_rates_for_trading_client` | Closed bars from an active `Mt5TradingClient` session |
| `collect_latest_closed_rates_for_accounts` | Multi-account closed bars with optional retry wrapper |
| `collect_latest_closed_rates_by_granularity` | Same data keyed by `(symbol, granularity_name)` |
| `collect_latest_rates_for_accounts` | Latest bars including the forming bar when `start_pos=0` |
| `collect_latest_rates_for_accounts_with_retries` | Bounded exponential backoff for transient MT5 errors |
| Symbol | Role |
| ------------------------------------------------ | ------------------------------------------------------------------------------- |
| `drop_forming_rate_bar` | Remove the last row from chronologically ordered rate data |
| `fetch_latest_closed_rates` | Single connected client: fetch `count + 1`, drop forming bar |
| `fetch_latest_closed_rates_for_trading_client` | Closed bars from an active `Mt5TradingClient` session; returns RangeIndex |
| `fetch_latest_closed_rates_indexed` | Same as above but returns a UTC `DatetimeIndex` named `"time"` (no time column) |
| `collect_latest_closed_rates_for_accounts` | Multi-account closed bars with optional retry wrapper |
| `collect_latest_closed_rates_by_granularity` | Same data keyed by `(symbol, granularity_name)` |
| `collect_latest_rates_for_accounts` | Latest bars including the forming bar when `start_pos=0` |
| `collect_latest_rates_for_accounts_with_retries` | Bounded exponential backoff for transient MT5 errors |
### SQLite history collection and rate loading
@@ -97,7 +105,7 @@ strategy entries, exits, Kelly sizing, or signal logic.
| `detect_position_side` | Net long / short / flat from open positions |
| `calculate_spread_ratio` | Relative bid-ask spread |
| `calculate_margin_and_volume`, `calculate_volume_by_margin`, `calculate_new_position_margin_ratio` | Margin budget and volume sizing |
| `normalize_order_volume`, `estimate_order_margin`, `calculate_positions_margin` | Broker volume normalization and margin totals |
| `normalize_order_volume`, `estimate_order_margin`, `calculate_positions_margin` | Broker volume normalization and margin totals |
| `determine_order_limits` | SL/TP price levels from ratios |
| `ensure_symbol_selected` | Select/verify Market Watch visibility |
| `place_market_order`, `close_open_positions`, `update_sltp_for_open_positions` | Order execution helpers (`dry_run` supported) |
docs/api/sdk.md
+22
View File
@@ -86,6 +86,28 @@ resolved = resolve_account_specs(accounts, server="Broker-Demo")
# resolved[0].login == "12345", resolved[0].server == "Broker-Demo"
```
Pass `allow_whole_dollar_env=True` to also expand strings whose **entire value**
is a bare `$ENV_NAME` identifier (no braces). This opt-in covers
`substitute_env_placeholders()`, `resolve_account_spec()`,
`resolve_account_specs()`, and `build_config()`. Note: `build_config` cannot
expand `login` because that parameter is `int | None`; use
`resolve_account_spec` for a string `login` placeholder. Partial strings such as
`"plan$pass"`, `"abc$ENV"`, or `"$ENV-suffix"` are never expanded — only an
exact `$IDENTIFIER` whole-string match qualifies. The default is `False` to
preserve backward compatibility.
```python
import os
from mt5cli import AccountSpec, resolve_account_specs
os.environ["MT5_PASSWORD"] = "secret"
accounts = [AccountSpec(symbols=["EURUSD"], password="$MT5_PASSWORD")]
resolved = resolve_account_specs(accounts, allow_whole_dollar_env=True)
# resolved[0].password == "secret"
```
### Throttled incremental history updates
`ThrottledHistoryUpdater` wraps `update_history()` with a minimum interval
docs/api/trading.md
+19 -10
View File
@@ -48,6 +48,7 @@ from mt5cli import (
determine_order_limits,
estimate_order_margin,
fetch_latest_closed_rates_for_trading_client,
fetch_latest_closed_rates_indexed,
get_account_snapshot,
get_positions_frame,
get_symbol_snapshot,
@@ -78,6 +79,14 @@ closed_bars = fetch_latest_closed_rates_for_trading_client(
granularity="M1",
count=100,
)
# Or fetch with a UTC DatetimeIndex instead of a "time" column:
indexed_bars = fetch_latest_closed_rates_indexed(
client,
symbol="EURUSD",
granularity="M1",
count=100,
)
# indexed_bars.index is a UTC-aware DatetimeIndex named "time"
sizing = calculate_margin_and_volume(
client,
"EURUSD",
@@ -174,16 +183,16 @@ through the stable package root without embedding entry/exit policy.
## Migration from application-local helpers
| Application-local concern | mt5cli replacement |
| -------------------------------------------------------- | ----------------------------------------------- |
| Manual terminal spawn/kill around trading code | `mt5_trading_session()` |
| Local position-side detection | `detect_position_side()` |
| Local margin/volume sizing | `calculate_margin_and_volume()` |
| Local broker volume step normalization | `normalize_order_volume()` |
| Local order or position margin estimation | `estimate_order_margin()`, `calculate_positions_margin()` |
| Local closed-bar fetch from a trading session | `fetch_latest_closed_rates_for_trading_client()` |
| Local SL/TP price derivation | `determine_order_limits()` |
| Throttled SQLite history loop with ad-hoc error handling | `ThrottledHistoryUpdater(suppress_errors=True)` |
| Application-local concern | mt5cli replacement |
| -------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| Manual terminal spawn/kill around trading code | `mt5_trading_session()` |
| Local position-side detection | `detect_position_side()` |
| Local margin/volume sizing | `calculate_margin_and_volume()` |
| Local broker volume step normalization | `normalize_order_volume()` |
| Local order or position margin estimation | `estimate_order_margin()`, `calculate_positions_margin()` |
| Local closed-bar fetch from a trading session | `fetch_latest_closed_rates_for_trading_client()`, `fetch_latest_closed_rates_indexed()` |
| Local SL/TP price derivation | `determine_order_limits()` |
| Throttled SQLite history loop with ad-hoc error handling | `ThrottledHistoryUpdater(suppress_errors=True)` |
Keep read-only data collection on `mt5_session()` / `Mt5CliClient`; use
`mt5_trading_session()` only where order placement or trading calculations are