diff --git a/README.md b/README.md index 085aca3..0942232 100644 --- a/README.md +++ b/README.md @@ -249,7 +249,7 @@ eurusd_m1 = rates["EURUSD", "M1"] # closed bars only ``` - **Credential resolution**: use `resolve_account_spec()` / `resolve_account_specs()` to merge explicit override values over `AccountSpec` fields and expand `${ENV_VAR}` placeholders (via `substitute_env_placeholders()`), raising `ValueError` for missing variables. This keeps secrets out of plan/config files without coupling to any strategy code. -- **Throttled history updates**: use `ThrottledHistoryUpdater` to wrap `update_history()` with a minimum `interval_seconds` between successful runs (monotonic clock). Call `should_update()` / `update(client, symbols)` from an application loop; errors propagate by default, or pass `suppress_errors=True` to swallow recoverable `Mt5*Error`, `sqlite3.Error`, `ValueError`, `OSError`, and MT5 client capability errors for history API methods without advancing the throttle (other `AttributeError` / `TypeError` values always propagate). +- **Throttled history updates**: use `ThrottledHistoryUpdater` to wrap `update_history()` with a minimum `interval_seconds` between successful runs (monotonic clock). Call `should_update()` / `update(client, symbols)` from an application loop; errors propagate by default, or pass `suppress_errors=True` to swallow recoverable `Mt5*Error`, `sqlite3.Error`, `ValueError`, `OSError`, and MT5 client capability errors for history API methods without advancing the throttle (other `AttributeError` / `TypeError` values always propagate). Pass `update_backend` to inject a custom history update callable (same keyword arguments as `update_history`) instead of monkey-patching `mt5cli.sdk.update_history`. - **Trading session helpers**: use `mt5_trading_session()` for a trading-capable `pdmt5.Mt5TradingClient` that initializes/logs in via `Mt5Config.path` and always shuts down safely. Pair with `detect_position_side()`, `calculate_margin_and_volume()`, and `determine_order_limits()` for generic position and sizing utilities. The read-only `mt5_session()` / `Mt5CliClient` SDK is unchanged. - **Granularity-keyed rate loading**: `load_rate_series_by_granularity()` builds targets with `build_rate_targets()`, loads them with `load_rate_series_from_sqlite()`, and returns a mapping keyed by `(symbol | None, granularity_name)` such as `("EURUSD", "M1")` to reduce downstream boilerplate. - **MT5 session helper**: use the `mt5_session()` context manager to attach to (or, when `Mt5Config.path` is set, launch) an MT5 terminal, log in, and yield a connected `MT5Client` that shuts down on exit. diff --git a/docs/api/public-contract.md b/docs/api/public-contract.md index adc3cd6..bd8244d 100644 --- a/docs/api/public-contract.md +++ b/docs/api/public-contract.md @@ -67,16 +67,16 @@ timestamp normalization in downstream apps. ### SQLite history collection and rate loading -| Symbol | Role | -| ----------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------- | -| `collect_history` | One-shot date-range export into SQLite | -| `update_history`, `update_history_with_config` | Incremental append from `MAX(time)` cursors | -| `ThrottledHistoryUpdater` | Minimum interval between successful incremental updates | -| `resolve_history_datasets`, `resolve_history_timeframes`, `resolve_history_tick_flags` | History pipeline configuration | -| `build_rate_view_name`, `resolve_rate_table_name`, `resolve_rate_view_name`, `resolve_rate_view_names`, `resolve_rate_tables` | Map symbols/timeframes to mt5cli-managed table or view names | -| `RateTarget`, `build_rate_targets` | Neutral `(symbol, timeframe)` series descriptors | -| `load_rate_data`, `load_rate_data_from_connection` | Load one table/view into a time-indexed DataFrame | -| `load_rate_series_from_sqlite`, `load_rate_series_by_granularity` | Load one or many series; fail clearly when managed views are missing | +| Symbol | Role | +| ----------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | +| `collect_history` | One-shot date-range export into SQLite | +| `update_history`, `update_history_with_config` | Incremental append from `MAX(time)` cursors | +| `ThrottledHistoryUpdater` | Minimum interval between successful incremental updates; optional `update_backend` injection | +| `resolve_history_datasets`, `resolve_history_timeframes`, `resolve_history_tick_flags` | History pipeline configuration | +| `build_rate_view_name`, `resolve_rate_table_name`, `resolve_rate_view_name`, `resolve_rate_view_names`, `resolve_rate_tables` | Map symbols/timeframes to mt5cli-managed table or view names | +| `RateTarget`, `build_rate_targets` | Neutral `(symbol, timeframe)` series descriptors | +| `load_rate_data`, `load_rate_data_from_connection` | Load one table/view into a time-indexed DataFrame | +| `load_rate_series_from_sqlite`, `load_rate_series_by_granularity` | Load one or many series; fail clearly when managed views are missing | Pass `require_existing=True` to rate view resolution helpers when downstream code must fail instead of receiving a best-guess view name. Multi-series loaders diff --git a/docs/api/sdk.md b/docs/api/sdk.md index 7b2b218..9621099 100644 --- a/docs/api/sdk.md +++ b/docs/api/sdk.md @@ -113,6 +113,28 @@ finally: client.shutdown() ``` +Pass `update_backend` to substitute the default `update_history` implementation +without monkey-patching `mt5cli.sdk.update_history`. The callable receives the +same keyword arguments as `update_history` (`client`, `output`, `symbols`, +`datasets`, `timeframes`, `flags`, `lookback_hours`, `with_views`, +`include_account_events`). The resolved backend is stored on +`updater.update_backend` for inspection or subclassing. + +```python +from mt5cli import ThrottledHistoryUpdater, update_history + + +def app_update_history(**kwargs) -> None: + update_history(**kwargs) # or delegate to application-specific logic + + +updater = ThrottledHistoryUpdater( + output="history.db", + interval_seconds=60, + update_backend=app_update_history, +) +``` + By default recoverable errors (`Mt5TradingError`, `Mt5RuntimeError`, `sqlite3.Error`, `ValueError`, `OSError`, and MT5 client capability `AttributeError` / `TypeError` for history API methods) propagate so the caller diff --git a/mt5cli/sdk.py b/mt5cli/sdk.py index bff6eee..94e92db 100644 --- a/mt5cli/sdk.py +++ b/mt5cli/sdk.py @@ -42,6 +42,8 @@ from .utils import coerce_login as _coerce_login if TYPE_CHECKING: from collections.abc import Callable, Iterator, Sequence + UpdateHistoryBackend = Callable[..., None] + T = TypeVar("T") logger = logging.getLogger(__name__) @@ -1044,10 +1046,16 @@ def update_history_with_config( # noqa: PLR0913 class ThrottledHistoryUpdater: """Throttled incremental SQLite history updater for long-running apps. - Wraps :func:`update_history` with a minimum interval between successful - updates, so a tight application loop can call :meth:`update` every - iteration without re-fetching MT5 history more often than desired. Timing - uses a monotonic clock, so it is unaffected by wall-clock changes. + Wraps :func:`update_history` (or a custom ``update_backend``) with a minimum + interval between successful updates, so a tight application loop can call + :meth:`update` every iteration without re-fetching MT5 history more often + than desired. Timing uses a monotonic clock, so it is unaffected by + wall-clock changes. + + Downstream applications may pass ``update_backend`` to substitute the + default :func:`update_history` implementation—for example to add + application-specific logging, metrics, or test doubles—without monkey- + patching ``mt5cli.sdk.update_history``. """ def __init__( @@ -1062,6 +1070,7 @@ class ThrottledHistoryUpdater: include_account_events: bool = True, interval_seconds: float = 0.0, suppress_errors: bool = False, + update_backend: UpdateHistoryBackend | None = None, ) -> None: """Initialize the throttled updater. @@ -1085,6 +1094,12 @@ class ThrottledHistoryUpdater: the throttle. Other ``AttributeError`` / ``TypeError`` values always propagate. When False (default), recoverable errors propagate so callers control logging. + update_backend: Callable invoked instead of :func:`update_history` + when :meth:`update` runs. Receives the same keyword arguments as + :func:`update_history` (``client``, ``output``, ``symbols``, + ``datasets``, ``timeframes``, ``flags``, ``lookback_hours``, + ``with_views``, ``include_account_events``). Defaults to + :func:`update_history`. """ self.output = output self.datasets = datasets @@ -1095,6 +1110,9 @@ class ThrottledHistoryUpdater: self.include_account_events = include_account_events self.interval_seconds = interval_seconds self.suppress_errors = suppress_errors + self.update_backend = ( + update_history if update_backend is None else update_backend + ) self._last_update_monotonic: float | None = None @property @@ -1145,7 +1163,7 @@ class ThrottledHistoryUpdater: lookback_hours=self.lookback_hours, date_to=None, ) - update_history( + self.update_backend( client=client, output=self.output, symbols=symbols, diff --git a/pyproject.toml b/pyproject.toml index 2e1581a..9cfbf68 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "mt5cli" -version = "0.8.1" +version = "0.8.2" description = "Generic MT5 data and execution infrastructure for Python applications" authors = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}] maintainers = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}] diff --git a/tests/test_sdk.py b/tests/test_sdk.py index 4aef807..5872934 100644 --- a/tests/test_sdk.py +++ b/tests/test_sdk.py @@ -2100,3 +2100,154 @@ class TestThrottledHistoryUpdater: assert updater.update(MagicMock(), []) is False update.assert_not_called() assert updater.last_update_monotonic is None + + def test_default_update_backend_is_update_history(self) -> None: + """Test the default backend resolves to update_history.""" + updater = ThrottledHistoryUpdater(output="history.db") + assert updater.update_backend is update_history + + def test_falsy_callable_update_backend_is_preserved( + self, + mocker: MockerFixture, + ) -> None: + """Test only None selects the default backend, not falsy callables.""" + + class FalsyCallable: + def __init__(self) -> None: + self.calls: list[dict[str, object]] = [] + + def __bool__(self) -> bool: + return False + + def __call__(self, **kwargs: object) -> None: + self.calls.append(kwargs) + + falsy_backend = FalsyCallable() + default_backend = mocker.patch("mt5cli.sdk.update_history") + updater = ThrottledHistoryUpdater( + output="history.db", + update_backend=falsy_backend, + ) + + assert updater.update_backend is falsy_backend + client = MagicMock() + assert updater.update(client, ["EURUSD"]) is True + assert len(falsy_backend.calls) == 1 + assert falsy_backend.calls[0]["client"] is client + assert falsy_backend.calls[0]["symbols"] == ["EURUSD"] + default_backend.assert_not_called() + + def test_custom_update_backend_receives_expected_kwargs( + self, + mocker: MockerFixture, + ) -> None: + """Test a custom backend receives update_history keyword arguments.""" + backend = mocker.Mock() + client = MagicMock() + updater = ThrottledHistoryUpdater( + output="history.db", + datasets={Dataset.rates}, + timeframes=["M1", "H1"], + flags="INFO", + lookback_hours=12.0, + with_views=True, + include_account_events=False, + update_backend=backend, + ) + + updater.update(client, ["EURUSD", "GBPUSD"]) + + backend.assert_called_once_with( + client=client, + output="history.db", + symbols=["EURUSD", "GBPUSD"], + datasets={Dataset.rates}, + timeframes=["M1", "H1"], + flags="INFO", + lookback_hours=12.0, + with_views=True, + include_account_events=False, + ) + + def test_throttled_calls_do_not_invoke_custom_backend( + self, + mocker: MockerFixture, + ) -> None: + """Test throttled update cycles skip the injected backend.""" + backend = mocker.Mock() + monotonic = mocker.patch("mt5cli.sdk.time.monotonic") + monotonic.side_effect = [100.0, 105.0, 200.0, 200.0] + client = MagicMock() + updater = ThrottledHistoryUpdater( + output="history.db", + interval_seconds=60, + update_backend=backend, + ) + + assert updater.update(client, ["EURUSD"]) is True + assert updater.update(client, ["EURUSD"]) is False + assert updater.update(client, ["EURUSD"]) is True + assert backend.call_count == 2 + + def test_successful_custom_backend_advances_throttle( + self, + mocker: MockerFixture, + ) -> None: + """Test a successful custom backend updates _last_update_monotonic.""" + backend = mocker.Mock() + monotonic = mocker.patch("mt5cli.sdk.time.monotonic", return_value=42.0) + updater = ThrottledHistoryUpdater( + output="history.db", + update_backend=backend, + ) + + assert updater.update(MagicMock(), ["EURUSD"]) is True + assert updater.last_update_monotonic is monotonic.return_value + monotonic.assert_called_once() + + def test_failed_custom_backend_does_not_advance_throttle( + self, + mocker: MockerFixture, + ) -> None: + """Test a failing custom backend leaves _last_update_monotonic unchanged.""" + backend = mocker.Mock(side_effect=Mt5RuntimeError("boom")) + updater = ThrottledHistoryUpdater( + output="history.db", + update_backend=backend, + ) + + with pytest.raises(Mt5RuntimeError, match="boom"): + updater.update(MagicMock(), ["EURUSD"]) + + assert updater.last_update_monotonic is None + + def test_custom_backend_suppresses_recoverable_errors_when_requested( + self, + mocker: MockerFixture, + ) -> None: + """Test suppress_errors swallows recoverable custom backend errors.""" + backend = mocker.Mock(side_effect=Mt5RuntimeError("boom")) + updater = ThrottledHistoryUpdater( + output="history.db", + suppress_errors=True, + update_backend=backend, + ) + + assert updater.update(MagicMock(), ["EURUSD"]) is False + assert updater.last_update_monotonic is None + + def test_custom_backend_propagates_errors_when_not_suppressed( + self, + mocker: MockerFixture, + ) -> None: + """Test recoverable custom backend errors propagate by default.""" + backend = mocker.Mock(side_effect=Mt5RuntimeError("boom")) + updater = ThrottledHistoryUpdater( + output="history.db", + update_backend=backend, + ) + + with pytest.raises(Mt5RuntimeError, match="boom"): + updater.update(MagicMock(), ["EURUSD"]) + + assert updater.last_update_monotonic is None diff --git a/uv.lock b/uv.lock index af48da2..17604fb 100644 --- a/uv.lock +++ b/uv.lock @@ -487,7 +487,7 @@ wheels = [ [[package]] name = "mt5cli" -version = "0.8.1" +version = "0.8.2" source = { editable = "." } dependencies = [ { name = "click" },