feat: add injectable update_backend to ThrottledHistoryUpdater (#38)

* feat: add injectable update_backend to ThrottledHistoryUpdater Allow downstream applications to substitute the history update backend via the ThrottledHistoryUpdater constructor without monkey-patching mt5cli.sdk.update_history. Defaults to update_history for backward compatibility. Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * chore: fix lint and format after QA Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * chore: bump version to 0.8.2 Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> * fix: use explicit None check for ThrottledHistoryUpdater backend Only None selects the default update_history backend so falsy callable objects with __bool__ returning False are preserved as custom backends. Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Daichi Narushima <dceoy@users.noreply.github.com>
2026-06-18 22:58:09 +09:00
parent 897f7f0a0d
commit 7de3ce0b7a
7 changed files with 209 additions and 18 deletions
@@ -249,7 +249,7 @@ eurusd_m1 = rates["EURUSD", "M1"]  # closed bars only
 ```

 - **Credential resolution**: use `resolve_account_spec()` / `resolve_account_specs()` to merge explicit override values over `AccountSpec` fields and expand `${ENV_VAR}` placeholders (via `substitute_env_placeholders()`), raising `ValueError` for missing variables. This keeps secrets out of plan/config files without coupling to any strategy code.
- **Throttled history updates**: use `ThrottledHistoryUpdater` to wrap `update_history()` with a minimum `interval_seconds` between successful runs (monotonic clock). Call `should_update()` / `update(client, symbols)` from an application loop; errors propagate by default, or pass `suppress_errors=True` to swallow recoverable `Mt5*Error`, `sqlite3.Error`, `ValueError`, `OSError`, and MT5 client capability errors for history API methods without advancing the throttle (other `AttributeError` / `TypeError` values always propagate).
+- **Throttled history updates**: use `ThrottledHistoryUpdater` to wrap `update_history()` with a minimum `interval_seconds` between successful runs (monotonic clock). Call `should_update()` / `update(client, symbols)` from an application loop; errors propagate by default, or pass `suppress_errors=True` to swallow recoverable `Mt5*Error`, `sqlite3.Error`, `ValueError`, `OSError`, and MT5 client capability errors for history API methods without advancing the throttle (other `AttributeError` / `TypeError` values always propagate). Pass `update_backend` to inject a custom history update callable (same keyword arguments as `update_history`) instead of monkey-patching `mt5cli.sdk.update_history`.
 - **Trading session helpers**: use `mt5_trading_session()` for a trading-capable `pdmt5.Mt5TradingClient` that initializes/logs in via `Mt5Config.path` and always shuts down safely. Pair with `detect_position_side()`, `calculate_margin_and_volume()`, and `determine_order_limits()` for generic position and sizing utilities. The read-only `mt5_session()` / `Mt5CliClient` SDK is unchanged.
 - **Granularity-keyed rate loading**: `load_rate_series_by_granularity()` builds targets with `build_rate_targets()`, loads them with `load_rate_series_from_sqlite()`, and returns a mapping keyed by `(symbol | None, granularity_name)` such as `("EURUSD", "M1")` to reduce downstream boilerplate.
 - **MT5 session helper**: use the `mt5_session()` context manager to attach to (or, when `Mt5Config.path` is set, launch) an MT5 terminal, log in, and yield a connected `MT5Client` that shuts down on exit.
@@ -67,16 +67,16 @@ timestamp normalization in downstream apps.

 ### SQLite history collection and rate loading

-| Symbol                                                                                                                        | Role                                                                 |
-| ----------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------- |
-| `collect_history`                                                                                                             | One-shot date-range export into SQLite                               |
-| `update_history`, `update_history_with_config`                                                                                | Incremental append from `MAX(time)` cursors                          |
-| `ThrottledHistoryUpdater`                                                                                                     | Minimum interval between successful incremental updates              |
-| `resolve_history_datasets`, `resolve_history_timeframes`, `resolve_history_tick_flags`                                        | History pipeline configuration                                       |
-| `build_rate_view_name`, `resolve_rate_table_name`, `resolve_rate_view_name`, `resolve_rate_view_names`, `resolve_rate_tables` | Map symbols/timeframes to mt5cli-managed table or view names         |
-| `RateTarget`, `build_rate_targets`                                                                                            | Neutral `(symbol, timeframe)` series descriptors                     |
-| `load_rate_data`, `load_rate_data_from_connection`                                                                            | Load one table/view into a time-indexed DataFrame                    |
-| `load_rate_series_from_sqlite`, `load_rate_series_by_granularity`                                                             | Load one or many series; fail clearly when managed views are missing |
+| Symbol                                                                                                                        | Role                                                                                         |
+| ----------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
+| `collect_history`                                                                                                             | One-shot date-range export into SQLite                                                       |
+| `update_history`, `update_history_with_config`                                                                                | Incremental append from `MAX(time)` cursors                                                  |
+| `ThrottledHistoryUpdater`                                                                                                     | Minimum interval between successful incremental updates; optional `update_backend` injection |
+| `resolve_history_datasets`, `resolve_history_timeframes`, `resolve_history_tick_flags`                                        | History pipeline configuration                                                               |
+| `build_rate_view_name`, `resolve_rate_table_name`, `resolve_rate_view_name`, `resolve_rate_view_names`, `resolve_rate_tables` | Map symbols/timeframes to mt5cli-managed table or view names                                 |
+| `RateTarget`, `build_rate_targets`                                                                                            | Neutral `(symbol, timeframe)` series descriptors                                             |
+| `load_rate_data`, `load_rate_data_from_connection`                                                                            | Load one table/view into a time-indexed DataFrame                                            |
+| `load_rate_series_from_sqlite`, `load_rate_series_by_granularity`                                                             | Load one or many series; fail clearly when managed views are missing                         |

 Pass `require_existing=True` to rate view resolution helpers when downstream
 code must fail instead of receiving a best-guess view name. Multi-series loaders
@@ -113,6 +113,28 @@ finally:
    client.shutdown()
 ```

+Pass `update_backend` to substitute the default `update_history` implementation
+without monkey-patching `mt5cli.sdk.update_history`. The callable receives the
+same keyword arguments as `update_history` (`client`, `output`, `symbols`,
+`datasets`, `timeframes`, `flags`, `lookback_hours`, `with_views`,
+`include_account_events`). The resolved backend is stored on
+`updater.update_backend` for inspection or subclassing.
+
+```python
+from mt5cli import ThrottledHistoryUpdater, update_history
+
+
+def app_update_history(**kwargs) -> None:
+    update_history(**kwargs)  # or delegate to application-specific logic
+
+
+updater = ThrottledHistoryUpdater(
+    output="history.db",
+    interval_seconds=60,
+    update_backend=app_update_history,
+)
+```
+
 By default recoverable errors (`Mt5TradingError`, `Mt5RuntimeError`,
 `sqlite3.Error`, `ValueError`, `OSError`, and MT5 client capability
 `AttributeError` / `TypeError` for history API methods) propagate so the caller
@@ -42,6 +42,8 @@ from .utils import coerce_login as _coerce_login
 if TYPE_CHECKING:
    from collections.abc import Callable, Iterator, Sequence

+    UpdateHistoryBackend = Callable[..., None]
+
 T = TypeVar("T")

 logger = logging.getLogger(__name__)
@@ -1044,10 +1046,16 @@ def update_history_with_config(  # noqa: PLR0913
 class ThrottledHistoryUpdater:
    """Throttled incremental SQLite history updater for long-running apps.

-    Wraps :func:`update_history` with a minimum interval between successful
-    updates, so a tight application loop can call :meth:`update` every
-    iteration without re-fetching MT5 history more often than desired. Timing
-    uses a monotonic clock, so it is unaffected by wall-clock changes.
+    Wraps :func:`update_history` (or a custom ``update_backend``) with a minimum
+    interval between successful updates, so a tight application loop can call
+    :meth:`update` every iteration without re-fetching MT5 history more often
+    than desired. Timing uses a monotonic clock, so it is unaffected by
+    wall-clock changes.
+
+    Downstream applications may pass ``update_backend`` to substitute the
+    default :func:`update_history` implementation—for example to add
+    application-specific logging, metrics, or test doubles—without monkey-
+    patching ``mt5cli.sdk.update_history``.
    """

    def __init__(
@@ -1062,6 +1070,7 @@ class ThrottledHistoryUpdater:
        include_account_events: bool = True,
        interval_seconds: float = 0.0,
        suppress_errors: bool = False,
+        update_backend: UpdateHistoryBackend | None = None,
    ) -> None:
        """Initialize the throttled updater.

@@ -1085,6 +1094,12 @@ class ThrottledHistoryUpdater:
                the throttle. Other ``AttributeError`` / ``TypeError`` values
                always propagate. When False (default), recoverable errors
                propagate so callers control logging.
+            update_backend: Callable invoked instead of :func:`update_history`
+                when :meth:`update` runs. Receives the same keyword arguments as
+                :func:`update_history` (``client``, ``output``, ``symbols``,
+                ``datasets``, ``timeframes``, ``flags``, ``lookback_hours``,
+                ``with_views``, ``include_account_events``). Defaults to
+                :func:`update_history`.
        """
        self.output = output
        self.datasets = datasets
@@ -1095,6 +1110,9 @@ class ThrottledHistoryUpdater:
        self.include_account_events = include_account_events
        self.interval_seconds = interval_seconds
        self.suppress_errors = suppress_errors
+        self.update_backend = (
+            update_history if update_backend is None else update_backend
+        )
        self._last_update_monotonic: float | None = None

    @property
@@ -1145,7 +1163,7 @@ class ThrottledHistoryUpdater:
                lookback_hours=self.lookback_hours,
                date_to=None,
            )
-            update_history(
+            self.update_backend(
                client=client,
                output=self.output,
                symbols=symbols,
@@ -1,6 +1,6 @@
 [project]
 name = "mt5cli"
-version = "0.8.1"
+version = "0.8.2"
 description = "Generic MT5 data and execution infrastructure for Python applications"
 authors = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
 maintainers = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
@@ -2100,3 +2100,154 @@ class TestThrottledHistoryUpdater:
        assert updater.update(MagicMock(), []) is False
        update.assert_not_called()
        assert updater.last_update_monotonic is None
+
+    def test_default_update_backend_is_update_history(self) -> None:
+        """Test the default backend resolves to update_history."""
+        updater = ThrottledHistoryUpdater(output="history.db")
+        assert updater.update_backend is update_history
+
+    def test_falsy_callable_update_backend_is_preserved(
+        self,
+        mocker: MockerFixture,
+    ) -> None:
+        """Test only None selects the default backend, not falsy callables."""
+
+        class FalsyCallable:
+            def __init__(self) -> None:
+                self.calls: list[dict[str, object]] = []
+
+            def __bool__(self) -> bool:
+                return False
+
+            def __call__(self, **kwargs: object) -> None:
+                self.calls.append(kwargs)
+
+        falsy_backend = FalsyCallable()
+        default_backend = mocker.patch("mt5cli.sdk.update_history")
+        updater = ThrottledHistoryUpdater(
+            output="history.db",
+            update_backend=falsy_backend,
+        )
+
+        assert updater.update_backend is falsy_backend
+        client = MagicMock()
+        assert updater.update(client, ["EURUSD"]) is True
+        assert len(falsy_backend.calls) == 1
+        assert falsy_backend.calls[0]["client"] is client
+        assert falsy_backend.calls[0]["symbols"] == ["EURUSD"]
+        default_backend.assert_not_called()
+
+    def test_custom_update_backend_receives_expected_kwargs(
+        self,
+        mocker: MockerFixture,
+    ) -> None:
+        """Test a custom backend receives update_history keyword arguments."""
+        backend = mocker.Mock()
+        client = MagicMock()
+        updater = ThrottledHistoryUpdater(
+            output="history.db",
+            datasets={Dataset.rates},
+            timeframes=["M1", "H1"],
+            flags="INFO",
+            lookback_hours=12.0,
+            with_views=True,
+            include_account_events=False,
+            update_backend=backend,
+        )
+
+        updater.update(client, ["EURUSD", "GBPUSD"])
+
+        backend.assert_called_once_with(
+            client=client,
+            output="history.db",
+            symbols=["EURUSD", "GBPUSD"],
+            datasets={Dataset.rates},
+            timeframes=["M1", "H1"],
+            flags="INFO",
+            lookback_hours=12.0,
+            with_views=True,
+            include_account_events=False,
+        )
+
+    def test_throttled_calls_do_not_invoke_custom_backend(
+        self,
+        mocker: MockerFixture,
+    ) -> None:
+        """Test throttled update cycles skip the injected backend."""
+        backend = mocker.Mock()
+        monotonic = mocker.patch("mt5cli.sdk.time.monotonic")
+        monotonic.side_effect = [100.0, 105.0, 200.0, 200.0]
+        client = MagicMock()
+        updater = ThrottledHistoryUpdater(
+            output="history.db",
+            interval_seconds=60,
+            update_backend=backend,
+        )
+
+        assert updater.update(client, ["EURUSD"]) is True
+        assert updater.update(client, ["EURUSD"]) is False
+        assert updater.update(client, ["EURUSD"]) is True
+        assert backend.call_count == 2
+
+    def test_successful_custom_backend_advances_throttle(
+        self,
+        mocker: MockerFixture,
+    ) -> None:
+        """Test a successful custom backend updates _last_update_monotonic."""
+        backend = mocker.Mock()
+        monotonic = mocker.patch("mt5cli.sdk.time.monotonic", return_value=42.0)
+        updater = ThrottledHistoryUpdater(
+            output="history.db",
+            update_backend=backend,
+        )
+
+        assert updater.update(MagicMock(), ["EURUSD"]) is True
+        assert updater.last_update_monotonic is monotonic.return_value
+        monotonic.assert_called_once()
+
+    def test_failed_custom_backend_does_not_advance_throttle(
+        self,
+        mocker: MockerFixture,
+    ) -> None:
+        """Test a failing custom backend leaves _last_update_monotonic unchanged."""
+        backend = mocker.Mock(side_effect=Mt5RuntimeError("boom"))
+        updater = ThrottledHistoryUpdater(
+            output="history.db",
+            update_backend=backend,
+        )
+
+        with pytest.raises(Mt5RuntimeError, match="boom"):
+            updater.update(MagicMock(), ["EURUSD"])
+
+        assert updater.last_update_monotonic is None
+
+    def test_custom_backend_suppresses_recoverable_errors_when_requested(
+        self,
+        mocker: MockerFixture,
+    ) -> None:
+        """Test suppress_errors swallows recoverable custom backend errors."""
+        backend = mocker.Mock(side_effect=Mt5RuntimeError("boom"))
+        updater = ThrottledHistoryUpdater(
+            output="history.db",
+            suppress_errors=True,
+            update_backend=backend,
+        )
+
+        assert updater.update(MagicMock(), ["EURUSD"]) is False
+        assert updater.last_update_monotonic is None
+
+    def test_custom_backend_propagates_errors_when_not_suppressed(
+        self,
+        mocker: MockerFixture,
+    ) -> None:
+        """Test recoverable custom backend errors propagate by default."""
+        backend = mocker.Mock(side_effect=Mt5RuntimeError("boom"))
+        updater = ThrottledHistoryUpdater(
+            output="history.db",
+            update_backend=backend,
+        )
+
+        with pytest.raises(Mt5RuntimeError, match="boom"):
+            updater.update(MagicMock(), ["EURUSD"])
+
+        assert updater.last_update_monotonic is None
@@ -487,7 +487,7 @@ wheels = [

 [[package]]
 name = "mt5cli"
-version = "0.8.1"
+version = "0.8.2"
 source = { editable = "." }
 dependencies = [
    { name = "click" },