Add incremental SQLite history SDK (#16)

* Add incremental SQLite history SDK for automated pipelines.

Extract sqlite history helpers into a dedicated module and expose update_history APIs that resume from existing MAX(time) values instead of re-fetching fixed date ranges.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix incremental history deals and stale rate view cleanup.

Fetch account events once during incremental updates, drop stale rate_* views when timeframes change, and avoid SQLite variable limits on wide frames.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix incremental deal filtering edge cases

Co-authored-by: Cursor <cursoragent@cursor.com>

* Address PR review feedback for incremental SQLite history.

Make rate views collision-free, batch incremental resume queries, scope deduplication to appended boundaries, validate before opening MT5, use atomic SQLite transactions, and expand docs/tests for the new helpers.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Document collect-history SQLite schema with ER diagram.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix account-event filtering and drop legacy rates resume.

Account events must follow only account_event_start, not per-symbol trade
cursors. Require normalized rates schema and fail fast when timeframe is missing.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Validate normalized rates schema before incremental resume.

Require symbol, timeframe, and time on existing rates tables with clear
ValueError messages, and add regression tests for malformed schemas.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>

README.md

@@ -87,7 +87,46 @@ mt5cli -o history.db collect-history \
   --timeframe M1 --flags ALL --if-exists append --with-views
 ```
 
-History orders and deals are fetched per symbol and concatenated, so the symbol filter is applied consistently across all datasets. The `cash_events` view is derived from symbol-filtered `history_deals`, so account-level cash events with empty or non-matching symbols may be excluded. The `rates` table records the requested `timeframe` so appended runs at different timeframes remain distinguishable. The `positions_reconstructed` view aggregates trade deals by `position_id`, excludes positions without closing deals, and uses volume-weighted open/close prices; reversal deals (`DEAL_ENTRY_INOUT`) are reported via `volume_reversal` / `reversal_count` columns and do not contribute to the weighted prices.
+History orders and deals are fetched per symbol and concatenated, so the symbol filter is applied consistently across all datasets. The `cash_events` view is derived from symbol-filtered `history_deals`, so account-level cash events with empty or non-matching symbols may be excluded. The `rates` table records the requested `timeframe` so appended runs at different timeframes remain distinguishable. The `positions_reconstructed` view aggregates trade deals by `position_id`, excludes positions without closing-side entries, and uses volume-weighted open/close prices; reversal deals (`DEAL_ENTRY_INOUT`) are reported via `volume_reversal` / `reversal_count` columns.
+
+### Incremental history SDK
+
+For automated pipelines, use the importable incremental API instead of re-fetching fixed date ranges:
+
+```python
+from pdmt5 import Mt5Config, Mt5DataClient
+from mt5cli import Dataset, update_history, update_history_with_config
+
+# Reuse an already-connected pdmt5 client (does not open/close MT5)
+client = Mt5DataClient(config=Mt5Config(login=12345))
+client.initialize_and_login_mt5()
+try:
+    update_history(
+        client=client,
+        output="history.db",
+        symbols=["EURUSD", "GBPUSD"],
+        datasets={Dataset.rates, Dataset.history_deals},
+        timeframes=["M1", "H1"],  # default: all fixed MT5 timeframes
+        lookback_hours=24,
+        create_rate_views=True,
+        with_views=True,
+        include_account_events=True,
+    )
+finally:
+    client.shutdown()
+
+# Standalone wrapper that opens and closes MT5 for you
+update_history_with_config(
+    output="history.db",
+    symbols=["EURUSD"],
+    config=Mt5Config(login=12345),
+)
+```
+
+- **`collect-history`**: explicit date-range export into SQLite.
+- **`update_history`**: incremental append based on existing SQLite `MAX(time)` per symbol (and timeframe for rates); account-level deals use a separate cursor when `include_account_events=True`.
+- **`rates` table**: normalized storage with `symbol` and `timeframe` columns.
+- **Rate compatibility views**: mt5cli manages all `rate_*` views. Naming is `rate_<symbol>__<timeframe>` when a symbol has one timeframe, otherwise `rate_<symbol>__<granularity>_<timeframe>` (for example `rate_EURUSD__M1_1`). Stale `rate_*` views are dropped and recreated when rates change for offline tools such as mteor optimize.
 
 ## Requirements