Add collect-history command for bulk data collection (#14)
* Add collect-history command for bulk SQLite export Bundles rates, ticks, history-orders, and history-deals for one or more symbols into a single SQLite database. Tick collection uses copy_ticks_range_as_df with a --flags option defaulting to ALL. With --with-views, optional cash_events and positions_reconstructed views are derived from history_deals when the required columns are present. * Extend collect-history with datasets, if-exists, timeframe, view fixes - Fetch history-orders and history-deals per symbol so --symbol applies consistently across all four datasets. - Add repeatable --dataset (rates, ticks, history-orders, history-deals) so ticks are no longer required and any subset can be collected. - Add --if-exists append|replace|fail to control SQLite table conflict behavior instead of hard-coding replace. - Record the requested timeframe in a timeframe column on the rates table so appended runs at different timeframes stay distinguishable. - Fix positions_reconstructed to exclude positions with no closing deals, use volume-weighted open/close prices, and report reversal deals (DEAL_ENTRY_INOUT) via volume_reversal / reversal_count without contributing to weighted prices. - Update tests, README, docs, and skill to match. * Address collect-history review feedback * Stream collect-history writes per symbol * Address PR cleanup for collect-history --------- Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -85,6 +85,35 @@ mt5cli --login 12345 --password mypass --server MyBroker-Demo \
|
||||
|
||||
Use `order-check` to validate a request payload before running `order-send --yes`.
|
||||
|
||||
### Bulk Collection
|
||||
|
||||
| Command | Description |
|
||||
| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `collect-history` | Collect rates, ticks, history-orders, and history-deals for one or more symbols into a single SQLite database (optional cash-event/position views) |
|
||||
|
||||
```bash
|
||||
mt5cli -o history.db collect-history \
|
||||
--symbol EURUSD --symbol GBPUSD \
|
||||
--date-from 2024-01-01 --date-to 2024-02-01 \
|
||||
--dataset rates --dataset history-deals \
|
||||
--timeframe M1 --flags ALL --if-exists append --with-views
|
||||
```
|
||||
|
||||
`collect-history` options:
|
||||
|
||||
| Option | Default | Description |
|
||||
| -------------- | ---------- | --------------------------------------------------------------------------------------------- |
|
||||
| `--symbol/-s` | _required_ | Symbol to collect (repeat for multiple). |
|
||||
| `--date-from` | _required_ | Start date in ISO 8601. |
|
||||
| `--date-to` | _required_ | End date in ISO 8601. |
|
||||
| `--dataset` | all four | Repeatable: `rates`, `ticks`, `history-orders`, `history-deals`. |
|
||||
| `--timeframe` | `M1` | Rates timeframe; recorded in a `timeframe` column on the `rates` table. |
|
||||
| `--flags` | `ALL` | Tick copy flags forwarded to `copy_ticks_range`. |
|
||||
| `--if-exists` | `fail` | `append`, `replace`, or `fail` when a target table already exists. |
|
||||
| `--with-views` | off | Add `cash_events` and `positions_reconstructed` views (requires the `history-deals` dataset). |
|
||||
|
||||
History orders and deals are fetched per symbol and concatenated, so the symbol filter is applied consistently across all datasets. The `cash_events` view is derived from symbol-filtered `history_deals`, so account-level cash events with empty or non-matching symbols may be excluded. The `positions_reconstructed` view excludes positions with no closing deal, uses volume-weighted open/close prices, and reports reversal deals (`DEAL_ENTRY_INOUT`) via `volume_reversal` / `reversal_count`.
|
||||
|
||||
## Global Options
|
||||
|
||||
| Option | Description |
|
||||
|
||||
Reference in New Issue
Block a user