gavindiaz/mt5cli
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add collect-history command for bulk data collection (#14)

Browse Source 
* Add collect-history command for bulk SQLite export

Bundles rates, ticks, history-orders, and history-deals for one or more
symbols into a single SQLite database. Tick collection uses
copy_ticks_range_as_df with a --flags option defaulting to ALL. With
--with-views, optional cash_events and positions_reconstructed views are
derived from history_deals when the required columns are present.

* Extend collect-history with datasets, if-exists, timeframe, view fixes

- Fetch history-orders and history-deals per symbol so --symbol applies
  consistently across all four datasets.
- Add repeatable --dataset (rates, ticks, history-orders, history-deals)
  so ticks are no longer required and any subset can be collected.
- Add --if-exists append|replace|fail to control SQLite table conflict
  behavior instead of hard-coding replace.
- Record the requested timeframe in a timeframe column on the rates
  table so appended runs at different timeframes stay distinguishable.
- Fix positions_reconstructed to exclude positions with no closing
  deals, use volume-weighted open/close prices, and report reversal
  deals (DEAL_ENTRY_INOUT) via volume_reversal / reversal_count without
  contributing to weighted prices.
- Update tests, README, docs, and skill to match.

* Address collect-history review feedback

* Stream collect-history writes per symbol

* Address PR cleanup for collect-history

---------

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Daichi Narushima
2026-05-29 01:22:59 +09:00
committed by GitHub
co-authored by Claude
parent c45efb953c
commit da74c11087
7 changed files with 1253 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files
skills/mt5cli/SKILL.md
+24 -15
View File
@@ -50,21 +50,22 @@ Global options MUST precede the subcommand.
## Commands
| Command | Required options | Optional options |
| ---------------- | ----------------------------------------------------- | --------------------------------------------------------------------------- |
| `rates-from` | `--symbol`, `--timeframe`, `--date-from`, `--count` | — |
| `rates-from-pos` | `--symbol`, `--timeframe`, `--start-pos`, `--count` | — |
| `rates-range` | `--symbol`, `--timeframe`, `--date-from`, `--date-to` | — |
| `ticks-from` | `--symbol`, `--date-from`, `--count`, `--flags` | — |
| `ticks-range` | `--symbol`, `--date-from`, `--date-to`, `--flags` | — |
| `account-info` | — | — |
| `terminal-info` | — | — |
| `symbols` | — | `--group` (e.g., `*USD*`) |
| `symbol-info` | `--symbol` | — |
| `orders` | — | `--symbol`, `--group`, `--ticket` |
| `positions` | — | `--symbol`, `--group`, `--ticket` |
| `history-orders` | — | `--date-from`, `--date-to`, `--group`, `--symbol`, `--ticket`, `--position` |
| `history-deals` | — | `--date-from`, `--date-to`, `--group`, `--symbol`, `--ticket`, `--position` |
| Command | Required options | Optional options |
| ----------------- | ----------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `rates-from` | `--symbol`, `--timeframe`, `--date-from`, `--count` | — |
| `rates-from-pos` | `--symbol`, `--timeframe`, `--start-pos`, `--count` | — |
| `rates-range` | `--symbol`, `--timeframe`, `--date-from`, `--date-to` | — |
| `ticks-from` | `--symbol`, `--date-from`, `--count`, `--flags` | — |
| `ticks-range` | `--symbol`, `--date-from`, `--date-to`, `--flags` | — |
| `account-info` | — | — |
| `terminal-info` | — | — |
| `symbols` | — | `--group` (e.g., `*USD*`) |
| `symbol-info` | `--symbol` | — |
| `orders` | — | `--symbol`, `--group`, `--ticket` |
| `positions` | — | `--symbol`, `--group`, `--ticket` |
| `history-orders` | — | `--date-from`, `--date-to`, `--group`, `--symbol`, `--ticket`, `--position` |
| `history-deals` | — | `--date-from`, `--date-to`, `--group`, `--symbol`, `--ticket`, `--position` |
| `collect-history` | `--symbol` (repeatable), `--date-from`, `--date-to` | `--dataset` (repeatable; rates/ticks/history-orders/history-deals; default all), `--timeframe` (M1; recorded on rates), `--flags` (ALL), `--if-exists` (append/replace/fail; default fail), `--with-views` (SQLite3 output only) |
## Examples
@@ -85,6 +86,14 @@ mt5cli -o data.db --table symbols symbols --group "*USD*"
# Historical deals filtered by symbol (using an already-logged-in MT5 terminal).
mt5cli -o deals.csv history-deals --symbol EURUSD --date-from 2024-01-01
# Bundle selected historical datasets into one SQLite db, appending to any
# existing tables, plus cash_events and positions_reconstructed views.
mt5cli -o history.db collect-history \
--symbol EURUSD --symbol GBPUSD \
--date-from 2024-01-01 --date-to 2024-02-01 \
--dataset rates --dataset history-deals \
--timeframe M1 --flags ALL --if-exists append --with-views
```
## Guidelines