da74c11087
* Add collect-history command for bulk SQLite export Bundles rates, ticks, history-orders, and history-deals for one or more symbols into a single SQLite database. Tick collection uses copy_ticks_range_as_df with a --flags option defaulting to ALL. With --with-views, optional cash_events and positions_reconstructed views are derived from history_deals when the required columns are present. * Extend collect-history with datasets, if-exists, timeframe, view fixes - Fetch history-orders and history-deals per symbol so --symbol applies consistently across all four datasets. - Add repeatable --dataset (rates, ticks, history-orders, history-deals) so ticks are no longer required and any subset can be collected. - Add --if-exists append|replace|fail to control SQLite table conflict behavior instead of hard-coding replace. - Record the requested timeframe in a timeframe column on the rates table so appended runs at different timeframes stay distinguishable. - Fix positions_reconstructed to exclude positions with no closing deals, use volume-weighted open/close prices, and report reversal deals (DEAL_ENTRY_INOUT) via volume_reversal / reversal_count without contributing to weighted prices. - Update tests, README, docs, and skill to match. * Address collect-history review feedback * Stream collect-history writes per symbol * Address PR cleanup for collect-history --------- Co-authored-by: Claude <noreply@anthropic.com>
9.0 KiB
9.0 KiB
name, description
| name | description |
|---|---|
| mt5cli | Use the `mt5cli` CLI to export MetaTrader 5 data (rates, ticks, account, symbols, orders, positions, history) to CSV, JSON, Parquet, or SQLite3. Invoke when the user asks to export, dump, download, or fetch MT5 market data or account data to a file. |
mt5cli
Export MetaTrader 5 data to CSV, JSON, Parquet, or SQLite3 via the mt5cli
command. Output format is auto-detected from the file extension (.csv,
.json, .parquet/.pq, .db/.sqlite/.sqlite3) or overridden with
--format/-f.
Requirements
- Python 3.11+ on Windows with MetaTrader 5 installed (pdmt5 requires the MT5 terminal).
- Install:
pip install -U mt5cli MetaTrader5. - In this repo, run via
uv run mt5cli ...oruv run python -m mt5cli ....
Invocation shape
mt5cli [GLOBAL OPTIONS] -o OUTPUT COMMAND [COMMAND OPTIONS]
Global options MUST precede the subcommand.
Global options (apply to every subcommand)
| Option | Purpose |
|---|---|
-o, --output PATH |
Output file path (required). |
-f, --format FORMAT |
csv, json, parquet, or sqlite3 (auto from extension). |
--table NAME |
Table name for SQLite3 output (default: data). |
--login INT |
MT5 trading account login. |
--password TEXT |
MT5 trading account password. |
--server TEXT |
MT5 trading server name. |
--path TEXT |
Path to MetaTrader 5 terminal EXE. |
--timeout INT |
Connection timeout in milliseconds. |
--log-level LEVEL |
DEBUG, INFO, WARNING (default), ERROR. |
Parameter value formats
- Datetimes (
--date-from,--date-to): ISO 8601 (2024-01-01or2024-01-01T12:00:00+00:00). Naive values are treated as UTC. - Timeframe (
--timeframe):M1,M2,M3,M4,M5,M6,M10,M12,M15,M20,M30,H1,H2,H3,H4,H6,H8,H12,D1,W1,MN1, or the raw integer. - Tick flags (
--flags):ALL,INFO,TRADE, or the raw integer.
Commands
| Command | Required options | Optional options |
|---|---|---|
rates-from |
--symbol, --timeframe, --date-from, --count |
— |
rates-from-pos |
--symbol, --timeframe, --start-pos, --count |
— |
rates-range |
--symbol, --timeframe, --date-from, --date-to |
— |
ticks-from |
--symbol, --date-from, --count, --flags |
— |
ticks-range |
--symbol, --date-from, --date-to, --flags |
— |
account-info |
— | — |
terminal-info |
— | — |
symbols |
— | --group (e.g., *USD*) |
symbol-info |
--symbol |
— |
orders |
— | --symbol, --group, --ticket |
positions |
— | --symbol, --group, --ticket |
history-orders |
— | --date-from, --date-to, --group, --symbol, --ticket, --position |
history-deals |
— | --date-from, --date-to, --group, --symbol, --ticket, --position |
collect-history |
--symbol (repeatable), --date-from, --date-to |
--dataset (repeatable; rates/ticks/history-orders/history-deals; default all), --timeframe (M1; recorded on rates), --flags (ALL), --if-exists (append/replace/fail; default fail), --with-views (SQLite3 output only) |
Examples
# Account snapshot as CSV.
mt5cli -o account.csv account-info
# EURUSD M1 bars (1000 rows) from a start date to Parquet.
mt5cli -o rates.parquet rates-from \
--symbol EURUSD --timeframe M1 --date-from 2024-01-01 --count 1000
# EURUSD tick stream for a date range to JSON.
mt5cli -o ticks.json ticks-range \
--symbol EURUSD --date-from 2024-01-01 --date-to 2024-01-02 --flags ALL
# USD symbols into a named table in SQLite3.
mt5cli -o data.db --table symbols symbols --group "*USD*"
# Historical deals filtered by symbol (using an already-logged-in MT5 terminal).
mt5cli -o deals.csv history-deals --symbol EURUSD --date-from 2024-01-01
# Bundle selected historical datasets into one SQLite db, appending to any
# existing tables, plus cash_events and positions_reconstructed views.
mt5cli -o history.db collect-history \
--symbol EURUSD --symbol GBPUSD \
--date-from 2024-01-01 --date-to 2024-02-01 \
--dataset rates --dataset history-deals \
--timeframe M1 --flags ALL --if-exists append --with-views
Guidelines
- Pick the output extension to avoid passing
--format. - Use
--tableonly with SQLite3 outputs; it is otherwise ignored. --countis required forrates-from,rates-from-pos, andticks-from. Preferrates-range/ticks-rangewhen a fixed window is known.- Credentials (
--login,--password,--server) are optional when the local MT5 terminal is already logged in. - Avoid passing
--passwordon the command line in shared or logged environments — it is visible inps, shell history, and CI logs. Prefer logging in through the MT5 terminal first, then omit credentials here. - Reach for
--log-level DEBUGwhen a command fails silently — MT5 connection errors surface there. - If the user asks to run from source in this repo, prefix with
uv run(e.g.,uv run mt5cli -o out.csv account-info).