Commit Graph

48 Commits

Author SHA1 Message Date
unknown 17293d2c67 error in mt5 parser - unused module import 2026-06-13 07:45:15 +10:00
unknown f60cc3790c Fix loading of new accounts without trades on live mt5 eas page 2026-06-13 07:42:38 +10:00
unknown 626ff6d91b * Reordered navigator live ea's at the top
* added the live ea's ftp html reports to be available via trade analysis page.
2026-05-23 07:35:33 +10:00
unknown e8ee01304a Updated portfolio buidler and master pages for bulk file selection or select and scan a folder for MT5 html back test reports 2026-05-12 13:51:56 +10:00
unknown 902547d99f chore: untrack machine-specific streamlit config 2026-05-12 07:33:04 +10:00
unknown 62848079b3 fix(uploaders): convert master/builder to single-file slot pattern
Multi-file uploaders trigger Windows browsers' folder-select mode
which bypasses the type filter. Switch to N single-file slots
(matching the working set comparator pattern) so the OS picker
stays in single-file mode where folder selection is impossible.
Default 5 slots, expandable up to 20 via dropdown.
2026-05-06 17:13:02 +10:00
unknown 995205ed16 fix(portfolio-master): visible skip warning + clearer caption
When a user selects a folder in the upload picker (browser behaviour
that bypasses type filter), all files come through and only valid
extensions get processed. Make the skipped count visible and update
the caption to set expectations.
2026-05-06 17:09:27 +10:00
unknown fbc63e8a0b fix(uploaders): restrict file types on portfolio master and builder pages
Mac Safari was offering folder uploads when type=None was set on
file_uploader. Set explicit type=[htm,html,csv] on both
portfolio master and builder pages so the OS picker restricts to
individual files of supported extensions.
2026-05-06 16:27:29 +10:00
unknown 045365838b Fixed bug on portfolio builder equity chart when selecting EA + strategy. 2026-05-03 19:59:22 +10:00
unknown 7cd1ab6a12 Modified EA Comparator to allow for up to 20 set files to be compared 2026-05-03 10:58:39 +10:00
unknown aefcb76261 Modified EA Comparator to allow for up to 20 set files to be compared 2026-05-03 10:54:47 +10:00
unknown 04e4dea081 feat: move refresh controls into account configuration expander
view_live_mt5_eas.py:
- Refresh All button and Auto-refresh input moved inside Account
  Configuration expander to eliminate wasted space on main page
- Updated Xm ago timestamp remains on main page below expander
2026-04-20 14:47:30 +10:00
unknown 92bdf09356 fix: account summary card text colour and size on dark theme
view_live_mt5_eas.py:
- Secondary text colour changed from #555/#666 to #A0A8B8 (silver grey)
  for readability on dark theme
- Font size bumped from 11-12px to 13px across all card labels:
  Updated timestamp, Balance/Current, Recovery, Loss streak,
  Stagnation, Today P&L, prop account progress bar labels
2026-04-20 14:41:44 +10:00
unknown 1cdca6cfcb fix: auto-refresh using st_autorefresh JS timer instead of sleep/rerun
view_live_mt5_eas.py:
- Replace broken sleep+rerun approach with streamlit-autorefresh
  which injects a JS timer — works without user interaction
- Graceful fallback if package not installed, shows install hint
- First page load always pulls fresh data from FTP
- Auto-refresh fires FTP pull when poll interval has elapsed

requirements.txt:
- Added streamlit-autorefresh
2026-04-20 14:12:45 +10:00
unknown a8ff2b729d Fix live mt5 ea's page to fetch ftp data, then update cache before rendering.
Fix Live mt5 ea's autofresh to fetch ftp data, update cache then render.
2026-04-20 14:00:53 +10:00
unknown 934b5fec71 fix: auto-refresh and open positions columns on Live MT5 EAs page
view_live_mt5_eas.py:
- Auto-refresh: added sleep+rerun loop at end of render so page
  keeps itself alive without user interaction
- Initialise ftp_last_auto_refresh to now on first page load so
  timer starts correctly rather than triggering immediately
- Open positions table: updated columns to match MT5 HTML report
  (Time, Position, Symbol, Type, Volume, Price, S/L, T/P,
  Market Price, Swap, Profit, Comment)

mt5_parser.py:
- parse_open_positions: added market_price, swap, profit, comment
  columns to match all 11 columns in MT5 HTML Open Positions table
2026-04-20 13:54:37 +10:00
unknown 6778d1b590 fix: auto-refresh and open positions columns on Live MT5 EAs page
view_live_mt5_eas.py:
- Auto-refresh: added sleep+rerun loop at end of render so page
  keeps itself alive without user interaction
- Initialise ftp_last_auto_refresh to now on first page load so
  timer starts correctly rather than triggering immediately
- Open positions table: updated columns to match MT5 HTML report
  (Time, Position, Symbol, Type, Volume, Price, S/L, T/P,
  Market Price, Swap, Profit, Comment)

mt5_parser.py:
- parse_open_positions: added market_price, swap, profit, comment
  columns to match all 11 columns in MT5 HTML Open Positions table
2026-04-20 13:52:37 +10:00
unknown b6ed4b6c4b fix: auto-refresh and open positions columns on Live MT5 EAs page
view_live_mt5_eas.py:
- Auto-refresh: added sleep+rerun loop at end of render so page
  keeps itself alive without user interaction
- Initialise ftp_last_auto_refresh to now on first page load so
  timer starts correctly rather than triggering immediately
- Open positions table: updated columns to match MT5 HTML report
  (Time, Position, Symbol, Type, Volume, Price, S/L, T/P,
  Market Price, Swap, Profit, Comment)

mt5_parser.py:
- parse_open_positions: added market_price, swap, profit, comment
  columns to match all 11 columns in MT5 HTML Open Positions table
2026-04-20 13:49:58 +10:00
unknown f77f44056c fix: open positions table now matches MT5 HTML report columns
mt5_parser.py:
- parse_open_positions: added market_price, swap, profit, comment
  columns to match all 11 columns in MT5 HTML Open Positions table

view_live_mt5_eas.py:
- Open positions table columns updated to match MT5 report order:
  Account, Time, Position, Symbol, Type, Volume, Price, S/L, T/P,
  Market Price, Swap, Profit, Comment
2026-04-20 13:17:11 +10:00
unknown 239ed043c6 fix: open positions table shows all columns, remove account numbers
view_live_mt5_eas.py:
- Open positions table now shows all columns: Account, Position,
  Symbol, Base, Type, Lots, Open Time, Open Price, SL, TP, Status
- Friendly display names and formatted Open Time (DD.MM.YYYY HH:MM)
- Any extra columns returned by parser appended automatically
- Replaced real account numbers in FTP setup guide CLI examples
  with generic placeholders (123456, 789012)

icmarkets_parser.py:
- Replaced real account numbers in docstring examples with
  generic placeholders (123456789, 987654321)

.gitignore:
- Added *.docx to prevent docs with sensitive info being committed
2026-04-20 13:00:55 +10:00
unknown 59420e50a5 fix: remove real account numbers from codebase
icmarkets_parser.py:
- Replaced real account numbers in docstring examples with
  generic placeholders (123456789, 987654321)

view_live_mt5_eas.py:
- Replaced real account numbers in FTP setup guide CLI
  examples with generic placeholders (123456, 789012)
2026-04-20 10:32:14 +10:00
unknown ba91d8fd13 fix: remove real account numbers from Live MT5 EAs setup guide
view_live_mt5_eas.py:
- Replaced real account numbers in CLI examples with generic
  placeholders (123456, 789012)

.gitignore:
- Added *.docx to prevent handoff/setup docs with sensitive
  information being committed to the repo
2026-04-20 10:30:35 +10:00
unknown fc42eeef02 feat: full FTP setup guide on Live MT5 EAs page when unconfigured
view_live_mt5_eas.py:
- Replaced bare error message with full inline setup guide when
  ftp_config.json is missing
- Part 1: FileZilla Server install, create FTP user, disable TLS,
  passive mode port range
- Part 2: MT5 terminal FTP publisher settings step-by-step table,
  common issues table
- Part 3: CLI verification commands for connection test, save
  credentials, pull and cache accounts
- Part 4: config file reference table and .gitignore entries
- Part 5: troubleshooting table
- Info banner at bottom prompting to refresh once CLI setup is done
- Page loads normally once ftp_config.json exists — no behaviour change
2026-04-20 09:46:55 +10:00
unknown 50d2df9c98 fix: bind to 0.0.0.0 for LAN mode instead of specific IP
view_settings.py:
- _write_server_config now always writes address = 0.0.0.0 for LAN mode
- Specific detected IP is display-only, not written to config.toml
- Fixes WinError 10049 when config.toml is shared across machines
  with different LAN IPs via git pull
2026-04-20 09:11:07 +10:00
unknown f0b30e2097 feat: network settings, update banner and changelog in settings page
view_settings.py:
- Network Access section: LAN vs localhost toggle, port input
- _get_lan_ip: detects first 192.168.x.x address, ignores PPPoE/WAN
- _write_server_config: writes [server] block to config.toml
- _write_config: now preserves existing [server] block when saving theme
- Custom hostname input with step-by-step hosts file instructions
  for Windows (C:\Windows\System32\drivers\etc\hosts) and macOS (/etc/hosts)
- iOS/Android note directing to IP address or static DHCP reservation
- _get_local_sha: reads .git/HEAD directly, no git CLI required
- _get_remote_info: fetches latest commit SHA from GitHub API, 1hr cache
- _get_recent_commits: fetches last 10 commits for changelog, 1hr cache
- check_for_update_banner: call from app.py to show top-of-page info
  banner when local SHA differs from latest GitHub commit
- Version and Changelog section in Settings: local vs remote SHA,
  update warning with git pull instructions, Check again button,
  last 10 commits as collapsible expanders with you-are-here marker
2026-04-20 08:56:37 +10:00
unknown 6f8e01bcf1 feat: Live MT5 EAs page + parser improvements
Live MT5 EAs page (view_ftp_tracker.py):
- FTP-based multi-account tracker replacing view_mt5_tracker.py
- Account configuration with add/remove, FTP folder validation, Demo/Personal/Prop types
- Prop account fields: profit target %, max loss %, daily loss % with EA hard stop banner
- Account summary cards: recovery factor, loss streak, stagnation days, today P&L, prop progress bars
- Calendar view: month/week/year, Mon-Fri only, weekly total column, $ or % toggle
- Open positions table parsed from MT5 HTML Open Positions section
- Symbol correlation heatmap (collapsible)
- Trade analysis section with full stats, equity/drawdown/daily P&L, DOW/hour, monthly table
- Analysis modes: Overall, By Account, By Symbol, By Algo, By Day of Week
- Dynamic combined balance field auto-updates from selected accounts
- Auto-refresh polling with session state timestamp guard
- Report date extracted from MT5 HTML header (Date: field)
- Drawdown $ / % toggle with spline smoothing

mt5_parser.py:
- calc_stats(df, deposit=0.0) — deposit-aware balance drawdown matching MT5 Balance Drawdown Maximal
- max_drawdown_pct and peak_equity added to calc_stats return
- peak_at_dd uses .loc[] fix (IndexError on filtered DataFrames)
- parse_open_positions() — parses Open Positions section from MT5 account HTML
- extract_strategy() — filters sl/tp/so close-reason comments, maps nan to Manual

ftp_sync_cli.py:
- New CLI tool for FTP connection testing and cache population

MT5Tools_FTP_Setup_Guide.docx:
- FileZilla Server setup, MT5 publisher config, CLI verification, troubleshooting
2026-04-20 08:30:46 +10:00
unknown e1b426c9b2 feat: HTML report generation for trade analysis
- Add _generate_html_report() function with self-contained HTML output
- Charts built with direct Plotly.newPlot JSON (no pio.to_html conflicts)
- Equity curve, drawdown, daily P&L, DOW, hour of day charts
- Both view overlays original and edited equity curves
- Stats grid with delta arrows in Both mode
- Monthly performance table with JS $ / % toggle
- Initial balance passed explicitly from session state at download time
- Position summary and trade log as collapsible <details> dropdowns
- Daily P&L x-axis labels hidden for readability
- Download button at top of Overall mode, filename includes view and timestamp
- Move Initial Balance input to filters row so it's set before report renders
- Remove duplicate deposit input from monthly table section
2026-04-18 12:06:01 +10:00
unknown fa5ceca982 feat: trade analysis overhaul — editing, grouping, comparison views
Trade Analysis:
- Add editable raw trade log with # index and Group column
- Group column merges multiple entries into one position on Update
- Reset button clears edited version back to original upload
- View selector: Original / Edited / Both — appears after first Update
- Both mode: stats show with coloured delta arrows vs edited version
- Both mode: equity, drawdown and daily P&L charts overlay both versions
- Both mode: strategy comparison table shows inline delta values
- Position Summary expander above monthly table — all positions listed
  (grouped with entry count, individual as single rows) sorted by open time
- Download exports edited version when available, original otherwise
- Trade # multiselect filter to show specific trades
- All charts and tables (DOW, hour, monthly) respect view selection
- Fix: Group column positioned second after # index
- Fix: DataFrame truth value error in download button
- Fix: Both mode now uses edited df for DOW/hour/monthly charts

mt5_parser:
- Add trading_days and trades_per_day to calc_stats
- Add ISO datetime format fallback for CSV imports
- Fix missing comment column KeyError in _enrich
- Fix NaT in date filter when open_time has null values
2026-04-18 08:54:02 +10:00
unknown 413d9bb3f4 fix: mt5_parser, trade analysis charts and batch backtest
mt5_parser:
- Remove deprecated infer_datetime_format (removed in pandas 2.0)
- Add ISO datetime format fallback for CSV imports (YYYY-MM-DD)
- Fix missing comment column in _enrich causing KeyError on plain CSVs

Trade Analysis:
- Add daily P&L bar chart below equity curve
- Add drawdown panel below daily P&L
- Fix NaT in date filter when open_time has null values

Batch Backtest:
- Fix read_utf16/write_utf16 to detect and preserve original encoding
- Fix _clear_use_default_flags to clear bit 2 on all parameters
- Fix snapshot approach for report finding — before/after file scan
- Fix cleanup to remove all new htm files after each run
- Fix report selection to prefer named file over newest fallback
- Apply all encoding and flag fixes to CLI script mt5_batch_backtest.py
2026-04-17 21:10:16 +10:00
unknown ab9e968def Portfolio builder/master improvements, batch backtest fixes, parser FIFO fix, theme updates 2026-04-17 09:42:08 +10:00
unknown 7e2015cb11 Fixed logic of portfolio builder and parsing of htm reports
- incorrectly managed in/out trades skipping trades when multiple open entries on the same strategy.
- fixed card color backgrounds in light theme
- fixed strategy filters in portfolio builder
2026-04-17 08:40:42 +10:00
unknown 0414fa649a updated batch backtest file name exports and bug in lot size manual allocating a lot size step when parsing to MT5 2026-04-16 15:44:22 +10:00
unknown a4a75eaaeb fix: batch backtest ini generation and report collection
- Replace Inputs= with [TesterInputs] section in ini files so MT5
  cannot override set file parameters with cached/default values
- Fix build_ini to use caller-supplied report_name instead of
  recomputing a generic name, ensuring each run gets a unique ini
- Replace shutil.copy2 + delete loop with shutil.move for htm reports
- Add companion PNG move (name, -holding, -mfemae, -hst) to reports folder
- Fix Streamlit folder warning appearing in wrong conditional branch

Applies to both mt5_batch_backtest.py and view_batch_backtest.py
2026-04-15 21:35:19 +10:00
unknown 2044d181f3 feat: trade analysis monthly table, equity overlays, chart fixes
- Add monthly performance table (year x month grid) with $ / % toggle
  and Initial Balance input alongside the toggle
- Add equity curve overlay toggles: Account total, By Strategy,
  By Symbol, By Day of Week — checkboxes above each chart
- Fix all chart backgrounds to transparent for light/dark compatibility
- Fix chart grid and font colours to work on both themes
- Monthly table and equity overlays added to Overall, By Strategy
  and By Symbol modes
2026-04-15 08:56:08 +10:00
unknown c4e1f4dfed docs: update install guide with new features and git update guide
- Section 4.1: add equity chart overlay toggles description
- Section 4.6: new Settings page docs — Dark/Light/Custom theme,
  colour pickers, text size, inject_theme_css() usage
- Section 4.7: Batch Backtest renumbered
- Section 9: new — Keeping Up to Date (git pull, pip install,
  restart, handling conflicts, checking current version)
2026-04-15 08:27:33 +10:00
unknown a784d26875 feat: trade analysis equity overlays, theme settings, chart fixes
Trade Analysis:
- Add toggleable equity curve overlays: Account total, By Strategy,
  By Symbol, By Day of Week — checkboxes above each chart
- Fix all chart backgrounds to transparent (inherit page theme)
- Fix grid and font colours to work on both dark and light backgrounds

Settings page:
- Full theme control: Dark / Light / Custom radio with Apply button
- Custom mode: colour pickers for accent, page bg, sidebar bg, text,
  font selector and base dark/light toggle
- Preview swatches before applying
- Discard / Reset to Dark button
- Text size: Normal / Large (+2px) / Extra Large (+4px) via CSS injection
- inject_theme_css() helper for app.py to apply font size on all pages
- Theme written to .streamlit/config.toml on apply
2026-04-15 08:23:56 +10:00
unknown 2215a5d727 fix: batch backtest instrument editing and report naming
- Add Manual (edit in table) instrument mode
- Add Lock Table / Unlock buttons — edits persist without rerun interference
- Fix report name to always include symbol and period: {stem}_{symbol}_{period}_{model}
- Fix report names rebuilt from actual Symbol values in locked/edited table
- Add instrument warning above preview table
- Fix run_batch missing report_names parameter (recurring)
- Fix progress bar using stored total count
- Add Cancel button and bb_total / bb_report_dir session state keys
2026-04-14 20:25:58 +10:00
unknown 789cf67bba fix: batch backtest, set comparator and config editor
Batch Backtest:
- Fix run_batch signature missing report_names parameter (recurring)
- Add inline config editor to Active Config expander — edit and save
  all settings directly from the UI without re-running CLI script

Set Comparator:
- Fix UTF-8/UTF-16 encoding detection using BOM check
- Fix parse to skip optimisation metadata lines (ParamName,F etc.)
- Fix export to preserve all original lines including metadata
- Add UBS format detection — warning shown for non-Ultimate Breakout files
- Fix view_set_comparator to unpack 4-value return from parse_set_file
2026-04-14 15:46:27 +10:00
unknown 25b734a559 fix: set comparator, batch backtest and trade analysis bugs
Set Comparator:
- Fix UTF-16/UTF-8 encoding detection for .set files (BOM check)
- Fix parse to skip optimisation metadata lines (ParamName,F etc.)
- Fix export to preserve all original lines including metadata
- Add format detection warning for non-Ultimate Breakout .set files

Batch Backtest:
- Fix run_batch signature missing report_names parameter (caused hang)
- Fix progress bar to use stored total count not live set_files length
- Add Windows-only check at render entry point
- Fix optimisation file exclusion to match anywhere in filename

Trade Analysis:
- Fix IC Markets upload error handling to show actual exception

requirements.txt:
- Add scipy, openpyxl, beautifulsoup4
2026-04-14 13:24:05 +10:00
unknown e3b0d4221d chore: fix gitignore — remove mt5_batch_config.json from tracking 2026-04-14 13:22:27 +10:00
unknown 831c0dcef1 docs: add README and install guide
- README.md: full feature overview, install steps, file structure, format support
- MT5_Tools_Install_Guide.docx: 8-section Word doc covering install, all pages,
  config reference and troubleshooting
2026-04-14 10:59:55 +10:00
unknown d1add02390 docs: add README and install guide
- README.md: full feature overview, install steps, file structure, format support
- MT5_Tools_Install_Guide.docx: 8-section Word doc covering install, all pages,
  config reference and troubleshooting
2026-04-14 10:58:59 +10:00
unknown ec80bde82d Feature:
- added IC markets trade export file to trade analysis with account filter option
2026-04-14 08:39:30 +10:00
unknown 3db0870844 feat: portfolio master algo overhaul + fixes
Search:
- Three modes: Exhaustive, Greedy (incremental), Monte Carlo, combined
- Combination count estimate + runtime warning before run
- Background thread with Cancel button (threading.Event, no session_state spam)

Scoring:
- Composite weighted score (Ret/DD, Stability, Stagnation, Win Rate,
  Growth Quality, Diversity) with normalised sliders
- Score, Stability, Growth Quality, Diversity now whole numbers
- Growth Quality = normalised slope x R2 rewards consistently rising curves
- Diversity bonus based on symbol variety + session overlap

Correlation:
- Average pairwise correlation output metric
- Conditional correlation (drawdown days only) computed and displayed
- Per-result correlation heatmaps in detail expanders

Results table:
- Summary explanation panel above table
- Max Pos Exposure removed (unreliable with current parser output)

Fixes:
- Duplicate plotly_chart element ID fix
- time import missing fix
2026-04-13 17:25:13 +10:00
unknown 06e23e1d7a fix: batch backtest lot size, DD% formula, max position exposure
- mt5_batch_backtest.py / view_batch_backtest.py: fix update_param to
  clear MT5 .set flag bit 2 (use-default) and update default field so
  manual lot size actually takes effect
- view_portfolio_master.py: Max DD% now uses deposit not running peak,
  Max Position Exposure rebuilt as proper timeline sweep
- view_portfolio_builder.py: Max DD% formula aligned to match master
- view_portfolio_master.py / view_portfolio_builder.py: results table
  now shows full stats columns including Stability score
2026-04-13 15:46:52 +10:00
unknown 0d5746c89a Added batch set file back test for strategy tester MT5 2026-04-12 19:19:32 +10:00
unknown 3b1f90ffa8 Initial commit - MT5 Tools dashboard 2026-04-12 18:27:19 +10:00
unknown 6b677b8259 Initial commit - MT5 Tools dashboard 2026-04-12 18:25:46 +10:00