Add methodology documentation for EA research

This document outlines the methodology for research, implementation, backtesting, and validation of trading strategies in the xauusd-mt5-ea-research repository. It includes a detailed process flow, repository structure, and guidelines for maintaining research integrity.

docs/EA-055_Double_Top_Bottom_Break/methodology.md

@@ -0,0 +1,751 @@
+# XAUUSD MT5 EA Research — Methodology
+
+## Purpose
+
+This document defines the standard research, implementation, backtesting and validation methodology used throughout the `xauusd-mt5-ea-research` repository.
+
+The objective is to convert trading ideas into explicit, reproducible MQL5 Expert Advisors and evaluate them using recorded empirical evidence.
+
+The repository is designed to preserve both successful and failed experiments.
+
+A strategy is not considered valid simply because it has been implemented, compiled successfully or produced a profitable individual backtest.
+
+---
+
+# 1. Repository Research Flow
+
+Every EA follows the same basic process:
+
+    Trading Idea
+        ↓
+    Define Hypothesis
+        ↓
+    Convert Hypothesis into Explicit Rules
+        ↓
+    Implement in MQL5
+        ↓
+    Compile and Verify
+        ↓
+    Run Baseline Backtest
+        ↓
+    Preserve Raw Evidence
+        ↓
+    Analyze Results
+        ↓
+    PASS / FAIL Baseline
+        ↓
+    Controlled Modification
+        ↓
+    Retest
+        ↓
+    Robustness Validation
+        ↓
+    Forward Test
+        ↓
+    Production Review
+
+The process is intentionally sequential.
+
+Later validation stages should not be treated as completed when earlier stages have not produced sufficient evidence.
+
+---
+
+# 2. Repository Structure
+
+The repository separates strategy implementation, empirical evidence and research documentation.
+
+    xauusd-mt5-ea-research/
+    │
+    ├── EAs/
+    │   └── EA-<>/
+    │       ├── EA-<>.mq5
+    │       └── README.md
+    │
+    ├── Backtest/
+    │   └── EA-<>/
+    │       ├── README.md
+    │       ├── Strategy Tester Report
+    │       └── Backtest charts
+    │
+    ├── Research/
+    │   └── README.md
+    │
+    ├── docs/
+    │   └── methodology.md
+    │
+    └── GitHub_Profile/
+        └── README.md
+
+Each layer has a different responsibility.
+
+## EAs
+
+Contains the actual MQL5 implementation.
+
+The EA source code is the authoritative record of what trading rules were executed.
+
+Each EA directory should contain:
+
+- MQL5 source code.
+- EA-specific README.
+- Strategy description.
+- Input parameters.
+- Entry logic.
+- Exit logic.
+- Position-management logic.
+
+---
+
+## Backtest
+
+Contains empirical evidence generated by MetaTrader 5 Strategy Tester.
+
+Each EA should have its own backtest directory.
+
+Raw Strategy Tester artifacts should be preserved whenever possible.
+
+Examples include:
+
+- HTML Strategy Tester report.
+- Balance/equity charts.
+- Entry distribution charts.
+- MFE/MAE charts.
+- Holding-time charts.
+- Parameter configuration.
+- README summarizing the test.
+
+Failed tests should not be deleted simply because the result is unfavorable.
+
+---
+
+## Research
+
+Contains the research reasoning behind the strategies.
+
+Research documentation should explain:
+
+- Trading hypothesis.
+- Pattern or market behavior being investigated.
+- Translation from discretionary concept to measurable rules.
+- Research questions.
+- Experimental findings.
+- Current research status.
+
+Research documentation should distinguish between:
+
+- Hypothesis.
+- Implementation.
+- Observation.
+- Empirical result.
+- Interpretation.
+
+A hypothesis must not be presented as an established trading edge without supporting evidence.
+
+---
+
+## docs
+
+Contains repository-wide documentation.
+
+`methodology.md` defines the common research and validation procedure used across all EAs.
+
+EA-specific results should remain inside the corresponding EA and Backtest directories rather than being duplicated here.
+
+---
+
+# 3. Strategy Identification
+
+Every strategy receives a unique EA identifier.
+
+Format:
+
+    EA-XXX_Strategy_Name
+
+Example:
+
+    EA-055_Double_Top_Bottom_Break
+
+The identifier should remain stable throughout the research lifecycle.
+
+Material changes to the trading hypothesis may require a separate EA identifier rather than silently replacing the original experiment.
+
+---
+
+# 4. Trading Hypothesis
+
+Every EA begins with a testable trading hypothesis.
+
+A useful hypothesis should identify:
+
+- Market behavior being investigated.
+- Conditions required for entry.
+- Expected direction after the signal.
+- Conditions that invalidate the setup.
+- Exit mechanism.
+- Risk-management assumptions.
+
+The hypothesis should be specific enough to translate into deterministic program logic.
+
+Statements such as:
+
+    "Buy when the market looks bullish."
+
+are not sufficiently objective.
+
+A rule such as:
+
+    "BUY when a completed candle closes above a defined breakout level."
+
+can be implemented and tested consistently.
+
+---
+
+# 5. Rule Formalization
+
+Before interpreting backtest performance, discretionary concepts should be converted into explicit rules.
+
+Examples include:
+
+- Swing High / Swing Low definition.
+- Trend definition.
+- Pattern width.
+- Pattern tolerance.
+- Breakout threshold.
+- Candle confirmation.
+- Spread limits.
+- Stop Loss.
+- Take Profit.
+- Break Even.
+- Trailing Stop.
+- Position limits.
+
+The objective is reproducibility.
+
+Given the same:
+
+- EA version.
+- Inputs.
+- Symbol.
+- Timeframe.
+- Historical data.
+- Broker specifications.
+- Testing environment.
+
+the strategy should produce reproducible behavior.
+
+---
+
+# 6. MQL5 Implementation
+
+Strategies are implemented as MetaTrader 5 Expert Advisors using MQL5.
+
+Each EA source file should contain sufficient information to identify:
+
+- EA name.
+- Version.
+- Input parameters.
+- Signal-generation logic.
+- Order-execution logic.
+- Position-management logic.
+
+Compilation success is required before backtesting.
+
+A successful compilation confirms only that the program can be built.
+
+It does not confirm that:
+
+- The strategy logic is correct.
+- Orders behave as intended.
+- The strategy is profitable.
+- Risk is acceptable.
+
+---
+
+# 7. Baseline Configuration
+
+The first valid implementation should be tested using a clearly documented baseline configuration.
+
+The baseline is not necessarily an optimized configuration.
+
+Its purpose is to establish a reference result.
+
+The baseline should record at minimum:
+
+- EA name/version.
+- Symbol.
+- Timeframe.
+- Test period.
+- Initial deposit.
+- Leverage.
+- Lot model.
+- Input parameters.
+- Historical-data quality.
+- Relevant broker/symbol information available in the report.
+
+Future experiments should be compared against a preserved baseline.
+
+---
+
+# 8. Backtesting
+
+MetaTrader 5 Strategy Tester is used to evaluate implemented EAs.
+
+Where available, high-quality historical tick data should be used.
+
+The test report should preserve enough information for another researcher to understand the exact conditions under which the result was generated.
+
+Important metrics include:
+
+- Total Net Profit.
+- Gross Profit.
+- Gross Loss.
+- Profit Factor.
+- Expected Payoff.
+- Maximum Drawdown.
+- Relative Drawdown.
+- Recovery Factor.
+- Sharpe Ratio.
+- Total Trades.
+- Win Rate.
+- Average Winning Trade.
+- Average Losing Trade.
+- Maximum Consecutive Wins.
+- Maximum Consecutive Losses.
+- Position Holding Time.
+
+No single metric should be used alone to determine strategy quality.
+
+---
+
+# 9. Evidence Preservation
+
+Raw evidence is part of the research record.
+
+The original MT5 report should be preserved rather than replacing it with manually transcribed statistics.
+
+A typical backtest directory may contain:
+
+    Backtest/
+    └── EA-XXX_Strategy_Name/
+        ├── README.md
+        ├── ReportTester-XXXXXXXX.html
+        ├── ReportTester-XXXXXXXX.png
+        ├── ReportTester-XXXXXXXX-hst.png
+        ├── ReportTester-XXXXXXXX-mfemae.png
+        └── ReportTester-XXXXXXXX-holding.png
+
+The README summarizes the experiment.
+
+The original Strategy Tester artifacts remain the primary evidence.
+
+---
+
+# 10. Baseline PASS / FAIL
+
+Every completed baseline test receives an explicit status.
+
+Possible states:
+
+    PASS
+    FAIL
+
+A FAIL result is a valid research result.
+
+It means that the tested implementation/configuration did not satisfy the current acceptance requirements.
+
+It should not be hidden or deleted.
+
+A PASS result means only that the experiment passed the defined stage criteria.
+
+It does not automatically mean that the EA is ready for live trading.
+
+---
+
+# 11. Minimum Baseline Evaluation
+
+At minimum, the following should be reviewed:
+
+## Profitability
+
+Check:
+
+- Net Profit.
+- Profit Factor.
+- Expected Payoff.
+- Average win/loss relationship.
+
+A strategy with negative expectancy should not pass profitability evaluation.
+
+---
+
+## Risk
+
+Check:
+
+- Balance Drawdown.
+- Equity Drawdown.
+- Consecutive losses.
+- Largest loss.
+- Recovery Factor.
+
+A profitable result with unacceptable drawdown should not automatically be considered successful.
+
+---
+
+## Trade Sample
+
+Check the number of trades generated.
+
+A very small trade sample may not provide sufficient evidence to draw meaningful conclusions.
+
+Trade count should always be interpreted together with:
+
+- Test duration.
+- Timeframe.
+- Strategy frequency.
+- Market conditions.
+
+---
+
+## Equity / Balance Behavior
+
+The shape of the balance/equity curve should be inspected.
+
+Important characteristics include:
+
+- Persistent upward or downward trend.
+- Long stagnation periods.
+- Sudden dependence on a small number of trades.
+- Large drawdown clusters.
+- Unstable performance.
+
+---
+
+# 12. Failed Experiments
+
+Failed experiments must be preserved.
+
+Example:
+
+    EA-055
+    Baseline
+    Net Profit: negative
+    Profit Factor: < 1
+    Drawdown: high
+    Result: FAIL
+
+This information is useful because it establishes what has already been tested.
+
+Removing failed results creates survivorship bias and makes future research harder to audit.
+
+---
+
+# 13. Controlled Modification
+
+After a baseline FAIL, changes should be introduced in a controlled manner.
+
+Examples:
+
+- Pattern detection.
+- Trend filter.
+- Volatility filter.
+- Session filter.
+- Breakout confirmation.
+- Stop Loss.
+- Take Profit.
+- Break Even.
+- Trailing Stop.
+- Spread filter.
+
+Whenever practical, avoid changing many unrelated components simultaneously.
+
+If multiple components are changed together and performance improves, it becomes difficult to determine which change produced the difference.
+
+---
+
+# 14. Optimization
+
+Optimization may be used after a valid baseline exists.
+
+Optimization is treated as a research tool, not proof of robustness.
+
+A configuration should not be considered validated simply because an optimizer finds profitable parameters on historical data.
+
+Optimization introduces overfitting risk.
+
+Therefore optimized results should be tested on data not used to select the parameters.
+
+---
+
+# 15. In-Sample and Out-of-Sample Testing
+
+When optimization begins, historical data should be separated where practical into:
+
+    In-Sample
+        ↓
+    Parameter Development / Optimization
+        ↓
+    Out-of-Sample
+        ↓
+    Independent Validation
+
+The In-Sample period may be used to develop or select parameters.
+
+The Out-of-Sample period should not be used to select those parameters.
+
+If performance disappears outside the development period, the strategy may be overfitted.
+
+---
+
+# 16. Robustness Testing
+
+A strategy that passes the baseline or optimization stage should undergo additional robustness testing before production consideration.
+
+Possible robustness checks include:
+
+- Different historical periods.
+- Different market regimes.
+- Different spreads.
+- Different execution assumptions.
+- Parameter sensitivity.
+- Nearby parameter values.
+- Alternative XAUUSD broker symbols.
+- Different but relevant timeframes.
+
+The purpose is not to find another perfect result.
+
+The purpose is to determine whether the strategy depends excessively on one exact historical configuration.
+
+---
+
+# 17. Parameter Sensitivity
+
+A potentially robust strategy should not normally depend on one extremely precise parameter combination.
+
+For example, if:
+
+    Parameter = 100 → highly profitable
+    Parameter = 99  → strongly losing
+    Parameter = 101 → strongly losing
+
+the result requires further investigation.
+
+Nearby parameter combinations should be examined when optimization is performed.
+
+A stable performance region is generally more informative than a single isolated optimum.
+
+---
+
+# 18. Transaction Costs
+
+XAUUSD strategies, particularly on low timeframes, can be sensitive to execution costs.
+
+Backtest interpretation should consider:
+
+- Spread.
+- Commission.
+- Slippage.
+- Broker execution.
+- Symbol contract specification.
+
+Strategies producing many short-duration trades require particular attention to transaction costs.
+
+A theoretical price-pattern edge may disappear when realistic execution costs are included.
+
+---
+
+# 19. Low-Timeframe Testing
+
+M1 and other low-timeframe strategies may generate large trade samples but are also exposed to:
+
+- Market noise.
+- Spread sensitivity.
+- Execution sensitivity.
+- Short holding periods.
+- False breakouts.
+
+A high number of trades does not automatically indicate a reliable strategy.
+
+Low-timeframe results should therefore receive additional execution and robustness testing.
+
+---
+
+# 20. Forward Testing
+
+Historical backtesting is not the final validation stage.
+
+Strategies that survive robustness testing should proceed to forward testing.
+
+Typical progression:
+
+    Backtest
+        ↓
+    Robustness Test
+        ↓
+    Demo / Forward Test
+        ↓
+    Small Controlled Live Test
+        ↓
+    Production Review
+
+Forward testing should compare actual execution behavior against expectations derived from historical testing.
+
+---
+
+# 21. Production Readiness
+
+An EA should not be classified as production-ready solely because one backtest is profitable.
+
+Production consideration requires evidence from multiple stages.
+
+A typical status progression is:
+
+    RESEARCH
+        ↓
+    IMPLEMENTED
+        ↓
+    BASELINE TESTED
+        ↓
+    BASELINE PASS
+        ↓
+    ROBUSTNESS TESTED
+        ↓
+    FORWARD TESTED
+        ↓
+    PRODUCTION REVIEW
+        ↓
+    APPROVED / REJECTED
+
+Human review remains required before production approval.
+
+---
+
+# 22. Research Integrity
+
+The repository follows several basic research-integrity rules.
+
+## Preserve Failures
+
+Do not delete unfavorable tests simply because later configurations perform better.
+
+## Preserve Raw Evidence
+
+Keep original Strategy Tester reports whenever possible.
+
+## Separate Facts from Hypotheses
+
+A strategy idea is not evidence.
+
+## Avoid Cherry-Picking
+
+Do not select only favorable historical periods without documenting the selection.
+
+## Avoid Silent Parameter Changes
+
+Parameter changes between experiments must be recorded.
+
+## Avoid Overfitting Claims
+
+Optimized historical performance should not be described as validated performance without independent testing.
+
+## Do Not Rewrite History
+
+Earlier results should remain available even after the EA evolves.
+
+---
+
+# 23. Documentation Standard
+
+Each EA should eventually provide enough information to answer four questions:
+
+### 1. Why was this strategy tested?
+
+Documented in:
+
+    Research/
+
+### 2. What exactly was implemented?
+
+Documented in:
+
+    EAs/
+
+### 3. What happened when it was tested?
+
+Documented in:
+
+    Backtest/
+
+### 4. What is the current research status?
+
+Recorded through the corresponding README files and preserved evidence.
+
+---
+
+# 24. Example — EA-055
+
+The first strategy currently documented using this methodology is:
+
+    EA-055_Double_Top_Bottom_Break
+
+Research hypothesis:
+
+    Double Top / Double Bottom
+        ↓
+    Neckline Breakout
+        ↓
+    Directional Entry
+
+Implementation:
+
+    MetaTrader 5
+    MQL5
+
+Recorded baseline:
+
+    Symbol: XAUUSD.PRO
+    Timeframe: M1
+    Period: 2026-01-02 → 2026-04-01
+    Initial Deposit: $1,000
+    Fixed Lot: 0.01
+    History Quality: 100% real ticks
+
+Baseline result:
+
+    Net Profit: -$489.98
+    Profit Factor: 0.86
+    Win Rate: 47.74%
+    Maximum Equity Drawdown: 52.16%
+    Total Trades: 2,719
+
+Status:
+
+    BASELINE FAIL
+
+EA-055 therefore remains a research strategy.
+
+The failed result is preserved as the reference point for future controlled experiments.
+
+---
+
+# 25. Core Principle
+
+The repository follows one central principle:
+
+**Code is implementation. Backtest is evidence. Research is interpretation. None of them alone proves a trading edge.**
+
+The objective is not to produce the largest number of profitable-looking backtests.
+
+The objective is to build a transparent and reproducible research record showing:
+
+    What was hypothesized
+        ↓
+    What was implemented
+        ↓
+    What was tested
+        ↓
+    What actually happened
+        ↓
+    What changed next
+        ↓
+    Whether the result survived further validation