MT5-EA-Sniper-Strategy/docs/STANDARDIZED_TEST_DATASETS.md

📊 Standardized Test Datasets for Consistent Backtesting

🎯 Objective

Establish standardized test datasets and scenarios to ensure consistent, repeatable backtesting results across all development iterations and feature implementations.

📋 Test Dataset Categories

Primary Test Datasets

Dataset 1: Baseline Performance (3 Months)

Name: BASELINE_3M_EURUSD
Symbol: EURUSD
Period: 2024-07-01 to 2024-09-25
Timeframe: M1
Characteristics: Mixed market conditions, normal volatility
Purpose: Primary regression testing and performance benchmarking
Expected Trades: 80-120
Expected Win Rate: 50-65%

Dataset 2: High Volatility Period (1 Month)

Name: HIGH_VOL_1M_EURUSD
Symbol: EURUSD
Period: 2024-08-01 to 2024-08-31 (NFP releases, ECB meetings)
Timeframe: M1
Characteristics: High volatility, frequent news events
Purpose: Stress testing and volatility handling validation
Expected Trades: 40-60
Expected Drawdown: Higher than baseline

Dataset 3: Low Volatility Period (1 Month)

Name: LOW_VOL_1M_EURUSD
Symbol: EURUSD
Period: 2024-07-15 to 2024-08-15 (Summer trading period)
Timeframe: M1
Characteristics: Low volatility, ranging markets
Purpose: Pattern detection accuracy in quiet markets
Expected Trades: 20-40
Expected Win Rate: Higher precision expected

Dataset 4: Trending Market (2 Weeks)

Name: TREND_2W_EURUSD
Symbol: EURUSD
Period: 2024-09-01 to 2024-09-15 (Strong directional move)
Timeframe: M1
Characteristics: Clear trending conditions
Purpose: Trend-following strategy validation
Expected Trades: 25-40
Expected R:R: Higher than baseline

Multi-Symbol Test Datasets

Dataset 5: Major Pairs Comparison (1 Month)

Symbols: EURUSD, GBPUSD, USDJPY, USDCHF
Period: 2024-08-01 to 2024-08-31
Purpose: Cross-pair performance validation
Expected Behavior: Consistent pattern detection across pairs

Dataset 6: Commodity Currency Test (1 Month)

Symbols: AUDUSD, NZDUSD, USDCAD
Period: 2024-08-01 to 2024-08-31
Purpose: Commodity-correlated pair testing
Expected Behavior: Different volatility patterns

Dataset 7: Gold Testing (2 Weeks)

Symbol: XAUUSD
Period: 2024-09-01 to 2024-09-15
Purpose: High-value, high-volatility instrument testing
Expected Behavior: Larger pip movements, different risk calculations

🔧 Test Dataset Configuration Scripts

Dataset Configuration Generator

File: generate_test_configs.ps1

# Generate standardized test configurations
$TestDatasets = @(
    @{
        Name = "BASELINE_3M_EURUSD"
        Symbol = "EURUSD"
        StartDate = "2024-07-01"
        EndDate = "2024-09-25"
        Description = "Primary baseline performance dataset"
        ExpectedTrades = "80-120"
        ExpectedWinRate = "50-65%"
        Purpose = "Regression testing and benchmarking"
    },
    @{
        Name = "HIGH_VOL_1M_EURUSD"
        Symbol = "EURUSD"
        StartDate = "2024-08-01"
        EndDate = "2024-08-31"
        Description = "High volatility stress testing"
        ExpectedTrades = "40-60"
        ExpectedWinRate = "45-60%"
        Purpose = "Volatility handling validation"
    },
    @{
        Name = "LOW_VOL_1M_EURUSD"
        Symbol = "EURUSD"
        StartDate = "2024-07-15"
        EndDate = "2024-08-15"
        Description = "Low volatility precision testing"
        ExpectedTrades = "20-40"
        ExpectedWinRate = "55-70%"
        Purpose = "Pattern detection accuracy in quiet markets"
    },
    @{
        Name = "TREND_2W_EURUSD"
        Symbol = "EURUSD"
        StartDate = "2024-09-01"
        EndDate = "2024-09-15"
        Description = "Trending market validation"
        ExpectedTrades = "25-40"
        ExpectedWinRate = "50-65%"
        Purpose = "Trend-following strategy validation"
    }
)

# Generate configuration files
foreach ($Dataset in $TestDatasets) {
    $ConfigContent = @"
# Test Dataset Configuration: $($Dataset.Name)

## Dataset Information
- **Name**: $($Dataset.Name)
- **Symbol**: $($Dataset.Symbol)
- **Period**: $($Dataset.StartDate) to $($Dataset.EndDate)
- **Description**: $($Dataset.Description)
- **Purpose**: $($Dataset.Purpose)

## Expected Results
- **Expected Trades**: $($Dataset.ExpectedTrades)
- **Expected Win Rate**: $($Dataset.ExpectedWinRate)

## Strategy Tester Configuration

Expert Advisor: SniperEA Symbol: $($Dataset.Symbol) Period: M1 Date Range: $($Dataset.StartDate) to $($Dataset.EndDate) Model: Every tick (based on all available least timeframes) Optimization: Disabled Deposit: 10000 USD Leverage: 1:100

## EA Parameters (Standard Configuration)

RiskPercent = 1.0 MinRR = 2.0 MaxPositionsPerSymbol = 3 MaxTotalPositions = 10 OBStrengthFilter = 1.0 MinFVGSize = 5.0 BOSConfirmationBars = 3 SweepDistanceThreshold = 10.0 UseTimeFilter = true SessionTimeStart = "08:00" SessionTimeEnd = "21:00" EnableDebugMode = true EnableDetailedLogging = true

## Test Execution Steps
1. Open Strategy Tester (Ctrl+R)
2. Apply configuration above
3. Click Start
4. Record results in test_results_$($Dataset.Name.ToLower()).json
5. Compare with expected results
6. Document any deviations

## Success Criteria
- Compilation successful
- Test completes without errors
- Results within expected ranges
- Pattern detection functioning
- Risk management compliance (exactly 1% per trade)
"@

    $ConfigFile = "test_config_$($Dataset.Name.ToLower()).md"
    $ConfigContent | Out-File $ConfigFile -Encoding UTF8
    Write-Host "Generated: $ConfigFile" -ForegroundColor Green
}

Write-Host "`nAll test dataset configurations generated!" -ForegroundColor Green

Test Results Template Generator

File: generate_results_templates.ps1

# Generate standardized results templates
$TestDatasets = @("BASELINE_3M_EURUSD", "HIGH_VOL_1M_EURUSD", "LOW_VOL_1M_EURUSD", "TREND_2W_EURUSD")

foreach ($Dataset in $TestDatasets) {
    $TemplateContent = @"
{
  "dataset_name": "$Dataset",
  "test_date": "",
  "ea_version": "",
  "test_duration_minutes": 0,
  "compilation_status": "SUCCESS/FAILED",
  "test_completion_status": "COMPLETED/FAILED",
  
  "performance_metrics": {
    "total_net_profit": 0.0,
    "gross_profit": 0.0,
    "gross_loss": 0.0,
    "profit_factor": 0.0,
    "expected_payoff": 0.0,
    "absolute_drawdown": 0.0,
    "maximal_drawdown": 0.0,
    "relative_drawdown_percent": 0.0,
    "total_trades": 0,
    "short_positions_won": 0,
    "long_positions_won": 0,
    "profit_trades_percent": 0.0,
    "loss_trades_percent": 0.0,
    "largest_profit_trade": 0.0,
    "largest_loss_trade": 0.0,
    "average_profit_trade": 0.0,
    "average_loss_trade": 0.0,
    "maximum_consecutive_wins": 0,
    "maximum_consecutive_losses": 0,
    "maximal_consecutive_profit": 0.0,
    "maximal_consecutive_loss": 0.0,
    "average_consecutive_wins": 0,
    "average_consecutive_losses": 0
  },
  
  "risk_management_validation": {
    "risk_per_trade_compliance": true,
    "position_limits_respected": true,
    "stop_loss_placement_accurate": true,
    "take_profit_calculation_correct": true,
    "lot_size_calculation_accurate": true
  },
  
  "pattern_detection_stats": {
    "order_blocks_detected": 0,
    "fair_value_gaps_detected": 0,
    "bos_events_detected": 0,
    "liquidity_sweeps_detected": 0,
    "valid_setups_identified": 0,
    "setup_conversion_rate_percent": 0.0
  },
  
  "session_analysis": {
    "london_session_trades": 0,
    "new_york_session_trades": 0,
    "asian_session_trades": 0,
    "off_hours_trades": 0
  },
  
  "comparison_with_expected": {
    "trades_within_expected_range": true,
    "win_rate_within_expected_range": true,
    "performance_meets_expectations": true,
    "deviations_noted": []
  },
  
  "issues_identified": [],
  "recommendations": [],
  "overall_assessment": "PASS/CONDITIONAL/FAIL",
  "notes": ""
}
"@

    $TemplateFile = "test_results_template_$($Dataset.ToLower()).json"
    $TemplateContent | Out-File $TemplateFile -Encoding UTF8
    Write-Host "Generated: $TemplateFile" -ForegroundColor Green
}

Write-Host "`nAll results templates generated!" -ForegroundColor Green

📊 Test Execution Automation

Batch Test Runner

File: run_standardized_tests.ps1

param(
    [string[]]$Datasets = @("BASELINE_3M_EURUSD", "HIGH_VOL_1M_EURUSD", "LOW_VOL_1M_EURUSD"),
    [switch]$Interactive = $true
)

Write-Host "=== Standardized Test Suite Execution ===" -ForegroundColor Green

foreach ($Dataset in $Datasets) {
    Write-Host "`n--- Testing Dataset: $Dataset ---" -ForegroundColor Yellow
    
    # Load test configuration
    $ConfigFile = "test_config_$($Dataset.ToLower()).md"
    if (-not (Test-Path $ConfigFile)) {
        Write-Host "Configuration file not found: $ConfigFile" -ForegroundColor Red
        continue
    }
    
    Write-Host "Configuration loaded: $ConfigFile" -ForegroundColor Cyan
    
    if ($Interactive) {
        Write-Host "`nStrategy Tester Configuration for $Dataset:" -ForegroundColor Magenta
        Write-Host "1. Open Strategy Tester (Ctrl+R)" -ForegroundColor White
        Write-Host "2. Configure according to $ConfigFile" -ForegroundColor White
        Write-Host "3. Start test and wait for completion" -ForegroundColor White
        Write-Host "4. Press Enter when test is complete..." -ForegroundColor White
        Read-Host
        
        # Collect results
        Write-Host "Enter test results for $Dataset:" -ForegroundColor Green
        $Results = @{}
        $Results.total_net_profit = Read-Host "Total Net Profit"
        $Results.profit_factor = Read-Host "Profit Factor"
        $Results.total_trades = Read-Host "Total Trades"
        $Results.profit_trades_percent = Read-Host "Winning Trades (%)"
        $Results.relative_drawdown_percent = Read-Host "Maximum Drawdown (%)"
        
        # Save results
        $TestResults = @{
            dataset_name = $Dataset
            test_date = Get-Date -Format "yyyy-MM-dd HH:mm:ss"
            ea_version = "current"
            performance_metrics = $Results
            test_completion_status = "COMPLETED"
        }
        
        $ResultsFile = "test_results_$($Dataset.ToLower())_$(Get-Date -Format 'yyyyMMdd_HHmmss').json"
        $TestResults | ConvertTo-Json -Depth 3 | Out-File $ResultsFile
        Write-Host "Results saved: $ResultsFile" -ForegroundColor Green
    }
}

Write-Host "`n=== Test Suite Execution Complete ===" -ForegroundColor Green

Results Analysis Script

File: analyze_test_results.ps1

param(
    [string]$ResultsPattern = "test_results_*.json"
)

Write-Host "=== Test Results Analysis ===" -ForegroundColor Green

$ResultFiles = Get-ChildItem -Filter $ResultsPattern | Sort-Object LastWriteTime -Descending

if ($ResultFiles.Count -eq 0) {
    Write-Host "No test result files found matching pattern: $ResultsPattern" -ForegroundColor Red
    exit
}

Write-Host "Found $($ResultFiles.Count) test result files" -ForegroundColor Cyan

$AllResults = @()
foreach ($File in $ResultFiles) {
    try {
        $Result = Get-Content $File.FullName | ConvertFrom-Json
        $AllResults += $Result
    } catch {
        Write-Host "Error reading $($File.Name): $($_.Exception.Message)" -ForegroundColor Red
    }
}

# Group by dataset
$GroupedResults = $AllResults | Group-Object dataset_name

Write-Host "`n=== Results Summary ===" -ForegroundColor Green
foreach ($Group in $GroupedResults) {
    Write-Host "`nDataset: $($Group.Name)" -ForegroundColor Yellow
    
    $LatestResult = $Group.Group | Sort-Object test_date -Descending | Select-Object -First 1
    
    Write-Host "  Latest Test: $($LatestResult.test_date)" -ForegroundColor Cyan
    Write-Host "  Profit Factor: $($LatestResult.performance_metrics.profit_factor)" -ForegroundColor Cyan
    Write-Host "  Win Rate: $($LatestResult.performance_metrics.profit_trades_percent)%" -ForegroundColor Cyan
    Write-Host "  Total Trades: $($LatestResult.performance_metrics.total_trades)" -ForegroundColor Cyan
    Write-Host "  Max Drawdown: $($LatestResult.performance_metrics.relative_drawdown_percent)%" -ForegroundColor Cyan
    
    # Performance assessment
    $PF = [double]$LatestResult.performance_metrics.profit_factor
    $WR = [double]$LatestResult.performance_metrics.profit_trades_percent
    $DD = [double]$LatestResult.performance_metrics.relative_drawdown_percent
    
    $Assessment = "GOOD"
    if ($PF -lt 1.2 -or $WR -lt 45 -or $DD -gt 15) {
        $Assessment = "NEEDS ATTENTION"
    }
    if ($PF -lt 1.0 -or $WR -lt 35 -or $DD -gt 25) {
        $Assessment = "POOR"
    }
    
    Write-Host "  Assessment: $Assessment" -ForegroundColor $(if($Assessment -eq "GOOD"){"Green"}elseif($Assessment -eq "NEEDS ATTENTION"){"Yellow"}else{"Red"})
}

# Generate summary report
$SummaryReport = @{
    analysis_date = Get-Date -Format "yyyy-MM-dd HH:mm:ss"
    total_datasets_analyzed = $GroupedResults.Count
    results_summary = @()
}

foreach ($Group in $GroupedResults) {
    $Latest = $Group.Group | Sort-Object test_date -Descending | Select-Object -First 1
    $SummaryReport.results_summary += @{
        dataset = $Group.Name
        latest_test_date = $Latest.test_date
        profit_factor = $Latest.performance_metrics.profit_factor
        win_rate = $Latest.performance_metrics.profit_trades_percent
        max_drawdown = $Latest.performance_metrics.relative_drawdown_percent
        total_trades = $Latest.performance_metrics.total_trades
    }
}

$SummaryFile = "test_analysis_summary_$(Get-Date -Format 'yyyyMMdd_HHmmss').json"
$SummaryReport | ConvertTo-Json -Depth 3 | Out-File $SummaryFile
Write-Host "`nSummary report saved: $SummaryFile" -ForegroundColor Green

🎯 Implementation for Your Current Setup

Immediate Next Steps:

1. Generate test configurations using the scripts above
2. Use your current Strategy Tester setup to run the BASELINE_3M_EURUSD test
3. Document the results using the standardized template
4. Establish this as your benchmark for all future development

Integration with Development Workflow:

• Before each feature: Run baseline test to establish pre-change performance
• After each feature: Run same test to validate no regression
• Weekly: Run multi-dataset test suite for comprehensive validation
• Before releases: Run full test suite across all datasets

---

Next Action: Generate the test configurations and run your first standardized baseline test using the Strategy Tester you currently have open.