MT5-EA-Sniper-Strategy/docs/STANDARDIZED_TEST_DATASETS.md

# 📊 Standardized Test Datasets for Consistent Backtesting

## 🎯 Objective
Establish standardized test datasets and scenarios to ensure consistent, repeatable backtesting results across all development iterations and feature implementations.

## 📋 Test Dataset Categories

### Primary Test Datasets

#### Dataset 1: Baseline Performance (3 Months)
```
Name: BASELINE_3M_EURUSD
Symbol: EURUSD
Period: 2024-07-01 to 2024-09-25
Timeframe: M1
Characteristics: Mixed market conditions, normal volatility
Purpose: Primary regression testing and performance benchmarking
Expected Trades: 80-120
Expected Win Rate: 50-65%
```

#### Dataset 2: High Volatility Period (1 Month)
```
Name: HIGH_VOL_1M_EURUSD
Symbol: EURUSD
Period: 2024-08-01 to 2024-08-31 (NFP releases, ECB meetings)
Timeframe: M1
Characteristics: High volatility, frequent news events
Purpose: Stress testing and volatility handling validation
Expected Trades: 40-60
Expected Drawdown: Higher than baseline
```

#### Dataset 3: Low Volatility Period (1 Month)
```
Name: LOW_VOL_1M_EURUSD
Symbol: EURUSD
Period: 2024-07-15 to 2024-08-15 (Summer trading period)
Timeframe: M1
Characteristics: Low volatility, ranging markets
Purpose: Pattern detection accuracy in quiet markets
Expected Trades: 20-40
Expected Win Rate: Higher precision expected
```

#### Dataset 4: Trending Market (2 Weeks)
```
Name: TREND_2W_EURUSD
Symbol: EURUSD
Period: 2024-09-01 to 2024-09-15 (Strong directional move)
Timeframe: M1
Characteristics: Clear trending conditions
Purpose: Trend-following strategy validation
Expected Trades: 25-40
Expected R:R: Higher than baseline
```

### Multi-Symbol Test Datasets

#### Dataset 5: Major Pairs Comparison (1 Month)
```
Symbols: EURUSD, GBPUSD, USDJPY, USDCHF
Period: 2024-08-01 to 2024-08-31
Purpose: Cross-pair performance validation
Expected Behavior: Consistent pattern detection across pairs
```

#### Dataset 6: Commodity Currency Test (1 Month)
```
Symbols: AUDUSD, NZDUSD, USDCAD
Period: 2024-08-01 to 2024-08-31
Purpose: Commodity-correlated pair testing
Expected Behavior: Different volatility patterns
```

#### Dataset 7: Gold Testing (2 Weeks)
```
Symbol: XAUUSD
Period: 2024-09-01 to 2024-09-15
Purpose: High-value, high-volatility instrument testing
Expected Behavior: Larger pip movements, different risk calculations
```

## 🔧 Test Dataset Configuration Scripts

### Dataset Configuration Generator

**File**: `generate_test_configs.ps1`
```powershell
# Generate standardized test configurations
$TestDatasets = @(
    @{
        Name = "BASELINE_3M_EURUSD"
        Symbol = "EURUSD"
        StartDate = "2024-07-01"
        EndDate = "2024-09-25"
        Description = "Primary baseline performance dataset"
        ExpectedTrades = "80-120"
        ExpectedWinRate = "50-65%"
        Purpose = "Regression testing and benchmarking"
    },
    @{
        Name = "HIGH_VOL_1M_EURUSD"
        Symbol = "EURUSD"
        StartDate = "2024-08-01"
        EndDate = "2024-08-31"
        Description = "High volatility stress testing"
        ExpectedTrades = "40-60"
        ExpectedWinRate = "45-60%"
        Purpose = "Volatility handling validation"
    },
    @{
        Name = "LOW_VOL_1M_EURUSD"
        Symbol = "EURUSD"
        StartDate = "2024-07-15"
        EndDate = "2024-08-15"
        Description = "Low volatility precision testing"
        ExpectedTrades = "20-40"
        ExpectedWinRate = "55-70%"
        Purpose = "Pattern detection accuracy in quiet markets"
    },
    @{
        Name = "TREND_2W_EURUSD"
        Symbol = "EURUSD"
        StartDate = "2024-09-01"
        EndDate = "2024-09-15"
        Description = "Trending market validation"
        ExpectedTrades = "25-40"
        ExpectedWinRate = "50-65%"
        Purpose = "Trend-following strategy validation"
    }
)

# Generate configuration files
foreach ($Dataset in $TestDatasets) {
    $ConfigContent = @"
# Test Dataset Configuration: $($Dataset.Name)

## Dataset Information
- **Name**: $($Dataset.Name)
- **Symbol**: $($Dataset.Symbol)
- **Period**: $($Dataset.StartDate) to $($Dataset.EndDate)
- **Description**: $($Dataset.Description)
- **Purpose**: $($Dataset.Purpose)

## Expected Results
- **Expected Trades**: $($Dataset.ExpectedTrades)
- **Expected Win Rate**: $($Dataset.ExpectedWinRate)

## Strategy Tester Configuration
```
Expert Advisor: SniperEA
Symbol: $($Dataset.Symbol)
Period: M1
Date Range: $($Dataset.StartDate) to $($Dataset.EndDate)
Model: Every tick (based on all available least timeframes)
Optimization: Disabled
Deposit: 10000 USD
Leverage: 1:100
```

## EA Parameters (Standard Configuration)
```
RiskPercent = 1.0
MinRR = 2.0
MaxPositionsPerSymbol = 3
MaxTotalPositions = 10
OBStrengthFilter = 1.0
MinFVGSize = 5.0
BOSConfirmationBars = 3
SweepDistanceThreshold = 10.0
UseTimeFilter = true
SessionTimeStart = "08:00"
SessionTimeEnd = "21:00"
EnableDebugMode = true
EnableDetailedLogging = true
```

## Test Execution Steps
1. Open Strategy Tester (Ctrl+R)
2. Apply configuration above
3. Click Start
4. Record results in test_results_$($Dataset.Name.ToLower()).json
5. Compare with expected results
6. Document any deviations

## Success Criteria
- Compilation successful
- Test completes without errors
- Results within expected ranges
- Pattern detection functioning
- Risk management compliance (exactly 1% per trade)
"@

    $ConfigFile = "test_config_$($Dataset.Name.ToLower()).md"
    $ConfigContent | Out-File $ConfigFile -Encoding UTF8
    Write-Host "Generated: $ConfigFile" -ForegroundColor Green
}

Write-Host "`nAll test dataset configurations generated!" -ForegroundColor Green
```

### Test Results Template Generator

**File**: `generate_results_templates.ps1`
```powershell
# Generate standardized results templates
$TestDatasets = @("BASELINE_3M_EURUSD", "HIGH_VOL_1M_EURUSD", "LOW_VOL_1M_EURUSD", "TREND_2W_EURUSD")

foreach ($Dataset in $TestDatasets) {
    $TemplateContent = @"
{
  "dataset_name": "$Dataset",
  "test_date": "",
  "ea_version": "",
  "test_duration_minutes": 0,
  "compilation_status": "SUCCESS/FAILED",
  "test_completion_status": "COMPLETED/FAILED",
  
  "performance_metrics": {
    "total_net_profit": 0.0,
    "gross_profit": 0.0,
    "gross_loss": 0.0,
    "profit_factor": 0.0,
    "expected_payoff": 0.0,
    "absolute_drawdown": 0.0,
    "maximal_drawdown": 0.0,
    "relative_drawdown_percent": 0.0,
    "total_trades": 0,
    "short_positions_won": 0,
    "long_positions_won": 0,
    "profit_trades_percent": 0.0,
    "loss_trades_percent": 0.0,
    "largest_profit_trade": 0.0,
    "largest_loss_trade": 0.0,
    "average_profit_trade": 0.0,
    "average_loss_trade": 0.0,
    "maximum_consecutive_wins": 0,
    "maximum_consecutive_losses": 0,
    "maximal_consecutive_profit": 0.0,
    "maximal_consecutive_loss": 0.0,
    "average_consecutive_wins": 0,
    "average_consecutive_losses": 0
  },
  
  "risk_management_validation": {
    "risk_per_trade_compliance": true,
    "position_limits_respected": true,
    "stop_loss_placement_accurate": true,
    "take_profit_calculation_correct": true,
    "lot_size_calculation_accurate": true
  },
  
  "pattern_detection_stats": {
    "order_blocks_detected": 0,
    "fair_value_gaps_detected": 0,
    "bos_events_detected": 0,
    "liquidity_sweeps_detected": 0,
    "valid_setups_identified": 0,
    "setup_conversion_rate_percent": 0.0
  },
  
  "session_analysis": {
    "london_session_trades": 0,
    "new_york_session_trades": 0,
    "asian_session_trades": 0,
    "off_hours_trades": 0
  },
  
  "comparison_with_expected": {
    "trades_within_expected_range": true,
    "win_rate_within_expected_range": true,
    "performance_meets_expectations": true,
    "deviations_noted": []
  },
  
  "issues_identified": [],
  "recommendations": [],
  "overall_assessment": "PASS/CONDITIONAL/FAIL",
  "notes": ""
}
"@

    $TemplateFile = "test_results_template_$($Dataset.ToLower()).json"
    $TemplateContent | Out-File $TemplateFile -Encoding UTF8
    Write-Host "Generated: $TemplateFile" -ForegroundColor Green
}

Write-Host "`nAll results templates generated!" -ForegroundColor Green
```

## 📊 Test Execution Automation

### Batch Test Runner

**File**: `run_standardized_tests.ps1`
```powershell
param(
    [string[]]$Datasets = @("BASELINE_3M_EURUSD", "HIGH_VOL_1M_EURUSD", "LOW_VOL_1M_EURUSD"),
    [switch]$Interactive = $true
)

Write-Host "=== Standardized Test Suite Execution ===" -ForegroundColor Green

foreach ($Dataset in $Datasets) {
    Write-Host "`n--- Testing Dataset: $Dataset ---" -ForegroundColor Yellow
    
    # Load test configuration
    $ConfigFile = "test_config_$($Dataset.ToLower()).md"
    if (-not (Test-Path $ConfigFile)) {
        Write-Host "Configuration file not found: $ConfigFile" -ForegroundColor Red
        continue
    }
    
    Write-Host "Configuration loaded: $ConfigFile" -ForegroundColor Cyan
    
    if ($Interactive) {
        Write-Host "`nStrategy Tester Configuration for $Dataset:" -ForegroundColor Magenta
        Write-Host "1. Open Strategy Tester (Ctrl+R)" -ForegroundColor White
        Write-Host "2. Configure according to $ConfigFile" -ForegroundColor White
        Write-Host "3. Start test and wait for completion" -ForegroundColor White
        Write-Host "4. Press Enter when test is complete..." -ForegroundColor White
        Read-Host
        
        # Collect results
        Write-Host "Enter test results for $Dataset:" -ForegroundColor Green
        $Results = @{}
        $Results.total_net_profit = Read-Host "Total Net Profit"
        $Results.profit_factor = Read-Host "Profit Factor"
        $Results.total_trades = Read-Host "Total Trades"
        $Results.profit_trades_percent = Read-Host "Winning Trades (%)"
        $Results.relative_drawdown_percent = Read-Host "Maximum Drawdown (%)"
        
        # Save results
        $TestResults = @{
            dataset_name = $Dataset
            test_date = Get-Date -Format "yyyy-MM-dd HH:mm:ss"
            ea_version = "current"
            performance_metrics = $Results
            test_completion_status = "COMPLETED"
        }
        
        $ResultsFile = "test_results_$($Dataset.ToLower())_$(Get-Date -Format 'yyyyMMdd_HHmmss').json"
        $TestResults | ConvertTo-Json -Depth 3 | Out-File $ResultsFile
        Write-Host "Results saved: $ResultsFile" -ForegroundColor Green
    }
}

Write-Host "`n=== Test Suite Execution Complete ===" -ForegroundColor Green
```

### Results Analysis Script

**File**: `analyze_test_results.ps1`
```powershell
param(
    [string]$ResultsPattern = "test_results_*.json"
)

Write-Host "=== Test Results Analysis ===" -ForegroundColor Green

$ResultFiles = Get-ChildItem -Filter $ResultsPattern | Sort-Object LastWriteTime -Descending

if ($ResultFiles.Count -eq 0) {
    Write-Host "No test result files found matching pattern: $ResultsPattern" -ForegroundColor Red
    exit
}

Write-Host "Found $($ResultFiles.Count) test result files" -ForegroundColor Cyan

$AllResults = @()
foreach ($File in $ResultFiles) {
    try {
        $Result = Get-Content $File.FullName | ConvertFrom-Json
        $AllResults += $Result
    } catch {
        Write-Host "Error reading $($File.Name): $($_.Exception.Message)" -ForegroundColor Red
    }
}

# Group by dataset
$GroupedResults = $AllResults | Group-Object dataset_name

Write-Host "`n=== Results Summary ===" -ForegroundColor Green
foreach ($Group in $GroupedResults) {
    Write-Host "`nDataset: $($Group.Name)" -ForegroundColor Yellow
    
    $LatestResult = $Group.Group | Sort-Object test_date -Descending | Select-Object -First 1
    
    Write-Host "  Latest Test: $($LatestResult.test_date)" -ForegroundColor Cyan
    Write-Host "  Profit Factor: $($LatestResult.performance_metrics.profit_factor)" -ForegroundColor Cyan
    Write-Host "  Win Rate: $($LatestResult.performance_metrics.profit_trades_percent)%" -ForegroundColor Cyan
    Write-Host "  Total Trades: $($LatestResult.performance_metrics.total_trades)" -ForegroundColor Cyan
    Write-Host "  Max Drawdown: $($LatestResult.performance_metrics.relative_drawdown_percent)%" -ForegroundColor Cyan
    
    # Performance assessment
    $PF = [double]$LatestResult.performance_metrics.profit_factor
    $WR = [double]$LatestResult.performance_metrics.profit_trades_percent
    $DD = [double]$LatestResult.performance_metrics.relative_drawdown_percent
    
    $Assessment = "GOOD"
    if ($PF -lt 1.2 -or $WR -lt 45 -or $DD -gt 15) {
        $Assessment = "NEEDS ATTENTION"
    }
    if ($PF -lt 1.0 -or $WR -lt 35 -or $DD -gt 25) {
        $Assessment = "POOR"
    }
    
    Write-Host "  Assessment: $Assessment" -ForegroundColor $(if($Assessment -eq "GOOD"){"Green"}elseif($Assessment -eq "NEEDS ATTENTION"){"Yellow"}else{"Red"})
}

# Generate summary report
$SummaryReport = @{
    analysis_date = Get-Date -Format "yyyy-MM-dd HH:mm:ss"
    total_datasets_analyzed = $GroupedResults.Count
    results_summary = @()
}

foreach ($Group in $GroupedResults) {
    $Latest = $Group.Group | Sort-Object test_date -Descending | Select-Object -First 1
    $SummaryReport.results_summary += @{
        dataset = $Group.Name
        latest_test_date = $Latest.test_date
        profit_factor = $Latest.performance_metrics.profit_factor
        win_rate = $Latest.performance_metrics.profit_trades_percent
        max_drawdown = $Latest.performance_metrics.relative_drawdown_percent
        total_trades = $Latest.performance_metrics.total_trades
    }
}

$SummaryFile = "test_analysis_summary_$(Get-Date -Format 'yyyyMMdd_HHmmss').json"
$SummaryReport | ConvertTo-Json -Depth 3 | Out-File $SummaryFile
Write-Host "`nSummary report saved: $SummaryFile" -ForegroundColor Green
```

## 🎯 Implementation for Your Current Setup

### Immediate Next Steps:
1. **Generate test configurations** using the scripts above
2. **Use your current Strategy Tester setup** to run the BASELINE_3M_EURUSD test
3. **Document the results** using the standardized template
4. **Establish this as your benchmark** for all future development

### Integration with Development Workflow:
- **Before each feature**: Run baseline test to establish pre-change performance
- **After each feature**: Run same test to validate no regression
- **Weekly**: Run multi-dataset test suite for comprehensive validation
- **Before releases**: Run full test suite across all datasets

---

**Next Action**: Generate the test configurations and run your first standardized baseline test using the Strategy Tester you currently have open.