- Adjusted session consistency test results to reflect empty DataFrame.
- Updated candidate counts and hashes in pipeline manifest explorer.
- Modified Explorer Diagnostics CSV files to reflect new execution counts and removed unnecessary columns.
- Revised signal loss reports for M15 and M5 to remove stage information while retaining lost counts.
- Enhanced explorer consistency report with updated generated and loaded counts, along with TBM, HMM, XGB pass rates, and executed trades.
- Introduced `pipeline_certification.py` for Phase 13 checks on the observability layer, including stage order, monotonicity, unique IDs, and stale reports.
- Implemented `pipeline_profiler.py` for event-based profiling, ensuring integrity across pipeline stages and providing methods for recording events and generating reports.
- Created `prediction.py` to standardize prediction results and ensure compatibility across different components.
- Added a smoke test script to validate module imports and function signatures before running notebooks.
- Implemented survival analysis report generation with CSV output.
- Added probability report generation with summary statistics.
- Introduced feature drift report with metrics for monitoring shifts.
- Created HMM regime report for per-timeframe analysis.
- Developed self-verification table to ensure required artifacts are present and contain data.
- Added new CSV files: probability_report_M5.csv and survival_analysis.csv.
- Enhanced pipeline_health.txt to include metrics for M15 and M5, highlighting generated, executed, and survival rates, along with bottleneck information.
- Added detailed rejection records for M15 and M5 in top100_rejected_M15.csv and top100_rejected_M5.csv, respectively, indicating the filtering stage and reasons for rejection.
- Set execution counts for initial cells to 1, 2, 3, 4, and 5 respectively.
- Replace error outputs with informative stdout messages for better clarity.
- Add error handling for missing bridge file with a clear message to run Strategy_Tester first.
This commit introduces a new smoke test script that exercises every phase of the pipeline observability process. The test generates synthetic data for two timeframes (M15 and M5), simulating various stages of the pipeline and verifying that all artifacts are produced and non-empty. The script includes assertions to ensure the integrity of the generated outputs and checks for expected conditions in the decision log, survival matrix, and audit JSON.
- Updated GoldRegime_X.code-workspace to remove deprecated server configuration.
- Added package.json and package-lock.json to manage dependencies, specifically @notionhq/client.
- Introduced patch_explorer_cell46.py to resolve ModuleNotFoundError in the PIPELINE VERIFICATION import cell of GoldRegimeX_Explorer.ipynb.
- Created patch_explorer_cell48.py to fix NameError for TIMEFRAMES in the Dataset Integrity verification cell, ensuring it can derive TIMEFRAMES from the pipeline if not already defined.
- Introduced `pipeline_verification.py` containing multiple verifier classes for dataset integrity, feature leakage, evaluation integrity, prediction alignment, and more.
- Added `SessionFilter` class in `session_filter.py` for unified session/time-of-day filtering.
- Implemented `VerificationResult` data class for structured verification results.
- Created utility functions for hashing and manifest building.
- Established a test suite for the session filter to ensure correct functionality.
- Updated `__init__.py` to include the new shared module.
Luke Mosoti