gavindiazandClaude Code
5ca012fe3c
- mtlib/: shared data loader, backtest engine, metrics, cli wrapper, walkforward, GBK-safe console, smoke test (python -m mtlib smoke) - strategies/xauusd_scalp/: single-source signal (generate_signals + generate_signal_at share _check), params dataclass, all runners - Unify 3 known drifts: session filter (discrete set), spread 0.16, backtest signature defaults 1.2/0.6/1.0/0.2 - Fix walkforward verdict (both halves PF>=1) + drop fake stability ratio - Root shims keep old commands working; data/ holds JSON files - Baseline check: new vs old engine identical trades, PnL, final equity Co-Authored-By: Claude Code <noreply@anthropic.com>
61 lines
2.1 KiB
Python
61 lines
2.1 KiB
Python
"""数据加载 - 全项目唯一入口。
|
|
|
|
- JSON 文件(含 'data' 键的值列表) -> 排序、数值列强制转换、dropna 的 DataFrame
|
|
- 路径优先取 data/ 目录, 迁移期回退到仓库根目录
|
|
- 语义必须与旧脚本严格一致(排序/reset_index/dropna), 冒烟测试锁定等价
|
|
"""
|
|
|
|
import json
|
|
from pathlib import Path
|
|
|
|
import pandas as pd
|
|
|
|
# 数值列(与 data_fetch_candles 输出的指标列一致)
|
|
DATA_COLS = ['open', 'high', 'low', 'close', 'tick_volume',
|
|
'rsi_14', 'macd_12_26_9', 'macd_h_12_26_9', 'macd_s_12_26_9',
|
|
'ema_20', 'ema_50', 'atr_14', 'bbl_20_2', 'bbm_20_2', 'bbu_20_2',
|
|
'bbb_20_2', 'bbp_20_2', 'stochk_14_3_3', 'stochd_14_3_3', 'obv',
|
|
'cmf_20', 'adx_14', 'dmp_14', 'dmn_14']
|
|
|
|
|
|
def project_root():
|
|
"""仓库根目录 = mtlib 的上一级。"""
|
|
return Path(__file__).resolve().parent.parent
|
|
|
|
|
|
def data_path(filename):
|
|
"""解析数据文件路径: 优先 data/ 目录, 回退仓库根(迁移期兼容)。"""
|
|
p = project_root() / "data" / filename
|
|
if p.exists():
|
|
return p
|
|
alt = project_root() / filename
|
|
if alt.exists():
|
|
return alt
|
|
raise FileNotFoundError(
|
|
f"数据文件不存在: 尝试过 {p} 和 {alt}; 请先用 tools/fetch_data.py 拉取")
|
|
|
|
|
|
def load_data(path=None):
|
|
"""加载 candle JSON 为回测用 DataFrame。
|
|
|
|
path: 文件路径或文件名; None 时按数据目录规则解析。
|
|
返回的 DataFrame 列顺序与原始文件一致, 数值列转 float, NaN 行剔除。
|
|
"""
|
|
if path is None:
|
|
path = data_path("xauusd_m1_20k.json")
|
|
else:
|
|
p = Path(path)
|
|
if not p.exists() and p.parent == Path('.'):
|
|
p = data_path(str(path))
|
|
path = p
|
|
|
|
with open(path, 'r', encoding='utf-8') as f:
|
|
raw = json.load(f)
|
|
df = pd.DataFrame(raw['data'])
|
|
df['time'] = pd.to_datetime(df['time'])
|
|
df = df.sort_values('time').reset_index(drop=True)
|
|
for col in DATA_COLS:
|
|
if col in df.columns:
|
|
df[col] = pd.to_numeric(df[col], errors='coerce')
|
|
df = df.dropna().reset_index(drop=True)
|
|
return df |