7.1 KiB
08 — The Workflow Cycle
Everything above is parts; this is the assembly line. One research idea travels through a fixed, repeatable cycle. The discipline of the cycle — especially the progressive scope (test the smallest thing first) — is what keeps you from overfitting and what makes results explainable.
1. The cycle
1. HYPOTHESIS — state the idea in one sentence, grounded in an existing engine + preset.
2. SCAFFOLD — create the iteration folder + run script + declared search space + run notes.
3. STATS — measure signals & filters on real history. Pick the primary timeframe. No tuning yet.
4. MINIMAL SCOPE — test the smallest version of the idea. Is there ANY edge? If no → stop here.
5. EXPAND + OPTUNA— only if minimal scope showed edge: add layers one at a time, then Optuna search.
6. MT5-VERIFY — run the 2–3 diverse finalists in the real terminal. Python ranks; MT5 decides.
7. APPROVAL — a human looks at the comparison and explicitly decides.
8. PROMOTE — approved result is copied into the locked registry with full context.
The early steps are cheap and exist to kill bad ideas before they cost compute. Most ideas should die at step 3 or 4. That's the system working, not failing.
2. Step-by-step
Step 1 — Hypothesis
Write it down in one sentence and tie it to something that already exists: "On instrument X, adding a daily-trend filter to preset Y should cut the drawdown from the counter-trend series without killing Net." A hypothesis that isn't grounded in an existing engine + baseline preset is a research project, not an iteration — narrow it first.
Step 2 — Scaffold
Create the iteration folder by convention (doc 02 §4):
strategies/<strategy>/iterations/<base-preset>-<approach>-<YYYY-MM-DD>/
├── README.md # the hypothesis, status, and (later) the result
├── parameter-space.md # the declared search space + the reasoning for each range
├── optimize.py # a self-contained snapshot of the run script
└── wizard-answers.yaml # written by the wizard at run time
Copy the run script from a previous iteration as a template and edit it — each iteration owns its snapshot (doc 04 Rule 5). A data/profile pre-check belongs here: confirm the instrument's data and config exist before scaffolding (for a brand-new symbol, do the data-download + config step from doc 07 §6 / doc 05 §1 first).
Step 3 — Stats (measure before you tune)
Before any parameter search, characterize the signals on real history:
- How often does the signal fire? What's the raw win rate, the average favorable vs adverse excursion?
- Multi-timeframe filter analysis: if the idea is a trend/regime filter, measure how well it separates good from bad entries on several timeframes (e.g. M15/H1/H4/D1) and pick the one primary timeframe where the separation is cleanest. Lock it for the rest of the iteration.
- Output: a short stats note + the chosen primary timeframe + the specific levers worth tuning. No parameter sweep yet.
This step routinely ends an iteration: if the signal has no separation on any timeframe, there's nothing to optimize, and you've spent minutes instead of hours learning that.
Step 4 — Minimal scope (the anti-overfit gate)
Test the smallest possible version of the idea against the baseline:
- Core engine behavior + the new filter, with exits minimal — e.g. for a trend strategy: trailing only, no break-even; for a grid: the first level only, no depth/martingale/second tier.
- One question: is there any edge at all over the baseline? A handful of A/B runs, not a search.
- If the minimal version shows no edge, stop. Piling parameters onto a non-edge just manufactures an overfit that looks good in-sample and dies in MT5. Most ideas should end here.
Step 5 — Expand + Optuna (only if step 4 showed edge)
Now add complexity one layer at a time, each as an A/B against the previous best:
- add break-even, then a second grid tier, then an ATR-based stop, then wider ranges — in sequence, keeping only what improves things.
- when the structure is set, run the Optuna study (doc 06): smoke-test first, then the full run in the background. Apply the constraints and the fragility guard. Select 2–3 diverse finalists (not top-N-by-score). Run the robustness layers on them.
Progressive expansion keeps every result interpretable ("the second tier added X, the ATR stop added Y") and keeps the search space small enough that Optuna isn't just mining noise.
Step 6 — MT5-verify (finalists only)
Run the 2–3 finalists through the bridge (doc 07). Use a fast tick model to start, real ticks for the
final path-sensitive check. Write the auto-verification.md comparison and judge each delta against the
expected fidelity gap (doc 03 §7). Don't verify every step of the search — you trust the Python
engine for ranking; MT5 is reserved for the numbers that might go live.
Step 7 — Approval
A human reviews the comparison and the robustness signals and explicitly decides: promote, discard, or iterate. Nothing auto-promotes. This is the gate where judgment — does this make sense, is the drawdown livable, is the trade count real — overrides any single metric.
Step 8 — Promote
On approval, copy the finalist into the locked registry (doc 04 Rule 7) with everything needed to trust and reproduce it: the params, the Python metrics, the MT5 report, and the context (period, instrument, why approved). Discarded iterations are archived, not deleted — a negative result is data, and re-testing a known dead end is waste.
3. When to start a new iteration vs use the scratchpad
- New iteration (full folder, will be tracked): a genuine improvement idea for a specific baseline; an engine update that requires re-verifying existing approved results; new data/period to re-test on.
- Scratchpad (
results/— never promoted): a quick experiment you don't intend to keep. Don't polluteiterations/or the registry with throwaways.
4. Naming convention
<base-preset>-<approach>-<YYYY-MM-DD>/
e.g. trend-filter-daily-ema-2026-06-15, grid-second-tier-atr-2026-07-01. The approach word is
free-form (coin it as needed); the date is the day of creation. Consistent names make the iterations/
folder a readable history of what you tried and when.
5. The cycle as a habit
Run it the same way every time and three good things happen automatically:
- Bad ideas die cheap (steps 3–4), so compute goes to ideas with a pulse.
- Surviving ideas are interpretable (step 5's one-layer-at-a-time), so you understand why they work — which is what tells you whether they'll keep working.
- Promoted results are trustworthy and reproducible (steps 6–8), so the registry stays a source of truth rather than a graveyard of optimistic backtests.
That's the whole lab: a fast engine you trust, a disciplined search you can't fool yourself with, and a real-terminal check before anything counts.