141 lines
7.1 KiB
Markdown
141 lines
7.1 KiB
Markdown
# 08 — The Workflow Cycle
|
||
|
||
Everything above is parts; this is the assembly line. One research idea travels through a fixed,
|
||
repeatable cycle. The discipline of the cycle — especially the **progressive scope** (test the
|
||
*smallest* thing first) — is what keeps you from overfitting and what makes results explainable.
|
||
|
||
---
|
||
|
||
## 1. The cycle
|
||
|
||
```
|
||
1. HYPOTHESIS — state the idea in one sentence, grounded in an existing engine + preset.
|
||
2. SCAFFOLD — create the iteration folder + run script + declared search space + run notes.
|
||
3. STATS — measure signals & filters on real history. Pick the primary timeframe. No tuning yet.
|
||
4. MINIMAL SCOPE — test the smallest version of the idea. Is there ANY edge? If no → stop here.
|
||
5. EXPAND + OPTUNA— only if minimal scope showed edge: add layers one at a time, then Optuna search.
|
||
6. MT5-VERIFY — run the 2–3 diverse finalists in the real terminal. Python ranks; MT5 decides.
|
||
7. APPROVAL — a human looks at the comparison and explicitly decides.
|
||
8. PROMOTE — approved result is copied into the locked registry with full context.
|
||
```
|
||
|
||
The early steps are cheap and exist to **kill bad ideas before they cost compute**. Most ideas should
|
||
die at step 3 or 4. That's the system working, not failing.
|
||
|
||
---
|
||
|
||
## 2. Step-by-step
|
||
|
||
### Step 1 — Hypothesis
|
||
Write it down in one sentence and tie it to something that already exists: *"On instrument X, adding a
|
||
daily-trend filter to preset Y should cut the drawdown from the counter-trend series without killing
|
||
Net."* A hypothesis that isn't grounded in an existing engine + baseline preset is a research project,
|
||
not an iteration — narrow it first.
|
||
|
||
### Step 2 — Scaffold
|
||
Create the iteration folder by convention (doc 02 §4):
|
||
|
||
```
|
||
strategies/<strategy>/iterations/<base-preset>-<approach>-<YYYY-MM-DD>/
|
||
├── README.md # the hypothesis, status, and (later) the result
|
||
├── parameter-space.md # the declared search space + the reasoning for each range
|
||
├── optimize.py # a self-contained snapshot of the run script
|
||
└── wizard-answers.yaml # written by the wizard at run time
|
||
```
|
||
|
||
Copy the run script from a previous iteration as a template and edit it — each iteration owns its
|
||
snapshot (doc 04 Rule 5). A data/profile pre-check belongs here: confirm the instrument's data and
|
||
config exist *before* scaffolding (for a brand-new symbol, do the data-download + config step from doc
|
||
07 §6 / doc 05 §1 first).
|
||
|
||
### Step 3 — Stats (measure before you tune)
|
||
Before any parameter search, **characterize the signals on real history**:
|
||
|
||
- How often does the signal fire? What's the raw win rate, the average favorable vs adverse excursion?
|
||
- **Multi-timeframe filter analysis:** if the idea is a trend/regime filter, measure how well it
|
||
separates good from bad entries on *several* timeframes (e.g. M15/H1/H4/D1) and pick the **one
|
||
primary timeframe** where the separation is cleanest. Lock it for the rest of the iteration.
|
||
- Output: a short stats note + the chosen primary timeframe + the specific levers worth tuning. **No
|
||
parameter sweep yet.**
|
||
|
||
This step routinely ends an iteration: if the signal has no separation on any timeframe, there's
|
||
nothing to optimize, and you've spent minutes instead of hours learning that.
|
||
|
||
### Step 4 — Minimal scope (the anti-overfit gate)
|
||
Test the **smallest possible version** of the idea against the baseline:
|
||
|
||
- Core engine behavior + the new filter, with **exits minimal** — e.g. for a trend strategy: trailing
|
||
only, *no* break-even; for a grid: the first level only, *no* depth/martingale/second tier.
|
||
- One question: **is there any edge at all** over the baseline? A handful of A/B runs, not a search.
|
||
- If the minimal version shows no edge, **stop**. Piling parameters onto a non-edge just manufactures an
|
||
overfit that looks good in-sample and dies in MT5. Most ideas should end here.
|
||
|
||
### Step 5 — Expand + Optuna (only if step 4 showed edge)
|
||
Now add complexity **one layer at a time**, each as an A/B against the previous best:
|
||
|
||
- add break-even, then a second grid tier, then an ATR-based stop, then wider ranges — *in sequence*,
|
||
keeping only what improves things.
|
||
- when the structure is set, run the **Optuna study** (doc 06): smoke-test first, then the full run in
|
||
the background. Apply the constraints and the fragility guard. Select **2–3 diverse finalists** (not
|
||
top-N-by-score). Run the robustness layers on them.
|
||
|
||
Progressive expansion keeps every result interpretable ("the second tier added X, the ATR stop added
|
||
Y") and keeps the search space small enough that Optuna isn't just mining noise.
|
||
|
||
### Step 6 — MT5-verify (finalists only)
|
||
Run the 2–3 finalists through the bridge (doc 07). Use a fast tick model to start, real ticks for the
|
||
final path-sensitive check. Write the `auto-verification.md` comparison and judge each delta against the
|
||
expected fidelity gap (doc 03 §7). **Don't verify every step of the search** — you trust the Python
|
||
engine for ranking; MT5 is reserved for the numbers that might go live.
|
||
|
||
### Step 7 — Approval
|
||
A human reviews the comparison and the robustness signals and **explicitly** decides: promote, discard,
|
||
or iterate. Nothing auto-promotes. This is the gate where judgment — does this make sense, is the
|
||
drawdown livable, is the trade count real — overrides any single metric.
|
||
|
||
### Step 8 — Promote
|
||
On approval, copy the finalist into the **locked registry** (doc 04 Rule 7) with everything needed to
|
||
trust and reproduce it: the params, the Python metrics, the MT5 report, and the context (period,
|
||
instrument, why approved). Discarded iterations are archived, not deleted — a negative result is data,
|
||
and re-testing a known dead end is waste.
|
||
|
||
---
|
||
|
||
## 3. When to start a new iteration vs use the scratchpad
|
||
|
||
- **New iteration** (full folder, will be tracked): a genuine improvement idea for a specific baseline;
|
||
an engine update that requires re-verifying existing approved results; new data/period to re-test on.
|
||
- **Scratchpad** (`results/` — never promoted): a quick experiment you don't intend to keep. Don't
|
||
pollute `iterations/` or the registry with throwaways.
|
||
|
||
---
|
||
|
||
## 4. Naming convention
|
||
|
||
```
|
||
<base-preset>-<approach>-<YYYY-MM-DD>/
|
||
```
|
||
|
||
e.g. `trend-filter-daily-ema-2026-06-15`, `grid-second-tier-atr-2026-07-01`. The *approach* word is
|
||
free-form (coin it as needed); the date is the day of creation. Consistent names make the `iterations/`
|
||
folder a readable history of what you tried and when.
|
||
|
||
---
|
||
|
||
## 5. The cycle as a habit
|
||
|
||
Run it the same way every time and three good things happen automatically:
|
||
|
||
- **Bad ideas die cheap** (steps 3–4), so compute goes to ideas with a pulse.
|
||
- **Surviving ideas are interpretable** (step 5's one-layer-at-a-time), so you understand *why* they
|
||
work — which is what tells you whether they'll keep working.
|
||
- **Promoted results are trustworthy and reproducible** (steps 6–8), so the registry stays a source of
|
||
truth rather than a graveyard of optimistic backtests.
|
||
|
||
That's the whole lab: a fast engine you trust, a disciplined search you can't fool yourself with, and a
|
||
real-terminal check before anything counts.
|
||
|
||
---
|
||
|
||
*Back to [`README.md`](README.md) · setup playbook [`CLAUDE.md`](CLAUDE.md).*
|