gavindiaz/manifoldbt
1
0
Fork 0
mirror of https://github.com/manifoldbt/manifoldbt.git synced 2026-08-24 22:48:05 +00:00
Code Issues Packages Projects Releases Wiki Activity

release: v0.15.0

Browse Source
This commit is contained in:
github-actions[bot]
2026-08-16 12:02:58 +00:00
parent 1bc5fe3038
commit 5452806314
12 changed files with 1783 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/entry-orders-plan.md
+263
View File
@@ -0,0 +1,263 @@
# Plan — ordres d'entrée conditionnels (market / limit / stop à prix déterminé)
Branche : `claude/limit-market-order-entries-mqvww2` (depuis `main` @ `c483032`, 0.14.1).
## État d'avancement
| Phase | Statut |
|---|---|
| 0 — garde-fous perf | fait (baseline benchée, goldens verts) |
| 1 — modèle de prix d'entrée (boucles générales) | **fait** |
| 2 — surface utilisateur, validation, observabilité | **fait** |
| 3 — ordre au repos dans le kernel rapide + CUDA | **pas fait** — voir §3 |
| 3b — prix piloté par série sur GPU | pas fait |
| 4 — mesure | fait — `perf/entry-orders.md` |
Ce qui est livré couvre les quatre déclencheurs (`Limit`, `Stop`, `StopLimit`,
`MarketIfTouched`), les trois formes de prix (`OffsetBps`, `Absolute`, `Signal`), le
time-in-force existant, le sizing optionnel au prix de l'ordre, la validation à la
compilation et le décompte des ordres non remplis. Une entrée conditionnelle reste hors du
kernel rapide : `fast_path_blocker` continue de la refuser et de le dire.
But : permettre à l'utilisateur de décrire **où** son entrée se remplit — pas seulement
« au close de la barre suivante » — sans casser la vitesse ni les résultats existants.
Contrainte cardinale : **zéro coût quand la fonctionnalité n'est pas utilisée**, et pas de
quatrième transcription de la règle de fill (cf. `refactor(engine): one fill/sizing rule
instead of three transcriptions`).
---
## 1. État des lieux
### Ce qui existe
`OrderConfig.limit_entry` (`crates/bt-core/src/orders.rs:27`) :
```rust
pub struct LimitOrderConfig {
pub offset_bps: f64, // distance depuis le close de la barre de SIGNAL
pub time_in_force: TimeInForce, // GTC | GTB(n) | IOC
}
```
Réellement branché, sur les deux boucles générales :
| Élément | Emplacement |
|---|---|
| Calcul du prix limite (close du signal, pas de look-ahead) | `orchestrator.rs:1628-1638`, `:2961-2969` |
| Gate de fill (`low <= limit` achat, `high >= limit` vente) | `orchestrator.rs:1679-1704` |
| TIF (IOC annule, GTB(n) expire, GTC persiste) | `orchestrator.rs:1689-1701` |
| Fill au prix limite, frais **maker**, slippage nul | `orchestrator.rs:1751-1762` |
| État de l'ordre au repos | `PendingOrder`, `orchestrator.rs:442-462` |
| Bracket SL/TP armé dès la première fraction remplie | `orchestrator.rs:1791-1800` |
| Tests | `crates/bt-core/tests/backtest_orders.rs:601`, `:647`, `:687` |
Exposition : dict brut via `ExecutionConfig.orders` (`config.py:24`) ou champ `orders` du
StrategyDef (`compiler.rs:24`), résolu par `effective_orders` (`orchestrator.rs:4578`) —
le per-stratégie écrase le global.
### Ce qui manque
1. **Aucun prix absolu ni piloté par une expression.** Seul « X bps sous le close du signal »
est exprimable. Pas de « achète à 60 000 », pas de « achète sur l'EMA20 / le VWAP /
`close - 2*atr(14)` / le low de la veille ».
2. **Pas de stop entry (breakout).** Le sens est codé en dur en passif
(`orchestrator.rs:1682-1686`).
3. **Pas de gestion de gap à l'entrée.** Le fill se fait au prix limite exact, sans regarder
l'open. Pour un *limit* c'est conservateur (on paye la limite alors que le gap aurait donné
mieux) — acceptable. Pour un *stop* ce serait faux et **optimiste** : la règle gap-aware
existe déjà côté sortie (`sim_fast_lite_core_single`, `orchestrator.rs:5931-5946`) et doit
être réutilisée telle quelle.
4. **Pas de méthode dans le builder Python.** `Strategy` a `.stop_loss()`, `.take_profit()`,
`.trailing_stop()` (`strategy.py:124-160`) mais rien côté entrée.
5. **`ExitReason::LimitExpiry` déclaré et jamais émis** (`orders.rs:124`) : un ordre annulé
disparaît sans trace. L'utilisateur ne peut pas savoir que sa stratégie n'a jamais été
remplie.
6. **Le sizing ignore le prix de fill.** `sanitize_target` reçoit le close de la barre courante
(`orchestrator.rs:1618-1625`) : en `FractionOfEquity`, une entrée limite à 2 % sous le close
achète 2 % de notionnel de trop.
### Le vrai coût : le fast path
`fast_path_blocker` refuse **toute** limit entry, sans condition :
```rust
// orchestrator.rs:4927
if orders.limit_entry.is_some() {
return Some("the strategy has a conditional entry order (limit / stop)");
}
```
Commentaire à l'appui (`orchestrator.rs:4612`) : *« A LIMIT ENTRY blocks regardless: it can rest
unfilled across bars, which the single-bar fill assumption here cannot express. »*
Conséquence : une entrée conditionnelle sort du kernel rapide **et** du GPU. Sur un sweep, c'est
le facteur ~25× qui disparaît. Si on rend la fonctionnalité attrayante sans toucher à ça, les
utilisateurs vont la mettre partout et perdre la vitesse qui est l'argument du produit.
Les trois niveaux d'exécution à garder en tête :
| Niveau | Fonction | Ordres au repos ? |
|---|---|---|
| Boucle générale (`run`, `run_lite_on_aligned`) | `orchestrator.rs:1357`, `:2154` | oui |
| Kernels rapides CPU | `simulate_fast:4995`, `simulate_fast_lite:5225`, `sim_fast_lite_core_single:5869` | non |
| Kernel CUDA (transpilé 1:1 du précédent) | `gpu_sweep.rs:1115` (`bracket_src`) | non |
---
## 2. Les possibilités (espace de conception)
### Axe A — déclencheur
| Mode | Sémantique | Frais | Statut |
|---|---|---|---|
| `Market` | fill à la barre d'exécution selon `execution_price` | taker + slippage | existe (défaut) |
| `Limit` | fill si le prix **touche** le niveau (passif) | maker, pas de slippage | existe |
| `Stop` | fill si le prix **franchit** le niveau (breakout) | taker + slippage + gap | **à faire** |
| `StopLimit` | déclenché au stop, rempli au mieux à la limite | maker | optionnel, phase ultérieure |
`Stop` est le symétrique exact du `check_stop` de sortie ; le code de trigger et de gap existe
déjà, il n'y a qu'à l'appeler côté entrée.
### Axe B — ancrage du prix
| Variante | Exemple | Coût runtime |
|---|---|---|
| `OffsetBps(f64)` | `-25 bps sous le close du signal` | nul (existe) |
| `Absolute(f64)` | `60_000.0` | nul (scalaire) |
| `Signal(String)` | `"entry_px"` avec `entry_px: "ema(close,20)"` | une slice `&[f64]` de plus |
`Signal` couvre tout le reste (ATR, VWAP, plus-haut de N barres, niveau de Fibonacci, prix
externe injecté en colonne exo) sans inventer de mini-langage : la série est déjà matérialisée
dans `symbol_envs` au moment où le sizing est évalué (`orchestrator.rs:1140-1175`), il suffit de
l'extraire comme on extrait `position_sizing`, de la passer dans `expand_to_fine_resolution` et
de la slicer par symbole comme `target_slices` (`orchestrator.rs:1314`).
`Absolute` est redondant avec `Signal` (`entry_px: "60000"`) mais reste utile : c'est un
scalaire, donc il ne bloque pas le GPU et ne coûte pas une série.
### Axe C — durée de vie
`GTC` / `GTB(n)` / `IOC` existent et suffisent. À ajouter seulement :
- annulation quand le signal s'inverse (aujourd'hui l'override existe mais n'est pas explicite,
`orchestrator.rs:1575-1594`) ;
- **observabilité** de l'annulation (voir §3 phase 2).
### Axe D — sizing
Deux comportements, à rendre explicite :
- `size_at_signal_close` (actuel, rétro-compatible) ;
- `size_at_fill_price` : la quantité est calculée sur le prix limite, ce que l'utilisateur
attend quasi toujours.
Défaut proposé : garder l'actuel pour ne rien casser, exposer l'option, la documenter.
---
## 3. Plan par phases
### Phase 0 — garde-fous (avant toute ligne de feature)
- `cargo bench -p bt-core` : figer `single_asset_50k`, `sweep_1000`, `multi_asset_10sym`
(méthodo min-médiane sur 3 passes, cf. `perf/BASELINE.md`).
- Test de non-régression « zéro coût » : une stratégie sans `orders` doit produire un résultat
**bit-à-bit** identique avant/après. Les tests de parité existent déjà
(`test_fast_lite_core_matches_simulate_fast_lite`, `orchestrator.rs:6844`) — les étendre.
### Phase 1 — modèle de prix d'entrée (boucle générale, CPU)
```rust
pub enum EntryPrice {
OffsetBps(f64),
Absolute(f64),
Signal(String),
}
pub enum EntryTrigger { Market, Limit, Stop }
pub struct EntryOrderConfig {
pub price: EntryPrice,
pub trigger: EntryTrigger,
pub time_in_force: TimeInForce,
pub size_at_fill_price: bool,
}
```
- Rétro-compatibilité serde : `{"offset_bps": 10.0, "time_in_force": "GTC"}` continue de
désérialiser en `Limit` + `OffsetBps` (alias serde + `#[serde(default)]`).
- Résolution du prix : une seule fonction `resolve_entry_price(...) -> f64`, appelée depuis les
**deux** boucles générales — pas deux transcriptions.
- Trigger `Stop` : réutiliser la règle gap-aware de la sortie, sans la recopier.
- Validation : `Signal(name)` inconnu → erreur dans `validate_strategy`
(`crates/bt-strategy/src/validate.rs`), pas un NaN silencieux à l'exécution.
- Tests : fill, non-fill, expiration, gap, short, multi-bar fill, `Signal` piloté par indicateur.
### Phase 2 — surface utilisateur
- `Strategy.limit_entry(...)`, `.stop_entry(...)`, `.market_entry(...)` (`strategy.py`), sur le
modèle exact de `.stop_loss()`.
- `OrderConfig` typé côté Python plutôt que dict brut (`config.py:10-56`).
- Émission de `ExitReason::LimitExpiry` : une entrée annulée doit être visible. Comme elle ne
produit pas de trade, l'exposer via un compteur (`orders_cancelled`) dans le résultat + un
warning quand le taux de non-remplissage dépasse un seuil — un backtest « parfait » qui n'a
jamais rempli est le piège n°1 des entrées limites.
- Doc : section dédiée dans `docs/strategy-authoring.md`, plus un exemple dans `examples/`.
### Phase 3 — perf : l'ordre au repos dans le kernel rapide
C'est la phase qui protège la vitesse. Même patron que `LiteBracket` (`orchestrator.rs:5751`) :
struct plate `Copy`, `NaN` = absent, pas d'enum, pas de branche quand la feature est absente.
```rust
#[derive(Clone, Copy, Default)]
struct LitePending {
limit_price: f64, // NaN = pas d'ordre au repos
remaining: f64,
bars_alive: f64,
is_buy: f64, // encodé en f64 pour rester transpilable
}
```
- Ajouter au `sim_fast_lite_core_single` en `Option<...>` comme `exits`, pour que le chemin
sans ordre reste byte-for-byte identique.
- Élargir `fast_path_blocker` : accepter `Limit`/`Stop` quand le prix est `OffsetBps` ou
`Absolute`. Garder le blocage pour `Signal` **tant que** la série n'est pas uploadée
(phase 3b), et surtout garder le message explicite — c'est ce qui dit à l'utilisateur ce
qu'il paye.
- CUDA : ajouter un `pending_src` sur le modèle de `bracket_src` (`gpu_sweep.rs:1115`) et étendre
`pack_cfg` (`gpu_sweep.rs:1084`). Ordre des opérations FP **strictement** préservé, sinon la
parité bit-à-bit saute.
- Tests de parité CPU/GPU étendus aux ordres au repos.
### Phase 3b (optionnelle) — prix d'entrée piloté par série sur GPU
Uploader la série `Signal(name)` comme une colonne de plus, au même titre que les closes. Ne le
faire que si la mesure de la phase 4 montre que `Signal` est le cas d'usage dominant.
### Phase 4 — mesure et documentation
- Re-bench, comparaison à la baseline de la phase 0, seuil de régression accepté : **0 %** sur
le chemin sans ordres, à documenter sur le chemin avec ordres.
- Consigner dans `perf/` comme les campagnes précédentes.
---
## 4. Risques
| Risque | Mitigation |
|---|---|
| Divergence FP CPU/CUDA (kernel transpilé à la main) | une seule règle partagée, ordre des opérations figé, tests de parité étendus |
| Régression silencieuse sur le chemin sans ordres | golden bit-à-bit en phase 0, `Option` partout |
| Utilisateur qui perd le fast path sans le savoir | message de `fast_path_blocker` explicite, déjà surfacé par le sweep GPU |
| Backtest flatteur parce que rien n'a été rempli | compteur d'annulations + warning (phase 2) |
| Sizing incohérent avec le prix de fill | option explicite, défaut inchangé |
## 5. Arbitrages à trancher
1. **Stop entry (breakout) dans le lot initial ?** Le code de trigger et de gap existe côté
sortie, le coût marginal est faible — recommandé oui.
2. **Phase 3 (kernel rapide + CUDA) maintenant ou après retour utilisateur ?** C'est la moitié
de l'effort. Sans elle, la fonctionnalité marche mais coûte le fast path.
3. **`size_at_fill_price` par défaut ?** Plus juste, mais change les résultats des stratégies
existantes utilisant `limit_entry`.
docs/strategy-authoring.md
+87 -7
View File
@@ -15,13 +15,14 @@ This guide describes how to define trading strategies using the manifoldbt Pytho
5. [Backtest Configuration](#backtest-configuration)
6. [Execution Model](#execution-model)
7. [Fee & Slippage Models](#fee--slippage-models)
8. [Orders (SL/TP/Trailing)](#orders-sltp-trailing)
9. [Cross-Asset References](#cross-asset-references)
10. [Dataset Auto-Resolution](#dataset-auto-resolution)
11. [Diagnostics](#diagnostics)
12. [Profiling](#profiling)
13. [Complete Examples](#complete-examples)
14. [Indicator Reference](#indicator-reference)
8. [Orders (SL/TP/Trailing)](#orders-sltptrailing)
9. [Entry Orders](#entry-orders)
10. [Cross-Asset References](#cross-asset-references)
11. [Dataset Auto-Resolution](#dataset-auto-resolution)
12. [Diagnostics](#diagnostics)
13. [Profiling](#profiling)
14. [Complete Examples](#complete-examples)
15. [Indicator Reference](#indicator-reference)
---
@@ -195,6 +196,11 @@ best = sweep.best("sharpe")
batch = mbt.run_sweep_lite(strategy, {"fast": range(5, 100), "slow": range(10, 500)}, config, store)
```
Grids this size need Pro. Community is capped at 256 backtests cumulatively per
Python session across all sweep/batch calls, and each sweep call waits 5 s
before starting; single `bt.run()` calls are never gated. See
`docs/sweep-combo-limit-plan.md`.
`run_sweep_lite` is optimized for large parameter grids (100k+ combos):
- Cartesian product expansion in Rust (no Python loop)
- Shared indicator cache (EMA(12) computed once, reused across combos)
@@ -312,6 +318,80 @@ strategy = (
---
## Entry Orders
By default an entry takes a market fill on the execution bar (see
[Execution Model](#execution-model)). Four order types let the entry rest at a
price instead:
| Builder method | Fills when | Fill price | Costs |
|---|---|---|---|
| `.limit_entry(...)` | price comes **to** the level | the level exactly | maker, no slippage |
| `.stop_entry(...)` | price breaks **through** the level | the level, or the open if the bar gapped through it | taker + slippage |
| `.market_if_touched(...)` | price comes **to** the level | the level | taker + slippage |
| `.stop_limit_entry(...)` | breaks through `stop`, then rests at `limit` | the limit | maker, no slippage |
### Where the level comes from
Every method takes exactly one of three price forms:
```python
.limit_entry(offset_bps=25) # 25 bps below the signal close (above, for a sell)
.limit_entry(price=60_000) # a fixed level
.limit_entry(signal="entry_px") # a level this strategy computes
```
`signal=` is the general form: name any signal the strategy defines and the
order rests on that series, read on the signal bar.
```python
from manifoldbt.indicators import atr, close, ema
trend = ema(close, 50)
entry_px = close - atr(14) # rest one ATR below the close
strategy = (
mbt.Strategy.create("pullback_entry")
.signal("trend", trend)
.signal("entry_px", entry_px) # named so the order can reference it
.size(mbt.when(close > trend, 1.0, 0.0))
.limit_entry(signal="entry_px", time_in_force={"GTB": 5})
.stop_loss(pct=3.0)
)
```
### Time in force
`"GTC"` (default, rests until filled or the signal changes), `{"GTB": n}`
(cancel after n bars), `"IOC"` (fill on the arrival bar or cancel).
### Two things to watch
**A resting entry can simply never fill.** A strategy whose entries never
trigger produces a flat equity curve with no drawdown, which reads as a clean
backtest. The engine counts unfilled entries and reports them:
```python
result = mbt.run_backtest(strategy, config)
for w in result.warnings:
print(w) # "N entry order(s) expired unfilled and M were still resting ..."
```
**Sizing uses the close, not the level.** In `FractionOfEquity` mode a target of
`1.0` is converted to units at the signal-bar close, so an entry resting 2% away
buys ~2% too much notional. `size_at_fill_price=True` sizes off the order's own
level instead. It is off by default because turning it on changes the results of
strategies written against the old behaviour.
### Cost
A conditional entry runs on the general simulation loop rather than the fast
kernel, so parameter sweeps over one are slower than sweeps over a market entry
and cannot use the GPU. `run_sweep` reports which setting took you off the fast
path.
---
## Cross-Asset References
Use `mbt.symbol_ref()` to reference another symbol's data in multi-asset strategies:
docs/sweep-combo-limit-plan.md
+177
View File
@@ -0,0 +1,177 @@
# Community sweep gating — shipped design
Two mechanisms, both in `crates/bt-python/src/lib.rs` around
`require_combo_limit`:
1. a **cumulative combo budget** per process (256), and
2. a **wall-clock rate gate** — 5 s between Community sweeps, serialised
machine-wide by an exclusive file lock.
Pro is untouched: the gate returns before both.
Measured effect on the bypass this exists to stop (one fresh interpreter per
slice, since the counter dies with the process):
| strategy | combos | Pro | bypass, sequential | bypass, parallel | friction |
|-------------------|--------|--------|--------------------|------------------|----------|
| RSI + EMA (light) | 5,000 | 5.9 s | 103 s | 95 s | 16.2x |
| RSI + EMA (light) | 20,000 | 23.3 s | 413 s | 390 s | 16.7x |
| SMA bands (heavy) | 5,000 | 10.0 s | 107 s | 95 s | 9.5x |
| SMA bands (heavy) | 20,000 | 40.0 s | 424 s | 391 s | 9.8x |
Friction is stable *within* a strategy (16.2 → 16.7, 9.5 → 9.8) and differs
*between* them: it no longer depends on grid size at all, only on how expensive
the strategy is per combination. At the previous 500-combo cap the same four
cells read 7.7x / 8.7x / 5.1x / 5.2x — halving the cap roughly doubled them, as
the model predicts.
Every cell read **1.0x** before this work: slicing a grid across processes cost
the same as one big licensed call.
## The hole
The Community limit (`COMMUNITY_MAX_SWEEP_COMBOS`, 500 at the time) used to be
checked **per call**. A parameter grid is sliceable, so splitting it bypassed
the cap:
```python
for i in range(0, 500, 5): # 100 calls x 500 combos
bt.run_sweep_lite(strat, {"dev_up": DU[i:i+5], "dev_dn": DD}, cf, store)
```
| run | time | per combo |
|------------------------------------|--------|-----------|
| 1 call x 50,000 combos (Pro) | 89.9 s | 1.80 ms |
| 100 calls x 500 combos (Community) | 90.3 s | 1.81 ms |
The split cost **+0.5%**, and per-call overhead is ~10 ms once data is cached,
so no per-call charge could make slicing expensive without hitting legitimate
small sweeps first.
## 1. Cumulative combo budget
A process-lifetime `AtomicU64`. Non-Pro calls are refused when
`total + n_combos > COMMUNITY_MAX_SWEEP_COMBOS`; otherwise the total is
CAS-incremented — a plain load+store would let two concurrent sweeps both slip
under the cap. A refused call consumes no budget and pays no wait. Pro skips
check and accounting. `_native._combo_budget()` exposes `(used, limit, is_pro)`
read-only.
All native fan-out entry points share one total: `run_sweep`, `run_sweep_lite`,
`run_batch`, `run_batch_lite`, `run_sweep_2d`, `run_stability`. A single
`bt.run()` is ungated.
The Python-side `_require_pro_over_combos` stays a fast-fail courtesy check for
the single-call case; the authoritative gate is native, since the Python layer
is trivially patchable.
This alone leaves the restart bypass wide open — the counter dies with the
process — which is what mechanism 2 prices.
## 2. Rate gate: 5 s, serialised by a file lock
Before each accepted Community call, the gate waits `SWEEP_MIN_INTERVAL` (5 s)
while holding an exclusive lock on `{license_dir}/sweep.lock`.
**The lock is the mechanism, not plumbing.** Waiting costs no cores, so N
concurrent processes would otherwise all clear the same 5 s — the way every
sleep-based limiter dies. Serialising the wait behind one machine-wide lock
makes N slices cost N x 5 s whatever the core count. The measurements confirm
it exactly: 19 slices in parallel took 95 s and 78 took 390 s, against a
theoretical floor of 95 s and 390 s. Parallelising buys nothing.
**Stateless on purpose.** An earlier shape recorded "last sweep at T" and only
waited the remainder — friendlier, since an idle user pays nothing. But the
record is the attack surface: restoring a copy from a minute ago grants an
immediate pass, and a MAC prevents forgery, not replay (a *stale* restored file
is the permissive one, so replay always favours the attacker). Here the file
carries no data, only the lock. Nothing to roll back; deleting it merely makes
the next caller recreate it, and the wait still happens.
**Fails closed on an unwritable state directory** (read-only HOME, locked-down
CI): the wait still happens, only without cross-process serialisation. Failing
open would make an unwritable HOME the bypass.
**Paid up front**, before results exist, so killing the process to dodge the
wait also discards the run. The GIL is released throughout.
Implemented with `std::fs::File::lock` — stable since Rust 1.89, so no new
dependency, but that is the minimum toolchain for this crate now.
### Why not a CPU penalty (what this replaced)
The previous mechanism burned ~3 s of counted CPU per process. It worked
(3.7x–6.2x) but had three defects the rate gate does not:
- **Cost in core-seconds, not seconds.** Friction was
`1 + fixed_cost / (Pro cost of the slice)`, so the same penalty bought 6.2x on
a light strategy, 3.9x on a heavy one, and would fall to ~1.2x on 3 years of
bars. The rate gate's floor is wall-clock, so strategy weight and dataset size
drop out.
- **Machine-dependent**, needing a runtime calibration and per-thread clamps to
stay near 3 s across CPUs. The rate gate needs neither.
- **Wasteful and visible.** Burning every core for 3 s is indefensible if a
user profiles it. Waiting costs nothing and leaves the machine usable.
## Tuning
Behaviour is now predictable:
```
friction ≈ (combos / cap) x 5 s / (Pro time for those combos)
```
The cap is therefore the lever, and its effect is calculable rather than
measured: halving it roughly doubles the friction, at the cost of a smaller free
allowance. Verified — moving from 500 to 256 took the four cells above from
5.1x–8.7x to 9.5x–16.7x, a 1.9x shift against a predicted 1.95x.
Current settings: `COMMUNITY_MAX_SWEEP_COMBOS = 256`, `SWEEP_MIN_INTERVAL = 5 s`.
A legitimate free user pays 5 s per sweep, capped at 256 combinations per
process — still a 16x16 grid.
## Rejected alternatives
- **Disk-persisted counter, or a required "ticket" file.** Deleted with `rm`,
or restored with `cp`; a hash prevents forgery, not replay.
- **Shared memory / sentinel process.** One `pkill` away, and a library
spawning hidden background processes is an antivirus profile on Windows.
- **Delay measured from a recorded last-sweep time.** Needs history, which is
restorable — see above. The stateless always-wait shape avoids it.
- **Plain sleep without the lock.** Overlaps for free across processes; this is
the single reason the lock exists.
- **Penalty scaled by combos, or by combos x dataset span.** Measured and
dropped: a legitimate 500-combo sweep reached 3 s and 4.4 s respectively.
- **`run_sweep_lite` restricted to Pro.** Would work — capability cannot be
sliced around — but turned down on positioning: the free tier should be
capped in *how much*, not degraded in *how well*.
## Known limits
- **The clock is the user's.** `faketime`/`LD_PRELOAD` can shorten the sleep,
and per-slice mount namespaces (`unshare`) give each process its own lock
file. Both are deliberate technical acts, unlike "restart Python".
- **A patched wheel defeats any client-side check.** This is the ceiling of the
whole approach; the goal is to make bypassing cost more than a subscription,
not to make it impossible.
- Escalation path if telemetry ever shows this is not enough:
`docs/sweep-key-rate-limit-plan.md`.
## Related findings (tracked separately)
1. `run_sweep` (full Arrow output) is OOM-killed around ~6.8 MB/combo: 2,000
combos ≈ 13.4 GB RSS, 3,000 → SIGKILL, no Python exception, no warning. Only
Pro users can reach it, since the Community cap keeps grids below the danger
zone — a paying-customer-only failure. Needs an upfront memory estimate and
a clean error pointing at `run_sweep_lite`.
2. Level-fill via `execution_source` regenerates the execution bar feed per
combination (~80x slower sweeps); an expression-based execution price would
move the level back into parameter space. Separate design.
## Measurement note
Per-combo cost is flat across grid size — 1.21 / 1.13 / 1.11 / 1.17 / 1.11
ms/combo at 500 / 2k / 10k / 50k / 200k on one strategy. An earlier revision of
this document claimed the big call degraded at scale; that was wrong, and came
from comparing grids with different *content*. Per-combo cost tracks trade
count, not grid size — which is also why friction varies by strategy.
docs/sweep-key-rate-limit-plan.md
+247
View File
@@ -0,0 +1,247 @@
# Server-issued sweep keys — escalation plan
Status: **specified, deliberately not built.** The shipped local rate gate
(`docs/sweep-combo-limit-plan.md`) measures 9.5x–16.7x at a 256-combo cap, and
lowering the cap further scales it linearly — the level this design was meant to
buy,
without a server, without making the free tier online-only, and without a
privacy review.
Build this **only** if telemetry shows the local gate being defeated at scale.
What would justify it: evidence of `faketime`/`LD_PRELOAD` clock manipulation
or per-slice namespaces in the wild, i.e. the two attacks the local gate cannot
answer. Nothing else here is worth the cost.
The rest of the document is the spec, kept complete so the decision can be
revisited without redoing the analysis.
## 1. What it does
A sweep must present a valid, unexpired, server-signed key. The key carries a
**budget of combinations**, not a single authorisation, so a burst of small
sweeps shares one round-trip.
| tier | rate | combos per key | TTL | round-trips |
|-----------|-----------------|----------------|-------|-----------------------|
| Community | 1 key per 5 s | 100 | 30 s | 1 per 100 combos |
| Pro | none | unlimited | 24 h | 1 per day |
Expected effect, from the measured Pro baselines (3 months of 1m bars, 4 cores):
| combos | Pro | Community floor | friction |
|--------|--------|---------------------|----------|
| 5,000 | 8.0 s | 50 keys → 4.2 min | ~31x |
| 20,000 | 26.6 s | 200 keys → 16.7 min | ~38x |
These hold on any machine, strategy and dataset size — but so does the shipped
local rate gate, for the same reason (a wall-clock floor rather than CPU work).
The server's only remaining advantage is that its clock is out of the user's
reach: no `faketime`, no namespace trick. That is the entire delta now.
## 2. Why the server, and why no offline lane
Every client-side variant was built or analysed and fails the same two ways
(measurements in the companion doc):
- **State dies with the process.** The cumulative counter resets on restart. On
disk it is one `rm` away; as a required "ticket" file it is one `cp` away —
a MAC prevents forgery, not replay, and a *stale* restored file is more
permissive, so replay always favours the attacker.
- **A fixed client *CPU* cost buys a variable ratio.** Friction was
`1 + fixed_cost / (Pro cost of the slice)`, so the CPU penalty gave 6.2x on a
light strategy and 3.9x on a heavy one. (The lock-serialised wall-clock gate
that replaced it does not have this defect — it is why this plan is no longer
urgent.)
**Why no offline lane.** "Free users get N combos offline, the key is only
needed beyond that" sounds like a friendly compromise and is worth nothing: the
offline allowance resets on restart, so an abuser stays in the offline lane
forever and never requests a key. Any tolerated offline mode is a hole exactly
the size of the allowance. The choice is binary — accept the network
requirement, or keep the local gate's 9.5x–16.7x.
## 3. Protocol
```
client server
|-- POST /api/sweep-key ---------------->| { device_hash, license_hash?,
| | version, requested_combos }
| | identify -> rate + quota check
|<---------- signed key -----------------| { combos, expires_at, device_hash,
| | nonce, tier, sig }
| verify sig (embedded public key) |
| spend from the budget as sweeps run |
```
**Key payload** (JSON, serialised canonically — reuse the existing licence rule
that payloads re-serialise byte-identically so signatures verify):
```json
{ "combos": 100, "expires_at": "2026-08-11T09:31:04.000Z",
"device_hash": "…", "nonce": "…", "tier": "community" }
```
Signed Ed25519, verified against a public key compiled into the wheel. Reuse
`bt-license`: `ed25519-dalek` is already a dependency, `license.rs` already
verifies this exact shape, and `PUBLIC_KEY_BYTES` is already stored XOR-masked
and split rather than as a contiguous blob. **Use a separate key pair from the
licence one** so a leaked sweep-signing key cannot mint licences.
`device_hash` is inside the signature, so a key copied to another machine is
rejected locally, with no round-trip.
### Why the two TTLs differ
The 5 s rhythm is **not** enforced by the TTL — the server refuses to mint
before 5 s have passed, which is server state. The short Community TTL exists
to prevent **stockpiling**: with a one-hour TTL an abuser requests a key every
5 s for an hour, banks 720 keys, then burns them in parallel and gets 72,000
combinations at once. A 30 s TTL makes an unused key worthless while still
covering a burst of small sweeps.
Pro's 24 h TTL is the opposite trade: one round-trip, then a full day offline.
Its exposure is key sharing, already covered by the device binding in the
signature plus the existing per-licence device limit.
### Rate limiting caps throughput, not volume
At one key per 5 s a patient Community user still reaches ~72,000 combinations
per hour. If the intent is a ceiling rather than a drip, the server must also
hold a **daily total per account**. Recommended: yes, with a generous limit
(e.g. 5,000/day) — it costs one counter and turns "slow" into "bounded".
## 4. Client implementation
All in `crates/bt-license` (transport, crypto, state) and
`crates/bt-python/src/lib.rs` (the gate).
**4.1 — `bt-license`: new `sweep_key` module**
- `pub fn acquire(requested: u32) -> Result<SweepKey, LicenseError>`
- Returns the process-cached key if unexpired with budget left.
- Otherwise `POST /api/sweep-key`, verify signature, verify `device_hash`
matches `device::device_hash()`, cache, return.
- **Cache in memory only, never on disk.** An on-disk key is a stockpile, and
the TTL is short enough that persistence buys nothing.
- Reuse the telemetry HTTP pattern (`reqwest` + a current-thread tokio runtime),
but **blocking** with a **500 ms timeout** — a library that freezes on its
first call gets uninstalled. Distinguish the three failures explicitly:
network unreachable, rate-limited (HTTP 429 + `retry_after`), quota exhausted
(HTTP 402).
- Clock skew: trust `expires_at` against local time but also keep a local
monotonic deadline from receipt; use whichever expires first, so a client
clock set backwards cannot extend a key.
**4.2 — `bt-python`: replace the gate**
`require_combo_limit(py, n_combos, label)` becomes `require_sweep_budget`:
1. Pro (`check_feature("sweep")`) → acquire/refresh the 24 h key, return.
2. Community → `sweep_key::acquire(n_combos)`; on success debit the budget, on
failure raise `PermissionError` with the server's reason.
3. Keep the in-process cumulative counter as a **local fast-fail** so an
over-budget call fails without a round-trip.
4. Release the GIL around the request (`py.allow_threads`).
Call sites are unchanged: `run_sweep`, `run_sweep_lite`, `run_batch`,
`run_batch_lite`, `run_sweep_2d`, `run_stability`. A single `bt.run()` stays
ungated.
**4.3 — Error messages.** Three distinct, actionable texts:
- rate-limited → `"Community sweeps are limited to one every 5 s (4 s
remaining). Upgrade at …"`
- daily quota → `"Community daily sweep quota reached (5,000 combinations).
Resets at 00:00 UTC. Upgrade at …"`
- offline → `"Sweeps require a connection to www.manifoldbt.com on the
Community tier. bt.run() and single backtests work offline."`
**4.4 — Remove** `community_rate_gate`, `SWEEP_MIN_INTERVAL` and the lock file.
The server rhythm replaces the local one; running both would charge the wait
twice.
## 5. Server implementation
- `POST /api/sweep-key` — authenticate (licence hash when present, device hash
otherwise), enforce rate + daily quota, sign, return. Reject unknown or
revoked licences.
- Storage: per-device `last_issued_at` and per-account `combos_today`. A single
small table; the rate check is one read + one write.
- **Rate-limit by IP as well as by device.** Rotating `device_hash` to mint a
fresh identity is the one bypass this design does not close; per-IP limits
and requiring an account for Community keys raise the bar substantially.
- Metrics to emit from day one: keys issued, refusals by reason, distinct
devices per account, combos per account per day. This is also the first real
usage data on the free tier.
- Key rotation: publish the sweep-signing public key with a version byte in the
payload so a future rotation does not break older wheels.
## 6. Rollout
1. **Ship telemetry first.** Add sweep-usage reporting to the current wheel and
watch for a release. If the data shows nobody is chaining sweeps, stop here
and keep the client-side gating — this design is not free.
2. **Server first, enforcement later.** Deploy the endpoint and let the client
request keys *without* enforcing, logging refusals it would have made.
Confirms capacity and false-positive rate against real usage.
3. **Enforce behind a version gate.** Only wheels ≥ the release that ships this
require a key; older wheels keep working, so the change cannot brick an
existing install.
4. **Announce before enforcing.** Community losing offline sweeps is a
user-visible regression and must be in the release notes, not discovered.
## 7. Non-technical prerequisites
- **Privacy.** Unlicensed installs will contact the server for the first time —
the current telemetry ping only fires when a licence is present
(`guard.rs: if let Some(ref lic) = license`). Needs a privacy notice, a
documented retention policy, and a GDPR review. `device_hash` is a salted
hash of a machine ID: pseudonymous, not anonymous.
- **Availability becomes a product dependency.** If the endpoint is down, no
Community user can sweep. Needs an uptime target, and a documented
fallback decision (fail-closed is what makes the design hold; a fail-open
switch is a bypass anyone can trigger by blocking a domain).
## 8. Attack surface after this lands
| attack | cost after |
|-------------------------------|-----------|
| loop inside one process | blocked by the key budget |
| fresh process per slice | useless — the server clock does not reset |
| copy the key file | no key file exists; in-memory only |
| replay a captured key | 30 s TTL, bound to the device |
| forge a key | Ed25519 |
| stockpile keys | TTL shorter than the accumulation window |
| parallelise | useless — the rate is per account, not per process |
| rotate `device_hash` | **works** — mitigated by per-IP limits and account requirement, not closed |
| block the network | fails closed: no sweep |
| patch the wheel | **works** — the ceiling of any client-side check |
The last two lines are the honest limit: this makes bypassing a deliberate,
technical act rather than a side effect of restarting Python.
## 9. Tests
- Community: two sweeps back to back — the second refused until 5 s elapse;
101 combinations in one call refused outright; a burst of five 20-combo
sweeps consumes one key, not five.
- Key expiry: a key held past its TTL is refused and transparently renewed.
- Signature: edited payload rejected; valid key minted for another
`device_hash` rejected.
- Clock: local clock moved backwards does not extend a key (monotonic deadline
wins).
- Pro: one key covers a 200,000-combo sweep; no rate limit; no penalty.
- Server unreachable: fails within the timeout with the offline message, never
hangs.
- Restart: a fresh process cannot obtain a key faster than the server rhythm —
the end-to-end property the whole design exists for. Same harness as the
400-slice bypass benchmark used for the CPU penalty.
## 10. Still to decide
1. Daily quota: on or off, and at what value.
2. Rate: 5 s is a guess. It sets the free tier's usable throughput; pick it
from the telemetry in step 6.1 rather than from intuition.
3. Whether Community keys require a registered account (email) or just a
device hash. Accounts make per-IP limits far more effective, at the cost of
a signup wall in front of a free tier.