Compare commits

..

20 Commits

Author SHA1 Message Date
github-actions[bot] 5a6948d321 chore(master): release 2.2.0 (#19) 2026-04-18 12:51:41 +02:00
TPTBusiness c62d9c4efa fix(kronos): replace rdagent_logger with stdlib logging for CI compatibility 2026-04-18 12:24:47 +02:00
TPTBusiness 1a1bd2c712 feat(fin_quant): auto-generate Kronos factor before loop start
Adds _ensure_kronos_factor_in_pool() which runs automatically at the
start of every fin_quant / predix quant invocation. If the Kronos
factor is not yet in results/factors/values/, it generates it
(stride=500, batch=32 GPU) and computes IC via evaluate_kronos_model.
Writes a StrategyOrchestrator-compatible JSON so the factor is
immediately available to strategy generation without manual steps.
Is a no-op when the factor already exists with a valid IC.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 12:18:51 +02:00
TPTBusiness 999bbac08d perf(kronos): batch GPU inference via predict_batch — 75x faster
Replace sequential predict() calls with predict_batch() in both
build_kronos_factor and evaluate_kronos_model. Up to batch_size windows
processed simultaneously on GPU, reducing per-window time from ~10s to
~0.13s (10 windows in 1.3s on RTX 5060 Ti, 75x speedup).

Adds --batch-size / -b option (default 32) to both kronos-factor and
kronos-eval CLI commands. Falls back to single inference per window if a
batch fails. Refactors timestamp prep into _build_window_inputs helper.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 12:10:30 +02:00
TPTBusiness ea87495a6a perf(kronos): batch GPU inference via predict_batch — 75x faster
Replace sequential predict() calls with predict_batch() in both
build_kronos_factor and evaluate_kronos_model. Up to batch_size windows
are processed simultaneously on GPU, reducing per-window time from ~10s
to ~0.13s (measured: 10 windows in 1.3s on RTX 5060 Ti).

Adds --batch-size / -b option (default 32) to both kronos-factor and
kronos-eval CLI commands. Falls back to single inference per window if
a batch fails. Refactors timestamp preparation into _build_window_inputs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 12:04:44 +02:00
TPTBusiness 3a1adf7e0a fix(kronos): pass actual datetime Series to Kronos predictor timestamps
KronosPredictor.predict() requires x_timestamp and y_timestamp to be
pandas Series of datetime values for its calc_time_stamps() helper.
Previously we passed integer ranges (after reset_index), which raised
AttributeError on .dt.minute. Fixed by extracting datetime index values
before resetting and using future_idx for y_timestamp.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 10:12:51 +02:00
TPTBusiness e819dcc3b9 fix(kronos): lazy torch import to fix CI ModuleNotFoundError
Move top-level `import torch` into _cuda_available() helper so
kronos_adapter.py can be imported in CI environments without torch.
All device defaults resolved at runtime via lazy detection.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 10:00:45 +02:00
TPTBusiness 016eed7df7 feat: add Kronos CLI commands, expand tests, document in README
- predix kronos-factor: generate KronosPredReturn alpha factor via CLI
- predix kronos-eval: evaluate Kronos IC/hit-rate vs LightGBM via CLI
- 19 tests covering adapter, factor builder, model evaluator, CLI (mock-based)
- README: Kronos section in Features + CLI commands table
- Total test suite: 153 passed

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 09:53:56 +02:00
TPTBusiness 3f3a23bd38 feat: integrate Kronos-mini OHLCV foundation model (Option A + B)
Add Kronos-mini (4.1M params, AAAI 2026, MIT) as:
- Option A: predicted-return alpha factor via rolling daily inference
  (kronos_factor_gen.py — stride=96 bars/day, ~2k inference calls)
- Option B: standalone model evaluator alongside LightGBM
  (kronos_model_eval.py — IC / hit-rate vs actual realized returns)

KronosAdapter wraps NeoQuasar/Kronos-mini + Kronos-Tokenizer-2k,
auto-detects GPU, gracefully degrades if ~/Kronos repo is missing.
Factor output: MultiIndex (datetime, instrument) with KronosPredReturn.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 09:49:25 +02:00
dependabot[bot] 6868002beb chore(deps): Bump azure-identity from 1.17.1 to 1.25.3 (#15)
Bumps [azure-identity](https://github.com/Azure/azure-sdk-for-python) from 1.17.1 to 1.25.3.
- [Release notes](https://github.com/Azure/azure-sdk-for-python/releases)
- [Commits](https://github.com/Azure/azure-sdk-for-python/compare/azure-identity_1.17.1...azure-identity_1.25.3)

---
updated-dependencies:
- dependency-name: azure-identity
  dependency-version: 1.25.3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-18 09:16:20 +02:00
dependabot[bot] 0302fbe66a chore(deps): Bump psutil from 6.1.0 to 6.1.1 (#17)
Bumps [psutil](https://github.com/giampaolo/psutil) from 6.1.0 to 6.1.1.
- [Changelog](https://github.com/giampaolo/psutil/blob/master/docs/changelog.rst)
- [Commits](https://github.com/giampaolo/psutil/compare/v6.1.0...v6.1.1)

---
updated-dependencies:
- dependency-name: psutil
  dependency-version: 6.1.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-18 09:16:18 +02:00
TPTBusiness 89ef9fb33d docs: fix duplicate sections, add hardware requirements and data setup guide
- Remove duplicate Configuration and CLI Commands sections
- Add System Requirements table (GPU VRAM, RAM, CUDA)
- Expand Data Setup with concrete step-by-step instructions
- Add prerequisites checklist to Quick Start (Docker, data, LLM health)
- Consolidate all CLI commands into one section

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 09:16:02 +02:00
dependabot[bot] c533a0346d chore(deps): Update snowballstemmer requirement from <3.0 to <4.0 (#16)
Updates the requirements on [snowballstemmer](https://github.com/snowballstem/snowball) to permit the latest version.
- [Changelog](https://github.com/snowballstem/snowball/blob/master/NEWS)
- [Commits](https://github.com/snowballstem/snowball/compare/v2.0.0...v3.0.1)

---
updated-dependencies:
- dependency-name: snowballstemmer
  dependency-version: 3.0.1
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-18 09:09:39 +02:00
dependabot[bot] d91263a70c chore(deps): Bump scipy from 1.14.1 to 1.15.3 (#14)
Bumps [scipy](https://github.com/scipy/scipy) from 1.14.1 to 1.15.3.
- [Release notes](https://github.com/scipy/scipy/releases)
- [Commits](https://github.com/scipy/scipy/compare/v1.14.1...v1.15.3)

---
updated-dependencies:
- dependency-name: scipy
  dependency-version: 1.15.3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-18 09:09:36 +02:00
dependabot[bot] e650b01da0 chore(deps): Bump dill from 0.3.9 to 0.4.1 (#13)
Bumps [dill](https://github.com/uqfoundation/dill) from 0.3.9 to 0.4.1.
- [Release notes](https://github.com/uqfoundation/dill/releases)
- [Commits](https://github.com/uqfoundation/dill/compare/0.3.9...0.4.1)

---
updated-dependencies:
- dependency-name: dill
  dependency-version: 0.4.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-18 09:09:33 +02:00
dependabot[bot] 670585f640 chore(deps): Bump github/codeql-action from 3 to 4 (#12)
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3 to 4.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-18 09:09:31 +02:00
dependabot[bot] bdb1d03357 chore(deps): Bump actions/deploy-pages from 4 to 5 (#11)
Bumps [actions/deploy-pages](https://github.com/actions/deploy-pages) from 4 to 5.
- [Release notes](https://github.com/actions/deploy-pages/releases)
- [Commits](https://github.com/actions/deploy-pages/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/deploy-pages
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-18 09:09:29 +02:00
dependabot[bot] 8433795a4a chore(deps): Bump actions/cache from 4 to 5 (#10)
Bumps [actions/cache](https://github.com/actions/cache) from 4 to 5.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-18 09:09:26 +02:00
dependabot[bot] 210c4e8ad3 chore(deps): Bump codecov/codecov-action from 4 to 6 (#9)
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4 to 6.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v4...v6)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-18 09:09:23 +02:00
dependabot[bot] 9e68c761b8 chore(deps): Bump actions/upload-artifact from 4 to 7 (#8)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4 to 7.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4...v7)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-18 09:09:20 +02:00
17 changed files with 1340 additions and 225 deletions
+1 -1
View File
@@ -43,7 +43,7 @@ jobs:
pytest test/backtesting/ -v --tb=short
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v4
uses: codecov/codecov-action@v6
with:
token: ${{ secrets.CODECOV_TOKEN }}
fail_ci_if_error: false
+1 -1
View File
@@ -56,6 +56,6 @@ jobs:
# Upload the SARIF file generated in the previous step
- name: Upload SARIF results file
uses: github/codeql-action/upload-sarif@v3
uses: github/codeql-action/upload-sarif@v4
with:
sarif_file: results.sarif
+2 -2
View File
@@ -33,7 +33,7 @@ jobs:
python-version: "3.10"
- name: Cache pip dependencies
uses: actions/cache@v4
uses: actions/cache@v5
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-docs-${{ hashFiles('**/pyproject.toml') }}
@@ -83,4 +83,4 @@ jobs:
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
uses: actions/deploy-pages@v5
+1 -1
View File
@@ -24,7 +24,7 @@ jobs:
python-version: "3.10"
- name: Cache pip dependencies
uses: actions/cache@v4
uses: actions/cache@v5
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-lint-${{ hashFiles('**/pyproject.toml') }}
+1 -1
View File
@@ -38,7 +38,7 @@ jobs:
- name: Upload results on failure
if: failure()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: test-results-py${{ matrix.python-version }}
path: |
+2 -2
View File
@@ -27,7 +27,7 @@ jobs:
python-version: "3.10"
- name: Cache pip dependencies
uses: actions/cache@v4
uses: actions/cache@v5
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-security-${{ hashFiles('**/requirements.txt') }}
@@ -53,7 +53,7 @@ jobs:
bandit -c .bandit.yml -r rdagent/ -ll || true
- name: Upload Bandit report
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
if: always()
with:
name: bandit-security-report
+27
View File
@@ -1,5 +1,32 @@
# Changelog
## [2.2.0](https://github.com/TPTBusiness/Predix/compare/v2.1.0...v2.2.0) (2026-04-18)
### Features
* add Kronos CLI commands, expand tests, document in README ([f911081](https://github.com/TPTBusiness/Predix/commit/f911081d1763d0dc4dd790b57dd97aae2dc62679))
* **fin_quant:** auto-generate Kronos factor before loop start ([277063f](https://github.com/TPTBusiness/Predix/commit/277063f3e36cd071db859cdc77f69135c1f0763b))
* integrate Kronos-mini OHLCV foundation model (Option A + B) ([4ae3b99](https://github.com/TPTBusiness/Predix/commit/4ae3b99f2450930f72e202a1a470c407bfde3328))
### Bug Fixes
* **kronos:** lazy torch import to fix CI ModuleNotFoundError ([ccc1d27](https://github.com/TPTBusiness/Predix/commit/ccc1d27dbe5ab06a57085a589d456ac7bf49cc08))
* **kronos:** pass actual datetime Series to Kronos predictor timestamps ([dc6e7ce](https://github.com/TPTBusiness/Predix/commit/dc6e7ce207d21fbc21976f2af7691058530fac2f))
* **kronos:** replace rdagent_logger with stdlib logging for CI compatibility ([b4558f2](https://github.com/TPTBusiness/Predix/commit/b4558f2456659c6109bd1b3cf100510491cd3e6c))
### Performance Improvements
* **kronos:** batch GPU inference via predict_batch — 75x faster ([74611d0](https://github.com/TPTBusiness/Predix/commit/74611d071ac123a655eb15d0737bb73b8c1bd2b0))
* **kronos:** batch GPU inference via predict_batch — 75x faster ([2babeb9](https://github.com/TPTBusiness/Predix/commit/2babeb95f42828e13a37dc16166c75538f33fd4b))
### Documentation
* fix duplicate sections, add hardware requirements and data setup guide ([6c771b3](https://github.com/TPTBusiness/Predix/commit/6c771b37e6f88526a896499e86929cfca2c199eb))
## [2.1.0](https://github.com/TPTBusiness/Predix/compare/v2.0.0...v2.1.0) (2026-04-18)
+215 -208
View File
@@ -20,6 +20,7 @@
<p align="center">
<a href="#installation">Installation</a> •
<a href="#no-gpu-use-openrouter">No GPU?</a> •
<a href="#quick-start">Quick Start</a> •
<a href="#configuration">Configuration</a> •
<a href="#features">Features</a>
@@ -101,11 +102,25 @@ All code in Predix is originally written and implemented independently. Predix e
## Installation
### System Requirements
| Component | Minimum | Recommended |
|-----------|---------|-------------|
| **GPU VRAM** | 8 GB | 16 GB (RTX 4080 / 5060 Ti) |
| **RAM** | 16 GB | 32 GB |
| **Storage** | 20 GB | 50 GB (models + data) |
| **OS** | Linux (Ubuntu 22.04+) | Linux |
| **CUDA** | 12.0+ | 12.4+ |
> Local LLMs require a CUDA-capable GPU. The default model (Qwen3.6-35B Q3) uses ~13.6 GB VRAM. CPU-only inference is possible but very slow (not recommended for production use).
### Prerequisites
- **Conda** (Miniconda or Anaconda) - Required for environment management
- **Docker** (required for sandboxed code execution)
- **Linux** (officially supported; macOS/Windows may work with adjustments)
- **Conda** (Miniconda or Anaconda) — required for environment management
- **Docker** required for sandboxed factor/model code execution (`docker run hello-world` to verify)
- **llama.cpp** — for local LLM inference (see [llama.cpp build guide](https://github.com/ggml-org/llama.cpp))
- **Ollama** — for embeddings (`nomic-embed-text`); install from [ollama.com](https://ollama.com) and run `ollama pull nomic-embed-text`
- **Linux** — officially supported; macOS/Windows may work with adjustments
### Quick Install
@@ -120,14 +135,71 @@ conda activate predix
# Install in editable mode
pip install -e .
# Verify Docker is accessible
docker run --rm hello-world
```
> **Important:** Predix requires a conda environment to manage dependencies properly.
> Using plain Python or other environment managers may cause conflicts.
### Configuration
---
## Data Setup
Predix requires **1-minute EUR/USD OHLCV data** in HDF5 format. This is a hard prerequisite — the system cannot run without it.
### Step 1: Get the data
Download 1-minute EUR/USD data (2020present) from any of these free sources:
| Source | Cost | Notes |
|--------|------|-------|
| **[Dukascopy](https://www.dukascopy.com/swiss/english/marketfeed/historical/)** | Free | Best quality free EUR/USD tick data |
| **[OANDA API](https://developer.oanda.com/)** | Free (demo) | Requires API key, programmatic access |
| **[TrueFX](https://truefx.com/)** | Free | Institutional-quality tick data |
| **[Kaggle](https://www.kaggle.com/datasets?search=EURUSD+1min)** | Free | Search "EURUSD 1 minute" |
| **MetaTrader 5** | Free | Export via `copy_rates_range()` |
### Step 2: Convert to HDF5
```python
import pandas as pd
df = pd.read_csv('eurusd_1min.csv', parse_dates=['datetime'])
df = df.rename(columns={'open': '$open', 'close': '$close',
'high': '$high', 'low': '$low', 'volume': '$volume'})
df['instrument'] = 'EURUSD'
df = df.set_index(['datetime', 'instrument'])
for col in ['$open', '$close', '$high', '$low', '$volume']:
df[col] = df[col].astype('float32')
import os
os.makedirs('git_ignore_folder/factor_implementation_source_data', exist_ok=True)
df.to_hdf('git_ignore_folder/factor_implementation_source_data/intraday_pv.h5', key='data', mode='w')
```
### Required HDF5 format
| Field | Type | Description |
|-------|------|-------------|
| **Index** | MultiIndex `(datetime, instrument)` | Timestamp + currency pair |
| **`$open`** | float32 | Open price |
| **`$close`** | float32 | Close price |
| **`$high`** | float32 | High price |
| **`$low`** | float32 | Low price |
| **`$volume`** | float32 | Tick volume |
**Save location:** `git_ignore_folder/factor_implementation_source_data/intraday_pv.h5`
---
## Configuration
### Environment Setup
Create a `.env` file in the project root:
1. **Create `.env` file:**
```bash
# Local LLM (llama.cpp)
OPENAI_API_KEY=local
@@ -143,7 +215,8 @@ EMBEDDING_MODEL=nomic-embed-text
QLIB_DATA_DIR=~/.qlib/qlib_data/eurusd_1min_data
```
2. **Start LLM server (llama.cpp):**
### LLM Server (llama.cpp)
```bash
~/llama.cpp/build/bin/llama-server \
--model ~/models/qwen3.6/Qwen3.6-35B-A3B-UD-Q3_K_XL.gguf \
@@ -158,56 +231,110 @@ QLIB_DATA_DIR=~/.qlib/qlib_data/eurusd_1min_data
--reasoning off
```
> **Important flags and token budget:**
> - `--ctx-size 240000 --parallel 2` — allocates **2 slots × 120,000 tokens each**. `fin_quant` prompts can reach 80k+ tokens with full factor history; a smaller slot causes silent overflow and empty/invalid responses.
>
> **Token budget breakdown per fin_quant request:**
> | Component | Approx. tokens |
> |---|---|
> | System prompt + scenario description | ~3,000 |
> | `MAX_FACTOR_HISTORY=5` past experiments × ~2,500 | ~12,500 |
> | RAG context + instructions | ~2,000 |
> | **Total** | **~17,500** (well within 120k slot) |
>
> Formula: `ctx_size / parallel` must satisfy `n_ctx_slot > MAX_FACTOR_HISTORY × 2500 + 5000`.
>
> - `--reasoning off` — **critical**: completely disables Qwen3 chain-of-thought. `--reasoning-budget 0` is not sufficient — it still starts and immediately aborts reasoning, producing empty JSON responses. Only `--reasoning off` prevents this entirely.
> **Important flags:**
> - `--ctx-size 240000 --parallel 2` — allocates **2 slots × 120,000 tokens each**. `fin_quant` prompts can reach 80k+ tokens with full factor history; a smaller slot causes silent overflow and empty responses.
> - `--reasoning off` — **critical**: completely disables Qwen3 chain-of-thought. `--reasoning-budget 0` is not sufficient and produces empty JSON responses.
> - `--n-gpu-layers 24` — 4 fewer than maximum on RTX 5060 Ti (16 GB), freeing ~500 MB VRAM for the larger KV cache.
> - `-ctk q4_0 -ctv q4_0` — quantises the KV cache to 4-bit, reducing VRAM from ~5 GB to ~1.3 GB at 240k context.
### Data Configuration
Edit [`data_config.yaml`](data_config.yaml) to customize walk-forward splits:
```yaml
instrument: EURUSD
frequency: 1min
data_path: ~/.qlib/qlib_data/eurusd_1min_data
train_start: "2022-03-14"
train_end: "2024-06-30"
valid_start: "2024-07-01"
valid_end: "2024-12-31"
test_start: "2025-01-01"
test_end: "2026-03-20"
market_context:
spread_bps: 1.5
target_arr: 9.62
max_drawdown: 20
```
---
## No GPU? Use OpenRouter
If you don't have a CUDA-capable GPU, you can run Predix using [OpenRouter](https://openrouter.ai) for LLM inference — no local model download required.
**1. Set up `.env` for OpenRouter:**
```bash
# Chat (OpenRouter)
OPENAI_API_KEY=sk-or-v1-<your-openrouter-key>
OPENAI_API_BASE=https://openrouter.ai/api/v1
CHAT_MODEL=qwen/qwen3-235b-a22b
# Embedding (Ollama — still required locally)
LITELLM_PROXY_API_KEY=local
LITELLM_PROXY_API_BASE=http://localhost:11434/v1
EMBEDDING_MODEL=nomic-embed-text
```
**2. Skip the llama-server step** — no local LLM server needed.
**3. Run with the OpenRouter backend:**
```bash
rdagent fin_quant --model openrouter
```
**4. Parallel runs** (uses API concurrency instead of GPU slots):
```bash
python predix_parallel.py --runs 5 --api-keys 1 -m openrouter
```
> Ollama is still required for embeddings even in the OpenRouter path. Install from [ollama.com](https://ollama.com) and run `ollama pull nomic-embed-text` once.
---
## Quick Start
### Prerequisites checklist
```bash
# 1. Docker running?
docker run --rm hello-world
# 2. Data in place?
ls git_ignore_folder/factor_implementation_source_data/intraday_pv.h5
# 3. LLM server running?
curl http://localhost:8081/health
```
### 1. Run Trading Loop
```bash
# Activate conda environment
conda activate predix
# Start EURUSD trading loop
rdagent fin_quant
# With options
# or with explicit options:
rdagent fin_quant --loop-n 5 --step-n 2
```
### 2. Monitor Results
```bash
# Start the UI dashboard
# Web dashboard
rdagent server_ui --port 19899 --log-dir git_ignore_folder/RD-Agent_workspace/
# then open http://127.0.0.1:19899
# Or open in browser
# http://127.0.0.1:19899
# Best strategies so far
python predix.py best
```
### 3. Loop Continuously
To run the trading loop continuously with auto-restart:
### 3. Run Continuously
```bash
# Simple loop
while true; do
rdagent fin_quant
sleep 5
@@ -218,39 +345,36 @@ done
## CLI Commands
### Trading Loop
### Factor & Strategy Loop
| Command | Description |
|---------|-------------|
| `rdagent fin_quant` | Start factor evolution loop |
| `rdagent fin_quant --loop-n 5` | Run 5 evolution loops |
| `rdagent fin_quant` | Start autonomous factor + model evolution loop |
| `rdagent fin_quant --loop-n 5` | Run exactly 5 evolution loops |
| `rdagent fin_quant --with-dashboard` | Start with web dashboard |
| `rdagent fin_quant --cli-dashboard` | Start with CLI Rich dashboard |
### Parallel Execution
| Command | Description |
|---------|-------------|
| `python predix_parallel.py --runs 5 --api-keys 1 -m openrouter` | Run 5 parallel factor evolutions |
| `python predix_parallel.py --runs 20 --api-keys 2 -m openrouter` | Run 20 runs with 2 API keys |
### AI Strategy Generation (with REAL OHLCV Backtest)
| Command | Description |
|---------|-------------|
| `python predix_gen_strategies_real_bt.py` | Generate 10 strategies with LLM + real backtest |
| `python predix_gen_strategies_real_bt.py 20` | Generate 20 strategies |
| `python predix_gen_strategies_real_bt.py 5` | Generate 5 strategies (faster) |
| `rdagent fin_factor` | Factor-only evolution |
| `rdagent fin_model` | Model-only evolution |
### Strategy Reports
| Command | Description |
|---------|-------------|
| `python predix.py best` | Show top strategies by composite score (Sharpe × DD × trade penalty) |
| `python predix.py best` | Show top strategies by composite score |
| `python predix.py best -n 20 -m sharpe` | Top 20 by Sharpe ratio |
| `python predix.py best --show NAME` | Full metadata for one strategy |
| `python predix_strategy_report.py` | Generate reports for ALL strategies |
| `python predix_strategy_report.py results/strategies_new/123_MyStrategy.json` | Report for single strategy |
| `python predix_gen_strategies_real_bt.py` | Generate 10 strategies with LLM + real backtest |
| `python predix_gen_strategies_real_bt.py 20` | Generate 20 strategies |
### Kronos Foundation Model
| Command | Description |
|---------|-------------|
| `python predix.py kronos-factor` | Generate Kronos predicted-return factor (daily stride, ~15 min GPU) |
| `python predix.py kronos-factor --pred 30` | 30-bar prediction horizon |
| `python predix.py kronos-factor --device cpu` | CPU inference (slower) |
| `python predix.py kronos-eval` | Evaluate Kronos IC / hit rate vs LightGBM baseline |
| `python predix.py kronos-eval --pred 96` | Daily horizon evaluation |
### Factor Evaluation
@@ -260,62 +384,21 @@ done
| `python predix.py top -n 20` | Show top 20 factors by IC |
| `python predix.py portfolio-simple` | Simple portfolio optimization |
### Other Utilities
### Parallel Execution
| Command | Description |
|---------|-------------|
| `python predix_parallel.py --runs 5 --api-keys 1 -m openrouter` | Run 5 parallel factor evolutions |
| `python predix_parallel.py --runs 20 --api-keys 2 -m openrouter` | Run 20 runs with 2 API keys |
### Monitoring & Debug
| Command | Description |
|---------|-------------|
| `rdagent server_ui --port 19899 --log-dir <path>` | Start web dashboard |
| `rdagent health_check` | Validate environment setup |
| `python predix_batch_backtest.py` | Batch backtest multiple factors |
| `python predix_parallel.py` | Parallel factor evolution |
| `python predix_rebacktest_strategies.py` | Re-backtest existing strategies |
| `python debug_backtest.py` | Debug backtest alignment & IC |
### Environment Options
| Env Variable | Description | Example |
|--------------|-------------|---------|
| `OPENROUTER_API_KEY` | OpenRouter API key | `sk-or-v1-...` |
| `OPENAI_API_KEY` | Alternative: OpenAI key | `sk-...` |
| `CHAT_MODEL` | LLM model | `openrouter/qwen/qwen3.6-plus:free` |
| `OPENROUTER_MODEL` | Specific OpenRouter model | `openrouter/qwen/qwen3.6-plus:free` |
| `NO_COLOR` | Disable ANSI colors | `1` |
---
## Configuration
### Data Configuration
Edit [`data_config.yaml`](data_config.yaml) to customize:
```yaml
instrument: EURUSD
frequency: 1min
data_path: ~/.qlib/qlib_data/eurusd_1min_data
# Walk-forward split
train_start: "2022-03-14"
train_end: "2024-06-30"
valid_start: "2024-07-01"
valid_end: "2024-12-31"
test_start: "2025-01-01"
test_end: "2026-03-20"
# Market context for LLM prompts
market_context:
spread_bps: 1.5
target_arr: 9.62 # Target annual return (%)
max_drawdown: 20 # Max drawdown (%)
```
### Environment Variables
| Variable | Description | Example |
|----------|-------------|---------|
| `CHAT_MODEL` | LLM for reasoning | `gpt-4o`, `deepseek-chat` |
| `EMBEDDING_MODEL` | Embedding model | `text-embedding-3-small` |
| `OPENAI_API_KEY` | API key for OpenAI | `sk-...` |
| `DEEPSEEK_API_KEY` | API key for DeepSeek | `sk-...` |
| `DS_LOCAL_DATA_PATH` | Local data directory | `./data` |
---
@@ -363,13 +446,32 @@ Real-time dashboard for monitoring:
- Cumulative returns and drawdowns
- Code diffs and implementation history
### 🤖 Kronos Foundation Model Integration
Predix integrates [Kronos-mini](https://github.com/shiyu-coder/Kronos) — a 4.1M parameter OHLCV foundation model pretrained on 12+ billion K-lines from 45 global exchanges (AAAI 2026, MIT):
- **Option A — Alpha Factor**: Rolling daily inference generates a `KronosPredReturn` factor. Every 96 bars (one trading day), Kronos predicts the next day's return from the previous 512 bars of EUR/USD OHLCV data. The factor is forward-filled to 1-min frequency and plugs directly into Predix's factor evaluation pipeline.
- **Option B — Model Evaluation**: Kronos runs alongside LightGBM as a standalone predictor. IC (Information Coefficient), IC IR, and directional hit rate are computed over the full dataset for direct comparison with LightGBM-generated models.
```bash
# One-time setup
git clone https://github.com/shiyu-coder/Kronos ~/Kronos
# Generate factor (Option A) — saves to results/factors/
python predix.py kronos-factor
# Evaluate as model (Option B) — prints IC vs LightGBM reference
python predix.py kronos-eval
```
### 🔒 Security & Quality
Automated quality assurance:
- **60 Integration Tests** - All features tested automatically
- **Bandit Security Scanner** - Pre-commit security checks
- **Pre-commit Hooks** - Tests run before EVERY commit
- **134+ Tests** — all features tested automatically on every commit
- **Bandit Security Scanner** — pre-commit security checks
- **Weekly Dependency Audit** — automated vulnerability scan via GitHub Actions
---
@@ -382,118 +484,23 @@ predix/
│ ├── components/ # Reusable agent components
│ │ ├── backtesting/ # Backtest engine & protections
│ │ │ ├── backtest_engine.py
│ │ │ ├── vbt_backtest.py # Unified backtest engine
│ │ │ ├── results_db.py
│ │ │ ── risk_management.py
│ │ │ └── protections/ # Trading protection system (NEW)
│ │ │ ├── base.py
│ │ │ ├── max_drawdown.py
│ │ │ ├── cooldown.py
│ │ │ ├── stoploss_guard.py
│ │ │ ├── low_performance.py
│ │ │ └── protection_manager.py
│ │ ├── coder/ # Factor & model coding
│ │ └── loader.py # Prompt & model loaders
│ │ │ ── protections/ # Trading protection system
│ │ └── coder/ # Factor & model coding (CoSTEER + Optuna)
│ ├── core/ # Core abstractions
│ ├── scenarios/ # Domain-specific scenarios
│ └── utils/ # Utilities
├── test/ # Test suite
── integration/ # Integration tests (60 tests)
│ │ └── test_all_features.py
│ └── backtesting/ # Unit tests
│ └── test_protections.py
├── constraints/ # Constraint definitions
├── docs/ # Documentation
├── test/ # Test suite (134 tests)
── backtesting/ # Backtest unit tests
├── web/ # Web UI frontend
├── data_config.yaml # Data configuration
├── data_config.yaml # Walk-forward split configuration
├── pyproject.toml # Project metadata
└── requirements.txt # Dependencies
```
---
## Data Setup
Predix requires **1-minute EUR/USD OHLCV data** in HDF5 format.
### Required Format
The data file must be saved as `intraday_pv.h5` with the following structure:
| Field | Type | Description |
|-------|------|-------------|
| **Index** | MultiIndex `(datetime, instrument)` | Timestamp + currency pair |
| **`$open`** | float32 | Open price |
| **`$close`** | float32 | Close price |
| **`$high`** | float32 | High price |
| **`$low`** | float32 | Low price |
| **`$volume`** | float32 | Tick volume |
**Save location:** `git_ignore_folder/factor_implementation_source_data/intraday_pv.h5`
### Where to Get Data
| Source | Cost | Notes |
|--------|------|-------|
| **[Dukascopy](https://www.dukascopy.com/swiss/english/marketfeed/historical/)** | Free | Best free EUR/USD tick data |
| **[OANDA API](https://developer.oanda.com/)** | Free (demo) | Requires API key |
| **[TrueFX](https://truefx.com/)** | Free | Institutional-quality data |
| **[Kaggle](https://www.kaggle.com/datasets?search=EURUSD+1min)** | Free | Search "EURUSD 1 minute" |
| **MetaTrader 5** | Free | Export via `copy_rates_range()` |
### Quick CSV Conversion
```python
import pandas as pd
df = pd.read_csv('eurusd_1min.csv', parse_dates=['datetime'])
df = df.rename(columns={'open': '$open', 'close': '$close',
'high': '$high', 'low': '$low', 'volume': '$volume'})
df['instrument'] = 'EURUSD'
df = df.set_index(['datetime', 'instrument'])
for col in ['$open', '$close', '$high', '$low', '$volume']:
df[col] = df[col].astype('float32')
df.to_hdf('intraday_pv.h5', key='data', mode='w')
```
Expected data columns: `$open`, `$close`, `$high`, `$low`, `$volume`
---
## CLI Commands
| Command | Description |
|---------|-------------|
| `rdagent fin_quant` | Full factor & model co-evolution |
| `rdagent fin_factor` | Factor-only evolution |
| `rdagent fin_model` | Model-only evolution |
| `rdagent fin_factor_report --report-folder=<path>` | Extract factors from financial reports |
| `rdagent general_model <paper-url>` | Extract model from research paper |
| `rdagent rl_trading --mode train --algorithm PPO` | Train RL trading agent |
| `rdagent rl_trading --mode backtest --model-path <path>` | Backtest with trained RL model |
| `rdagent data_science --competition <name>` | Kaggle/data science competition mode |
| `rdagent ui --port 19899 --log-dir <path>` | Start monitoring dashboard |
| `rdagent health_check` | Validate environment setup |
### RL Trading Examples
```bash
# Train new RL agent with PPO
rdagent rl_trading --mode train --algorithm PPO --total-timesteps 100000
# Backtest with trained model
rdagent rl_trading --mode backtest --model-path models/rl_trader.zip
# Disable trading protections (not recommended)
rdagent rl_trading --mode backtest --no-with-protections
# Get help
rdagent rl_trading --help
```
**Note:** RL Trading works without `stable-baselines3` (uses simple fallback strategy). For full RL features, install: `pip install -r requirements/rl.txt`
---
## Requirements
Core dependencies (see [`requirements.txt`](requirements.txt) for full list):
+4 -4
View File
@@ -1,5 +1,5 @@
azure-identity==1.17.1
dill==0.3.9
azure-identity==1.25.3
dill==0.4.1
pillow==10.4.0
psutil==6.1.0
scipy==1.14.1
psutil==6.1.1
scipy==1.15.3
+4 -4
View File
@@ -1,5 +1,5 @@
azure-identity==1.17.1
dill==0.3.9
azure-identity==1.25.3
dill==0.4.1
pillow==10.4.0
psutil==6.1.0
scipy==1.14.1
psutil==6.1.1
scipy==1.15.3
+251
View File
@@ -22,6 +22,98 @@ app = typer.Typer(help="Predix - AI Quantitative Trading Agent")
console = Console()
def _ensure_kronos_factor_in_pool(con) -> None:
"""Auto-generate Kronos factor and register it in the StrategyOrchestrator pool.
Runs before fin_quant starts. If the Kronos parquet already exists in
results/factors/values/ and has a matching JSON with ic, it's a no-op.
Otherwise, generates the factor (stride=500 for speed) and computes IC.
"""
import json as _json
from datetime import datetime as _dt
data_path = Path("git_ignore_folder/factor_implementation_source_data/intraday_pv.h5")
if not data_path.exists():
return # No data — skip silently
factor_name = "KronosPredReturn_p96"
factors_dir = Path("results/factors")
values_dir = factors_dir / "values"
json_path = factors_dir / f"{factor_name}.json"
parquet_path = values_dir / f"{factor_name}.parquet"
# Already in pool with IC — nothing to do
if json_path.exists() and parquet_path.exists():
try:
existing = _json.loads(json_path.read_text())
if existing.get("ic") is not None:
return
except Exception:
pass
con.print("\n[bold yellow]Kronos Factor[/bold yellow] not in pool — generating automatically...")
con.print(" [dim]stride=500 (~4500 windows), batch=32 — ~5-10 min on GPU[/dim]")
try:
from rdagent.components.coder.kronos_adapter import _cuda_available, build_kronos_factor, evaluate_kronos_model
_device = "cuda" if _cuda_available() else "cpu"
# Generate factor values
factor_df = build_kronos_factor(
hdf5_path=data_path,
context_bars=100,
pred_bars=96,
stride_bars=500,
device=_device,
batch_size=32,
)
# Save parquet to values/ directory (where StrategyOrchestrator looks)
values_dir.mkdir(parents=True, exist_ok=True)
factor_df.to_parquet(parquet_path)
# Quick IC evaluation (stride=2000 → ~1100 windows, fast)
con.print(" [dim]Computing IC...[/dim]")
metrics = evaluate_kronos_model(
hdf5_path=data_path,
context_bars=100,
pred_bars=96,
stride_bars=2000,
device=_device,
batch_size=32,
)
ic = metrics.get("IC_mean", 0.0) or 0.0
hit_rate = metrics.get("hit_rate", 0.5)
# Write JSON metadata compatible with StrategyOrchestrator
factors_dir.mkdir(parents=True, exist_ok=True)
meta = {
"factor_name": factor_name,
"status": "success",
"ic": ic,
"hit_rate": hit_rate,
"model": "NeoQuasar/Kronos-mini",
"context_bars": 100,
"pred_bars": 96,
"stride_bars": 500,
"device": _device,
"generated_at": _dt.now().isoformat(),
"n_bars": len(factor_df),
"n_non_nan": int(factor_df["KronosPredReturn"].notna().sum()),
}
json_path.write_text(_json.dumps(meta, indent=2))
color = "green" if abs(ic) > 0.01 else "yellow"
con.print(
f" [bold {color}]Kronos Factor ready:[/bold {color}] IC={ic:.4f}, "
f"Hit-Rate={hit_rate:.1%} — added to strategy pool"
)
except Exception as e:
con.print(f" [yellow]Kronos Factor generation failed ({e}) — continuing without it[/yellow]")
@app.command()
def quant(
model: str = typer.Option(
@@ -226,6 +318,9 @@ def quant(
threading.Thread(target=start_cli_dash, daemon=True).start()
time.sleep(1)
# ---- Kronos Factor: auto-generate if not in pool ----
_ensure_kronos_factor_in_pool(console)
# ---- Start fin_quant ----
from rdagent.app.qlib_rd_loop.quant import main as fin_quant
from rdagent.log.daily_log import session as _daily_session
@@ -1583,5 +1678,161 @@ def best(
console.print(f"[green]Exported {len(top)} strategies (code stripped) → {export}[/green]")
@app.command("kronos-factor")
def kronos_factor(
context: int = typer.Option(512, "--context", "-c", help="Context window in bars (max 512 for Kronos-mini)"),
pred: int = typer.Option(96, "--pred", "-p", help="Prediction horizon in bars (default 96 = 1 trading day at 1-min)"),
stride: int = typer.Option(None, "--stride", "-s", help="Stride between windows (default: same as --pred)"),
device: str = typer.Option(None, "--device", "-d", help="Device: cuda or cpu (default: auto-detect)"),
batch_size: int = typer.Option(32, "--batch-size", "-b", help="Windows per GPU batch (higher = faster on GPU, more VRAM)"),
output: str = typer.Option(None, "--output", "-o", help="Output parquet path (default: results/factors/kronos_pred_return_p<pred>.parquet)"),
):
"""Generate Kronos-mini predicted-return alpha factor (Option A).
Runs Kronos-mini (4.1M params OHLCV foundation model, AAAI 2026) on rolling
windows of EUR/USD 1-min data and saves a predicted-return factor in Predix's
standard MultiIndex (datetime, instrument) format.
Strategy: every STRIDE bars, use the previous CONTEXT bars as input and
predict the next PRED bars. Windows are processed in GPU batches of BATCH_SIZE
for full GPU utilization (5-20x faster than sequential). Default (--pred 96) =
one trading day at 1-min frequency, ~2 000 windows total.
Requires:
~/Kronos repo (git clone https://github.com/shiyu-coder/Kronos ~/Kronos)
git_ignore_folder/factor_implementation_source_data/intraday_pv.h5
Examples:
$ predix kronos-factor # Default: daily stride, GPU
$ predix kronos-factor --pred 30 --device cpu # 30-bar horizon, CPU
$ predix kronos-factor --context 256 --pred 48
See Also:
predix kronos-eval - Evaluate Kronos as model and compute IC vs LightGBM
predix top - Show top factors by IC
"""
from rdagent.components.coder.kronos_adapter import _cuda_available
_device = device or ("cuda" if _cuda_available() else "cpu")
_stride = stride or pred
data_path = Path("git_ignore_folder/factor_implementation_source_data/intraday_pv.h5")
if not data_path.exists():
console.print(f"[red]ERROR: Data not found at {data_path}[/red]")
console.print("Run data conversion first — see README Data Setup section.")
raise typer.Exit(1)
console.print(f"[bold]Kronos Factor Generator[/bold]")
console.print(f" Context: [cyan]{context}[/cyan] bars | Pred: [cyan]{pred}[/cyan] bars | Device: [cyan]{_device}[/cyan]")
from rdagent.components.coder.kronos_adapter import build_kronos_factor
factor_df = build_kronos_factor(
hdf5_path=data_path,
context_bars=context,
pred_bars=pred,
stride_bars=_stride,
device=_device,
batch_size=batch_size,
)
out_dir = Path("results/factors")
out_dir.mkdir(parents=True, exist_ok=True)
out_path = Path(output) if output else out_dir / f"kronos_pred_return_p{pred}.parquet"
factor_df.to_parquet(out_path)
import json as _json
from datetime import datetime as _dt
meta = {
"factor_name": f"KronosPredReturn_p{pred}",
"description": f"Kronos-mini predicted return, {pred}-bar horizon",
"model": "NeoQuasar/Kronos-mini",
"context_bars": context,
"pred_bars": pred,
"stride_bars": _stride,
"device": _device,
"generated_at": _dt.now().isoformat(),
"n_bars": len(factor_df),
"n_non_nan": int(factor_df["KronosPredReturn"].notna().sum()),
"parquet_path": str(out_path),
}
meta_path = out_path.with_suffix(".json")
meta_path.write_text(_json.dumps(meta, indent=2))
console.print(f"\n[green]Factor saved:[/green] {out_path}")
console.print(f" Shape: {factor_df.shape} | Non-NaN: {meta['n_non_nan']}")
console.print(f" Metadata: {meta_path}")
console.print("\n[dim]Use 'predix top' to compare with other factors.[/dim]")
@app.command("kronos-eval")
def kronos_eval(
context: int = typer.Option(512, "--context", "-c", help="Context window in bars"),
pred: int = typer.Option(30, "--pred", "-p", help="Prediction horizon in bars"),
stride: int = typer.Option(None, "--stride", "-s", help="Stride between evaluations (default: same as --pred)"),
device: str = typer.Option(None, "--device", "-d", help="Device: cuda or cpu (default: auto-detect)"),
batch_size: int = typer.Option(32, "--batch-size", "-b", help="Windows per GPU batch (higher = faster on GPU, more VRAM)"),
):
"""Evaluate Kronos-mini as standalone model — IC and hit rate vs LightGBM (Option B).
Runs Kronos inference on the full EUR/USD dataset and computes:
- IC (Information Coefficient): correlation between predicted and actual returns
- IC IR: IC / std — risk-adjusted signal strength (>0.5 = good)
- Hit Rate: directional accuracy (>50% = useful signal)
Results are printed and saved to results/kronos/ for comparison with LightGBM
models generated by fin_quant.
Requires:
~/Kronos repo (git clone https://github.com/shiyu-coder/Kronos ~/Kronos)
git_ignore_folder/factor_implementation_source_data/intraday_pv.h5
Examples:
$ predix kronos-eval # Default: 30-bar horizon
$ predix kronos-eval --pred 96 --device cuda # Daily horizon, GPU
$ predix kronos-eval --context 256 --pred 15 # Shorter horizon
See Also:
predix kronos-factor - Generate Kronos factor for the factor pipeline
predix best - Show top strategies
"""
from rdagent.components.coder.kronos_adapter import _cuda_available
_device = device or ("cuda" if _cuda_available() else "cpu")
_stride = stride or pred
data_path = Path("git_ignore_folder/factor_implementation_source_data/intraday_pv.h5")
if not data_path.exists():
console.print(f"[red]ERROR: Data not found at {data_path}[/red]")
raise typer.Exit(1)
console.print(f"[bold]Kronos Model Evaluator[/bold] (alongside LightGBM)")
console.print(f" Context: [cyan]{context}[/cyan] bars | Pred: [cyan]{pred}[/cyan] bars | Device: [cyan]{_device}[/cyan]")
console.print(" Running evaluation...")
from rdagent.components.coder.kronos_adapter import evaluate_kronos_model
metrics = evaluate_kronos_model(
hdf5_path=data_path,
context_bars=context,
pred_bars=pred,
stride_bars=_stride,
device=_device,
batch_size=batch_size,
)
console.print(f"\n[bold]Kronos-mini Results[/bold]")
console.print(f" Predictions: [cyan]{metrics['n_predictions']}[/cyan]")
console.print(f" IC (mean): [{'green' if metrics['IC_mean'] > 0.02 else 'yellow'}]{metrics['IC_mean']:.4f}[/]")
console.print(f" IC IR: [{'green' if metrics['IC_IR'] > 0.5 else 'yellow'}]{metrics['IC_IR']:.4f}[/] (>0.5 = strong signal)")
console.print(f" Hit Rate: [{'green' if metrics['hit_rate'] > 0.52 else 'yellow'}]{metrics['hit_rate']:.2%}[/] (>50% = directionally useful)")
console.print(f"\n[dim]Reference: LightGBM baseline IC typically 0.010.05 on 1-min EUR/USD[/dim]")
import json as _json
out_dir = Path("results/kronos")
out_dir.mkdir(parents=True, exist_ok=True)
out_path = out_dir / f"kronos_eval_ctx{context}_pred{pred}.json"
out_path.write_text(_json.dumps({**metrics, "context_bars": context, "pred_bars": pred}, indent=2))
console.print(f"\n[green]Results saved:[/green] {out_path}")
if __name__ == "__main__":
app()
+380
View File
@@ -0,0 +1,380 @@
"""
Kronos Foundation Model Adapter for Predix.
Wraps the Kronos-mini OHLCV foundation model (4.1M params, AAAI 2026, MIT)
for use as:
- Factor (Option A): predicted next-day return signal
- Model alongside LightGBM (Option B): IC/Sharpe evaluation
Kronos repo: https://github.com/shiyu-coder/Kronos
HuggingFace: NeoQuasar/Kronos-mini | NeoQuasar/Kronos-Tokenizer-2k
"""
from __future__ import annotations
import sys
from pathlib import Path
from typing import Optional
import numpy as np
import pandas as pd
import logging
logger = logging.getLogger(__name__)
def _cuda_available() -> bool:
try:
import torch
return torch.cuda.is_available()
except ImportError:
return False
KRONOS_REPO = Path.home() / "Kronos"
_KRONOS_AVAILABLE: Optional[bool] = None
def _ensure_kronos() -> bool:
global _KRONOS_AVAILABLE
if _KRONOS_AVAILABLE is not None:
return _KRONOS_AVAILABLE
if not KRONOS_REPO.exists():
logger.warning(f"Kronos repo not found at {KRONOS_REPO}. Clone with: git clone https://github.com/shiyu-coder/Kronos ~/Kronos")
_KRONOS_AVAILABLE = False
return False
repo_str = str(KRONOS_REPO)
if repo_str not in sys.path:
sys.path.insert(0, repo_str)
try:
import model as _ # noqa: F401
_KRONOS_AVAILABLE = True
except ImportError as e:
logger.warning(f"Failed to import Kronos model: {e}")
_KRONOS_AVAILABLE = False
return _KRONOS_AVAILABLE
def _ohlcv_from_predix(df: pd.DataFrame) -> pd.DataFrame:
"""Convert Predix HDF5 format ($open/$close/...) to Kronos format (open/close/...)."""
col_map = {"$open": "open", "$high": "high", "$low": "low", "$close": "close", "$volume": "volume"}
renamed = df.rename(columns=col_map)
cols = [c for c in ["open", "high", "low", "close", "volume"] if c in renamed.columns]
return renamed[cols].astype(float)
def _build_window_inputs(
ohlcv_df: pd.DataFrame,
pred_bars: int,
freq: str,
) -> tuple[pd.DataFrame, pd.Series, pd.Series]:
"""Prepare (ctx_df, x_timestamp, y_timestamp) for one Kronos window."""
last_ts = ohlcv_df.index[-1]
future_idx = pd.date_range(start=last_ts, periods=pred_bars + 1, freq=freq)[1:]
x_timestamp = pd.Series(ohlcv_df.index.values)
y_timestamp = pd.Series(future_idx)
ctx = ohlcv_df.copy().reset_index(drop=True)
return ctx, x_timestamp, y_timestamp
class KronosAdapter:
"""
Loads Kronos-mini once and provides rolling-window OHLCV inference.
Usage:
adapter = KronosAdapter(device="cuda")
adapter.load()
pred_return = adapter.predict_return(ohlcv_df, context_bars=512, pred_bars=96)
"""
MODEL_ID = "NeoQuasar/Kronos-mini"
TOKENIZER_ID = "NeoQuasar/Kronos-Tokenizer-2k"
def __init__(self, device: Optional[str] = None, max_context: int = 512):
self.device = device or ("cuda" if _cuda_available() else "cpu")
self.max_context = max_context
self._predictor = None
def load(self) -> "KronosAdapter":
if self._predictor is not None:
return self
if not _ensure_kronos():
raise RuntimeError("Kronos not available — see warning above.")
from model import Kronos, KronosTokenizer, KronosPredictor # type: ignore
logger.info(f"Loading Kronos-mini from HuggingFace ({self.MODEL_ID})...")
tokenizer = KronosTokenizer.from_pretrained(self.TOKENIZER_ID)
model = Kronos.from_pretrained(self.MODEL_ID)
self._predictor = KronosPredictor(model, tokenizer, device=self.device, max_context=self.max_context)
logger.info("Kronos-mini loaded.")
return self
def predict_next_bars(
self,
ohlcv_df: pd.DataFrame,
context_bars: int,
pred_bars: int,
temperature: float = 1.0,
top_p: float = 0.9,
) -> pd.DataFrame:
"""
Run Kronos on `context_bars` of OHLCV data, returning `pred_bars` predicted bars.
Args:
ohlcv_df: DataFrame with columns open/high/low/close[/volume], DatetimeIndex.
context_bars: Number of history bars to feed as context.
pred_bars: Number of future bars to predict.
Returns:
DataFrame with predicted open/high/low/close/volume, indexed by future timestamps.
"""
if self._predictor is None:
raise RuntimeError("Call .load() first.")
if len(ohlcv_df) < context_bars:
raise ValueError(f"Need at least {context_bars} bars, got {len(ohlcv_df)}")
freq = ohlcv_df.index.freq or pd.infer_freq(ohlcv_df.index[:100]) or "1min"
ctx, x_timestamp, y_timestamp = _build_window_inputs(ohlcv_df.iloc[-context_bars:], pred_bars, freq)
future_idx = pd.DatetimeIndex(y_timestamp)
pred_df = self._predictor.predict(
df=ctx,
x_timestamp=x_timestamp,
y_timestamp=y_timestamp,
pred_len=pred_bars,
T=temperature,
top_p=top_p,
sample_count=1,
verbose=False,
)
pred_df.index = future_idx
return pred_df
def predict_next_bars_batch(
self,
ohlcv_windows: list,
pred_bars: int,
temperature: float = 1.0,
top_p: float = 0.9,
) -> list:
"""
Batch inference: run Kronos on multiple context windows simultaneously.
All windows must have the same number of bars. Processing them together
saturates the GPU and is typically 5-20x faster than sequential calls.
Args:
ohlcv_windows: List of OHLCV DataFrames, each with a DatetimeIndex.
pred_bars: Number of future bars to predict per window.
Returns:
List of prediction DataFrames (one per input window), same order.
"""
if self._predictor is None:
raise RuntimeError("Call .load() first.")
if not ohlcv_windows:
return []
freq = ohlcv_windows[0].index.freq or pd.infer_freq(ohlcv_windows[0].index[:100]) or "1min"
df_list, x_ts_list, y_ts_list, future_idxs = [], [], [], []
for win in ohlcv_windows:
ctx, x_ts, y_ts = _build_window_inputs(win, pred_bars, freq)
df_list.append(ctx)
x_ts_list.append(x_ts)
y_ts_list.append(y_ts)
future_idxs.append(pd.DatetimeIndex(y_ts))
pred_dfs = self._predictor.predict_batch(
df_list=df_list,
x_timestamp_list=x_ts_list,
y_timestamp_list=y_ts_list,
pred_len=pred_bars,
T=temperature,
top_p=top_p,
sample_count=1,
verbose=False,
)
for pred_df, future_idx in zip(pred_dfs, future_idxs):
pred_df.index = future_idx
return pred_dfs
def predict_return(
self,
ohlcv_df: pd.DataFrame,
context_bars: int = 512,
pred_bars: int = 1,
) -> float:
"""
Predict the average return over the next `pred_bars` using the last `context_bars`.
Returns the predicted log-return (predicted_close / last_close - 1).
"""
pred = self.predict_next_bars(ohlcv_df, context_bars=context_bars, pred_bars=pred_bars)
last_close = float(ohlcv_df["close"].iloc[-1])
pred_close = float(pred["close"].iloc[-1])
return pred_close / last_close - 1.0
def build_kronos_factor(
hdf5_path,
context_bars: int = 512,
pred_bars: int = 96,
stride_bars: int = 96,
device: Optional[str] = None,
batch_size: int = 32,
) -> pd.DataFrame:
"""
Generate the Kronos predicted-return factor for all EUR/USD 1-min bars.
Strategy:
Every `stride_bars` bars, run Kronos on the previous `context_bars` and
predict the next `pred_bars`. Windows are processed in GPU batches of
`batch_size` for full GPU utilization. The predicted log-return is
forward-filled across the predicted window.
Returns:
MultiIndex (datetime, instrument) DataFrame with column "KronosPredReturn".
"""
device = device or ("cuda" if _cuda_available() else "cpu")
logger.info(f"Loading data from {hdf5_path}...")
raw = pd.read_hdf(hdf5_path, key="data")
instrument = raw.index.get_level_values("instrument").unique()[0]
df = raw.xs(instrument, level="instrument")
ohlcv = _ohlcv_from_predix(df)
adapter = KronosAdapter(device=device, max_context=min(context_bars, 512))
adapter.load()
bar_indices = list(range(context_bars, len(ohlcv), stride_bars))
n_windows = len(bar_indices)
logger.info(
f"Running Kronos batch inference: {n_windows} windows "
f"(batch={batch_size}, stride={stride_bars}, ctx={context_bars}, pred={pred_bars}, device={device})"
)
factor_values: dict = {}
for batch_start in range(0, n_windows, batch_size):
batch_idx = bar_indices[batch_start : batch_start + batch_size]
windows = [ohlcv.iloc[i - context_bars : i] for i in batch_idx]
last_closes = [float(ohlcv["close"].iloc[i - 1]) for i in batch_idx]
try:
pred_dfs = adapter.predict_next_bars_batch(windows, pred_bars=pred_bars)
for pred_df, last_close in zip(pred_dfs, last_closes):
for ts, row in pred_df.iterrows():
factor_values[ts] = float(row["close"]) / last_close - 1.0
except Exception as e:
logger.warning(f"Batch {batch_start // batch_size + 1} failed ({e}), retrying individually...")
for bar_idx, win, last_close in zip(batch_idx, windows, last_closes):
try:
pred = adapter.predict_next_bars(win, context_bars=context_bars, pred_bars=pred_bars)
for ts, row in pred.iterrows():
factor_values[ts] = float(row["close"]) / last_close - 1.0
except Exception as e2:
logger.warning(f" Single inference failed at bar {bar_idx}: {e2}")
done = min(batch_start + batch_size, n_windows)
if done % max(batch_size, 100) < batch_size or done == n_windows:
logger.info(f" {done}/{n_windows} windows done")
if not factor_values:
raise RuntimeError("No Kronos predictions were generated.")
factor_series = pd.Series(factor_values, name="KronosPredReturn")
factor_series = factor_series.reindex(ohlcv.index, method="ffill")
result = factor_series.to_frame()
result.index = pd.MultiIndex.from_arrays(
[ohlcv.index, [instrument] * len(ohlcv)],
names=["datetime", "instrument"],
)
logger.info(f"Kronos factor built: {len(result)} bars, {result['KronosPredReturn'].notna().sum()} non-NaN")
return result
def evaluate_kronos_model(
hdf5_path,
context_bars: int = 512,
pred_bars: int = 30,
stride_bars: int = 30,
device: Optional[str] = None,
batch_size: int = 32,
) -> dict:
"""
Evaluate Kronos as a standalone model (Option B, alongside LightGBM).
Computes IC (Information Coefficient) between Kronos predicted returns and
actual realized returns on the test set.
Returns:
dict with keys: IC_mean, IC_std, IC_IR (IC / std), hit_rate, n_predictions
"""
device = device or ("cuda" if _cuda_available() else "cpu")
raw = pd.read_hdf(hdf5_path, key="data")
instrument = raw.index.get_level_values("instrument").unique()[0]
df = raw.xs(instrument, level="instrument")
ohlcv = _ohlcv_from_predix(df)
adapter = KronosAdapter(device=device, max_context=min(context_bars, 512))
adapter.load()
n = len(ohlcv)
bar_indices = list(range(context_bars, n - pred_bars, stride_bars))
logger.info(
f"Evaluating Kronos: {len(bar_indices)} windows "
f"(batch={batch_size}, ctx={context_bars}, pred={pred_bars}, device={device})"
)
predicted_returns = []
actual_returns = []
for batch_start in range(0, len(bar_indices), batch_size):
batch_idx = bar_indices[batch_start : batch_start + batch_size]
windows = [ohlcv.iloc[i - context_bars : i] for i in batch_idx]
last_closes = [float(ohlcv["close"].iloc[i - 1]) for i in batch_idx]
actuals = [
float(ohlcv["close"].iloc[i + pred_bars - 1]) / float(ohlcv["close"].iloc[i - 1]) - 1.0
for i in batch_idx
]
try:
pred_dfs = adapter.predict_next_bars_batch(windows, pred_bars=pred_bars)
for pred_df, last_close, actual_ret in zip(pred_dfs, last_closes, actuals):
pred_ret = float(pred_df["close"].iloc[-1]) / last_close - 1.0
predicted_returns.append(pred_ret)
actual_returns.append(actual_ret)
except Exception as e:
logger.warning(f"Batch {batch_start // batch_size + 1} failed ({e}), retrying individually...")
for bar_idx, win, last_close, actual_ret in zip(batch_idx, windows, last_closes, actuals):
try:
pred = adapter.predict_next_bars(win, context_bars=context_bars, pred_bars=pred_bars)
pred_ret = float(pred["close"].iloc[-1]) / last_close - 1.0
predicted_returns.append(pred_ret)
actual_returns.append(actual_ret)
except Exception:
pass
pred_arr = np.array(predicted_returns)
actual_arr = np.array(actual_returns)
ic = np.corrcoef(pred_arr, actual_arr)[0, 1] if len(pred_arr) > 1 else float("nan")
ic_std = float(
np.std([
np.corrcoef(pred_arr[i : i + 50], actual_arr[i : i + 50])[0, 1]
for i in range(0, len(pred_arr) - 50, 10)
])
) if len(pred_arr) > 60 else float("nan")
hit_rate = float(np.mean(np.sign(pred_arr) == np.sign(actual_arr)))
return {
"IC_mean": float(ic),
"IC_std": ic_std,
"IC_IR": float(ic / ic_std) if ic_std and ic_std > 0 else float("nan"),
"hit_rate": hit_rate,
"n_predictions": len(pred_arr),
}
# BATCH_INFERENCE_v2
+1
View File
@@ -5,6 +5,7 @@ python-Levenshtein
scikit-learn
filelock
loguru
psutil
fire
fuzzywuzzy
openai
+1 -1
View File
@@ -13,4 +13,4 @@ sphinx-togglebutton
sphinx_rtd_theme
# snowballstemmer, a dependency of sphinx, was released on 2025-05-08 with version 3.0.0,
# which causes errors in the build process. So we've limited the version for now.
snowballstemmer<3.0
snowballstemmer<4.0
+95
View File
@@ -0,0 +1,95 @@
#!/usr/bin/env python
"""
Option A: Generate Kronos predicted-return factor from EUR/USD 1-min data.
Runs Kronos-mini inference in daily strides (96 bars/day) over all available
OHLCV data and saves the resulting factor for use in Predix's factor pipeline.
Usage:
conda activate predix
python scripts/kronos_factor_gen.py
python scripts/kronos_factor_gen.py --context 512 --pred 96 --device cuda
python scripts/kronos_factor_gen.py --device cpu # slower but no GPU needed
"""
import argparse
import json
from datetime import datetime
from pathlib import Path
from dotenv import load_dotenv
load_dotenv(Path(__file__).parent.parent / ".env")
import pandas as pd
import torch
DATA_PATH = Path("git_ignore_folder/factor_implementation_source_data/intraday_pv.h5")
OUTPUT_DIR = Path("results/factors")
def main():
parser = argparse.ArgumentParser(description="Generate Kronos predicted-return factor")
parser.add_argument("--context", type=int, default=512, help="Context window in bars (max 512 for Kronos-mini)")
parser.add_argument("--pred", type=int, default=96, help="Prediction horizon in bars (default: 96 = 1 trading day)")
parser.add_argument("--stride", type=int, default=None, help="Stride between windows (default: same as --pred)")
parser.add_argument("--device", type=str, default="cuda" if torch.cuda.is_available() else "cpu")
parser.add_argument("--output", type=str, default=None, help="Output parquet path (default: auto)")
args = parser.parse_args()
stride = args.stride or args.pred
print(f"Kronos Factor Generator")
print(f" Data: {DATA_PATH}")
print(f" Context: {args.context} bars")
print(f" Pred: {args.pred} bars ({args.pred} min = {args.pred/96:.1f} trading days)")
print(f" Stride: {stride} bars")
print(f" Device: {args.device}")
print()
if not DATA_PATH.exists():
print(f"ERROR: Data not found at {DATA_PATH}")
print("Run data conversion first — see README Data Setup section.")
raise SystemExit(1)
from rdagent.components.coder.kronos_adapter import build_kronos_factor
factor_df = build_kronos_factor(
hdf5_path=DATA_PATH,
context_bars=args.context,
pred_bars=args.pred,
stride_bars=stride,
device=args.device,
)
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
out_path = args.output or OUTPUT_DIR / f"kronos_pred_return_p{args.pred}.parquet"
factor_df.to_parquet(out_path)
print(f"\nFactor saved to: {out_path}")
print(f"Shape: {factor_df.shape}")
print(f"Non-NaN: {factor_df['KronosPredReturn'].notna().sum()}")
print(f"\nSample (first 5):")
print(factor_df.head())
# Save metadata for predix.py top / best integration
meta = {
"factor_name": f"KronosPredReturn_p{args.pred}",
"description": f"Kronos-mini predicted return, {args.pred}-bar horizon",
"model": "NeoQuasar/Kronos-mini",
"context_bars": args.context,
"pred_bars": args.pred,
"stride_bars": stride,
"device": args.device,
"generated_at": datetime.now().isoformat(),
"n_bars": len(factor_df),
"n_non_nan": int(factor_df["KronosPredReturn"].notna().sum()),
"parquet_path": str(out_path),
}
meta_path = out_path.with_suffix(".json")
with open(meta_path, "w") as f:
json.dump(meta, f, indent=2)
print(f"Metadata saved to: {meta_path}")
if __name__ == "__main__":
main()
+77
View File
@@ -0,0 +1,77 @@
#!/usr/bin/env python
"""
Option B: Evaluate Kronos-mini as a model alongside LightGBM.
Computes IC (Information Coefficient) and hit rate for Kronos predictions
vs actual realized returns. Results are printed for comparison with LightGBM.
Usage:
conda activate predix
python scripts/kronos_model_eval.py
python scripts/kronos_model_eval.py --pred 30 --context 512 --device cuda
"""
import argparse
import json
from pathlib import Path
from dotenv import load_dotenv
load_dotenv(Path(__file__).parent.parent / ".env")
import torch
DATA_PATH = Path("git_ignore_folder/factor_implementation_source_data/intraday_pv.h5")
OUTPUT_DIR = Path("results/kronos")
def main():
parser = argparse.ArgumentParser(description="Evaluate Kronos as model (alongside LightGBM)")
parser.add_argument("--context", type=int, default=512, help="Context window in bars")
parser.add_argument("--pred", type=int, default=30, help="Prediction horizon in bars")
parser.add_argument("--stride", type=int, default=None, help="Stride between evaluations (default: pred)")
parser.add_argument("--device", type=str, default="cuda" if torch.cuda.is_available() else "cpu")
args = parser.parse_args()
stride = args.stride or args.pred
print(f"Kronos Model Evaluator (alongside LightGBM)")
print(f" Context: {args.context} bars | Pred: {args.pred} bars | Device: {args.device}")
print()
if not DATA_PATH.exists():
print(f"ERROR: Data not found at {DATA_PATH}")
raise SystemExit(1)
from rdagent.components.coder.kronos_adapter import evaluate_kronos_model
print("Running evaluation (this may take several minutes)...")
metrics = evaluate_kronos_model(
hdf5_path=DATA_PATH,
context_bars=args.context,
pred_bars=args.pred,
stride_bars=stride,
device=args.device,
)
print("\n" + "=" * 50)
print("Kronos-mini Model Evaluation Results")
print("=" * 50)
print(f" Predictions: {metrics['n_predictions']}")
print(f" IC (mean): {metrics['IC_mean']:.4f}")
print(f" IC (std): {metrics['IC_std']:.4f}")
print(f" IC IR: {metrics['IC_IR']:.4f} (>0.5 = good)")
print(f" Hit Rate: {metrics['hit_rate']:.2%} (>50% = directionally useful)")
print("=" * 50)
print()
print("Reference: LightGBM baseline IC typically 0.010.05 on 1-min EUR/USD")
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
out = OUTPUT_DIR / f"kronos_eval_ctx{args.context}_pred{args.pred}.json"
with open(out, "w") as f:
json.dump({**metrics, "context_bars": args.context, "pred_bars": args.pred}, f, indent=2)
print(f"\nResults saved to: {out}")
if __name__ == "__main__":
main()
+277
View File
@@ -0,0 +1,277 @@
"""Tests for KronosAdapter and CLI commands — mock-based, no real model download needed."""
import json
import numpy as np
import pandas as pd
import pytest
from pathlib import Path
from unittest.mock import patch, MagicMock
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _make_ohlcv(n: int = 600, freq: str = "1min") -> pd.DataFrame:
"""Synthetic 1-min OHLCV DataFrame."""
idx = pd.date_range("2024-01-01", periods=n, freq=freq)
close = 1.1000 + np.cumsum(np.random.randn(n) * 0.0001)
return pd.DataFrame({
"open": close + np.random.randn(n) * 0.00005,
"high": close + np.abs(np.random.randn(n) * 0.0001),
"low": close - np.abs(np.random.randn(n) * 0.0001),
"close": close,
"volume": np.abs(np.random.randn(n) * 100),
}, index=idx)
def _make_predix_hdf5(tmp_path: Path, n: int = 300) -> Path:
"""Write a minimal Predix-format HDF5 file and return its path."""
idx = pd.MultiIndex.from_arrays(
[pd.date_range("2024-01-01", periods=n, freq="1min"), ["EURUSD"] * n],
names=["datetime", "instrument"],
)
df = pd.DataFrame({
"$open": (np.random.rand(n) + 1.1).astype("float32"),
"$close": (np.random.rand(n) + 1.1).astype("float32"),
"$high": (np.random.rand(n) + 1.11).astype("float32"),
"$low": (np.random.rand(n) + 1.09).astype("float32"),
"$volume": (np.random.rand(n) * 100).astype("float32"),
}, index=idx)
h5 = tmp_path / "intraday_pv.h5"
df.to_hdf(h5, key="data", mode="w")
return h5
def _make_mock_adapter():
"""Return a mock KronosAdapter whose predict_next_bars is deterministic."""
class MockAdapter:
def load(self): return self
def predict_next_bars(self, ohlcv_df, context_bars, pred_bars, **kw):
idx = pd.date_range(ohlcv_df.index[-1], periods=pred_bars + 1, freq="1min")[1:]
last_close = float(ohlcv_df["close"].iloc[-1])
return pd.DataFrame({
"open": last_close * 1.001,
"close": last_close * 1.002,
"high": last_close * 1.003,
"low": last_close * 0.999,
"volume": 100.0,
}, index=idx)
def predict_return(self, ohlcv_df, context_bars=512, pred_bars=1):
return 0.001
return MockAdapter()
# ---------------------------------------------------------------------------
# Unit tests: _ohlcv_from_predix
# ---------------------------------------------------------------------------
class TestOhlcvConversion:
def test_renames_dollar_columns(self):
from rdagent.components.coder.kronos_adapter import _ohlcv_from_predix
idx = pd.MultiIndex.from_arrays(
[pd.date_range("2024-01-01", periods=3, freq="1min"), ["EURUSD"] * 3],
names=["datetime", "instrument"],
)
df = pd.DataFrame({
"$open": [1.1, 1.2, 1.3], "$high": [1.15, 1.25, 1.35],
"$low": [1.05, 1.15, 1.25], "$close": [1.12, 1.22, 1.32],
"$volume": [100.0, 200.0, 300.0],
}, index=idx)
result = _ohlcv_from_predix(df)
assert list(result.columns) == ["open", "high", "low", "close", "volume"]
def test_no_dollar_columns_passthrough(self):
from rdagent.components.coder.kronos_adapter import _ohlcv_from_predix
df = pd.DataFrame({"open": [1.0], "close": [1.1], "high": [1.2], "low": [0.9], "volume": [100.0]})
result = _ohlcv_from_predix(df)
assert "close" in result.columns
def test_output_is_float64(self):
from rdagent.components.coder.kronos_adapter import _ohlcv_from_predix
df = pd.DataFrame({
"$open": np.array([1.1], dtype="float32"),
"$close": np.array([1.1], dtype="float32"),
"$high": np.array([1.1], dtype="float32"),
"$low": np.array([1.1], dtype="float32"),
"$volume": np.array([100.0], dtype="float32"),
})
result = _ohlcv_from_predix(df)
assert result["close"].dtype == np.float64
# ---------------------------------------------------------------------------
# Unit tests: KronosAdapter availability check
# ---------------------------------------------------------------------------
class TestKronosAvailability:
def test_unavailable_without_repo(self, tmp_path, monkeypatch):
import rdagent.components.coder.kronos_adapter as mod
monkeypatch.setattr(mod, "KRONOS_REPO", tmp_path / "nonexistent")
monkeypatch.setattr(mod, "_KRONOS_AVAILABLE", None)
assert mod._ensure_kronos() is False
def test_load_raises_without_repo(self, tmp_path, monkeypatch):
import rdagent.components.coder.kronos_adapter as mod
monkeypatch.setattr(mod, "KRONOS_REPO", tmp_path / "nonexistent")
monkeypatch.setattr(mod, "_KRONOS_AVAILABLE", None)
adapter = mod.KronosAdapter()
with pytest.raises(RuntimeError, match="Kronos not available"):
adapter.load()
def test_predict_without_load_raises(self):
from rdagent.components.coder.kronos_adapter import KronosAdapter
adapter = KronosAdapter()
with pytest.raises(RuntimeError, match="Call .load()"):
adapter.predict_next_bars(_make_ohlcv(100), 50, 10)
# ---------------------------------------------------------------------------
# Unit tests: build_kronos_factor
# ---------------------------------------------------------------------------
class TestBuildKronosFactor:
def test_output_has_correct_multiindex(self, tmp_path, monkeypatch):
import rdagent.components.coder.kronos_adapter as mod
monkeypatch.setattr(mod, "KronosAdapter", lambda **kw: _make_mock_adapter())
h5 = _make_predix_hdf5(tmp_path)
result = mod.build_kronos_factor(h5, context_bars=100, pred_bars=20, stride_bars=20, device="cpu")
assert result.index.names == ["datetime", "instrument"]
assert result.index.nlevels == 2
def test_output_column_name(self, tmp_path, monkeypatch):
import rdagent.components.coder.kronos_adapter as mod
monkeypatch.setattr(mod, "KronosAdapter", lambda **kw: _make_mock_adapter())
h5 = _make_predix_hdf5(tmp_path)
result = mod.build_kronos_factor(h5, context_bars=100, pred_bars=20, stride_bars=20, device="cpu")
assert "KronosPredReturn" in result.columns
def test_output_has_non_nan_values(self, tmp_path, monkeypatch):
import rdagent.components.coder.kronos_adapter as mod
monkeypatch.setattr(mod, "KronosAdapter", lambda **kw: _make_mock_adapter())
h5 = _make_predix_hdf5(tmp_path)
result = mod.build_kronos_factor(h5, context_bars=100, pred_bars=20, stride_bars=20, device="cpu")
assert result["KronosPredReturn"].notna().sum() > 0
def test_output_length_matches_input(self, tmp_path, monkeypatch):
import rdagent.components.coder.kronos_adapter as mod
monkeypatch.setattr(mod, "KronosAdapter", lambda **kw: _make_mock_adapter())
n = 300
h5 = _make_predix_hdf5(tmp_path, n=n)
result = mod.build_kronos_factor(h5, context_bars=100, pred_bars=20, stride_bars=20, device="cpu")
assert len(result) == n
def test_forward_fill_propagates_signal(self, tmp_path, monkeypatch):
"""Values within a predicted window should be forward-filled, not NaN."""
import rdagent.components.coder.kronos_adapter as mod
monkeypatch.setattr(mod, "KronosAdapter", lambda **kw: _make_mock_adapter())
h5 = _make_predix_hdf5(tmp_path, n=300)
result = mod.build_kronos_factor(h5, context_bars=100, pred_bars=20, stride_bars=20, device="cpu")
non_nan_ratio = result["KronosPredReturn"].notna().mean()
assert non_nan_ratio > 0.5, f"Expected >50% non-NaN, got {non_nan_ratio:.2%}"
def test_raises_on_missing_hdf5(self, tmp_path, monkeypatch):
import rdagent.components.coder.kronos_adapter as mod
monkeypatch.setattr(mod, "KronosAdapter", lambda **kw: _make_mock_adapter())
with pytest.raises(Exception):
mod.build_kronos_factor(tmp_path / "missing.h5", context_bars=50, pred_bars=10, stride_bars=10)
# ---------------------------------------------------------------------------
# Unit tests: evaluate_kronos_model
# ---------------------------------------------------------------------------
class TestEvaluateKronosModel:
def test_returns_required_keys(self, tmp_path, monkeypatch):
import rdagent.components.coder.kronos_adapter as mod
monkeypatch.setattr(mod, "KronosAdapter", lambda **kw: _make_mock_adapter())
h5 = _make_predix_hdf5(tmp_path, n=400)
metrics = mod.evaluate_kronos_model(h5, context_bars=100, pred_bars=20, stride_bars=20, device="cpu")
for key in ["IC_mean", "IC_std", "IC_IR", "hit_rate", "n_predictions"]:
assert key in metrics, f"Missing key: {key}"
def test_hit_rate_in_valid_range(self, tmp_path, monkeypatch):
import rdagent.components.coder.kronos_adapter as mod
monkeypatch.setattr(mod, "KronosAdapter", lambda **kw: _make_mock_adapter())
h5 = _make_predix_hdf5(tmp_path, n=400)
metrics = mod.evaluate_kronos_model(h5, context_bars=100, pred_bars=20, stride_bars=20, device="cpu")
assert 0.0 <= metrics["hit_rate"] <= 1.0
def test_n_predictions_positive(self, tmp_path, monkeypatch):
import rdagent.components.coder.kronos_adapter as mod
monkeypatch.setattr(mod, "KronosAdapter", lambda **kw: _make_mock_adapter())
h5 = _make_predix_hdf5(tmp_path, n=400)
metrics = mod.evaluate_kronos_model(h5, context_bars=100, pred_bars=20, stride_bars=20, device="cpu")
assert metrics["n_predictions"] > 0
# ---------------------------------------------------------------------------
# Integration tests: CLI commands (via typer test runner)
# ---------------------------------------------------------------------------
class TestCLICommands:
def test_kronos_factor_missing_data_exits(self, tmp_path, monkeypatch):
"""kronos-factor exits with code 1 when HDF5 data is missing."""
from typer.testing import CliRunner
import predix as predix_mod
monkeypatch.chdir(tmp_path)
runner = CliRunner()
result = runner.invoke(predix_mod.app, ["kronos-factor"])
assert result.exit_code == 1
def test_kronos_eval_missing_data_exits(self, tmp_path, monkeypatch):
"""kronos-eval exits with code 1 when HDF5 data is missing."""
from typer.testing import CliRunner
import predix as predix_mod
monkeypatch.chdir(tmp_path)
runner = CliRunner()
result = runner.invoke(predix_mod.app, ["kronos-eval"])
assert result.exit_code == 1
def test_kronos_factor_runs_with_mock(self, tmp_path, monkeypatch):
"""kronos-factor completes and saves parquet + json when adapter is mocked."""
from typer.testing import CliRunner
import rdagent.components.coder.kronos_adapter as mod
import predix as predix_mod
monkeypatch.setattr(mod, "KronosAdapter", lambda **kw: _make_mock_adapter())
data_dir = tmp_path / "git_ignore_folder" / "factor_implementation_source_data"
data_dir.mkdir(parents=True)
_make_predix_hdf5(data_dir.parent.parent, n=300)
h5_src = tmp_path / "intraday_pv.h5"
# Put HDF5 where the CLI expects it
import shutil
src = _make_predix_hdf5(tmp_path, n=300)
shutil.copy(src, data_dir / "intraday_pv.h5")
monkeypatch.chdir(tmp_path)
runner = CliRunner()
result = runner.invoke(predix_mod.app, [
"kronos-factor", "--context", "100", "--pred", "20", "--device", "cpu"
])
assert result.exit_code == 0, result.output
assert "saved" in result.output.lower()
def test_kronos_eval_runs_with_mock(self, tmp_path, monkeypatch):
"""kronos-eval completes and prints IC metrics when adapter is mocked."""
from typer.testing import CliRunner
import rdagent.components.coder.kronos_adapter as mod
import predix as predix_mod
monkeypatch.setattr(mod, "KronosAdapter", lambda **kw: _make_mock_adapter())
data_dir = tmp_path / "git_ignore_folder" / "factor_implementation_source_data"
data_dir.mkdir(parents=True)
_make_predix_hdf5(data_dir.parent.parent, n=400)
src = _make_predix_hdf5(tmp_path, n=400)
import shutil
shutil.copy(src, data_dir / "intraday_pv.h5")
monkeypatch.chdir(tmp_path)
runner = CliRunner()
result = runner.invoke(predix_mod.app, [
"kronos-eval", "--context", "100", "--pred", "20", "--device", "cpu"
])
assert result.exit_code == 0, result.output
assert "IC" in result.output