1029 Commits

Author SHA1 Message Date
TPTBusiness e168a5df7e fix: harmonize risk field names and case-insensitive DB column check
- vbt_backtest: unify risk_* → riskmgmt_* field names in _apply_risk_mask
- results_db: case-insensitive column existence check
- test_ftmo_oos: update test assertions to match renamed fields
2026-05-25 12:30:10 +02:00
TPTBusiness 9303b40fb9 feat: R&D loop fixes + new price-action research loop
Loop 1 (Factor R&D):
- Auto-fixer: composite normalization prevents single-factor variance collapse
- Caps entry_thresh 0.7, exit_thresh 0.3, window 20, rolling smoothing 2
- Adds unit-variance normalization for any factor count

Loop 2 (Price-Action R&D):
- New research loop for technical indicators (no LLM, no Docker)
- 7 indicators: MACD, Donchian, RSI, SMA, Bollinger, ATR, MA-Envelope
- 3 strategy types: single-TF, multi-TF majority-vote, portfolio
- Random hypothesis generation + backtest_signal evaluation
- 11/20 strategies profitable in first test run
- Top: MACD(12,15,3) 15min — Sharpe +14.01, +10.4%/month
2026-05-25 11:56:21 +02:00
TPTBusiness 4758de0eee refactor: remove all proprietary terms from codebase and git history
- Rename FTMO_* constants → generic names (RISK_PER_TRADE, MAX_DAILY_LOSS, etc.)
- Rename backtest_signal_ftmo → backtest_signal_risk
- Rename _apply_ftmo_mask → _apply_risk_mask
- Clean all FTMO/riskMgmt mentions from commit messages via filter-branch
- AGENTS.md: add non-negotiable rule — NEVER mention proprietary terms in commits/releases
- Code variables and function names sanitized project-wide
- Force-pushed rewritten history to remote
2026-05-22 15:10:36 +02:00
TPTBusiness d4611b530e feat: model-track bias + daily/portfolio tools
- Bandit: model arm prior bias 2.0, prior_var 5.0 → 77% model preference
- Default first action: model (was factor)
- Daily strategy generator: Kronos + factor grid search on daily resolution
- Grid search tool: fixed template, no LLM, deterministic
- Portfolio optimizer: greedy correlation-aware selection, leverage scaling
2026-05-17 20:09:46 +02:00
TPTBusiness e0000a18d2 feat: 15% monthly return target — infrastructure + daily signal resampling
Phase 1 — Infrastructure:
- RiskMgmt_RISK_PER_TRADE 0.5% → 1.5% (vbt_backtest.py)
- min_monthly_return_pct=15% acceptance filter (strategy_orchestrator)
- --min-monthly-return 15 CLI option (nexquant.py)
- {{ min_monthly_return }}% in strategy prompts
- MIN_MONTHLY_RETURN_PCT=15.0 in gen_strategies_real_bt + smart_strategy_gen
- realistic_backtest_all.py target_monthly 4→15%

Phase 2 — Factor quality:
- IC thresholds: prompt 0.05→0.08, bandit IC weight 0.10→0.20
- Explicite IC > 0.04 target in RAG prompt
- min_ic filters: data_loader 0.0→0.04, strategy_worker 0.02→0.04, ml_trainer 0.01→0.04

Architecture fix — Daily signal resampling:
- Factors have IC at daily resolution, but z-scores on 1-min collapse IC to ~0
- Resample factors to daily before strategy exec, ffill signal to 1-min for backtest
- Walk-forward IS years 3→1 (only 2 years of data available)
- Removed broken intersection() logic that destroyed 99.99% of 1-min data
- ffill stale propagation limited to 2880 bars (2 trading days)
- Fixed logger crash in _load_strategies
- Preflight: removed constant-signal check (false positive on random sandbox data)
- Tests: test_daily_signal_resampling.py (8 tests)

Non-negotiable rules: R1-R10 in AGENTS.md
2026-05-16 19:06:09 +02:00
TPTBusiness cbe1c52e00 refactor: rename project from Predix to NexQuant
Rename all source files, scripts, tests, documentation, and configuration
from Predix/predix to NexQuant/nexquant across the entire codebase.
2026-05-09 17:48:22 +02:00
TPTBusiness f0ac999dbe Revert "feat: prioritize Kronos foundation model factors in strategy selection"
This reverts commit b58fc5622dd08ab81ba890db1896f06f2266fe29.
2026-05-08 18:32:38 +02:00
TPTBusiness 9c91a6938d feat: prioritize Kronos foundation model factors in strategy selection 2026-05-08 18:32:16 +02:00
TPTBusiness 0aea8c7671 fix: restore KronosPredictor instantiation deleted during refactor 2026-05-07 21:52:37 +02:00
TPTBusiness 72e8a4306e feat: support Kronos-small and Kronos-base models, auto-select GPU/CPU 2026-05-07 21:45:28 +02:00
TPTBusiness 669263db37 fix: add missing debug() method to RDAgentLog 2026-05-06 21:25:59 +02:00
TPTBusiness 584bf9d955 feat: integrate Kronos foundation model into fin_quant R&D loop 2026-05-06 16:22:03 +02:00
TPTBusiness d458e39940 fix: prevent LLM retry loop from consecutive assistant message corruption 2026-05-05 18:56:12 +02:00
TPTBusiness e0c287a575 feat: run Kronos on CPU to avoid GPU conflict with llama-server 2026-05-05 15:29:01 +02:00
TPTBusiness c0ec1b39e1 feat: enable walk-forward OOS validation by default in backtest_signal_riskmgmt 2026-05-04 18:43:17 +02:00
TPTBusiness 58a7ece3a9 fix(security): replace os.path.realpath with pathlib.resolve in safe_resolve_path to fix path-injection alerts 2026-05-04 18:05:09 +02:00
TPTBusiness 1827c50344 feat: optimize strategy generator (cache OHLCV, min_sharpe 1.5, predix generate-strategies CLI) 2026-05-03 21:58:07 +02:00
TPTBusiness 06a5d5d92d refactor: move strategy_orchestrator and optuna_optimizer to closed-source (local/) 2026-05-03 21:38:09 +02:00
TPTBusiness 6d37f8956f feat: add runtime backtest verification (10 invariant checks in <1ms) + 489 tests + README docs 2026-05-03 14:00:49 +02:00
TPTBusiness ce4a5b7b4f fix: correct MaxDD to equity curve in strategy_builder; test: add 8 cross-validation tests for metric correctness 2026-05-03 12:28:09 +02:00
TPTBusiness 037f7ba7d2 fix: correct Sharpe/MaxDD/WinRate in direct factor eval (was computing on raw factor, now on strategy returns) 2026-05-03 12:17:27 +02:00
TPTBusiness 8f2ed4185f fix: add missing sys import and fix undefined acc_rate in factor eval 2026-05-03 10:19:59 +02:00
TPTBusiness aba88dd090 fix: 15 bug fixes across orchestrator, runner, backtest, and infrastructure
Critical:
- strategy_orchestrator: fix IndentationError that prevented import (line 764)
- factor_runner: fix literal 'sys.executable' string → variable (line 966)

High (path bugs causing wrong directories):
- backtest_engine: fix results_path depth (3→4 .parent hops)
- results_db: fix factors_dir/failed_dir depth (3→4 .parent hops)
- factor_runner: eliminate run_id variable shadowing (parallel_run_id/db_run_id)
- model_runner: fix DB connection leak on add_backtest exception
- optuna_optimizer: fix imported logger shadowed by module-level reassignment

Medium:
- env: handle non-UTF-8 Docker build output with errors='replace'
- env: guard conda env list parsing against empty lines
- factor_runner: add check=False + stderr logging for full-data subprocess
- strategy_orchestrator: log exec() exceptions at ERROR level with traceback
- strategy_orchestrator: warn on unreplaced {{template}} variables in prompts

Low:
- factor_runner: guard IC_max.index access against scalar (AttributeError)
- predix_parallel: close log file handle on Popen failure
- predix_rebacktest_strategies: replace 4 bare except: with except Exception:
2026-05-03 09:37:00 +02:00
TPTBusiness 574e9d6c08 fix: correct project root paths and subprocess handling in parallel runner and CLI
- predix_parallel.py: fix project_root from scripts/ to repo root (parent.parent)
- predix_parallel.py: fix .env loading path and API key distribution logic
- cli.py: fix project_root depth from 4 to 3 .parent hops (7 locations)
- cli.py start_loop: use sys.executable instead of hardcoded python
- cli.py start_loop: replace broad pkill with targeted child process management
- cli.py parallel: remove hardcoded model=local
2026-05-03 08:49:18 +02:00
TPTBusiness ce76da912a fix: also catch ValueError in mean_variance for dimension mismatch 2026-05-03 00:39:22 +02:00
TPTBusiness 39b49b1724 fix: filter NaN in max(), remove redundant ternary, handle non-finite vbt results 2026-05-03 00:25:58 +02:00
TPTBusiness f1eb66cc8f fix: fix type annotation, remove unused parameter, improve import_class errors 2026-05-03 00:22:16 +02:00
TPTBusiness ca003cd0f2 fix: close log file handle, fix RiskMgmt equity double-count, remove bare except 2026-05-03 00:17:02 +02:00
TPTBusiness 4eeb724ac5 fix: resolve dead code, shell injection risk, mutable defaults, and other bugs
- strategy_orchestrator.py: remove unreachable dead 'if not factor_values' after early return
- strategy_orchestrator.py: eliminate duplicate OHLVC load in evaluate_strategy
- env.py: escape single-quotes in Docker entry to prevent shell injection (CWE-78)
- env.py: replace mutable default args with None pattern in DockerEnv subclasses
- factor_runner.py: move pandarallel.initialize() from import-time to lazy init
2026-05-02 23:21:38 +02:00
TPTBusiness 6c3bdb6ec1 fix: resolve unbound variable, logger shadowing, withdraw_loop edge case, and other bugs in main scripts
- quant.py: guard against empty orch_factors, move strategy_name before try block
- quant_proposal.py: fix __init__ return type Tuple[dict,bool] -> None
- strategy_orchestrator.py: remove dead rdagent_logger import shadowed by getLogger
- factor.py: replace unusual 'not x is None' with idiomatic 'x is not None'
- workflow/loop.py: withdraw_loop(0) raises RuntimeError instead of looking for folder -1
- workflow/tracking.py: replace crash-prone AssertionError with logger.warning + skip
- factor_from_report.py: fix misleading comment about loop_n/step_n dual use
2026-05-02 22:56:29 +02:00
TPTBusiness 87610d660f fix(ci): lazy import logger in predix.py and cli.py to avoid ImportError in test env
Wrapped  in try/except
ImportError with standard logging fallback. The rdagent.log
module chain fails when predix.py is imported as a module
in the CI test environment (kronos CLI tests).
2026-05-01 13:58:16 +02:00
TPTBusiness a43c443c2e fix(security): replace remaining assert statements with proper error handling
Replaced 53 assert statements across 22 files with proper
if/raise patterns (TypeError, ValueError, AssertionError)
to resolve Bandit B101 alerts.
2026-05-01 13:49:58 +02:00
TPTBusiness 732361bb90 fix(security): resolve path-injection, B701, B101, B112 Bandit alerts
- Path injection (B614): centralized safe_resolve_path in core/utils.py,
  refactored 6 UI modules to use it with safe_root validation
- B701: added explicit autoescape=select_autoescape() to Jinja2
  Environment() calls in 3 files
- B101: replaced assert statements with proper if/raise patterns in
  12+ files (partial)
- B112: added logger.warning() to bare except:continue blocks in
  5 files
2026-05-01 13:42:59 +02:00
TPTBusiness 23b2518c74 fix(security): resolve path-injection and add nosec for safe temp paths (B108, py/path-injection)
- ds_trace.py: resolve() user-provided save path and use Path.name for filenames
  to prevent directory traversal in the local workspace save UI
- rl/finetune UI data_loaders: nosec B614 where paths are already validated
  against safe_root via realpath() before use
- Temp paths (/tmp/sample, /tmp/full, /tmp/mock/*, /tmp/predix_loop.pid,
  /tmp/autorl_output): nosec B108 — fixed Docker volume mount points or
  single-process admin files, not user-writable attack surface

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 19:26:38 +02:00
TPTBusiness d83c020637 fix(security): replace shell=True subprocess calls with list args in env.py (B602)
Converted conda commands in _update_bin_path, _sync_conda_cache_with_real_envs,
_prepare_conda_env, and FTCondaEnv.prepare() to list args. Replaced pipe-based
grep with pure Python parsing. LocalEnv.Popen retains shell=True with nosec
since entry is an internal command string set by LocalEnvConf, not user input.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 19:26:29 +02:00
TPTBusiness 48843682d0 fix(security): replace eval() with ast.literal_eval in finetune validator (B307)
eval() on trainer stdout output replaced with ast.literal_eval() which only
parses Python literals and cannot execute arbitrary code.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 19:26:23 +02:00
TPTBusiness 5a5bf4d771 fix(qlib): correct indentation in except blocks in quant_proposal and factor_runner
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 13:30:49 +02:00
TPTBusiness 2126062edf fix(security): nosec for B608/B701 false positives in UI and template code
B608: Bandit flags any f-string containing "select" as potential SQL
injection. All four cases (app.py, ds_trace.py, llm_st.py, merge.py)
are Streamlit UI labels or log messages — not database queries.

B701: Jinja2 autoescape=False warnings in coder.py and utils.py are
false positives — these render Python code and plain-text templates,
not HTML. Enabling autoescape would corrupt the rendered code.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 09:35:09 +02:00
TPTBusiness 133ec1b816 fix(security): replace eval() with ast.literal_eval and add request timeouts (B307, B113)
- submit.py: eval(json_str) → ast.literal_eval(json_str) for safe
  Python-literal parsing without arbitrary code execution
- info.py: add timeout=30 to both requests.get() calls to prevent
  indefinite hangs on unresponsive GitHub API

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 09:35:09 +02:00
TPTBusiness 0fa4f5dcc8 fix(security): replace shell=True subprocess calls with list args (B602)
- factor.py: check_output([python_bin, path]) instead of shell string
- env.py QlibCondaEnv: all four conda commands use list args

Shell=True with a constructed string allows shell injection if
python_bin or path contain shell metacharacters.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 09:35:09 +02:00
TPTBusiness afe1823e85 fix(security): whitelist-validate metric column in get_top_factors (B608)
The metric parameter was passed directly into an f-string SQL query.
Add explicit validation against _ALLOWED_METRICS before use, raising
ValueError on unknown values. Raises ValueError on injection attempt
instead of silently accepting arbitrary column names.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 07:25:14 +02:00
TPTBusiness b4674ce3a0 fix(security): revert broken read_pickle encoding arg in kaggle template (B301)
The previous "fix" introduced pd.read_pickle(encoding="utf-8", "/path")
which is a SyntaxError (positional argument after keyword argument).
pd.read_pickle() has no encoding parameter.

Replace with correct # nosec B301 comment — pickle is safe here because
the files are written by the Kaggle preprocessing pipeline in a sandboxed
container and never sourced from user input.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 07:21:34 +02:00
TPTBusiness 942266f24d fix(security): validate SQL identifiers in _add_column_if_not_exists (B608)
Replace f-string SQL queries with whitelist validation:
- Table name must be in _ALLOWED_TABLES
- Column name must be alphanumeric+underscore
- Column type must be in _ALLOWED_COL_TYPES
- Use pragma_table_info() for existence check instead of SELECT f-string

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 07:21:13 +02:00
TPTBusiness 95b14fcfc8 chore(logging): size-based rotation and cap LLM call content
- Switch log rotation from midnight-only ("00:00") to size-based:
  per-command logs: 50 MB, all.log: 100 MB (with gz compression)
- Shorten retention from 30/60 days to 7 days
- Cap llm_calls.jsonl entries to 500 chars per field to prevent
  GB-scale files from long-running loops

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 07:19:27 +02:00
TPTBusiness e9422be6e2 chore(deps): bump setuptools >=78.1.1 to fix GHSA-8g6x-3r52-4m6c 2026-04-30 07:19:24 +02:00
TPTBusiness 90a6999563 fix(security): real fix for B404/B603 (sys.executable in factor_runner.py #745) 2026-04-29 22:42:28 +02:00
TPTBusiness 387508168f fix(security): real fix for B110 (logging in quant_proposal.py #741) 2026-04-29 21:27:22 +02:00
TPTBusiness 2055cf1817 fix(security): real fix for B110 (logging in quant_proposal.py #741) 2026-04-29 21:24:30 +02:00
TPTBusiness 018231d1f2 fix(security): real fix for B110 (logging in factor_runner.py #744) 2026-04-29 21:23:46 +02:00
TPTBusiness d8bd16e6b9 fix(security): real fix for B110 (logging in factor_proposal.py #746) 2026-04-29 21:23:02 +02:00