Commit Graph

1347 Commits

Author SHA1 Message Date
TPTBusiness 49ebc19fd0 test: add 39 tests for fx_config, utils, predix_full_eval, exceptions, and log 2026-05-03 11:18:27 +02:00
TPTBusiness 58eb551898 test: add 4 tests for QuantTrace and QlibQuantHypothesis 2026-05-03 11:14:33 +02:00
TPTBusiness 2e18b7c104 test: add 24 tests for factor/model scenarios and experiments 2026-05-03 11:12:23 +02:00
TPTBusiness 28743eb043 test: add 33 tests for app config, strategy builder, and quant scenario 2026-05-03 11:09:55 +02:00
TPTBusiness f6f5531427 test: expand pre-commit to run qlib+backtesting unit tests (160+ tests) 2026-05-03 11:05:39 +02:00
github-actions[bot] bd9362e2f3 chore(master): release 1.4.2 (#48)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-05-03 11:02:32 +02:00
TPTBusiness a4d4266295 test: add 129 tests for critical untested code (core, CoSTEER, factor_coder, model_coder, qlib pipeline) 2026-05-03 10:59:24 +02:00
TPTBusiness 8f2ed4185f fix: add missing sys import and fix undefined acc_rate in factor eval 2026-05-03 10:19:59 +02:00
github-actions[bot] 7f24af6f05 chore(master): release 1.4.1 (#47)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-05-03 09:39:56 +02:00
TPTBusiness aba88dd090 fix: 15 bug fixes across orchestrator, runner, backtest, and infrastructure
Critical:
- strategy_orchestrator: fix IndentationError that prevented import (line 764)
- factor_runner: fix literal 'sys.executable' string → variable (line 966)

High (path bugs causing wrong directories):
- backtest_engine: fix results_path depth (3→4 .parent hops)
- results_db: fix factors_dir/failed_dir depth (3→4 .parent hops)
- factor_runner: eliminate run_id variable shadowing (parallel_run_id/db_run_id)
- model_runner: fix DB connection leak on add_backtest exception
- optuna_optimizer: fix imported logger shadowed by module-level reassignment

Medium:
- env: handle non-UTF-8 Docker build output with errors='replace'
- env: guard conda env list parsing against empty lines
- factor_runner: add check=False + stderr logging for full-data subprocess
- strategy_orchestrator: log exec() exceptions at ERROR level with traceback
- strategy_orchestrator: warn on unreplaced {{template}} variables in prompts

Low:
- factor_runner: guard IC_max.index access against scalar (AttributeError)
- predix_parallel: close log file handle on Popen failure
- predix_rebacktest_strategies: replace 4 bare except: with except Exception:
2026-05-03 09:37:00 +02:00
TPTBusiness 28766c932e test: add regression tests for background task path and env bugs
- Verify parallel runner project_root is repo root, not scripts/
- Verify .env loading from correct path
- Verify API key distribution (single key, multi-key comma-separated)
- Verify CLI project_root depth (3 .parent hops, not 4)
- Verify start_loop uses sys.executable and child_proc, not pkill
- Verify parallel_cli does not hardcode model=local
- Verify all referenced scripts exist at resolved paths
2026-05-03 08:57:56 +02:00
TPTBusiness 574e9d6c08 fix: correct project root paths and subprocess handling in parallel runner and CLI
- predix_parallel.py: fix project_root from scripts/ to repo root (parent.parent)
- predix_parallel.py: fix .env loading path and API key distribution logic
- cli.py: fix project_root depth from 4 to 3 .parent hops (7 locations)
- cli.py start_loop: use sys.executable instead of hardcoded python
- cli.py start_loop: replace broad pkill with targeted child process management
- cli.py parallel: remove hardcoded model=local
2026-05-03 08:49:18 +02:00
TPTBusiness ce76da912a fix: also catch ValueError in mean_variance for dimension mismatch 2026-05-03 00:39:22 +02:00
TPTBusiness 02ac3f7aae test: add direct unit tests for _apply_riskmgmt_mask, safe_resolve_path, import_class, and _add_column_if_not_exists 2026-05-03 00:35:57 +02:00
TPTBusiness 39b49b1724 fix: filter NaN in max(), remove redundant ternary, handle non-finite vbt results 2026-05-03 00:25:58 +02:00
TPTBusiness f1eb66cc8f fix: fix type annotation, remove unused parameter, improve import_class errors 2026-05-03 00:22:16 +02:00
TPTBusiness ca003cd0f2 fix: close log file handle, fix RiskMgmt equity double-count, remove bare except 2026-05-03 00:17:02 +02:00
TPTBusiness 4eeb724ac5 fix: resolve dead code, shell injection risk, mutable defaults, and other bugs
- strategy_orchestrator.py: remove unreachable dead 'if not factor_values' after early return
- strategy_orchestrator.py: eliminate duplicate OHLVC load in evaluate_strategy
- env.py: escape single-quotes in Docker entry to prevent shell injection (CWE-78)
- env.py: replace mutable default args with None pattern in DockerEnv subclasses
- factor_runner.py: move pandarallel.initialize() from import-time to lazy init
2026-05-02 23:21:38 +02:00
TPTBusiness 6c3bdb6ec1 fix: resolve unbound variable, logger shadowing, withdraw_loop edge case, and other bugs in main scripts
- quant.py: guard against empty orch_factors, move strategy_name before try block
- quant_proposal.py: fix __init__ return type Tuple[dict,bool] -> None
- strategy_orchestrator.py: remove dead rdagent_logger import shadowed by getLogger
- factor.py: replace unusual 'not x is None' with idiomatic 'x is not None'
- workflow/loop.py: withdraw_loop(0) raises RuntimeError instead of looking for folder -1
- workflow/tracking.py: replace crash-prone AssertionError with logger.warning + skip
- factor_from_report.py: fix misleading comment about loop_n/step_n dual use
2026-05-02 22:56:29 +02:00
github-actions[bot] 82633d328a chore(master): release 1.4.0 (#46)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-05-01 15:59:06 +02:00
TPTBusiness c5d919f581 feat(optimizer): add max_positions parameter to Optuna search space
Add max_positions (1-5) as an optimizable hyperparameter across all
three Optuna search stages (coarse, fine, very fine). The parameter
scales effective position size as min(position_size_pct × max_positions,
1.0), allowing the optimizer to discover pyramiding strategies.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 15:58:01 +02:00
github-actions[bot] 44ed82283d chore(master): release 1.3.11 (#45)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-05-01 13:58:55 +02:00
TPTBusiness 87610d660f fix(ci): lazy import logger in predix.py and cli.py to avoid ImportError in test env
Wrapped  in try/except
ImportError with standard logging fallback. The rdagent.log
module chain fails when predix.py is imported as a module
in the CI test environment (kronos CLI tests).
2026-05-01 13:58:16 +02:00
github-actions[bot] c0516d60f9 chore(master): release 1.3.10 (#44)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-05-01 13:50:38 +02:00
TPTBusiness a43c443c2e fix(security): replace remaining assert statements with proper error handling
Replaced 53 assert statements across 22 files with proper
if/raise patterns (TypeError, ValueError, AssertionError)
to resolve Bandit B101 alerts.
2026-05-01 13:49:58 +02:00
github-actions[bot] 9e58c64805 chore(master): release 1.3.9 (#43)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-05-01 13:43:52 +02:00
TPTBusiness 732361bb90 fix(security): resolve path-injection, B701, B101, B112 Bandit alerts
- Path injection (B614): centralized safe_resolve_path in core/utils.py,
  refactored 6 UI modules to use it with safe_root validation
- B701: added explicit autoescape=select_autoescape() to Jinja2
  Environment() calls in 3 files
- B101: replaced assert statements with proper if/raise patterns in
  12+ files (partial)
- B112: added logger.warning() to bare except:continue blocks in
  5 files
2026-05-01 13:42:59 +02:00
github-actions[bot] dff6262871 chore(master): release 1.3.8 (#42)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-04-30 20:01:29 +02:00
TPTBusiness 23b2518c74 fix(security): resolve path-injection and add nosec for safe temp paths (B108, py/path-injection)
- ds_trace.py: resolve() user-provided save path and use Path.name for filenames
  to prevent directory traversal in the local workspace save UI
- rl/finetune UI data_loaders: nosec B614 where paths are already validated
  against safe_root via realpath() before use
- Temp paths (/tmp/sample, /tmp/full, /tmp/mock/*, /tmp/predix_loop.pid,
  /tmp/autorl_output): nosec B108 — fixed Docker volume mount points or
  single-process admin files, not user-writable attack surface

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 19:26:38 +02:00
TPTBusiness d83c020637 fix(security): replace shell=True subprocess calls with list args in env.py (B602)
Converted conda commands in _update_bin_path, _sync_conda_cache_with_real_envs,
_prepare_conda_env, and FTCondaEnv.prepare() to list args. Replaced pipe-based
grep with pure Python parsing. LocalEnv.Popen retains shell=True with nosec
since entry is an internal command string set by LocalEnvConf, not user input.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 19:26:29 +02:00
TPTBusiness 48843682d0 fix(security): replace eval() with ast.literal_eval in finetune validator (B307)
eval() on trainer stdout output replaced with ast.literal_eval() which only
parses Python literals and cannot execute arbitrary code.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 19:26:23 +02:00
TPTBusiness 5a5bf4d771 fix(qlib): correct indentation in except blocks in quant_proposal and factor_runner
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 13:30:49 +02:00
TPTBusiness 02c830d4e2 fix(deps): relax aiohttp constraint to >=3.13.4 for litellm compatibility
litellm 1.83.14 pins aiohttp==3.13.4 exactly; requiring >=3.13.5 caused
an unresolvable conflict in CI. aiohttp 3.13.4 still patches all four CVEs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 09:44:09 +02:00
github-actions[bot] 1328a8f0c4 chore(master): release 1.3.7 (#41)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-04-30 09:35:48 +02:00
TPTBusiness 2126062edf fix(security): nosec for B608/B701 false positives in UI and template code
B608: Bandit flags any f-string containing "select" as potential SQL
injection. All four cases (app.py, ds_trace.py, llm_st.py, merge.py)
are Streamlit UI labels or log messages — not database queries.

B701: Jinja2 autoescape=False warnings in coder.py and utils.py are
false positives — these render Python code and plain-text templates,
not HTML. Enabling autoescape would corrupt the rendered code.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 09:35:09 +02:00
TPTBusiness 133ec1b816 fix(security): replace eval() with ast.literal_eval and add request timeouts (B307, B113)
- submit.py: eval(json_str) → ast.literal_eval(json_str) for safe
  Python-literal parsing without arbitrary code execution
- info.py: add timeout=30 to both requests.get() calls to prevent
  indefinite hangs on unresponsive GitHub API

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 09:35:09 +02:00
TPTBusiness 0fa4f5dcc8 fix(security): replace shell=True subprocess calls with list args (B602)
- factor.py: check_output([python_bin, path]) instead of shell string
- env.py QlibCondaEnv: all four conda commands use list args

Shell=True with a constructed string allows shell injection if
python_bin or path contain shell metacharacters.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 09:35:09 +02:00
github-actions[bot] a076b58304 chore(master): release 1.3.6 (#40)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-04-30 07:27:44 +02:00
TPTBusiness afe1823e85 fix(security): whitelist-validate metric column in get_top_factors (B608)
The metric parameter was passed directly into an f-string SQL query.
Add explicit validation against _ALLOWED_METRICS before use, raising
ValueError on unknown values. Raises ValueError on injection attempt
instead of silently accepting arbitrary column names.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 07:25:14 +02:00
TPTBusiness b4674ce3a0 fix(security): revert broken read_pickle encoding arg in kaggle template (B301)
The previous "fix" introduced pd.read_pickle(encoding="utf-8", "/path")
which is a SyntaxError (positional argument after keyword argument).
pd.read_pickle() has no encoding parameter.

Replace with correct # nosec B301 comment — pickle is safe here because
the files are written by the Kaggle preprocessing pipeline in a sandboxed
container and never sourced from user input.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 07:21:34 +02:00
TPTBusiness 942266f24d fix(security): validate SQL identifiers in _add_column_if_not_exists (B608)
Replace f-string SQL queries with whitelist validation:
- Table name must be in _ALLOWED_TABLES
- Column name must be alphanumeric+underscore
- Column type must be in _ALLOWED_COL_TYPES
- Use pragma_table_info() for existence check instead of SELECT f-string

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 07:21:13 +02:00
TPTBusiness 95b14fcfc8 chore(logging): size-based rotation and cap LLM call content
- Switch log rotation from midnight-only ("00:00") to size-based:
  per-command logs: 50 MB, all.log: 100 MB (with gz compression)
- Shorten retention from 30/60 days to 7 days
- Cap llm_calls.jsonl entries to 500 chars per field to prevent
  GB-scale files from long-running loops

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 07:19:27 +02:00
TPTBusiness e9422be6e2 chore(deps): bump setuptools >=78.1.1 to fix GHSA-8g6x-3r52-4m6c 2026-04-30 07:19:24 +02:00
TPTBusiness 161c4043d7 chore(deps): bump aiohttp >=3.13.5 and scipy >=1.15.3
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 07:19:24 +02:00
dependabot[bot] 6b3669c3be chore(deps): Update litellm requirement from >=1.73 to >=1.83.14 (#35)
Updates the requirements on [litellm](https://github.com/BerriAI/litellm) to permit the latest version.
- [Release notes](https://github.com/BerriAI/litellm/releases)
- [Commits](https://github.com/BerriAI/litellm/commits)

---
updated-dependencies:
- dependency-name: litellm
  dependency-version: 1.83.14
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-30 07:19:24 +02:00
dependabot[bot] a7fc33b9a8 chore(deps): Update lightgbm requirement from >=3.3.0 to >=3.3.5 (#34)
Updates the requirements on [lightgbm](https://github.com/microsoft/LightGBM) to permit the latest version.
- [Release notes](https://github.com/microsoft/LightGBM/releases)
- [Commits](https://github.com/microsoft/LightGBM/compare/v3.3.0...v3.3.5)

---
updated-dependencies:
- dependency-name: lightgbm
  dependency-version: 3.3.5
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-30 07:19:24 +02:00
dependabot[bot] 1448106230 chore(deps): Update stable-baselines3 requirement (#33)
Updates the requirements on [stable-baselines3](https://github.com/DLR-RM/stable-baselines3) to permit the latest version.
- [Release notes](https://github.com/DLR-RM/stable-baselines3/releases)
- [Commits](https://github.com/DLR-RM/stable-baselines3/compare/v2.0.0...v2.8.0)

---
updated-dependencies:
- dependency-name: stable-baselines3
  dependency-version: 2.8.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-30 07:19:24 +02:00
dependabot[bot] 5eeaf1f496 chore(deps): Bump googleapis/release-please-action from 4 to 5 (#32)
Bumps [googleapis/release-please-action](https://github.com/googleapis/release-please-action) from 4 to 5.
- [Release notes](https://github.com/googleapis/release-please-action/releases)
- [Changelog](https://github.com/googleapis/release-please-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/googleapis/release-please-action/compare/v4...v5)

---
updated-dependencies:
- dependency-name: googleapis/release-please-action
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-30 07:19:24 +02:00
TPTBusiness 90a6999563 fix(security): real fix for B404/B603 (sys.executable in factor_runner.py #745) 2026-04-29 22:42:28 +02:00
TPTBusiness 387508168f fix(security): real fix for B110 (logging in quant_proposal.py #741) 2026-04-29 21:27:22 +02:00