mirror of
https://github.com/NicolasBohn/NexQuant.git
synced 2026-07-27 23:47:46 +00:00
feat: Add GitHub infrastructure, CI/CD pipelines, and examples
- Add GitHub issue templates (bug, feature, docs) - Add pull request template with closed-source checklist - Add CODEOWNERS for code review assignment - Add CI/CD workflows (ci, lint, security, docs, release) - pytest + coverage with Python 3.10/3.11 matrix - Ruff + MyPy code quality checks - Bandit + safety security scanning - Sphinx docs + GitHub Pages deployment - Automated PyPI releases on tag push - Add 6 comprehensive examples + Jupyter quickstart - 01_factor_discovery.py (LLM factor generation) - 02_factor_evolution.py (factor optimization) - 03_strategy_generation.py (IC-weighted combination) - 04_backtest_simple.py (strategy backtesting) - 05_model_training.py (XGBoost/LSTM training) - 06_rl_trading_agent.py (PPO/DQN/A2C agents) - notebooks/quickstart.ipynb (interactive tutorial) - Restructure .gitignore with explicit closed-source sections - Add CI/coverage/license badges to README - Complete CLI docstrings for all 9 commands - Add data_config.yaml for quant loop configuration
This commit is contained in:
@@ -0,0 +1,42 @@
|
||||
# CODEOWNERS
|
||||
# Diese Datei definiert die Verantwortlichen für Code-Reviews
|
||||
# Siehe: https://docs.github.com/en/repositories/working-with-files/managing-files/about-code-owners
|
||||
|
||||
# Core Maintainer (Standard-Reviewer für alle Änderungen)
|
||||
* @nico
|
||||
|
||||
# RD-Agent Core-Module
|
||||
/rdagent/core/ @nico
|
||||
/rdagent/components/ @nico
|
||||
/rdagent/app/ @nico
|
||||
|
||||
# Trading-Spezifika
|
||||
/rdagent/scenarios/ @nico
|
||||
/prompts/ @nico
|
||||
|
||||
# Dokumentation
|
||||
/docs/ @nico
|
||||
/README.md @nico
|
||||
/examples/ @nico
|
||||
/CONTRIBUTING.md @nico
|
||||
/CODE_OF_CONDUCT.md @nico
|
||||
|
||||
# Konfiguration & Build
|
||||
/pyproject.toml @nico
|
||||
/requirements.txt @nico
|
||||
/setup.py @nico
|
||||
/Makefile @nico
|
||||
|
||||
# CI/CD & Security
|
||||
/.github/ @nico
|
||||
/.pre-commit-config.yaml @nico
|
||||
/.bandit.yml @nico
|
||||
/SECURITY.md @nico
|
||||
|
||||
# Dashboard & Visualization
|
||||
/dashboard/ @nico
|
||||
/web/ @nico
|
||||
|
||||
# Data Pipeline
|
||||
/data/ @nico
|
||||
/scripts/download*.py @nico
|
||||
@@ -0,0 +1,58 @@
|
||||
---
|
||||
name: 🐛 Bug Report
|
||||
about: Create a report to help us improve PREDIX
|
||||
title: '[Bug] '
|
||||
labels: 'bug, needs-triage'
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
## Beschreibung
|
||||
<!-- Eine klare und prägnante Beschreibung des Bugs -->
|
||||
|
||||
## Reproduktionsschritte
|
||||
<!-- Schritte zum Reproduzieren des Verhaltens -->
|
||||
|
||||
1. Schritt 1: `...`
|
||||
2. Schritt 2: `...`
|
||||
3. Schritt 3: `...`
|
||||
4. Fehler tritt auf
|
||||
|
||||
## Erwartetes Verhalten
|
||||
<!-- Eine klare Beschreibung dessen, was passieren sollte -->
|
||||
|
||||
## Tatsächliches Verhalten
|
||||
<!-- Was passiert tatsächlich? -->
|
||||
|
||||
## Environment
|
||||
|
||||
<!-- Bitte fülle die folgenden Informationen aus -->
|
||||
|
||||
- **OS:** [z.B. Linux, macOS, Windows]
|
||||
- **Python-Version:** [z.B. 3.10, 3.11]
|
||||
- **PREDIX-Version:** [z.B. v2.0.0, main-branch]
|
||||
- **Installation:** [z.B. pip, conda, from source]
|
||||
|
||||
## Logs & Screenshots
|
||||
|
||||
<!-- Füge relevante Logs oder Screenshots hinzu -->
|
||||
|
||||
<details>
|
||||
<summary>Log Output (klicken zum Aufklappen)</summary>
|
||||
|
||||
```
|
||||
Hier die Log-Ausgabe einfügen
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
## Zusätzliche Kontext
|
||||
|
||||
<!-- Weitere Informationen zum Problem -->
|
||||
|
||||
### Data Configuration
|
||||
- [ ] Ich habe sichergestellt, dass die Daten korrekt geladen sind
|
||||
- [ ] `qlib init` wurde erfolgreich ausgeführt
|
||||
|
||||
### Workaround
|
||||
<!-- Falls vorhanden: Gibt es einen Workaround? -->
|
||||
@@ -0,0 +1,47 @@
|
||||
---
|
||||
name: 💡 Feature Request
|
||||
about: Suggest an idea for PREDIX
|
||||
title: '[Feature] '
|
||||
labels: 'enhancement, needs-triage'
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
## Problem-Beschreibung
|
||||
<!-- Bezieht sich dein Feature auf ein Problem? Bitte beschreibe es -->
|
||||
<!-- Beispiel: "Ich bin immer frustriert, wenn ich..." -->
|
||||
|
||||
## Lösungsvorschlag
|
||||
<!-- Eine klare und prägnante Beschreibung dessen, was du gerne hättest -->
|
||||
|
||||
## Alternativen
|
||||
<!-- Hast du alternative Lösungen in Betracht gezogen? -->
|
||||
|
||||
## Zusätzliche Kontext
|
||||
<!-- Weitere Informationen, Screenshots oder Mockups -->
|
||||
|
||||
## Use Case
|
||||
<!-- Wie würde dieses Feature deinen Workflow verbessern? -->
|
||||
|
||||
### Checkliste
|
||||
<!-- Bitte bestätige die folgenden Punkte mit [x] -->
|
||||
|
||||
- [ ] Ich habe die [Dokumentation](https://github.com/nico/Predix/tree/main/docs) gelesen
|
||||
- [ ] Ich habe geprüft, ob dieses Feature bereits als [bestehendes Issue](https://github.com/nico/Predix/issues) existiert
|
||||
- [ ] Dieses Feature ist relevant für **Open-Source** (keine closed-source Komponenten)
|
||||
|
||||
## Impact
|
||||
|
||||
<!-- Wer würde von diesem Feature profitieren? -->
|
||||
|
||||
- [ ] Alle PREDIX-Nutzer
|
||||
- [ ] Spezifische Nutzer (z.B. FX-Trader, Qlib-Nutzer)
|
||||
- [ ] Entwickler/Contributors
|
||||
|
||||
## Priorität
|
||||
|
||||
<!-- Wie dringend ist dieses Feature? -->
|
||||
|
||||
- [ ] Niedrig (Nice-to-have)
|
||||
- [ ] Mittel (Würde den Workflow verbessern)
|
||||
- [ ] Hoch (Blockiert meine Arbeit)
|
||||
@@ -0,0 +1,58 @@
|
||||
---
|
||||
name: 📚 Documentation Improvement
|
||||
about: Suggest improvements to PREDIX documentation
|
||||
title: '[Docs] '
|
||||
labels: 'documentation'
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
## Aktueller Zustand
|
||||
<!-- Welche Seite/Welcher Teil der Dokumentation ist betroffen? -->
|
||||
|
||||
**URL/Datei:** `z.B. README.md, docs/quickstart.rst`
|
||||
|
||||
**Aktueller Inhalt:**
|
||||
<!-- Zitat oder Beschreibung des aktuellen Zustands -->
|
||||
|
||||
## Verbesserungsvorschlag
|
||||
<!-- Was sollte geändert/hinzugefügt werden? -->
|
||||
|
||||
## Beispiel/Begründung
|
||||
<!-- Warum ist diese Verbesserung notwendig? -->
|
||||
|
||||
### Art der Verbesserung
|
||||
|
||||
- [ ] Tippfehler/Grammatik
|
||||
- [ ] Fehlende Erklärung
|
||||
- [ ] Veraltetes Beispiel
|
||||
- [ ] Neues Beispiel hinzufügen
|
||||
- [ ] Struktur/Navigation verbessern
|
||||
- [ ] API-Dokumentation erweitern
|
||||
- [ ] Troubleshooting-Sektion
|
||||
|
||||
## Betroffene Nutzergruppe
|
||||
|
||||
<!-- Wer profitiert von dieser Verbesserung? -->
|
||||
|
||||
- [ ] Neueinsteiger
|
||||
- [ ] Fortgeschrittene Nutzer
|
||||
- [ ] Developers/Contributors
|
||||
- [ ] Alle
|
||||
|
||||
## Vorschlag (Optional)
|
||||
|
||||
<!-- Hast du bereits einen konkreten Formulierungsvorschlag? -->
|
||||
|
||||
<details>
|
||||
<summary>Vorgeschlagener Text (klicken zum Aufklappen)</summary>
|
||||
|
||||
```markdown
|
||||
Hier den verbesserten Text einfügen
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
## Zusätzliche Kontext
|
||||
|
||||
<!-- Weitere Informationen -->
|
||||
@@ -0,0 +1,91 @@
|
||||
# Pull Request
|
||||
|
||||
## Beschreibung
|
||||
|
||||
<!--
|
||||
Eine klare und prägnante Beschreibung der Änderungen.
|
||||
Beziehe dich auf das zugehörige Issue (falls vorhanden).
|
||||
-->
|
||||
|
||||
**Fixes:** #<!-- Issue-Nummer -->
|
||||
|
||||
## Typ
|
||||
|
||||
<!-- Bitte zutreffendes ankreuzen [x] -->
|
||||
|
||||
- [ ] 🐛 Bug Fix
|
||||
- [ ] ✨ Neue Funktion
|
||||
- [ ] 📚 Dokumentation
|
||||
- [ ] 🧹 Code Cleanup/Refactoring
|
||||
- [ ] ⚡ Performance-Verbesserung
|
||||
- [ ] 🔧 Konfiguration/Build
|
||||
- [ ] 🧪 Tests
|
||||
|
||||
## Changes
|
||||
|
||||
<!-- Welche Dateien wurden geändert und warum? -->
|
||||
|
||||
- `Datei1.py`: Beschreibung der Änderung
|
||||
- `Datei2.py`: Beschreibung der Änderung
|
||||
|
||||
## Testing
|
||||
|
||||
<!-- Wie wurden die Änderungen getestet? -->
|
||||
|
||||
### Tests hinzugefügt/aktualisiert
|
||||
|
||||
- [ ] Ja, Unit Tests
|
||||
- [ ] Ja, Integration Tests
|
||||
- [ ] Nein, aber manuell getestet
|
||||
- [ ] Nicht zutreffend
|
||||
|
||||
### Testing Notes
|
||||
|
||||
<!-- Beschreibe deine Testing-Schritte -->
|
||||
|
||||
```bash
|
||||
# Beispiel: Tests ausführen
|
||||
pytest test/ -v --cov=rdagent
|
||||
|
||||
# Beispiel: CLI Command testen
|
||||
rdagent COMMAND --help
|
||||
```
|
||||
|
||||
## Checklist
|
||||
|
||||
<!-- Bitte alle zutreffenden Punkte ankreuzen [x] -->
|
||||
|
||||
- [ ] Meine Änderungen folgen dem [Coding Style](CONTRIBUTING.md)
|
||||
- [ ] Ich habe [CONTRIBUTING.md](CONTRIBUTING.md) gelesen und befolgt
|
||||
- [ ] Tests wurden hinzugefügt oder aktualisiert
|
||||
- [ ] Dokumentation wurde aktualisiert (`docs/` oder README.md)
|
||||
- [ ] CHANGELOG.md wurde aktualisiert (falls zutreffend)
|
||||
- [ ] Pre-commit Hooks bestanden (`pre-commit run --all-files`)
|
||||
- [ ] Keine closed-source Assets committen (siehe unten)
|
||||
|
||||
## ⚠️ Closed-Source Check
|
||||
|
||||
<!--
|
||||
KRITISCH: Bitte bestätige, dass KEINE der folgenden Dateien committen wurden:
|
||||
-->
|
||||
|
||||
- [ ] `git_ignore_folder/` – Trading-Skripte, OHLCV-Daten, Credentials
|
||||
- [ ] `results/` – Backtest-Ergebnisse, Strategien, Logs
|
||||
- [ ] `.env` – API-Keys, Credentials
|
||||
- [ ] `models/local/` – Eigene verbesserte Modelle
|
||||
- [ ] `prompts/local/` – Eigene verbesserte Prompts
|
||||
- [ ] `rdagent/scenarios/qlib/local/` – Closed-Source Komponenten
|
||||
- [ ] `*.db` – SQLite-Datenbanken
|
||||
- [ ] `*.log` – Log-Files
|
||||
|
||||
## Screenshots (falls relevant)
|
||||
|
||||
<!-- Vorher/Nachher-Vergleiche, UI-Änderungen etc. -->
|
||||
|
||||
| Vorher | Nachher |
|
||||
|--------|---------|
|
||||
| <!-- Screenshot --> | <!-- Screenshot --> |
|
||||
|
||||
## Zusätzliche Kontext
|
||||
|
||||
<!-- Weitere Informationen zu den Änderungen -->
|
||||
@@ -0,0 +1,84 @@
|
||||
name: CI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ main, develop ]
|
||||
paths-ignore:
|
||||
- '**.md'
|
||||
- 'docs/**'
|
||||
- 'LICENSE'
|
||||
pull_request:
|
||||
branches: [ main ]
|
||||
paths-ignore:
|
||||
- '**.md'
|
||||
- 'docs/**'
|
||||
- 'LICENSE'
|
||||
|
||||
env:
|
||||
PYTHONUNBUFFERED: "1"
|
||||
|
||||
jobs:
|
||||
test:
|
||||
name: Test (Python ${{ matrix.python-version }}, ${{ matrix.os }})
|
||||
runs-on: ${{ matrix.os }}
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
python-version: ["3.10", "3.11"]
|
||||
os: [ubuntu-latest]
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
|
||||
- name: Cache pip dependencies
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ~/.cache/pip
|
||||
key: ${{ runner.os }}-py${{ matrix.python-version }}-pip-${{ hashFiles('**/requirements.txt', '**/pyproject.toml') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-py${{ matrix.python-version }}-pip-
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -e ".[test]"
|
||||
pip install -r requirements.txt
|
||||
|
||||
- name: List installed packages
|
||||
run: pip list
|
||||
|
||||
- name: Run tests with coverage
|
||||
run: |
|
||||
pytest test/ \
|
||||
-v \
|
||||
--cov=rdagent \
|
||||
--cov-report=xml \
|
||||
--cov-report=html \
|
||||
--cov-report=term-missing \
|
||||
--durations=10 \
|
||||
-x
|
||||
|
||||
- name: Upload coverage to Codecov
|
||||
if: matrix.python-version == '3.10' && github.ref == 'refs/heads/main'
|
||||
uses: codecov/codecov-action@v4
|
||||
with:
|
||||
file: ./coverage.xml
|
||||
flags: unittests
|
||||
name: codecov-umbrella
|
||||
fail_ci_if_error: false
|
||||
env:
|
||||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
|
||||
|
||||
- name: Upload coverage HTML report
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: coverage-html-report-py${{ matrix.python-version }}
|
||||
path: htmlcov/
|
||||
retention-days: 7
|
||||
@@ -0,0 +1,83 @@
|
||||
name: Documentation
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ main ]
|
||||
paths:
|
||||
- 'docs/**'
|
||||
- 'README.md'
|
||||
- '**/*.rst'
|
||||
- '.github/workflows/docs.yml'
|
||||
pull_request:
|
||||
branches: [ main ]
|
||||
paths:
|
||||
- 'docs/**'
|
||||
- 'README.md'
|
||||
- '**/*.rst'
|
||||
|
||||
jobs:
|
||||
docs:
|
||||
name: Build Documentation
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.10"
|
||||
|
||||
- name: Cache pip dependencies
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ~/.cache/pip
|
||||
key: ${{ runner.os }}-pip-docs-${{ hashFiles('**/pyproject.toml') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-pip-docs-
|
||||
|
||||
- name: Install docs dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -e ".[docs]"
|
||||
|
||||
- name: Build Sphinx documentation
|
||||
run: |
|
||||
cd docs
|
||||
make clean
|
||||
make html SPHINXOPTS="-W --keep-going" || {
|
||||
echo "::error::Sphinx build failed with warnings"
|
||||
exit 1
|
||||
}
|
||||
|
||||
- name: Check for broken links
|
||||
run: |
|
||||
cd docs
|
||||
make linkcheck || {
|
||||
echo "::warning::Some links are broken (non-blocking)"
|
||||
exit 0
|
||||
}
|
||||
|
||||
- name: Upload docs artifact
|
||||
if: github.ref == 'refs/heads/main'
|
||||
uses: actions/upload-pages-artifact@v3
|
||||
with:
|
||||
path: docs/_build/html
|
||||
|
||||
deploy:
|
||||
name: Deploy to GitHub Pages
|
||||
needs: docs
|
||||
if: github.ref == 'refs/heads/main'
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
pages: write
|
||||
id-token: write
|
||||
environment:
|
||||
name: github-pages
|
||||
url: ${{ steps.deployment.outputs.page_url }}
|
||||
|
||||
steps:
|
||||
- name: Deploy to GitHub Pages
|
||||
id: deployment
|
||||
uses: actions/deploy-pages@v4
|
||||
@@ -0,0 +1,81 @@
|
||||
name: Code Quality
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ main, develop ]
|
||||
pull_request:
|
||||
branches: [ main ]
|
||||
|
||||
jobs:
|
||||
lint:
|
||||
name: Lint & Format
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.10"
|
||||
|
||||
- name: Cache pip dependencies
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ~/.cache/pip
|
||||
key: ${{ runner.os }}-pip-lint-${{ hashFiles('**/pyproject.toml') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-pip-lint-
|
||||
|
||||
- name: Install lint dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install ruff mypy
|
||||
|
||||
- name: Run Ruff (linter)
|
||||
run: |
|
||||
echo "=== Running Ruff Linter ==="
|
||||
ruff check . --statistics || {
|
||||
echo "::error::Ruff linter found issues. Run: ruff check . --fix"
|
||||
exit 1
|
||||
}
|
||||
|
||||
- name: Run Ruff (formatter)
|
||||
run: |
|
||||
echo "=== Running Ruff Formatter ==="
|
||||
ruff format --check . || {
|
||||
echo "::error::Ruff formatter found issues. Run: ruff format ."
|
||||
exit 1
|
||||
}
|
||||
|
||||
- name: Run MyPy (type checker)
|
||||
run: |
|
||||
echo "=== Running MyPy Type Checker ==="
|
||||
mypy rdagent/ \
|
||||
--ignore-missing-imports \
|
||||
--no-strict-optional \
|
||||
--follow-imports=skip \
|
||||
--warn-return-any || {
|
||||
echo "::warning::MyPy found type issues (non-blocking)"
|
||||
# Non-blocking: MyPy warnings don't fail the build
|
||||
exit 0
|
||||
}
|
||||
|
||||
- name: Check for trailing whitespace
|
||||
run: |
|
||||
echo "=== Checking for trailing whitespace ==="
|
||||
if grep -rIn '[[:space:]]$' --include='*.py' --include='*.md' --include='*.rst' . | grep -v '.git'; then
|
||||
echo "::error::Found trailing whitespace. Please remove it."
|
||||
exit 1
|
||||
fi
|
||||
echo "✓ No trailing whitespace found"
|
||||
|
||||
- name: Check for merge conflicts
|
||||
run: |
|
||||
echo "=== Checking for merge conflict markers ==="
|
||||
if grep -rn '<<<<<<< HEAD\|=======\|>>>>>>>' --include='*.py' --include='*.md' . | grep -v '.git'; then
|
||||
echo "::error::Found merge conflict markers. Please resolve them."
|
||||
exit 1
|
||||
fi
|
||||
echo "✓ No merge conflict markers found"
|
||||
@@ -0,0 +1,97 @@
|
||||
name: Release
|
||||
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- 'v*'
|
||||
- 'V*'
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
packages: write
|
||||
|
||||
jobs:
|
||||
release:
|
||||
name: Create Release
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.10"
|
||||
|
||||
- name: Install build tools
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install build twine
|
||||
|
||||
- name: Build package
|
||||
run: python -m build
|
||||
|
||||
- name: Check package
|
||||
run: twine check dist/*
|
||||
|
||||
- name: Verify no closed-source assets
|
||||
run: |
|
||||
echo "=== Verifying Release Package ==="
|
||||
|
||||
# Extract and check contents
|
||||
python -c "
|
||||
import tarfile
|
||||
import sys
|
||||
|
||||
with tarfile.open('dist/' + [f for f in __import__('os').listdir('dist') if f.endswith('.tar.gz')][0], 'r:gz') as tar:
|
||||
names = tar.getnames()
|
||||
closed_patterns = ['git_ignore_folder', 'results/', '.env', 'models/local', 'prompts/local']
|
||||
|
||||
found = False
|
||||
for name in names:
|
||||
for pattern in closed_patterns:
|
||||
if pattern in name:
|
||||
print(f'ERROR: Found closed-source asset: {name}')
|
||||
found = True
|
||||
|
||||
if found:
|
||||
sys.exit(1)
|
||||
print('✓ No closed-source assets in package')
|
||||
"
|
||||
|
||||
- name: Generate release notes
|
||||
id: release_notes
|
||||
run: |
|
||||
# Try to find changelog for this version
|
||||
VERSION=${GITHUB_REF#refs/tags/}
|
||||
CHANGELOG_FILE="changelog/${VERSION}.md"
|
||||
|
||||
if [ -f "$CHANGELOG_FILE" ]; then
|
||||
echo "body_path=$CHANGELOG_FILE" >> $GITHUB_OUTPUT
|
||||
elif [ -f "CHANGELOG.md" ]; then
|
||||
# Extract section for this version from main changelog
|
||||
echo "body_path=CHANGELOG.md" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "body_path=" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Create GitHub Release
|
||||
uses: softprops/action-gh-release@v2
|
||||
with:
|
||||
body_path: ${{ steps.release_notes.outputs.body_path || '' }}
|
||||
files: dist/*
|
||||
draft: false
|
||||
prerelease: false
|
||||
generate_release_notes: true
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Publish to PyPI
|
||||
if: startsWith(github.ref, 'refs/tags/v')
|
||||
env:
|
||||
TWINE_USERNAME: __token__
|
||||
TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
|
||||
run: twine upload dist/*
|
||||
@@ -0,0 +1,135 @@
|
||||
name: Security Scan
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ main ]
|
||||
pull_request:
|
||||
branches: [ main ]
|
||||
schedule:
|
||||
# Weekly on Monday at 6:00 UTC
|
||||
- cron: '0 6 * * 1'
|
||||
|
||||
jobs:
|
||||
security:
|
||||
name: Security Analysis
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.10"
|
||||
|
||||
- name: Cache pip dependencies
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ~/.cache/pip
|
||||
key: ${{ runner.os }}-pip-security-${{ hashFiles('**/requirements.txt') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-pip-security-
|
||||
|
||||
- name: Install security tools
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install bandit safety
|
||||
|
||||
- name: Run Bandit (code security)
|
||||
run: |
|
||||
echo "=== Running Bandit Security Scan ==="
|
||||
bandit \
|
||||
-c .bandit.yml \
|
||||
-r rdagent/ \
|
||||
-f json \
|
||||
-o bandit-report.json \
|
||||
--exit-zero || true
|
||||
|
||||
# Show summary
|
||||
bandit -c .bandit.yml -r rdagent/ -ll || true
|
||||
|
||||
- name: Upload Bandit report
|
||||
uses: actions/upload-artifact@v4
|
||||
if: always()
|
||||
with:
|
||||
name: bandit-security-report
|
||||
path: bandit-report.json
|
||||
retention-days: 30
|
||||
|
||||
- name: Check dependencies for vulnerabilities
|
||||
run: |
|
||||
echo "=== Checking Dependencies for Vulnerabilities ==="
|
||||
safety check --json || {
|
||||
echo "::warning::Some dependencies have known vulnerabilities"
|
||||
echo "Please review and update dependencies."
|
||||
exit 0 # Non-blocking
|
||||
}
|
||||
|
||||
- name: Check for exposed secrets
|
||||
run: |
|
||||
echo "=== Scanning for Exposed Secrets ==="
|
||||
|
||||
# Check for common secret patterns
|
||||
PATTERNS=(
|
||||
"api_key\s*=\s*['\"][^'\"]+['\"]"
|
||||
"secret\s*=\s*['\"][^'\"]+['\"]"
|
||||
"password\s*=\s*['\"][^'\"]+['\"]"
|
||||
"token\s*=\s*['\"][^'\"]+['\"]"
|
||||
"PRIVATE.KEY"
|
||||
"BEGIN RSA PRIVATE KEY"
|
||||
)
|
||||
|
||||
FOUND_SECRETS=0
|
||||
for pattern in "${PATTERNS[@]}"; do
|
||||
if grep -rInE "$pattern" --include='*.py' --include='*.yml' --include='*.yaml' --include='*.json' . | \
|
||||
grep -v '.git' | \
|
||||
grep -v 'test/' | \
|
||||
grep -v 'example' | \
|
||||
grep -v '# ' | \
|
||||
grep -v 'os.environ' | \
|
||||
grep -v 'getenv' | \
|
||||
grep -v 'argparse'; then
|
||||
FOUND_SECRETS=1
|
||||
fi
|
||||
done
|
||||
|
||||
if [ $FOUND_SECRETS -eq 1 ]; then
|
||||
echo "::error::Potential secrets exposure detected!"
|
||||
echo "Please review the output above and remove any hardcoded credentials."
|
||||
echo "Use environment variables or .env files instead."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "✓ No exposed secrets found"
|
||||
|
||||
- name: Verify closed-source files not committed
|
||||
run: |
|
||||
echo "=== Verifying No Closed-Source Assets Committed ==="
|
||||
|
||||
CLOSED_PATTERNS=(
|
||||
"git_ignore_folder/"
|
||||
"results/"
|
||||
".env"
|
||||
"models/local/"
|
||||
"prompts/local/"
|
||||
"rdagent/scenarios/qlib/local/"
|
||||
"*.db"
|
||||
"*.log"
|
||||
)
|
||||
|
||||
FOUND_CLOSED=0
|
||||
for pattern in "${CLOSED_PATTERNS[@]}"; do
|
||||
if git ls-files | grep -q "$pattern"; then
|
||||
echo "::error::Found closed-source asset: $pattern"
|
||||
FOUND_CLOSED=1
|
||||
fi
|
||||
done
|
||||
|
||||
if [ $FOUND_CLOSED -eq 1 ]; then
|
||||
echo "CRITICAL: Closed-source assets must not be committed to the repository!"
|
||||
echo "Please remove them and add to .gitignore if needed."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "✓ No closed-source assets found"
|
||||
Reference in New Issue
Block a user