feat: Add GitHub infrastructure, CI/CD pipelines, and examples

- Add GitHub issue templates (bug, feature, docs)
- Add pull request template with closed-source checklist
- Add CODEOWNERS for code review assignment
- Add CI/CD workflows (ci, lint, security, docs, release)
  - pytest + coverage with Python 3.10/3.11 matrix
  - Ruff + MyPy code quality checks
  - Bandit + safety security scanning
  - Sphinx docs + GitHub Pages deployment
  - Automated PyPI releases on tag push
- Add 6 comprehensive examples + Jupyter quickstart
  - 01_factor_discovery.py (LLM factor generation)
  - 02_factor_evolution.py (factor optimization)
  - 03_strategy_generation.py (IC-weighted combination)
  - 04_backtest_simple.py (strategy backtesting)
  - 05_model_training.py (XGBoost/LSTM training)
  - 06_rl_trading_agent.py (PPO/DQN/A2C agents)
  - notebooks/quickstart.ipynb (interactive tutorial)
- Restructure .gitignore with explicit closed-source sections
- Add CI/coverage/license badges to README
- Complete CLI docstrings for all 9 commands
- Add data_config.yaml for quant loop configuration
This commit is contained in:
TPTBusiness
2026-04-11 21:40:18 +02:00
parent b42627c335
commit 5738e47ffa
23 changed files with 3275 additions and 175 deletions
+42
View File
@@ -0,0 +1,42 @@
# CODEOWNERS
# Diese Datei definiert die Verantwortlichen für Code-Reviews
# Siehe: https://docs.github.com/en/repositories/working-with-files/managing-files/about-code-owners
# Core Maintainer (Standard-Reviewer für alle Änderungen)
* @nico
# RD-Agent Core-Module
/rdagent/core/ @nico
/rdagent/components/ @nico
/rdagent/app/ @nico
# Trading-Spezifika
/rdagent/scenarios/ @nico
/prompts/ @nico
# Dokumentation
/docs/ @nico
/README.md @nico
/examples/ @nico
/CONTRIBUTING.md @nico
/CODE_OF_CONDUCT.md @nico
# Konfiguration & Build
/pyproject.toml @nico
/requirements.txt @nico
/setup.py @nico
/Makefile @nico
# CI/CD & Security
/.github/ @nico
/.pre-commit-config.yaml @nico
/.bandit.yml @nico
/SECURITY.md @nico
# Dashboard & Visualization
/dashboard/ @nico
/web/ @nico
# Data Pipeline
/data/ @nico
/scripts/download*.py @nico
+58
View File
@@ -0,0 +1,58 @@
---
name: 🐛 Bug Report
about: Create a report to help us improve PREDIX
title: '[Bug] '
labels: 'bug, needs-triage'
assignees: ''
---
## Beschreibung
<!-- Eine klare und prägnante Beschreibung des Bugs -->
## Reproduktionsschritte
<!-- Schritte zum Reproduzieren des Verhaltens -->
1. Schritt 1: `...`
2. Schritt 2: `...`
3. Schritt 3: `...`
4. Fehler tritt auf
## Erwartetes Verhalten
<!-- Eine klare Beschreibung dessen, was passieren sollte -->
## Tatsächliches Verhalten
<!-- Was passiert tatsächlich? -->
## Environment
<!-- Bitte fülle die folgenden Informationen aus -->
- **OS:** [z.B. Linux, macOS, Windows]
- **Python-Version:** [z.B. 3.10, 3.11]
- **PREDIX-Version:** [z.B. v2.0.0, main-branch]
- **Installation:** [z.B. pip, conda, from source]
## Logs & Screenshots
<!-- Füge relevante Logs oder Screenshots hinzu -->
<details>
<summary>Log Output (klicken zum Aufklappen)</summary>
```
Hier die Log-Ausgabe einfügen
```
</details>
## Zusätzliche Kontext
<!-- Weitere Informationen zum Problem -->
### Data Configuration
- [ ] Ich habe sichergestellt, dass die Daten korrekt geladen sind
- [ ] `qlib init` wurde erfolgreich ausgeführt
### Workaround
<!-- Falls vorhanden: Gibt es einen Workaround? -->
@@ -0,0 +1,47 @@
---
name: 💡 Feature Request
about: Suggest an idea for PREDIX
title: '[Feature] '
labels: 'enhancement, needs-triage'
assignees: ''
---
## Problem-Beschreibung
<!-- Bezieht sich dein Feature auf ein Problem? Bitte beschreibe es -->
<!-- Beispiel: "Ich bin immer frustriert, wenn ich..." -->
## Lösungsvorschlag
<!-- Eine klare und prägnante Beschreibung dessen, was du gerne hättest -->
## Alternativen
<!-- Hast du alternative Lösungen in Betracht gezogen? -->
## Zusätzliche Kontext
<!-- Weitere Informationen, Screenshots oder Mockups -->
## Use Case
<!-- Wie würde dieses Feature deinen Workflow verbessern? -->
### Checkliste
<!-- Bitte bestätige die folgenden Punkte mit [x] -->
- [ ] Ich habe die [Dokumentation](https://github.com/nico/Predix/tree/main/docs) gelesen
- [ ] Ich habe geprüft, ob dieses Feature bereits als [bestehendes Issue](https://github.com/nico/Predix/issues) existiert
- [ ] Dieses Feature ist relevant für **Open-Source** (keine closed-source Komponenten)
## Impact
<!-- Wer würde von diesem Feature profitieren? -->
- [ ] Alle PREDIX-Nutzer
- [ ] Spezifische Nutzer (z.B. FX-Trader, Qlib-Nutzer)
- [ ] Entwickler/Contributors
## Priorität
<!-- Wie dringend ist dieses Feature? -->
- [ ] Niedrig (Nice-to-have)
- [ ] Mittel (Würde den Workflow verbessern)
- [ ] Hoch (Blockiert meine Arbeit)
@@ -0,0 +1,58 @@
---
name: 📚 Documentation Improvement
about: Suggest improvements to PREDIX documentation
title: '[Docs] '
labels: 'documentation'
assignees: ''
---
## Aktueller Zustand
<!-- Welche Seite/Welcher Teil der Dokumentation ist betroffen? -->
**URL/Datei:** `z.B. README.md, docs/quickstart.rst`
**Aktueller Inhalt:**
<!-- Zitat oder Beschreibung des aktuellen Zustands -->
## Verbesserungsvorschlag
<!-- Was sollte geändert/hinzugefügt werden? -->
## Beispiel/Begründung
<!-- Warum ist diese Verbesserung notwendig? -->
### Art der Verbesserung
- [ ] Tippfehler/Grammatik
- [ ] Fehlende Erklärung
- [ ] Veraltetes Beispiel
- [ ] Neues Beispiel hinzufügen
- [ ] Struktur/Navigation verbessern
- [ ] API-Dokumentation erweitern
- [ ] Troubleshooting-Sektion
## Betroffene Nutzergruppe
<!-- Wer profitiert von dieser Verbesserung? -->
- [ ] Neueinsteiger
- [ ] Fortgeschrittene Nutzer
- [ ] Developers/Contributors
- [ ] Alle
## Vorschlag (Optional)
<!-- Hast du bereits einen konkreten Formulierungsvorschlag? -->
<details>
<summary>Vorgeschlagener Text (klicken zum Aufklappen)</summary>
```markdown
Hier den verbesserten Text einfügen
```
</details>
## Zusätzliche Kontext
<!-- Weitere Informationen -->
+91
View File
@@ -0,0 +1,91 @@
# Pull Request
## Beschreibung
<!--
Eine klare und prägnante Beschreibung der Änderungen.
Beziehe dich auf das zugehörige Issue (falls vorhanden).
-->
**Fixes:** #<!-- Issue-Nummer -->
## Typ
<!-- Bitte zutreffendes ankreuzen [x] -->
- [ ] 🐛 Bug Fix
- [ ] ✨ Neue Funktion
- [ ] 📚 Dokumentation
- [ ] 🧹 Code Cleanup/Refactoring
- [ ] ⚡ Performance-Verbesserung
- [ ] 🔧 Konfiguration/Build
- [ ] 🧪 Tests
## Changes
<!-- Welche Dateien wurden geändert und warum? -->
- `Datei1.py`: Beschreibung der Änderung
- `Datei2.py`: Beschreibung der Änderung
## Testing
<!-- Wie wurden die Änderungen getestet? -->
### Tests hinzugefügt/aktualisiert
- [ ] Ja, Unit Tests
- [ ] Ja, Integration Tests
- [ ] Nein, aber manuell getestet
- [ ] Nicht zutreffend
### Testing Notes
<!-- Beschreibe deine Testing-Schritte -->
```bash
# Beispiel: Tests ausführen
pytest test/ -v --cov=rdagent
# Beispiel: CLI Command testen
rdagent COMMAND --help
```
## Checklist
<!-- Bitte alle zutreffenden Punkte ankreuzen [x] -->
- [ ] Meine Änderungen folgen dem [Coding Style](CONTRIBUTING.md)
- [ ] Ich habe [CONTRIBUTING.md](CONTRIBUTING.md) gelesen und befolgt
- [ ] Tests wurden hinzugefügt oder aktualisiert
- [ ] Dokumentation wurde aktualisiert (`docs/` oder README.md)
- [ ] CHANGELOG.md wurde aktualisiert (falls zutreffend)
- [ ] Pre-commit Hooks bestanden (`pre-commit run --all-files`)
- [ ] Keine closed-source Assets committen (siehe unten)
## ⚠️ Closed-Source Check
<!--
KRITISCH: Bitte bestätige, dass KEINE der folgenden Dateien committen wurden:
-->
- [ ] `git_ignore_folder/` Trading-Skripte, OHLCV-Daten, Credentials
- [ ] `results/` Backtest-Ergebnisse, Strategien, Logs
- [ ] `.env` API-Keys, Credentials
- [ ] `models/local/` Eigene verbesserte Modelle
- [ ] `prompts/local/` Eigene verbesserte Prompts
- [ ] `rdagent/scenarios/qlib/local/` Closed-Source Komponenten
- [ ] `*.db` SQLite-Datenbanken
- [ ] `*.log` Log-Files
## Screenshots (falls relevant)
<!-- Vorher/Nachher-Vergleiche, UI-Änderungen etc. -->
| Vorher | Nachher |
|--------|---------|
| <!-- Screenshot --> | <!-- Screenshot --> |
## Zusätzliche Kontext
<!-- Weitere Informationen zu den Änderungen -->
+84
View File
@@ -0,0 +1,84 @@
name: CI
on:
push:
branches: [ main, develop ]
paths-ignore:
- '**.md'
- 'docs/**'
- 'LICENSE'
pull_request:
branches: [ main ]
paths-ignore:
- '**.md'
- 'docs/**'
- 'LICENSE'
env:
PYTHONUNBUFFERED: "1"
jobs:
test:
name: Test (Python ${{ matrix.python-version }}, ${{ matrix.os }})
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11"]
os: [ubuntu-latest]
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Cache pip dependencies
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: ${{ runner.os }}-py${{ matrix.python-version }}-pip-${{ hashFiles('**/requirements.txt', '**/pyproject.toml') }}
restore-keys: |
${{ runner.os }}-py${{ matrix.python-version }}-pip-
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -e ".[test]"
pip install -r requirements.txt
- name: List installed packages
run: pip list
- name: Run tests with coverage
run: |
pytest test/ \
-v \
--cov=rdagent \
--cov-report=xml \
--cov-report=html \
--cov-report=term-missing \
--durations=10 \
-x
- name: Upload coverage to Codecov
if: matrix.python-version == '3.10' && github.ref == 'refs/heads/main'
uses: codecov/codecov-action@v4
with:
file: ./coverage.xml
flags: unittests
name: codecov-umbrella
fail_ci_if_error: false
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
- name: Upload coverage HTML report
if: always()
uses: actions/upload-artifact@v4
with:
name: coverage-html-report-py${{ matrix.python-version }}
path: htmlcov/
retention-days: 7
+83
View File
@@ -0,0 +1,83 @@
name: Documentation
on:
push:
branches: [ main ]
paths:
- 'docs/**'
- 'README.md'
- '**/*.rst'
- '.github/workflows/docs.yml'
pull_request:
branches: [ main ]
paths:
- 'docs/**'
- 'README.md'
- '**/*.rst'
jobs:
docs:
name: Build Documentation
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.10"
- name: Cache pip dependencies
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-docs-${{ hashFiles('**/pyproject.toml') }}
restore-keys: |
${{ runner.os }}-pip-docs-
- name: Install docs dependencies
run: |
python -m pip install --upgrade pip
pip install -e ".[docs]"
- name: Build Sphinx documentation
run: |
cd docs
make clean
make html SPHINXOPTS="-W --keep-going" || {
echo "::error::Sphinx build failed with warnings"
exit 1
}
- name: Check for broken links
run: |
cd docs
make linkcheck || {
echo "::warning::Some links are broken (non-blocking)"
exit 0
}
- name: Upload docs artifact
if: github.ref == 'refs/heads/main'
uses: actions/upload-pages-artifact@v3
with:
path: docs/_build/html
deploy:
name: Deploy to GitHub Pages
needs: docs
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
permissions:
pages: write
id-token: write
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
+81
View File
@@ -0,0 +1,81 @@
name: Code Quality
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main ]
jobs:
lint:
name: Lint & Format
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.10"
- name: Cache pip dependencies
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-lint-${{ hashFiles('**/pyproject.toml') }}
restore-keys: |
${{ runner.os }}-pip-lint-
- name: Install lint dependencies
run: |
python -m pip install --upgrade pip
pip install ruff mypy
- name: Run Ruff (linter)
run: |
echo "=== Running Ruff Linter ==="
ruff check . --statistics || {
echo "::error::Ruff linter found issues. Run: ruff check . --fix"
exit 1
}
- name: Run Ruff (formatter)
run: |
echo "=== Running Ruff Formatter ==="
ruff format --check . || {
echo "::error::Ruff formatter found issues. Run: ruff format ."
exit 1
}
- name: Run MyPy (type checker)
run: |
echo "=== Running MyPy Type Checker ==="
mypy rdagent/ \
--ignore-missing-imports \
--no-strict-optional \
--follow-imports=skip \
--warn-return-any || {
echo "::warning::MyPy found type issues (non-blocking)"
# Non-blocking: MyPy warnings don't fail the build
exit 0
}
- name: Check for trailing whitespace
run: |
echo "=== Checking for trailing whitespace ==="
if grep -rIn '[[:space:]]$' --include='*.py' --include='*.md' --include='*.rst' . | grep -v '.git'; then
echo "::error::Found trailing whitespace. Please remove it."
exit 1
fi
echo "✓ No trailing whitespace found"
- name: Check for merge conflicts
run: |
echo "=== Checking for merge conflict markers ==="
if grep -rn '<<<<<<< HEAD\|=======\|>>>>>>>' --include='*.py' --include='*.md' . | grep -v '.git'; then
echo "::error::Found merge conflict markers. Please resolve them."
exit 1
fi
echo "✓ No merge conflict markers found"
+97
View File
@@ -0,0 +1,97 @@
name: Release
on:
push:
tags:
- 'v*'
- 'V*'
permissions:
contents: write
packages: write
jobs:
release:
name: Create Release
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.10"
- name: Install build tools
run: |
python -m pip install --upgrade pip
pip install build twine
- name: Build package
run: python -m build
- name: Check package
run: twine check dist/*
- name: Verify no closed-source assets
run: |
echo "=== Verifying Release Package ==="
# Extract and check contents
python -c "
import tarfile
import sys
with tarfile.open('dist/' + [f for f in __import__('os').listdir('dist') if f.endswith('.tar.gz')][0], 'r:gz') as tar:
names = tar.getnames()
closed_patterns = ['git_ignore_folder', 'results/', '.env', 'models/local', 'prompts/local']
found = False
for name in names:
for pattern in closed_patterns:
if pattern in name:
print(f'ERROR: Found closed-source asset: {name}')
found = True
if found:
sys.exit(1)
print('✓ No closed-source assets in package')
"
- name: Generate release notes
id: release_notes
run: |
# Try to find changelog for this version
VERSION=${GITHUB_REF#refs/tags/}
CHANGELOG_FILE="changelog/${VERSION}.md"
if [ -f "$CHANGELOG_FILE" ]; then
echo "body_path=$CHANGELOG_FILE" >> $GITHUB_OUTPUT
elif [ -f "CHANGELOG.md" ]; then
# Extract section for this version from main changelog
echo "body_path=CHANGELOG.md" >> $GITHUB_OUTPUT
else
echo "body_path=" >> $GITHUB_OUTPUT
fi
- name: Create GitHub Release
uses: softprops/action-gh-release@v2
with:
body_path: ${{ steps.release_notes.outputs.body_path || '' }}
files: dist/*
draft: false
prerelease: false
generate_release_notes: true
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Publish to PyPI
if: startsWith(github.ref, 'refs/tags/v')
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
run: twine upload dist/*
+135
View File
@@ -0,0 +1,135 @@
name: Security Scan
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
schedule:
# Weekly on Monday at 6:00 UTC
- cron: '0 6 * * 1'
jobs:
security:
name: Security Analysis
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.10"
- name: Cache pip dependencies
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-security-${{ hashFiles('**/requirements.txt') }}
restore-keys: |
${{ runner.os }}-pip-security-
- name: Install security tools
run: |
python -m pip install --upgrade pip
pip install bandit safety
- name: Run Bandit (code security)
run: |
echo "=== Running Bandit Security Scan ==="
bandit \
-c .bandit.yml \
-r rdagent/ \
-f json \
-o bandit-report.json \
--exit-zero || true
# Show summary
bandit -c .bandit.yml -r rdagent/ -ll || true
- name: Upload Bandit report
uses: actions/upload-artifact@v4
if: always()
with:
name: bandit-security-report
path: bandit-report.json
retention-days: 30
- name: Check dependencies for vulnerabilities
run: |
echo "=== Checking Dependencies for Vulnerabilities ==="
safety check --json || {
echo "::warning::Some dependencies have known vulnerabilities"
echo "Please review and update dependencies."
exit 0 # Non-blocking
}
- name: Check for exposed secrets
run: |
echo "=== Scanning for Exposed Secrets ==="
# Check for common secret patterns
PATTERNS=(
"api_key\s*=\s*['\"][^'\"]+['\"]"
"secret\s*=\s*['\"][^'\"]+['\"]"
"password\s*=\s*['\"][^'\"]+['\"]"
"token\s*=\s*['\"][^'\"]+['\"]"
"PRIVATE.KEY"
"BEGIN RSA PRIVATE KEY"
)
FOUND_SECRETS=0
for pattern in "${PATTERNS[@]}"; do
if grep -rInE "$pattern" --include='*.py' --include='*.yml' --include='*.yaml' --include='*.json' . | \
grep -v '.git' | \
grep -v 'test/' | \
grep -v 'example' | \
grep -v '# ' | \
grep -v 'os.environ' | \
grep -v 'getenv' | \
grep -v 'argparse'; then
FOUND_SECRETS=1
fi
done
if [ $FOUND_SECRETS -eq 1 ]; then
echo "::error::Potential secrets exposure detected!"
echo "Please review the output above and remove any hardcoded credentials."
echo "Use environment variables or .env files instead."
exit 1
fi
echo "✓ No exposed secrets found"
- name: Verify closed-source files not committed
run: |
echo "=== Verifying No Closed-Source Assets Committed ==="
CLOSED_PATTERNS=(
"git_ignore_folder/"
"results/"
".env"
"models/local/"
"prompts/local/"
"rdagent/scenarios/qlib/local/"
"*.db"
"*.log"
)
FOUND_CLOSED=0
for pattern in "${CLOSED_PATTERNS[@]}"; do
if git ls-files | grep -q "$pattern"; then
echo "::error::Found closed-source asset: $pattern"
FOUND_CLOSED=1
fi
done
if [ $FOUND_CLOSED -eq 1 ]; then
echo "CRITICAL: Closed-source assets must not be committed to the repository!"
echo "Please remove them and add to .gitignore if needed."
exit 1
fi
echo "✓ No closed-source assets found"