288b1546b2
* feat: broaden CPU and platform coverage across PyPI, nodes, and crates.io Replace static SIMD with runtime CPU-feature dispatch and expand the release wheel matrix so one set of artifacts runs on any target CPU and platform without illegal-instruction crashes. Rust core: - Add multiversion runtime dispatch (crates/ferro_ta_core/src/simd.rs); drop compile-time `wide`. `simd` feature is now default-on and forwarded through the pyo3 crate, and stays compatible with #![forbid(unsafe_code)]. Packaging: - abi3-py310: one cp310-abi3 wheel per platform (covers CPython 3.10+). - CI matrix adds Linux aarch64 + musllinux (x86_64/aarch64) and Windows arm64. Node/Docker + docs: - api/Dockerfile: document baseline+dispatch (no target-cpu pin) and add a fail-fast import check; aarch64 containers now install cleanly. - Rewrite docs/guides/simd.md; fix stale `wide` mention in ADR 0003. - Add ADR 0006 (CPU coverage strategy). Also bundles in-flight release prep already staged in the tree (DTW exception types, SBOM/provenance security, supporting docs). * fix(ci): clear cargo-deny and pip-audit failures; apply dependency bumps cargo-deny (advisories): - Ignore pyo3 RUSTSEC-2026-0176 / RUSTSEC-2026-0177 in deny.toml with a documented rationale: ferro-ta uses neither affected code path (PyList/PyTuple nth iterators; PyCFunction::new_closure). Upstream fix needs pyo3 >=0.29 (large API migration), tracked as a follow-up. pip-audit: - Bump dev lockfile idna 3.18, pytest 9.1.1, urllib3 2.7.0 to clear PYSEC-2026-215, CVE-2025-71176, PYSEC-2026-141/142. Dependency bumps (supersede open dependabot PRs; they auto-close on merge): - cargo: log 0.4.32, serde_json 1.0.150, rayon 1.12.0 - api/requirements.txt: uvicorn>=0.49.0, pydantic>=2.13.4, ferro-ta>=1.1.4 - CI actions: deploy-pages v5, upload-pages-artifact v5, action-gh-release v3 The open `wide` 1.5.0 bump (PR #24) is obsolete — the crate is removed in this branch. * chore: address CodeRabbit review; remove docs/adr section CodeRabbit findings: - CI sbom job: add `attestations: write` so attest-build-provenance can run (it had only contents:write + id-token:write). - simd.rs: vectorize `wma_seed` with lane-local accumulators — it was scalar behind the multiversion wrapper, adding dispatch overhead for no SIMD gain. - CHANGELOG: consolidate the duplicate `### Changed` heading. - python/ferro_ta/__init__.py: also re-export the `FerroTaError` alias. - docs/guides/dtw.md: soften "byte-for-byte" parity to within-tolerance. Remove docs/adr/ at maintainer request and clean up the ADR links in the SIMD and DTW guides. The ADR files remain in commit 9506a30 if ever needed.
460 lines
15 KiB
YAML
460 lines
15 KiB
YAML
name: CI
|
|
|
|
on:
|
|
push:
|
|
branches: ["main"]
|
|
pull_request:
|
|
branches: ["main"]
|
|
release:
|
|
types: [published]
|
|
workflow_dispatch:
|
|
inputs:
|
|
release:
|
|
description: "Simulate a release publish (runs build + publish jobs)"
|
|
type: boolean
|
|
default: true
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
# -------------------------------------------------------------------------
|
|
# Version consistency — Cargo.toml and pyproject.toml must have same version
|
|
# -------------------------------------------------------------------------
|
|
changelog-check:
|
|
name: CHANGELOG has [Unreleased] entry
|
|
runs-on: ubuntu-latest
|
|
if: github.event_name == 'pull_request'
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
- name: Check CHANGELOG.md
|
|
run: python3 scripts/check_changelog.py
|
|
|
|
version-check:
|
|
name: Version consistency (Cargo.toml == pyproject.toml)
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
- name: Check version parity
|
|
run: |
|
|
CARGO_VERSION=$(grep '^version' Cargo.toml | head -1 | sed 's/version = "\(.*\)"/\1/')
|
|
PYPROJECT_VERSION=$(grep '^version' pyproject.toml | head -1 | sed 's/version = "\(.*\)"/\1/')
|
|
echo "Cargo.toml version: $CARGO_VERSION"
|
|
echo "pyproject.toml version: $PYPROJECT_VERSION"
|
|
if [ "$CARGO_VERSION" != "$PYPROJECT_VERSION" ]; then
|
|
echo "ERROR: Version mismatch! Update both files before releasing."
|
|
exit 1
|
|
fi
|
|
echo "OK: versions match ($CARGO_VERSION)"
|
|
|
|
# -------------------------------------------------------------------------
|
|
# Modularized CI suites.
|
|
# Keep release publishing jobs in this file so trusted-publisher bindings
|
|
# remain anchored to CI.yml for PyPI and crates.io.
|
|
# -------------------------------------------------------------------------
|
|
rust-suite:
|
|
name: Rust and audit
|
|
uses: ./.github/workflows/ci-rust.yml
|
|
|
|
python-suite:
|
|
name: Python quality and tests
|
|
uses: ./.github/workflows/ci-python.yml
|
|
|
|
wasm:
|
|
name: WASM binding
|
|
uses: ./.github/workflows/ci-wasm.yml
|
|
|
|
docs:
|
|
name: Documentation
|
|
uses: ./.github/workflows/ci-docs.yml
|
|
with:
|
|
upload-pages-artifact: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
|
|
|
|
# -------------------------------------------------------------------------
|
|
# Deploy docs to GitHub Pages (on push to main only)
|
|
# -------------------------------------------------------------------------
|
|
deploy-docs:
|
|
name: Deploy docs to GitHub Pages
|
|
runs-on: ubuntu-latest
|
|
needs: docs
|
|
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
|
environment:
|
|
name: github-pages
|
|
url: ${{ steps.deployment.outputs.page_url }}
|
|
permissions:
|
|
pages: write
|
|
id-token: write
|
|
steps:
|
|
- name: Deploy to GitHub Pages
|
|
id: deployment
|
|
uses: actions/deploy-pages@v5
|
|
|
|
# -------------------------------------------------------------------------
|
|
# CI gate — all required jobs must pass before this job succeeds.
|
|
# Set "ci-complete" as a required status check in branch protection rules
|
|
# to block merging of PRs that fail any required job.
|
|
# -------------------------------------------------------------------------
|
|
ci-complete:
|
|
name: CI complete (required gate)
|
|
runs-on: ubuntu-latest
|
|
needs:
|
|
- changelog-check
|
|
- version-check
|
|
- rust-suite
|
|
- python-suite
|
|
- wasm
|
|
- docs
|
|
if: always()
|
|
steps:
|
|
- name: Check all required jobs passed
|
|
run: |
|
|
results='${{ toJSON(needs) }}'
|
|
echo "Job results: $results"
|
|
failed="$(
|
|
RESULTS_JSON="$results" python3 - <<'PY'
|
|
import json
|
|
import os
|
|
|
|
needs = json.loads(os.environ["RESULTS_JSON"])
|
|
failed = [
|
|
name
|
|
for name, data in needs.items()
|
|
if data["result"] not in ("success", "skipped")
|
|
]
|
|
if failed:
|
|
print(" ".join(failed))
|
|
PY
|
|
)"
|
|
if [ -n "$failed" ]; then
|
|
echo "FAILED jobs: $failed"
|
|
exit 1
|
|
fi
|
|
echo "All required CI jobs passed."
|
|
|
|
# -------------------------------------------------------------------------
|
|
# Build wheels for all platforms and publish to PyPI on release
|
|
# Keep release jobs here because trusted-publisher configuration points to
|
|
# CI.yml specifically.
|
|
# -------------------------------------------------------------------------
|
|
# One abi3 wheel per (platform, arch). abi3-py310 means a single wheel
|
|
# covers CPython 3.10+ on each target, so there is no python-version axis.
|
|
# extension-module + abi3 link no libpython, which is what lets the linux
|
|
# jobs cross-compile aarch64/musl from an x86_64 runner via maturin-action.
|
|
|
|
build-wheels-linux:
|
|
name: Build wheels (linux-gnu / ${{ matrix.target }})
|
|
runs-on: ubuntu-latest
|
|
if: (github.event_name == 'release' && github.event.action == 'published') || (github.event_name == 'workflow_dispatch' && inputs.release)
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
target: [x86_64, aarch64]
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- name: Build abi3 manylinux wheel
|
|
uses: PyO3/maturin-action@v1
|
|
with:
|
|
command: build
|
|
target: ${{ matrix.target }}
|
|
args: --release --out dist
|
|
manylinux: "2_17"
|
|
|
|
- name: Upload wheels as artifact
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: wheels-linux-gnu-${{ matrix.target }}
|
|
path: dist/*.whl
|
|
|
|
build-wheels-musllinux:
|
|
name: Build wheels (linux-musl / ${{ matrix.target }})
|
|
runs-on: ubuntu-latest
|
|
if: (github.event_name == 'release' && github.event.action == 'published') || (github.event_name == 'workflow_dispatch' && inputs.release)
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
target: [x86_64, aarch64]
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- name: Build abi3 musllinux wheel
|
|
uses: PyO3/maturin-action@v1
|
|
with:
|
|
command: build
|
|
target: ${{ matrix.target }}
|
|
args: --release --out dist
|
|
manylinux: musllinux_1_2
|
|
|
|
- name: Upload wheels as artifact
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: wheels-linux-musl-${{ matrix.target }}
|
|
path: dist/*.whl
|
|
|
|
build-wheels-macos:
|
|
name: Build wheels (macos / universal2)
|
|
runs-on: macos-latest
|
|
if: (github.event_name == 'release' && github.event.action == 'published') || (github.event_name == 'workflow_dispatch' && inputs.release)
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version: "3.10"
|
|
|
|
- name: Build universal2 abi3 wheel
|
|
uses: PyO3/maturin-action@v1
|
|
with:
|
|
command: build
|
|
target: universal2-apple-darwin
|
|
args: --release --out dist
|
|
|
|
- name: Upload wheels as artifact
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: wheels-macos-universal2
|
|
path: dist/*.whl
|
|
|
|
build-wheels-windows:
|
|
name: Build wheels (windows / ${{ matrix.platform.arch }})
|
|
runs-on: ${{ matrix.platform.runner }}
|
|
if: (github.event_name == 'release' && github.event.action == 'published') || (github.event_name == 'workflow_dispatch' && inputs.release)
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
platform:
|
|
- runner: windows-latest
|
|
arch: x64
|
|
target: x64
|
|
- runner: windows-11-arm
|
|
arch: arm64
|
|
target: aarch64
|
|
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version: "3.11"
|
|
architecture: ${{ matrix.platform.arch }}
|
|
|
|
- name: Build abi3 wheel
|
|
uses: PyO3/maturin-action@v1
|
|
with:
|
|
command: build
|
|
target: ${{ matrix.platform.target }}
|
|
args: --release --out dist
|
|
|
|
- name: Upload wheels as artifact
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: wheels-windows-${{ matrix.platform.arch }}
|
|
path: dist/*.whl
|
|
|
|
build-sdist:
|
|
name: Build source distribution
|
|
runs-on: ubuntu-latest
|
|
if: (github.event_name == 'release' && github.event.action == 'published') || (github.event_name == 'workflow_dispatch' && inputs.release)
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- name: Build sdist
|
|
uses: PyO3/maturin-action@v1
|
|
with:
|
|
command: sdist
|
|
args: --out dist
|
|
|
|
- name: Upload sdist as artifact
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: sdist
|
|
path: dist/*.tar.gz
|
|
|
|
# -------------------------------------------------------------------------
|
|
# Publish to PyPI
|
|
# -------------------------------------------------------------------------
|
|
publish:
|
|
name: Publish to PyPI
|
|
runs-on: ubuntu-latest
|
|
needs:
|
|
- build-wheels-linux
|
|
- build-wheels-musllinux
|
|
- build-wheels-macos
|
|
- build-wheels-windows
|
|
- build-sdist
|
|
if: (github.event_name == 'release' && github.event.action == 'published') || (github.event_name == 'workflow_dispatch' && inputs.release)
|
|
environment:
|
|
name: pypi
|
|
url: https://pypi.org/p/ferro-ta
|
|
permissions:
|
|
id-token: write
|
|
attestations: write
|
|
contents: read
|
|
steps:
|
|
- name: Download all wheels
|
|
uses: actions/download-artifact@v8
|
|
with:
|
|
pattern: wheels-*
|
|
merge-multiple: true
|
|
path: dist
|
|
|
|
- name: Download source distribution
|
|
uses: actions/download-artifact@v8
|
|
with:
|
|
name: sdist
|
|
path: dist
|
|
|
|
- name: Generate SLSA build provenance attestations
|
|
uses: actions/attest-build-provenance@v2
|
|
with:
|
|
subject-path: "dist/*"
|
|
|
|
- name: Verify distribution coverage
|
|
run: |
|
|
python3 - <<'PY'
|
|
import fnmatch
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
dist = Path("dist")
|
|
files = sorted(p.name for p in dist.iterdir() if p.is_file())
|
|
print("Distributions:")
|
|
for name in files:
|
|
print(f" - {name}")
|
|
|
|
# One abi3 wheel (cp310-abi3) per platform/arch — covers CPython 3.10+.
|
|
expected = [
|
|
"ferro_ta-*-cp310-abi3-manylinux*_x86_64.whl",
|
|
"ferro_ta-*-cp310-abi3-manylinux*_aarch64.whl",
|
|
"ferro_ta-*-cp310-abi3-musllinux*_x86_64.whl",
|
|
"ferro_ta-*-cp310-abi3-musllinux*_aarch64.whl",
|
|
"ferro_ta-*-cp310-abi3-macosx*_universal2.whl",
|
|
"ferro_ta-*-cp310-abi3-win_amd64.whl",
|
|
"ferro_ta-*-cp310-abi3-win_arm64.whl",
|
|
"ferro_ta-*.tar.gz",
|
|
]
|
|
|
|
missing = [
|
|
pattern for pattern in expected
|
|
if not any(fnmatch.fnmatch(name, pattern) for name in files)
|
|
]
|
|
if missing:
|
|
print("Missing expected distributions:")
|
|
for pattern in missing:
|
|
print(f" - {pattern}")
|
|
sys.exit(1)
|
|
PY
|
|
|
|
- name: Publish to PyPI
|
|
uses: pypa/gh-action-pypi-publish@release/v1
|
|
|
|
# -------------------------------------------------------------------------
|
|
# Publish ferro_ta_core to crates.io (requires CARGO_REGISTRY_TOKEN secret)
|
|
# -------------------------------------------------------------------------
|
|
publish-cratesio:
|
|
name: Publish to crates.io
|
|
runs-on: ubuntu-latest
|
|
if: (github.event_name == 'release' && github.event.action == 'published') || (github.event_name == 'workflow_dispatch' && inputs.release)
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- name: Install Rust
|
|
uses: dtolnay/rust-toolchain@v1
|
|
with:
|
|
toolchain: stable
|
|
|
|
- name: Publish ferro_ta_core to crates.io
|
|
run: cargo publish -p ferro_ta_core --token ${{ secrets.CARGO_REGISTRY_TOKEN }}
|
|
|
|
# -------------------------------------------------------------------------
|
|
# SBOM generation — Software Bill of Materials for supply-chain transparency
|
|
# Generates SBOMs for both Python (syft/CycloneDX) and Rust (cargo-cyclonedx)
|
|
# and uploads them as GitHub Release assets.
|
|
# -------------------------------------------------------------------------
|
|
sbom:
|
|
name: Generate SBOM (Python + Rust)
|
|
runs-on: ubuntu-latest
|
|
needs:
|
|
- publish
|
|
- publish-cratesio
|
|
if: (github.event_name == 'release' && github.event.action == 'published') || (github.event_name == 'workflow_dispatch' && inputs.release)
|
|
permissions:
|
|
contents: write
|
|
id-token: write
|
|
attestations: write
|
|
steps:
|
|
- uses: actions/checkout@v6
|
|
|
|
- name: Set up Python 3.12
|
|
uses: actions/setup-python@v6
|
|
with:
|
|
python-version: "3.12"
|
|
|
|
- name: Install maturin
|
|
run: pip install maturin numpy
|
|
|
|
- name: Build and install ferro_ta wheel
|
|
run: |
|
|
maturin build --release --out dist
|
|
pip install dist/*.whl
|
|
|
|
- name: Generate Python SBOM (CycloneDX via anchore/sbom-action)
|
|
uses: anchore/sbom-action@v0.24.0
|
|
with:
|
|
artifact-name: ferro-ta-python-sbom.spdx.json
|
|
output-file: ferro-ta-python-sbom.spdx.json
|
|
format: spdx-json
|
|
|
|
- name: Install Rust stable
|
|
uses: dtolnay/rust-toolchain@v1
|
|
with:
|
|
toolchain: stable
|
|
|
|
- name: Install cargo-cyclonedx
|
|
run: cargo install cargo-cyclonedx --locked
|
|
|
|
- name: Generate Rust SBOM (CycloneDX)
|
|
run: cargo cyclonedx --format json --override-filename ferro-ta-rust-sbom.cdx
|
|
|
|
- name: Validate SBOMs (schema check)
|
|
run: |
|
|
python3 -c "import json, sys; json.load(open('ferro-ta-python-sbom.spdx.json')); print('Python SBOM valid JSON')"
|
|
python3 -c "import json, sys; data=json.load(open('ferro-ta-rust-sbom.cdx.json')); assert data.get('bomFormat')=='CycloneDX', 'not CycloneDX'; print('Rust SBOM valid CycloneDX')"
|
|
|
|
- name: Install cosign
|
|
uses: sigstore/cosign-installer@v3
|
|
|
|
- name: Sign SBOMs with cosign (keyless)
|
|
env:
|
|
COSIGN_EXPERIMENTAL: "1"
|
|
run: |
|
|
cosign sign-blob --yes ferro-ta-python-sbom.spdx.json --output-signature ferro-ta-python-sbom.spdx.json.sig --output-certificate ferro-ta-python-sbom.spdx.json.pem
|
|
cosign sign-blob --yes ferro-ta-rust-sbom.cdx.json --output-signature ferro-ta-rust-sbom.cdx.json.sig --output-certificate ferro-ta-rust-sbom.cdx.json.pem
|
|
|
|
- name: Attest SBOM provenance
|
|
uses: actions/attest-build-provenance@v2
|
|
with:
|
|
subject-path: |
|
|
ferro-ta-python-sbom.spdx.json
|
|
ferro-ta-rust-sbom.cdx.json
|
|
|
|
- name: Upload Python SBOM to release
|
|
uses: softprops/action-gh-release@v3
|
|
with:
|
|
files: |
|
|
ferro-ta-python-sbom.spdx.json
|
|
ferro-ta-python-sbom.spdx.json.sig
|
|
ferro-ta-python-sbom.spdx.json.pem
|
|
|
|
- name: Upload Rust SBOM to release
|
|
uses: softprops/action-gh-release@v3
|
|
with:
|
|
files: |
|
|
ferro-ta-rust-sbom.cdx.json
|
|
ferro-ta-rust-sbom.cdx.json.sig
|
|
ferro-ta-rust-sbom.cdx.json.pem
|