d59cd44043cf6660b469a57dc252a35eddc7c8dc
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 kingchenc
|
b06ce2678a |
Restructure the Go binding for self-contained distribution (#236)
## Problem
Plain `go get` + `go build` of the Go binding never worked **as a dependency**:
- cgo `CFLAGS` pointed at `${SRCDIR}/../c/include` — the parent dir is **outside** the Go module, so a proxy-fetched module has no header.
- the library under `./lib` was git-ignored and never shipped; the README told users to `cargo build` it from the workspace, which only works inside a clone, not from the read-only module cache.
cgo has no build hook and Go has no registry, so the only way a consumer's `go get`+build can find the lib is for it to be committed **inside the module the consumer pulls**. Per the design decision, that module is a separate **`wickra-go`** repo (keeps this repo free of committed binaries), populated by the release pipeline — this PR is the in-repo restructure that makes that possible.
## Changes
- **Vendor the header** at `bindings/go/include/wickra.h` (committed copy of `bindings/c/include/wickra.h`); `CFLAGS` → `-I${SRCDIR}/include`. A **CI drift check** fails if the copy goes stale.
- **Per-platform libraries**: cgo `LDFLAGS` become per `GOOS`/`GOARCH`, linking `${SRCDIR}/lib/<goos>_<goarch>/`.
- **CI** stages the host library into `lib/<goos>_<goarch>/` (`RUNNER_OS`/`RUNNER_ARCH` — note `macos-latest` is arm64) and exports `WICKRA_GO_LIBDIR` for the Windows PATH; libraries stay git-ignored here.
- **README**: install via the `wickra-go` module + a contributor build section.
## Validation
- Local **windows/amd64**: `gofmt` clean, `go vet`, `go build`, `go test` all green against the staged library + vendored header.
- Linux/macOS arches → the 3-OS `Go on …` CI job.
Follow-up (Stage 2, separate): create `wickra-lib/wickra-go` + a `release.yml` mirror job (source + 6 platform libs → `lib/<goos>_<goarch>/`, commit + tag), and point the docs at the new import path. Part of the self-contained gap (`todo-11`).
|
||
|
kingchenc
|
5b7523265c |
Make the R binding self-contained (fetch the C ABI at install time) (#235)
## Problem
The R package built **only** inside the dev/CI workspace. `src/Makevars` and `configure.win` hard-required `WICKRA_INCLUDE_DIR` / `WICKRA_LIB_DIR` pointing at a pre-built `libwickra` (`configure.win` literally `: "${WICKRA_LIB_DIR:?...}"`), and there was no Unix `configure`. So **r-universe** and any plain `install.packages` / `install_github` failed — R had no working end-user install path.
## Fix
Fetch the prebuilt `wickra-c-<triple>.tar.gz` release asset matching the package version at install time and bundle the library into the package (same outcome as the C# / Java bindings, which bundle per-platform native libs):
- **New POSIX `configure`** (the Unix hook R lacked): detect OS/arch → triple, download + `untar` via **base R** (no curl/wget system dep), stage `wickra.h` + `libwickra.{so,dylib}` into `src/`, generate `src/Makevars` from `Makevars.in` with an rpath (`$ORIGIN` Linux, `@loader_path` + `install_name_tool` macOS) so the bundled lib resolves post-install.
- **`configure.win`**: drop the hard `WICKRA_LIB_DIR` requirement; download the Windows triple when unset, then keep the existing `wickra_abi.dll` rename + `objdump`/`dlltool` import-lib dance (sourcing the dll/header from the asset).
- **`install.libs.R`** also bundles `libwickra.dylib` (macOS); the `*.so`/`*.dll` globs already covered Linux/Windows.
- `WICKRA_INCLUDE_DIR`/`WICKRA_LIB_DIR` stay as an optional **dev override**.
- **CI (3 OS)** keeps building against the locally built C ABI (version-independent — avoids the chicken-egg of downloading the in-flight version) but **no longer exports `LD_LIBRARY_PATH`/`DYLD_LIBRARY_PATH`**, so it now verifies the bundled rpath — the real self-contained path users and r-universe get.
`Makevars` is now generated from `Makevars.in`; `SystemRequirements` + ignore/attributes files updated.
## Validation
- The Windows `objdump`/`dlltool` import-lib dance was smoke-tested locally against the real v0.7.9 asset (2412 `wickra_` exports → import lib built).
- Linux/macOS rpath bundling can't be tested on this Windows host → **the 3-OS `r` CI job is the gate** (now without the loader-path mask).
- Follow-up (separate, after release): set up `wickra-lib/wickra-lib.r-universe.dev` (`packages.json` → `bindings/r`).
Closes the R half of the self-contained-distribution gap (`todo-10`).
|
||
|
kingchenc
|
167f7b3ffe |
Add the Java binding over the C ABI hub (Panama/FFM) (#233)
Adds a Java binding (`bindings/java`) over the C ABI hub — the fourth language stecker after C#, Go and R, reaching the hub through the Java Foreign Function & Memory API (Panama, `java.lang.foreign`, final in Java 22) rather than JNI or jextract. ## What's here - **`bindings/java`** — a Maven module (`org.wickra:wickra`) exposing all 514 indicators as idiomatic `AutoCloseable` classes. The downcall handles (`internal/NativeMethods.java`), the per-indicator wrappers and the output records are generated from `bindings/c/include/wickra.h` (same eight-archetype taxonomy as the C#/Go/R generators: scalar/batch, multi-output, bars, profile, values-profile, array-input). The opaque handle is a `MemorySegment` freed by a registered `java.lang.ref.Cleaner` action; multi-output returns a `record` (`null` at warmup), bars a `record[]`, profiles a record with a trailing `double[]`. The hand-written `WickraNative` resolves the native library (a bundled per-platform copy, or a `target/release` fallback for local development) and validates it against a sentinel symbol. repr(C) struct offsets are computed in the generator so the FFM reads land on the exact bytes. - **`examples/java`** — the full example suite mirroring C/C#/Go/R: streaming, backtest, multi_timeframe, parallel_assets (parallel streams), three strategies, and `FetchBtcusdt`/`LiveBinance`. - **CI** — a `java` job builds the C ABI library, sets up JDK 22 (Temurin, with a CDN-flake retry), runs the archetype test suite and the seven offline examples on Linux, macOS and Windows. - **Release** — a gated `java-publish` job (skipped until the `JAVA_PUBLISH_ENABLED` repository variable is set) stages the native libraries from the `wickra-c-<triple>.tar.gz` assets into the binding's resources and deploys to Maven Central with GPG signing. Independent of the GitHub-release job, like the NuGet job. - **Docs** — Java added to the README languages table, project layout, building/testing and comparison table, CONTRIBUTING, ARCHITECTURE, the examples index, the issue/PR templates, the About-description template, and the other binding READMEs. ## Requirements Java 22+ (the FFM API is final since Java 22). The binding requires `--enable-native-access=ALL-UNNAMED` at runtime; the test and example runners pass it automatically. No Rust crate or `Cargo.toml` change — the Java binding is standalone and additive. The generated `*.java` are committed (like the node `index.js`/`index.d.ts`); the generator stays private. |
||
|
kingchenc
|
b7ef63400d |
Add the R binding over the C ABI hub (#230)
Adds an R binding (`bindings/r`) over the C ABI hub — the third language stecker after C# and Go, reaching the hub through R's native `.Call` interface (not extendr). ## What's here - **`bindings/r`** — an R package exposing all 514 indicators as constructors that return a `wickra_indicator` object with generic `update`/`batch`/`reset` methods. The C glue (`src/wickra.c`) and R wrappers (`R/indicators.R`) are generated from `bindings/c/include/wickra.h` (same archetype taxonomy as the C#/Go generators: scalar/batch, multi-output, bars, profile, profile-values, array-input). The opaque handle is an R external pointer freed by a registered finalizer; multi-output returns a named vector (`NA` at warmup), bars a matrix, profiles a list. - **`examples/r`** — the full example suite mirroring C/C#/Go: streaming, backtest, multi_timeframe, parallel_assets (`mclapply`), three strategies, and `fetch_btcusdt`/`live_binance`. - **CI** — an `r` job builds the C ABI library, installs the package, runs the `testthat` suite and the offline examples on Linux, macOS and Windows (`R CMD check` is clean: 0 warnings, 0 notes). - **Docs** — R added to the README languages table, project layout, building/testing, CONTRIBUTING binding table + regenerate note, ARCHITECTURE, examples index, issue/PR templates, the About-description template, and the other binding READMEs. ## Linking / distribution The package compiles a thin `.Call` glue layer against the prebuilt C ABI library (header via `WICKRA_INCLUDE_DIR`, library via `WICKRA_LIB_DIR`). On Windows the package's own `wickra.dll` would collide with the C ABI's `wickra.dll`, so `configure.win` stages a renamed copy (`wickra_abi.dll`) and builds an import library referencing it; `install.libs.R` bundles the DLL and `.onLoad` puts it on the load path. On Linux/macOS the rpath locates the shared library. No `release.yml` change — R is distributed via r-universe / source install (gated). No Rust crate or `Cargo.toml` change — the R package is standalone and additive. |
||
|
kingchenc
|
23d636fd97 |
Add the Go binding over the C ABI hub (#228)
Adds a Go binding (`bindings/go`) over the C ABI hub — the second language stecker after C#. ## What's here - **`bindings/go`** — a cgo binding exposing all 514 indicators as idiomatic Go types with `New<Indicator>` constructors and `Update`/`Batch`/`Reset`/`Close` methods. The wrappers in `indicators_gen.go` are generated from `bindings/c/include/wickra.h` (same archetype taxonomy as the C# generator: scalar/batch, multi-output, bars, profile, profile-values, array-input). Opaque handles are freed by `Close()` with a `runtime.SetFinalizer` backstop; pointer arguments are caller-owned, panics never cross the boundary. - **`examples/go`** — the full example suite mirroring C/C#: streaming, backtest, multi_timeframe, parallel_assets (goroutine fan-out), three strategies, and `fetch_btcusdt`/`live_binance`. - **CI** — a `go` job builds the C ABI library, stages it, and runs `gofmt`/`go vet`/`go test` plus the offline examples on Linux, macOS and Windows. - **Docs** — Go added to the README languages table, project layout, building/testing, CONTRIBUTING binding table + regenerate note, ARCHITECTURE, examples index, issue/PR templates, the About-description template, and the other binding READMEs. ## Linking / distribution The binding links the prebuilt C ABI library via cgo (`libwickra.so`/`.dylib`/`wickra.dll` staged under `bindings/go/lib`, gitignored). The native libraries are already shipped per target triple by the existing `c-abi-build` release job; distribution is via the subdirectory module tag `bindings/go/vX.Y.Z` (gated), so `release.yml` needs no new publish job. No Rust crate or `Cargo.toml` change — the Go module is standalone and additive. Not for merge yet (gated, per request). |
||
|
kingchenc
|
91f6f67257 |
Add C# (.NET) binding over the C ABI hub (#226)
The first language stecker on the C ABI hub: a .NET binding exposing all 514 indicators as idiomatic `IDisposable` classes, generated from `wickra.h`. ## What's here - **`bindings/csharp/`** — the `Wickra` .NET 8 package. `[LibraryImport]` source-generated P/Invoke (`NativeMethods.g.cs`) plus idiomatic wrappers (`Indicators.g.cs`), both generated from the committed `bindings/c/include/wickra.h`. The binding owns no indicator maths — it only marshals types across the C ABI. - **Marshalling, verified end-to-end against the native library.** Opaque handles cross as `nint` kept alive per call via a `SafeHandle`; `bool` as `[MarshalAs(U1)]` (Rust `bool` is one byte); a self-correcting `DllImportResolver` validates the loaded library actually exports the Wickra ABI. Tests cover one representative per FFI archetype (scalar, candle, pairwise, multi-output, bars, profile, values-profile, order-book / array-input) plus exact Sma reference values. - **NuGet packaging** — `dotnet pack` produces `Wickra.<version>.nupkg`; the release pipeline stages prebuilt native libraries under `runtimes/<rid>/native/` for six target triples (win/linux/osx × x64/arm64). - **`examples/csharp/`** — nine examples mirroring `examples/c/`: streaming, backtest, multi_timeframe, parallel_assets, three strategies, and fetch_btcusdt + live_binance. - **CI** — a `csharp` job on the three OSes builds the C ABI, tests the binding, and runs the offline examples. **Release** — a gated `csharp-publish` job packs and pushes to NuGet (gated on `NUGET_API_KEY`, independent of the GitHub-release job so a C# hiccup never blocks the C/C++ asset release). - **Docs consistency wave** — README, CONTRIBUTING, CHANGELOG, examples/README, the issue / PR templates, `sync-about.yml`, and `.gitattributes`. The native Python / Node / WASM bindings and the C ABI are untouched; this is additive. Publishing to NuGet stays gated behind the release tag and the secret. |
||
|
kingchenc
|
91e05e3c26 |
C ABI hub crate (bindings/c) foundation (#222)
## What
Introduces `wickra-c` — a `cdylib` + `staticlib` that exposes the Rust core over a **C ABI**. This is the hub every C-capable language (C, C++, Go, C#, Java, R) links against, instead of re-wiring each indicator natively. The native Python/Node/WASM bindings are untouched; this is purely additive, for ecosystems without first-class Rust tooling.
## Scope (foundation slice)
This PR deliberately validates the **whole pipeline end to end with one indicator (SMA)** before scaling to all 514, so the CI / cross-OS / header-drift mechanics are proven green first.
- Opaque `*mut T` handles; `wickra_<ind>_{new,update,batch,reset,free}`.
- NaN sentinel for warmup / NULL handles; caller-owned batch buffers; every function NULL-safe.
- cbindgen generates and commits `bindings/c/include/wickra.h` with opaque handle typedefs.
- A C smoke example (`examples/c/`) links the header + compiled library and runs (CMake + ctest).
- A `c-abi` CI job builds the library and runs the smoke test on **Linux, macOS and Windows**, plus a header drift check on Linux.
## Notes
- The per-indicator FFI blocks are plain `#[no_mangle]` functions, **not** a macro: cbindgen cannot see macro-generated functions on stable Rust (macro expansion needs nightly), so the blocks are written literally and will be generated mechanically by the ScriptHelpers `capi` wrapper in a follow-up (same model as the committed-but-generated Node `index.js`).
- `bindings/c` cannot inherit the workspace `forbid(unsafe_code)` lint (the C boundary needs raw pointers), so it mirrors every workspace lint and only relaxes `unsafe_code`. The Rust core stays `unsafe`-forbidden.
## Follow-ups (separate PRs)
- ScriptHelpers `capi` generator + wire the scalar family (~235).
- Hand-written blocks for multi-output / custom-input / bars (~279).
- Docs consistency wave (README / docs / webpage: Python·Node·WASM·Rust → +C).
- Release wiring (native-lib matrix + header/lib GH-release assets) — gated.
|
||
|
kingchenc
|
5b23b36261 |
ci: pin CI dependency installs by hash (Scorecard PinnedDependencies) (#114)
* ci: use npm ci instead of npm install for reproducible installs Pins the node binding dependency install to the committed package-lock.json integrity hashes (OpenSSF Scorecard PinnedDependencies). npm ci installs strictly from the lockfile; npm install could resolve newer patch versions. Covers ci.yml and both release.yml node steps. * ci: hash-pin Python dev tooling in ci.yml (Scorecard #19) Replaces the unpinned 'pip install maturin pytest numpy hypothesis' with a hash-locked '--require-hashes -r' install (OpenSSF Scorecard PinnedDependencies). Two lock files are needed because numpy publishes no single release with wheels for both cp39 and cp313 (<=2.0.2 has cp39 only, >=2.1 drops cp39): ci-dev-py39.txt numpy 2.0.2 (Python 3.9, + tomli/exceptiongroup) ci-dev-py3.txt numpy 2.4.6 (Python 3.10+) The step selects the file by matrix.python-version under shell: bash. Both are generated from ci-dev.in via uv (scripts/update-lockfiles.sh, added next). * ci: hash-pin Python deps in bench.yml (Scorecard #16) Replaces the unpinned 'pip install maturin numpy pandas talipp finta' with a hash-locked '--require-hashes -r .github/requirements/bench.txt' install. bench.yml runs on a single Python version (3.11), so one lock file (generated from bench.in via uv) is sufficient. * build: add scripts/update-lockfiles.sh to regenerate all lockfiles One command refreshes every committed lockfile across languages: Cargo.lock and fuzz/Cargo.lock (cargo update), the Node binding package-lock.json, and the hash-pinned Python requirements under .github/requirements/ (uv pip compile --generate-hashes). Uses uv for the Python locks so a target Python version's hashed transitive closure can be resolved without that interpreter installed (needed for the numpy cp39/cp313 split); bootstraps uv if absent. .gitattributes pins *.sh to LF so the script stays runnable on Linux/macOS. * ci: split ci-dev requirements per Python version + Dependabot rehash Splits the single ci-dev.in into ci-dev-py39.in (numpy <2.1, the last series with cp39 wheels) and ci-dev-py3.in (3.10+), giving a 1:1 .in->.txt layout. The cap keeps Python 3.9 permanently installable and stops Dependabot from proposing 3.9-breaking numpy bumps. Adds a Dependabot pip entry on /.github/requirements so the hash-locked tooling is kept current automatically; the canonical manual refresh stays scripts/update-lockfiles.sh. Only the '# via -r' provenance lines in the .txt change; no package versions or hashes move. * ci: cache pip and npm downloads in the PR-loop jobs Adds setup-python cache: pip (ci.yml python matrix + bench.yml, keyed on the hash-locked requirements) and setup-node cache: npm (ci.yml node job, keyed on bindings/node/package-lock.json), on both the primary and retry setup steps. Scoped to jobs that actually install dependencies; the examples-smoke and clippy-bindings jobs install nothing and are left uncached. release.yml is intentionally left out: it runs only on tag push (not the PR loop) and is the publish-critical path, so no caching is added there. * docs: document hash-pinned requirements and update-lockfiles.sh Updates the lockfile-policy table: the bindings/python row no longer claims CI installs tooling unpinned, and a new .github/requirements row documents the hash-locked CI/bench tooling and the per-Python-version ci-dev split. Adds a paragraph pointing contributors at scripts/update-lockfiles.sh (uv-based, self-bootstrapping) as the canonical lockfile refresh. * docs: changelog entry for hash-pinned CI dependency installs |