feat(data-layer): TickAggregator (tick-to-candle) in all 10 languages (#309)

* feat(data-layer): TickAggregator in Node, WASM, Python + C ABI hub

First data-layer feature (F2): roll trade ticks up into fixed-timeframe OHLCV
candles, exposed natively and over the C ABI.

- wickra-data wired as a binding dependency (workspace dep; its wickra-core dep
  is default-features=false so it never forces rayon into the rayon-free WASM
  build — native bindings re-enable parallel through their own dependency).
- Node `TickAggregator(bucket, gapFill?)` -> `push(price, size, ts): Candle[]`;
  WASM the same (array of objects); Python `push(...) -> list[tuple]`.
- C ABI: `WickraCandle` struct + `wickra_tick_aggregator_new/push/free` (push
  writes candles into a caller buffer and returns the count), generated via the
  capi generator's new DATA_LAYER section; cbindgen now parses wickra-data so
  `TickAggregator` is a forward-declared opaque; header vendored to bindings/go.

Verified bit-identical across Node/WASM/Python/C/C++ (o=100 h=101 l=100 c=101
v=3 ts=0 for the shared 3-tick probe). WIP: Go/C#/Java/R generated bindings and
the cross-language golden are still pending.

* feat(data-layer): TickAggregator in Go, C#, Java, R (lossless push/drain)

Complete F2 across all 10 languages: the C-ABI tick aggregator now uses a
two-step push/drain so gap-fill candles are never lost, and the four generated
bindings expose it idiomatically.

- C ABI redesigned: opaque TickAggregator handle (inner aggregator + pending
  buffer); push consumes a tick and returns the closed-candle count, drain copies
  them into a count-sized caller buffer.
- Go: NewTickAggregator + Push(price,size,ts) []Candle; C#: TickAggregator +
  Candle[] Push(...); Java: TickAggregator + Candle[] push(...); R: TickAggregator
  constructor + push() S3 generic returning an (n x 6) numeric matrix.
- Candle output record generated per language from WickraCandle.

Verified bit-identical to the native bindings (o=100 h=101 l=100 c=101 v=3 ts=0)
in Go, C#, Java, and R at runtime; R passes R CMD check (pre-existing doc
warnings only). WIP: cross-language data-layer golden + CHANGELOG still pending.

* test(data-layer): cross-language golden for the tick aggregator + CHANGELOG

gen_golden emits a deterministic tick stream (testdata/golden/data_ticks.csv) and
the reference candle streams with and without gap filling (data_candles.csv,
data_candles_gap.csv). Every binding replays the shared ticks through its
TickAggregator and checks the candles bit-for-bit (fp tolerance) against the Rust
reference:

- Node / WASM / Python / Go / C# / Java / R: a dedicated parity test each.
- C / C++: data_layer_test.c (compiled as both, run as ctest).

The gap-fill fixture closes several candles from a single push, exercising the
lossless push/drain path. Records the feature under CHANGELOG [Unreleased].

* fix(examples): rename the CSV-loader candle to WickraBar

The example CSV helper (wickra_csv.h) defined its own struct WickraCandle, which
now collides with the public C ABI WickraCandle (the tick aggregator output) in
any example that includes both headers (backtest, multi_timeframe, the strategy
examples). The public type owns the name; rename the example loader's bar to
WickraBar. The generated golden_test.c is untouched (its only match was the
unrelated WickraCandleVolumeOutput).
This commit is contained in:
kingchenc
2026-06-15 21:24:33 +02:00
committed by GitHub
parent fd9f4c8bc6
commit 8a103ef920
48 changed files with 1368 additions and 29 deletions
@@ -0,0 +1,59 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.IO;
using Wickra;
using Xunit;
// Cross-language data-layer parity: replay the shared golden tick stream through
// the TickAggregator and check the candles against the Rust reference, with and
// without gap filling.
public class DataLayerTests
{
private static string GoldenDir([System.Runtime.CompilerServices.CallerFilePath] string file = "") =>
Path.GetFullPath(Path.Combine(Path.GetDirectoryName(file)!, "..", "..", "..", "testdata", "golden"));
private static double[][] Read(string name)
{
var lines = File.ReadAllLines(Path.Combine(GoldenDir(), name + ".csv"));
var rows = new List<double[]>();
for (var i = 1; i < lines.Length; i++)
{
if (lines[i].Length == 0)
{
continue;
}
rows.Add(Array.ConvertAll(lines[i].Split(','), s => double.Parse(s, CultureInfo.InvariantCulture)));
}
return rows.ToArray();
}
[Theory]
[InlineData(false, "data_candles")]
[InlineData(true, "data_candles_gap")]
public void TickAggregatorMatchesGolden(bool gapFill, string fixture)
{
var ticks = Read("data_ticks");
using var agg = new TickAggregator(1000, gapFill);
var got = new List<double[]>();
foreach (var t in ticks)
{
foreach (var c in agg.Push(t[0], t[1], (long)t[2]))
{
got.Add(new[] { c.Open, c.High, c.Low, c.Close, c.Volume, (double)c.Timestamp });
}
}
var want = Read(fixture);
Assert.Equal(want.Length, got.Count);
for (var i = 0; i < got.Count; i++)
{
for (var j = 0; j < 6; j++)
{
var tol = 1e-9 * Math.Max(1, Math.Abs(want[i][j]));
Assert.True(
Math.Abs(got[i][j] - want[i][j]) <= tol,
$"{fixture} row {i} col {j}: {got[i][j]} vs {want[i][j]}");
}
}
}
}
@@ -16,6 +16,7 @@ public readonly record struct AutoFibOutput(double Level0, double Level236, doub
public readonly record struct BollingerOutput(double Upper, double Middle, double Lower, double Stddev);
public readonly record struct BomarBandsOutput(double Upper, double Middle, double Lower);
public readonly record struct CamarillaPivotsOutput(double Pp, double R1, double R2, double R3, double R4, double S1, double S2, double S3, double S4);
public readonly record struct Candle(double Open, double High, double Low, double Close, double Volume, double Timestamp);
public readonly record struct CandleVolumeOutput(double Body, double Width);
public readonly record struct CentralPivotRangeOutput(double Pivot, double Tc, double Bc);
public readonly record struct ChandeKrollStopOutput(double StopLong, double StopShort);
@@ -34704,6 +34705,51 @@ public sealed class Thrusting : IDisposable
public void Dispose() => _handle.Dispose();
}
public sealed class TickAggregator : IDisposable
{
private readonly WickraHandle _handle;
public TickAggregator(long bucket, bool gapFill)
{
var ptr = NativeMethods.wickra_tick_aggregator_new(bucket, gapFill);
if (ptr == nint.Zero)
{
throw new ArgumentException("invalid TickAggregator parameters");
}
_handle = new WickraHandle(ptr, NativeMethods.wickra_tick_aggregator_free);
}
/// <summary>Feed one trade tick; returns the candles it closed.</summary>
public Candle[] Push(double price, double size, long timestamp)
{
var count = (long)NativeMethods.wickra_tick_aggregator_push(_handle.DangerousGetHandle(), price, size, timestamp);
GC.KeepAlive(_handle);
if (count <= 0)
{
return Array.Empty<Candle>();
}
var buffer = new WickraCandle[count];
unsafe
{
fixed (WickraCandle* ptr = buffer)
{
NativeMethods.wickra_tick_aggregator_drain(_handle.DangerousGetHandle(), ptr, (nuint)count);
}
}
GC.KeepAlive(_handle);
var result = new Candle[count];
for (var i = 0; i < count; i++)
{
result[i] = new Candle(buffer[i].open, buffer[i].high, buffer[i].low, buffer[i].close, buffer[i].volume, buffer[i].timestamp);
}
return result;
}
public void Dispose() => _handle.Dispose();
}
public sealed class TickBars : IDisposable
{
private readonly WickraHandle _handle;
@@ -12404,6 +12404,18 @@ internal static partial class NativeMethods
[LibraryImport(WickraNative.LibraryName)]
internal static partial void wickra_footprint_free(nint handle);
[LibraryImport(WickraNative.LibraryName)]
internal static partial nint wickra_tick_aggregator_new(long bucket, [MarshalAs(UnmanagedType.U1)] bool gapFill);
[LibraryImport(WickraNative.LibraryName)]
internal static partial nint wickra_tick_aggregator_push(nint handle, double price, double size, long timestamp);
[LibraryImport(WickraNative.LibraryName)]
internal static unsafe partial nint wickra_tick_aggregator_drain(nint handle, WickraCandle* @out, nuint cap);
[LibraryImport(WickraNative.LibraryName)]
internal static partial void wickra_tick_aggregator_free(nint handle);
}
[StructLayout(LayoutKind.Sequential)]
@@ -12503,6 +12515,17 @@ internal static partial class NativeMethods
public double s4;
}
[StructLayout(LayoutKind.Sequential)]
internal struct WickraCandle
{
public double open;
public double high;
public double low;
public double close;
public double volume;
public long timestamp;
}
[StructLayout(LayoutKind.Sequential)]
internal struct WickraCandleVolumeOutput
{