chore: release v1.0.3
This commit is contained in:
Pratik Bhadane
+122
-21
@@ -1,20 +1,130 @@
|
||||
Benchmarks
|
||||
==========
|
||||
|
||||
The authoritative benchmark workflow is in ``benchmarks/``:
|
||||
The benchmark suite is meant to support a narrow claim: ferro-ta is often
|
||||
faster on selected indicators, and the evidence is published in a reproducible
|
||||
form.
|
||||
|
||||
What is published
|
||||
-----------------
|
||||
|
||||
The authoritative benchmark workflow lives in ``benchmarks/``:
|
||||
|
||||
- Cross-library speed suite: ``benchmarks/test_speed.py``
|
||||
- Cross-library accuracy suite: ``benchmarks/test_accuracy.py``
|
||||
- TA-Lib head-to-head speed script: ``benchmarks/bench_vs_talib.py``
|
||||
- TA-Lib head-to-head script: ``benchmarks/bench_vs_talib.py``
|
||||
- Table generation from benchmark JSON: ``benchmarks/benchmark_table.py``
|
||||
- Perf-contract artifact bundle: ``benchmarks/run_perf_contract.py``
|
||||
|
||||
Run the cross-library speed suite on 100,000 bars:
|
||||
Latest checked-in TA-Lib artifact
|
||||
---------------------------------
|
||||
|
||||
The current checked-in TA-Lib comparison artifact benchmarks contiguous
|
||||
``float64`` arrays at 10k and 100k bars on an ``Apple M3 Max`` with 14 logical
|
||||
cores, about 38.7 GB RAM, ``CPython 3.13.5``, and ``Rust 1.91.1`` using the
|
||||
default release profile (``lto = true``, ``codegen-units = 1``).
|
||||
|
||||
Summary from ``benchmarks/artifacts/latest/benchmark_vs_talib.json``:
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
|
||||
* - Size
|
||||
- Rows
|
||||
- ferro-ta wins
|
||||
- Median speedup
|
||||
- TA-Lib wins or ties
|
||||
* - ``10,000``
|
||||
- 12
|
||||
- 6
|
||||
- ``1.0850x``
|
||||
- ``EMA``, ``RSI``, ``ATR``, ``STOCH``, ``ADX``, ``OBV``
|
||||
* - ``100,000``
|
||||
- 12
|
||||
- 6
|
||||
- ``1.0784x``
|
||||
- ``EMA``, ``RSI``, ``ATR``, ``STOCH``, ``ADX``, ``OBV``
|
||||
|
||||
Examples from the 100k-bar run:
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
|
||||
* - Indicator
|
||||
- ferro-ta
|
||||
- TA-Lib
|
||||
- Speedup
|
||||
- Read
|
||||
* - ``SMA``
|
||||
- ``0.0985 ms``
|
||||
- ``0.2241 ms``
|
||||
- ``2.2751x``
|
||||
- clear ferro-ta win
|
||||
* - ``BBANDS``
|
||||
- ``0.2122 ms``
|
||||
- ``0.4966 ms``
|
||||
- ``2.3402x``
|
||||
- clear ferro-ta win
|
||||
* - ``MACD``
|
||||
- ``0.5152 ms``
|
||||
- ``0.7111 ms``
|
||||
- ``1.3801x``
|
||||
- ferro-ta win
|
||||
* - ``STOCH``
|
||||
- ``1.7064 ms``
|
||||
- ``0.7603 ms``
|
||||
- ``0.4455x``
|
||||
- TA-Lib win
|
||||
* - ``ADX``
|
||||
- ``0.7910 ms``
|
||||
- ``0.5769 ms``
|
||||
- ``0.7294x``
|
||||
- TA-Lib win
|
||||
* - ``ATR``
|
||||
- ``0.5087 ms``
|
||||
- ``0.5147 ms``
|
||||
- ``1.0118x``
|
||||
- tie on this machine
|
||||
|
||||
Methodology notes
|
||||
-----------------
|
||||
|
||||
- The head-to-head script uses the same synthetic OHLCV generator, the same
|
||||
parameters, and the same contiguous ``float64`` array layout for both
|
||||
libraries.
|
||||
- Reported speedup is ``TA-Lib median time / ferro-ta median time``.
|
||||
- The script uses 1 warmup run and 7 measured runs per case, and now records
|
||||
the full per-run timing samples, not just one selected number.
|
||||
- Published JSON artifacts include machine/runtime metadata, git metadata, Rust
|
||||
toolchain and build-profile metadata, per-run variance statistics, and
|
||||
Python-tracked peak allocation snapshots.
|
||||
- Allocation snapshots are based on ``tracemalloc`` and capture Python-tracked
|
||||
allocations only; they are not full native RSS profiles.
|
||||
- If your workload uses non-contiguous arrays, different dtypes, or different
|
||||
batch sizes, benchmark that exact workload. Those factors can materially
|
||||
change the result.
|
||||
|
||||
Reproduce the TA-Lib comparison
|
||||
-------------------------------
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install ta-lib
|
||||
python benchmarks/bench_vs_talib.py --sizes 10000 100000 --json benchmark_vs_talib.json
|
||||
|
||||
The JSON output is the main artifact to review when publishing performance
|
||||
claims.
|
||||
|
||||
Cross-library suite
|
||||
-------------------
|
||||
|
||||
Run the broader speed suite on 100,000 bars:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
uv run pytest benchmarks/test_speed.py --benchmark-only --benchmark-json=benchmarks/results.json -v
|
||||
|
||||
Selected results on a modern CPU (100,000 bars):
|
||||
Selected throughput examples from the checked-in table:
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
@@ -38,25 +148,16 @@ Selected results on a modern CPU (100,000 bars):
|
||||
* - ``STOCH``
|
||||
- 33 M bars/s
|
||||
|
||||
Multi-size and JSON output
|
||||
--------------------------
|
||||
Perf-contract artifacts
|
||||
-----------------------
|
||||
|
||||
To build the markdown comparison table from the JSON output:
|
||||
Use the perf-contract runner when you want a compact, machine-readable artifact
|
||||
bundle for single-series latency, batch throughput, streaming throughput, and
|
||||
hotspot attribution:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
uv run python benchmarks/benchmark_table.py
|
||||
uv run python benchmarks/run_perf_contract.py --output-dir benchmarks/artifacts/latest
|
||||
|
||||
Comparison with TA-Lib
|
||||
----------------------
|
||||
|
||||
To measure speedup vs TA-Lib on the same data and parameters, run:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pip install ta-lib
|
||||
python benchmarks/bench_vs_talib.py --sizes 10000 100000 --json benchmark_vs_talib.json
|
||||
|
||||
See the README “Performance vs TA-Lib” section for methodology and a
|
||||
representative comparison table. The script prints a table of median times and
|
||||
speedup (TA-Lib time / ferro_ta time); use ``--json out.json`` to save results.
|
||||
See ``benchmarks/README.md`` for the detailed benchmark playbook and the
|
||||
checked-in comparison tables.
|
||||
|
||||
Reference in New Issue
Block a user