Refactor documentation for clarity and detail

This commit is contained in:
Miha Kralj
2025-12-17 23:00:52 -08:00
parent 1084644a3d
commit 5d03dec741
38 changed files with 3141 additions and 2006 deletions
+85 -41
View File
@@ -1,68 +1,112 @@
# TBar Struct
# TBar: OHLCV Bar Struct
`TBar` is a lightweight, immutable struct representing a single OHLCV (Open, High, Low, Close, Volume) bar. It is designed for high-performance financial data processing with minimal memory overhead.
## What It Does
## Key Features
`TBar` is a lightweight, immutable struct representing a single OHLCV (Open, High, Low, Close, Volume) bar. It serves as the fundamental unit for price data in QuanTAlib, designed to hold market data with minimal memory overhead while providing convenient accessors for common price derivations.
- **Memory Efficient**: Pure data type occupying exactly 48 bytes (1 `long` + 5 `double`s).
- **Immutable**: Thread-safe by design.
- **Zero-Copy Conversions**: Efficiently converts to `TValue` for individual price components (Open, High, Low, Close, Volume).
- **Computed Properties**: Provides on-demand calculation of common price averages (HL2, HLC3, etc.) without storage overhead.
- **SIMD Compatible**: Layout is optimized for potential vectorization in collection types.
## Design Philosophy
## Structure Definition
Financial data processing often involves millions of bars. Storing these as classes would create massive GC pressure and memory fragmentation. `TBar` is designed as a **pure data struct** to ensure:
* **Compactness**: Occupies exactly 48 bytes (1 `long` + 5 `double`s), fitting efficiently in memory.
* **Immutability**: Thread-safe by default; values cannot change once created.
* **Zero-Cost Abstractions**: Computed properties (like `HL2`) are calculated on-demand, requiring no extra storage.
## How It Works
`TBar` is a `readonly record struct` that stores:
* **Time**: Timestamp in ticks.
* **Open, High, Low, Close**: Price components.
* **Volume**: Traded volume.
It includes implicit conversions to `double` (defaulting to Close price) and `TValue` (Time + Close), allowing it to be used interchangeably with simpler types in many contexts.
## Structure
### Definition
```csharp
public readonly struct TBar : IEquatable<TBar>
{
public readonly long Time; // Unix ticks
public readonly double Open;
public readonly double High;
public readonly double Low;
public readonly double Close;
public readonly double Volume;
}
public readonly record struct TBar(long Time, double Open, double High, double Low, double Close, double Volume);
```
## Properties
### Core Properties
| Property | Type | Description |
|----------|------|-------------|
| `Time` | `long` | Timestamp in ticks. |
| `Time` | `long` | Timestamp in ticks (UTC). |
| `Open` | `double` | Opening price. |
| `High` | `double` | Highest price. |
| `Low` | `double` | Lowest price. |
| `Close` | `double` | Closing price. |
| `Volume` | `double` | Traded volume. |
| `AsDateTime` | `DateTime` | `Time` converted to UTC DateTime. |
### Computed Averages
These properties are calculated on the fly:
- `HL2`: (High + Low) / 2
- `OC2`: (Open + Close) / 2
- `OHL3`: (Open + High + Low) / 3
- `HLC3`: (High + Low + Close) / 3
- `OHLC4`: (Open + High + Low + Close) / 4
- `HLCC4`: (High + Low + Close + Close) / 4
### Computed Properties (Zero-Storage)
| Property | Formula | Description |
|----------|---------|-------------|
| `HL2` | `(H + L) / 2` | Median Price. |
| `OC2` | `(O + C) / 2` | Midpoint Price. |
| `OHL3` | `(O + H + L) / 3` | Typical Price (Variant). |
| `HLC3` | `(H + L + C) / 3` | Typical Price. |
| `OHLC4` | `(O + H + L + C) / 4` | Weighted Close. |
| `HLCC4` | `(H + L + 2C) / 4` | Weighted Close (Variant). |
### TValue Accessors
Efficiently access components as `TValue` (Time-Value pair):
- `O`: (Time, Open)
- `H`: (Time, High)
- `L`: (Time, Low)
- `C`: (Time, Close)
- `V`: (Time, Volume)
Efficiently extracts components as `TValue` pairs:
* `O`, `H`, `L`, `C`, `V`
## Usage
### Creating a TBar
### Creating a Bar
```csharp
long now = DateTime.UtcNow.Ticks;
var bar = new TBar(now, 100.0, 105.0, 95.0, 102.0, 1000.0);
var bar = new TBar(DateTime.UtcNow, 100, 105, 95, 102, 1000);
```
### Implicit Conversions
```csharp
double closePrice = bar; // Implicitly converts to Close price
TValue value = bar; // Implicitly converts to (Time, Close)
DateTime dt = bar; // Implicitly converts to DateTime
TBar bar = ...;
// Treat as double (uses Close price)
double price = bar;
// Treat as TValue (Time + Close)
TValue tv = bar;
// Treat as DateTime
DateTime dt = bar;
```
### Using Computed Properties
```csharp
// Calculate Typical Price on the fly
double typical = bar.HLC3;
```
## Performance Profile
* **Memory**: 48 bytes per instance.
* **Allocation**: 0 bytes (Stack allocated).
* **Access**: Direct field access (no property overhead).
## Integration
`TBar` is the primary input for:
* **TBarSeries**: A collection of bars.
* **Indicators**: Some indicators (like ATR) require full `TBar` input rather than just a single value.
## Architecture Notes
* **SkipLocalsInit**: Marked with `[SkipLocalsInit]` for performance in tight loops.
* **AggressiveInlining**: All computed properties are inlined to ensure they are as fast as writing the formula manually.
## References
* [OHLC Chart](https://en.wikipedia.org/wiki/Open-high-low-close_chart)
* [C# Record Structs](https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/builtin-types/record)
+75 -24
View File
@@ -1,26 +1,46 @@
# TBarSeries Class
# TBarSeries: OHLCV Data Container
`TBarSeries` is a high-performance collection of OHLCV bars implemented using a Structure of Arrays (SoA) layout. This design optimizes memory access patterns and enables efficient SIMD operations while providing convenient object-oriented views.
## What It Does
## Key Features
`TBarSeries` is a high-performance collection of OHLCV bars. It is the primary data structure for managing historical and real-time market data in QuanTAlib. It uses a **Structure of Arrays (SoA)** layout to optimize memory access and enable efficient SIMD operations across individual price components.
- **Structure of Arrays (SoA)**: Stores Time, Open, High, Low, Close, and Volume in separate contiguous arrays rather than an array of structs. This improves cache locality for operations that only need specific components (e.g., calculating SMA on Close prices).
- **Zero-Copy Views**: Exposes `TSeries` properties (`Open`, `High`, `Low`, `Close`, `Volume`) that view the underlying data without copying.
- **Streaming Support**: Efficiently handles real-time data updates with `Add(bar, isNew: false)`.
- **Memory Efficient**: Minimizes object overhead by using shared internal lists.
## Design Philosophy
## Class Definition
A naive implementation of a bar series would be a `List<TBar>`. However, this is inefficient for technical analysis. Most indicators only need one component at a time (e.g., SMA uses Close prices). Iterating over a `List<TBar>` to get Close prices loads unnecessary Open, High, Low, and Volume data into the CPU cache, wasting bandwidth.
`TBarSeries` solves this by storing each component in its own contiguous array. This allows:
* **Component Views**: You can access `Close` prices as a `TSeries` without copying data.
* **Cache Efficiency**: Iterating over `Close` prices loads *only* Close prices.
* **Unified Time**: All component series share a single Time array, ensuring synchronization.
## How It Works
Internally, `TBarSeries` maintains six parallel lists:
1. `_t` (Time)
2. `_o` (Open)
3. `_h` (High)
4. `_l` (Low)
5. `_c` (Close)
6. `_v` (Volume)
It exposes these internal lists as `TSeries` properties (`Open`, `High`, `Low`, `Close`, `Volume`), which act as read-only views into the master data.
## Structure
### Definition
```csharp
public class TBarSeries : IReadOnlyList<TBar>
{
// Views
// Component Views (TSeries)
public TSeries Open { get; }
public TSeries High { get; }
public TSeries Low { get; }
public TSeries Close { get; }
public TSeries Volume { get; }
// Aliases
public TSeries O => Open;
public TSeries H => High;
@@ -30,41 +50,72 @@ public class TBarSeries : IReadOnlyList<TBar>
}
```
## Core Methods
### Core Methods
| Method | Description |
|--------|-------------|
| `Add(TBar bar, bool isNew = true)` | Adds a new bar or updates the last one. |
| `Add(DateTime time, double o, double h, double l, double c, double v, bool isNew)` | Adds raw values directly. |
| `Add(TBar bar, bool isNew)` | Adds a bar or updates the last one. |
| `Add(DateTime time, double o, double h, double l, double c, double v)` | Adds raw values directly. |
| `Count` | Returns the number of bars. |
| `Last` | Returns the most recent `TBar`. |
## Usage
### Creating and Populating
```csharp
var bars = new TBarSeries();
// Add a new bar
long now = DateTime.UtcNow.Ticks;
bars.Add(new TBar(now, 100, 105, 95, 102, 1000), isNew: true);
bars.Add(new TBar(DateTime.UtcNow, 100, 105, 95, 102, 1000));
// Update the last bar (e.g., real-time feed update)
bars.Add(new TBar(now, 100, 106, 95, 104, 1500), isNew: false);
// Add raw values
bars.Add(DateTime.UtcNow, 100, 105, 95, 102, 1000);
```
### Accessing Data
```csharp
// Access entire bar
// Get the last full bar
TBar lastBar = bars.Last;
// Access specific component series (Zero-Copy)
// Get the Close series (Zero-Copy)
TSeries closes = bars.Close;
double lastClose = closes.Last.Value;
// Access via indexer
TBar firstBar = bars[0];
// Calculate SMA on Close prices
var sma = new Sma(14);
var result = sma.Calculate(bars.Close);
```
### Performance Note
Because `TBarSeries` uses SoA layout, iterating over a single component (like `Close` prices) is extremely cache-efficient. The CPU prefetcher can load contiguous doubles without loading the interleaved Open, High, Low, or Volume data.
### Streaming Updates
```csharp
// New minute starts
bars.Add(newBar, isNew: true);
// Price updates within the same minute
bars.Add(updatedBar, isNew: false); // Updates the last bar in place
```
## Performance Profile
* **Memory Layout**: SoA (Structure of Arrays).
* **Component Access**: Zero-copy `TSeries` views.
* **Iteration**: Cache-friendly for single-component analysis.
## Integration
`TBarSeries` is the standard input for multi-input indicators (like ATR, ADX) and the primary data source for trading strategies.
* **Indicators**: Can be passed to indicators that require full bar data.
* **Strategies**: Provides the historical context needed for signal generation.
## Architecture Notes
* **Shared Storage**: The `TSeries` views (`Open`, `Close`, etc.) do not own their data; they point to the internal lists of the `TBarSeries`. This means modifying the `TBarSeries` automatically updates all views.
* **Synchronization**: Because all views share the same `_t` (Time) list, they are guaranteed to be perfectly synchronized.
## References
* [Structure of Arrays (SoA)](https://en.wikipedia.org/wiki/AOS_and_SOA)
* [Data Locality](https://gameprogrammingpatterns.com/data-locality.html)
+90 -32
View File
@@ -1,56 +1,114 @@
# TSeries: Time Series Data
# TSeries: Time Series Data Container
## Overview
## What It Does
`TSeries` is a high-performance container for time-series data. Unlike a standard `List<TValue>`, it uses a **Structure of Arrays (SoA)** layout internally. This means it stores timestamps and values in separate contiguous arrays (`List<long>` and `List<double>`).
`TSeries` is a high-performance, memory-efficient container for time-series data. Unlike standard collections (like `List<TValue>`), it uses a **Structure of Arrays (SoA)** layout internally. This means it stores timestamps and values in separate contiguous arrays, optimizing memory access patterns for numerical processing and SIMD vectorization.
This layout is critical for performance because it allows:
1. **SIMD Optimization**: The `Values` property returns a `ReadOnlySpan<double>` that can be directly processed by CPU vector instructions (AVX/SSE).
2. **Cache Locality**: Iterating over values doesn't load timestamps into the CPU cache, and vice versa.
## Design Philosophy
Standard object-oriented collections (Array of Structures - AoS) are cache-inefficient for numerical algorithms. When calculating a moving average, the CPU only needs the values, but an AoS layout forces it to load interleaved timestamps into the cache, wasting bandwidth.
`TSeries` solves this by decoupling time and value storage:
* **Cache Locality**: Iterating over values loads only values.
* **SIMD Readiness**: The internal value array can be exposed directly as a `Span<double>` for AVX/SSE processing.
* **Zero-Copy Views**: Data is accessed without defensive copying, ensuring maximum throughput.
## How It Works
`TSeries` maintains two parallel internal lists:
1. `List<long> _t`: Stores timestamps.
2. `List<double> _v`: Stores values.
It implements `IReadOnlyList<TValue>`, allowing it to be treated as a standard collection of `TValue` structs when needed, but its true power lies in its column-oriented properties (`Values`, `Times`).
## Structure
```csharp
public class TSeries : IReadOnlyList<TValue>
{
// Internal SoA storage
protected readonly List<long> _t;
protected readonly List<double> _v;
### Definition
// Public accessors
public ReadOnlySpan<double> Values => ...; // Zero-copy access
public ReadOnlySpan<long> Times => ...; // Zero-copy access
public TValue Last { get; }
public int Count { get; }
}
```csharp
public class TSeries : IReadOnlyList<TValue>, ITValuePublisher
```
## Key Features
### Core Properties
* **SoA Layout**: Optimized for numerical computing and SIMD.
* **Zero-Copy Access**: `Values` and `Times` properties expose internal storage as Spans without copying.
* **Streaming Support**: The `Add` method supports `isNew` parameter to handle intra-bar updates (replacing the last value instead of appending).
* **Event Publishing**: Optional `Pub` event for reactive pipelines.
| Property | Type | Description |
|----------|------|-------------|
| `Values` | `ReadOnlySpan<double>` | Direct access to the value array (SIMD-ready). |
| `Times` | `ReadOnlySpan<long>` | Direct access to the timestamp array. |
| `Last` | `TValue` | The most recent time-value pair. |
| `Count` | `int` | Number of elements in the series. |
| `Name` | `string` | Optional identifier for the series. |
### Events
| Event | Type | Description |
|-------|------|-------------|
| `Pub` | `Action<TValue>` | Fired whenever a new value is added or updated. |
## Usage
### Creating and Adding Data
### Creating and Populating
```csharp
var series = new TSeries();
series.Add(DateTime.Now, 100.0); // isNew=true by default
// Add a new bar (isNew = true by default)
series.Add(DateTime.UtcNow, 100.0);
// Add multiple values
series.Add(new List<double> { 1.0, 2.0, 3.0 });
```
### Streaming Updates
### Streaming Updates (Real-time)
`TSeries` supports "bar updates" where the last value changes until the bar closes.
```csharp
// New bar
// New minute starts
series.Add(time, 100.0, isNew: true);
// Update current bar (e.g. price change within same minute)
series.Add(time, 101.0, isNew: false);
// Price updates within the same minute
series.Add(time, 101.0, isNew: false); // Overwrites last value
series.Add(time, 102.0, isNew: false); // Overwrites last value
```
### SIMD Processing
```csharp
// Calculate average using SIMD
double avg = series.Values.AverageSIMD();
// Calculate average using SIMD (via Span)
double sum = 0;
foreach (var v in series.Values) { sum += v; } // Compiler vectorizes this
```
### Reactive Subscription
```csharp
series.Pub += (item) => Console.WriteLine($"New value: {item}");
```
## Performance Profile
* **Memory Layout**: SoA (Structure of Arrays).
* **Access Speed**: O(1) for random access.
* **Iteration**: Cache-friendly linear scan.
* **SIMD**: Fully supported via `Values` span.
## Integration
`TSeries` is the standard output format for all indicators in QuanTAlib.
* **Input**: Can be fed into indicators via `Update(TSeries)`.
* **Output**: Indicators return `TSeries` from their `Calculate` methods.
* **Visualization**: Easily mappable to charting libraries due to separate Time/Value arrays.
## Architecture Notes
* **CollectionsMarshal**: Uses `CollectionsMarshal.AsSpan` to expose internal list storage as spans without copying. This is unsafe if the list is modified during span access, but provides maximum performance for single-threaded algorithms.
* **Virtual Methods**: `Add` is virtual to allow derived classes (like `TBarSeries` components) to intercept updates if necessary.
## References
* [Data-Oriented Design](https://en.wikipedia.org/wiki/Data-oriented_design)
* [SIMD in .NET](https://learn.microsoft.com/en-us/dotnet/standard/simd)
-69
View File
@@ -1,69 +0,0 @@
#!meta
{"kernelInfo":{"defaultKernelName":"csharp","items":[{"name":"csharp"},{"name":"fsharp","languageName":"F#","aliases":["f#","fs"]},{"name":"html","languageName":"HTML"},{"name":"http","languageName":"HTTP"},{"name":"javascript","languageName":"JavaScript","aliases":["js"]},{"name":"mermaid","languageName":"Mermaid"},{"name":"pwsh","languageName":"PowerShell","aliases":["powershell"]},{"name":"value"}]}}
#!markdown
# TValue Examples
This notebook demonstrates the usage of `TValue`, the fundamental data structure in QuanTAlib.
For detailed documentation, see [TValue.md](TValue.md).
#!csharp
// Reference the library
#r "..\..\bin\QuanTAlib.dll"
using System;
using QuanTAlib;
#!markdown
## Creating TValue
You can create a `TValue` using `DateTime` or `ticks`.
#!csharp
// Using DateTime
var now = DateTime.UtcNow;
var val1 = new TValue(now, 100.5);
Console.WriteLine($"Created TValue: Time={val1.AsDateTime}, Value={val1.Value}");
// Using Ticks
long ticks = now.AddMinutes(1).Ticks;
var val2 = new TValue(ticks, 101.0);
Console.WriteLine($"Created TValue: Time={val2.AsDateTime}, Value={val2.Value}");
#!markdown
## Implicit Conversions
`TValue` supports implicit conversions to `double` and `DateTime` for convenience.
#!csharp
double d = val1; // Implicitly gets Value
DateTime t = val1; // Implicitly gets Time (as DateTime)
Console.WriteLine($"Double: {d}");
Console.WriteLine($"DateTime: {t}");
// Arithmetic operations using implicit conversion
double result = val1 + 5.0;
Console.WriteLine($"Result (100.5 + 5.0): {result}");
#!markdown
## Immutability
`TValue` is immutable. You cannot change its properties after creation.
#!csharp
// val1.Value = 200; // Error: Property or indexer 'TValue.Value' cannot be assigned to -- it is read only
// To "change" a value, create a new instance
var val3 = new TValue(val1.Time, 200.0);
Console.WriteLine($"New TValue: {val3.Value}");
+83 -21
View File
@@ -1,37 +1,99 @@
# TValue: Time-Value Pair
## Overview
## What It Does
`TValue` is the fundamental building block of QuanTAlib. It represents a single data point in a time series, consisting of a timestamp and a double-precision floating-point value.
`TValue` is the fundamental atomic unit of data in QuanTAlib. It represents a single point in a time series, consisting of a timestamp and a double-precision floating-point value. It serves as the standard input and output format for all indicators and data streams.
It is implemented as a lightweight `readonly struct` to ensure immutability and high performance (stack allocation, no GC overhead).
## Design Philosophy
In high-frequency trading and quantitative analysis, memory allocation is a critical bottleneck. `TValue` is designed as a **lightweight, immutable struct** to ensure:
* **Zero Heap Allocation**: Being a struct, it lives on the stack or embedded in arrays, avoiding Garbage Collector (GC) pressure.
* **Thread Safety**: Immutability guarantees safe concurrent access.
* **Minimal Footprint**: Occupies exactly 16 bytes (8 bytes for `long` Time + 8 bytes for `double` Value), fitting efficiently in CPU cache lines.
## How It Works
`TValue` is implemented as a `readonly record struct`. It encapsulates:
* **Time**: A `long` representing ticks (UTC).
* **Value**: A `double` representing the data magnitude.
It supports implicit conversions to `double` (extracting the value) and `DateTime` (extracting the time), making it syntactically fluid to use in calculations.
## Structure
### Definition
```csharp
public readonly struct TValue
{
public readonly long Time; // Ticks (UTC)
public readonly double Value; // Data value
public readonly bool IsNew; // Metadata for streaming (optional usage)
}
public readonly record struct TValue(long Time, double Value);
```
## Key Features
### Properties
* **Lightweight**: 24 bytes (long + double + bool + padding).
* **Immutable**: Thread-safe by design.
* **Implicit Conversions**: Can be implicitly converted to `double` (returns Value) and `DateTime` (returns Time).
* **Performance**: Designed for high-frequency trading and large dataset processing.
| Property | Type | Description |
|----------|------|-------------|
| `Time` | `long` | Timestamp in ticks (UTC). |
| `Value` | `double` | The data value. |
| `AsDateTime` | `DateTime` | Helper to view `Time` as a `DateTime` object. |
### Constructors
| Constructor | Description |
|-------------|-------------|
| `new TValue(long time, double value)` | Creates a TValue from raw ticks. |
| `new TValue(DateTime time, double value)` | Creates a TValue from a DateTime object. |
## Usage
`TValue` is used throughout the library for:
* Input to indicators (`Update(TValue)`).
* Output from indicators (`Value` property).
* Elements in `TSeries`.
### Creating TValues
## Constructors
```csharp
// From DateTime
var t1 = new TValue(DateTime.UtcNow, 100.5);
* `new TValue(long time, double value, bool isNew = true)`
* `new TValue(DateTime time, double value, bool isNew = true)`
// From Ticks
var t2 = new TValue(DateTime.UtcNow.Ticks, 100.5);
```
### Implicit Conversions
```csharp
TValue tv = new TValue(DateTime.UtcNow, 42.0);
// Implicitly converts to double
double val = tv; // 42.0
// Implicitly converts to DateTime
DateTime dt = tv; // DateTime object
```
### String Representation
```csharp
Console.WriteLine(tv); // Output: "[2024-01-01 12:00:00, 42.00]"
```
## Performance Profile
* **Memory**: 16 bytes per instance.
* **Allocation**: 0 bytes (Stack allocated).
* **Copying**: Cheap (fits in two 64-bit registers).
## Integration
`TValue` is the primary currency of the library:
* **Indicators**: `Update(TValue input)` accepts it.
* **Series**: `TSeries` stores collections of it.
* **Events**: `ITValuePublisher` broadcasts it.
## Architecture Notes
* **SkipLocalsInit**: The struct is marked with `[SkipLocalsInit]` to suppress zero-initialization of locals, squeezing out nanoseconds in tight loops.
* **AggressiveInlining**: All accessors and operators are inlined to ensure zero abstraction penalty.
## References
* [Structure of Arrays (SoA)](https://en.wikipedia.org/wiki/AOS_and_SOA)
* [C# Struct Performance](https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/builtin-types/struct)