11 Commits

Author SHA1 Message Date
gavindiaz de5893de3b 添加详细说明文档 2026-07-07 20:25:32 +00:00
Michael Halls-Moore 2c53efb4f3 Merge pull request #28 from mhallsmoore/position_fix
Position fix
2015-07-15 08:24:12 +01:00
Michael Halls-Moore 553edab0db Added basic logging capability to trading.py and related classes. 2015-07-13 19:22:27 +01:00
Michael Halls-Moore 675412c125 Modified the position handling to fix a pricing bug, so that locally handled Portfolio values match those of OANDA (up to slippage). 2015-07-13 16:30:55 +01:00
Michael Halls-Moore be9ef2b54f Merge pull request #24 from mhallsmoore/backtest_class
Backtest class
2015-06-30 09:48:56 +01:00
Michael Halls-Moore c273962a04 Modified README to detail new backtest interface. 2015-06-30 09:48:25 +01:00
Michael Halls-Moore 784cfd2508 Added a Backtest class, which replaces the script in backtest.py. Also added an examples directory, to make strategy testing straightforward. 2015-06-23 11:52:44 +01:00
Michael Halls-Moore d191aad641 Modified README to include better installation instructions. 2015-06-03 15:39:35 +01:00
Michael Halls-Moore 458f263722 Multi-day backtesting now supported. 2015-06-03 09:23:40 +01:00
Michael Halls-Moore 17b36c5def Modified heartbeat comment in trading.py. Also added a script in the /scripts directory that generates simulated forex tick data in the style of Dukascopy. 2015-05-27 18:10:55 +01:00
Michael Halls-Moore 40eeecf282 Merge pull request #12 from mhallsmoore/unrealised_pnl
Added the ability for the backtester to use unrealised PnL from the P…
2015-05-15 13:50:58 +01:00
18 changed files with 1365 additions and 213 deletions
+763
View File
@@ -0,0 +1,763 @@
# QSForex 详细说明文档
---
## 一、项目简介
**QSForex** 是一个开源的、基于事件驱动架构的外汇(Forex)量化交易系统,支持**回测**和**实盘交易**。项目由 QuantStart.com 创始人 Michael Halls-Moore 于 2015 年开发,采用 MIT 开源许可证,目前处于 "alpha" 阶段。
### 核心特性
| 特性 | 说明 |
|------|------|
| 事件驱动架构 | 回测与实盘使用同一套事件接口,策略无需修改即可从回测切换到实盘 |
| Tick 级精度 | 支持逐 tick 级别的多日、多货币对回测 |
| 交易成本 | 回测中默认包含点差(Spread)成本 |
| 多货币对 | 支持同时交易多个货币对 |
| 实盘交易 | 通过 OANDA Brokerage API 进行实盘交易 |
| 绩效分析 | 支持权益曲线、收益率、最大回撤等指标计算与可视化 |
| 单元测试 | 核心计算逻辑有完整的单元测试覆盖 |
---
## 二、项目目录结构
```
qsforex/
├── __init__.py # 顶层包入口
├── settings.py # 全局配置(OANDA 凭证、数据路径、杠杆等)
├── logging.conf # 日志配置
├── requirements.txt # Python 依赖包列表
├── event/ # ====== 事件系统(核心通信机制) ======
│ ├── __init__.py
│ └── event.py # 定义 TickEvent / SignalEvent / OrderEvent
├── data/ # ====== 数据层 ======
│ ├── __init__.py
│ ├── price.py # 历史 CSV 数据读取器(回测用)
│ └── streaming.py # OANDA Streaming API 实时数据流(实盘用)
├── strategy/ # ====== 策略层 ======
│ ├── __init__.py
│ └── strategy.py # TestStrategy + MovingAverageCrossStrategy
├── portfolio/ # ====== 组合与仓位管理 ======
│ ├── __init__.py
│ ├── portfolio.py # 资金管理、风险控制、信号执行
│ ├── position.py # 单个仓位的 P&L 计算
│ ├── portfolio_test.py # 组合模块单元测试
│ └── position_test.py # 仓位模块单元测试
├── execution/ # ====== 执行层 ======
│ ├── __init__.py
│ └── execution.py # SimulatedExecution + OANDAExecutionHandler
├── backtest/ # ====== 回测引擎 ======
│ ├── __init__.py
│ ├── backtest.py # Backtest 主循环
│ └── output.py # 可视化输出(权益曲线、回撤图)
├── performance/ # ====== 绩效计算 ======
│ ├── __init__.py
│ └── performance.py # 最大回撤(MDD)计算
├── trading/ # ====== 实盘交易 ======
│ ├── __init__.py
│ └── trading.py # 多线程实时交易循环
├── examples/ # ====== 示例 ======
│ ├── __init__.py
│ └── mac.py # 移动平均线交叉策略回测示例
└── scripts/ # ====== 辅助脚本 ======
├── generate_simulated_pair.py # 生成模拟外汇 tick 数据
└── test_performance.py # 绩效计算调试脚本
```
---
## 三、事件驱动架构详解
### 3.1 概述
整个系统围绕一个**事件队列(Event Queue)** 运转。事件是系统中各组件之间通信的唯一方式。这种设计使得各模块高度解耦,策略代码在回测和实盘之间无需修改即可复用。
### 3.2 事件类型
系统定义了三种事件,按处理顺序依次流转:
```
数据源 → TickEvent → 策略 → SignalEvent → 组合 → OrderEvent → 执行引擎
```
#### TickEvent(行情事件)
由数据层产生,包含市场报价信息。
```python
class TickEvent(Event):
def __init__(self, instrument, time, bid, ask):
self.type = 'TICK'
self.instrument = instrument # 货币对,如 "GBPUSD"
self.time = time # 时间戳
self.bid = bid # 买价
self.ask = ask # 卖价
```
#### SignalEvent(信号事件)
由策略层产生,表示一个交易决策。
```python
class SignalEvent(Event):
def __init__(self, instrument, order_type, side, time):
self.type = 'SIGNAL'
self.instrument = instrument # 货币对
self.order_type = order_type # 订单类型(当前仅支持 "market"
self.side = side # 方向:"buy" 或 "sell"
self.time = time # 产生信号的 tick 时间
```
#### OrderEvent(订单事件)
由组合层产生,表示一个需要发送到经纪商的实际订单。
```python
class OrderEvent(Event):
def __init__(self, instrument, units, order_type, side):
self.type = 'ORDER'
self.instrument = instrument # 货币对
self.units = units # 交易数量
self.order_type = order_type # 订单类型
self.side = side # 方向
```
### 3.3 事件流向图
```
┌──────────────┐
│ 数据源 │
│ (CSV/OANDA) │
└──────┬───────┘
│ TickEvent
┌───────────────────────┐
│ 事件队列 (Queue) │
└───────┬───────────────┘
┌──────────────┼──────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ 策略 │ │ 组合 │ │ 执行 │
│ (Strategy)│ │(Portfolio)│ │(Execution)│
└─────┬────┘ └─────┬────┘ └──────────┘
│ │
│ SignalEvent │ OrderEvent
└─────────────┘
(放回事件队列)
```
---
## 四、各模块详细说明
### 4.1 数据层(data/
#### 4.1.1 PriceHandler(抽象基类)
[price.py](file:///c:/Users/Administrator/Desktop/qsforex/data/price.py#L15-L76)
所有数据处理器继承自 `PriceHandler`,提供以下核心功能:
- **`_set_up_prices_dict()`**:构建价格字典,同时维护正向和反向汇率对。例如,当交易 `GBPUSD` 时,系统自动计算并维护 `USDGBP` 的 bid/ask,用于跨货币对的盈亏换算。
- **`invert_prices()`**:将 bid/ask 价格取倒数,生成反向汇率对。
#### 4.1.2 HistoricCSVPriceHandler(历史数据处理器)
[price.py](file:///c:/Users/Administrator/Desktop/qsforex/data/price.py#L77-L202)
用于回测场景,从 CSV 文件读取历史 tick 数据。
**数据格式**:兼容 DukasCopy Historical Data Feed 格式,文件名格式为 `{货币对}_{YYYYMMDD}.csv`,例如 `GBPUSD_20140112.csv`
CSV 文件内部格式:
```
Time,Ask,Bid,AskVolume,BidVolume
12.01.2014 22:00:10.123,1.50200,1.50180,1.50,1.50
```
**处理流程**
1. 扫描 `CSV_DATA_DIR` 目录下所有符合格式的 CSV 文件
2. 提取所有日期,去重排序
3. 按天加载数据,使用 Pandas 将多个货币对的数据按时间合并排序
4. `stream_next_tick()` 方法每次被调用时,按时间顺序输出下一个 tick
5. 每输出一个 tick,同时更新正向和反向汇率对的价格
#### 4.1.3 StreamingForexPrices(实时数据流处理器)
[streaming.py](file:///c:/Users/Administrator/Desktop/qsforex/data/streaming.py#L16-L91)
用于实盘交易场景,通过 OANDA 的 Streaming API 获取实时价格。
**处理流程**
1. 建立到 OANDA streaming 端点的 HTTPS 连接
2. 使用 `requests.Session` 的流式传输持续接收数据
3. 逐行解析 JSON 格式的 tick 数据
4. 生成 `TickEvent` 放入事件队列
5. 同步更新正向和反向汇率对价格
---
### 4.2 策略层(strategy/
[strategy.py](file:///c:/Users/Administrator/Desktop/qsforex/strategy/strategy.py)
策略层负责从 `TickEvent` 中分析市场数据,产生交易信号。所有策略必须实现 `calculate_signals(event)` 方法。
#### 4.2.1 TestStrategy(测试策略)
简单的测试策略,每 5 个 tick 交替买入/卖出。设计目的是持续"穿越点差",因此必然亏损,仅用于验证系统运行是否正常。
```
逻辑:
tick 0-4: 无操作
tick 5: BUY → 持仓
tick 10: SELL → 平仓
tick 15: BUY → 持仓
...循环
```
#### 4.2.2 MovingAverageCrossStrategy(双均线交叉策略)
经典的移动平均线交叉策略,仅做多。
**参数**
| 参数 | 默认值 | 说明 |
|------|--------|------|
| `short_window` | 500 | 短期 SMA 窗口(tick 数) |
| `long_window` | 2000 | 长期 SMA 窗口(tick 数) |
**信号逻辑**
- 短期 SMA 上穿长期 SMA 且未持仓 → 产生 `BUY` 信号(开多仓)
- 短期 SMA 下穿长期 SMA 且已持仓 → 产生 `SELL` 信号(平多仓)
**性能优化**:使用**滚动 SMA 算法**,避免每次重新计算完整均线:
```python
def calc_rolling_sma(self, sma_m_1, window, price):
return ((sma_m_1 * (window - 1)) + price) / window
```
该算法将时间复杂度从 O(n) 降为 O(1)。
**多货币对支持**:通过 `pairs_dict` 为每个货币对维护独立的 tick 计数、持仓状态和 SMA 值。
---
### 4.3 组合与仓位管理(portfolio/
#### 4.3.1 Position(仓位类)
[position.py](file:///c:/Users/Administrator/Desktop/qsforex/portfolio/position.py#L5-L108)
管理单个货币对的交易仓位,支持多空双向。
**核心属性**
| 属性 | 说明 |
|------|------|
| `home_currency` | 账户计价货币(如 GBP |
| `position_type` | 仓位方向:`"long"``"short"` |
| `currency_pair` | 交易货币对(如 EURUSD |
| `units` | 持仓数量 |
| `avg_price` | 平均持仓成本 |
| `cur_price` | 当前市场价格 |
| `profit_base` | 以计价货币计算的浮动盈亏 |
| `profit_perc` | 浮动盈亏百分比 |
**关键方法**
| 方法 | 说明 |
|------|------|
| `calculate_pips()` | 计算当前盈亏点数 |
| `calculate_profit_base()` | 计算以账户货币计价的盈亏金额 |
| `update_position_price()` | 更新当前价格并重新计算盈亏 |
| `add_units(units)` | 加仓,重新计算平均成本 |
| `remove_units(units)` | 减仓,返回已实现盈亏 |
| `close_position()` | 平仓,返回已实现盈亏 |
**跨货币对换算**:对于 EUR/USD 仓位,账户以 GBP 计价时的盈亏计算:
```
Profit_GBP = Pips × USD/GBP_Rate × Units
```
#### 4.3.2 Portfolio(组合类)
[portfolio.py](file:///c:/Users/Administrator/Desktop/qsforex/portfolio/portfolio.py#L20-L193)
管理整个交易组合,负责资金管理、风险控制和信号执行。
**核心属性**
| 属性 | 默认值 | 说明 |
|------|--------|------|
| `equity` | 100,000.00 | 账户总权益 |
| `balance` | = equity | 当前余额 |
| `leverage` | 20 | 杠杆倍数 |
| `risk_per_trade` | 0.02 (2%) | 单笔交易风险比例 |
| `trade_units` | equity × 0.02 | 单笔交易头寸规模 |
**关键方法**
| 方法 | 说明 |
|------|------|
| `calc_risk_position_size()` | 计算基于风险的仓位大小 |
| `add_new_position()` | 创建新仓位 |
| `add_position_units()` | 对已有仓位加仓 |
| `remove_position_units()` | 对已有仓位减仓 |
| `close_position()` | 平掉整个仓位 |
| `update_portfolio(tick_event)` | 更新所有持仓的浮动盈亏 |
| `execute_signal(signal_event)` | 执行交易信号,生成 OrderEvent |
| `output_results()` | 输出回测结果到 CSV |
| `create_equity_file()` | 创建回测结果 CSV 文件 |
**信号执行逻辑** [execute_signal](file:///c:/Users/Administrator/Desktop/qsforex/portfolio/portfolio.py#L135-L192)
| 信号方向 | 当前仓位 | 动作 |
|---------|---------|------|
| `buy` | 无仓位 | 创建多头仓位 |
| `buy` | 有多头仓位 | 加仓 |
| `buy` | 有空头仓位 | 若 units 相等则平仓 |
| `sell` | 无仓位 | 创建空头仓位 |
| `sell` | 有多头仓位 | 若 units 相等则平仓 |
| `sell` | 有空头仓位 | 加仓 |
---
### 4.4 执行层(execution/
[execution.py](file:///c:/Users/Administrator/Desktop/qsforex/execution/execution.py)
#### 4.4.1 ExecutionHandler(抽象基类)
定义了 `execute_order()` 接口,所有执行处理器必须实现此方法。
#### 4.4.2 SimulatedExecution(模拟执行)
回测模式下的执行器,`execute_order()` 方法为空操作。实际的成交处理由 `Portfolio` 对象直接完成。
#### 4.4.3 OANDAExecutionHandlerOANDA 执行器)
实盘模式下的执行器,通过 OANDA REST API 发送订单。
**下单流程**
1. 建立到 OANDA API 域名的 HTTPS 连接
2. 构造 HTTP POST 请求,包含认证信息和订单参数
3. 发送到 `/v1/accounts/{account_id}/orders` 端点
4. 记录 API 响应日志
---
### 4.5 回测引擎(backtest/
#### 4.5.1 Backtest 类
[backtest.py](file:///c:/Users/Administrator/Desktop/qsforex/backtest/backtest.py#L12-L82)
回测引擎的核心类,将所有组件组装在一起运行回测。
**初始化参数**
| 参数 | 说明 |
|------|------|
| `pairs` | 交易货币对列表 |
| `data_handler` | 数据处理器类(如 `HistoricCSVPriceHandler` |
| `strategy` | 策略类(如 `MovingAverageCrossStrategy` |
| `strategy_params` | 策略参数字典 |
| `portfolio` | 组合管理类 |
| `execution` | 执行处理器类 |
| `equity` | 初始资金 |
| `heartbeat` | 事件循环心跳间隔(秒) |
| `max_iters` | 最大迭代次数 |
**回测主循环** `_run_backtest()`
```python
while iters < max_iters and ticker.continue_backtest:
try:
event = events.get(False) # 非阻塞获取事件
except queue.Empty:
ticker.stream_next_tick() # 没有事件则推进下一个 tick
else:
if event.type == 'TICK':
strategy.calculate_signals(event) # 策略分析
portfolio.update_portfolio(event) # 更新盈亏
elif event.type == 'SIGNAL':
portfolio.execute_signal(event) # 执行信号
elif event.type == 'ORDER':
execution.execute_order(event) # 执行订单
time.sleep(heartbeat)
iters += 1
```
**关键设计**:当事件队列为空时,主动从数据源获取下一个 tick 并放入队列,而非等待。这种"拉取"模式确保回测的确定性和可复现性。
#### 4.5.2 可视化输出
[output.py](file:///c:/Users/Administrator/Desktop/qsforex/backtest/output.py)
读取 `equity.csv` 文件,使用 Matplotlib + Seaborn 绘制三张图表:
1. **权益曲线**(上图):组合价值随时间变化
2. **期间收益率**(中图):每个 tick 的收益率
3. **回撤曲线**(下图):从历史高点的回撤幅度
---
### 4.6 绩效计算(performance/
[performance.py](file:///c:/Users/Administrator/Desktop/qsforex/performance/performance.py)
`create_drawdowns(pnl)` 函数:
- 计算权益曲线的**峰值到谷值的最大回撤(Maximum Drawdown, MDD**
- 计算**回撤持续时间**
- 返回 `(drawdown_series, max_drawdown, max_duration)`
---
### 4.7 实盘交易(trading/
[trading.py](file:///c:/Users/Administrator/Desktop/qsforex/trading/trading.py)
实盘交易的入口,使用**双线程**架构:
```
线程 1(价格流): StreamingForexPrices.stream_to_queue()
└── 持续从 OANDA 接收实时价格,生成 TickEvent 放入队列
线程 2(交易循环): trade()
└── 从队列获取事件 → 策略 → 组合 → 执行
```
---
## 五、配置文件说明
### 5.1 settings.py
```python
# OANDA 环境配置
ENVIRONMENTS = {
"streaming": {
"real": "stream-fxtrade.oanda.com",
"practice": "stream-fxpractice.oanda.com",
"sandbox": "stream-sandbox.oanda.com"
},
"api": {
"real": "api-fxtrade.oanda.com",
"practice": "api-fxpractice.oanda.com",
"sandbox": "api-sandbox.oanda.com"
}
}
# 数据目录和输出目录(从环境变量读取)
CSV_DATA_DIR = os.environ.get('QSFOREX_CSV_DATA_DIR', None)
OUTPUT_RESULTS_DIR = os.environ.get('QSFOREX_OUTPUT_RESULTS_DIR', None)
# 交易环境选择
DOMAIN = "practice" # "practice" / "real" / "sandbox"
# OANDA 认证
ACCESS_TOKEN = os.environ.get('OANDA_API_ACCESS_TOKEN', None)
ACCOUNT_ID = os.environ.get('OANDA_API_ACCOUNT_ID', None)
# 账户设置
BASE_CURRENCY = "GBP" # 账户基础货币
EQUITY = Decimal("100000.00") # 初始资金
```
### 5.2 logging.conf
[logging.conf](file:///c:/Users/Administrator/Desktop/qsforex/logging.conf)
配置了 root logger 和 `qsforex.trading.trading` logger,均输出到控制台,级别为 DEBUG。
---
## 六、使用流程
### 6.1 回测流程
```
步骤 1: 生成模拟数据
└── python scripts/generate_simulated_pair.py GBPUSD
步骤 2: 运行回测
└── python examples/mac.py
步骤 3: 查看结果
└── python backtest/output.py
```
### 6.2 实盘交易流程
```
步骤 1: 配置 OANDA 凭证(settings.py
步骤 2: 编写策略(继承策略基类)
步骤 3: 启动交易
└── python trading/trading.py
```
---
## 七、技术栈
| 类别 | 依赖包 | 版本 |
|------|--------|------|
| 数值计算 | numpy | 1.9.2 |
| 科学计算 | scipy | 0.15.1 |
| 数据处理 | pandas | 0.16.1 |
| 可视化 | matplotlib | 1.4.3 |
| 统计可视化 | seaborn | 0.5.1 |
| 机器学习 | scikit-learn | 0.16.1 |
| HTTP 请求 | requests | 2.7.0 |
| 测试框架 | nose | 1.3.6 |
| Mock | mock | 1.0.1 |
| 交互环境 | ipython | 3.1.0 |
---
## 八、已知问题与改进建议
### 8.1 兼容性问题
| 问题 | 严重程度 | 说明 |
|------|---------|------|
| Python 2.7 语法 | 高 | 使用 `from __future__``try: import Queue``__metaclass__` 等旧式写法,无法在 Python 3 上直接运行 |
| 依赖版本过旧 | 高 | pandas 0.16、numpy 1.9 发布于 2015 年,存在安全漏洞 |
| OANDA API 版本 | 高 | 使用 OANDA v1 API,该版本已被废弃,需迁移到 v3 REST API |
### 8.2 代码缺陷
| 问题 | 位置 | 说明 |
|------|------|------|
| 缺少 `self.` 前缀 | [portfolio.py L165](file:///c:/Users/Administrator/Desktop/qsforex/portfolio/portfolio.py#L165) | `add_position_units` 应为 `self.add_position_units`,会导致 `NameError` |
| 无 setup.py | 根目录 | 缺少标准的 Python 包安装配置,无法通过 `pip install` 安装 |
| 硬编码 | [generate_simulated_pair.py](file:///c:/Users/Administrator/Desktop/qsforex/scripts/generate_simulated_pair.py) | 模拟数据生成的日期、初始价格等参数硬编码 |
### 8.3 改进建议
1. **Python 3 迁移**:将所有语法升级到 Python 3.6+,移除 `from __future__` 导入
2. **依赖更新**:升级到最新版本的 pandas、numpy 等核心依赖
3. **API 升级**:将 OANDA API 从 v1 迁移到 v3
4. **包管理现代化**:添加 `setup.py``pyproject.toml`
5. **Bug 修复**:修复 portfolio.py 中缺失 `self.` 的问题
6. **配置外部化**:将模拟数据生成参数从硬编码改为配置文件或命令行参数
---
## 九、扩展指南
### 9.1 编写自定义策略
只需继承或模仿现有策略,实现 `calculate_signals(event)` 方法:
```python
class MyCustomStrategy(object):
def __init__(self, pairs, events):
self.pairs = pairs
self.events = events
# 初始化你的策略状态
def calculate_signals(self, event):
if event.type == 'TICK':
# 你的策略逻辑
# 当满足条件时,生成 SignalEvent
signal = SignalEvent(pair, "market", "buy", event.time)
self.events.put(signal)
```
然后在回测或实盘入口中替换策略类即可。
### 9.2 添加新的数据源
继承 `PriceHandler` 基类,实现数据获取和 `stream_next_tick()` 方法。
---
## 十、许可证
MIT License - Copyright (c) 2015 Michael Halls-Moore
---
> **风险提示**:外汇保证金交易具有高风险,可能不适合所有投资者。过去的表现不代表未来的结果。高杠杆可能对您不利也可能对您有利。在决定投资外汇之前,您应仔细考虑您的投资目标、经验水平和风险承受能力。
---
## 十一、可复用模块分析 —— 多项目整合视角
> 本章节专门为**综合多个项目构建自有交易系统**的开发者编写,帮助你快速识别 QSForex 中最值得借鉴和复用的模块,以及它在多项目组合中的定位。
### 11.1 本项目在量化交易系统中的定位
QSForex 的核心价值在于**轻量级、教学友好、架构清晰**。它不是一个功能完备的生产级系统,而是一个**架构原型**——用最少的代码完整展示了事件驱动交易系统的骨架。在多项目整合中,QSForex 适合作为:
| 角色 | 说明 |
|------|------|
| **架构参考蓝图** | 事件驱动模式是量化交易系统的主流范式,本项目的实现简洁易懂 |
| **快速原型验证** | 当你有一个新策略想法时,可以最快速度搭出回测环境 |
| **教学/入门材料** | 代码量小、注释清晰,适合团队新成员快速理解量化系统全貌 |
---
### 11.2 模块可复用性矩阵
以下按照**可复用价值从高到低**排列,并标注每个模块在多项目整合中的建议处理方式。
#### ⭐⭐⭐⭐⭐ 强烈推荐复用/借鉴
| 模块 | 核心价值 | 可复用部分 | 跨市场适用性 |
|------|---------|-----------|-------------|
| **事件驱动架构** | ⭐⭐⭐⭐⭐ | 整个事件队列 + 事件类型设计 | 股票/期货/加密货币均可 |
| **回测引擎** | ⭐⭐⭐⭐⭐ | `Backtest._run_backtest()` 主循环 | 通用,只需替换数据源 |
| **仓位管理** | ⭐⭐⭐⭐⭐ | `Position` 类的盈亏计算逻辑 | 外汇特有,但多空双向模式通用 |
| **滚动 SMA 算法** | ⭐⭐⭐⭐ | `calc_rolling_sma()` 高效增量计算 | 所有时间序列策略通用 |
#### 详细分析
**① 事件驱动架构(最核心的复用价值)**
```
事件类型设计 → 事件队列 → 事件分发循环
```
这是 QSForex 的灵魂。无论你最终使用什么市场、什么语言、什么数据库,这个**三层解耦**的模式是量化交易系统的事实标准:
```
数据层 ──TickEvent──→ 策略层 ──SignalEvent──→ 组合层 ──OrderEvent──→ 执行层
↑ ↑ ↑ ↑
可替换 可替换 可替换 可替换
```
**迁移建议**
- 保留事件类型定义(TickEvent / SignalEvent / OrderEvent),按需扩展(如增加 `FillEvent``RiskEvent`
- 保留事件队列 + `while` 循环分发的模式
- 替换各层具体实现即可适配任何市场
**② 回测引擎主循环**
[backtest.py](file:///c:/Users/Administrator/Desktop/qsforex/backtest/backtest.py#L41-L64) 中的 `_run_backtest()` 方法是经典的回测主循环模式:
```python
while iters < max_iters and ticker.continue_backtest:
try:
event = events.get(False) # 非阻塞取事件
except queue.Empty:
ticker.stream_next_tick() # 无事件时推进数据
else:
if event.type == 'TICK':
strategy.calculate_signals(event)
portfolio.update_portfolio(event)
elif event.type == 'SIGNAL':
portfolio.execute_signal(event)
elif event.type == 'ORDER':
execution.execute_order(event)
```
**迁移建议**
- 这个循环结构可以直接移植到任何市场(股票、期货、加密货币)
- 只需实现对应市场的 `PriceHandler` 子类即可
- 可以扩展为支持 `BarEvent`K线事件)以适配日线级别策略
**③ 仓位盈亏计算**
[position.py](file:///c:/Users/Administrator/Desktop/qsforex/portfolio/position.py) 中的 `Position` 类实现了完整的:
- 多空双向持仓
- 平均成本计算(加仓/减仓时重新计算)
- 已实现/未实现盈亏分离
- 跨货币换算
**迁移建议**
- 外汇特有的汇率换算部分可以剥离,核心的 `avg_price` 加权平均算法是通用的
- 期货/股票场景下,需增加保证金计算、合约乘数等字段
- 加密货币场景几乎可以直接复用
**④ 滚动 SMA 增量算法**
[strategy.py](file:///c:/Users/Administrator/Desktop/qsforex/strategy/strategy.py#L69-L70) 中的高效均线计算:
```python
def calc_rolling_sma(self, sma_m_1, window, price):
return ((sma_m_1 * (window - 1)) + price) / window
```
这个算法将 SMA 计算从 O(n) 优化到 O(1),在高频回测中效果显著。
---
#### ⭐⭐⭐ 可借鉴思路,需重新实现
| 模块 | 核心价值 | 需改造的原因 |
|------|---------|-------------|
| **数据层** | CSV 多文件按天加载、多货币对按时间归并 | 格式绑定 DukasCopy,且 Python 2.7 的 pandas 语法已过时 |
| **信号执行逻辑** | buy/sell 与多空仓位的状态机 | 逻辑有 Bug(缺少 `self.`),且不支持部分平仓 |
| **绩效计算** | 最大回撤计算 | 实现过于简单,现代系统需要 Sharpe/Sortino/Calmar 等 |
| **可视化输出** | 三图布局(权益+收益+回撤) | matplotlib 版本过旧,可用 plotly/bokeh 替换 |
---
#### ⭐⭐ 参考价值有限,建议替换
| 模块 | 原因 |
|------|------|
| **OANDA 执行器** | 使用已废弃的 OANDA v1 API,必须用 v3 重写或换经纪商 |
| **实时数据流** | 同上,API 版本过时 |
| **模拟数据生成** | 仅生成随机游走数据,无市场微观结构,建议用专业回测数据源 |
| **日志配置** | 可用 structlog 或 loguru 替代 |
---
### 11.3 多项目整合的推荐架构
如果你正在综合多个项目构建自己的交易系统,推荐的分层架构如下:
```
┌─────────────────────────────────────────────────────┐
│ 策略层 (Strategy) │
│ 从 QSForex 借鉴:事件接口 + calculate_signals │
│ 从其他项目借鉴:因子计算、ML模型、信号组合 │
├─────────────────────────────────────────────────────┤
│ 组合层 (Portfolio) │
│ 从 QSForex 借鉴:多空仓位管理 + 风险头寸计算 │
│ 从其他项目借鉴:凯利公式、VaR、组合优化 │
├─────────────────────────────────────────────────────┤
│ 执行层 (Execution) │
│ 从 QSForex 借鉴:SimulatedExecution(回测模式) │
│ 从其他项目借鉴:多经纪商适配、订单路由、智能下单 │
├─────────────────────────────────────────────────────┤
│ 数据层 (Data) │
│ 从 QSForex 借鉴:多标的按时间归并的思想 │
│ 从其他项目借鉴:实时数据库、数据清洗、复权处理 │
├─────────────────────────────────────────────────────┤
│ 事件总线 (Event Bus) │
│ 从 QSForex 借鉴:Events Queue + 事件类型体系 ★★★ │
│ 从其他项目借鉴:Redis/Kafka 分布式事件总线 │
└─────────────────────────────────────────────────────┘
```
---
### 11.4 QSForex 在多项目组合中的角色总结
| 维度 | QSForex 的定位 |
|------|---------------|
| **架构教学** | 🥇 最佳入门教材——事件驱动模式的最简实现 |
| **回测引擎** | 🥈 可作为快速原型验证工具 |
| **策略开发** | 🥈 内置策略简单,但策略接口设计值得参考 |
| **生产就绪度** | ❌ 不适合直接用于生产环境 |
| **跨市场支持** | ❌ 仅外汇,需大量改造才能支持股票/期货 |
| **代码质量** | ⚠️ 有已知 Bug,需要修复后再复用 |
**推荐的复用策略**
1. **提取事件驱动骨架**:将 `event/` + `backtest/` 的核心循环提取为通用框架
2. **修复并提取仓位管理**:修复 `portfolio.py` 的 Bug,抽象出通用的 `Position` 基类
3. **保留策略接口规范**:统一 `calculate_signals(event)` 签名作为所有策略的入口规范
4. **替换其他所有模块**:数据源、执行器、绩效分析、可视化全部用现代方案重新实现
> **一句话总结**QSForex 的**事件驱动架构 + 回测主循环 + 仓位管理**是最值得带走的核心资产,其余模块建议用其他项目的更成熟实现替代。
+113 -14
View File
@@ -1,30 +1,129 @@
# QuantStart Forex
QSForex is an open-source work-in-progress event-driven backtesting and live trading platform for use in the foreign exchange ("forex") markets.
QSForex is an open-source event-driven backtesting and live trading platform for use in the foreign exchange ("forex") markets, currently in an "alpha" state.
It has been created as part of the Forex Trading Diary series on QuantStart.com, predominantly for education purposes, but also to provide the systematic trading community with a robust trading engine that allows straightforward forex strategy implementation and testing.
It has been created as part of the Forex Trading Diary series on QuantStart.com to provide the systematic trading community with a robust trading engine that allows straightforward forex strategy implementation and testing.
The software is provided under a permissive "MIT" license (see below).
# Current Features
* Currently in alpha mode - very early stage!
* Live trading with one particular forex broker, OANDA, via their Rest API
* Event-driven architecture with price streaming (via the OANDA API)
* Basic local portfolio replication of live trades (ultimately for backtesting purposes)
* **Open-Source** - QSForex has been released under an extremely permissive open-source MIT License, which allows full usage in both research and commercial applications, without restriction, but with no warranty of any kind whatsoever.
* **Free** - QSForex is completely free and costs nothing to download or use.
* **Collaboration** - As QSForex is open-source many developers collaborate to improve the software. New features are added frequently. Any bugs are quickly determined and fixed.
* **Software Development** - QSForex is written in the Python programming language for straightforward cross-platform support. QSForex contains a suite of unit tests for the majority of its calculation code and new tests are constantly added for new features.</li>
* **Event-Driven Architecture** - QSForex is completely event-driven both for backtesting and live trading, which leads to straightforward transitioning of strategies from a research/testing phase to a live trading implementation.
* **Transaction Costs** - Spread costs are included by default for all backtested strategies.
* **Backtesting** - QSForex features intraday tick-resolution multi-day multi-currency pair backtesting.
* **Trading** - QSForex currently supports live intraday trading using the OANDA Brokerage API across a portfolio of pairs.
* **Performance Metrics** - QSForex currently supports basic performance measurement and equity visualisation via the Matplotlib and Seaborn visualisation libraries.
# Installation and Usage
At this stage, since the software is in alpha mode, the installation instructions are somewhat more involved. As the software evolves they will become more straightforward and documentation will become more extensive. However, the basic approach is as follows:
1) Visit http://www.oanda.com/ and setup an account to obtain the API authentication credentials, which you will need to carry out live trading. I explain how to carry this out in this article: https://www.quantstart.com/articles/Forex-Trading-Diary-1-Automated-Forex-Trading-with-the-OANDA-API.
1. Setup an account with OANDA and obtain the API authentication credentials
2. Clone the repository into a suitable location on your machine
3. Create two environment variables: OANDA_API_ACCESS_TOKEN and OANDA_API_ACCOUNT_ID, which must contain, respectively, the OANDA API Access Token and the OANDA API Account ID, as provided by OANDA
4. Create a virtual environment ("virtualenv") for the QSForex code and utilise pip to install the requirements - `pip install -r requirements.txt`
5. Modify `strategy.py` to create an event-driven strategy class
6. Execute `trading.py` to carry out practice/live trading
2) Clone this git repository into a suitable location on your machine using the following command in your terminal: ```git clone https://github.com/mhallsmoore/qsforex.git```. Alternative you can download the zip file of the current master branch at https://github.com/mhallsmoore/qsforex/archive/master.zip.
If you have any questions about the installation then please feel free to email me at mike AT quantstart DOT com. You might also wish to have a look at the Forex Trading Diary series on QuantStart in order to gain some intuition about how the system is built.
3) Create a set of environment variables for all of the settings found in the ```settings.py``` file in the application root directory. Alternatively, you can "hard code" your specific settings by overwriting the ```os.environ.get(...)``` calls for each setting:
```
# The data directory used to store your backtesting CSV files
CSV_DATA_DIR = "/path/to/your/csv/data/dir"
# The directory where the backtest.csv and equity.csv files
# will be stored after a backtest is carried out
OUTPUT_RESULTS_DIR = "/path/to/your/output/results/dir"
# Change DOMAIN to "real" if you wish to carry out live trading
DOMAIN = "practice"
# Your OANDA API Access Token (found in your Account Details on their website)
ACCESS_TOKEN = "1234123412341234"
# Your OANDA Account ID (found in your Account Details on their website)
ACCOUNT_ID = "1234123412341234"
# Your base currency (e.g. "GBP", "USD", "EUR" etc.)
BASE_CURRENCY = "GBP"
# Your account equity in the base currency (for backtesting)
EQUITY = Decimal("100000.00")
```
4) Create a virtual environment ("virtualenv") for the QSForex code and utilise pip to install the requirements. For instance in a Unix-based system (Mac or Linux) you might create such a directory as follows by entering the following commands in the terminal:
```
mkdir -p ~/venv/qsforex
cd ~/venv/qsforex
virtualenv .
```
This will create a new virtual environment to install the packages into. Assuming you downloaded the QSForex git repository into an example directory such as ```~/projects/qsforex/``` (change this directory below to wherever you installed QSForex), then in order to install the packages you will need to run the following commands:
```
source ~/venv/qsforex/bin/activate
pip install -r ~/projects/qsforex/requirements.txt
```
This will take some time as NumPy, SciPy, Pandas, Scikit-Learn and Matplotlib must be compiled. There are many packages required for this to work, so please take a look at these two articles for more information:
* https://www.quantstart.com/articles/Quick-Start-Python-Quantitative-Research-Environment-on-Ubuntu-14-04
* https://www.quantstart.com/articles/Easy-Multi-Platform-Installation-of-a-Scientific-Python-Stack-Using-Anaconda
You will also need to create a symbolic link from your ```site-packages``` directory to your QSForex installation directory in order to be able to call ```import qsforex``` within the code. To do this you will need a command similar to the following:
```
ln -s ~/projects/qsforex/ ~/venv/qsforex/lib/python2.7/site-packages/qsforex
```
Make sure to change ```~/projects/qsforex``` to your installation directory and ```~/venv/qsforex/lib/python2.7/site-packages/``` to your virtualenv site packages directory.
You will now be able to run the subsequent commands correctly.
## Practice/Live Trading
5) At this stage, if you simply wish to carry out practice or live trading then you can run ```python trading/trading.py```, which will use the default ```TestStrategy``` trading strategy. This simply buys or sells a currency pair every 5th tick. It is purely for testing - do not use it in a live trading environment!
If you wish to create a more useful strategy, then simply create a new class with a descriptive name, e.g. ```MeanReversionMultiPairStrategy``` and ensure it has a ```calculate_signals``` method. You will need to pass this class the ```pairs``` list as well as the ```events``` queue, as in ```trading/trading.py```.
Please look at ```strategy/strategy.py``` for details.
## Backtesting
6) In order to carry out any backtesting it is necessary to generate simulated forex data or download historic tick data. If you wish to simply try the software out, the quickest way to generate an example backtest is to generate some simulated data. The current data format used by QSForex is the same as that provided by the DukasCopy Historical Data Feed at https://www.dukascopy.com/swiss/english/marketwatch/historical/.
To generate some historical data, make sure that the ```CSV_DATA_DIR``` setting in ```settings.py``` is to set to a directory where you want the historical data to live. You then need to run ```generate_simulated_pair.py```, which is under the ```scripts/``` directory. It expects a single command line argument, which in this case is the currency pair in ```BBBQQQ``` format. For example:
```
cd ~/projects/qsforex
python scripts/generate_simulated_pair.py GBPUSD
```
At this stage the script is hardcoded to create a single month's data for January 2014. That is, you will see individual files, of the format ```BBBQQQ_YYYYMMDD.csv``` (e.g. ```GBPUSD_20140112.csv```) appear in your ```CSV_DATA_DIR``` for all business days in that month. If you wish to change the month/year of the data output, simply modify the file and re-run.
7) Now that the historical data has been generated it is possible to carry out a backtest. The backtest file itself is stored in ```backtest/backtest.py```, but this only contains the ```Backtest``` class. To actually execute a backtest you need to instantiate this class and provide it with the necessary modules.
The best way to see how this is done is to look at the example Moving Average Crossover implementation in the ```examples/mac.py``` file and use this as a template. This makes use of the ```MovingAverageCrossStrategy``` which is found in ```strategy/strategy.py```. This defaults to trading both GBP/USD and EUR/USD to demonstrate multiple currency pair usage. It uses data found in ```CSV_DATA_DIR```.
To execute the example backtest, simply run the following:
```
python examples/mac.py
```
**This will take some time.** On my Ubuntu desktop system at home, with the historical data generated via ```generate_simulated_pair.py```, it takes around 5-10 mins to run. A large part of this calculation occurs at the end of the actual backtest, when the drawdown is being calculated, so please remember that the code has not hung up! Please leave it until completion.
8) If you wish to view the performance of the backtest you can simply use ```output.py``` to view an equity curve, period returns (i.e. tick-to-tick returns) and a drawdown curve:
```
python backtest/output.py
```
And that's it! At this stage you are ready to begin creating your own backtests by modifying or appending strategies in ```strategy/strategy.py``` and using real data downloaded from DukasCopy (https://www.dukascopy.com/swiss/english/marketwatch/historical/).
If you have any questions about the installation then please feel free to email me at mike@quantstart.com.
If you have any bugs or other issues that you think may be due to the codebase specifically, feel free to open a Github issue here: https://github.com/mhallsmoore/qsforex/issues
# License Terms
+68 -67
View File
@@ -1,82 +1,83 @@
from __future__ import print_function
import copy
try:
import Queue as queue
except ImportError:
import queue
import threading
import time
from decimal import Decimal, getcontext
from qsforex.execution.execution import SimulatedExecution
from qsforex.portfolio.portfolio import Portfolio
from qsforex import settings
from qsforex.strategy.strategy import TestStrategy, MovingAverageCrossStrategy
from qsforex.data.price import HistoricCSVPriceHandler
def backtest(
events, ticker, strategy, portfolio,
execution, heartbeat, max_iters=200000
class Backtest(object):
"""
Enscapsulates the settings and components for carrying out
an event-driven backtest on the foreign exchange markets.
"""
def __init__(
self, pairs, data_handler, strategy,
strategy_params, portfolio, execution,
equity=100000.0, heartbeat=0.0,
max_iters=10000000000
):
"""
Carries out an infinite while loop that polls the
events queue and directs each event to either the
strategy component of the execution handler. The
loop will then pause for "heartbeat" seconds and
continue unti the maximum number of iterations is
exceeded.
"""
iters = 0
while True and iters < max_iters:
ticker.stream_next_tick()
try:
event = events.get(False)
except queue.Empty:
pass
else:
if event is not None:
if event.type == 'TICK':
strategy.calculate_signals(event)
portfolio.update_portfolio(event)
elif event.type == 'SIGNAL':
portfolio.execute_signal(event)
elif event.type == 'ORDER':
execution.execute_order(event)
time.sleep(heartbeat)
iters += 1
portfolio.output_results()
"""
Initialises the backtest.
"""
self.pairs = pairs
self.events = queue.Queue()
self.csv_dir = settings.CSV_DATA_DIR
self.ticker = data_handler(self.pairs, self.events, self.csv_dir)
self.strategy_params = strategy_params
self.strategy = strategy(
self.pairs, self.events, **self.strategy_params
)
self.equity = equity
self.heartbeat = heartbeat
self.max_iters = max_iters
self.portfolio = portfolio(
self.ticker, self.events, equity=self.equity, backtest=True
)
self.execution = execution()
def _run_backtest(self):
"""
Carries out an infinite while loop that polls the
events queue and directs each event to either the
strategy component of the execution handler. The
loop will then pause for "heartbeat" seconds and
continue unti the maximum number of iterations is
exceeded.
"""
print("Running Backtest...")
iters = 0
while iters < self.max_iters and self.ticker.continue_backtest:
try:
event = self.events.get(False)
except queue.Empty:
self.ticker.stream_next_tick()
else:
if event is not None:
if event.type == 'TICK':
self.strategy.calculate_signals(event)
self.portfolio.update_portfolio(event)
elif event.type == 'SIGNAL':
self.portfolio.execute_signal(event)
elif event.type == 'ORDER':
self.execution.execute_order(event)
time.sleep(self.heartbeat)
iters += 1
if __name__ == "__main__":
heartbeat = 0.0
events = queue.Queue()
equity = settings.EQUITY
def _output_performance(self):
"""
Outputs the strategy performance from the backtest.
"""
print("Calculating Performance Metrics...")
self.portfolio.output_results()
# Load the historic CSV tick data files
pairs = ["GBPUSD"]
csv_dir = settings.CSV_DATA_DIR
if csv_dir is None:
print("No historic data directory provided - backtest terminating.")
sys.exit()
# Create the historic tick data streaming class
ticker = HistoricCSVPriceHandler(pairs, events, csv_dir)
# Create the strategy/signal generator, passing the
# instrument and the events queue
strategy = MovingAverageCrossStrategy(
pairs, events, 500, 2000
)
# Create the portfolio object to track trades
portfolio = Portfolio(
ticker, events, equity=equity, backtest=True
)
# Create the simulated execution handler
execution = SimulatedExecution()
# Carry out the backtest loop
backtest(events, ticker, strategy, portfolio, execution, heartbeat)
def simulate_trading(self):
"""
Simulates the backtest and outputs portfolio performance.
"""
self._run_backtest()
self._output_performance()
print("Backtest complete.")
+71 -28
View File
@@ -1,12 +1,16 @@
from __future__ import print_function
import datetime
from decimal import Decimal, getcontext, ROUND_HALF_DOWN
import os
import os.path
import re
import time
import numpy as np
import pandas as pd
from qsforex import settings
from qsforex.event.event import TickEvent
@@ -96,9 +100,32 @@ class HistoricCSVPriceHandler(PriceHandler):
self.csv_dir = csv_dir
self.prices = self._set_up_prices_dict()
self.pair_frames = {}
self._open_convert_csv_files()
self.file_dates = self._list_all_file_dates()
self.continue_backtest = True
self.cur_date_idx = 0
self.cur_date_pairs = self._open_convert_csv_files_for_day(
self.file_dates[self.cur_date_idx]
)
def _open_convert_csv_files(self):
def _list_all_csv_files(self):
files = os.listdir(settings.CSV_DATA_DIR)
pattern = re.compile("[A-Z]{6}_\d{8}.csv")
matching_files = [f for f in files if pattern.search(f)]
matching_files.sort()
return matching_files
def _list_all_file_dates(self):
"""
Removes the pair, underscore and '.csv' from the
dates and eliminates duplicates. Returns a list
of date strings of the form "YYYYMMDD".
"""
csv_files = self._list_all_csv_files()
de_dup_csv = list(set([d[7:-4] for d in csv_files]))
de_dup_csv.sort()
return de_dup_csv
def _open_convert_csv_files_for_day(self, date_str):
"""
Opens the CSV files from the data directory, converting
them into pandas DataFrames within a pairs dictionary.
@@ -109,13 +136,24 @@ class HistoricCSVPriceHandler(PriceHandler):
in a chronological fashion.
"""
for p in self.pairs:
pair_path = os.path.join(self.csv_dir, '%s.csv' % p)
pair_path = os.path.join(self.csv_dir, '%s_%s.csv' % (p, date_str))
self.pair_frames[p] = pd.io.parsers.read_csv(
pair_path, header=True, index_col=0, parse_dates=True,
pair_path, header=True, index_col=0,
parse_dates=True, dayfirst=True,
names=("Time", "Ask", "Bid", "AskVolume", "BidVolume")
)
self.pair_frames[p]["Pair"] = p
self.all_pairs = pd.concat(self.pair_frames.values()).sort().iterrows()
return pd.concat(self.pair_frames.values()).sort().iterrows()
def _update_csv_for_day(self):
try:
dt = self.file_dates[self.cur_date_idx+1]
except IndexError: # End of file dates
return False
else:
self.cur_date_pairs = self._open_convert_csv_files_for_day(dt)
self.cur_date_idx += 1
return True
def stream_next_tick(self):
"""
@@ -130,30 +168,35 @@ class HistoricCSVPriceHandler(PriceHandler):
well as updating the current bid/ask and inverse bid/ask.
"""
try:
index, row = next(self.all_pairs)
index, row = next(self.cur_date_pairs)
except StopIteration:
return
else:
getcontext().rounding = ROUND_HALF_DOWN
pair = row["Pair"]
bid = Decimal(str(row["Bid"])).quantize(
Decimal("0.00001")
)
ask = Decimal(str(row["Ask"])).quantize(
Decimal("0.00001")
)
# End of the current days data
if self._update_csv_for_day():
index, row = next(self.cur_date_pairs)
else: # End of the data
self.continue_backtest = False
return
getcontext().rounding = ROUND_HALF_DOWN
pair = row["Pair"]
bid = Decimal(str(row["Bid"])).quantize(
Decimal("0.00001")
)
ask = Decimal(str(row["Ask"])).quantize(
Decimal("0.00001")
)
# Create decimalised prices for traded pair
self.prices[pair]["bid"] = bid
self.prices[pair]["ask"] = ask
self.prices[pair]["time"] = index
# Create decimalised prices for traded pair
self.prices[pair]["bid"] = bid
self.prices[pair]["ask"] = ask
self.prices[pair]["time"] = index
# Create decimalised prices for inverted pair
inv_pair, inv_bid, inv_ask = self.invert_prices(pair, bid, ask)
self.prices[inv_pair]["bid"] = inv_bid
self.prices[inv_pair]["ask"] = inv_ask
self.prices[inv_pair]["time"] = index
# Create decimalised prices for inverted pair
inv_pair, inv_bid, inv_ask = self.invert_prices(pair, bid, ask)
self.prices[inv_pair]["bid"] = inv_bid
self.prices[inv_pair]["ask"] = inv_ask
self.prices[inv_pair]["time"] = index
# Create the tick event for the queue
tev = TickEvent(pair, index, bid, ask)
self.events_queue.put(tev)
# Create the tick event for the queue
tev = TickEvent(pair, index, bid, ask)
self.events_queue.put(tev)
+10 -4
View File
@@ -1,9 +1,11 @@
from __future__ import print_function
from decimal import Decimal, getcontext, ROUND_HALF_DOWN
import requests
import logging
import json
import requests
from qsforex.event.event import TickEvent
from qsforex.data.price import PriceHandler
@@ -19,6 +21,7 @@ class StreamingForexPrices(PriceHandler):
self.events_queue = events_queue
self.pairs = pairs
self.prices = self._set_up_prices_dict()
self.logger = logging.getLogger(__name__)
def invert_prices(self, pair, bid, ask):
"""
@@ -38,12 +41,13 @@ class StreamingForexPrices(PriceHandler):
def connect_to_stream(self):
pairs_oanda = ["%s_%s" % (p[:3], p[3:]) for p in self.pairs]
pair_list = ",".join(pairs_oanda)
try:
requests.packages.urllib3.disable_warnings()
s = requests.Session()
url = "https://" + self.domain + "/v1/prices"
headers = {'Authorization' : 'Bearer ' + self.access_token}
params = {'instruments' : pairs_oanda, 'accountId' : self.account_id}
params = {'instruments' : pair_list, 'accountId' : self.account_id}
req = requests.Request('GET', url, headers=headers, params=params)
pre = req.prepare()
resp = s.send(pre, stream=True, verify=False)
@@ -62,10 +66,12 @@ class StreamingForexPrices(PriceHandler):
dline = line.decode('utf-8')
msg = json.loads(dline)
except Exception as e:
print("Caught exception when converting message into json\n" + str(e))
self.logger.error(
"Caught exception when converting message into json: %s" % str(e)
)
return
if "instrument" in msg or "tick" in msg:
print(msg)
self.logger.debug(msg)
getcontext().rounding = ROUND_HALF_DOWN
instrument = msg["tick"]["instrument"].replace("_", "")
time = msg["tick"]["time"]
+29 -2
View File
@@ -10,15 +10,33 @@ class TickEvent(Event):
self.bid = bid
self.ask = ask
def __str__(self):
return "Type: %s, Instrument: %s, Time: %s, Bid: %s, Ask: %s" % (
str(self.type), str(self.instrument),
str(self.time), str(self.bid), str(self.ask)
)
def __repr__(self):
return str(self)
class SignalEvent(Event):
def __init__(self, instrument, order_type, side, time):
self.type = 'SIGNAL'
self.instrument = instrument
self.order_type = order_type
self.side = side
self.side = side
self.time = time # Time of the last tick that generated the signal
def __str__(self):
return "Type: %s, Instrument: %s, Order Type: %s, Side: %s" % (
str(self.type), str(self.instrument),
str(self.order_type), str(self.side)
)
def __repr__(self):
return str(self)
class OrderEvent(Event):
def __init__(self, instrument, units, order_type, side):
@@ -26,4 +44,13 @@ class OrderEvent(Event):
self.instrument = instrument
self.units = units
self.order_type = order_type
self.side = side
self.side = side
def __str__(self):
return "Type: %s, Instrument: %s, Units: %s, Order Type: %s, Side: %s" % (
str(self.type), str(self.instrument), str(self.units),
str(self.order_type), str(self.side)
)
def __repr__(self):
return str(self)
View File
+29
View File
@@ -0,0 +1,29 @@
from __future__ import print_function
from qsforex.backtest.backtest import Backtest
from qsforex.execution.execution import SimulatedExecution
from qsforex.portfolio.portfolio import Portfolio
from qsforex import settings
from qsforex.strategy.strategy import MovingAverageCrossStrategy
from qsforex.data.price import HistoricCSVPriceHandler
if __name__ == "__main__":
# Trade on GBP/USD and EUR/USD
pairs = ["GBPUSD", "EURUSD"]
# Create the strategy parameters for the
# MovingAverageCrossStrategy
strategy_params = {
"short_window": 500,
"long_window": 2000
}
# Create and execute the backtest
backtest = Backtest(
pairs, HistoricCSVPriceHandler,
MovingAverageCrossStrategy, strategy_params,
Portfolio, SimulatedExecution,
equity=settings.EQUITY
)
backtest.simulate_trading()
+4 -2
View File
@@ -5,6 +5,7 @@ try:
import httplib
except ImportError:
import http.client as httplib
import logging
try:
from urllib import urlencode
except ImportError:
@@ -47,6 +48,7 @@ class OANDAExecutionHandler(ExecutionHandler):
self.access_token = access_token
self.account_id = account_id
self.conn = self.obtain_connection()
self.logger = logging.getLogger(__name__)
def obtain_connection(self):
return httplib.HTTPSConnection(self.domain)
@@ -68,6 +70,6 @@ class OANDAExecutionHandler(ExecutionHandler):
"/v1/accounts/%s/orders" % str(self.account_id),
params, headers
)
response = self.conn.getresponse().read()
print(response)
response = self.conn.getresponse().read().decode("utf-8").replace("\n","").replace("\t","")
self.logger.debug(response)
+28
View File
@@ -0,0 +1,28 @@
[loggers]
keys=root,qsforex.trading.trading
[handlers]
keys=consoleHandler
[formatters]
keys=simpleFormatter
[logger_root]
level=DEBUG
handlers=consoleHandler
[logger_qsforex.trading.trading]
level=DEBUG
handlers=consoleHandler
qualname=qsforex.trading.trading
propagate=0
[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=simpleFormatter
args=(sys.stdout,)
[formatter_simpleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=
+3 -3
View File
@@ -26,7 +26,7 @@ def create_drawdowns(pnl):
# Loop over the index range
for t in range(1, len(idx)):
hwm.append(max(hwm[t-1], pnl[t]))
drawdown[t]= (hwm[t]-pnl[t])
duration[t]= (0 if drawdown[t] == 0 else duration[t-1]+1)
hwm.append(max(hwm[t-1], pnl.ix[t]))
drawdown.ix[t]= (hwm[t]-pnl.ix[t])
duration.ix[t]= (0 if drawdown.ix[t] == 0 else duration.ix[t-1]+1)
return drawdown, drawdown.max(), duration.max()
+74 -56
View File
@@ -2,6 +2,7 @@ from __future__ import print_function
from copy import deepcopy
from decimal import Decimal, getcontext, ROUND_HALF_DOWN
import logging
import os
import pandas as pd
@@ -14,9 +15,9 @@ from qsforex.settings import OUTPUT_RESULTS_DIR
class Portfolio(object):
def __init__(
self, ticker, events, home_currency="GBP", leverage=20,
equity=Decimal("100000.00"), risk_per_trade=Decimal("0.02"),
backtest=True
self, ticker, events, home_currency="GBP",
leverage=20, equity=Decimal("100000.00"),
risk_per_trade=Decimal("0.02"), backtest=True
):
self.ticker = ticker
self.events = events
@@ -28,7 +29,9 @@ class Portfolio(object):
self.backtest = backtest
self.trade_units = self.calc_risk_position_size()
self.positions = {}
self.backtest_file = self.create_equity_file()
if self.backtest:
self.backtest_file = self.create_equity_file()
self.logger = logging.getLogger(__name__)
def calc_risk_position_size(self):
return self.equity * self.risk_per_trade
@@ -114,62 +117,77 @@ class Portfolio(object):
if currency_pair in self.positions:
ps = self.positions[currency_pair]
ps.update_position_price()
out_line = "%s,%s" % (tick_event.time, self.balance)
for pair in self.ticker.pairs:
if pair in self.positions:
out_line += ",%s" % self.positions[currency_pair].profit_base
else:
out_line += ",0.00"
out_line += "\n"
if self.backtest:
out_line = "%s,%s" % (tick_event.time, self.balance)
for pair in self.ticker.pairs:
if pair in self.positions:
out_line += ",%s" % self.positions[pair].profit_base
else:
out_line += ",0.00"
out_line += "\n"
print(out_line[:-2])
self.backtest_file.write(out_line)
self.backtest_file.write(out_line)
def execute_signal(self, signal_event):
side = signal_event.side
currency_pair = signal_event.instrument
units = int(self.trade_units)
time = signal_event.time
# If there is no position, create one
if currency_pair not in self.positions:
if side == "buy":
position_type = "long"
def execute_signal(self, signal_event):
# Check that the prices ticker contains all necessary
# currency pairs prior to executing an order
execute = True
tp = self.ticker.prices
for pair in tp:
if tp[pair]["ask"] is None or tp[pair]["bid"] is None:
execute = False
# All necessary pricing data is available,
# we can execute
if execute:
side = signal_event.side
currency_pair = signal_event.instrument
units = int(self.trade_units)
time = signal_event.time
# If there is no position, create one
if currency_pair not in self.positions:
if side == "buy":
position_type = "long"
else:
position_type = "short"
self.add_new_position(
position_type, currency_pair,
units, self.ticker
)
# If a position exists add or remove units
else:
position_type = "short"
self.add_new_position(
position_type, currency_pair,
units, self.ticker
)
ps = self.positions[currency_pair]
# If a position exists add or remove units
if side == "buy" and ps.position_type == "long":
add_position_units(currency_pair, units)
elif side == "sell" and ps.position_type == "long":
if units == ps.units:
self.close_position(currency_pair)
# TODO: Allow units to be added/removed
elif units < ps.units:
return
elif units > ps.units:
return
elif side == "buy" and ps.position_type == "short":
if units == ps.units:
self.close_position(currency_pair)
# TODO: Allow units to be added/removed
elif units < ps.units:
return
elif units > ps.units:
return
elif side == "sell" and ps.position_type == "short":
add_position_units(currency_pair, units)
order = OrderEvent(currency_pair, units, "market", side)
self.events.put(order)
self.logger.info("Portfolio Balance: %s" % self.balance)
else:
ps = self.positions[currency_pair]
if side == "buy" and ps.position_type == "long":
add_position_units(currency_pair, units)
elif side == "sell" and ps.position_type == "long":
if units == ps.units:
self.close_position(currency_pair)
# TODO: Allow units to be added/removed
elif units < ps.units:
return
elif units > ps.units:
return
elif side == "buy" and ps.position_type == "short":
if units == ps.units:
self.close_position(currency_pair)
# TODO: Allow units to be added/removed
elif units < ps.units:
return
elif units > ps.units:
return
elif side == "sell" and ps.position_type == "short":
add_position_units(currency_pair, units)
order = OrderEvent(currency_pair, units, "market", side)
self.events.put(order)
self.logger.info("Unable to execute order as price data was insufficient.")
+4 -4
View File
@@ -212,7 +212,7 @@ class TestPortfolio(unittest.TestCase):
)
self.assertTrue(rpu)
self.assertEqual(ps.units, Decimal("7000"))
self.assertEqual(self.port.balance, Decimal("99988.84"))
self.assertEqual(self.port.balance, Decimal("99988.83"))
def test_close_position_long(self):
position_type = "long"
@@ -265,7 +265,7 @@ class TestPortfolio(unittest.TestCase):
cp = self.port.close_position(currency_pair)
self.assertTrue(cp)
self.assertRaises(ps) # Key doesn't exist
self.assertEqual(self.port.balance, Decimal("100026.64"))
self.assertEqual(self.port.balance, Decimal("100026.63"))
def test_close_position_short(self):
position_type = "short"
@@ -312,13 +312,13 @@ class TestPortfolio(unittest.TestCase):
)
self.assertTrue(rpu)
self.assertEqual(ps.units, Decimal("7000"))
self.assertEqual(self.port.balance, Decimal("99988.84"))
self.assertEqual(self.port.balance, Decimal("99988.83"))
# Close the position
cp = self.port.close_position(currency_pair)
self.assertTrue(cp)
self.assertRaises(ps) # Key doesn't exist
self.assertEqual(self.port.balance, Decimal("99962.80"))
self.assertEqual(self.port.balance, Decimal("99962.77"))
if __name__ == "__main__":
+6 -8
View File
@@ -24,7 +24,7 @@ class Position(object):
ticker_cur = self.ticker.prices[self.currency_pair]
if self.position_type == "long":
self.avg_price = Decimal(str(ticker_cur["ask"]))
self.cur_price = Decimal(str(ticker_cur["bid"]))
self.cur_price = Decimal(str(ticker_cur["bid"]))
else:
self.avg_price = Decimal(str(ticker_cur["bid"]))
self.cur_price = Decimal(str(ticker_cur["ask"]))
@@ -83,11 +83,11 @@ class Position(object):
ticker_cp = self.ticker.prices[self.currency_pair]
ticker_qh = self.ticker.prices[self.quote_home_currency_pair]
if self.position_type == "long":
remove_price = ticker_cp["ask"]
qh_close = ticker_qh["bid"]
else:
remove_price = ticker_cp["bid"]
qh_close = ticker_qh["ask"]
else:
remove_price = ticker_cp["ask"]
qh_close = ticker_qh["bid"]
self.units -= dec_units
self.update_position_price()
# Calculate PnL
@@ -99,11 +99,9 @@ class Position(object):
ticker_cp = self.ticker.prices[self.currency_pair]
ticker_qh = self.ticker.prices[self.quote_home_currency_pair]
if self.position_type == "long":
remove_price = ticker_cp["ask"]
qh_close = ticker_qh["bid"]
else:
remove_price = ticker_cp["bid"]
qh_close = ticker_qh["ask"]
else:
qh_close = ticker_qh["bid"]
self.update_position_price()
# Calculate PnL
pnl = self.calculate_pips() * qh_close * self.units
+80
View File
@@ -0,0 +1,80 @@
from __future__ import print_function
import calendar
import copy
import datetime
import os, os.path
import sys
import numpy as np
import pandas as pd
from qsforex import settings
def month_weekdays(year_int, month_int):
"""
Produces a list of datetime.date objects representing the
weekdays in a particular month, given a year.
"""
cal = calendar.Calendar()
return [
d for d in cal.itermonthdates(year_int, month_int)
if d.weekday() < 5 and d.year == year_int
]
if __name__ == "__main__":
try:
pair = sys.argv[1]
except IndexError:
print("You need to enter a currency pair, e.g. GBPUSD, as a command line parameter.")
else:
np.random.seed(42) # Fix the randomness
S0 = 1.5000
spread = 0.002
mu_dt = 1400 # Milliseconds
sigma_dt = 100 # Millseconds
ask = copy.deepcopy(S0) + spread / 2.0
bid = copy.deepcopy(S0) - spread / 2.0
days = month_weekdays(2014, 1) # January 2014
current_time = datetime.datetime(
days[0].year, days[0].month, days[0].day, 0, 0, 0,
)
# Loop over every day in the month and create a CSV file
# for each day, e.g. "GBPUSD_20150101.csv"
for d in days:
print(d.day)
current_time = current_time.replace(day=d.day)
outfile = open(
os.path.join(
settings.CSV_DATA_DIR,
"%s_%s.csv" % (
pair, d.strftime("%Y%m%d")
)
),
"w")
outfile.write("Time,Ask,Bid,AskVolume,BidVolume\n")
# Create the random walk for the bid/ask prices
# with fixed spread between them
while True:
dt = abs(np.random.normal(mu_dt, sigma_dt))
current_time += datetime.timedelta(0, 0, 0, dt)
if current_time.day != d.day:
outfile.close()
break
else:
W = np.random.standard_normal() * dt / 1000.0 / 86400.0
ask += W
bid += W
ask_volume = 1.0 + np.random.uniform(0.0, 2.0)
bid_volume = 1.0 + np.random.uniform(0.0, 2.0)
line = "%s,%s,%s,%s,%s\n" % (
current_time.strftime("%d.%m.%Y %H:%M:%S.%f")[:-3],
"%0.5f" % ask, "%0.5f" % bid,
"%0.2f00" % ask_volume, "%0.2f00" % bid_volume
)
outfile.write(line)
+35
View File
@@ -0,0 +1,35 @@
"""
This is a small helper script written to help debug issues
with performance calculation, that avoids having to re-run
the full backtest.
In this case it simply works off the "backtest.csv" file that
is produced from a backtest.py run.
"""
import os
import pandas as pd
from qsforex.performance.performance import create_drawdowns
from qsforex.settings import OUTPUT_RESULTS_DIR
if __name__ == "__main__":
in_filename = "backtest.csv"
out_filename = "equity.csv"
in_file = os.path.join(OUTPUT_RESULTS_DIR, in_filename)
out_file = os.path.join(OUTPUT_RESULTS_DIR, out_filename)
# Create equity curve dataframe
df = pd.read_csv(in_file, index_col=0)
df.dropna(inplace=True)
df["Total"] = df.sum(axis=1)
df["Returns"] = df["Total"].pct_change()
df["Equity"] = (1.0+df["Returns"]).cumprod()
# Create drawdown statistics
drawdown, max_dd, dd_duration = create_drawdowns(df["Equity"])
df["Drawdown"] = drawdown
df.to_csv(out_file, index=True)
+34 -22
View File
@@ -1,3 +1,5 @@
import copy
from qsforex.event.event import SignalEvent
@@ -18,7 +20,7 @@ class TestStrategy(object):
self.invested = False
def calculate_signals(self, event):
if event.type == 'TICK':
if event.type == 'TICK' and event.instrument == self.pairs[0]:
if self.ticks % 5 == 0:
if self.invested == False:
signal = SignalEvent(self.pairs[0], "market", "buy", event.time)
@@ -52,39 +54,49 @@ class MovingAverageCrossStrategy(object):
short_window=500, long_window=2000
):
self.pairs = pairs
self.events = events
self.ticks = 0
self.invested = False
self.pairs_dict = self.create_pairs_dict()
self.events = events
self.short_window = short_window
self.long_window = long_window
self.short_sma = None
self.long_sma = None
def create_pairs_dict(self):
attr_dict = {
"ticks": 0,
"invested": False,
"short_sma": None,
"long_sma": None
}
pairs_dict = {}
for p in self.pairs:
pairs_dict[p] = copy.deepcopy(attr_dict)
return pairs_dict
def calc_rolling_sma(self, sma_m_1, window, price):
return ((sma_m_1 * (window - 1)) + price) / window
def calculate_signals(self, event):
if event.type == 'TICK':
pair = event.instrument
price = event.bid
if self.ticks == 0:
self.short_sma = price
self.long_sma = price
pd = self.pairs_dict[pair]
if pd["ticks"] == 0:
pd["short_sma"] = price
pd["long_sma"] = price
else:
self.short_sma = self.calc_rolling_sma(
self.short_sma, self.short_window, price
pd["short_sma"] = self.calc_rolling_sma(
pd["short_sma"], self.short_window, price
)
self.long_sma = self.calc_rolling_sma(
self.long_sma, self.long_window, price
pd["long_sma"] = self.calc_rolling_sma(
pd["long_sma"], self.long_window, price
)
# Only start the strategy when we have created an accurate short window
if self.ticks > self.short_window:
if self.short_sma > self.long_sma and not self.invested:
signal = SignalEvent(self.pairs[0], "market", "buy", event.time)
if pd["ticks"] > self.short_window:
if pd["short_sma"] > pd["long_sma"] and not pd["invested"]:
signal = SignalEvent(pair, "market", "buy", event.time)
self.events.put(signal)
self.invested = True
if self.short_sma < self.long_sma and self.invested:
signal = SignalEvent(self.pairs[0], "market", "sell", event.time)
pd["invested"] = True
if pd["short_sma"] < pd["long_sma"] and pd["invested"]:
signal = SignalEvent(pair, "market", "sell", event.time)
self.events.put(signal)
self.invested = False
self.ticks += 1
pd["invested"] = False
pd["ticks"] += 1
+14 -3
View File
@@ -1,5 +1,7 @@
import copy
from decimal import Decimal, getcontext
import logging
import logging.config
try:
import Queue as queue
except ImportError:
@@ -30,25 +32,32 @@ def trade(events, strategy, portfolio, execution, heartbeat):
else:
if event is not None:
if event.type == 'TICK':
logger.info("Received new tick event: %s", event)
strategy.calculate_signals(event)
portfolio.update_portfolio(event)
elif event.type == 'SIGNAL':
logger.info("Received new signal event: %s", event)
portfolio.execute_signal(event)
elif event.type == 'ORDER':
logger.info("Received new order event: %s", event)
execution.execute_order(event)
time.sleep(heartbeat)
if __name__ == "__main__":
# Set up logging
logging.config.fileConfig('../logging.conf')
logger = logging.getLogger('qsforex.trading.trading')
# Set the number of decimal places to 2
getcontext().prec = 2
heartbeat = 0.0 # Half a second between polling
heartbeat = 0.0 # Time in seconds between polling
events = queue.Queue()
equity = settings.EQUITY
# Trade "Cable"
pairs = ["GBPUSD"]
# Pairs to include in streaming data set
pairs = ["EURUSD", "GBPUSD"]
# Create the OANDA market price streaming class
# making sure to provide authentication commands
@@ -86,5 +95,7 @@ if __name__ == "__main__":
price_thread = threading.Thread(target=prices.stream_to_queue, args=[])
# Start both threads
logger.info("Starting trading thread")
trade_thread.start()
logger.info("Starting price streaming thread")
price_thread.start()