Files
mt5cli/pyproject.toml
T
Daichi Narushima 1ffac45d57 feat: add publish_grafana_copy, Grafana examples, and optional OTel metrics (#89)
* feat: Grafana copy publishing, dashboard examples, and optional OTel metrics

Implements three observability improvements:

#82 — publish_grafana_copy(): Uses SQLite online backup API (WAL-safe) to
atomically publish a consistent read-only copy beside the target. Adds
--publish-copy option to grafana-schema and snapshot CLI commands.

#83 — examples/grafana/: Minimal working Grafana setup with docker-compose,
provisioning datasource/dashboard YAML, and three dashboard JSON files
(mt5cli-overview, mt5cli-trades, mt5cli-market). All queries use grafana_*
views; no credentials or private paths included.

#84 — mt5cli/telemetry.py: Optional OTel metrics behind mt5cli[otel] extra.
Base install is unaffected. Adds _Mt5Metrics singleton (no-op until
configure_metrics() is called), wraps update_history() and
update_observability() with record_history_update / record_snapshot_update
context managers, and emits account/position gauges from snapshots.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: replace ambiguous multiplication sign in comment

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: normalize markdown formatting in grafana README

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: preserve file mode on Grafana copy and fix unsupported time macro

- publish_grafana_copy: chmod temp file to match the existing target's
  permissions (or 0o644 when no prior target exists) before atomic
  replace, so Grafana running as a different OS user (e.g. UID 472 in
  Docker) can read the published database
- mt5cli-market.json: replace unsupported \$__timeFilter(time) with the
  epoch-based filter supported by frser-sqlite-datasource:
  "time" >= \$__from / 1000 AND "time" < \$__to / 1000

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: skip Windows-incompatible mode test, rename compose file to compose.yaml

- Skip test_overwrite_preserves_existing_target_mode on win32 since
  Windows chmod does not preserve Unix group/other permission bits
- Simplify test_fresh_target_has_readable_permissions to check owner
  read bit only (portable across platforms)
- Rename docker-compose.yml -> compose.yaml (modern Compose convention)
- Update README and test reference to match new filename

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: rename *.yaml to *.yml in examples/grafana

Renames compose.yaml, mt5cli-sqlite.yaml, and mt5cli.yaml to .yml;
updates README and test references accordingly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: format Grafana dashboards and expand qa script to include JSON

- Update qa.sh prettier pattern to format JSON files alongside markdown
- Reformat Grafana dashboard JSONs with consistent spacing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address owner review comments before merge

- qa.sh: fix Prettier glob from `{,d,json}` to `{md,json}` so Markdown
  files are actually formatted by local QA (P2)
- compose.yml: add GF_INSTALL_PLUGINS env var so the frser-sqlite-datasource
  plugin is installed at container start (P1)
- telemetry.py: replace no-op get_meter() call with a real SDK MeterProvider
  pipeline; add optional `readers` kwarg so callers can inject custom readers
  (e.g. InMemoryMetricReader in tests) without needing the OTLP package (P1)
- sdk.py: aggregate profit and volume by symbol before emitting gauge values
  so hedging accounts with multiple same-symbol positions emit one point per
  symbol instead of overwriting with each row (P2)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: emit mt5_history_update_rows_total via conn.total_changes delta

The counter was registered but never incremented, making the advertised
history-update throughput metric permanently zero. Add add_history_rows()
to _Mt5Metrics and call it in update_history() using the SQLite
total_changes delta measured around write_incremental_datasets().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address three owner review comments

- compose.yml: replace soft fallback with :? error expansion so Compose
  refuses to start when MT5CLI_DB_PATH is unset or empty (P1)
- README.md: tell native Windows users to copy only the datasource
  provisioning file; the dashboards yml contains a Docker-specific path
  that is invalid on Windows (P2)
- telemetry.py / sdk.py: emit mt5_terminal_connected,
  mt5_terminal_trade_allowed, and mt5_terminal_trade_expert gauges via a
  new record_terminal_state() method called from _snapshot_terminal(),
  completing the connection-status metric surface from issue #84 (P2)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: add snapshot freshness panel and win-rate column to dashboards

- mt5cli-overview.json: add a full-width "Last Snapshot" stat panel
  (dateTimeFromNow unit) below the account stats, querying
  MAX(time)*1000 from grafana_account_snapshots so users can tell
  whether Grafana is reading a current published copy (#83)
- mt5cli-trades.json: add win_rate_pct computed column to the Trade
  Statistics by Symbol table via 100.0 * winning_deals / NULLIF(
  total_deals, 0), with a percent unit override and "Win Rate (%)"
  display label (#83)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: reject same source and target path in publish_grafana_copy

Adds an early same-path guard to publish_grafana_copy: resolves both
paths before any I/O and raises ValueError if they are identical,
preventing the function from overwriting the live source database with
its own backup copy. Also adds a unit test for the rejected case.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address ruff EM102/TRY003/E501 in same-path guard

Assigns the ValueError message to a variable before raising and
shortens the test docstring to stay within the 88-char line limit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: apply ruff format to publish_grafana_copy error message

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: remove grafana_ticks panel from default market dashboard

The Tick Bid/Ask panel queried grafana_ticks which only exists when users
collect tick data (opt-in). Users following the default OHLCV-only setup
path hit "no such table: grafana_ticks" on dashboard load.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: close SQLite connections before atomic replace in publish_grafana_copy

Wrap both src and dst connections with contextlib.closing() so they are
explicitly closed before tmp_path.replace(target_path) runs. Without
this, sqlite3.Connection's context manager only commits/rolls back but
leaves the file handle open, which can cause PermissionError on Windows.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: rename history.grafana.db to history.mt5cli.db in Grafana examples

frser-sqlite-datasource blocks paths containing "grafana.db" via its
internal blocklist. Rename the recommended published filename in the
README, compose comment, and datasource provisioning comment to avoid
a blocked/denied datasource for native Windows users.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: update Docker Compose quick-start to pass MT5CLI_DB_PATH

The compose.yml already required MT5CLI_DB_PATH via ${MT5CLI_DB_PATH:?...},
but the README still showed bare `docker compose up -d`. Update the section
to show the env-var-prefixed invocation and document the .env file alternative.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: agent <agent@localhost>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-28 17:14:17 +09:00

196 lines
5.1 KiB
TOML

[project]
name = "mt5cli"
version = "1.1.0"
description = "Generic MT5 data and execution infrastructure for Python applications"
authors = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
maintainers = [{name = "dceoy", email = "dceoy@users.noreply.github.com"}]
license = "MIT"
license-files = ["LICENSE"]
readme = "README.md"
requires-python = ">= 3.11, < 3.14"
dependencies = [
"pdmt5>=1.0.0",
"click >= 8.1.0",
"typer >= 0.15.0",
]
classifiers = [
"Development Status :: 3 - Alpha",
"Environment :: Console",
"License :: OSI Approved :: MIT License",
"Operating System :: Microsoft :: Windows",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Intended Audience :: Financial and Insurance Industry",
"Topic :: Office/Business :: Financial :: Investment",
]
[project.optional-dependencies]
parquet = ["pyarrow >= 19.0.0"]
otel = [
"opentelemetry-api",
"opentelemetry-sdk",
"opentelemetry-exporter-otlp-proto-http",
]
[project.scripts]
mt5cli = "mt5cli.cli:main"
[project.urls]
Repository = "https://github.com/dceoy/mt5cli.git"
[tool.uv]
required-environments = ["platform_system == 'Windows'"]
[dependency-groups]
dev = [
"ruff >= 0.11.0",
"pyright >= 1.1.407",
"pytest>=9.0.3",
"pytest-mock >= 3.12.0",
"pytest-cov >= 5.0.0",
"pandas-stubs >= 2.2.3.250527",
"pyarrow >= 19.0.0",
"opentelemetry-api",
"opentelemetry-sdk",
"mkdocs >= 1.6.1",
"mkdocs-material >= 9.7.6",
"mkdocstrings[python] >= 1.0.4",
"pymdown-extensions >= 10.21.2",
]
[tool.ruff]
line-length = 88
exclude = ["build", ".venv"]
preview = true
[tool.ruff.lint]
select = [
"F", # Pyflakes (F)
"E", # pycodestyle error (E)
"W", # pycodestyle warning (W)
"C90", # mccabe (C90)
"I", # isort (I)
"N", # pep8-naming (N)
"D", # pydocstyle (D)
"UP", # pyupgrade (UP)
"S", # flake8-bandit (S)
"B", # flake8-bugbear (B)
"C4", # flake8-comprehensions (C4)
"SIM", # flake8-simplify (SIM)
"ARG", # flake8-unused-arguments (ARG)
"PD", # pandas-vet (PD)
"PLC", # Pylint convention (PLC)
"PLE", # Pylint error (PLE)
"PLR", # Pylint refactor (PLR)
"PLW", # Pylint warning (PLW)
"FLY", # flynt (FLY)
"NPY", # NumPy-specific rules (NPY)
"PERF", # Perflint (PERF)
"FURB", # refurb (FURB)
"RUF", # Ruff-specific rules (RUF)
"YTT", # flake8-2020 (YTT)
"ANN", # flake8-annotations (ANN)
"ASYNC", # flake8-async (ASYNC)
"BLE", # flake8-blind-except (BLE)
"FBT", # flake8-boolean-trap (FBT)
"A", # flake8-builtins (A)
"COM", # flake8-commas (COM)
"DTZ", # flake8-datetimez (DTZ)
"T10", # flake8-debugger (T10)
"DJ", # flake8-django (DJ)
"EM", # flake8-errmsg (EM)
"EXE", # flake8-executable (EXE)
"FA", # flake8-future-annotations (FA)
"ISC", # flake8-implicit-str-concat (ISC)
"ICN", # flake8-import-conventions (ICN)
"LOG", # flake8-logging (LOG)
"G", # flake8-logging-format (G)
"INP", # flake8-no-pep420 (INP)
"PIE", # flake8-pie (PIE)
"T20", # flake8-print (T20)
"PYI", # flake8-pyi (PYI)
"PT", # flake8-pytest-style (PT)
"Q", # flake8-quotes (Q)
"RSE", # flake8-raise (RSE)
"SLF", # flake8-self (SLF)
"SLOT", # flake8-slots (SLOT)
"TID", # flake8-tidy-imports (TID)
"TCH", # flake8-type-checking (TCH)
"INT", # flake8-gettext (INT)
"PTH", # flake8-use-pathlib (PTH)
"TD", # flake8-todos (TD)
"FIX", # flake8-fixme (FIX)
"ERA", # eradicate (ERA)
"PGH", # pygrep-hooks (PGH)
"TRY", # tryceratops (TRY)
"FAST", # FastAPI (FAST)
"AIR", # Airflow (AIR)
"DOC", # pydoclint (DOC)
]
ignore = [
"COM812", # missing-trailing-comma
"FBT001", # boolean-type-hint-positional-argument
"FBT002", # boolean-default-value-positional-argument
]
[tool.ruff.lint.per-file-ignores]
"tests/**/*.py" = [
"DOC201", # Missing return documentation
"DOC501", # Raised exception missing from docstring
"PLC2701", # Private name import
"PLR0904", # Too many public methods
"PLR2004", # Magic value used in comparison
"PLR6301", # Method could be function/static/classmethod
"S101", # Use of assert (acceptable in tests)
"S106", # Possible hardcoded password
"SLF001", # Private member accessed
]
[tool.ruff.lint.pydocstyle]
convention = "google"
[tool.ruff.lint.pylint]
max-args = 10
max-public-methods = 40
[tool.pyright]
exclude = ["build", ".venv"]
venvPath = "."
venv = ".venv"
typeCheckingMode = "strict"
reportMissingTypeStubs = false
[tool.pytest.ini_options]
addopts = [
"--cov=mt5cli",
"--cov-branch",
"--doctest-modules",
"--capture=no",
]
pythonpath = ["."]
testpaths = ["tests"]
python_files = ["test_*.py", "*_test.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]
minversion = "6.0"
[tool.coverage.run]
source = ["mt5cli"]
omit = [
"**/__init__.py",
"**/__main__.py",
"tests/**",
]
[tool.coverage.report]
show_missing = true
fail_under = 100
exclude_also = [
"if TYPE_CHECKING:",
"^\\s+\\.\\.\\.$",
]
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"