VIOLET V2e: Nautilus Binance data-client spike — GO (qualified)
90s BTCUSDT-PERP capture on the prod host, public data, dummy creds (factory demands key env vars even for the data client; public endpoints ignore the header). NT internal dispatch p50 0.23ms / p99 3.3ms — inside VIOLET reactor budgets; 77 ticks/s sustained, zero drops. Qualifications: NT owns its (uvloop) event loop -> recommended separate feed process bridging via Zinc/HZ; exchange->cb p50 135ms is network+NTP+throttle, a relative baseline only; PRODGREEN-era launcher API calls no longer exist in 1.219 (FUTURES->USDT_FUTURE) — pin version + API smoke on adoption. Exec-client spike remains keys-blocked; executor decision at the V4 gate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
51
prod/VIOLET_dev/nautilus_spike/FINDINGS.md
Normal file
51
prod/VIOLET_dev/nautilus_spike/FINDINGS.md
Normal file
@@ -0,0 +1,51 @@
|
||||
# V2e spike findings — NautilusTrader Binance DATA client as VIOLET feed
|
||||
|
||||
Date: 2026-06-12 · NT 1.219.0 · prod host · public mainnet market data,
|
||||
no real credentials · 90 s BTCUSDT-PERP trade+quote capture
|
||||
(`spike_report.json`, raw numbers; script `spike_nt_binance_feed.py`).
|
||||
|
||||
## Verdict: **GO (qualified)** for NT as the alpha-side Binance feed
|
||||
|
||||
NT's internal dispatch (WS frame → tick object → message bus → strategy
|
||||
callback) costs **p50 0.23 ms / p99 3.3 ms** — comfortably inside the
|
||||
VIOLET reactor budgets (V0 gate: reaction p99 < 10 ms). Throughput at one
|
||||
instrument was 77 ticks/s sustained (847 trades + 6,885 quotes / 90 s)
|
||||
with zero drops; build 0.03 s; first tick 1.1 s after start. The Rust WS
|
||||
stack is healthy on this host.
|
||||
|
||||
Qualifications that shape the integration:
|
||||
|
||||
1. **Loop ownership.** NT installs uvloop and requires a current event
|
||||
loop at node construction (py3.12 raises otherwise). The node owns its
|
||||
loop. In-process coexistence with the VIOLET reactor means running the
|
||||
reactor ON NT's loop — or (recommended) running NT as a separate feed
|
||||
process bridging ticks via Zinc/HZ, which preserves VIOLET's loop
|
||||
discipline and isolates NT lifecycle issues. Decision belongs at the
|
||||
V4 gate per the charter.
|
||||
2. **Keys demanded even for data.** The Binance factory builds an HTTP
|
||||
client (instrument definitions) with MANDATORY `BINANCE_FUTURES_API_KEY/
|
||||
SECRET` env vars. Public endpoints ignore the header, so **dummy values
|
||||
work** — but this is undocumented behavior to pin a test on; a real
|
||||
read-only key is the clean path if adopted.
|
||||
3. **exchange→callback p50 135 ms / p90 827 ms** is dominated by factors
|
||||
outside NT: host→Binance network distance, host NTP skew, and Binance's
|
||||
own stream throttling on the quote side (the bimodal 135 ms vs 827 ms
|
||||
split tracks trades-vs-quotes). Read it as a relative baseline for the
|
||||
lead/lag study, not as NT overhead. For exit-pricing purposes the venue
|
||||
(BingX) feed remains authoritative — this feed is the ALPHA side.
|
||||
4. **API drift is real**: the PRODGREEN-era launcher in-tree uses
|
||||
`BinanceAccountType.FUTURES` and config fields that no longer exist in
|
||||
1.219 (`USDT_FUTURE` now, no `paper_trading` on data config). Any
|
||||
adoption must pin the NT version and carry an API-surface smoke test.
|
||||
|
||||
## What was NOT tested (out of spike scope)
|
||||
Multi-instrument fan-out (the scan universe is ~50 symbols), reconnect
|
||||
behavior under network failure, memory growth over hours, NT exec clients
|
||||
(the BingX ExecutionClient skeleton remains the V2-deferred,
|
||||
keys-blocked item; executor decision at the V4 gate).
|
||||
|
||||
## Next concrete step if adopted
|
||||
Separate `violet_feed_nt` process: NT node + probe-style strategy
|
||||
publishing normalized ticks (asset, bid, ask, last, ts_event, mono_ns) to
|
||||
a Zinc region / HZ map; VIOLET's VenuePriceFeedPort consumes it like any
|
||||
other plane with its own staleness budget.
|
||||
147
prod/VIOLET_dev/nautilus_spike/spike_nt_binance_feed.py
Normal file
147
prod/VIOLET_dev/nautilus_spike/spike_nt_binance_feed.py
Normal file
@@ -0,0 +1,147 @@
|
||||
#!/usr/bin/env python3
|
||||
"""VIOLET V2e spike: NautilusTrader Binance DATA client as a feed candidate.
|
||||
|
||||
Standalone — NEVER imported by the violet package. Public market data only
|
||||
(no API keys). Boots a minimal TradingNode with the Binance USDT-futures
|
||||
data client, subscribes BTCUSDT-PERP trade + quote ticks, and measures:
|
||||
|
||||
- dispatch_ns: strategy-callback wall time − tick.ts_init (NT internal
|
||||
dispatch: WS parse → object → message bus → strategy)
|
||||
- exchange_ns: strategy-callback wall time − tick.ts_event (end-to-end
|
||||
exchange-to-callback; includes network + host NTP skew — read as a
|
||||
relative number, not absolute truth)
|
||||
- inter-tick cadence (ts_init deltas)
|
||||
|
||||
Usage: python spike_nt_binance_feed.py [--seconds 120] [--out report.json]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, "/mnt/dolphinng5_predict")
|
||||
|
||||
from prod.clean_arch.violet.clock import LatencyHistogram # read-only reuse
|
||||
|
||||
from nautilus_trader.adapters.binance.common.enums import BinanceAccountType
|
||||
from nautilus_trader.adapters.binance.config import BinanceDataClientConfig
|
||||
from nautilus_trader.adapters.binance.factories import BinanceLiveDataClientFactory
|
||||
from nautilus_trader.config import LoggingConfig, TradingNodeConfig
|
||||
from nautilus_trader.live.node import TradingNode
|
||||
from nautilus_trader.model.identifiers import InstrumentId, TraderId
|
||||
from nautilus_trader.trading.strategy import Strategy
|
||||
|
||||
IID = InstrumentId.from_str("BTCUSDT-PERP.BINANCE")
|
||||
|
||||
|
||||
class FeedProbe(Strategy):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.dispatch = LatencyHistogram("nt_dispatch")
|
||||
self.exchange = LatencyHistogram("exchange_to_cb")
|
||||
self.cadence = LatencyHistogram("inter_tick")
|
||||
self.quotes = 0
|
||||
self.trades = 0
|
||||
self._last_init_ns = 0
|
||||
self.first_tick_wall = 0.0
|
||||
self.started_wall = 0.0
|
||||
|
||||
def on_start(self) -> None:
|
||||
self.started_wall = time.time()
|
||||
self.subscribe_trade_ticks(IID)
|
||||
self.subscribe_quote_ticks(IID)
|
||||
|
||||
def _measure(self, ts_init: int, ts_event: int) -> None:
|
||||
now = time.time_ns()
|
||||
if not self.first_tick_wall:
|
||||
self.first_tick_wall = time.time()
|
||||
self.dispatch.record(max(0, now - int(ts_init)))
|
||||
self.exchange.record(max(0, now - int(ts_event)))
|
||||
if self._last_init_ns:
|
||||
self.cadence.record(max(0, int(ts_init) - self._last_init_ns))
|
||||
self._last_init_ns = int(ts_init)
|
||||
|
||||
def on_trade_tick(self, tick) -> None:
|
||||
self.trades += 1
|
||||
self._measure(tick.ts_init, tick.ts_event)
|
||||
|
||||
def on_quote_tick(self, tick) -> None:
|
||||
self.quotes += 1
|
||||
self._measure(tick.ts_init, tick.ts_event)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--seconds", type=float, default=120.0)
|
||||
ap.add_argument("--out", type=str, default="")
|
||||
args = ap.parse_args()
|
||||
|
||||
probe = FeedProbe()
|
||||
config = TradingNodeConfig(
|
||||
trader_id=TraderId("VIOLET-SPIKE-001"),
|
||||
data_clients={"BINANCE": BinanceDataClientConfig(
|
||||
account_type=BinanceAccountType.USDT_FUTURE,
|
||||
testnet=False, # public mainnet market data
|
||||
)},
|
||||
exec_clients={}, # NO execution client, ever
|
||||
logging=LoggingConfig(log_level="WARNING"),
|
||||
)
|
||||
# FINDING: NT 1.219 (uvloop) requires a current event loop at
|
||||
# construction — py3.12 no longer auto-creates one. The node OWNS this
|
||||
# loop for its lifetime: in-process coexistence with the VIOLET reactor
|
||||
# means sharing one loop on NT's terms (or a separate feed process).
|
||||
import asyncio
|
||||
loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(loop)
|
||||
node = TradingNode(config=config, loop=loop)
|
||||
node.add_data_client_factory("BINANCE", BinanceLiveDataClientFactory)
|
||||
t_build0 = time.time()
|
||||
node.build()
|
||||
build_s = time.time() - t_build0
|
||||
node.trader.add_strategy(probe)
|
||||
|
||||
timer = threading.Timer(args.seconds, node.stop)
|
||||
timer.daemon = True
|
||||
timer.start()
|
||||
t_run0 = time.time()
|
||||
try:
|
||||
node.run()
|
||||
finally:
|
||||
try:
|
||||
node.dispose()
|
||||
except Exception:
|
||||
pass
|
||||
ran_s = time.time() - t_run0
|
||||
|
||||
ticks = probe.trades + probe.quotes
|
||||
report = {
|
||||
"utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
|
||||
"nt_version": __import__("nautilus_trader").__version__,
|
||||
"build_seconds": round(build_s, 2),
|
||||
"run_seconds": round(ran_s, 2),
|
||||
"time_to_first_tick_s": (round(probe.first_tick_wall - probe.started_wall, 2)
|
||||
if probe.first_tick_wall else None),
|
||||
"trades": probe.trades,
|
||||
"quotes": probe.quotes,
|
||||
"ticks_per_s": round(ticks / max(1e-9, ran_s), 1),
|
||||
"nt_dispatch": probe.dispatch.to_dict(),
|
||||
"exchange_to_cb": probe.exchange.to_dict(),
|
||||
"inter_tick": probe.cadence.to_dict(),
|
||||
}
|
||||
blob = json.dumps(report, indent=2)
|
||||
print(blob)
|
||||
print(probe.dispatch.report())
|
||||
print(probe.exchange.report())
|
||||
print(probe.cadence.report())
|
||||
if args.out:
|
||||
Path(args.out).write_text(blob)
|
||||
return 0 if ticks > 0 else 2
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
46
prod/VIOLET_dev/nautilus_spike/spike_report.json
Normal file
46
prod/VIOLET_dev/nautilus_spike/spike_report.json
Normal file
@@ -0,0 +1,46 @@
|
||||
{
|
||||
"utc": "2026-06-12T22:53:35Z",
|
||||
"nt_version": "1.219.0",
|
||||
"build_seconds": 0.03,
|
||||
"run_seconds": 100.13,
|
||||
"time_to_first_tick_s": 1.13,
|
||||
"trades": 847,
|
||||
"quotes": 6885,
|
||||
"ticks_per_s": 77.2,
|
||||
"nt_dispatch": {
|
||||
"name": "nt_dispatch",
|
||||
"count": 7732,
|
||||
"retained": 7732,
|
||||
"overflow_dropped": 0,
|
||||
"min_ms": 0.031212,
|
||||
"p50_ms": 0.23393499999999998,
|
||||
"p90_ms": 1.491198,
|
||||
"p99_ms": 3.3297529999999997,
|
||||
"p999_ms": 4.192426,
|
||||
"max_ms": 4.358944
|
||||
},
|
||||
"exchange_to_cb": {
|
||||
"name": "exchange_to_cb",
|
||||
"count": 7732,
|
||||
"retained": 7732,
|
||||
"overflow_dropped": 0,
|
||||
"min_ms": 129.48001399999998,
|
||||
"p50_ms": 135.40447799999998,
|
||||
"p90_ms": 826.803426,
|
||||
"p99_ms": 1032.154036,
|
||||
"p999_ms": 1034.66528,
|
||||
"max_ms": 1034.749529
|
||||
},
|
||||
"inter_tick": {
|
||||
"name": "inter_tick",
|
||||
"count": 7731,
|
||||
"retained": 7731,
|
||||
"overflow_dropped": 0,
|
||||
"min_ms": 0.019243,
|
||||
"p50_ms": 0.05735,
|
||||
"p90_ms": 9.996751999999999,
|
||||
"p99_ms": 269.134981,
|
||||
"p999_ms": 621.126351,
|
||||
"max_ms": 1556.423341
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user