598 lines
27 KiB
Markdown
598 lines
27 KiB
Markdown
|
|
# PINK Capital & Trading Bookkeeping Refactor — Design Spec
|
|||
|
|
|
|||
|
|
**Status**: DESIGN (pre-implementation)
|
|||
|
|
**Date**: 2026-06-07
|
|||
|
|
**Author**: Crush AI, per PINK operator directive
|
|||
|
|
**Scope**: Refactor PINK to use DITAv2 kernel's accounting as the single bookkeeping authority, eliminate double-accounting, maintain BLUE observability compatibility
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 1. PROBLEM STATEMENT
|
|||
|
|
|
|||
|
|
### 1.1 Current State (Post-Flaw-Fix, Pre-This-Refactor)
|
|||
|
|
|
|||
|
|
The 13 structural flaws have been fixed. The kernel works. 157 live BingX testnet
|
|||
|
|
E2E scenarios pass (147 cleanly; 10 fail on test-design issues unrelated to the
|
|||
|
|
kernel). 645+ offline tests are green.
|
|||
|
|
|
|||
|
|
**But the bookkeeping architecture has a structural tension:**
|
|||
|
|
|
|||
|
|
The kernel (`ExecutionKernel` in `rust_backend.py`) has **two parallel accounting
|
|||
|
|
surfaces** that are both live:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
Surface A — Python AccountProjection (account.py)
|
|||
|
|
self.account.snapshot.capital ← mutated by account.settle()
|
|||
|
|
self.account.snapshot.realized_pnl ← accumulated by settle()
|
|||
|
|
self.account.snapshot.peak_capital ← tracked in observe_slots()
|
|||
|
|
|
|||
|
|
Surface B — Rust Kernel Atomic K/E Account (in lib.rs)
|
|||
|
|
k_capital = seed + Σrealized − Σfee − Σfunding
|
|||
|
|
k_realized_pnl ← accumulated in apply_fill()
|
|||
|
|
k_fees_paid ← accumulated in apply_fill()
|
|||
|
|
e_wallet_balance ← from WS/REST exchange facts
|
|||
|
|
reconcile_status/delta ← K-vs-E classifier
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
`ExecutionKernel.snapshot()` **merges both** into one dict (lines 1164-1195),
|
|||
|
|
but they are **independently maintained**:
|
|||
|
|
|
|||
|
|
| Responsibility | Python AccountProjection | Rust K-Account |
|
|||
|
|
|---|---|---|
|
|||
|
|
| `capital` | `settle(incremental_pnl)` on each fill | `seed + Σrealized − Σfee − Σfunding` |
|
|||
|
|
| `realized_pnl` | Accumulated via `settle()` | Accumulated in `apply_fill` |
|
|||
|
|
| `fees_paid` | Accumulated via `settle(fees=)` | Accumulated in `apply_fill` |
|
|||
|
|
| `peak_capital` | Tracked in `observe_slots()` | Tracked in Rust snapshot |
|
|||
|
|
| `open_notional` | Computed in `observe_slots()` | Computed in Rust snapshot |
|
|||
|
|
| `unrealized_pnl` | Read from slot in `observe_slots()` | Computed in Rust snapshot |
|
|||
|
|
| `equity` | `capital + unrealized_pnl` | `k_capital + k_unrealized` |
|
|||
|
|
|
|||
|
|
Both surfaces see the same fills and should agree, but they are **fed by different
|
|||
|
|
code paths**:
|
|||
|
|
|
|||
|
|
- Python `settle()` fires in `on_venue_event()` (rust_backend.py:1033-1036)
|
|||
|
|
- Rust K-account fires in `apply_fill()` inside `on_venue_event` Rust FFI call
|
|||
|
|
|
|||
|
|
These are called sequentially but they can **drift** if:
|
|||
|
|
- A fill event is accepted by Rust but the Python settle path has a different
|
|||
|
|
PnL formula (e.g. fee timing differences)
|
|||
|
|
- `observe_slots()` recomputes `unrealized_pnl` from slot fields while Rust
|
|||
|
|
uses its own mark-price computation
|
|||
|
|
- `_last_settled_pnl` dict tracks per-slot PnL settlement in Python but the
|
|||
|
|
Rust kernel tracks realized PnL per-slot independently
|
|||
|
|
|
|||
|
|
### 1.2 The Persistence Layer's Problem
|
|||
|
|
|
|||
|
|
`PinkClickHousePersistence` reads capital/peak/trade_seq from
|
|||
|
|
`self.account.snapshot` (the Python AccountProjection). It also has its own
|
|||
|
|
`_leg_state` dict for per-leg PnL tracking. This creates **three** parallel
|
|||
|
|
tracking layers:
|
|||
|
|
|
|||
|
|
1. Rust K-account (authoritative for K-vs-E reconciliation)
|
|||
|
|
2. Python AccountProjection (authoritative for persistence reads)
|
|||
|
|
3. Persistence `_leg_state` (authoritative for per-leg PnL deltas)
|
|||
|
|
|
|||
|
|
The `_leg_state` dict is the most dangerous:
|
|||
|
|
- Never cleared when a trade closes (grows unbounded)
|
|||
|
|
- Not synchronized with the kernel
|
|||
|
|
- `pnl_leg` can double-count if a sync step and async pump observe the same fill
|
|||
|
|
- Fallback paths set `prev_realized=0` on reconcile-detected positions
|
|||
|
|
|
|||
|
|
### 1.3 What Prior Attempts Got Wrong
|
|||
|
|
|
|||
|
|
From `SPRINT2_ACCOUNTING_PARITY.md` and `CRITICAL_DITAv2_FLAWS.md`:
|
|||
|
|
|
|||
|
|
1. **Terminal-only PnL settlement (Flaw 5)**: Settle fired only on CLOSED state,
|
|||
|
|
not on partial fills. Fixed with incremental delta settle, but the delta
|
|||
|
|
tracking (`_last_settled_pnl`) lives in Python, not in the kernel.
|
|||
|
|
|
|||
|
|
2. **Double-close paths (Flaw 4)**: Two independent blocks in `apply_fill` each
|
|||
|
|
set CLOSED. Fixed with single `should_close`, but the settlement still fires
|
|||
|
|
from two code paths (Python settle + Rust K-account).
|
|||
|
|
|
|||
|
|
3. **External balance overwrites**: Reconciliation used to overwrite
|
|||
|
|
`account.snapshot.capital` mid-loop. Fixed by restricting external writes to
|
|||
|
|
startup-only, but the kernel's `snapshot()` still returns two capital values
|
|||
|
|
(`capital` from Python, `k_capital` from Rust) and consumers don't know which
|
|||
|
|
to trust.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 2. DESIGN GOALS
|
|||
|
|
|
|||
|
|
| # | Goal | Success Criterion |
|
|||
|
|
|---|---|---|
|
|||
|
|
| G1 | **Single capital authority** | One canonical capital number, not two. All consumers read the same source. |
|
|||
|
|
| G2 | **No parallel PnL tracking** | Eliminate `_leg_state` from persistence. Per-leg PnL comes from the kernel. |
|
|||
|
|
| G3 | **Incremental settle in kernel, not Python** | Move `_last_settled_pnl` into the Rust kernel or a kernel-owned Python adapter. The bridge should not manage settlement state. |
|
|||
|
|
| G4 | **BLUE schema compatibility** | `trade_events`, `position_state`, `account_events`, `policy_events` rows maintain the same columns and semantics. TUI/CH queries unchanged. |
|
|||
|
|
| G5 | **Reconciliation as safety rail, not primary** | Exchange facts (E-block) flow into reconcile classification only. They never directly mutate capital. |
|
|||
|
|
| G6 | **Crash-safe, replayable** | Full state can be saved/restored. No in-memory-only tracking that would be lost on crash. |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 3. PROPOSED ARCHITECTURE
|
|||
|
|
|
|||
|
|
### 3.1 Canonical Authority: Rust K-Account
|
|||
|
|
|
|||
|
|
**The Rust kernel's K-account becomes the single capital authority.**
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
┌─────────────────────────────────┐
|
|||
|
|
│ RUST KERNEL (lib.rs) │
|
|||
|
|
│ │
|
|||
|
|
│ K-ACCOUNT (authoritative) │
|
|||
|
|
│ k_capital = seed + Σrealized │
|
|||
|
|
│ − Σfee − Σfunding │
|
|||
|
|
│ k_realized_pnl (per-slot) │
|
|||
|
|
│ k_fees_paid │
|
|||
|
|
│ k_funding_net │
|
|||
|
|
│ k_peak_capital │
|
|||
|
|
│ k_open_notional │
|
|||
|
|
│ k_unrealized_pnl │
|
|||
|
|
│ k_equity │
|
|||
|
|
│ │
|
|||
|
|
│ E-BLOCK (exchange facts) │
|
|||
|
|
│ e_wallet_balance │
|
|||
|
|
│ e_available_margin │
|
|||
|
|
│ e_used_margin │
|
|||
|
|
│ e_positions[] │
|
|||
|
|
│ │
|
|||
|
|
│ RECONCILE │
|
|||
|
|
│ status / delta / explanation │
|
|||
|
|
└──────────┬───────────────────────┘
|
|||
|
|
│
|
|||
|
|
┌──────────▼───────────────────────┐
|
|||
|
|
│ ExecutionKernel.snapshot() │
|
|||
|
|
│ (ONE account dict, from Rust) │
|
|||
|
|
└──────────┬───────────────────────┘
|
|||
|
|
│
|
|||
|
|
┌───────────────────┼───────────────────┐
|
|||
|
|
│ │ │
|
|||
|
|
┌──────▼──────┐ ┌───────▼───────┐ ┌──────▼──────┐
|
|||
|
|
│ PinkDirect │ │ Persistence │ │ Hz State │
|
|||
|
|
│ Runtime │ │ (CH writer) │ │ Writer │
|
|||
|
|
│ │ │ │ │ │
|
|||
|
|
│ reads │ │ reads │ │ reads │
|
|||
|
|
│ snapshot() │ │ snapshot() │ │ snapshot() │
|
|||
|
|
└─────────────┘ └───────────────┘ └─────────────┘
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 3.2 Eliminate the Python AccountProjection as Capital Authority
|
|||
|
|
|
|||
|
|
The Python `AccountProjection` class stays as a **read-only compatibility shim**
|
|||
|
|
but **stops being mutated**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# BEFORE (current):
|
|||
|
|
self.account.settle(incremental_pnl) # mutates Python capital
|
|||
|
|
self.account.observe_slots(slots) # recomputes unrealized/open_notional
|
|||
|
|
|
|||
|
|
# AFTER (proposed):
|
|||
|
|
# account.settle() is removed from the hot path
|
|||
|
|
# account.observe_slots() is removed from the hot path
|
|||
|
|
# All values come from kernel.snapshot()["account"] which reads Rust directly
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
The `AccountProjection` class is kept only because:
|
|||
|
|
- `PinkClickHousePersistence.__init__` takes it as a constructor argument
|
|||
|
|
- Test code references `kernel.account.snapshot.capital`
|
|||
|
|
- BLUE compat requires `account_events` rows with the same schema
|
|||
|
|
|
|||
|
|
It becomes a **lazy proxy** that reads from the Rust backend on each access:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
class AccountProjectionProxy:
|
|||
|
|
"""Read-only proxy — delegates all reads to the Rust kernel snapshot."""
|
|||
|
|
def __init__(self, kernel):
|
|||
|
|
self._kernel = kernel
|
|||
|
|
|
|||
|
|
@property
|
|||
|
|
def snapshot(self):
|
|||
|
|
return self._kernel._rust_account_snapshot()
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 3.3 Eliminate `_leg_state` from Persistence
|
|||
|
|
|
|||
|
|
Replace `_leg_state` with **kernel-owned per-slot leg tracking**:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
BEFORE:
|
|||
|
|
persistence._leg_state[trade_id] = {
|
|||
|
|
prev_realized: float,
|
|||
|
|
prev_size: float,
|
|||
|
|
prev_leg_id: int
|
|||
|
|
}
|
|||
|
|
→ double-counts on sync+async, never cleared, drifts
|
|||
|
|
|
|||
|
|
AFTER:
|
|||
|
|
kernel.slot(i).realized_pnl → cumulative realized for this slot
|
|||
|
|
kernel.slot(i).active_leg_index → current exit leg
|
|||
|
|
kernel.slot(i).size → remaining position size
|
|||
|
|
|
|||
|
|
Per-leg PnL = current_slot.realized_pnl - prev_call_slot.realized_pnl
|
|||
|
|
(tracked in a local ephemeral variable, not persistent state)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
The persistence layer reads from `slot_dict` (which comes from the kernel) and
|
|||
|
|
computes per-leg deltas **statelessly**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
def _write_trade_exit_leg(self, snapshot, decision, intent, slot_dict, ...):
|
|||
|
|
cur_realized = float(slot_dict.get("realized_pnl", 0.0))
|
|||
|
|
# Statelessly compute the leg delta from the outcome's emitted_events
|
|||
|
|
# rather than tracking _leg_state across calls
|
|||
|
|
leg_pnl = sum(
|
|||
|
|
float(getattr(e, 'realized_pnl', 0.0) or 0.0)
|
|||
|
|
for e in (outcome.emitted_events or ())
|
|||
|
|
if getattr(e, 'kind', None) in (KernelEventKind.FULL_FILL, KernelEventKind.PARTIAL_FILL)
|
|||
|
|
)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 3.4 Move `_last_settled_pnl` into the Kernel
|
|||
|
|
|
|||
|
|
The per-slot settlement tracking (`_last_settled_pnl` dict in
|
|||
|
|
`rust_backend.py:664`) is moved **inside the Rust kernel**:
|
|||
|
|
|
|||
|
|
```rust
|
|||
|
|
// In lib.rs, KernelSlot struct:
|
|||
|
|
pub last_settled_pnl: f64, // tracks what has been pushed to K-account
|
|||
|
|
|
|||
|
|
// In apply_fill():
|
|||
|
|
slot.last_settled_pnl = slot.realized_pnl; // atomic with the fill
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
This ensures the settlement delta is always consistent with the slot state —
|
|||
|
|
no Python dict can drift from the Rust state.
|
|||
|
|
|
|||
|
|
**Python bridge change**:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
# BEFORE:
|
|||
|
|
incremental_pnl = slot.realized_pnl - self._last_settled_pnl.get(slot.slot_id, 0.0)
|
|||
|
|
if abs(incremental_pnl) > 1e-12:
|
|||
|
|
self.account.settle(incremental_pnl)
|
|||
|
|
self._last_settled_pnl[slot.slot_id] = slot.realized_pnl
|
|||
|
|
|
|||
|
|
# AFTER:
|
|||
|
|
# No Python settle call at all — Rust kernel's apply_fill already updated
|
|||
|
|
# k_capital atomically. snapshot() reads k_capital directly.
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 3.5 Unified `snapshot()` Output
|
|||
|
|
|
|||
|
|
The merged snapshot dict simplifies to read Rust K-account directly:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
def snapshot(self) -> Dict[str, Any]:
|
|||
|
|
rust_snap = _get_rust().snapshot(self._backend)
|
|||
|
|
rust_account = rust_snap.get("account", {})
|
|||
|
|
return {
|
|||
|
|
"control": self.control.as_dict(),
|
|||
|
|
"slots": [self._get_slot(i).to_dict() for i in range(self.max_slots)],
|
|||
|
|
"account": {
|
|||
|
|
# Single canonical capital — from Rust K-account
|
|||
|
|
"capital": rust_account.get("k_capital", 0.0),
|
|||
|
|
"equity": rust_account.get("k_equity", 0.0),
|
|||
|
|
"realized_pnl": rust_account.get("k_realized_pnl", 0.0),
|
|||
|
|
"unrealized_pnl": rust_account.get("k_unrealized_pnl", 0.0),
|
|||
|
|
"fees_paid": rust_account.get("k_fees_paid", 0.0),
|
|||
|
|
"funding_paid": rust_account.get("k_funding_net", 0.0),
|
|||
|
|
"open_positions": rust_account.get("open_positions", 0),
|
|||
|
|
"open_notional": rust_account.get("k_open_notional", 0.0),
|
|||
|
|
"peak_capital": rust_account.get("k_peak_capital", 0.0),
|
|||
|
|
# E-block (exchange facts — for reconcile/observability only)
|
|||
|
|
"e_wallet_balance": rust_account.get("e_wallet_balance", 0.0),
|
|||
|
|
"e_available_margin": rust_account.get("e_available_margin", 0.0),
|
|||
|
|
"e_used_margin": rust_account.get("e_used_margin", 0.0),
|
|||
|
|
# Reconcile
|
|||
|
|
"available_capital": rust_account.get("available_capital", rust_account.get("k_capital", 0.0)),
|
|||
|
|
"reconcile_status": rust_account.get("reconcile_status", "OK"),
|
|||
|
|
"reconcile_delta": rust_account.get("reconcile_delta", 0.0),
|
|||
|
|
"event_seq": rust_account.get("event_seq", 0),
|
|||
|
|
},
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
**Key change**: `"capital"` now reads from `k_capital` (Rust), not
|
|||
|
|
`self.account.snapshot.capital` (Python). The Python AccountProjection is no
|
|||
|
|
longer the source.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 4. MIGRATION PATH (4 Phases)
|
|||
|
|
|
|||
|
|
### Phase 1: Rust Kernel Settlement Consolidation (Low Risk)
|
|||
|
|
|
|||
|
|
**Goal**: Make the Rust K-account the single capital mutation point.
|
|||
|
|
|
|||
|
|
**Changes**:
|
|||
|
|
1. In `lib.rs`: ensure `apply_fill()` updates `k_capital` atomically with
|
|||
|
|
`k_realized_pnl` and `k_fees_paid` in the same mutation block.
|
|||
|
|
2. In `lib.rs`: add `last_settled_pnl: f64` to the slot struct.
|
|||
|
|
3. In `rust_backend.py`: remove the `_last_settled_pnl` dict and the
|
|||
|
|
`self.account.settle()` call from `on_venue_event()`.
|
|||
|
|
4. In `rust_backend.py`: `snapshot()["account"]["capital"]` reads
|
|||
|
|
`k_capital` from Rust, not `self.account.snapshot.capital`.
|
|||
|
|
|
|||
|
|
**Test gate**: All 645+ offline tests + 35 flaw tests must pass.
|
|||
|
|
|
|||
|
|
### Phase 2: Python AccountProjection → Proxy (Medium Risk)
|
|||
|
|
|
|||
|
|
**Goal**: Eliminate the Python AccountProjection as a mutable state holder.
|
|||
|
|
|
|||
|
|
**Changes**:
|
|||
|
|
1. Add `AccountProjectionProxy` that reads from Rust snapshot.
|
|||
|
|
2. `ExecutionKernel.account` becomes this proxy (property, not stored field).
|
|||
|
|
3. `PinkClickHousePersistence.__init__` receives the proxy instead of the
|
|||
|
|
mutable `AccountProjection`. All `_capital()` / `_peak_capital()` /
|
|||
|
|
`_trade_seq()` reads go through the proxy → Rust.
|
|||
|
|
4. Remove `observe_slots()` calls from the hot path (Rust already computes
|
|||
|
|
open_notional, unrealized, etc.)
|
|||
|
|
|
|||
|
|
**Test gate**: All offline tests + live e2e canary.
|
|||
|
|
|
|||
|
|
### Phase 3: Persistence `_leg_state` Elimination (Medium Risk)
|
|||
|
|
|
|||
|
|
**Goal**: Remove per-leg tracking from persistence.
|
|||
|
|
|
|||
|
|
**Changes**:
|
|||
|
|
1. Compute per-leg PnL from `outcome.emitted_events` statelessly.
|
|||
|
|
2. Remove `self._leg_state` dict from `PinkClickHousePersistence`.
|
|||
|
|
3. Gate on `outcome` (which carries the events) rather than size-delta tracking.
|
|||
|
|
4. Add a `_prev_realized` ephemeral in `_write_trade_exit_leg` (local variable,
|
|||
|
|
not persistent state).
|
|||
|
|
|
|||
|
|
**Test gate**: Multi-leg exit tests (offline + live).
|
|||
|
|
|
|||
|
|
### Phase 4: Cleanup & Documentation (Low Risk)
|
|||
|
|
|
|||
|
|
**Goal**: Remove dead code, update docs.
|
|||
|
|
|
|||
|
|
**Changes**:
|
|||
|
|
1. Remove `AccountProjection.settle()` and `AccountProjection.observe_slots()`
|
|||
|
|
from the hot path (keep for tests/backward-compat but mark deprecated).
|
|||
|
|
2. Update `SYSTEM_BIBLE_v7.md` §38.7 with the new authority model.
|
|||
|
|
3. Update `SPRINT2_ACCOUNTING_PARITY.md` to document the completed refactor.
|
|||
|
|
4. Add a `CAPITAL_AUTHORITY.md` doc explaining the single-source model.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 5. INVARIANTS TO ENFORCE
|
|||
|
|
|
|||
|
|
### 5.1 Capital Invariants
|
|||
|
|
|
|||
|
|
| # | Invariant | Enforcement |
|
|||
|
|
|---|---|---|
|
|||
|
|
| I1 | `snapshot()["account"]["capital"] == k_capital` (Rust) | snapshot() reads Rust directly |
|
|||
|
|
| I2 | Capital is mutated **only** by `apply_fill()` in Rust | No Python `settle()` calls in hot path |
|
|||
|
|
| I3 | External capital writes happen **only** at `set_seed_capital()` (startup) | Audit all `account.snapshot.capital =` assignments |
|
|||
|
|
| I4 | `k_capital == seed + Σrealized − Σfee − Σfunding` (always reconstructable) | Rust invariant in `apply_fill` |
|
|||
|
|
| I5 | `capital >= 0` always (clamp at seed level) | Rust guard |
|
|||
|
|
|
|||
|
|
### 5.2 PnL Invariants
|
|||
|
|
|
|||
|
|
| # | Invariant | Enforcement |
|
|||
|
|
|---|---|---|
|
|||
|
|
| I6 | Per-slot realized PnL accumulates monotonically within a trade | Rust `apply_fill` only adds, never subtracts |
|
|||
|
|
| I7 | ENTER resets `slot.realized_pnl = 0.0` and `last_settled_pnl = 0.0` | Rust ENTER handler |
|
|||
|
|
| I8 | Per-leg PnL in persistence = `Σ(event.realized_pnl for events in this leg)` | Stateless computation from outcome |
|
|||
|
|
| I9 | Total realized PnL across all slots = `account.realized_pnl` | Snapshot cross-check |
|
|||
|
|
|
|||
|
|
### 5.3 Reconciliation Invariants
|
|||
|
|
|
|||
|
|
| # | Invariant | Enforcement |
|
|||
|
|
|---|---|---|
|
|||
|
|
| I10 | E-facts never mutate K-capital | `on_account_event` only updates E-block fields |
|
|||
|
|
| I11 | `reconcile_status == ERROR` → freeze ENTERs | `is_capital_frozen()` gate in `process_intent` |
|
|||
|
|
| I12 | Reconcile is advisory: K-account is always internally consistent | Rust K-account is a pure fold, no external mutation |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 6. BLUE OBSERVABILITY COMPATIBILITY
|
|||
|
|
|
|||
|
|
### 6.1 ClickHouse Schema (Unchanged)
|
|||
|
|
|
|||
|
|
All existing tables keep their columns. The source of values changes but the
|
|||
|
|
schema does not:
|
|||
|
|
|
|||
|
|
| Table | Key Columns | Source Change |
|
|||
|
|
|---|---|---|
|
|||
|
|
| `trade_events` | `capital_before`, `capital_after`, `pnl`, `pnl_pct` | `capital_*` from `k_capital` (was Python) |
|
|||
|
|
| `position_state` | `capital`, `equity`, `open_notional` | All from Rust K-account |
|
|||
|
|
| `account_events` | `capital`, `equity`, `reconcile_status` | All from Rust snapshot |
|
|||
|
|
| `status_snapshots` | `capital`, `peak_capital`, `trade_seq` | From Rust K-account |
|
|||
|
|
| `policy_events` | (no capital fields) | Unchanged |
|
|||
|
|
| `trade_exit_legs` | `pnl_leg`, `cumulative_pnl` | Computed from `outcome.emitted_events` |
|
|||
|
|
|
|||
|
|
### 6.2 Hazelcast Schema (Unchanged)
|
|||
|
|
|
|||
|
|
`_hz_publish()` reads `slot_dict` and `acc` from `kernel.snapshot()`. The dict
|
|||
|
|
shape is identical; values just come from Rust instead of Python.
|
|||
|
|
|
|||
|
|
### 6.3 TUI Compatibility
|
|||
|
|
|
|||
|
|
`dolphin_status.py` reads from ClickHouse tables. No query changes needed.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 7. RISK ANALYSIS
|
|||
|
|
|
|||
|
|
### 7.1 What Could Go Wrong
|
|||
|
|
|
|||
|
|
| Risk | Severity | Mitigation |
|
|||
|
|
|---|---|---|
|
|||
|
|
| Rust K-account formula drifts from Python AccountProjection | HIGH | Phase 1 test gate: add assertions `k_capital ≈ python_capital` before removing Python settle |
|
|||
|
|
| `_leg_state` removal breaks multi-leg exit rows in CH | MEDIUM | Phase 3 test gate: compare CH rows before/after for multi-leg scenarios |
|
|||
|
|
| `AccountProjectionProxy` breaks test code that expects mutable `.snapshot` | LOW | Proxy returns a read-only dataclass; tests that write `account.snapshot.capital = X` are migrated to `set_seed_capital()` |
|
|||
|
|
| Live BingX timing exposes new race | MEDIUM | Phase 2 canary test before full suite |
|
|||
|
|
|
|||
|
|
### 7.2 What We Explicitly Do NOT Change
|
|||
|
|
|
|||
|
|
1. **Rust FSM logic** — all 13 flaw fixes remain untouched
|
|||
|
|
2. **Venue adapter** — BingX submission/cancel paths unchanged
|
|||
|
|
3. **Decision/Intent engines** — policy layer unchanged
|
|||
|
|
4. **WS account stream** — event ingestion path unchanged
|
|||
|
|
5. **Kernel save/restore** — crash recovery format unchanged (it already
|
|||
|
|
serializes the Rust state including K-account)
|
|||
|
|
6. **Fee model** — calibration and prediction logic unchanged
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 8. FILE CHANGE MANIFEST
|
|||
|
|
|
|||
|
|
| File | Phase | Change |
|
|||
|
|
|---|---|---|
|
|||
|
|
| `_rust_kernel/src/lib.rs` | 1 | Add `last_settled_pnl` to slot; ensure `apply_fill` updates K-account atomically |
|
|||
|
|
| `_rust_kernel/Cargo.toml` | — | (no change) |
|
|||
|
|
| `rust_backend.py` | 1+2 | Remove `_last_settled_pnl` dict; remove `account.settle()` calls; `snapshot()` reads Rust K-account; add `AccountProjectionProxy` |
|
|||
|
|
| `account.py` | 2 | `AccountProjection` gains `deprecated` markers; `AccountProjectionProxy` added (or in rust_backend.py) |
|
|||
|
|
| `pink_clickhouse.py` | 3 | Remove `_leg_state`; compute per-leg PnL from `outcome.emitted_events` |
|
|||
|
|
| `pink_direct.py` | 2 | Remove direct `kernel.account.snapshot.capital =` writes (only at startup via `set_seed_capital`) |
|
|||
|
|
| `launcher.py` | — | (no change — kernel construction unchanged) |
|
|||
|
|
| `contracts.py` | — | (no change) |
|
|||
|
|
| `SYSTEM_BIBLE_v7.md` | 4 | Update §38.7 accounting authority model |
|
|||
|
|
| `CAPITAL_AUTHORITY.md` | 4 | NEW — documents the single-source model |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 9. DECISION RECORD
|
|||
|
|
|
|||
|
|
### 9.1 Why Rust K-Account as Authority (not Python)?
|
|||
|
|
|
|||
|
|
The Rust kernel is:
|
|||
|
|
- **Atomic**: `apply_fill` updates slot + K-account in one FFI call. Python
|
|||
|
|
`settle` is a separate call that can fail independently.
|
|||
|
|
- **Crash-safe**: State is serializable via `save_state()`. Python `_last_settled_pnl`
|
|||
|
|
is in-memory only.
|
|||
|
|
- **Deterministic**: The fold `seed + Σrealized − Σfee − Σfunding` is pure. Python
|
|||
|
|
`settle()` has clamping logic (`min_capital`, `max_capital`) that can mask errors.
|
|||
|
|
- **Test-covered**: 385 FSM matrix tests + 105 extended tests exercise the Rust
|
|||
|
|
state machine directly.
|
|||
|
|
|
|||
|
|
### 9.2 Why Not Remove Python AccountProjection Entirely?
|
|||
|
|
|
|||
|
|
Three blockers:
|
|||
|
|
1. `PinkClickHousePersistence.__init__` takes it as a constructor arg — changing
|
|||
|
|
this signature cascades through all test construction code.
|
|||
|
|
2. `to_account_event()` builds the CH row payload from the snapshot — this method
|
|||
|
|
lives on `AccountProjection`.
|
|||
|
|
3. Tests directly access `kernel.account.snapshot.capital` — a proxy preserves
|
|||
|
|
backward compat while delegating to Rust.
|
|||
|
|
|
|||
|
|
The proxy approach lets us migrate incrementally: the Python class stays as a
|
|||
|
|
read-only facade, and all mutations happen in Rust.
|
|||
|
|
|
|||
|
|
### 9.3 Why Stateless Per-Leg PnL (not a kernel field)?
|
|||
|
|
|
|||
|
|
Per-leg PnL is a **presentation concern**, not a state concern. The kernel
|
|||
|
|
tracks cumulative realized PnL per slot. A "leg" is just a fill event within
|
|||
|
|
the slot's lifetime. Computing `pnl_leg = Σ(event.realized_pnl for this outcome)`
|
|||
|
|
from the outcome's `emitted_events` is:
|
|||
|
|
- Stateless (no tracking dict)
|
|||
|
|
- Correct (uses the actual fill events, not a delta from a potentially-stale prev)
|
|||
|
|
- Crash-safe (nothing to lose)
|
|||
|
|
- Idempotent (same outcome → same leg PnL)
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 10. OPEN QUESTIONS
|
|||
|
|
|
|||
|
|
1. **Should `k_capital` include unrealized PnL in `equity`?**
|
|||
|
|
- Current: `equity = capital + unrealized_pnl` (Python)
|
|||
|
|
- Rust: `k_equity = k_capital + k_unrealized`
|
|||
|
|
- Answer: Yes, keep both. Capital is realized-only; equity includes unrealized.
|
|||
|
|
|
|||
|
|
2. **Should funding fees be settled incrementally or at close?**
|
|||
|
|
- Current: `on_account_event(FUNDING_FEE)` folds into `k_funding_net`
|
|||
|
|
- This already works correctly — no change needed.
|
|||
|
|
|
|||
|
|
3. **Trade_seq tracking** — currently in Python `AccountProjection.snapshot.trade_seq`.
|
|||
|
|
Should this move to Rust?
|
|||
|
|
- Recommendation: Yes, in Phase 1. It's a counter incremented on each fill.
|
|||
|
|
- Risk: Low. It's a simple `+= 1` in `apply_fill`.
|
|||
|
|
|
|||
|
|
4. **Min/max capital clamping** — Python `AccountProjection.settle()` clamps to
|
|||
|
|
`[min_capital, max_capital]`. Rust K-account does not clamp. Should Rust clamp?
|
|||
|
|
- Recommendation: No clamping in Rust. Clamping masks errors. If capital goes
|
|||
|
|
negative, the reconcile ERROR will freeze ENTERs. Let observability catch it.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## APPENDIX A: Current Capital Flow (Before Refactor)
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
FILL EVENT ARRIVES (on_venue_event)
|
|||
|
|
│
|
|||
|
|
├── Rust FFI: on_venue_event(backend, event)
|
|||
|
|
│ └── apply_fill() updates slot.realized_pnl in Rust
|
|||
|
|
│ (slot state is now authoritative for position/PnL)
|
|||
|
|
│
|
|||
|
|
├── Python: incremental_pnl = slot.realized_pnl - _last_settled_pnl[slot_id]
|
|||
|
|
│ └── account.settle(incremental_pnl) ← mutates Python capital
|
|||
|
|
│ (Python AccountProjection is now authoritative for capital)
|
|||
|
|
│
|
|||
|
|
├── Python: account.observe_slots(slots)
|
|||
|
|
│ └── recomputes open_notional, unrealized, equity
|
|||
|
|
│
|
|||
|
|
└── snapshot() returns BOTH:
|
|||
|
|
capital = account.snapshot.capital (Python)
|
|||
|
|
k_capital = rust_account.k_capital (Rust)
|
|||
|
|
Consumers read "capital" → Python authority
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## APPENDIX B: Proposed Capital Flow (After Refactor)
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
FILL EVENT ARRIVES (on_venue_event)
|
|||
|
|
│
|
|||
|
|
├── Rust FFI: on_venue_event(backend, event)
|
|||
|
|
│ └── apply_fill() updates:
|
|||
|
|
│ slot.realized_pnl
|
|||
|
|
│ slot.last_settled_pnl = slot.realized_pnl
|
|||
|
|
│ k_capital += incremental_pnl - incremental_fee
|
|||
|
|
│ k_realized_pnl += incremental_pnl
|
|||
|
|
│ k_fees_paid += fee
|
|||
|
|
│ (ALL atomic in one FFI call)
|
|||
|
|
│
|
|||
|
|
└── snapshot() returns:
|
|||
|
|
capital = k_capital (Rust — single authority)
|
|||
|
|
k_capital = k_capital (same value, kept for compat)
|
|||
|
|
No Python settle() call. No _last_settled_pnl dict.
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 8. CRITICAL BUG FIXES APPLIED — 2026-06-08
|
|||
|
|
|
|||
|
|
These fixes were applied before the Phase-1 refactor above. They make PINK tradeable on the current pre-refactor codebase and are NOT superseded by the refactor.
|
|||
|
|
|
|||
|
|
### 8.1 Defect A — BingX fee sign convention (FIXED)
|
|||
|
|
|
|||
|
|
**Root cause**: `bingx_user_stream._normalise_order()` preserved BingX's `"n"` field verbatim. BingX VST sends `"n"` NEGATIVE for costs; the Rust kernel expected POSITIVE for costs. A cost of 31 VST arrived as -31 → kernel treated it as a rebate → `k_maker_rebates` grew instead of `k_taker_fees` → `k_fees_paid` became negative → `k_capital` was inflated by ~93 VST per round trip → `reconcile_status="ERROR"` → `capital_frozen=True` → all future ENTERs blocked.
|
|||
|
|
|
|||
|
|
**Fix**: `bingx_user_stream.py` line 337: `fee = -raw_fee` (flip sign at the BingX boundary only — do not touch the kernel).
|
|||
|
|
|
|||
|
|
### 8.2 Defect B — Opening fill fee dropped (FIXED)
|
|||
|
|
|
|||
|
|
**Root cause**: BingX ENTER fills report filled quantity only in `"z"` (cumFilledQty), not `"l"` (lastFilledQty). `_normalise_order` used only `"l"` → `fill_qty=0` → `apply_predicted_fill` computed zero opening fee → k-account never recorded the opening-leg cost.
|
|||
|
|
|
|||
|
|
**Fix**: `bingx_user_stream.py` — fall back to `"z"` when `"l"` is zero or absent.
|
|||
|
|
|
|||
|
|
### 8.3 Architectural Fix — WARN zone unfreezes (FIXED)
|
|||
|
|
|
|||
|
|
**Root cause**: `lib.rs reconcile()` kept `capital_frozen=True` when transitioning through WARN (0.01–20 USDT delta). During a live trade, the ENTER predicted-fee phase temporarily pushes `delta > 20` (ERROR, freeze set), then the EXIT's realized PnL brings it back into WARN (< 20). WARN kept the freeze → permanent block after first trade.
|
|||
|
|
|
|||
|
|
**Fix**: `lib.rs` — WARN unfreezes. Only `ERROR` (≥ 20 USDT, unexplained) freezes. WARN = "in-flight settlement, tradeable".
|
|||
|
|
|
|||
|
|
### 8.4 Test coverage
|
|||
|
|
|
|||
|
|
`test_fee_sign_accounting.py` — 34 tests, all green. Includes:
|
|||
|
|
- Prove-Broken (4): characterises pre-fix buggy behaviour
|
|||
|
|
- Prove-Fixed (7): T-2 and T-1 ground-truth round-trips
|
|||
|
|
- Boundary layer (8): sign translation + cumQty fallback
|
|||
|
|
- Full round-trip integration (10): Rust kernel end-to-end
|
|||
|
|
- Edge cases (4): rebates, partials, adversarial inputs
|
|||
|
|
|
|||
|
|
**Acceptance criterion met**: `|k_capital − bingx_wallet_balance| < 1.0 USDT` and `capital_frozen=False` after every round-trip trade.
|