hjnormey/siloqy
Public Access
1
0
Fork 0
Code Issues 7 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

PINK DITAv2 Sprint 2-3: accounting parity + multi-leg groundwork

Browse Source 
Sprint 2 (accounting + observability parity, PINK scope):
- Verified pink_clickhouse.py writes the 8 BLUE-legacy row families at
  matching schema and that capital authority in pink_direct.step() is
  solely kernel.account (no balance-poll overwrite in the hot loop).
- Report: prod/clean_arch/dita_v2/SPRINT2_ACCOUNTING_PARITY.md.

Sprint 3 offline groundwork (no exchange contact):
- Add _write_trade_exit_leg to pink_clickhouse.py: one BLUE-schema-faithful
  trade_exit_legs row per exit leg, with isolated (non-cumulative) per-leg
  deltas tracked via _leg_state (reset on ENTER). Closes the docstring gap.
- New offline suite test_pink_multi_exit_groundwork.py (3 passed):
  * Flaw 4 — two-leg exit closes once, realized accrues per leg, closed
    slot rejects further EXIT (no double-close).
  * Overshoot invariant — a final EXIT requesting more than the remaining
    size CLAMPS (size to 0, no oversell), retiring the Sprint 0 cumulative-
    ratio risk empirically.
  * trade_exit_legs delta + full BLUE column-set assertions.
- Persistence regression after edits: 10 passed.

BLUE untouched: no changes to dolphin.* / DOLPHIN_*_BLUE / nautilus_event_trader.py.
Live VST multi-leg run remains deferred pending explicit authorization.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Codex
2026-05-30 19:21:45 +02:00
parent 3d7b00e28d
commit d4b73b236a
12 changed files with 3527 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

605
prod/docs/PINK_BINGX_SIMPLIFICATION_SPEC_2026-05-22.md Normal file
View File

@@ -0,0 +1,605 @@
# PINK BingX Simplification Spec
Status: Draft for implementation review
Date: 2026-05-22
Owner: Runtime / Trading Systems
Scope: PINK only, with BLUE parity preserved for algorithm comparison
## 1. Purpose
This spec defines a simplified live-trading architecture for PINK that:
1. Preserves the BLUE algorithm exactly.
2. Makes every engine action observable.
3. Uses the exchange as the authoritative source of live position truth.
4. Reuses the existing data structures needed for BLUE/PINK comparison.
5. Reduces hidden state and duplicate decision paths.
6. Keeps PINK mechanically comparable to BLUE wherever the exchange model allows it.
This document does **not** change the signal math, thresholds, or TP/exit logic.
It only simplifies how those decisions move through the system and how they are recorded.
Where BingX semantics differ from BLUE's historical execution surface, the difference must be isolated behind the execution boundary rather than pushed into the engine.
## 2. Design Goals
The architecture must satisfy all of the following:
- Faithfulness to BLUE's original algorithm.
- Full observability of actions as:
- fired
- requested
- sent
- acknowledged
- executed
- reflected on BingX
- Minimal complexity.
- Maximum reuse of existing tables, maps, and record shapes.
- Clean comparability between BLUE and PINK.
- No second domain-level truth source.
## 3. Non-Goals
This spec does not:
- Change the trading signal formula.
- Change the TP value or exit semantics.
- Add a second live source of truth.
- Replace supervisor with a new process manager.
- Introduce a new order ledger when existing tables can be reused.
## 4. Core Principle
PINK must be exchange-led.
That means:
- BingX position state is authoritative for whether the slot is open.
- BingX open-order state is authoritative for whether an exit is pending.
- Account state is a projection of confirmed exchange events.
- Local engine state is a projection of exchange state plus decision metadata.
- ClickHouse is the durable audit trail.
- Hazelcast is the live control/state bus.
- The TUI is a derived view only.
If local state and BingX state disagree, the system must reconcile toward BingX.
BLUE comparability rule:
- The engine-side lifecycle, state names, and record shapes should remain BLUE-compatible unless BingX makes that impossible.
- Any unavoidable exchange-specific deviation must be isolated in the execution adapter and event normalization path.
- The engine itself should remain oblivious to BingX quirks except for the minimal authority rules needed to stay safe.
## 5. Minimal State Model
The system should keep only these live state categories:
- Decision state
- what the engine decided
- Order state
- what was requested and acknowledged
- Position state
- what BingX currently holds
- Account state
- capital, leverage, open notional
- Terminal trade state
- completed trades only
Everything else should be derived from those categories.
The simplification target is not "remove layers entirely".
It is "make the layers explicit and narrow":
```text
engine intent
-> execution facade
-> exchange adapter
-> exchange
-> event normalization
-> durable ledger
```
The `execution facade` is where BLUE-compatible semantics are preserved.
The `exchange adapter` is where BingX-specific request/response shapes live.
`event normalization` is a thin technical return channel inside the execution boundary:
- dedupe exchange callbacks
- normalize terminal states
- map exchange facts into canonical trade/account events
- update projections and durable rows
It is not a separate policy or trading layer.
This spec uses the following DITA split:
- `Decision`
- pure signal evaluation
- `Intent`
- candidate selection and sizing proposal
- `Trade`
- single-slot lifecycle state machine
- `Account`
- projection of confirmed execution facts
## 6. Existing Data Structures to Reuse
This spec reuses the current structures instead of introducing parallel ones.
### 6.1 ClickHouse tables
- `dolphin_pink.position_state`
- lifecycle source for open and closed trade status
- `dolphin_pink.trade_events`
- terminal ledger for completed trades
- `dolphin_pink.account_events`
- capital and exposure snapshots
- `dolphin_pink.v7_decision_events`
- decision trail
- `dolphin_pink.adaptive_exit_shadow`
- shadow-only exit analysis
### 6.2 Hazelcast maps
- `DOLPHIN_STATE_PINK`
- `DOLPHIN_PNL_PINK`
- `DOLPHIN_FEATURES`
- `DOLPHIN_SAFETY`
- `DOLPHIN_HEARTBEAT`
### 6.3 Exchange-side sources
- `user/positions`
- `trade/openOrders`
- `trade/allOrders`
- `trade/allFillOrders`
## 7. Authoritative Precedence
Live truth must be resolved in this order:
```text
BingX user/positions
↓
BingX trade/openOrders
↓
BingX journal snapshot
↓
ClickHouse account_events / position_state
↓
Hazelcast engine snapshot
↓
Supervisor log fallback
```
Rules:
- The first matching live BingX signal wins.
- Local snapshots may lag and must not override BingX.
- Log parsing is a last resort only.
For BLUE comparability:
- The adapter must emit the same semantic milestones BLUE would expose, even if the physical exchange response is different.
- If BingX cannot express a BLUE milestone exactly, preserve the closest semantic equivalent and annotate the deviation in the event payload.
## 8. High-Level Data Flow
```text
+------------------+
| Binance data |
| / HZ features |
+---------+--------+
|
v
+------------------+
| DolphinActor |
| (BLUE logic) |
+---------+--------+
|
v
+------------------+
| NDAlphaEngine |
| single slot only |
+---------+--------+
|
decision / request
|
v
+------------------+
| Execution facade |
| BLUE-compatible |
+---------+--------+
|
exchange-specific request
|
v
+------------------+
| BingXExecClient |
+---------+--------+
|
v
+------------------+
| BingX VST |
| positions/orders |
+---------+--------+
|
poll / ack / fill / close
|
v
+------------------+
| journal snapshot |
+---------+--------+
|
v
+----------------+----------------+
| ClickHouse + Hazelcast + TUI |
+---------------------------------+
```
## 9. Order Lifecycle
The system should treat every trade as a simple state machine.
```text
EMPTY
|
v
DECISION_CREATED
|
v
ORDER_REQUESTED
|
v
ORDER_SENT
|
v
ORDER_ACKNOWLEDGED
|
v
POSITION_OPENED
|
v
POSITION_UPDATED
|
v
EXIT_REQUESTED
|
v
EXIT_SENT
|
v
EXIT_ACKNOWLEDGED
|
v
POSITION_CLOSED
|
v
TRADE_TERMINAL_WRITTEN
|
v
EMPTY
```
Rules:
- A trade is not "closed" until BingX no longer reports the position.
- A terminal close row is not optional.
- The close row must be written after exchange-event normalization confirms terminality, not before.
## 10. Open / Update / Close Mechanics
### 10.1 Open
1. Engine produces a decision.
2. Actor converts it into an intent.
3. Execution facade normalizes the request into a BLUE-compatible action record.
4. Execution client submits the request to BingX.
5. BingX acknowledges or rejects.
6. BingX position becomes authoritative once open.
7. Event normalization updates `position_state` and account projections.
### 10.2 Update
1. Execution client polls `openOrders`.
2. Execution facade records the requested action.
3. Execution client polls `user/positions`.
4. Execution client refreshes account state.
5. Journal snapshot is persisted.
6. ClickHouse rows are appended.
7. Hazelcast state is refreshed.
8. TUI renders the derived result.
### 10.3 Close
1. Engine or exit manager requests exit.
2. Execution facade normalizes the exit into the same lifecycle that BLUE would represent.
3. Exit order is submitted reduce-only.
4. BingX confirms fill or terminal state.
5. Exchange position disappears.
6. Event normalization emits the terminal close fact.
7. `trade_events` close row is written.
8. `position_state` is updated to closed.
## 11. Reconciliation Model
In this spec, "reconciliation" is not a first-class domain layer.
It is the thin adapter-side return path that converts BingX facts into canonical events and projections.
The simplified model is:
```text
engine intent
-> exchange submission
-> exchange state
-> event normalization
-> durable ledger
```
Not:
```text
engine intent
-> local inferred close
-> maybe exchange close later
```
The second pattern is what creates ghost closes and confusing TUI state.
The return path must remain thin and mostly transparent:
- confirm what BingX actually did
- translate exchange reality into canonical engine state and durable ledger rows
- backfill only the minimum terminal bookkeeping needed to keep the audit trail complete
It must not:
- make trading decisions
- invent or reinterpret strategy state
- act as a second policy layer
- override engine intent except where required to reflect BingX authority
In other words:
```text
policy lives in the engine
translation lives in the execution boundary
truth lives on BingX
```
If the return path starts shaping strategy behavior, the architecture has drifted.
## 12. ClickHouse Accounting Contract
### 12.1 `account_events`
This table must represent the latest authoritative snapshot of:
- capital
- open positions
- open notional
- leverage
- fills metadata
It is not the source of truth for execution. It is the projection of confirmed execution facts and the best table for capital-path replay.
### 12.2 `position_state`
This table must represent per-trade lifecycle state.
Required lifecycle states:
- `OPEN`
- `EXIT_REQUESTED`
- `EXIT_ACKED`
- `CLOSED`
- `RECONCILED`
This table is the canonical lifecycle projection, not a second engine.
### 12.3 `trade_events`
This table must represent terminal closed trades only.
Rules:
- one terminal row per completed trade
- dedupe by `trade_id`
- never infer a close row from a fill snapshot alone
### 12.4 `status_snapshots`
When capital replay is needed, `status_snapshots` remains the preferred capital-path source because it captures:
- capital
- posture
- `trades_executed`
- `rm`
- `vol_ok`
- related snapshot state
`trade_events` alone is not enough for capital replay.
## 13. PINK and BLUE Comparison Rules
PINK must remain structurally comparable to BLUE.
That means:
- same trade identity model
- same key fields for open/close events
- same exit reason vocabulary
- same capital accounting semantics
- same bar and hold semantics
Namespace differences are allowed.
Semantic differences are not.
The DITA split must stay semantically compatible with BLUE:
- decision semantics preserved
- intent selection preserved
- trade lifecycle compatible
- account projection comparable
- return-channel normalization exchange-specific only
## 14. Simplification Rules
To reduce bugs, do the following:
### 14.1 Keep one authoritative open-slot view
Do not maintain competing local definitions of "open trade".
### 14.2 Stop inventing closed trades in the TUI
The TUI may display:
- open positions
- terminal trades
- fills
It must not convert fills into fake closes.
### 14.3 Remove recovery ambiguity
At startup:
- BingX positions are imported
- stale local slots are cleared
- journal state is restored only when it does not contradict BingX
- account projection is rebuilt from confirmed exchange facts, not from intent history
### 14.4 Keep the event trail append-only
If a state needs correction, emit a new event.
Do not rewrite history.
## 15. ASCII Failure Modes
### 15.1 Ghost close
```text
EXIT_REQUESTED
|
v
EXIT_SENT
|
+--> local snapshot says CLOSED
|
+--> BingX still shows position OPEN
|
v
BUG: local UI looks flat, exchange is not flat
```
### 15.2 Missing terminal row
```text
EXIT_ACKNOWLEDGED
|
v
POSITION_CLOSED on BingX
|
v
trade_events row missing
|
v
BUG: replay/debug cannot prove the close
```
### 15.3 Duplicate ledger row
```text
trade_events insert
|
+--> duplicate insert for same trade_id
|
v
BUG: replay capital is overstated unless deduped
```
## 16. Acceptance Criteria
The simplification is acceptable only if all of the following hold:
1. BLUE algorithm behavior is preserved exactly.
2. PINK trades can be compared to BLUE trades using the same structures.
3. Every order action is visible in the trail.
4. Every close can be traced to BingX terminal state.
5. TUI never invents a close.
6. Capital replay can be reconstructed from `status_snapshots` plus deduped trade rows.
7. BingX remains the authoritative open-position source.
## 17. Implementation Boundaries
The following are the expected boundaries for any implementation work:
- Launcher layer
- namespace wiring only
- Actor layer
- engine-slot projection and adapter ingress
- Execution facade layer
- BLUE-compatible action normalization
- order lifecycle event emission
- BingX execution layer
- order submit / poll / reconcile / snapshot
- Journal layer
- durable bridge into ClickHouse
- Observability layer
- derived display only
The return path should be treated as a translation boundary, not a policy boundary.
Its ideal steady state is nearly invisible.
Any new BingX-specific behavior should go in the execution or adapter-ingress path, not in the engine decision logic.
## 18. Recommended Simplified Architecture
```text
[decision]
|
v
[intent]
|
v
[trade FSM]
|
v
[execution adapter]
|
v
[BingX order/position]
|
v
[event normalization]
|
v
[ClickHouse account + trade ledger]
|
v
[TUI / replay]
```
This is the simplest version that still preserves BLUE faithfulness and auditability.
## 19. Open Questions
These are implementation questions, not design blockers:
- Should PINK `trade_events` remain fully separate from BLUE-compatible schema, or only namespace-tagged?
- Should the TUI use `account_events` or `position_state` as the primary open-trade panel source?
- Should `position_state` become the canonical lifecycle table for all live strategies, or only PINK first?
- Should any exchange callback normalization be shared with BLUE, or remain PINK-only until parity is proven?
## 20. Final Decision
The target simplification is:
- one engine
- one exchange authority
- one append-only audit trail
- one derived TUI
- one replay path
Anything that introduces a second truth source should be removed or demoted.
Reconciliation, if the term is retained at all, should mean only the thin adapter-side normalization of BingX facts into canonical events and account projection. It should not exist as a policy layer.