sentiment-engine/prod/docs/BUGCLASS_INDETERMINATE_OUTCOME_20260713.md

# Bug class: **indeterminate-outcome collapse**
### "not done" is NOT the same as "tried and failed" — and neither is "tried, outcome unknown"

**Status:** doctrine. Codebase-wide sweep assigned to codex.
**Discovered by:** HJ, from the 2026-07-13 orphan incident.

---

## The formal statement

Any operation carrying an **external side effect** (venue order, cancel, transfer,
ClickHouse insert, journal write, shm publish) has **three** terminal outcomes, not two:

| Outcome | Effect exists? | Is rollback sound? |
|---|---|---|
| **NOT_ATTEMPTED** — we never issued it | provably no | **YES** |
| **ATTEMPTED_REFUSED** — the peer *proved* it refused (venue error code, validation reject) | provably no | **YES** |
| **ATTEMPTED_INDETERMINATE** — issued, outcome unknown | **maybe** | **NO — NEVER** |

The bug is **collapsing three states into two**: treating INDETERMINATE as FAILED, and
then applying a rollback that is only sound for the first two.

**The invariant that gets violated:** the *effect* (the venue order) and the *record*
(the kernel FSM) are **not atomic**. Any failure between them leaves them divergent. This
is the classic dual-write / write-then-record problem, and it has exactly one correct
answer: **when you cannot prove the effect did not happen, you must not assert that it
didn't. You must reconcile against the authoritative source** (the venue).

**Unknown is not flat. Unknown is not failed. Unknown is UNKNOWN.**

---

## How it presented on 2026-07-13

```python
# bingx_venue.submit_async()
receipt = await backend.submit_intent(legacy)   # <-- POINT OF NO RETURN: order is LIVE
events  = self._events_from_submit(...)         #     bookkeeping
self._publish_telemetry(..., order_id=...)      # <-- TypeError (signature skew)
return events                                   #     never reached
```
```python
# rust_backend, the caller
try:
    emitted_events = await submit_async(intent)
except Exception:                                # <-- ONE channel for BOTH meanings
    -> synthetic REJECTED -> FSM rollback to IDLE
```

`submit_async` signals *"the venue never took the order"* by **raising**, and signals
*"a bug fired somewhere inside me"* by **raising**. Same channel. The caller cannot tell
them apart, so it takes the interpretation written for the timeout case and asserts a
fact it cannot know: *no order exists*.

The venue kept 6 SHORT positions. The kernel believed it was flat. No SL, TP, MAX_HOLD or
ADVSL can protect a position the kernel does not know it holds — which is why **execution
truth (tier B) outranks capital preservation (tier C)** in `UV_EXEC_PRIORITY_LADDER`.

Telemetry never had a "veto". It **hijacked the failure channel**. It was the bullet;
`except Exception -> assume not-done` is the gun. *Any* post-ack line — a log call, a dict
build, an events-builder edge case — would have produced the identical 6 orphans.

---

## THE SECOND INSTANCE — older, still live, and worse

The "safe" pre-ack branch is **itself an instance of the same bug**:

```python
except Exception as _submit_exc:
    # venue.submit() failed (e.g. BingX timeout) ... feed a synthetic REJECTED
```

**A timeout is the definition of INDETERMINATE.** The request may have reached BingX and
filled. Rolling back to IDLE on a timeout asserts "no order exists" on exactly the
evidence that cannot support it. This has been in the code far longer than tonight's skew
and produces orphans on every venue timeout.

Only a **proven** refusal may roll back:
- connection refused / DNS failure *before the request was sent* → NOT_ATTEMPTED ✔
- venue returned an explicit rejection code (validation, insufficient margin) → REFUSED ✔
- **timeout, connection reset, 5xx, unparseable response → INDETERMINATE ✘ must reconcile**

---

## The fix pattern (minimally invasive — no new seams, no re-architecture)

1. **Mark the point of no return.** Everything after the side-effect call is fenced; it
   may not reach the caller through the bare-exception channel.
2. **Type the channel.** `VenuePostAckError` (added, `venue.py`) carries the receipt and
   means *the effect exists*. Pre-ack failures keep raising normally.
3. **Callers branch on it.** `rust_backend` catches `VenuePostAckError` first: **no
   synthetic REJECT, no rollback**. Leave the slot in its working state and let execution
   truth settle it — the E-feed delivers `FULL_FILL` from the account stream independently.
4. **Non-critical code cannot use the failure channel at all.** Telemetry is off the exec
   path entirely (bounded-memory, lossless spill lane).
5. **Indeterminate ⇒ reconcile.** Never rollback. Query the venue; it is the authority.

No architectural change. No new seam. One typed exception, one fence, one branch.

---

## The codebase-wide sweep (codex)

**Search pattern:** any `try:` whose body performs an external side effect and whose
`except` assumes the effect did not happen (rollback / mark-failed / synthesize-reject /
blind-retry / "assume flat" / return default).

Known/suspect sites:
- `bingx_venue.submit / submit_async` ✔ fixed
- `bingx_venue.cancel / cancel_async` — same shape, unaudited
- `rust_backend` pre-ack timeout branch — **BUG, live** (see above)
- `EKBridge.mirror_future`, `e_feed._publish_converted` — failure of the E→K mirror
- `ch_writer` / journal lane inserts — "insert failed" vs "insert maybe landed" (duplicates)
- `ExecutionRouter` maker/taker requote + amend paths
- promotion bridge (`uv.promotion`) receipt handling
- any `except Exception: pass` / `return None` / `return False` after an I/O call

**Every site must answer one question:** *can this except-branch PROVE the effect did not
happen?* If no → it must not assert absence. Reconcile or escalate to UNKNOWN.

---

## Tests demanded (nuclear grade)

1. **Fault-injection matrix.** For each side-effecting call: inject an exception at
   **every statement after the effect** (parametrize over injection index) and assert the
   kernel **never** concludes the effect is absent. This is the test that would have
   caught tonight's bug at *any* of the post-ack lines, not just the telemetry one.
2. **Timeout is indeterminate.** Venue times out *after* the exchange filled → assert NO
   rollback, assert reconcile is triggered, assert the position is adopted, not orphaned.
3. **Property/fuzz.** Random interleavings of {ack, no-ack, timeout, post-ack raise,
   partial fill, duplicate ack}; invariant: `kernel_believes_flat ⇒ venue_is_flat`. This
   invariant is the whole ballgame; it must never be violated for any interleaving.
4. **Signature-skew guard.** `inspect.signature().bind()` over every call site of every
   telemetry/journal helper — a pure binding test that survives re-vendoring.
5. **Losslessness.** Telemetry/journal lanes: wedge the sink, flood N records, assert
   `dropped == 0` and `on_disk + in_memory == N`, and assert memory stays bounded.
6. **Mutation litmus.** Remove each fence → the corresponding test must go RED. If
   removing the guard keeps the suite green, the test is theatre.