Files
sentiment-engine/MALKHUT/docs/OB_MICROSTRUCTURE_STUDY.md
Codex bb229833d3 malkhut: Flight9 learnings — markout=quality, queue×flow, depth-for-size
Fable's Flight9/BLUE generalizable features incorporated:

1. Slippage model gains trade_flow_intensity parameter:
   - Estimated from book imbalance (proxy for trade arrivals)
   - More flow → better fills (lower slippage)
   - Fable: 'fill = queue position × trade-flow intensity'

2. Markout = quality concept documented:
   - Score fills by post-fill markout, not just fill/no-fill
   - Maker fills are adversely selected

3. Depth-for-size documented:
   - Spread lies; key on depth-within-K-bps vs order notional

4. Measured fees:
   - BingX maker=2.00bp, taker=5.016bp (over 1,455 fills)
   - BingX commission = NEGATIVE (debit)

5. OB study updated with Flight9 learnings
2026-07-17 16:18:43 +02:00

447 lines
21 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Order Book Microstructure Study — MALKHUT
# Compiled from live Binance/BingX API data + academic literature
# (Bouchaud, Cont/Stoikov, Cartea/Jaimungal)
# Last updated: 2026-07-14
## 1. Order Book Depth — Power-Law Decay
The order book depth at distance `d` (in bps from mid) follows a power law:
D(d) = A * d^(1 - alpha)
where:
A = amplitude (USD depth at 1 bps from mid)
alpha = decay exponent (flatter = more depth at distance)
### Per-Asset Parameters
| Asset | A (amplitude USD) | alpha | Fragility | Depth@10bps USD | Depth@100bps USD | Template |
|--------|-------------------|-------|-----------|------------------|-------------------|-----------------|
| BTC | 750,000 | 0.70 | 0.10 | 5,983,000 | 20,000,000 | institutional |
| ETH | 600,000 | 0.75 | 0.12 | 2,095,000 | 7,200,000 | institutional |
| SOL | 400,000 | 0.85 | 0.18 | 954,000 | 4,000,000 | mid_cap_l1 |
| BNB | 500,000 | 0.78 | 0.12 | 2,000,000 | 8,000,000 | institutional |
| DOGE | 22,000 | 1.00 | 0.30 | 432,000 | 2,772,000 | retail_meme |
| ADA | 60,000 | 0.90 | 0.20 | 110,000 | 1,500,000 | mid_cap_l1 |
| AVAX | 55,000 | 0.88 | 0.18 | 107,000 | 1,200,000 | mid_cap_l1 |
| UNI | 30,000 | 0.95 | 0.25 | 53,000 | 800,000 | mid_cap_l1 |
| LINK | 80,000 | 0.87 | 0.17 | 129,000 | 1,800,000 | mid_cap_l1 |
| MATIC | 35,000 | 0.92 | 0.22 | 55,000 | 900,000 | mid_cap_l1 |
| AAVE | 20,000 | 0.95 | 0.25 | 35,000 | 600,000 | mid_cap_l1 |
| DOT | 70,000 | 0.88 | 0.18 | 120,000 | 1,500,000 | mid_cap_l1 |
| ATOM | 25,000 | 0.92 | 0.22 | 45,000 | 700,000 | mid_cap_l1 |
### Interpretation
- **alpha < 0.80** = institutional blue-chip (BTC, ETH, BNB). Depth is spread
relatively evenly. You can walk the book for $1M+ before seeing 10 bps slippage.
- **alpha 0.85-0.92** = mid-cap L1 (SOL, ADA, AVAX, DOT, LINK). Decent depth at
the touch, but thins rapidly. $100K order → 2-5 bps slippage.
- **alpha >= 0.95** = retail/thin (UNI, DOGE, AAVE). Almost all depth at the top
1-2 levels. Any meaningful order walks the book significantly.
- **Fragility factor** = fraction of depth that vanishes during stress events.
BTC loses 10% of depth; DOGE loses 30%. This is the "flash crash amplifier."
### Depth During Stress
During market stress, depth collapses to fragility_factor * normal depth:
D_stress(d) = D_normal(d) * fragility_factor
For BTC: $750K * 0.10 = $75K at 1 bps during stress.
For DOGE: $22K * 0.30 = $6.6K at 1 bps during stress.
Market makers pull within 1ms of flash crash onset. Recovery takes 30-300 seconds.
## 2. Spread Profiles
| Asset | Normal (bps) | Stress Multiplier | Interpretation |
|--------|-------------|-------------------|-----------------------------------|
| BTC | 0.01 | 50x | 0.01 bps normal, 0.5 bps stress |
| ETH | 0.02 | 50x | 0.02 bps normal, 1.0 bps stress |
| BNB | 0.50 | 15x | 0.50 bps normal, 7.5 bps stress |
| SOL | 1.26 | 8x | 1.26 bps normal, 10.1 bps stress |
| DOGE | 1.35 | 15x | 1.35 bps normal, 20.3 bps stress |
| ADA | 5.95 | 12x | 5.95 bps normal, 71.4 bps stress |
| AVAX | 1.48 | 10x | 1.48 bps normal, 14.8 bps stress |
| UNI | 2.76 | 12x | 2.76 bps normal, 33.1 bps stress |
| LINK | 1.25 | 8x | 1.25 bps normal, 10.0 bps stress |
| MATIC | 1.50 | 10x | 1.50 bps normal, 15.0 bps stress |
| AAVE | 2.50 | 12x | 2.50 bps normal, 30.0 bps stress |
| DOT | 1.00 | 8x | 1.00 bps normal, 8.0 bps stress |
| ATOM | 2.00 | 10x | 2.00 bps normal, 20.0 bps stress |
### Key Insight for Strategy Design
BTC/ETH spreads are 10-500x tighter than alts. This means:
- BTC: spread cost is negligible; profitability depends on fill quality + adverse selection
- ADA/ATOM: spread cost is 10-60 bps round-trip; must capture >= spread to be profitable
- Stress spreads can be 50x normal for BTC — but that's still only 0.5 bps
## 3. Order Flow Characteristics
| Asset | Orders/sec (normal) | Orders/sec (stress) | Cancel/Fill | Median $ | P99 $ | Avg Trade $ |
|--------|---------------------|---------------------|-------------|-----------|-----------|-------------|
| BTC | 300 | 5,000 | 20.0 | 643 | 200,000 | 5,000 |
| ETH | 250 | 4,000 | 18.0 | 500 | 150,000 | 4,000 |
| SOL | 100 | 1,500 | 10.0 | 800 | 150,000 | 2,000 |
| DOGE | 80 | 800 | 8.0 | 96 | 52,000 | 200 |
| ADA | 60 | 600 | 7.0 | 200 | 40,000 | 500 |
| AVAX | 70 | 700 | 8.0 | 300 | 60,000 | 800 |
| UNI | 50 | 500 | 6.0 | 150 | 30,000 | 400 |
| LINK | 90 | 1,200 | 9.0 | 250 | 80,000 | 1,000 |
| MATIC | 55 | 550 | 7.0 | 180 | 35,000 | 400 |
| AAVE | 40 | 400 | 6.0 | 500 | 50,000 | 1,500 |
| DOT | 65 | 650 | 7.5 | 350 | 45,000 | 700 |
| ATOM | 45 | 450 | 6.5 | 200 | 35,000 | 500 |
### Cancel/Fill Ratio Interpretation
- **BTC 20x**: For every fill, 20 orders are cancelled. This is pure HFT MM churn.
The MM quotes aggressively, pulls when toxicity rises, re-quotes wider.
- **DOGE 8x**: Less MM activity, more genuine intent. Retail orders are more sticky.
- **Alts 5-15x**: Range between MM-dominated (higher) and retail-dominated (lower).
### Order Size Distribution
All assets follow a power-law tail: log-normal body + Pareto tail.
- BTC P50 = $643 (median order), P99 = $200K. Tail exponent ~2.5.
- DOGE P50 = $96, P99 = $52K. Tail exponent ~2.0 (fatter tail = more whale orders).
- The P99 order is 300-600x the P50. These are institutional block trades.
### Order Arrival Process
Orders arrive as a self-exciting Hawkes process (not Poisson):
- Clustering: a fill begets more fills within 10-100ms
- BTC: 300 orders/sec normal, 5000 during events (17x burst)
- Burst magnitude correlates with volatility regime
## 4. Market Maker Behavior
| Asset | Max Inventory | Skew Tol. | Pull Speed | Margin |
|--------|--------------|-----------|------------|--------|
| BTC | $10M | 15 bps | 3 ms | 0.5 bps|
| ETH | $8M | 15 bps | 3 ms | 0.5 bps|
| SOL | $5M | 20 bps | 10 ms | 0.8 bps|
| BNB | $5M | 20 bps | 5 ms | 0.6 bps|
| DOGE | $500K | 40 bps | 25 ms | 2.0 bps|
| ADA | $1M | 30 bps | 20 ms | 1.5 bps|
| AVAX | $1.5M | 25 bps | 15 ms | 1.0 bps|
| UNI | $300K | 50 bps | 30 ms | 2.5 bps|
| LINK | $2M | 25 bps | 12 ms | 1.0 bps|
| MATIC | $400K | 35 bps | 22 ms | 2.0 bps|
| AAVE | $200K | 45 bps | 28 ms | 2.0 bps|
| DOT | $1.5M | 25 bps | 15 ms | 1.0 bps|
| ATOM | $600K | 35 bps | 20 ms | 1.5 bps|
### Key Dynamics
- **Pull speed** = how fast MM withdraws quotes after detecting toxicity.
BTC MMs are 10x faster than DOGE MMs (3ms vs 25ms). This means:
- BTC: you must be fast or you're picking off stale quotes
- DOGE: stale quotes persist longer → latency arbitrage more viable
- **Margin** = minimum edge the MM requires to quote.
BTC 0.5 bps = MM breaks even on a 0.5 bps spread after fees.
DOGE 2.5 bps = MM needs 2.5 bps edge, because adverse selection is higher.
- **Max inventory** = position limit before MM widens quotes or stops quoting.
BTC $10M vs DOGE $500K. Ratio is 20x, matching the depth ratio.
## 5. Volatility Regimes
| Asset | Ann. Vol (normal) | Ann. Vol (crisis) | GARCH alpha | GARCH beta | Half-life |
|--------|-------------------|-------------------|-------------|------------|-----------|
| BTC | 35% | 100% | 0.10 | 0.88 | 48 hrs |
| ETH | 66.5% | 130% | 0.12 | 0.86 | 40 hrs |
| SOL | 71.9% | 150% | 0.13 | 0.84 | 32 hrs |
| DOGE | 77.9% | 200% | 0.15 | 0.82 | 24 hrs |
| BNB | 50% | 120% | 0.11 | 0.87 | 42 hrs |
### GARCH Interpretation
- **alpha + beta = persistence.** BTC: 0.10 + 0.88 = 0.98. Very persistent.
After a shock, volatility takes ~48 hours (half-life) to decay to 50%.
- **Crisis vol is 2-3x normal vol.** BTC goes from 35% → 100% annualized.
- **Smaller caps have faster decay.** DOGE half-life 24h vs BTC 48h.
DOGE returns to calm faster but also spikes faster.
## 6. Intraday Patterns
| Asset | Peak Hour (UTC) | Trough Hour (UTC) | Peak/Trough Ratio |
|--------|-----------------|-------------------|-------------------|
| BTC | 15:00 | 19:00 | 7.4x |
| ETH | 15:00 | 19:00 | 7.0x |
| DOGE | 15:00 | 10:00 | 4.5x |
### Session Analysis
- **US session (13:00-21:00 UTC)**: Highest volume, tightest spreads, deepest books.
The 15:00 UTC peak = US market open overlap with EU close.
- **Asia session (00:00-08:00 UTC)**: Lowest volume, widest spreads.
- **Weekend**: Volume drops 30-50%, vol drops to 0.65-0.70x, spreads widen 10-30%.
## 7. Cross-Asset Correlations
| Pair | Normal | Crash | Interpretation |
|-----------------|--------|--------|-------------------------------|
| BTC-ETH | 0.70-0.92 | 0.93-0.98 | Near-perfect in crashes |
| BTC-SOL | 0.60-0.80 | 0.85-0.95 | High in crashes |
| BTC-DOGE | 0.45-0.65 | 0.80-0.90 | Moderate normal, high crash |
| BTC-LINK | 0.55-0.75 | 0.85-0.92 | Similar to SOL |
### "Correlations go to 1 in crashes"
This is the single most important portfolio-level fact. During normal times,
diversification works. During crashes, EVERYTHING correlates with BTC.
A "diversified" alt portfolio provides zero downside protection.
## 8. BingX-Specific Behavior vs Binance
| Metric | BingX / Binance Ratio | Interpretation |
|-----------------------|-----------------------|------------------------------------|
| Perp spread | 1.7-12.6x wider | BingX has less MM competition |
| BTC depth | 20x thinner | Much thinner books |
| Taker fee | 1.25x higher | 0.05% vs 0.04% |
| API latency | 2-3x higher | 100ms vs 40-50ms |
| Funding rate corr. | R² ~ 0.90, 0-8h lag | Use Binance as leading indicator |
| Spot spread | 18-389x wider | NEVER use BingX spot |
### Practical Implications
- **BingX is 10-20x harder to trade profitably** than Binance for the same strategy.
Thinner books + wider spreads + higher fees + higher latency.
- **Cross-exchange arbitrage** between BingX and Binance is real but latency-limited.
The 0-8h funding rate lag creates opportunities.
- **BingX perp is viable** for market making (wider spread = more edge) but
requires wider quotes and lower aggression.
## 9. Book Fragility & Cascade Dynamics
### Flash Crash Anatomy
1. **Trigger**: Large market sell hits thin book (0.05x depth at that moment)
2. **Cascade**: Price drops through stop levels → forced liquidations → more selling
3. **MM withdrawal**: All MMs pull quotes within 1-5ms
4. **Depth vacuum**: Book goes from $750K to $75K (BTC) or $22K to $6.6K (DOGE)
5. **Recovery**: 30-300 seconds for MMs to re-quote, 10-60 minutes for depth to normalize
### Per-Asset Cascade Characteristics
| Asset | Trigger Price Drop | Liquidation Speed | Recovery |
|--------|-------------------|-------------------|------------|
| BTC | 6.5% | slow | fast |
| ETH | 4.0% | medium | medium |
| SOL | 5.0% | medium | medium |
| DOGE | 4.0% | fast | slow |
| UNI | 3.5% | fast | slow |
| AAVE | 4.0% | fast | slow |
### OI/MCap Ratio (Liquidation Pressure)
- BTC: 0.5% of market cap in open interest → low cascade risk
- DOGE: 1.4% → moderate cascade risk
- ETH: 1.9% → higher cascade risk
- The ratio directly predicts how much forced selling occurs per 1% price drop
### Liquidation Trigger Threshold
The price drop needed to trigger cascade liquidations:
- BTC: 6.5% (hard to trigger → "too big to cascade")
- DOGE: 4.0% (easier to trigger → more volatile cascades)
- UNI: 3.5% (very easy to trigger → most fragile)
### Recovery Asymmetry
Crashes are FAST (milliseconds for MM withdrawal, seconds for liquidations)
but recovery is SLOW (minutes to hours for depth normalization).
This asymmetry is exploitable: buy the dip 30-60 seconds after the crash,
when depth is still thin but selling pressure is exhausted.
## 10. Retail vs Institutional Composition
| Asset | Retail Ratio | Institutional Gap | Implication |
|--------|-------------|-------------------|--------------------------------|
| BTC | 0.35 | 0.04 | Most institutional, best flows |
| ETH | 0.40 | 0.06 | Near-institutional |
| BNB | 0.50 | 0.20 | Balanced |
| SOL | 0.72 | 0.75 | Retail-dominated |
| DOGE | 0.80 | 0.31 | Heavily retail |
| UNI | 0.60 | 0.25 | Mixed |
| AAVE | 0.55 | 0.20 | Mixed |
### Trading Implications
- **Institutional assets (BTC, ETH)**: Tighter spreads, deeper books, more
efficient pricing. Edge comes from execution quality, not information.
- **Retail assets (DOGE, SOL)**: Wider spreads, more predictable order flow,
more stale-quote opportunities. Edge comes from toxicity detection + latency.
- **The institutional gap** = difference in quote persistence between institutional
and retail orders. Higher gap = more predictable behavior = more exploitable.
## 11. Funding Rates (BingX Perps)
| Asset | Mean (8h) | Std (8h) | Positive % | Basis Typical |
|--------|-----------|----------|------------|---------------|
| BTC | 0.59 bps | 0.22 bps | 100% | 4.0 bps |
| ETH | 0.50 bps | 0.25 bps | 95% | 3.5 bps |
| SOL | 0.30 bps | 0.35 bps | 65% | 2.5 bps |
| DOGE | 0.39 bps | 0.34 bps | 80% | 2.0 bps |
| BNB | 0.40 bps | 0.25 bps | 90% | 3.0 bps |
| ADA | 0.20 bps | 0.40 bps | 55% | 1.5 bps |
| AVAX | 0.15 bps | 0.35 bps | 50% | 2.0 bps |
| UNI | 0.10 bps | 0.30 bps | 45% | 1.5 bps |
| LINK | 0.25 bps | 0.30 bps | 60% | 2.0 bps |
| MATIC | 0.12 bps | 0.32 bps | 48% | 1.8 bps |
| AAVE | 0.08 bps | 0.28 bps | 40% | 1.2 bps |
| DOT | 0.18 bps | 0.30 bps | 55% | 2.0 bps |
| ATOM | 0.10 bps | 0.28 bps | 42% | 1.5 bps |
### Key Insight
BTC funding is ALWAYS positive (100% of time) at ~0.59 bps/8h. This means:
- Longs ALWAYS pay shorts on BTC perps
- Being short BTC perp earns a steady 0.59 bps every 8 hours
- This is "free money" for short-biased strategies (which MALKHUT is)
The funding rate is the single most predictable return stream in crypto perps.
## 12. Expected Slippage Model
For a market order of size $X:
slippage_bps = sum_{d=1}^{D} (A * d^(-alpha)) for cumulative depth >= X
Example for BTC ($750K amplitude, alpha=0.70):
- $10K order: ~0.05 bps (negligible)
- $100K order: ~0.5 bps (one tick)
- $1M order: ~3.5 bps (walks the book meaningfully)
- $10M order: ~15 bps (aggressive, will move the market)
Example for DOGE ($22K amplitude, alpha=1.00):
- $1K order: ~0.5 bps
- $10K order: ~5 bps
- $100K order: ~50 bps (very aggressive, huge impact)
### Implication
BTC allows $100K orders with <1 bps slippage. DOGE requires $1K orders for
the same. Position sizing must account for the book's capacity, not just
the strategy's signal.
## 13. Fill Quality Optimization (MALKHUT Core)
MALKHUT is an execution improvement engine. Fill quality IS the primary aim.
### Fill Quality Metrics
For each CWM transition, MALKHUT computes:
- **slippage_bps**: aggressive fills — how far from mid?
- **price_improvement_bps**: passive fills — how much better than touch?
- **levels_consumed**: queue depth of fill
- **post_fill_adverse_bps**: price movement after fill (negative = adverse)
- **fill_value_score**: composite = quality - adverse * 0.5
### How This Connects to the OB
The OB microstructure directly determines fill quality:
| OB Characteristic | Impact on Fill Quality |
|-------------------|----------------------|
| **Depth at touch** | More depth = more fill opportunities for passive orders |
| **Spread** | Tighter spread = smaller price improvement possible |
| **Depth decay (alpha)** | Steeper decay = fills walk the book faster = higher slippage |
| **Cancel/fill ratio** | Higher ratio = more queue churn = harder to get fills |
| **MM pull speed** | Faster pull = stale quotes less likely = harder to snipe |
| **Book fragility** | During stress, depth drops 70-95% = fills at worse prices |
### Per-Asset Fill Quality Expectations
| Asset | Expected Fill Quality | Why |
|-------|----------------------|-----|
| BTC | Excellent | Deep book, tight spread, fast MM re-quote |
| ETH | Good | Similar to BTC, slightly thinner |
| SOL | Moderate | Mid-depth, moderate spread |
| DOGE | Poor | Thin book, wide spread, slow MM |
| ADA | Poor | Thin book, wide spread |
### Optimization Strategy
1. **Venue selection**: choose venues with better fill quality (Binance > BingX)
2. **Order type selection**: use POST_ONLY on tight-spread assets, MARKET on thin-spread
3. **Offset optimization**: CMA-ES learns optimal offset per (asset, regime, venue)
4. **Size optimization**: CMA-ES learns optimal size per (asset, regime, venue)
5. **Timing optimization**: CMA-ES learns when to quote vs when to wait
The PerformanceMatrix tracks `avg_fill_value_score` per (regime, strategy, venue),
enabling the system to learn: "In this regime, on this venue, this strategy
achieves the best fill quality."
## 14. Flight9/BLUE Fill Learnings (Fable, 2026-07-16)
Real FLIGHT9/BLUE fill backfill — generalizable features, not hardcoded thresholds.
### Markout = Quality
Fill quality is measured by **post-fill markout** (price move N ticks after fill),
not just fill/no-fill. A maker fill at a good quoted price can still be a bad fill
if the market moves adversely after execution. The system scores fills by markout.
**Generalizable:** score fills by post-fill markout. Model fill-CONDITIONAL-on-adverse-flow.
### Fill = Queue × Flow Intensity
Fill probability is driven by **trade-arrival intensity** (the tape), not the static book.
A resting maker fills only when trades print through its level for enough volume to clear
the queue ahead. The HftBacktestCWM queue model is the right substrate — it needs real
trade-flow intensity as input.
**On testnet:** maker fill-rate ~0% (no flow). This is an artifact, not a signal.
### Depth-for-Size, Not Spread
Spread alone LIES: an asset with 1.5bp spread behind ~$977 of depth is unfillable at any
size. Generalizable: key fill viability on **depth-within-K-bps** relative to order
notional, not spread.
### Measured Fees (Venue-Parameterized)
| Venue | Maker | Taker | Sign |
|-------|-------|-------|------|
| BingX | 2.00 bp | 5.016 bp | NEGATIVE = DEBIT |
Generalize: parameterize fee + sign per venue from measurement, never assume.
### Realized Friction (F9 VST Fills)
- Taker: ~20-27 bp adverse on thin/mid books
- Maker: saves ~3-4 bp WHEN it fills
- Maker fill-rate: ~0% on VST (no flow — testnet artifact)
### Counterfactual Maker Fill (Real Binance Book)
- ~50% fill on liquid books (at-touch upper bound)
- Queue + venue-thinness reduce it
### Validation Methodology
When validating against a crude fill sim, optimize on **RELATIVE lift** between two policies
through the SAME sim — fidelity bias cancels in the difference. Trust the ordering of
policies even when absolute fill rates are approximate.