Files
sentiment-engine/MALKHUT/docs/OB_MICROSTRUCTURE_STUDY.md

457 lines
21 KiB
Markdown
Raw Normal View History

# Order Book Microstructure Study — MALKHUT
# Compiled from live Binance/BingX API data + academic literature
# (Bouchaud, Cont/Stoikov, Cartea/Jaimungal)
# Last updated: 2026-07-14
## 1. Order Book Depth — Power-Law Decay
The order book depth at distance `d` (in bps from mid) follows a power law:
D(d) = A * d^(1 - alpha)
where:
A = amplitude (USD depth at 1 bps from mid)
alpha = decay exponent (flatter = more depth at distance)
### Per-Asset Parameters
| Asset | A (amplitude USD) | alpha | Fragility | Depth@10bps USD | Depth@100bps USD | Template |
|--------|-------------------|-------|-----------|------------------|-------------------|-----------------|
| BTC | 750,000 | 0.70 | 0.10 | 5,983,000 | 20,000,000 | institutional |
| ETH | 600,000 | 0.75 | 0.12 | 2,095,000 | 7,200,000 | institutional |
| SOL | 400,000 | 0.85 | 0.18 | 954,000 | 4,000,000 | mid_cap_l1 |
| BNB | 500,000 | 0.78 | 0.12 | 2,000,000 | 8,000,000 | institutional |
| DOGE | 22,000 | 1.00 | 0.30 | 432,000 | 2,772,000 | retail_meme |
| ADA | 60,000 | 0.90 | 0.20 | 110,000 | 1,500,000 | mid_cap_l1 |
| AVAX | 55,000 | 0.88 | 0.18 | 107,000 | 1,200,000 | mid_cap_l1 |
| UNI | 30,000 | 0.95 | 0.25 | 53,000 | 800,000 | mid_cap_l1 |
| LINK | 80,000 | 0.87 | 0.17 | 129,000 | 1,800,000 | mid_cap_l1 |
| MATIC | 35,000 | 0.92 | 0.22 | 55,000 | 900,000 | mid_cap_l1 |
| AAVE | 20,000 | 0.95 | 0.25 | 35,000 | 600,000 | mid_cap_l1 |
| DOT | 70,000 | 0.88 | 0.18 | 120,000 | 1,500,000 | mid_cap_l1 |
| ATOM | 25,000 | 0.92 | 0.22 | 45,000 | 700,000 | mid_cap_l1 |
### Interpretation
- **alpha < 0.80** = institutional blue-chip (BTC, ETH, BNB). Depth is spread
relatively evenly. You can walk the book for $1M+ before seeing 10 bps slippage.
- **alpha 0.85-0.92** = mid-cap L1 (SOL, ADA, AVAX, DOT, LINK). Decent depth at
the touch, but thins rapidly. $100K order → 2-5 bps slippage.
- **alpha >= 0.95** = retail/thin (UNI, DOGE, AAVE). Almost all depth at the top
1-2 levels. Any meaningful order walks the book significantly.
- **Fragility factor** = fraction of depth that vanishes during stress events.
BTC loses 10% of depth; DOGE loses 30%. This is the "flash crash amplifier."
### Depth During Stress
During market stress, depth collapses to fragility_factor * normal depth:
D_stress(d) = D_normal(d) * fragility_factor
For BTC: $750K * 0.10 = $75K at 1 bps during stress.
For DOGE: $22K * 0.30 = $6.6K at 1 bps during stress.
Market makers pull within 1ms of flash crash onset. Recovery takes 30-300 seconds.
## 2. Spread Profiles
| Asset | Normal (bps) | Stress Multiplier | Interpretation |
|--------|-------------|-------------------|-----------------------------------|
| BTC | 0.01 | 50x | 0.01 bps normal, 0.5 bps stress |
| ETH | 0.02 | 50x | 0.02 bps normal, 1.0 bps stress |
| BNB | 0.50 | 15x | 0.50 bps normal, 7.5 bps stress |
| SOL | 1.26 | 8x | 1.26 bps normal, 10.1 bps stress |
| DOGE | 1.35 | 15x | 1.35 bps normal, 20.3 bps stress |
| ADA | 5.95 | 12x | 5.95 bps normal, 71.4 bps stress |
| AVAX | 1.48 | 10x | 1.48 bps normal, 14.8 bps stress |
| UNI | 2.76 | 12x | 2.76 bps normal, 33.1 bps stress |
| LINK | 1.25 | 8x | 1.25 bps normal, 10.0 bps stress |
| MATIC | 1.50 | 10x | 1.50 bps normal, 15.0 bps stress |
| AAVE | 2.50 | 12x | 2.50 bps normal, 30.0 bps stress |
| DOT | 1.00 | 8x | 1.00 bps normal, 8.0 bps stress |
| ATOM | 2.00 | 10x | 2.00 bps normal, 20.0 bps stress |
### Key Insight for Strategy Design
BTC/ETH spreads are 10-500x tighter than alts. This means:
- BTC: spread cost is negligible; profitability depends on fill quality + adverse selection
- ADA/ATOM: spread cost is 10-60 bps round-trip; must capture >= spread to be profitable
- Stress spreads can be 50x normal for BTC — but that's still only 0.5 bps
## 3. Order Flow Characteristics
| Asset | Orders/sec (normal) | Orders/sec (stress) | Cancel/Fill | Median $ | P99 $ | Avg Trade $ |
|--------|---------------------|---------------------|-------------|-----------|-----------|-------------|
| BTC | 300 | 5,000 | 20.0 | 643 | 200,000 | 5,000 |
| ETH | 250 | 4,000 | 18.0 | 500 | 150,000 | 4,000 |
| SOL | 100 | 1,500 | 10.0 | 800 | 150,000 | 2,000 |
| DOGE | 80 | 800 | 8.0 | 96 | 52,000 | 200 |
| ADA | 60 | 600 | 7.0 | 200 | 40,000 | 500 |
| AVAX | 70 | 700 | 8.0 | 300 | 60,000 | 800 |
| UNI | 50 | 500 | 6.0 | 150 | 30,000 | 400 |
| LINK | 90 | 1,200 | 9.0 | 250 | 80,000 | 1,000 |
| MATIC | 55 | 550 | 7.0 | 180 | 35,000 | 400 |
| AAVE | 40 | 400 | 6.0 | 500 | 50,000 | 1,500 |
| DOT | 65 | 650 | 7.5 | 350 | 45,000 | 700 |
| ATOM | 45 | 450 | 6.5 | 200 | 35,000 | 500 |
### Cancel/Fill Ratio Interpretation
- **BTC 20x**: For every fill, 20 orders are cancelled. This is pure HFT MM churn.
The MM quotes aggressively, pulls when toxicity rises, re-quotes wider.
- **DOGE 8x**: Less MM activity, more genuine intent. Retail orders are more sticky.
- **Alts 5-15x**: Range between MM-dominated (higher) and retail-dominated (lower).
### Order Size Distribution
All assets follow a power-law tail: log-normal body + Pareto tail.
- BTC P50 = $643 (median order), P99 = $200K. Tail exponent ~2.5.
- DOGE P50 = $96, P99 = $52K. Tail exponent ~2.0 (fatter tail = more whale orders).
- The P99 order is 300-600x the P50. These are institutional block trades.
### Order Arrival Process
Orders arrive as a self-exciting Hawkes process (not Poisson):
- Clustering: a fill begets more fills within 10-100ms
- BTC: 300 orders/sec normal, 5000 during events (17x burst)
- Burst magnitude correlates with volatility regime
## 4. Market Maker Behavior
| Asset | Max Inventory | Skew Tol. | Pull Speed | Margin |
|--------|--------------|-----------|------------|--------|
| BTC | $10M | 15 bps | 3 ms | 0.5 bps|
| ETH | $8M | 15 bps | 3 ms | 0.5 bps|
| SOL | $5M | 20 bps | 10 ms | 0.8 bps|
| BNB | $5M | 20 bps | 5 ms | 0.6 bps|
| DOGE | $500K | 40 bps | 25 ms | 2.0 bps|
| ADA | $1M | 30 bps | 20 ms | 1.5 bps|
| AVAX | $1.5M | 25 bps | 15 ms | 1.0 bps|
| UNI | $300K | 50 bps | 30 ms | 2.5 bps|
| LINK | $2M | 25 bps | 12 ms | 1.0 bps|
| MATIC | $400K | 35 bps | 22 ms | 2.0 bps|
| AAVE | $200K | 45 bps | 28 ms | 2.0 bps|
| DOT | $1.5M | 25 bps | 15 ms | 1.0 bps|
| ATOM | $600K | 35 bps | 20 ms | 1.5 bps|
### Key Dynamics
- **Pull speed** = how fast MM withdraws quotes after detecting toxicity.
BTC MMs are 10x faster than DOGE MMs (3ms vs 25ms). This means:
- BTC: you must be fast or you're picking off stale quotes
- DOGE: stale quotes persist longer → latency arbitrage more viable
- **Margin** = minimum edge the MM requires to quote.
BTC 0.5 bps = MM breaks even on a 0.5 bps spread after fees.
DOGE 2.5 bps = MM needs 2.5 bps edge, because adverse selection is higher.
- **Max inventory** = position limit before MM widens quotes or stops quoting.
BTC $10M vs DOGE $500K. Ratio is 20x, matching the depth ratio.
## 5. Volatility Regimes
| Asset | Ann. Vol (normal) | Ann. Vol (crisis) | GARCH alpha | GARCH beta | Half-life |
|--------|-------------------|-------------------|-------------|------------|-----------|
| BTC | 35% | 100% | 0.10 | 0.88 | 48 hrs |
| ETH | 66.5% | 130% | 0.12 | 0.86 | 40 hrs |
| SOL | 71.9% | 150% | 0.13 | 0.84 | 32 hrs |
| DOGE | 77.9% | 200% | 0.15 | 0.82 | 24 hrs |
| BNB | 50% | 120% | 0.11 | 0.87 | 42 hrs |
### GARCH Interpretation
- **alpha + beta = persistence.** BTC: 0.10 + 0.88 = 0.98. Very persistent.
After a shock, volatility takes ~48 hours (half-life) to decay to 50%.
- **Crisis vol is 2-3x normal vol.** BTC goes from 35% → 100% annualized.
- **Smaller caps have faster decay.** DOGE half-life 24h vs BTC 48h.
DOGE returns to calm faster but also spikes faster.
## 6. Intraday Patterns
| Asset | Peak Hour (UTC) | Trough Hour (UTC) | Peak/Trough Ratio |
|--------|-----------------|-------------------|-------------------|
| BTC | 15:00 | 19:00 | 7.4x |
| ETH | 15:00 | 19:00 | 7.0x |
| DOGE | 15:00 | 10:00 | 4.5x |
### Session Analysis
- **US session (13:00-21:00 UTC)**: Highest volume, tightest spreads, deepest books.
The 15:00 UTC peak = US market open overlap with EU close.
- **Asia session (00:00-08:00 UTC)**: Lowest volume, widest spreads.
- **Weekend**: Volume drops 30-50%, vol drops to 0.65-0.70x, spreads widen 10-30%.
## 7. Cross-Asset Correlations
| Pair | Normal | Crash | Interpretation |
|-----------------|--------|--------|-------------------------------|
| BTC-ETH | 0.70-0.92 | 0.93-0.98 | Near-perfect in crashes |
| BTC-SOL | 0.60-0.80 | 0.85-0.95 | High in crashes |
| BTC-DOGE | 0.45-0.65 | 0.80-0.90 | Moderate normal, high crash |
| BTC-LINK | 0.55-0.75 | 0.85-0.92 | Similar to SOL |
### "Correlations go to 1 in crashes"
This is the single most important portfolio-level fact. During normal times,
diversification works. During crashes, EVERYTHING correlates with BTC.
A "diversified" alt portfolio provides zero downside protection.
## 8. BingX-Specific Behavior vs Binance
| Metric | BingX / Binance Ratio | Interpretation |
|-----------------------|-----------------------|------------------------------------|
| Perp spread | 1.7-12.6x wider | BingX has less MM competition |
| BTC depth | 20x thinner | Much thinner books |
| Taker fee | 1.25x higher | 0.05% vs 0.04% |
| API latency | 2-3x higher | 100ms vs 40-50ms |
| Funding rate corr. | R² ~ 0.90, 0-8h lag | Use Binance as leading indicator |
| Spot spread | 18-389x wider | NEVER use BingX spot |
### Practical Implications
- **BingX is 10-20x harder to trade profitably** than Binance for the same strategy.
Thinner books + wider spreads + higher fees + higher latency.
- **Cross-exchange arbitrage** between BingX and Binance is real but latency-limited.
The 0-8h funding rate lag creates opportunities.
- **BingX perp is viable** for market making (wider spread = more edge) but
requires wider quotes and lower aggression.
## 9. Book Fragility & Cascade Dynamics
### Flash Crash Anatomy
1. **Trigger**: Large market sell hits thin book (0.05x depth at that moment)
2. **Cascade**: Price drops through stop levels → forced liquidations → more selling
3. **MM withdrawal**: All MMs pull quotes within 1-5ms
4. **Depth vacuum**: Book goes from $750K to $75K (BTC) or $22K to $6.6K (DOGE)
5. **Recovery**: 30-300 seconds for MMs to re-quote, 10-60 minutes for depth to normalize
### Per-Asset Cascade Characteristics
| Asset | Trigger Price Drop | Liquidation Speed | Recovery |
|--------|-------------------|-------------------|------------|
| BTC | 6.5% | slow | fast |
| ETH | 4.0% | medium | medium |
| SOL | 5.0% | medium | medium |
| DOGE | 4.0% | fast | slow |
| UNI | 3.5% | fast | slow |
| AAVE | 4.0% | fast | slow |
### OI/MCap Ratio (Liquidation Pressure)
- BTC: 0.5% of market cap in open interest → low cascade risk
- DOGE: 1.4% → moderate cascade risk
- ETH: 1.9% → higher cascade risk
- The ratio directly predicts how much forced selling occurs per 1% price drop
### Liquidation Trigger Threshold
The price drop needed to trigger cascade liquidations:
- BTC: 6.5% (hard to trigger → "too big to cascade")
- DOGE: 4.0% (easier to trigger → more volatile cascades)
- UNI: 3.5% (very easy to trigger → most fragile)
### Recovery Asymmetry
Crashes are FAST (milliseconds for MM withdrawal, seconds for liquidations)
but recovery is SLOW (minutes to hours for depth normalization).
This asymmetry is exploitable: buy the dip 30-60 seconds after the crash,
when depth is still thin but selling pressure is exhausted.
## 10. Retail vs Institutional Composition
| Asset | Retail Ratio | Institutional Gap | Implication |
|--------|-------------|-------------------|--------------------------------|
| BTC | 0.35 | 0.04 | Most institutional, best flows |
| ETH | 0.40 | 0.06 | Near-institutional |
| BNB | 0.50 | 0.20 | Balanced |
| SOL | 0.72 | 0.75 | Retail-dominated |
| DOGE | 0.80 | 0.31 | Heavily retail |
| UNI | 0.60 | 0.25 | Mixed |
| AAVE | 0.55 | 0.20 | Mixed |
### Trading Implications
- **Institutional assets (BTC, ETH)**: Tighter spreads, deeper books, more
efficient pricing. Edge comes from execution quality, not information.
- **Retail assets (DOGE, SOL)**: Wider spreads, more predictable order flow,
more stale-quote opportunities. Edge comes from toxicity detection + latency.
- **The institutional gap** = difference in quote persistence between institutional
and retail orders. Higher gap = more predictable behavior = more exploitable.
## 11. Funding Rates (BingX Perps)
| Asset | Mean (8h) | Std (8h) | Positive % | Basis Typical |
|--------|-----------|----------|------------|---------------|
| BTC | 0.59 bps | 0.22 bps | 100% | 4.0 bps |
| ETH | 0.50 bps | 0.25 bps | 95% | 3.5 bps |
| SOL | 0.30 bps | 0.35 bps | 65% | 2.5 bps |
| DOGE | 0.39 bps | 0.34 bps | 80% | 2.0 bps |
| BNB | 0.40 bps | 0.25 bps | 90% | 3.0 bps |
| ADA | 0.20 bps | 0.40 bps | 55% | 1.5 bps |
| AVAX | 0.15 bps | 0.35 bps | 50% | 2.0 bps |
| UNI | 0.10 bps | 0.30 bps | 45% | 1.5 bps |
| LINK | 0.25 bps | 0.30 bps | 60% | 2.0 bps |
| MATIC | 0.12 bps | 0.32 bps | 48% | 1.8 bps |
| AAVE | 0.08 bps | 0.28 bps | 40% | 1.2 bps |
| DOT | 0.18 bps | 0.30 bps | 55% | 2.0 bps |
| ATOM | 0.10 bps | 0.28 bps | 42% | 1.5 bps |
### Key Insight
BTC funding is ALWAYS positive (100% of time) at ~0.59 bps/8h. This means:
- Longs ALWAYS pay shorts on BTC perps
- Being short BTC perp earns a steady 0.59 bps every 8 hours
- This is "free money" for short-biased strategies (which MALKHUT is)
The funding rate is the single most predictable return stream in crypto perps.
## 12. Expected Slippage Model
For a market order of size $X:
slippage_bps = sum_{d=1}^{D} (A * d^(-alpha)) for cumulative depth >= X
Example for BTC ($750K amplitude, alpha=0.70):
- $10K order: ~0.05 bps (negligible)
- $100K order: ~0.5 bps (one tick)
- $1M order: ~3.5 bps (walks the book meaningfully)
- $10M order: ~15 bps (aggressive, will move the market)
Example for DOGE ($22K amplitude, alpha=1.00):
- $1K order: ~0.5 bps
- $10K order: ~5 bps
- $100K order: ~50 bps (very aggressive, huge impact)
### Implication
BTC allows $100K orders with <1 bps slippage. DOGE requires $1K orders for
the same. Position sizing must account for the book's capacity, not just
the strategy's signal.
## 13. Fill Quality Optimization (MALKHUT Core)
MALKHUT is an execution improvement engine. Fill quality IS the primary aim.
### Fill Quality Metrics
For each CWM transition, MALKHUT computes:
- **slippage_bps**: aggressive fills — how far from mid?
- **price_improvement_bps**: passive fills — how much better than touch?
- **levels_consumed**: queue depth of fill
- **post_fill_adverse_bps**: price movement after fill (negative = adverse)
- **fill_value_score**: composite = quality - adverse * 0.5
### How This Connects to the OB
The OB microstructure directly determines fill quality:
| OB Characteristic | Impact on Fill Quality |
|-------------------|----------------------|
| **Depth at touch** | More depth = more fill opportunities for passive orders |
| **Spread** | Tighter spread = smaller price improvement possible |
| **Depth decay (alpha)** | Steeper decay = fills walk the book faster = higher slippage |
| **Cancel/fill ratio** | Higher ratio = more queue churn = harder to get fills |
| **MM pull speed** | Faster pull = stale quotes less likely = harder to snipe |
| **Book fragility** | During stress, depth drops 70-95% = fills at worse prices |
### Per-Asset Fill Quality Expectations
| Asset | Expected Fill Quality | Why |
|-------|----------------------|-----|
| BTC | Excellent | Deep book, tight spread, fast MM re-quote |
| ETH | Good | Similar to BTC, slightly thinner |
| SOL | Moderate | Mid-depth, moderate spread |
| DOGE | Poor | Thin book, wide spread, slow MM |
| ADA | Poor | Thin book, wide spread |
### Optimization Strategy
1. **Venue selection**: choose venues with better fill quality (Binance > BingX)
2. **Order type selection**: use POST_ONLY on tight-spread assets, MARKET on thin-spread
3. **Offset optimization**: CMA-ES learns optimal offset per (asset, regime, venue)
4. **Size optimization**: CMA-ES learns optimal size per (asset, regime, venue)
5. **Timing optimization**: CMA-ES learns when to quote vs when to wait
The PerformanceMatrix tracks `avg_fill_value_score` per (regime, strategy, venue),
enabling the system to learn: "In this regime, on this venue, this strategy
achieves the best fill quality."
## 14. Flight9/BLUE Fill Learnings (Fable, 2026-07-16)
Real FLIGHT9/BLUE fill backfill — generalizable features, not hardcoded thresholds.
### Markout = Quality
Fill quality is measured by **post-fill markout** (price move N ticks after fill),
not just fill/no-fill. A maker fill at a good quoted price can still be a bad fill
if the market moves adversely after execution. The system scores fills by markout.
**Generalizable:** score fills by post-fill markout. Model fill-CONDITIONAL-on-adverse-flow.
### Fee Model (BingX, No Rebates)
BingX reports fees as NEGATIVE. MALKHUT convention: positive = cost.
- Maker fee: 2.0 bps (you pay)
- Taker fee: 5.0 bps (you pay)
- Fee savings: 3.0 bps (maker saves 3bp vs taker)
- There are NO rebates on BingX.
The system learns: prefer maker when fee savings > fill probability cost.
### Fill = Queue × Flow Intensity
Fill probability is driven by **trade-arrival intensity** (the tape), not the static book.
A resting maker fills only when trades print through its level for enough volume to clear
the queue ahead. The HftBacktestCWM queue model is the right substrate — it needs real
trade-flow intensity as input.
**On testnet:** maker fill-rate ~0% (no flow). This is an artifact, not a signal.
### Depth-for-Size, Not Spread
Spread alone LIES: an asset with 1.5bp spread behind ~$977 of depth is unfillable at any
size. Generalizable: key fill viability on **depth-within-K-bps** relative to order
notional, not spread.
### Measured Fees (Venue-Parameterized)
| Venue | Maker | Taker | Sign |
|-------|-------|-------|------|
| BingX | 2.00 bp | 5.016 bp | NEGATIVE = DEBIT |
Generalize: parameterize fee + sign per venue from measurement, never assume.
### Realized Friction (F9 VST Fills)
- Taker: ~20-27 bp adverse on thin/mid books
- Maker: saves ~3-4 bp WHEN it fills
- Maker fill-rate: ~0% on VST (no flow — testnet artifact)
### Counterfactual Maker Fill (Real Binance Book)
- ~50% fill on liquid books (at-touch upper bound)
- Queue + venue-thinness reduce it
### Validation Methodology
When validating against a crude fill sim, optimize on **RELATIVE lift** between two policies
through the SAME sim — fidelity bias cancels in the difference. Trust the ordering of
policies even when absolute fill rates are approximate.