malkhut(wire): fill quality as PRIMARY optimization target

Fill quality is MALKHUT's core aim. Wired end-to-end:

1. FillQuality state (state.py):
   - slippage_bps, price_improvement_bps, levels_consumed
   - is_maker_fill, rolling_fill_rate, post_fill_adverse_bps
   - fill_value_score: composite metric for optimization
   - Added to MarketWorldState.fill_quality field

2. HftBacktestCWM.transition() (hft_cwm.py):
   - _compute_fill_quality() computes all metrics per transition
   - Fill quality now tracked for every CWM step
   - Empty book guards added for safety

3. MinimalCryptoLOBCWM.transition() (core.py):
   - Same fill quality computation for deterministic fallback
   - Empty book guards added

4. Reward function (hft_cwm.py):
   - fill_quality_reward = w_fill_probability * fill_value_score (PRIMARY)
   - Bonus for maker fills that improve price
   - Penalty for adverse selection after fill
   - Base reward (PnL, adverse selection, fees) preserved

5. PerformanceMatrix (selector.py):
   - RegimeStrategyScore: 4 new fill quality fields
   - record(): accepts fill_rate, slippage, price_improvement, fill_value_score
   - EMA updates for all fill quality metrics

6. EpisodeResult (cma_trainer.py):
   - avg_fill_value_score, avg_price_improvement_bps, avg_post_fill_adverse_bps
   - Accumulated per-step during _run_episode
   - Recorded to PerformanceMatrix in evaluate_candidate

All 1379+ tests green.
This commit is contained in:
Codex
2026-07-15 15:22:25 +02:00
parent fa76070c79
commit 618ad723e3
5 changed files with 284 additions and 29 deletions

View File

@@ -277,6 +277,10 @@ class EpisodeResult:
final_equity: float = 0.0
max_position_qty: float = 0.0
diagnostics: Mapping[str, Any] = field(default_factory=dict)
# Fill quality (PRIMARY metrics)
avg_fill_value_score: float = 0.0
avg_price_improvement_bps: float = 0.0
avg_post_fill_adverse_bps: float = 0.0
# ==============================================================================
@@ -1056,6 +1060,10 @@ class PolicyEvaluator:
drawdown_bps=result.max_drawdown_bps,
adverse_fill_ratio=result.adverse_fill_count / max(result.order_count, 1),
venue=venue_tag,
fill_rate=result.fill_ratio,
slippage_bps=result.avg_slippage_bps,
price_improvement_bps=result.avg_price_improvement_bps,
fill_value_score=result.avg_fill_value_score,
)
score = self._robust_score(results, params)
return score, results
@@ -1108,6 +1116,11 @@ class PolicyEvaluator:
entropy_sum = 0.0
equity_start = state.account.equity
cancel_count = 0
# Fill quality accumulation (PRIMARY metrics)
fq_fill_value_sum = 0.0
fq_price_improve_sum = 0.0
fq_adverse_sum = 0.0
fq_count = 0
for step in range(scenario.max_steps):
# Plan with minimal overhead
@@ -1145,6 +1158,14 @@ class PolicyEvaluator:
cp_actions = tuple(cp.rollout_action(state, rng) for cp in scenario.counterparties)
next_state = cwm.transition(state, (action, *cp_actions))
# Accumulate fill quality from transition
if next_state.fill_quality:
fq = next_state.fill_quality
fq_fill_value_sum += fq.fill_value_score
fq_price_improve_sum += fq.price_improvement_bps
fq_adverse_sum += fq.post_fill_adverse_bps
fq_count += 1
pnl = next_state.account.equity - equity_start
pnl_bps = 10_000.0 * pnl / max(equity_start, 1.0)
total_pnl_bps = pnl_bps
@@ -1159,6 +1180,7 @@ class PolicyEvaluator:
state = next_state
steps = step + 1 if scenario.max_steps > 0 else 0
fq_n = max(fq_count, 1)
return EpisodeResult(
scenario_id=scenario.scenario_id, policy_version=params.version,
seed=rng_seed, steps=steps, pnl_bps=total_pnl_bps, realized_pnl=0.0,
@@ -1172,6 +1194,9 @@ class PolicyEvaluator:
policy_entropy_avg=entropy_sum / max(steps, 1),
final_equity=state.account.equity, max_position_qty=0.0,
diagnostics={"scenario_tags": scenario.tags},
avg_fill_value_score=fq_fill_value_sum / fq_n,
avg_price_improvement_bps=fq_price_improve_sum / fq_n,
avg_post_fill_adverse_bps=fq_adverse_sum / fq_n,
)
def _robust_score(self, results: list[EpisodeResult], params: FulfilmentPolicyParams) -> float: