Model Backtest

Segment Router v1.3 improves versus the best v1.1 hybrid on the same rows. Keep it in research/shadow until live tracking confirms the routing policy. The router is walk-forward: each target quarter can only use earlier quarters to decide which model wins in each segment.

Do not chase generic feature tweaks next. Use v1.4 to build specialist heads for weak action types and show calibrated confidence in current predictions so high raw probabilities do not imply false certainty.

Directional model does not yet beat the trim/exit majority baseline. Keep it as a diagnostic and improve feature/threshold design before using it in predictions.

Build/Keep exception model finds a higher-quality subset, but needs stronger recent consistency before it can affect predictions.

Positive macro offsets are future-oracle diagnostics. They can tell us whether Duquesne anticipated later macro changes, but they must be converted into point-in-time expectation proxies before any production use.

Best setup: Current raw + proxy. These proxies are built from point-in-time market expectation signals, so the improvement is production-safe, though still below the primary router.

Boundary labels are ADD/REDUCE target actions with absolute next-quarter share change from 5% to below 10%; these sit near the current 5% label threshold and should be treated as noisier than EXIT or large share changes.

The production-safe allocation predictor is not yet accurate enough to replace the existing router. Its useful signal is narrower: it improves Build/Keep recall while hurting overall directional accuracy, so it should feed a future Build/Keep specialist rather than the global model.

v2.1 barely improves global 4-class accuracy, but it improves ADD and Build/Keep recall. The standalone gates are only mildly better than base rate, so the channel signal should be used as a feature layer rather than a standalone trade signal.

Build/Keep exception model has not yet cleared the precision and coverage bar. Keep it as diagnostics. The best pocket is high precision but very low coverage, so it stays research-only until live shadow tracking proves recent consistency.

v2.3 tests the Druckenmiller-style idea that market internals may lead official macro data. The current result is useful but narrow: directional accuracy and Build/Keep recall improve slightly, while global 4-class accuracy and channel allocation accuracy decline. Keep it research-only until it proves broader lift in live shadow tracking.

A hierarchical/rules hybrid is the best candidate, but the pure hierarchical model has not cleared the promotion bar. Keep rules in production. Best pure hierarchical model is Hierarchical regularized at 41%(+0.6 pts vs rules). Hybrid lift is promising, but production still needs a pure-model or live-shadow confirmation before replacing rules.

Macro-first v1.2 is now measurable, but it has not cleared the production bar. Keep rules-v0.6 as production and use v1.2 for diagnostics/shadow comparison. Best candidate is Macro prior blend 10%; macro is now an explicit upstream prior, but the current backtest says it should remain research-only.

Keep rules-v0.6 as production. Model v1.1 has a shadow snapshot, but no future 13F has arrived yet for live validation.

Model v1 is training successfully, but it has not beaten the existing rules with enough margin. Keep rules-v0.6 as production and use Model v1 for diagnostics and shadow tracking. Current pure-model leader is All features 3-seed bag at 40%; rules remain 40% on the same walk-forward rows.

This is a dependency-free baseline, not the production predictor. It trains only on prior target quarters and compares against rules-v0.6 on the same evaluated rows.

Ablation uses the same walk-forward online softmax setup as the ML baseline and changes only the feature columns. Current result: behavior remains the strongest foundation; company, portfolio, and macro layers help only when combined with behavior.

Top probabilities are calibrated against historical walk-forward hit rates. Use them as conviction rankings plus calibrated ranges, not literal certainty.

Thresholds are gates for whether to treat a model prediction as actionable. They do not change the predicted ADD/HOLD/REDUCE/EXIT label.

Keep the production company overlay weak for now; higher company weights did not create a large enough aggregate gain.

Keep segmented company weighting in research mode; it did not beat the simple 1x company overlay.

A perfect probability model would have actual hit rates close to the average predicted probability in each bucket. The current rules model is useful directionally, but still overconfident in higher-probability buckets.