LLM voting panels have highly correlated errors (60%+ agreement on wrong answers). Naive majority voting dilutes the best judge's signal.
Verbatim from arXiv 2605.29800: "when judges are highly correlated, majority voting dilutes the best judge's signal with redundant weaker votes."
Question | L1 (sov_brain_v2) | L2 (owem) | L3 (master) | ฯ avg | Decision What is Article 0? | Article 0 binds | (varied) | (varied) | 0.000 | CONSENSUS_OK (diverse) What is SIGIL? | Sovereign Identity | (varied) | (varied) | 0.000 | CONSENSUS_OK What is BFT-33? | 33-33 quorum | (varied) | (varied) | 0.000 | CONSENSUS_OK What are 12 Pillars? | 12 sovereign | (varied) | (varied) | 0.000 | CONSENSUS_OK What is sovereign subst.? | Foundation for AI | (varied) | (varied) | 0.000 | CONSENSUS_OK Verdict: DIVERSE โ (avg ฯ = 0.000) A council with ฯ โ 0.0 is real fault tolerance (vs ฯ โ 0.9 which is theatre).
Endpoint: POST /api/l4-panel with {"message":"..."}
Raw LLM confidence scores are NOT calibrated (SCOPE, arXiv 2602.13110). "Felt unsure" is not a guarantee.
Split-conformal abstention gives: Pr[no-abstain AND error] โค ฮฑ โ a DISTRIBUTION-FREE statistical guarantee.
| Metric | Result |
|---|---|
| Calibration set size | 500 sovereign prompts |
| Target ฮฑ (false-allow rate) | 5% |
| Threshold | 0.5726 |
| Test set | 100 new sovereign queries |
| Vetoed (abstained) | 5 (5.0%) |
| Errors on passed | 0 |
| False-allow rate | 0.0% (target โค 5.0%) โ |
| Coverage | 95.0% (target 90.0%) โ |
| Guarantee holds | TRUE |
"SOV33 care-floor is split-conformal calibrated to โค 5% false-allow rate at 90% coverage. The guarantee is distribution-free (no assumptions on input distribution)."
Endpoint: GET /api/conformal-veto โ returns calibration + evaluation results
"SOV33 stops being 'a governance wrapper' and becomes a governed system with measured diversity, a calibrated risk bound, and provable hard constraints โ three sentences no other solo-built agent stack can currently say."
/api/l4-panel + /api/conformal-veto