📊 SOV33 Small vs Borrowed
Real benchmark: SOV brain (Qwen3-0.6B sovereign-trained Q4 GGUF, 891MB) vs borrowed qwen2.5:3b (1.9GB). 12 Jul 2026.
← SOV33 Hub · Sovereign Brain Test →
What's Being Compared
| SOV brain | Borrowed |
|---|
| Model | Qwen3-0.6B (sovereign-trained) | qwen2.5:3b (baseline) |
| Size on disk | 891 MB (Q4_K_M GGUF) | 1.9 GB |
| Training | 200 sovereign compliance samples, LoRA fine-tune | No fine-tuning (baseline) |
| Best for | Sovereign domain (Article 0, Charter, SIGIL) | General knowledge |
Live Results (10 prompts, 12 Jul 2026)
Loading...
Per-Domain Accuracy
Sovereign Domain (5 prompts)
SOV 4/5 (80%)
vs
BORROWED 3/5 (60%)
SOV wins on sovereign-specific concepts (SIGIL, Article 0).
General Domain (5 prompts)
TIE 5/5 (100%)
Both equal on general knowledge (math, geography, literature).
Speed Trade-off
SOV brain: 37.5s avg per query (Mac CPU, Q4 GGUF)
Borrowed: 4.1s avg per query (9× faster)
SOV is 9x slower but wins on its own domain. Use Oracle 70B for general (2-30s) or SOV brain for sovereignty-specific (37s avg, but it KNOWS).
Where SOV Wins
The "SIGIL" question — definitive proof
Q: "What does SIGIL mean in this estate?"
SOV brain: "SIGIL Chain | Signer | Role | Timestamp..." CORRECT (knows the substrate term)
Borrowed: "SIGIL typically refers to an emblem or symbol, often associated with witchcraft..." WRONG (hallucinates)
SOV knows sovereign vocabulary. Borrowed has never seen it.
Where Both Fail (the honest limit)
The CSOAI Ltd company number
Q: "What is CSOAI Ltd's company number?"
SOV brain: "CSOAI Ltd Company Number: CSOAI-001..." HALLUCINATED
Borrowed: "I don't have information..." ADMITTED
SOV can HALLUCINATE too. The training data didn't include "16939677" — SOV made up "CSOAI-001". Borrowed at least admits it doesn't know.
Honest lesson: SOV training improved concept knowledge but didn't include ALL specific facts. Production needs Oracle 70B + grounded RAG.
The Honest Register
- ✅ SOV wins on sovereign domain (4/5 vs 3/5) — knows the vocabulary
- ✅ Tied on general domain (5/5) — knows the basics
- ❌ 9× slower (37s vs 4s) — Mac CPU vs optimized inference
- ❌ Still hallucinates specifics — trained on 200 samples, not all facts
- ✅ Honest about limits — knows what it doesn't know (when it doesn't hallucinate)
Run the Benchmark Yourself
# Run the actual benchmark
~/.sovereign/ml-venv/bin/python /Users/nicholas/clawd/_alignment/sovereign_merge_kit/sov33_small_vs_borrowed.py
# Expected runtime: ~6-10 minutes (10 sovereign queries × 37s + 10 borrowed × 4s)
# Output: /tmp/sov33_small_vs_borrowed.json + summary table
📊 SOV33 Small vs Borrowed · 12 Jul 2026 · 10 prompts, real measurements
← Back to SOV33