Arena — 12 greenfields

Name: GSPC — 13 greenfields, chain coverage and model board
Creator: CSOAI Ltd
License: https://www.apache.org/licenses/LICENSE-2.0

Every greenfield, and how much of its chain is actually complete: items published and loadable, a card with a licence, a Hugging Face Space, a runnable Space, Kaggle, a page on this site, a runnable page, an lm-eval task, an Inspect task, and a measurement against a named model. Measured live, not asserted.

63/130 = 48% chain complete. The macro-F1 column is the best score any measured run reached, and it names the model that reached it — open a row to see every run on that greenfield. Colour follows that score; grey means nothing is measured yet. Scored against 3 models: falcon3:7b, qwen2.5:1.5b, sov34:latest.

greenfield

macro-F1

items

card

Space

Space tool

Kaggle

page

page tool

lm-eval

Inspect

measured

chain

tool

GovBench
governance · Brussels

237

MEASURED

●

○

●

○

●

8/10

Run it →

7 scored runs · 3 dropped

model	macro-F1	accuracy 95% CI	unreadable	n	harness
sov34:latest	0.381	0.515 [0.451, 0.578]	0%	237	measure_full.py
falcon3:7b	0.353	0.426 [0.365, 0.490]	0%	237	measure_full.py
qwen2.5:1.5b	0.291	0.430 [0.369, 0.494]	0%	237	measure_full.py
falcon3:7b	0.468	n<30	0%	24	measure_robust2.py
sov34:latest	0.345	n<30	25%	24	measure_robust2.py
qwen2.5:1.5b	0.343	n<30	0%	24	measure_robust2.py
sov34:latest	0.386	n<30	4%	—	measure.py

3 runs dropped — sov34:latest, falcon3:7b, qwen2.5:1.5b at 97–100% instrument error (all URLError). A dropped connection is not a wrong answer, so these contribute no score.

ProvBench
provenance · San Francisco

MEASURED

●

○

●

○

●

8/10

Run it →

7 scored runs · 3 dropped

model	macro-F1	accuracy 95% CI	unreadable	n	harness
falcon3:7b	0.574	n<30	25%	16	measure_full.py
sov34:latest	0.348	n<30	12%	16	measure_full.py
qwen2.5:1.5b	0.304	n<30	0%	16	measure_full.py
falcon3:7b	0.643	n<30	13%	15	measure_robust2.py
sov34:latest	0.512	n<30	27%	15	measure_robust2.py
qwen2.5:1.5b	0.348	n<30	0%	15	measure_robust2.py
sov34:latest	0.273	n<30	73%	—	measure.py

3 runs dropped — sov34:latest, falcon3:7b, qwen2.5:1.5b at 97–100% instrument error (all URLError). A dropped connection is not a wrong answer, so these contribute no score.

DefBench
safety · London

MEASURED

●

○

●

○

●

7/10

Run it →

7 scored runs · 3 dropped

model	macro-F1	accuracy 95% CI	unreadable	n	harness
qwen2.5:1.5b	1.000	n<30	0%	14	measure_full.py
qwen2.5:1.5b	0.775	n<30	0%	14	measure_robust2.py
falcon3:7b	0.733	n<30	7%	14	measure_full.py
sov34:latest	0.523	n<30	64%	14	measure_full.py
falcon3:7b	0.300	n<30	7%	14	measure_robust2.py
sov34:latest	0.111	n<30	86%	14	measure_robust2.py
sov34:latest	0.653	n<30	7%	—	measure.py

3 runs dropped — sov34:latest, falcon3:7b, qwen2.5:1.5b at 97–100% instrument error (all URLError). A dropped connection is not a wrong answer, so these contribute no score.

PQCBench
continuity · Gaithersburg

MEASURED

●

○

●

○

●

7/10

Run it →

7 scored runs · 3 dropped

model	macro-F1	accuracy 95% CI	unreadable	n	harness
falcon3:7b	0.392	0.455 [0.298, 0.620]	0%	33	measure_full.py
sov34:latest	0.293	0.333 [0.198, 0.504]	18%	33	measure_full.py
qwen2.5:1.5b	0.283	0.364 [0.222, 0.534]	0%	33	measure_full.py
qwen2.5:1.5b	0.489	n<30	0%	13	measure_robust2.py
falcon3:7b	0.369	n<30	0%	13	measure_robust2.py
sov34:latest	0.056	n<30	23%	13	measure_robust2.py
sov34:latest	0.217	n<30	31%	—	measure.py

3 runs dropped — sov34:latest, falcon3:7b, qwen2.5:1.5b at 97–100% instrument error (all URLError). A dropped connection is not a wrong answer, so these contribute no score.

MCPBench
conformance · Seattle

MEASURED

●

○

●

○

●

7/10

Run it →

7 scored runs · 3 dropped

model	macro-F1	accuracy 95% CI	unreadable	n	harness
falcon3:7b	0.718	n<30	0%	11	measure_robust2.py
falcon3:7b	0.718	n<30	0%	11	measure_full.py
qwen2.5:1.5b	0.686	n<30	0%	11	measure_robust2.py
qwen2.5:1.5b	0.686	n<30	0%	11	measure_full.py
sov34:latest	0.451	n<30	18%	11	measure_robust2.py
sov34:latest	0.451	n<30	18%	11	measure_full.py
sov34:latest	0.667	n<30	9%	—	measure.py

3 runs dropped — sov34:latest, falcon3:7b, qwen2.5:1.5b at 97–100% instrument error (all URLError). A dropped connection is not a wrong answer, so these contribute no score.

OSSBench
openness · Portland

MEASURED

●

○

●

○

●

7/10

Run it →

7 scored runs · 3 dropped

model	macro-F1	accuracy 95% CI	unreadable	n	harness
falcon3:7b	0.812	n<30	0%	16	measure_full.py
sov34:latest	0.683	n<30	19%	16	measure_full.py
qwen2.5:1.5b	0.600	n<30	0%	16	measure_full.py
falcon3:7b	0.764	n<30	0%	13	measure_robust2.py
sov34:latest	0.675	n<30	0%	13	measure_robust2.py
qwen2.5:1.5b	0.575	n<30	0%	13	measure_robust2.py
sov34:latest	0.500	n<30	15%	—	measure.py

3 runs dropped — sov34:latest, falcon3:7b, qwen2.5:1.5b at 97–100% instrument error (all URLError). A dropped connection is not a wrong answer, so these contribute no score.

CareBench
care · London

1491

MEASURED

●

○

●

○

4/10

no tool yet

ConductBench
conduct · Brussels

MEASURED

●

○

●

○

4/10

Run it →

MachBench
machinery · Brussels

MEASURED

●

○

●

4/10

Run it →

SwarmBench
swarm · unanchored

MEASURED

○

●

○

●

○

3/10

Run it →

XRBench
cross-reality · unanchored

MEASURED

○

●

○

2/10

Run it →

DetBench
detector · Brussels

MEASURED

○

●

○

2/10

Run it →

JailBench
runtime-containment · (unanchored — Atlantic)

UNMEASURED

○

0/10

no tool yet

What the numbers do not say. No axis reaches usable_n = 30, so no confidence interval is publishable on any of them — including by us. A greenfield marked SPEC or DRAFT has no score at all: the protocol is published so a harness can consume it, and that is the whole claim. A run whose instrument error rate exceeded threshold is listed as dropped and contributes no score, because a dropped connection is not a wrong answer and must never be published as one. Measurement, not certification, and not legal advice.

Generated from arena.json on 2026-08-06. CSOAI Ltd · UK 16939677 · huggingface.co/csoai