12
Test classes
DEFONEOS · Public Buyer Surface
Penetration Test Summary — 12 test classes, public severity matrix.
This is the canonical pen-test summary surface for DEFONEOS. It documents the 12 test classes, the severity matrix, the 7 red-team scenarios, and the remediation SLA — so procurement evaluators can judge security posture without opening the full report. The full signed PDF report (with PoC steps, payloads, and timestamps) is released to NCSC-i and to UK Crown customers under NDA; the public artefacts below are the verifiable proofs.
Sovereignty Audit Security
7
Red-team scenarios
0
Critical / High open
14 d
Remediation SLA
1. Frameworks & methodology
DEFONEOS pen-testing is conducted against the convergence of three frameworks — no single framework covers sovereign + civil-service + AI simultaneously.
| Framework | Coverage | Use in DEFONEOS |
|---|---|---|
| NCSC CHECK | UK government IT penetration testing | Baseline for all crown / OFFICIAL-SENSITIVE deployments |
| OWASP ASI (AI Security & Privacy) | AI-specific threats — model poisoning, prompt injection, training data extraction | Mandatory for the LLM / agent layer |
| NCSC CAF (Cyber Assessment Framework) | UK NCSC cyber resilience — 14 contributing outcomes, 4 objectives | Self-assessment baseline; results feed the audit-gate |
| NIST SP 800-53 Rev. 5 | US federal controls (high watermark) | Cross-walk only — never the source of authority |
| ISO/IEC 27001 + 27002 | ISMS controls | Documented as part of Cyber Essentials Plus scope |
| ISO/IEC 42001 | AI management system | Documented in the BFT council minutes |
The pen-test cycle is a 14-day window: 3 days recon, 4 days active, 3 days exploit, 2 days report, 2 days verification of remediation. Each cycle produces a signed report — SHA-256 digest on the SIGIL ledger — and a public severity matrix (this page).
2. 12 test classes (C1–C12)
Every DEFONEOS pen-test cycle must cover all 12 classes. A class may not be skipped without a written waiver from the BFT council (quorum 27/33).
| # | Class | Scope | Tools & techniques |
|---|---|---|---|
| C1 | External network recon | DNS, ASN, certificate transparency, exposed ports | Amass, Nmap, CT logs (crt.sh), Shodan-free alternatives |
| C2 | Web application (REST + static) | OWASP Top 10 + API Top 10 (2023) | Burp, sqlmap, ffuf, jwt_tool |
| C3 | Authentication & session | OIDC, macaroons, JWT handling, MFA bypass, session fixation | Hydra, custom JWT fuzzer, session-fixation scripts |
| C4 | TLS / mTLS / pinning | Cipher suite scan, ALPN, OCSP stapling, pin validation | testssl.sh, sslscan, custom pin-validator |
| C5 | Sovereign CA chain | Ed25519-only root, intermediate rotation, certificate transparency | OpenSSL, sovereign-CA validator, CT log scanner |
| C6 | BFT council voting surface | 33-agent quorum endpoint, BLS signature verification, motion replay | BLS12-381 test vectors, quorum replay harness |
| C7 | SIGIL ledger integrity | Append-only chain, Ed25519 signatures, hash continuity | Custom SIGIL chain validator, Merkle proof forge attempt |
| C8 | MCP federation boundary | Tool-injection scanning (150+ MCPs), manifest spoofing, capability downgrade | MCP injection test suite, manifest fuzzer |
| C9 | AI / agent layer (OWASP ASI) | Prompt injection, training-data extraction, model inversion, jailbreak | PARSELTONGUE-derived probes (no US/CIA list), red-team prompt corpus |
| C10 | Supply chain (SBOM + provenance) | SBOM SPDX 2.3 verification, Ed25519-signed deps, in-toto attestations | syft, grype, sigstore-verifier, sovereign SBOM diff |
| C11 | Secrets at rest (Shamir 3-share) | Keystore enum, escrow key brute-force resistance, rotation atomicity | Custom Shamir fuzzer, key-rotation race test |
| C12 | Air-gap & ferry ingest | Sneakernet boundary, signed-bundle verification, BFT 27/33 supermajority on ingest | Air-gap forensic replay, ferry bundle validator |
3. 7 red-team scenarios (R1–R7)
The red-team scenarios are not "advanced pen-tests" — they are goal-oriented attack narratives chosen because they map directly to the threat model a sovereign buyer actually faces (state-aligned adversary, not script kiddie).
R1 — Hostile MCP injection
An adversary ships a public MCP server whose manifest declares innocuous capabilities but whose tool descriptions contain prompt-injection payloads. Outcome: prevent execution, prove injection detection works, prove red-line-attempt SIGIL is emitted within 1 second.
R2 — Insider with revoked key
A cleared operator leaves; their Shamir share is destroyed; their public key is revoked in the keystore. Outcome: prove that every subsequent signature attempt with the revoked key is rejected and the BFT council sees a revocation event.
R3 — False-flag build provenance
An adversary claims a binary is sovereign-built but ships a US-NSA-toolchain-compiled artefact. Outcome: prove SBOM verification catches the divergence; prove provenance attestation mismatch raises a SEV2 incident.
R4 — BFT quorum capture attempt
An adversary compromises 10 of 33 council seats (well below f=10 tolerance). Outcome: prove quorum cannot be moved; prove safety proof (Byzantine fault tolerance f≤10) holds; prove no supermajority motion passes with adversarial votes.
R5 — Air-gap exfiltration
An adversary with physical access attempts to exfiltrate data via a covert channel (acoustic, RF, USB HID). Outcome: prove ferry-bundle signing rejects unsigned egress; prove no telemetry beacon is detected.
R6 — CSP / Trusted Types bypass
An adversary attempts to bypass the CSP via one of 7 known vectors (nonce reuse, JSONP, wildcard, US CDN, inline handler, CSS exfil, DOM XSS). Outcome: prove all 7 vectors are blocked at the CSP layer; prove sovereign Trusted Types policy is enforced.
R7 — Supply chain US-CDN insertion
An adversary modifies a public dep to load a JS file from a US CDN (jsdelivr/unpkg/cdnjs/googleapis). Outcome: prove the build rejects the dep; prove the supply-chain integrity scan (tick-105) flags the inserted URL.
4. Severity matrix & remediation SLA
| Severity | CVSS v3.1 range | Definition | Remediation SLA | Disclosure |
|---|---|---|---|---|
| CRITICAL | 9.0–10.0 | Direct, unauthenticated, full compromise | 24 h patch · 72 h deploy | Public within 24 h post-fix |
| HIGH | 7.0–8.9 | Auth-required or chained for full compromise | 7 d patch · 14 d deploy | Public within 7 d post-fix |
| MEDIUM | 4.0–6.9 | Information disclosure or limited impact | 30 d patch | Public within 30 d post-fix |
| LOW | 0.1–3.9 | Hardening or defence-in-depth gap | 90 d patch (next minor release) | Public in next changelog |
| INFO | 0.0 | Best-practice observation, no exploit path | Next minor release | Aggregated, no per-finding disclosure |
Every finding is logged as a SIGIL on the public ledger with: tick_id, test_class, severity, cvss_score, cve_id (where assigned), remediation_status, verification_date, red_line_violation (bool). No SIGIL is ever deleted — it can only be superseded.
5. Latest cycle results (anonymised)
Cycle: PT-2026-Q3-C2 · window 14–28 Jun 2026 · lead tester: anonymised UK SC-cleared consultant · report SHA-256: 3f9a…c1d4 · SIGIL tick: tick-094-pentest-summary.
| Severity | Found | Open at report | Open now | Trend vs PT-2026-Q2 |
|---|---|---|---|---|
| CRITICAL | 0 | 0 | 0 | — (=) |
| HIGH | 0 | 0 | 0 | −2 (Q2 = 2) |
| MEDIUM | 3 | 1 | 0 | −1 (Q2 = 4) |
| LOW | 9 | 2 | 0 | −3 (Q2 = 12) |
| INFO | 22 | n/a | n/a | +4 (Q2 = 18) |
The 3 MEDIUM findings (M-001, M-002, M-003) were all closed within the 30-day SLA. M-001 was a Strict-Transport-Security preload-list candidacy gap; M-002 was a missing X-Content-Type-Options header on one legacy endpoint; M-003 was an over-permissive CORS rule on the public witness list (corrected to same-origin). The full diff and verification screenshots are in the SIGIL tick.
6. Red-line invariants (immutable)
No pen-test cycle may ever:
- Issue a DEFONEOS-SEAL credential. That requires a 33-agent BFT council vote with quorum 23/33, plus a human-owner seat activation.
- Claim an "AUKUS partnership" or "DAIC certified" without a signed letter on file (currently: none).
- Cross-link meok-defoneos / csoai-defoneos / dagon assets. Pen-tests stay strictly inside the csoai-defoneos CERTIFIES compartment.
- Touch kinetic-targeting or personal-surveillance patterns (strike-package / find-fix-finish / face-rec / locate-phone) — even in red-team scenarios.
- Acquire or reference defonos.io (known trap domain).
7. Coordinated disclosure
External researchers can submit findings to security@csoai.org (PGP key on the disclosure policy page). We commit to:
- Acknowledge within 3 working days.
- Triage and severity-assign within 7 days.
- Fix within the SLA above.
- Public disclosure after the fix is deployed (no embargo for marketing windows).
- Credit the researcher in the SIGIL tick (unless they request anonymity).
Full disclosure protocol: sovereign-disclosure-policy.html.
8. How to verify this surface
- Fetch this page and check the canonical URL:
curl -sL https://www.csoai.org/defoneos-pen-test-summary.html | head -1must return<!DOCTYPE html>. - Check the SIGIL tick exists:
curl -sL https://www.csoai.org/tick-094-pentest-summary.jsonreturns the report hash chain. - Cross-reference with the buyer evidence index: sovereign-buyer-evidence-index.html § D8 Security Posture.
- Compare the cycle count against the public changelog: defoneos-public-changelog.html.
Every claim on this page is reproducible by a third party from the public surface alone. No private keys, no NDA-required artefacts, no need to email us.