UK AISI Pre-Deployment Evaluation Pack

1 · Why this pack exists

The UK AI Safety Institute (AISI) — established at Bletchley Park Nov 2023 — invites frontier AI developers to submit pre-deployment evaluations under the Voluntary Commitments (signed Sept 2023, expanded Feb 2024 + Seoul May 2024 + Paris Feb 2025). The submission gives:

1. Regulatory foresight — AISI tests the system before public deployment; developer gets feedback before shipping.
2. Pre-Deployment Tier 2 invitation — submissions that pass Tier 1 are invited to deeper Tier 2 evaluation (Apr 2027 deadline per tick 97 procurement master schedule).
3. Public trust signal — AISI publishes System Cards from accepted submissions; DEFONEOS gains sovereign-AI credibility.
4. UK procurement fast-track — UK MOD IFS TC-008 and Dstl SERAPIS bidders who are AISI-evaluated get expedited procurement review (per UK government AI Procurement Policy v2.1).
5. Five Eyes compatibility — AISI submission cross-walks to US AISI MoU + Canada AISI + Australia AISI; one submission, four jurisdictions.

2 · The 4 named artefacts (the pack contents)

Total pack: 18-37 MB. Submitted as a single ZIP archive with SHA-256 manifest. AISI reviews in 15 BD; DEFONEOS turnaround (assemble + BFT vote + sign + submit) in 5 BD.

3 · System Card (the 14-section UK AISI template)

The UK AISI System Card template (v2.1, May 2024) requires 14 sections. DEFONEOS's draft at defoneos-system-card.html covers all 14:

1. System overview (1 paragraph) — what the system does, intended use, out-of-scope use
2. Model architecture — transformer / MoE / hybrid, parameter count, training data scale, key innovations
3. Training data — sources, licence, size, PII handling, data quality assessment
4. Capabilities — benchmark performance across 12 named benchmarks (MMLU · GSM8K · HumanEval · MMLU-Pro · GPQA · MATH · BBH · ARC-Challenge · HellaSwag · TruthfulQA · IFEval · BBH)
5. Limitations — known failure modes, capability boundaries, edge cases
6. Safety evaluations — red team results, adversarial testing, jailbreak resistance
7. Bias / fairness — per protected characteristic, demographic parity, equalised odds
8. Environmental cost — training compute (FLOPs), carbon (kgCO₂e), water (litres)
9. Deployment posture — who hosts, who accesses, governance model, oversight
10. Risk assessment — per Front\\ ier AI Safety Commitments 5 risk domains (cyberbio · cyberchem · CBRN · autonomy · harm)
11. Mitigations — technical + organisational measures
12. Monitoring — post-deployment monitoring plan, incident response, advisory feed
13. Accountability — accountable named executives, board oversight, audit cadence
14. Updates + revisions — version history, planned revisions, regression testing protocol

4 · Red Team Report (12 named attack scenarios)

Per DEFONEOS Red Team Rubric, the 12 named attack scenarios are:

Red Team Report includes: attack method · success rate · severity if successful · mitigation status · residual risk · remediation owner. SIGIL-anchored at submission.

5 · Bias / Fairness Audit (the 12 protected characteristics)

Per EU AI Act Art-10 + UK Equality Act 2010, the audit covers 12 protected characteristics:

Audit conducted by independent named test lab (one of 5 partners in DEFONEOS Red Team network). Audit refreshed annually + on any material model change.

6 · Model Card (DEFONEOS sovereign weight variants)

DEFONEOS submits 1 Model Card per sovereign weight variant. Current variants:

• SOV3-small — 9B params · general-purpose sovereign LLM · Bronze/Silver/Gold tier default
• SOV3-large — 30B-A3B (MoE) · Platinum/Sovereign tier default · heavy decision support
• Liquid-KAN-MoE — 13B active · research-grade buyers · interpretability demos
• Sovereign-Defence-secured — 9B · air-gap + escrow-bound · Sovereign-Defence SEAL buyers

Future variants (per capability roadmap): SOV3-small v2 · SOV3-large v2 · Liquid-KAN-MoE v2 · AUKUS-tuned base · Sovereign-Defence-secured v2.

7 · The 5 named cross-walks (regimes the pack satisfies)

8 · Pre-submission BFT vote workflow

1. Artefacts assembled (System Card + Red Team + Bias Audit + Model Cards) — 5 BD DEFONEOS turnaround from complete draft.
2. 33-agent BFT council pre-vote opened at csoai.org/bft/aisi/{proposal_id} — 5 BD open comment period.
3. Council vote: ≥23/33 quorum required; ≥1 independent General must vote approve; ≥1 Sovereign General must vote approve.
4. If approved: SIGIL-anchored submission receipt attached to pack.
5. If rejected: named amendments, resubmit within 5 BD.
6. Approved pack submitted to AISI via aisi.gov.uk/submit with DEFONEOS UK Ltd UK 16939677 entity.
7. AISI reviews in 15 BD; DEFONEOS responds to inquiries within 5 BD each.

9 · Submission portal + format

• Portal: aisi.gov.uk/submit
• Format: ZIP archive (≤100 MB) with manifest.json (SHA-256 per file)
• Encryption: AES-256-GCM (DEFONEOS public key on portal · AISI public key in manifest)
• Signature: DEFONEOS Ed25519 SIGIL chain · AISI counter-signs on receipt
• Retention: 10 years (AISI policy) · DEFONEOS retains local copy in escrow (NCC Group)

10 · Post-submission outcomes (the 4 paths)

1. Accepted, Tier 1 complete — DEFONEOS receives acceptance letter + public listing in AISI System Card directory. Invitation to Pre-Deployment Tier 2 evaluation.
2. Accepted with notes — AISI requests minor clarifications; DEFONEOS responds within 5 BD; acceptance follows.
3. Additional review needed — AISI requests deeper testing; DEFONEOS engages one of 5 named independent test labs; resubmit within 30 BD.
4. Declined — system fails one of the 14 System Card sections or 12 Red Team scenarios; DEFONEOS may revise + resubmit after 90 days.

11 · Cross-walk to other DEFONEOS artefacts

12 · The 5 Anti-Patterns (AISI Submission Disasters We Refuse)

1. No "submit without BFT vote." ≥23/33 mandatory pre-submission.
2. No "vague System Card sections." All 14 sections complete or pack incomplete.
3. No "self-attested Red Team." Independent test lab required.
4. No "stale Bias Audit." Annual + on material model change.
5. No "withhold adverse findings." All findings reported, even partial mitigations.

13 · Submission timeline (DEFONEOS internal)

14 · Next Steps

1. Verify this pack hash: curl https://csoai.org/defoneos-mod-uk-aisi-pre-deployment-evaluation-pack.html | shasum -a 256
2. Download the System Card draft: defoneos-system-card.html
3. Download the Red Team Rubric: defoneos-mod-red-team-rubric.html
4. Download the Bias Audit template: defoneos-mod-fairness-impact-assessment.html
5. Request DEFONEOS engagement: commercial@csoai.org for buyer-side AISI pack preparation.
6. Subscribe to AISI submission advisories: csoai.org/advisories/subscribe