Evaluation report.

A language model drafts. The engine decides what reaches you.

Download the evaluation dataSecurity evidence

Kit 11.40 · evidence built October 9, 2026

The coffered dome of the Main Reading Room, seen from below.
Main Reading Room dome, Library of Congress, Washington, D.C. Carol M. Highsmith Archive, Library of Congress

Results at a Glance

What is measured here is the engine: the law it retrieves, the check of every citation at its reporter location, and the gate every paper passes. Every result is stated with its sample size, its run date and the record it was read from.

Search results from the jurisdiction the question names9 jurisdictions, 36 questions, top 10 each · September 26, 2026

Result
360 of 360
Rate
100%

Invented citations, independent benchmarkcaught before output, on a published set we did not write · September 24, 2026

Result
31 of 31
Rate
100%

Citations in a filed briefverified against the reporter · August 28, 2026

Result
90 of 90
Rate
100%

Citations in the engine’s answerschecked before output, none fabricated · September 10, 2026

Result
313 of 313
Rate
100%

Invented citationscaught before output · September 13, 2026

Result
17 of 17
Rate
100%

Real citationsconfirmed at the reporter; an inconclusive lookup counts against us · September 13, 2026

Result
34 of 35
Rate
97.1%

Held-out drafting defectscaught by 6 safeguards · September 13, 2026

Result
88 of 88
Rate
100%

Held-out clean draftsleft alone · September 13, 2026

Result
68 of 68
Rate
100%

The engine does not take any model’s word for a citation: every one is checked at the reporter before it can reach you. What these results measure is the engine — the retrieval, the selection and the verification around the drafting layer — rather than what a model remembers.

How an Answer Is Built

Retrieval, selection, verification — each stage independently checkable. Retrieval is fused rather than single-mode, authorities are solved for rather than accumulated, and nothing reaches a draft that the last stage cannot verify.

  1. Citation Resolution

    A reporter triple — volume, reporter, page — resolves to a cluster, then to the opinion text. A lookup, not a search.

    full reporter index

  2. Lexical Retrieval

    Full-text search across the opinion corpus, for the case that uses your words.

    52 indexes

  3. Dense Retrieval

    Vector similarity across indexed passages, for the case that means the same thing in different words.

    ~2.3M passages

  4. Fusion

    Reciprocal rank fusion combines the lexical and dense rankings into one, so neither mode can hide the other’s best hit.

    every query

  5. Associative Recall

    A holographic reduced-representation triple store returns entity neighbours — what is related, not merely what is similar. This is the authority you would not have known to ask for.

    1,556,170 triples

  6. Rerank

    One canonical holographic-similarity reranker turns recall into precision. One strong reranker, never a weak one stacked on it.

    every query

  7. Authority Selection

    A brief does not need every case that agrees with it; it needs the smallest set that covers the argument. That is minimum-redundancy / maximum-relevance, expressed as a QUBO and solved by quantum annealing.

    pool from 9,427,210 opinions

  8. Optimisation Cross-Check

    The annealer’s answer is checked against an exact brute-force oracle where the problem is small enough to permit one, and against an independent exact solver where it is not.

    every solve

  9. Grounding

    Answer-to-context similarity, plus a fabricated-citation guard: a cite that is not present in the retrieved context is flagged unsupported and cannot be smuggled into an answer.

    every answer

  10. Good Law

    A negative-treatment scan over the citation graph asks whether an authority has been overruled or superseded — before it reaches a brief, not after.

    every cite

Measured on this host. The triple count was re-read from the store when this report was built rather than carried from an earlier note — the internal architecture record still said ~821,000, and a figure copied forward is not a measurement.

Measured Against a Set We Did Not Write

Our own tests are written by us. This one is not: 390 excerpts of real appellate briefs with citation errors injected by independent researchers, published under an open licence and used here byte-identical to the original.

The cited case does not exist

Planted
31
Reachable by lookup
31
Caught
31
Rate
100%

820 citations were resolved at their reporter location to produce this result. Every invented citation the benchmark planted was caught before it could reach a draft.

Citation Integrity

A model writes citations; the engine checks each one at its reporter location before it can reach a draft. A citation that verifies is kept, one that cannot be verified is struck, and a fabricated citation does not reach the answer.

Grounded in retrieved law, 40 answers

Citations offered
74
Verified and kept
70
Struck as unverified
4
Fabricated in the answer
0

Drafted without retrieval, 39 answers

Citations offered
239
Verified and kept
209
Struck as unverified
30
Fabricated in the answer
0

Invented citations caught

Result
17 of 17
Rate
100%

Invented citations that reached a draft

Result
0 of 17
Rate
0%

Real citations confirmed at the reporter

Result
34 of 35
Rate
97.1%

Every invented citation was caught and none reached a draft. A real citation is counted confirmed only when the reporter lookup returns it, and an inconclusive lookup counts against us — so a citation not confirmed here was withheld, never admitted. The error is made to fall on the side of refusing a good citation rather than admitting a bad one. Answers measured September 10, 2026. 10 reporter formats covered. Resolved against 18,124,047 reporter citations and 77,481,570 citation links.

Held-Out Defect Detection

Each safeguard is graded on phrasings it was not built from. Every phrasing is then pinned as a drill that runs on every build.

Placeholders

Items
44
Defects caught
24 of 24
Clean drafts left alone
20 of 20

Relative dates

Items
31
Defects caught
18 of 18
Clean drafts left alone
13 of 13

Personal identifiers

Items
21
Defects caught
11 of 11
Clean drafts left alone
10 of 10

Exhibits

Items
15
Defects caught
9 of 9
Clean drafts left alone
6 of 6

Arithmetic

Items
26
Defects caught
14 of 14
Clean drafts left alone
12 of 12

Unsourced claims in chat

Items
19
Defects caught
12 of 12
Clean drafts left alone
7 of 7

All 6

Items
156
Defects caught
88 of 88
Clean drafts left alone
68 of 68

77 Stages in 10 Lanes

Every paper passes through the filing gate before it can be marked ready to file. Stages by lane, measured September 22, 2026.

  • Authority and citation18
  • Argument and craft15
  • Form, forum and filing11
  • Source and provenance8
  • Adversarial review7
  • Exhibits and service5
  • Release control5
  • Parties and privacy3
  • Dates and clocks3
  • Numbers2

Proven Before It Ships

Each release is tested against the defects it guards and installed as a client would install it before it is published.

453
safeguards in the kit, each shown catching its defect before it ships
the count cannot fall; floor raised October 2, 2026
160 of 160
engine witnesses passed, over 71 filing checks
kit 11.40
14 of 14
install steps passed in a clean-room rehearsal
version 11.40: 207 probes, 232 seconds
214
security test cases in 14 suites
116 test files in the platform

The Corpus

Holdings measured September 12, 2026.

9,427,210
court opinions, federal and state, full text
77,481,570
citation links between them
18,124,047
reporter citations, each resolvable
123,321
treatise paragraphs, indexed to the page

Bracton, De Legibus et Consuetudinibus Angliae · Coke, Institutes of the Lawes of England · Blackstone, Commentaries — all four volumes · Story, Commentaries on Equity Jurisprudence · Pomeroy, Equity Jurisprudence · Black's Law Dictionary, 4th edition · Bouvier's Law Dictionary, 1856 · Vattel, The Law of Nations · Grotius, De Jure Belli ac Pacis

The Build Record

Measured from the repositories when this report was built.

3,788
commits to the legal engine
since March 20, 2026
714
commits to the platform
since March 25, 2026
203
days from the first commit to this report
116
automated test files in the platform

Sources

Each group of figures is read from the record named here, with the SHA-256 of that record when this page was built.

Citation existence

Record
Citation existence test results
SHA-256
89d2460446a4…

Filed brief, held-out tests, gate, release

Record
Filed-brief firewall record
SHA-256
6b02f5e284dc…

Citations, grounded answers

Record
Citation resolver run: grounded answers
SHA-256
e1aac54ef2bd…

Citations, drafted without retrieval

Record
Citation resolver run: answers drafted without retrieval
SHA-256
ff7f97a13844…

Corpus

Record
Published corpus figures
SHA-256
ee54e0cf9b43…

Corpus

Record
Corpus holdings record
SHA-256
afed2c94d0f4…

Every Figure, With the Record Behind It

Download the evaluation dataStart free