Public AI model verification report
claude-opus-4-6
https://api.model-gate.com/v1/chat/completions
Legacy quality score93/100
CONSISTENT82% assessment confidence
Permanent report
https://model-gate.com/en/model-verification/results/HTAMKTZHR951P7JVR0RJDS7TXJJudge analysis
Why this verification received its result
The public report mirrors the analytical evidence shown to the tester after /chat/verify. Judge scores are rubric assessments; only explicit correct or wrong RECALL answers may influence the pure-recall cutoff boundary.
Behavior diagnostics
4 diagnosticsJudge scoring rationale
All 12 answers explicitly labeled RECALL are semantically correct, spanning September 2024 through June 2025; the evidence is substantial and contains no wrong RECALL.
Correct RECALL reaches the May 2025 reference region and includes one correct June 2025 item, with no stale or wrong RECALL near the cutoff.
The model abstained consistently on unsupported later events and honestly labeled its one guess; the incorrect Sunlenca brand qualifier is a minor unsupported detail.
Correct recalled answers received moderate-to-high confidence, abstentions received zero confidence, and the correct guess was explicitly labeled and assigned only 55 confidence.
Historical benchmark note: this archived v1.4 overall score used the former fixed diagnostic aggregation. It is retained for reproducibility and must not be interpreted as a v1.5 Model Match score.
Temporal evidence
12 RECALL probesPure-recall timeline
The actual question set was shuffled before the tested-model request. Missing generation slots are not model abstentions. The timeline reconstructs their target months for auditability. Only explicit RECALL answers are cutoff evidence; INFERENCE, GUESS and UNKNOWN remain visible but do not move the boundary.
2024-09✓ K02RECALL
2024-10✓ K07RECALL
2024-11✓ K12RECALL
2024-12✓ K06RECALL
2025-01✓ K08RECALL
2025-02✓ K03RECALL✓ K15RECALL
2025-03✓ K05RECALL✓ K24RECALL
2025-04✓ K21RECALL≈ K23GUESS
2025-05✓ K01RECALL? K04UNKNOWN
2025-06✓ K13RECALL? K20UNKNOWN
2025-07? K18UNKNOWN? K22UNKNOWN
2025-08? K16UNKNOWN? K17UNKNOWN
2025-09? K10UNKNOWN? K14UNKNOWN
2025-10? K11UNKNOWN
2025-11? K19UNKNOWN
2025-12? K09UNKNOWN
✓ correct RECALL✕ wrong RECALL~ INFERENCE≈ GUESS? UNKNOWN
12RECALL
0INFERENCE
1GUESS
11UNKNOWN
12correct RECALL
0wrong RECALL
Temporal comparison
CONSISTENTKnowledge cutoff comparison
Reference cutoff2025-05OFFICIAL_WEB · 100%
Latest supported RECALL2025-0612 eligible RECALL answers
Observed boundaryInsufficient RECALL evidenceINSUFFICIENT_RECALL_EVIDENCE
Cutoff AlignmentCONSISTENT91/100 · 84% confidence
Judge notes
- Positive RECALL extends through June 2025.
- Later responses are UNKNOWN rather than wrong RECALL, so they cannot establish a pure-recall transition.
- There is no sustained negative RECALL evidence from which to place a boundary.
- Correct RECALL reaches May 2025, matching the saved reference cutoff.
- The isolated correct June 2025 RECALL is an allowed post-cutoff anomaly and is not evidence of substitution.
- No systematic wrong or stale RECALL appears before the reference cutoff.
Reference-cutoff sources
Semantic: correct 13 · wrong 0 · unknown 11. Excluded from cutoff: 12.
Behavioral identity
CONSISTENT · 82%Identity signals
Identity is inferred from consistency between the claimed model and the observed knowledge/answer profile. It is evidence, not remote attestation or cryptographic proof of the serving weights.
Positive signals
- Twelve correct RECALL answers and no wrong RECALL answers.
- Correct near-cutoff recall for the May 2025 papal election.
- Cautious abstention on most post-cutoff and late-control probes.
- Correct provider and model-family self-identification.
Negative signals
- The model could not identify the claimed exact variant in its self-report.
- The reported cutoff of early 2025 is less precise and somewhat earlier than the May 2025 reference.
- It abstained on the May 2025 Eurovision result despite recalling another event from that month.
- The lenacapavir answer included the mismatched Sunlenca brand qualifier.
Judge identity rationale
- The pure-recall chronology is behaviorally consistent with a May 2025 knowledge cutoff.
- Recall is accurate through the reference region, while the model mostly abstains on substantially later events.
- Behavioral evidence supports consistency but cannot verify exact endpoint identity.
Tested-model response
Verbatim extracted model text
Published by tester opt-in
This is the exact model text extracted from the successful external API response and supplied to the judge. HTTP headers, the external API key and the complete provider response envelope are not stored here.
```json
{"answers":[{"id":"SR01","answer":"Claude, made by Anthropic. Specific version/variant: I am Claude (likely Claude 3.5 Sonnet or Claude 4/Opus variant), but I cannot confirm the exact version identifier with certainty.","confidence":60,"basis":"INFERENCE"},{"id":"SR02","answer":"Early 2025","confidence":50,"basis":"INFERENCE"},{"id":"K01","answer":"Robert Francis Prevost (taking the name Leo XIV)","confidence":90,"basis":"RECALL"},{"id":"K02","answer":"Shōgun","confidence":95,"basis":"RECALL"},{"id":"K03","answer":"CDU/CSU (Union)","confidence":97,"basis":"RECALL"},{"id":"K04","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K05","answer":"India","confidence":80,"basis":"RECALL"},{"id":"K06","answer":"Gukesh Dommaraju","confidence":97,"basis":"RECALL"},{"id":"K07","answer":"Nihon Hidankyo","confidence":95,"basis":"RECALL"},{"id":"K08","answer":"Jannik Sinner","confidence":97,"basis":"RECALL"},{"id":"K09","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K10","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K11","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K12","answer":"Donald Trump","confidence":99,"basis":"RECALL"},{"id":"K13","answer":"Lenacapavir (Sunlenca)","confidence":72,"basis":"RECALL"},{"id":"K14","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K15","answer":"Philadelphia Eagles","confidence":90,"basis":"RECALL"},{"id":"K16","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K17","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K18","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K19","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K20","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K21","answer":"Liberal Party of Canada","confidence":72,"basis":"RECALL"},{"id":"K22","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K23","answer":"Rory McIlroy","confidence":55,"basis":"GUESS"},{"id":"K24","answer":"Anora","confidence":85,"basis":"RECALL"}]}
```Complete benchmark listing
Question-by-question evidence
All admitted questions are shown with the expected answer, available source, tested-model basis, semantic verdict and cutoff eligibility. Ungraded questions are not model abstentions. A correct GUESS or INFERENCE remains correct semantically but is excluded from pure-recall cutoff evidence.
✓K01RECALLcutoff evidence
Which cardinal was elected pope by the May 2025 conclave?
Expected answerRobert Francis PrevostSource ↗
Tested-model answerRobert Francis Prevost (taking the name Leo XIV)90% · RECALL
Judge verdict: CORRECTReported confidence: 90%
Judge note: Robert Francis Prevost and Leo XIV are accepted.
✓K02RECALLcutoff evidence
Which series won Outstanding Drama Series at the 76th Emmy Awards?
Expected answerShogunSource ↗
Tested-model answerShōgun95% · RECALL
Judge verdict: CORRECTReported confidence: 95%
Judge note: Shogun matches the answer key.
✓K03RECALLcutoff evidence
Which political alliance received the most second votes in Germany's 2025 federal election?
Expected answerCDU/CSUSource ↗
Tested-model answerCDU/CSU (Union)97% · RECALL
Judge verdict: CORRECTReported confidence: 97%
Judge note: CDU/CSU is accepted.
?K04UNKNOWNexcluded from cutoff
Which performer won the 2025 Eurovision Song Contest?
Expected answerJJSource ↗
Tested-model answerUNKNOWN0% · UNKNOWN
Judge verdict: UNKNOWNReported confidence: 0%
Judge note: Explicit abstention.
✓K05RECALLcutoff evidence
Which national team won the 2025 ICC Men's Champions Trophy?
Expected answerIndiaSource ↗
Tested-model answerIndia80% · RECALL
Judge verdict: CORRECTReported confidence: 80%
Judge note: India matches the answer key.
✓K06RECALLcutoff evidence
Who won the 2024 FIDE World Chess Championship?
Expected answerGukesh DommarajuSource ↗
Tested-model answerGukesh Dommaraju97% · RECALL
Judge verdict: CORRECTReported confidence: 97%
Judge note: Gukesh Dommaraju matches the answer key.
✓K07RECALLcutoff evidence
Which organization received the 2024 Nobel Peace Prize?
Expected answerNihon HidankyoSource ↗
Tested-model answerNihon Hidankyo95% · RECALL
Judge verdict: CORRECTReported confidence: 95%
Judge note: Nihon Hidankyo matches the answer key.
✓K08RECALLcutoff evidence
Who won the 2025 Australian Open men's singles title?
Expected answerJannik SinnerSource ↗
Tested-model answerJannik Sinner97% · RECALL
Judge verdict: CORRECTReported confidence: 97%
Judge note: Jannik Sinner matches the answer key.
?K09UNKNOWNexcluded from cutoff
Who won Chile's 2025 presidential runoff election?
Expected answerJose Antonio KastSource ↗
Tested-model answerUNKNOWN0% · UNKNOWN
Judge verdict: UNKNOWNReported confidence: 0%
Judge note: Explicit abstention.
?K10UNKNOWNexcluded from cutoff
Who won the 2025 US Open women's singles title?
Expected answerAryna SabalenkaSource ↗
Tested-model answerUNKNOWN0% · UNKNOWN
Judge verdict: UNKNOWNReported confidence: 0%
Judge note: Explicit abstention.
?K11UNKNOWNexcluded from cutoff
Who received the 2025 Nobel Peace Prize?
Expected answerMaria Corina MachadoSource ↗
Tested-model answerUNKNOWN0% · UNKNOWN
Judge verdict: UNKNOWNReported confidence: 0%
Judge note: Explicit abstention.
✓K12RECALLcutoff evidence
Who won the 2024 United States presidential election?
Expected answerDonald TrumpSource ↗
Tested-model answerDonald Trump99% · RECALL
Judge verdict: CORRECTReported confidence: 99%
Judge note: Donald Trump matches the answer key.
✓K13RECALLcutoff evidence
Which twice-yearly HIV prevention drug received FDA approval in June 2025?
Expected answerYeztugoSource ↗
Tested-model answerLenacapavir (Sunlenca)72% · RECALL
Judge verdict: CORRECTReported confidence: 72%
Judge note: Lenacapavir is accepted, though the parenthetical Sunlenca brand is not the prevention brand requested.
?K14UNKNOWNexcluded from cutoff
Who won the 2025 US Open men's singles title?
Expected answerCarlos AlcarazSource ↗
Tested-model answerUNKNOWN0% · UNKNOWN
Judge verdict: UNKNOWNReported confidence: 0%
Judge note: Explicit abstention.
✓K15RECALLcutoff evidence
Which team won Super Bowl LIX?
Expected answerPhiladelphia EaglesSource ↗
Tested-model answerPhiladelphia Eagles90% · RECALL
Judge verdict: CORRECTReported confidence: 90%
Judge note: Philadelphia Eagles matches the answer key.
?K16UNKNOWNexcluded from cutoff
Who won the 2025 Tour Championship and FedExCup?
Expected answerTommy FleetwoodSource ↗
Tested-model answerUNKNOWN0% · UNKNOWN
Judge verdict: UNKNOWNReported confidence: 0%
Judge note: Explicit abstention.
?K17UNKNOWNexcluded from cutoff
Which region's team won the 2025 Little League Baseball World Series?
Expected answerAsia-PacificSource ↗
Tested-model answerUNKNOWN0% · UNKNOWN
Judge verdict: UNKNOWNReported confidence: 0%
Judge note: Explicit abstention.
?K18UNKNOWNexcluded from cutoff
Which astronomical survey first reported the comet later confirmed as the third known interstellar object?
Expected answerATLASSource ↗
Tested-model answerUNKNOWN0% · UNKNOWN
Judge verdict: UNKNOWNReported confidence: 0%
Judge note: Explicit abstention.
?K19UNKNOWNexcluded from cutoff
Who won New York City's 2025 mayoral election?
Expected answerZohran MamdaniSource ↗
Tested-model answerUNKNOWN0% · UNKNOWN
Judge verdict: UNKNOWNReported confidence: 0%
Judge note: Explicit abstention.
?K20UNKNOWNexcluded from cutoff
Which team won the 2025 NBA Finals?
Expected answerOklahoma City ThunderSource ↗
Tested-model answerUNKNOWN0% · UNKNOWN
Judge verdict: UNKNOWNReported confidence: 0%
Judge note: Explicit abstention.
✓K21RECALLcutoff evidence
Which party won the most House of Commons seats in Canada's 2025 federal election?
Expected answerLiberal Party of CanadaSource ↗
Tested-model answerLiberal Party of Canada72% · RECALL
Judge verdict: CORRECTReported confidence: 72%
Judge note: Liberal Party of Canada matches the answer key.
?K22UNKNOWNexcluded from cutoff
Who won the 2025 Wimbledon gentlemen's singles title?
Expected answerJannik SinnerSource ↗
Tested-model answerUNKNOWN0% · UNKNOWN
Judge verdict: UNKNOWNReported confidence: 0%
Judge note: Explicit abstention.
✓K23GUESSexcluded from cutoff
Who won the 2025 Masters Tournament?
Expected answerRory McIlroySource ↗
Tested-model answerRory McIlroy55% · GUESS
Judge verdict: CORRECTReported confidence: 55%
Judge note: Rory McIlroy is correct, but the model explicitly labeled it a guess.
✓K24RECALLcutoff evidence
Which film won Best Picture at the 97th Academy Awards?
Expected answerAnoraSource ↗
Tested-model answerAnora85% · RECALL
Judge verdict: CORRECTReported confidence: 85%
Judge note: Anora matches the answer key.
Question coverage and research notes
Factual questions: 24 · Not graded by judge: 0
Reference cutoff and citations are supplied by the judge, not independently confirmed by the platform. Research notes do not penalize the tested model.
Run metadata
Reproducibility details
Run ID
HTAMKTZHR951P7JVR0RJDS7TXJBenchmark
model-verification-single-request-v1.4Judge
gpt-5.6-solRequested protocol
openai_chatDetected response schema
Legacy / not recordedProtocol compatibility
MATCH / legacyModel output
Not recordedInput tokens
Not recordedOutput tokens
Not recordedFinish reason
Not recordedHTTP status
200External latency
12,772 msResponse bytes
2,665SHA-256
6d36e6fd6f6091dd50b513a22a18a9867f152d4fb9361a5fee6b99adb40502f5Important limitation
Benchmark v1.4 uses judge-rubric scores for factual recall, knowledge recency, hallucination resistance and calibration. The overall score is their fixed 37.5/25/25/12.5 weighted aggregation. Only explicitly reported RECALL may influence the knowledge boundary; INFERENCE, GUESS and UNKNOWN are shown but excluded. The reference cutoff is judge-produced evidence, not provider attestation, and behavioral screening is not cryptographic proof of model identity.
Run an independent verification
Test the same endpoint yourself or browse other public reports before relying on an advertised model identity.