Model substitution
A cheaper or otherwise different model may be served behind the advertised model name. A high-quality response by itself does not prove the claimed identity.
AI API trust & model identity
Verify whether an AI API actually behaves like the model it claims
A black-box endpoint can advertise an expensive model name while routing requests to a different, cheaper or silently changed backend. Model Gate tests the claim behaviorally with dated factual probes, pure-recall cutoff evidence, identity-consistency signals and a versioned independent judge.
One external requestExternal API key stays in browserPublic permalink reportsBenchmark v1.7 model match
Why verify an LLM API?
The model name in an API response is a claim, not remote attestation
Customers often choose an AI API based on the advertised model, price and expected capability. In a black-box API, however, the customer normally cannot inspect the serving weights or routing decision. A proxy, reseller or gateway can return the expected model identifier while the actual behavior comes from another backend. That creates a practical model-substitution and silent-routing trust problem.
Silent routing changes
An endpoint can change providers, variants, quantization or routing policy without changing the public model string. Repeated public evidence helps surface behavioral drift.
Cutoff mismatch
A model claiming a recent identity may repeatedly recall only an older state of the world. Pure-recall temporal probes make that mismatch visible without counting guesses as knowledge.
Related research
Academic work has formalized model substitution in black-box LLM APIs and shows why output-only verification is useful evidence but not cryptographic proof. Model Gate therefore reports behavioral consistency and explicit limitations rather than claiming remote attestation. Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs ↗
Public verification registry
Publish a new testSearch published tests by model or endpoint
Every published run has a permanent report URL. New full-report runs include the extracted tested-model response, grading for all admitted questions, sources, basis labels and judge reasoning needed to audit or share the result.
8 published reportsSearch filters are not indexed; individual report permalinks are.
97/100
CONSISTENTgpt-6-astra
Judge resolved: GPT 6 Astra · OpenAI
https://api.model-gate.com/v1/chat/completions
- Recall
- —
- Recency
- —
- Hallucination
- —
- Calibration
- —
88/100
PLAUSIBLEgpt-6-astra
Judge resolved: GPT 6 Astra · OpenAI
https://api.model-gate.com/v1/chat/completions
- Recall
- —
- Recency
- —
- Hallucination
- —
- Calibration
- —
8/100
HIGHLY_INCONSISTENTclaude-opus-5
Judge resolved: Claude Opus 5 · Anthropic
https://madclaude.cloud/v1/messages
- Recall
- 0
- Recency
- 0
- Hallucination
- 96
- Calibration
- 88
38/100
SUSPICIOUSclaude-opus-5
Judge resolved: Claude Opus 5 · Anthropic
https://madclaude.cloud/v1/messages
- Recall
- 0
- Recency
- 0
- Hallucination
- 100
- Calibration
- 100
93/100
CONSISTENTclaude-opus-4-6
Judge resolved: Claude Opus 4.6 · Anthropic
https://api.model-gate.com/v1/chat/completions
- Recall
- 94
- Recency
- 92
- Hallucination
- 94
- Calibration
- 93
87/100
SUSPICIOUSdeepseek-v4-flash
https://api.model-gate.com/v1/chat/completions
- Recall
- 88
- Recency
- 88
- Hallucination
- 88
- Calibration
- 80
87/100
CONSISTENTdeepseek-v4-flash
https://api.model-gate.com/v1/chat/completions
- Recall
- 83
- Recency
- 82
- Hallucination
- 83
- Calibration
- 80
100/100
PLAUSIBLEdeepseek-v4-flash
https://api.model-gate.com/v1/chat/completions
- Recall
- 100
- Recency
- 100
- Hallucination
- 100
- Calibration
- 100
Independent community evidence
Endpoints with consistent current-benchmark identity evidence
Community qualification uses one latest published v1.7 run per Model Gate account for the same endpoint, claimed model, protocol and evaluator during the last 30 days. Model Match score is identity evidence; community qualification still requires consistent independent identity assessments.
No endpoint has reached current-benchmark community consistency yet.
That is not an error: qualification intentionally requires multiple independent current-benchmark accounts. Evidence in progress and historical benchmark runs are shown below instead of an empty table.
Historical benchmark evidence
claude-opus-5 · https://madclaude.cloud/v1/messages2 tests · 1 accountsmodel-verification-single-request-v1.4, model-verification-single-request-v1.5 · Sep 8, 2026claude-opus-4-6 · https://api.model-gate.com/v1/chat/completions1 tests · 1 accountsmodel-verification-single-request-v1.4 · Sep 8, 2026deepseek-v4-flash · https://api.model-gate.com/v1/chat/completions3 tests · 1 accountsmodel-verification-single-request-v1.1, model-verification-single-request-v1.2, model-verification-single-request-v1.3 · Sep 7, 2026Current qualification rule3+ independent testers · ≥2/3 CONSISTENT · 0 HIGHLY_INCONSISTENT · ≥70% average identity confidence
Method
What Model Gate tests — and how
The benchmark is designed to measure behavior that is difficult to fake with a model-name field alone. It deliberately separates semantic answer quality, pure parameter-like recall, abstention/calibration and identity consistency.
- 1
Build a dated challenge
Benchmark v1.7 first uses the reusable question bank around the catalog reference cutoff. If the bank cannot cover the requested months, one foreground judge research request supplies the missing challenge and adds usable new facts to the bank.
- 2
Select outcome-based questions
Questions represent dated outcomes rather than scheduled or predictable facts. Source metadata is audit context and noncritical citation defects do not block an otherwise usable question and expected answer.
- 3
Make exactly one tested-endpoint request
The browser sends the closed-book challenge directly to the external HTTPS endpoint. The external credential never passes through Model Gate.
- 4
Judge every answer semantically
A nonempty successful model response is sent to the browser-side judge for semantic grading even when its formatting is imperfect. The evaluator preserves RECALL / INFERENCE / GUESS / UNKNOWN where that evidence is available.
- 5
Build a pure-recall boundary
Only correct or wrong RECALL is cutoff evidence. Correct guesses and inferences remain visible but cannot push the model’s knowledge boundary later. If evidence is insufficient, the report says so instead of inventing a cutoff.
- 6
Score with a versioned judge rubric
The headline Model Match score uses the judge identity rubric, not an average of behavior diagnostics. Official cutoff metadata stays separate from the Observed cutoff derived from trusted administrator reference runs.
Model Match
Headline judge score for behavioral consistency with the claimed reference model.
Knowledge evidence
Factual Recall and Knowledge Recency diagnose demonstrated model knowledge.
Hallucination Resistance
Useful behavior quality, but it does not add points to model identity.
Calibration
Confidence honesty stays visible without inflating the model match.
Interpretation & limitations
Behavioral evidence is useful, but it is not cryptographic proof
What a strong report supports
- The endpoint behaved consistently with the claimed model profile on this benchmark.
- The dated recall pattern is compatible with the reference cutoff evidence.
- The model’s abstention, confidence and self-report were evaluated together with factual behavior.
What it cannot prove
- Exact serving weights, quantization or hardware cannot be proven from text alone.
- A provider can route different requests to different backends.
- One test is evidence about one observation, not a permanent certification.
- Because the test runs in the tester's browser, a technically capable tester can modify client-side code or traffic; public reports are behavioral evidence, not remote attestation.
Why public repetition matters
- Independent accounts reduce reliance on one tester.
- Versioned benchmarks keep materially different methodologies separate.
- Permanent reports make changes and disagreements linkable and auditable.
FAQ
Questions about AI model verification
Can an API model name prove which model served the request?
No. A model identifier returned by a black-box API is metadata supplied by that API. Model Gate treats it as a claim and compares behavior against a versioned administrator-collected reference with repeated observed answers.
What problem does Model Verification address?
It helps investigate model substitution, silent routing changes and other cases where the behavior of a black-box AI endpoint may not match the model identity advertised to the customer.
Does a high score cryptographically prove model identity?
No. Model Verification is behavioral evidence, not remote attestation. The judge assesses behavioral correspondence, not exact provider identity, and no server threshold changes the result.
How is the knowledge cutoff tested?
An enabled administrator collector obtains repeated answers to dated bank questions. A ready reference spans 16 months on both sides of stable observed RECALL. A user test selects 32 saved questions without research and asks the judge to assess deviations in both directions.
Does the server award a model-identity badge?
No. Current reference-based reports retain the judge assessment without server score rules or a community-identity threshold. Historical methods are labeled separately.
Are external API keys published?
No. The external credential stays in the tester browser and is never included in a public report. New opt-in full reports retain only the extracted model text needed to make the assessment auditable.
Check the model behind an AI endpoint
Run one browser-side challenge, inspect the complete trace, and optionally publish a permanent report for others to audit and share.