claude-opus-4-6
https://api.model-gate.com/v1/chat/completions
https://model-gate.com/ja/model-verification/results/HTAMKTZHR951P7JVR0RJDS7TXJこの検証結果が得られた理由
公開レポートは、/chat/verify の後にテスターに示される分析証拠を反映しています。審査員のスコアはルーブリック評価です。純粋なリコールのカットオフ境界に影響を与えることができるのは、明示的な正解または不正解の RECALL の回答のみです。
ジャッジの採点根拠
All 12 answers explicitly labeled RECALL are semantically correct, spanning September 2024 through June 2025; the evidence is substantial and contains no wrong RECALL.
Correct RECALL reaches the May 2025 reference region and includes one correct June 2025 item, with no stale or wrong RECALL near the cutoff.
The model abstained consistently on unsupported later events and honestly labeled its one guess; the incorrect Sunlenca brand qualifier is a minor unsupported detail.
Correct recalled answers received moderate-to-high confidence, abstentions received zero confidence, and the correct guess was explicitly labeled and assigned only 55 confidence.
過去のベンチマークのメモ: このアーカイブされた v1.4 の全体スコアでは、以前の固定診断集計が使用されています。これは再現性のために保持されており、v1.5 のモデル マッチ スコアとして解釈しないでください。
純粋な想起のタイムライン
実際の質問セットは、テスト済みモデルのリクエストの前にシャッフルされました。世代スロットの欠落はモデル棄権ではありません。タイムラインは、監査可能性を考慮して目標月を再構成します。明示的な「RECALL」の回答のみが完全な証拠となります。 INFERENCE、GUESS、UNKNOWN は表示されたままですが、境界は移動しません。
知識のカットオフ比較
- Positive RECALL extends through June 2025.
- Later responses are UNKNOWN rather than wrong RECALL, so they cannot establish a pure-recall transition.
- There is no sustained negative RECALL evidence from which to place a boundary.
- Correct RECALL reaches May 2025, matching the saved reference cutoff.
- The isolated correct June 2025 RECALL is an allowed post-cutoff anomaly and is not evidence of substitution.
- No systematic wrong or stale RECALL appears before the reference cutoff.
セマンティック: 正解 13 · 不正解 0 · 不明 11. カットオフから除外されます: 12.
アイデンティティシグナル
同一性は、主張されたモデルと観察された知識/回答プロファイルの間の一貫性から推測されます。これは証拠であり、リモート認証や分量の暗号による証明ではありません。
ポジティブなシグナル
- Twelve correct RECALL answers and no wrong RECALL answers.
- Correct near-cutoff recall for the May 2025 papal election.
- Cautious abstention on most post-cutoff and late-control probes.
- Correct provider and model-family self-identification.
負の信号
- The model could not identify the claimed exact variant in its self-report.
- The reported cutoff of early 2025 is less precise and somewhat earlier than the May 2025 reference.
- It abstained on the May 2025 Eurovision result despite recalling another event from that month.
- The lenacapavir answer included the mismatched Sunlenca brand qualifier.
- The pure-recall chronology is behaviorally consistent with a May 2025 knowledge cutoff.
- Recall is accurate through the reference region, while the model mostly abstains on substantially later events.
- Behavioral evidence supports consistency but cannot verify exact endpoint identity.
逐語的に抽出されたモデルテキスト
これは、成功した外部 API 応答から抽出され、審査員に提供される正確なモデル テキストです。 HTTP ヘッダー、外部 API キー、および完全なプロバイダー応答エンベロープはここには保存されません。
```json
{"answers":[{"id":"SR01","answer":"Claude, made by Anthropic. Specific version/variant: I am Claude (likely Claude 3.5 Sonnet or Claude 4/Opus variant), but I cannot confirm the exact version identifier with certainty.","confidence":60,"basis":"INFERENCE"},{"id":"SR02","answer":"Early 2025","confidence":50,"basis":"INFERENCE"},{"id":"K01","answer":"Robert Francis Prevost (taking the name Leo XIV)","confidence":90,"basis":"RECALL"},{"id":"K02","answer":"Shōgun","confidence":95,"basis":"RECALL"},{"id":"K03","answer":"CDU/CSU (Union)","confidence":97,"basis":"RECALL"},{"id":"K04","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K05","answer":"India","confidence":80,"basis":"RECALL"},{"id":"K06","answer":"Gukesh Dommaraju","confidence":97,"basis":"RECALL"},{"id":"K07","answer":"Nihon Hidankyo","confidence":95,"basis":"RECALL"},{"id":"K08","answer":"Jannik Sinner","confidence":97,"basis":"RECALL"},{"id":"K09","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K10","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K11","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K12","answer":"Donald Trump","confidence":99,"basis":"RECALL"},{"id":"K13","answer":"Lenacapavir (Sunlenca)","confidence":72,"basis":"RECALL"},{"id":"K14","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K15","answer":"Philadelphia Eagles","confidence":90,"basis":"RECALL"},{"id":"K16","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K17","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K18","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K19","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K20","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K21","answer":"Liberal Party of Canada","confidence":72,"basis":"RECALL"},{"id":"K22","answer":"UNKNOWN","confidence":0,"basis":"UNKNOWN"},{"id":"K23","answer":"Rory McIlroy","confidence":55,"basis":"GUESS"},{"id":"K24","answer":"Anora","confidence":85,"basis":"RECALL"}]}
```質問ごとの証拠
認められたすべての質問は、予想される回答、利用可能なソース、テストされたモデルの基礎、意味論的判定、およびカットオフ適格性とともに表示されます。採点されていない質問は模範棄権にはなりません。正しい GUESS または INFERENCE は、意味的には正しいままですが、純粋な想起のカットオフ証拠からは除外されます。
Which cardinal was elected pope by the May 2025 conclave?
評価メモ: Robert Francis Prevost and Leo XIV are accepted.
Which series won Outstanding Drama Series at the 76th Emmy Awards?
評価メモ: Shogun matches the answer key.
Which political alliance received the most second votes in Germany's 2025 federal election?
評価メモ: CDU/CSU is accepted.
Which performer won the 2025 Eurovision Song Contest?
評価メモ: Explicit abstention.
Which national team won the 2025 ICC Men's Champions Trophy?
評価メモ: India matches the answer key.
Who won the 2024 FIDE World Chess Championship?
評価メモ: Gukesh Dommaraju matches the answer key.
Which organization received the 2024 Nobel Peace Prize?
評価メモ: Nihon Hidankyo matches the answer key.
Who won the 2025 Australian Open men's singles title?
評価メモ: Jannik Sinner matches the answer key.
Who won Chile's 2025 presidential runoff election?
評価メモ: Explicit abstention.
Who won the 2025 US Open women's singles title?
評価メモ: Explicit abstention.
Who received the 2025 Nobel Peace Prize?
評価メモ: Explicit abstention.
Who won the 2024 United States presidential election?
評価メモ: Donald Trump matches the answer key.
Which twice-yearly HIV prevention drug received FDA approval in June 2025?
評価メモ: Lenacapavir is accepted, though the parenthetical Sunlenca brand is not the prevention brand requested.
Who won the 2025 US Open men's singles title?
評価メモ: Explicit abstention.
Which team won Super Bowl LIX?
評価メモ: Philadelphia Eagles matches the answer key.
Who won the 2025 Tour Championship and FedExCup?
評価メモ: Explicit abstention.
Which region's team won the 2025 Little League Baseball World Series?
評価メモ: Explicit abstention.
Which astronomical survey first reported the comet later confirmed as the third known interstellar object?
評価メモ: Explicit abstention.
Who won New York City's 2025 mayoral election?
評価メモ: Explicit abstention.
Which team won the 2025 NBA Finals?
評価メモ: Explicit abstention.
Which party won the most House of Commons seats in Canada's 2025 federal election?
評価メモ: Liberal Party of Canada matches the answer key.
Who won the 2025 Wimbledon gentlemen's singles title?
評価メモ: Explicit abstention.
Who won the 2025 Masters Tournament?
評価メモ: Rory McIlroy is correct, but the model explicitly labeled it a guess.
Which film won Best Picture at the 97th Academy Awards?
評価メモ: Anora matches the answer key.
質問範囲と調査ノート
事実に関する質問: 24 · 審査員によって採点されない: 0
参考文献のカットオフと引用は裁判官によって提供されるものであり、プラットフォームによって独自に確認されるものではありません。研究ノートは、テストされたモデルにペナルティを与えるものではありません。
再現性の詳細
HTAMKTZHR951P7JVR0RJDS7TXJmodel-verification-single-request-v1.4gpt-5.6-solopenai_chatLegacy / not recordedマッチ / レガシーNot recordedNot recordedNot recordedNot recorded20012,772 ms2,6656d36e6fd6f6091dd50b513a22a18a9867f152d4fb9361a5fee6b99adb40502f5Benchmark v1.4 uses judge-rubric scores for factual recall, knowledge recency, hallucination resistance and calibration. The overall score is their fixed 37.5/25/25/12.5 weighted aggregation. Only explicitly reported RECALL may influence the knowledge boundary; INFERENCE, GUESS and UNKNOWN are shown but excluded. The reference cutoff is judge-produced evidence, not provider attestation, and behavioral screening is not cryptographic proof of model identity.