docs

Benchmark methodology

The public benchmark methodology is intentionally partial: readers can see what the score is meant to represent without getting access to protected prompts, grading rules, or answer keys.

← Back to docs

In practice, that means publishing category-level outcomes and leaderboard positions while keeping the exact evaluation assets private enough to preserve the usefulness of future comparisons.

This tradeoff is necessary for any benchmark that also supports fraud or mismatch detection. Once the full protected asset set is exposed, providers can optimize against the test rather than against genuine model behavior.

You should treat methodology pages as a guide to interpretation: what is measured, what is not measured, and why a benchmark score must be combined with verification evidence.