Check your own benchmark, task by task
Lookups, search and the CSV for all 500 SWE-bench Verified tasks are free. For your own benchmark, pick how you want to buy: every task checked for a stated fix, patch overlap and fix-commit date, with the evidence quoted.
Pro
€49 / month
For teams that check a benchmark every release.
- 3,000 task checks a month
- 10 API keys
- CI templates, so a new release is checked on every build
No payment now.
Audit pack
€29 one audit
One benchmark, up to 500 tasks, one report.
- A per-task leakage report with quoted evidence
- Fix-commit dates against the model cutoff you choose
- Export to CSV and a public page you can cite
No payment now.
Running a lab eval team or checking many benchmarks at once? Tell us what you need and bulk checks are priced per check.
Want to see the check first? Browse all 500 Verified tasks or read how often the flags hold up.