Leaklens

← All entries

Six more coding benchmarks checked by rule, on samples

function ensurePrecision(value, step) {
mui__material-ui-16252 · mui/material-ui · fix commit 16 June 2019
SWE-PolyBench: all 5 lines the gold patch adds are already in the issue text.
  • Tasks whose issue text already contains lines the gold patch adds (Tier C, by rule, no model read): SWE-bench Pro 0 of 100 sampled (of 731 public tasks), SWE-rebench 0 of 100 (of 6,542), SWE-Gym 1 of 100 (of 2,438), SWE-PolyBench 4 of 100 (of 2,110), Multi-SWE-bench 1 of its 300-task flash set. See the table →
  • OmniCode, all 494 tasks in its three main files: 33 flagged, of which 29 are SWE-bench Verified tasks it carries and 4 are its own. Open the OmniCode page →
  • Each sample is a reproducible random draw. Tier A (the fix stated in words, read by a model) was not run on these samples; run it on any of them with leaklens check --judge. Read the CLI docs →

From an agent: the same checks over MCP, read-only, no key. Connect it

Check your own benchmark’s tasks

This is the check Leaklens runs on every Verified and Lite task. Upload your benchmark and get the same per-task report, with the evidence quoted. Sign in with an email, no card.

Run it on your benchmark, free See plans

Older: 10 November 2026