Six more coding benchmarks checked by rule, on samples
function ensurePrecision(value, step) {
SWE-PolyBench: all 5 lines the gold patch adds are already in the issue text.
- Tasks whose issue text already contains lines the gold patch adds (Tier C, by rule, no model read): SWE-bench Pro 0 of 100 sampled (of 731 public tasks), SWE-rebench 0 of 100 (of 6,542), SWE-Gym 1 of 100 (of 2,438), SWE-PolyBench 4 of 100 (of 2,110), Multi-SWE-bench 1 of its 300-task flash set. See the table →
- OmniCode, all 494 tasks in its three main files: 33 flagged, of which 29 are SWE-bench Verified tasks it carries and 4 are its own. Open the OmniCode page →
- Each sample is a reproducible random draw. Tier A (the fix stated in words, read by a model) was not run on these samples; run it on any of them with leaklens check --judge. Read the CLI docs →
From an agent: the same checks over MCP, read-only, no key. Connect it