Leaklens

Your score, outside the headline flags

Paste the resolved instance IDs from a SWE-bench Verified or Lite run, or drop its results file. You get the score on all tasks, the score on the tasks outside the Leaklens headline flags, and a table row and citation to paste into a paper. It runs in your browser: nothing is sent or stored, and there is no sign-in.

What the headline flags are

A headline flag is Tier A where both checks agreed on a quote that states the fix, or Tier C (the issue repeats lines the gold patch adds): 149 of 500 Verified tasks and 84 of 300 Lite tasks. The tasks outside the headline flags are not a clean set. In Verified, the 351 include 69 tasks a third check settled as stating the fix, 101 that point to the fix (Tier B) and 60 disputed tasks; in Lite, the 216 include 9, 63 and 54. Their task pages say so, and they are counted apart from the headline, not removed. How each tier is computed is on the methodology page; how far to trust a flag is on the validation page.

The same numbers on your machine

This page runs the same code and the same flag file as the open engine, github.com/Vyls42/leaklens (MIT). No network, no model:

node bin/leaklens.mjs rescore results.json --benchmark verified --label "your model"

The per-task flags

One row per task, with tier, state, whether it counts in the headline, the quoted sentence, the fix commit date and the methodology version: SWE-bench Verified (CSV, 500 rows) · SWE-bench Lite (CSV, 300 rows) · the flag file this page uses (JSON). The flags for 135 public leaderboard entries are already applied in the rescored Verified leaderboard.