SWE-Bench Pro (public sample): every task, with its verdict
SWE-Bench Pro, published by Scale AI. Paper authors: Xiang Deng, Jeff Da, Edwin Pan, Yannis Yiming He, Charles Ide, Kanak Garg, Niklas Lauffer, Andrew Park, Nitin Pasari, Chetan Rane, Karmini Sampath, Maya Krishnan, Srivatsa Kundurthy, Sean Hendryx, Zifan Wang, Vijay Bharadwaj, Jeff Holm, Raja Aluri, Chen Bo Calvin Zhang, Noah Jacobson, Bing Liu and Brad Kenstler. SWE-Bench Pro (arXiv) · This page covers a random 100-task sample of the 731 public tasks (v1 config of the ScaleAI/SWE-bench_Pro dataset). The sample is the 100 instance IDs with the lowest SHA-256 of "own1:swe-bench-pro:<instance_id>", so anyone can redraw it. · Report a correction · flags show what the public text and git history contain: public exposure was possible, nothing more.
| Task | Verdict | Issue title |
|---|---|---|
| instance_navidrome__navidrome-3977ef6e0f287f598b6e4009876239d6f13b686d | no overlap found (rule check) | Hasher lacks deterministic seeding needed for stable “random” ordering |
| instance_navidrome__navidrome-55bff343cdaad1f04496f724eda4b55d422d7f17 | no overlap found (rule check) | Simplify SQLite3 access by reverting read/write separation |
| instance_navidrome__navidrome-56303cde23a4122d2447cbb266f942601a78d7e4 | no overlap found (rule check) | Scanner does not support R128 gain tags for track and album |
| instance_navidrome__navidrome-69e0a266f48bae24a11312e9efbe495a337e4c84 | no overlap found (rule check) | Issue Title: Remove size from public image ID JWT. |
| instance_navidrome__navidrome-6bd4c0f6bfa653e9b8b27cfdc2955762d371d6e9 | no overlap found (rule check) | Possible to remove authentication? |
| instance_navidrome__navidrome-89b12b34bea5687c70e4de2109fd1e7330bb2ba2 | no overlap found (rule check) | Last.fm API not getting correct Artist info |