SWE-PolyBench (full set sample): every task, with its verdict
SWE-PolyBench, published by Amazon Science. Paper authors: Muhammad Shihab Rashid, Christian Bock, Yuan Zhuang, Alexander Buchholz, Tim Esler, Simon Valentin, Luca Franceschi, Martin Wistuba, Prabhu Teja Sivaprasad, Woo Jung Kim, Anoop Deoras, Giovanni Zappella and Laurent Callot. SWE-PolyBench (arXiv) · This page covers a random 100-task sample of the 2,110 tasks (AmazonScience/SWE-PolyBench on Hugging Face). The sample is the 100 instance IDs with the lowest SHA-256 of "own1:swe-polybench:<instance_id>", so anyone can redraw it. · Report a correction · flags show what the public text and git history contain: public exposure was possible, nothing more.
| Task | Verdict | Issue title |
|---|---|---|
| prettier__prettier-14109 | Tier C: the issue text repeats lines of the gold patch | less format problem |
| prettier__prettier-10346 | no overlap found (rule check) | Markdown code-block with wrong language yields garbage |
| prettier__prettier-14170 | no overlap found (rule check) | Angular templates self closing tags |
| prettier__prettier-6133 | no overlap found (rule check) | Parens incorrectly dropped when trying to type a function expression |
| prettier__prettier-830 | no overlap found (rule check) | Unstable behaviours when using comments around variables |
| prettier__prettier-8465 | no overlap found (rule check) | Pug plugin doesn't work for the .vue files |
| prettier__prettier-8675 | no overlap found (rule check) | SCSS: unexpected format |