SWE-Bench Pro (public sample): every task, with its verdict
SWE-Bench Pro, published by Scale AI. Paper authors: Xiang Deng, Jeff Da, Edwin Pan, Yannis Yiming He, Charles Ide, Kanak Garg, Niklas Lauffer, Andrew Park, Nitin Pasari, Chetan Rane, Karmini Sampath, Maya Krishnan, Srivatsa Kundurthy, Sean Hendryx, Zifan Wang, Vijay Bharadwaj, Jeff Holm, Raja Aluri, Chen Bo Calvin Zhang, Noah Jacobson, Bing Liu and Brad Kenstler. SWE-Bench Pro (arXiv) · This page covers a random 100-task sample of the 731 public tasks (v1 config of the ScaleAI/SWE-bench_Pro dataset). The sample is the 100 instance IDs with the lowest SHA-256 of "own1:swe-bench-pro:<instance_id>", so anyone can redraw it. · Report a correction · flags show what the public text and git history contain: public exposure was possible, nothing more.
| Task | Verdict | Issue title |
|---|---|---|
| instance_future-architect__vuls-01441351c3407abfc21c48a38e28828e1b504e0c | no overlap found (rule check) | SNMP2CPE fails to emit correct CPEs for Fortinet – FortiSwitch-108E case |
| instance_future-architect__vuls-030b2e03525d68d74cb749959aac2d7f3fc0effa | no overlap found (rule check) | Outdated security‑update mapping for certain Windows releases in the Vuls scanner. |
| instance_future-architect__vuls-5af1a227339e46c7abf3f2815e4c636a0c01098e | no overlap found (rule check) | Issue Title: Incorrect detection of running kernel package versions when multiple variants are installed |
| instance_future-architect__vuls-dc496468b9e9fb73371f9606cdcdb0f8e12e70ca | no overlap found (rule check) | Reorganize syslog configuration into a dedicated configuration component. |
| instance_future-architect__vuls-f6cc8c263dc00329786fa516049c60d4779c4a07 | no overlap found (rule check) | Package name parsing produces incorrect namespace, name, or subpath in PURLs |