SWE-Bench Pro (public sample): every task, with its verdict
SWE-Bench Pro, published by Scale AI. Paper authors: Xiang Deng, Jeff Da, Edwin Pan, Yannis Yiming He, Charles Ide, Kanak Garg, Niklas Lauffer, Andrew Park, Nitin Pasari, Chetan Rane, Karmini Sampath, Maya Krishnan, Srivatsa Kundurthy, Sean Hendryx, Zifan Wang, Vijay Bharadwaj, Jeff Holm, Raja Aluri, Chen Bo Calvin Zhang, Noah Jacobson, Bing Liu and Brad Kenstler. SWE-Bench Pro (arXiv) · This page covers a random 100-task sample of the 731 public tasks (v1 config of the ScaleAI/SWE-bench_Pro dataset). The sample is the 100 instance IDs with the lowest SHA-256 of "own1:swe-bench-pro:<instance_id>", so anyone can redraw it. · Report a correction · flags show what the public text and git history contain: public exposure was possible, nothing more.
| Task | Verdict | Issue title |
|---|---|---|
| instance_gravitational__teleport-0415e422f12454db0c22316cf3eaa5088d6b6322 | no overlap found (rule check) | Multi-Device U2F Authentication Restricted to Single Token Selection |
| instance_gravitational__teleport-46a13210519461c7cec8d643bfbe750265775b41 | no overlap found (rule check) | `tctl auth sign --format=kubernetes` uses incorrect port from proxy public address |
| instance_gravitational__teleport-4f771403dc4177dc26ee0370f7332f3fe54bee0f-vee9b09fb20c43af7e520f57e9239bbcf46b7113d | no overlap found (rule check) | Foundational buffering and deadline primitives for resilient connections |
| instance_gravitational__teleport-ac2fb2f9b4fd1896b554d3011df23d3d71295779 | no overlap found (rule check) | Auth service crashing |
| instance_gravitational__teleport-b1bcd8b90c474a35bb11cc3ef4cc8941e1f8eab2-vee9b09fb20c43af7e520f57e9239bbcf46b7113d | no overlap found (rule check) | Incorrect counting of authenticated HTTP connections in ingress reporter metrics |
| instance_gravitational__teleport-c335534e02de143508ebebc7341021d7f8656e8f | no overlap found (rule check) | Issues with certificate validation in tsh proxy ssh |
| instance_gravitational__teleport-dd3977957a67bedaf604ad6ca255ba8c7b6704e9 | no overlap found (rule check) | Support additional principals for Teleport services. |
| instance_gravitational__teleport-f432a71a13e698b6e1c4672a2e9e9c1f32d35c12 | no overlap found (rule check) | Add KeyStore interface and rawKeyStore implementation to manage cryptographic keys |