Leaklens

SWE-Gym (training set sample): every task, with its verdict

SWE-Gym, published by the SWE-Gym authors. Paper authors: Jiayi Pan, Xingyao Wang, Graham Neubig, Navdeep Jaitly, Heng Ji, Alane Suhr and Yizhe Zhang. SWE-Gym (arXiv) · This page covers a random 100-task sample of the 2,438 training tasks (SWE-Gym/SWE-Gym on Hugging Face). The sample is the 100 instance IDs with the lowest SHA-256 of "own1:swe-gym:<instance_id>", so anyone can redraw it. · Report a correction · flags show what the public text and git history contain: public exposure was possible, nothing more.

Download CSV
TaskVerdictIssue title
iterative__dvc-1684no overlap found (rule check)dvc repro --force is ignored when stage does not change
iterative__dvc-1749no overlap found (rule check)cache: dir option is not working correctly
iterative__dvc-2254no overlap found (rule check)run: running the same command twice on a callback stage doesn't reproduce
iterative__dvc-3576no overlap found (rule check)metrics diff with missing old value
iterative__dvc-4978no overlap found (rule check)dvc init fails when .dvc directory is git-ignored
iterative__dvc-5005no overlap found (rule check)dvc 1.10.2 does not provide the extra 'webdavs'
iterative__dvc-5747no overlap found (rule check)push: remote.jobs in .dvc/config is ignored
iterative__dvc-6565no overlap found (rule check)http: timeouts after 60 seconds (when using with aiohttp-retry)
iterative__dvc-6799no overlap found (rule check)exp show: --show-md for experiment tables
iterative__dvc-7103no overlap found (rule check)exp init: change stage name to `train`
iterative__dvc-9838no overlap found (rule check)repro: External outputs with persist: true and cache: false raise AssertionError