Leaklens

Multi-SWE-bench (flash): every task, with its verdict

Multi-SWE-bench, published by ByteDance Seed. Paper authors: Daoguang Zan, Zhirong Huang, Wei Liu, Hanwu Chen, Linhao Zhang, Shulin Xin, Lu Chen, Qi Liu, Xiaojian Zhong, Aoyan Li, Siyao Liu, Yongsheng Xiao, Liangqiang Chen, Yuyu Zhang, Jing Su, Tianyu Liu, Rui Long, Kai Shen and Liang Xiang. Multi-SWE-bench (arXiv) · Report a correction · flags show what the public text and git history contain: public exposure was possible, nothing more.

Download CSV
TaskVerdictIssue title
BurntSushi__ripgrep-1294no overlap found (rule check)For to --iglob even if -g
BurntSushi__ripgrep-1367no overlap found (rule check)match bug
BurntSushi__ripgrep-1642no overlap found (rule check)Search does not stop with --max-count and --context with matches in the context lines
BurntSushi__ripgrep-1980no overlap found (rule check)Support apply smart case to each individual search pattern separately
BurntSushi__ripgrep-2209no overlap found (rule check)Adding --replace to a --multiline search can expand what content is matched
BurntSushi__ripgrep-2295no overlap found (rule check)Only some ignores applied when searching a sub-directory
BurntSushi__ripgrep-2488no overlap found (rule check)Patterns may affect each other when multiple patterns are provided
BurntSushi__ripgrep-2610no overlap found (rule check)Option to print file paths as file URLs
BurntSushi__ripgrep-454no overlap found (rule check)`--only-matching` only prints first match of matched line N times
BurntSushi__ripgrep-723no overlap found (rule check)Fixed width line numbers
BurntSushi__ripgrep-727no overlap found (rule check)Suggest --fixed-strings on invalid regexps
BurntSushi__ripgrep-954no overlap found (rule check)use different exit codes for "no match" and "error"