Skip to content

Issues: swe-bench/SWE-bench

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

29 Open 158 Closed
29 Open 158 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

blackboxai.com is still visible. this shouldnt be the case cause they are scammers and fakers bug Something isn't working
#299 opened Jan 24, 2025 by stjema
Support sweagent merge-preds json output bug Something isn't working
#298 opened Jan 24, 2025 by dashesy
Building base image (sweb.base.py.x86_64:latest) bug Something isn't working
#297 opened Jan 22, 2025 by kamuy-shennai
Unable to run evaluation on gold predictions with princeton-nlp/SWE-bench_Multimodal dataset bug Something isn't working
#296 opened Jan 22, 2025 by thetonywu
Request to add released version tags in GitHub
#295 opened Jan 22, 2025 by MattFisher
How stable is run_evaluation.py with gold patch for SWE-bench_Verified? documentation Improvements or additions to documentation
#294 opened Jan 20, 2025 by MarcCote
1
Environment images failed to build in running "swebench.harness.run_evaluation" bug Something isn't working
#293 opened Jan 18, 2025 by haoxianghz
Some SWE-bench_Verified test specs are missing test parameters bug Something isn't working
#292 opened Jan 17, 2025 by jsdelfino
Truncated parametrized test names in SWE-bench_Verified bug Something isn't working
#290 opened Jan 16, 2025 by jsdelfino
Tests incorrectly named [100%] in SWE-bench-Verified instances pytest-5262 and pytest-7521 bug Something isn't working
#289 opened Jan 16, 2025 by jsdelfino
How can I populate the FAIL_TO_PASS and PASS_TO_PASS fields? documentation Improvements or additions to documentation
#287 opened Jan 15, 2025 by ErsjanKeri
2
Failed to build matplotlib images.
#286 opened Jan 13, 2025 by kohankhaki
4
failed to build env images bug Something isn't working
#283 opened Jan 10, 2025 by jianzzz
1
Misleading removed images count. bug Something isn't working
#282 opened Jan 10, 2025 by SmartManoj
"Successful" patches fail on unexecuted developer-written tests bug Something isn't working
#280 opened Jan 10, 2025 by PrinzyW
Bug Report: Error Building Instance Image sweb.eval.x86_64.astropy__astropy-14995:latest bug Something isn't working
#277 opened Jan 7, 2025 by MichaelYang-lyx
1
Incorrect unit tests in FAIL_TO_PASS and PASS_TO_PASS bug Something isn't working
#275 opened Dec 17, 2024 by WuYff
1
pvlib/pvlib-python in dev set failed for Gold Patch bug Something isn't working
#274 opened Dec 16, 2024 by thaiminhpv
1
JavaScript version of the SWEbench dataset?
#269 opened Dec 1, 2024 by calcsam
5 Instances in Verified Fail for Gold Patch bug Something isn't working
#267 opened Nov 30, 2024 by wistuba
4
PASS_TO_PASS and FAIL_TO_PASS test cases documentation Improvements or additions to documentation
#257 opened Nov 21, 2024 by chenzimin
What decides whether a dependency is part of the "instance" or "environment" level cache? documentation Improvements or additions to documentation
#256 opened Nov 19, 2024 by max-kaufmann
The patch in mwaskom__seaborn-2848 corresponds to the wrong problem statement bug Something isn't working
#255 opened Nov 19, 2024 by wangwhh
The environment building failed for astropy__astropy__1.3 for unsupported setuptools version. bug Something isn't working
#251 opened Nov 15, 2024 by DavdGao
1
pyvista__pyvista-4315 isn't working bug Something isn't working
#246 opened Nov 6, 2024 by thaiminhpv
1
ProTip! Find all open issues with in progress development work with linked:pr.