-
Notifications
You must be signed in to change notification settings - Fork 209
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve accounting of tests marked as driver bugs #2177
base: master
Are you sure you want to change the base?
Conversation
"successful bugs" reads a bit weird, maybe "fixed" would be better? Just a nit though, I do agree that we should only count bugs as actual bugs if the corresponding test fails. |
Right, I actually used "fixed" at first. Then I thought that a passing test doesn't say all that much about bugs being fixed. With flaking tests, tests that only fail on some hardware, etc. That made me mirror the Happy to use whatever terminology you'd prefer though. |
Can't it just be explicitly named? Something like "not our bugs", "driver bugs" or "invalid bugs" etc. Or does that sound weird. |
Currently all tests marked as driver bugs get counted as "bug" no matter if they succeed or fail. This makes it clearer how many of them actually fail.
b937f59
to
0190102
Compare
Calling them "driver bugs" is clearer than just "bugs", especially since another category is called "failures".
0190102
to
eba6ce6
Compare
|
Forgive me if I'm misunderstanding, but this is bugs that are from native DX12 yes? If so, perhaps "Bugs replicated" might be a preferred term? |
No, these are bugs in the underlying Vulkan driver and should ideally be fixed there. Modulo misattributions of course. |
Before:
After: