@devvmh , thank you for making this proposal. Have you looked at Gradle Build Scans: https://scans.gradle.com/?
It seems like they can help you answer your questions and will also provide a more user-friendly interface than a console output.

Here is an example for a flaky test in JUnit codebase, which shows all "flaky" tests in a build (i.e., ones that passed on retry): https://ge.junit.org/s/tcwmbmjybjktg/tests/overview?outcome=FLAKY. You can also navigate to individual methods and get details about individual retries: https://ge.junit.org/s/tcwmbmjybjktg/tests/task/:junit-jupiter-engine:test/details/org.junit.jupiter.engine.extension.TimeoutExtensionTests$SeparateThread/oneThreadStuckForeverAndOtherTestsInSameThread()?top-execution=1

Feel free to reopen the issue if you feel like the Build Scan does not cover your use case.

Build Scans don't cover my use case. For me, as the application owner, it's not too hard to learn about this edge case and remember it. However, there's a population of developers (around 100-200 per year, with some repeat offenders and many new each year) who are contributing more casually, or even once total in their lifetime. For them, they are overwhelmed with documentation and wikis to read to learn about contribution, so being aware of this problem is the issue. Build scans might help me target the flaky tests, but I already know which tests are flaky (it's the JSP tests that spin up a Tomcat server, and the issue is with the test framework I'm using, not with any individual test).

For them, they are likely to be exposed to this problem for the first time by seeing the sample build output I shared above, which is very confusing. I could add a target right before :test to say "hey, check build scans" or just explain this problem, but seems a bit heavy handed to me

(by the way, the problem has gotten better since I posted this last August, and I'm not really sure why. I might end up dropping the plugin and just relying on retrying the full build when a test is flaky)

So summarizing

1. Still love this plugin! Thanks so much for creating it!
2. No, build scans don't solve my problem
3. I won't re-open since I'd rather not clog up the queue if others aren't experiencing the issue.

@devvmh , I see; thank you for explaining your use case. I understand first-time contributors will likely pay little attention to the Build Scan, as it requires a certain habit to be developed. We have a couple of requests in the queue regarding adding callbacks to the plugin, allowing users to perform arbitrary actions on retries or when they complete. I linked this issue to the tracking epic: #271. I feel like you'd be able to achieve your use case by logging something in the callback action.

I learned more about this since April and I found that my company doesn't allow the usage of https://scans.gradle.com/ :(