UI Indicating Progress of the test run (or that it's working at all)

[strickinato]

2 Helpful reasons for output

Psychological

With large amounts of test files, running tests can take long (in problematic case we spoke of in particular, 68717 ms). During this time, it doesn't feel very good as the developer to watch nothing happen on the screen.

Helpful

In the case that you won't take the next action until all tests pass (like commit or push), it's useful to know as soon as possible when tests break. In the current state, we don't know until the end if there are any test breaks- but it would be very helpful to see them as they come out so as a developer I can either:

• End early
• Start fixing tests as others run

@stoeffel may have additional cases

[zwilias]

Yeah, I've been running with --report json (and then re-running without it on failure) for that reason. I've been looking at other test runners, and I'm starting to think that perhaps we should have a more verbose runner by default (print every success and failure + runtime) with a --quiet option. This is sort of a reversal from the "add a --verbose flag" proposal but ultimately, it feels like a good idea.

Would love to hear what Richard thinks about this, too.

[rtfeldman]

I'd rather print "Running test 23 of 348" and update that in-place (when at a terminal, as opposed to CI) than print successes.

Much less noise!

[avh4]

Was starting to write a reply to #620, but realized it fits better here:

---

Options for TUI for enabling progress output:

• (1) always show progress at some interval when using the "console" reporter (imo, preferred, since I think when most people want this, they'd rather not have an extra step of opting in, and I'm not seeing a reason having this on by default would be bad)
• (2) show progress at some interval by default and have a --no-progress to disable it (presumably used on CI). I guess progress would mess up json/junit output, so maybe it should only apply to "console" reporter?
• (3) have --progress option to enable showing progress at some interval (this doesn't seem ideal because it seems like most people would want this on by default, right?)

What is "at some interval" for showing progress? Is there even a tradeoff here that matters in typical use of elm-test? Probably just whichever's easier would be sufficient.

• whenever the coordinator receives an update from one of the workers
• every time the coordinator receives the status of X (10?, 100?) tests from the workers
• every X (1?, 10?) seconds