[Suggestion] Show images one by one as they get created and not at the end of creating the whole batch

[johnpccd]

Here is a quick suggestion: Show images one by one as they get created and not at the end of creating the whole batch

[AK391]

this can be done now with gradio==3.2.1b1, see #694

[johnpccd]

that is cool but I actually meant spitting out each image once it's finalized

[micwalk]

This is the most elegant solution. Even though the Gradio example demonstrates replacing the output with an increasingly detailed image, it should also be possible to iteratively update with an array of increasing length. What must be done in this repo is to update to the new version then convert the txt2img / img2img function to use yield instead of return. Also in another thread it was mentioned that the new version of Gradio breaks some other UI things.

[mchaker]

If you press "Refresh" in the UI, it shows the currently completed images.

[DasWookie]

I would agree with this. The only possible catch is that hitting refresh can result in a not insubstantial hang of the UI while waiting for the result to load. I too find myself clicking refresh a lot while doing long runs. If you hit refresh at the wrong time, you can end up not getting everything as well in the UI.

[mchaker]

Good point -- and if you press Refresh after the job is complete, the results disappear from the UI. Hmm...

[DasWookie]

I'm spelunking through the code now, trying to figure out how to get both of these ideas to work. This is:

1. Emit each step generation to an image, so for a 65 steps generation, you'd get 65 images

• This might be the easiest to implement, as I suspect a call to image.save after each ddim_step would handle this. That's just a guess tho, and so far I'm not seeing any place to, say for txt2img or img2img, drop in an "extra" image.save

2. Auto-load each completed image into the UI, so if you generate 100 images, the images would load into the UI automatically 1 at a time, 1, 2, 3, ... 98, 99, 100. Without the need to hit refresh.

• No idea how hard this would be. Also probably relies on fixing the results getting obliterated once a run completes. So, seems problematic. Possible, but harder to do I suspect. I'm just getting started on digging into the WebUI code tho.

[mchaker]

Thank you so much for doing so!

[cobryan05]

I'm spelunking through the code now, trying to figure out how to get both of these ideas to work. This is:

1. Emit each step generation to an image, so for a 65 steps generation, you'd get 65 images

• This might be the easiest to implement, as I suspect a call to image.save after each ddim_step would handle this. That's just a guess tho, and so far I'm not seeing any place to, say for txt2img or img2img, drop in an "extra" image.save

This PR should basically take care of point 1. #611 . I don't really know what's going on with the repo re-org right now, so even though it says it was merged I have no idea where it went.

2. Auto-load each completed image into the UI, so if you generate 100 images, the images would load into the UI automatically 1 at a time, 1, 2, 3, ... 98, 99, 100. Without the need to hit refresh.

Number 2 might be more complicated. Doable, but more complicated. gradio puts components in the 'loading' state when something with them marked as an output is executing. I don't think you can do anything with the component in this state... and have written a bunch of code on that assumption, but I can't at the moment really recall if I actually tested 😆 . But, it's a pain to 'share' the output gallery, and if you are executing code with the component marked as output then the user can't browse it. Maybe gradio's new iteration API has solved this.

I didn't originally implement it because I was afraid when each update 'refreshed' the gallery that a user browsing the gallery would lose their place in it during the refresh, but this doesn't seem to be the case. I also worried that for large batch sizes (or if you have upscaling enabled) that the auto-refresh would take so long that the next batch would be ready before it finished, triggering the next update, essentially locking the user out of the gallery...

• No idea how hard this would be. Also probably relies on fixing the results getting obliterated once a run completes. So, seems problematic. Possible, but harder to do I suspect. I'm just getting started on digging into the WebUI code tho.

6e3463a should take care of freeing the server memory as soon as the gallery loads it. This will make it free it when the next job starts instead.