Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Research preview streaming with the new Generator Handler #35

Open
davefojtik opened this issue Aug 16, 2024 · 0 comments
Open

Research preview streaming with the new Generator Handler #35

davefojtik opened this issue Aug 16, 2024 · 0 comments
Labels
good first issue Good issue to start contributing optimization Performance and code improvements

Comments

@davefojtik
Copy link
Owner

davefojtik commented Aug 16, 2024

RunPod's new Generator Handler allows fetching /stream endpoint to get continuous updates on a job without the need for RunPod to send updates to your server as it had to be done previously.

The maximum size for a payload that can be sent using yield to stream results is 1 MB.

Needed steps are:

  • Check if this limit is not exceeded in any situation.
  • Change the handler type to the Generator one
  • Add code that will yield previews to the handler
  • Test it
@davefojtik davefojtik added optimization Performance and code improvements good first issue Good issue to start contributing labels Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good issue to start contributing optimization Performance and code improvements
Projects
None yet
Development

No branches or pull requests

1 participant