Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run multiple worker loops #1064

Merged
merged 3 commits into from
Dec 13, 2023
Merged

Run multiple worker loops #1064

merged 3 commits into from
Dec 13, 2023

Conversation

matthew-white
Copy link
Member

Closes #533. Instead of running a single worker loop, Backend will now run 4. I didn't change the timing of how a worker waits between work.

What has been done to verify that this works as intended?

Existing tests continue to pass, but tests usually run workers manually (via exhaust()). When they do run a loop, it's at most one loop. I think it's hard to really test multiple loops and that the best thing to do is just to get the PR merged before regression testing begins. If we run into performance or other issues during regression testing, we can address them then.

Why is this the best possible solution? Were any other approaches considered?

I probably could have called the existing worker() function multiple times, but I noticed that each invocation of worker() would create functions to check, run, and report. Instead, I created a new function named workerQueue() that will create all worker-related functions at once. That reduces the number of duplicate functions and I think is a little easier to understand.

How does this change affect users? Describe intentional changes to behavior and behavior that could have accidentally been affected by code changes. In other words, what are the regression risks?

We already have concurrent workers in place because of our use of pm2, so I'm hoping we won't run into any issues.

Before submitting this PR, please make sure you have:

  • run make test-full and confirmed all checks still pass OR confirm CircleCI build passes
  • verified that any code from external sources are properly credited in comments or that everything is internally sourced

This prevents now() from being passed an error. Previously, if the
UPDATE to increment the failure count was rejected, now() would be
passed an error, which isn't something it knows how to handle.
@matthew-white matthew-white requested a review from ktuite December 12, 2023 23:54
@matthew-white matthew-white merged commit 6efdd5b into master Dec 13, 2023
4 checks passed
@matthew-white matthew-white deleted the multiple-workers branch December 13, 2023 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create more or faster workers
2 participants