Add ability to pause jobs by queue, job class, or label (experimental) #1575
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bensheldon
commented
Jan 18, 2025
bensheldon
force-pushed
the
enable-pause
branch
5 times, most recently
from
736b14e to
240888b
Compare
|
Exciting! |
|
It's happening! |
bensheldon
commented
Jan 19, 2025
app/models/good_job/job.rb
Outdated
Comment on lines
99
to
125
| # Exclude jobs that are paused via queue_name or job_class. | ||
| # Only applies when enable_pause configuration is true. | ||
| # @!method exclude_paused | ||
| # @!scope class | ||
| # @return [ActiveRecord::Relation] | ||
| scope :exclude_paused, lambda { | ||
| return all unless GoodJob.configuration.enable_pauses | ||
|
|
||
| paused_query = GoodJob::Setting.where(key: GoodJob::Setting::PAUSES) | ||
| paused_queues_query = paused_query.select("jsonb_array_elements_text(value->'queues')") | ||
| paused_job_classes_query = paused_query.select("jsonb_array_elements_text(value->'job_classes')") | ||
| paused_labels_query = paused_query.select("jsonb_array_elements_text(value->'labels')") | ||
|
|
||
| where.not(queue_name: paused_queues_query) | ||
| .where.not(job_class: paused_job_classes_query) | ||
| .where( | ||
| Arel::Nodes::Not.new( | ||
| Arel::Nodes::NamedFunction.new( | ||
| "COALESCE", [ | ||
| Arel::Nodes::InfixOperation.new('&&', arel_table['labels'], Arel::Nodes::NamedFunction.new('ARRAY', [paused_labels_query.arel])), | ||
| Arel::Nodes::SqlLiteral.new('FALSE'), | ||
| ] | ||
| ) | ||
| ) | ||
| ) | ||
| } | ||
|
|
There was a problem hiding this comment.
This is the hot query, now with labels, which makes it slightly more complicated.
bensheldon
force-pushed
the
enable-pause
branch
6 times, most recently
from
43e560a to
d6f3d73
Compare
bensheldon
commented
Jan 19, 2025
bensheldon
changed the title
bensheldon
force-pushed
the
enable-pause
branch
from
76994df to
96f5270
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Technical Description
Paused values are stored in the GoodJob::Setting table. During the fetch-and-lock query, this record is loaded as sub=selects that add additional scopes to the query that exclude them. Paused jobs will appear as "queued" (a visible "paused" status in the Dashboard is tbd).
I'm unsure of the performance impact of the query scopes, so I've made this feature "opt-in".
Query right now looks like this:
Connects to #919, #1574
I put the pause/unpause buttons on the Performance page. I think that could be problematic, though I'm not sure if it will be more or less so than how it affects the fetch-and-lock query to add new conditions.
Future thoughts: