feat: Implement running Prefect task after models #52
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a new optional
run_task_after_model
parameter to theDbtDagOptions
class. When a Prefect task is provided as an argument,generate_tasks_dag
will inject that task into the DAG, running it after each model.The task must have exactly one required parameter:
model_id
of typestr
, which is supplied the valuenode.unique_id
. This enables the Prefect task to operate according to the specific model which it is running after. Of course, tasks which don't require this parameter can simply discard it.The implementation maintains consistency with the existing
run_test_after_model
parameter, and ensures that both can be used independently. If arguments are provided for both, the Prefect task depends on (waits for) the dbt test task.Additionally, this PR removes the parameter
run_test_after_model
fromgenerate_tasks_dag
, which already receivesdag_options
whererun_test_after_model
andrun_task_after_model
are both defined. This saves unnecessary argument passing and null checking.Resolves #51.