Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix deserializing callbacks to proper types #45305

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

mobuchowski
Copy link
Contributor

Without this fix, the callback requests are deserialized as a base class CallbackRequest and are not executed - they are silently failing through this check.

def _execute_callbacks(
    dagbag: DagBag, callback_requests: list[CallbackRequest], log: FilteringBoundLogger
) -> None:
    for request in callback_requests:
        log.debug("Processing Callback Request", request=request.to_json())
        if isinstance(request, TaskCallbackRequest):
            raise NotImplementedError(
                "Haven't coded Task callback yet - https://github.com/apache/airflow/issues/44354!"
            )
            # _execute_task_callbacks(dagbag, request)
        elif isinstance(request, DagCallbackRequest):
            _execute_dag_callbacks(dagbag, request, log)

@boring-cyborg boring-cyborg bot added the area:Scheduler including HA (high availability) scheduler label Dec 30, 2024
@mobuchowski mobuchowski marked this pull request as ready for review December 30, 2024 21:31
Copy link
Contributor

@amoghrajesh amoghrajesh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Callbacks haven't yet been ported to TASK SDK @mobuchowski
Is this an attempt to do that?

@mobuchowski
Copy link
Contributor Author

@amoghrajesh dag ones (DagCallbackRequest) should work, and are mostly implemented in some form. This PR makes basic ones work. There are missing things - like the context is empty dict - but neverthless is improving the situation.

My main motivation is unblocking ticket #44547 that has been really hard to get to get merged.

@mobuchowski mobuchowski force-pushed the tasksdk-deserialize-callbacks branch from fc00586 to 879a9aa Compare December 31, 2024 08:31
@mobuchowski mobuchowski force-pushed the tasksdk-deserialize-callbacks branch from 879a9aa to 7b9eb4a Compare December 31, 2024 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:Scheduler including HA (high availability) scheduler
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants