Skip to content

Commit

Permalink
fix(taskworker): Avoid crash with a grpc bug (#84267)
Browse files Browse the repository at this point in the history
In the case when the worker receives a gRPC error message from the
broker, log the exception and
continue processing instead of crashing the entire application.
  • Loading branch information
evanh authored Jan 29, 2025
1 parent e3b70b9 commit 0745e6a
Showing 1 changed file with 13 additions and 5 deletions.
18 changes: 13 additions & 5 deletions src/sentry/taskworker/worker.py
Original file line number Diff line number Diff line change
Expand Up @@ -336,11 +336,19 @@ def _drain_result(self, fetch: bool = True) -> bool:
if not self._child_tasks.full():
fetch_next = FetchNextTask(namespace=self._namespace)

next_task = self.client.update_task(
task_id=result.task_id,
status=result.status,
fetch_next_task=fetch_next,
)
try:
next_task = self.client.update_task(
task_id=result.task_id,
status=result.status,
fetch_next_task=fetch_next,
)
except grpc.RpcError as e:
logger.exception(
"taskworker.drain_result.update_task_failed",
extra={"task_id": result.task_id, "error": e},
)
return False

if next_task:
try:
self._child_tasks.put(next_task, block=False)
Expand Down

0 comments on commit 0745e6a

Please sign in to comment.