diff --git a/docs/user-guide/sinks/retry-strategy.md b/docs/user-guide/sinks/retry-strategy.md new file mode 100644 index 0000000000..a5b2a7264b --- /dev/null +++ b/docs/user-guide/sinks/retry-strategy.md @@ -0,0 +1,68 @@ +# Retry Strategy + +### Overview +The `RetryStrategy` is used to configure the behavior for a sink after encountering failures during a write operation. +This structure allows the user to specify how Numaflow should respond to different fail-over scenarios for Sinks, ensuring that the writing can be resilient and handle +unexpected issues efficiently. + + +### Struct Explanation + + +`retryStrategy` is optional, and can be added to the Sink spec configurations where retry logic is necessary. + + + +```yaml +sink: + retryStrategy: + # Optional + backoff: + duration: 1s # Optional + steps: 3 # Optional, number of retries (including the 1st try) + # Optional + onFailure: retry|fallback|drop +``` +Note: If no custom fields are defined for retryStrategy then the **default** values are used. + +- `BackOff` - Defines the timing for retries, including the interval and the maximum attempts. + - `duration`: the time interval to wait before retry attempts + - Default: _1ms_ + - `steps`: the limit on the number of times to try the sink write operation including retries + - Default: _Infinite_ +- `OnFailure` - Specifies the action to be undertaken if number of retries are exhausted + - retry: continue with the retry logic again + - fallback: write the leftover messages to a [fallback](https://numaflow.numaproj.io/user-guide/sinks/fallback/) sink + - drop: any messages left to be processed are dropped + - Default: _retry_ + + +### Constraints + +1) If the `onFailure` is defined as fallback, then there should be a fallback sink specified in the spec. + +2) The steps defined should always be `> 0` + + +## Example + +```yaml + sink: + retryStrategy: + backoff: + interval: "500ms" + steps: 10 + onFailure: "fallback" + udsink: + container: + image: my-sink-image + fallback: + udsink: + container: + image: my-fallback-sink +``` +### Explanation + +- Normal Operation: Data is processed by the primary sink container specified by `UDSink`. +The system retries up to 10 times for a batch write operation to succeed with an interval of 500 milliseconds between each retry. +- After Maximum Retries: If all retries fail, data is then routed to a fallback sink instead. diff --git a/mkdocs.yml b/mkdocs.yml index 7c9161d480..7e57a6af21 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -66,6 +66,7 @@ nav: - user-guide/sinks/blackhole.md - User-defined Sinks: "user-guide/sinks/user-defined-sinks.md" - Fallback Sink: "user-guide/sinks/fallback.md" + - Retry Strategy: "user-guide/sinks/retry-strategy.md" - User-defined Functions: - Overview: "user-guide/user-defined-functions/user-defined-functions.md" - Map: