Describing Kafka consumer behaviour when consuming from multiple topi…

…cs (#3931)

* Describing Kafka consumer behaviour when consuming from multiple topics

Signed-off-by: Simon Headley <[email protected]>

* Further specifying the behaviour of Kafka consumer groups on topics.

Signed-off-by: Simon Headley <[email protected]>

* Changing 'this will guarantee' to 'this guarantees' as per PR review

Signed-off-by: Simon Headley <[email protected]>

* Update daprdocs/content/en/reference/components-reference/supported-pubsub/setup-apache-kafka.md

Co-authored-by: Hannah Hunter <[email protected]>
Signed-off-by: Mark Fussell <[email protected]>

---------

Signed-off-by: Simon Headley <[email protected]>
Signed-off-by: Mark Fussell <[email protected]>
Co-authored-by: Mark Fussell <[email protected]>
Co-authored-by: Hannah Hunter <[email protected]>

daprdocs/content/en/reference/components-reference/supported-pubsub/setup-apache-kafka.md

@@ -299,6 +299,44 @@ auth:
   secretStore: <SECRET_STORE_NAME>
 ```
 
+## Consuming from multiple topics
+
+When consuming from multiple topics using a single pub/sub component, there is no guarantee about how the consumers in your consumer group are balanced across the topic partitions. 
+
+For instance, let's say you are subscribing to two topics with 10 partitions per topic and you have 20 replicas of your service consuming from the two topics. There is no guarantee that 10 will be assigned to the first topic and 10 to the second topic. Instead, the partitions could be divided unequally, with more than 10 assigned to the first topic and the rest assigned to the second topic. 
+
+This can result in idle consumers listening to the first topic and over-extended consumers on the second topic, or vice versa. This same behavior can be observed when using auto-scalers such as HPA or KEDA.
+
+If you run into this particular issue, it is recommended that you configure a single pub/sub component per topic with uniquely defined consumer groups per component. This guarantees that all replicas of your service are fully allocated to the unique consumer group, where each consumer group targets one specific topic.
+
+For example, you may define two Dapr components with the following configuration:
+
+```yaml
+apiVersion: dapr.io/v1alpha1
+kind: Component
+metadata:
+  name: kafka-pubsub-topic-one
+spec:
+  type: pubsub.kafka
+  version: v1
+  metadata:
+  - name: consumerGroup
+    value: "{appID}-topic-one"
+```
+
+```yaml
+apiVersion: dapr.io/v1alpha1
+kind: Component
+metadata:
+  name: kafka-pubsub-topic-two
+spec:
+  type: pubsub.kafka
+  version: v1
+  metadata:
+  - name: consumerGroup
+    value: "{appID}-topic-two"
+```
+
 ## Sending and receiving multiple messages
 
 Apache Kafka component supports sending and receiving multiple messages in a single operation using the bulk Pub/sub API.