[Questions] How to narrow down why consumer may be getting duplicate deliveires

[Haileab-TG]

Community Support Policy

• I have read RabbitMQ's Community Support Policy
• I run RabbitMQ 4.x, the only series currently covered by community support
• I promise to provide all relevant information (versions, logs from all nodes, rabbitmq-diagnostics output, detailed reproduction steps)

RabbitMQ version used

4.0.4

Erlang version used

26.2.x

Operating system (distribution) used

Linux

How is RabbitMQ deployed?

Community Docker image

rabbitmq-diagnostics status output

See https://www.rabbitmq.com/docs/cli to learn how to use rabbitmq-diagnostics

# PASTE OUTPUT HERE, BETWEEN BACKTICKS

Logs from node 1 (with sensitive values edited out)

See https://www.rabbitmq.com/docs/logging to learn how to collect logs

# PASTE LOG HERE, BETWEEN BACKTICKS

Logs from node 2 (if applicable, with sensitive values edited out)

See https://www.rabbitmq.com/docs/logging to learn how to collect logs

# PASTE LOG HERE, BETWEEN BACKTICKS

Logs from node 3 (if applicable, with sensitive values edited out)

See https://www.rabbitmq.com/docs/logging to learn how to collect logs

# PASTE LOG HERE, BETWEEN BACKTICKS

rabbitmq.conf

See https://www.rabbitmq.com/docs/configure#config-location to learn how to find rabbitmq.conf file location

# PASTE rabbitmq.conf HERE, BETWEEN BACKTICKS

Steps to deploy RabbitMQ cluster

N/A

Steps to reproduce the behavior in question

N/A

advanced.config

See https://www.rabbitmq.com/docs/configure#config-location to learn how to find advanced.config file location

# PASTE advanced.config HERE, BETWEEN BACKTICKS

Application code

# PASTE CODE HERE, BETWEEN BACKTICKS

Kubernetes deployment file

# Relevant parts of K8S deployment that demonstrate how RabbitMQ is deployed
# PASTE YAML HERE, BETWEEN BACKTICKS

What problem are you trying to solve?

I have consumers that are consuming the same message multiple time sometimes after the first message completes getting processed by the consumer and sometimes while getting processed. So, I need a full picture on what is going on with rabbitmq, are the queue getting requeue for any reason if so what is the reason. I read the logging documentation it is so vague, can't figure out how to achieve this type of logging. Tried firehose, but best it provided was about message getting published and received, no info what happen after received like ack successfully or requeue due something/

[kjnilsson]

You have provided very little information but I suspect there is an issue with your consumer client application. Perhaps you are closing the channel too soon after acking such that the ack never reaches there queue. If you provide some sample code that reproduces the issue then we can perhaps advise further.

[michaelklishin]

@Haileab-TG so you need a full picture but you have provided absolutely no details for the core team to work with? Yeah, since we have infinite time for guessing.

I read the logging documentation it is so vague, can't figure out how to achieve this type of
logging

The logging guide is not vague. RabbitMQ does not log operations on individual messages, it's not practically possible when rates can reach tens of thousands, hundreds or (mostly relevant for streams and superstreams) thousands of millions.

A traffic capture is by far the best source of information. To a certain extent, logs and internal events (consumer registration events specifically) could help.

Please take it from here. Our team won't spend time guessing what your application does.

[michaelklishin]

Also worth considering that

1. If you bind a queue N times to an exchange, you will not get duplicate deliveries (with the exchange types available in the distribution and not counting the Consistent Hashing Exchange where multiple bindings are explicitly prohibited in recent versions)
2. If a message is routed to N queues and a consumer consumes from all or some of them, you will get duplicate deliveries. RabbitMQ streams support deduplication but every message routed to a queue is (semantically) a complete copy of the same original message routed to other exchanges