rabbitmqctl status errors and claims that rabbit_amqp1_0:list_local/0 is an undefined function

[hzsunzixiang]

Describe the bug

When running rabbitmqctl status in RabbitMQ version 4.0, an error occurs:

Status of node rabbit@rabbitmq4-2 ...
Error:
{:undef, [{:rabbit_amqp1_0, :list_local, [], []}, {:rabbit_networking, :local_connections, 0, [file: ~c"rabbit_networking.erl", line: 465]}, {:rabbit, :status, 0, [file: ~c"rabbit.erl", line: 771]}]}

When I changed the name of list_local/0 to something like list_local_my/0, it ran normally.

Reproduction steps

running rabbitmqctl status in RabbitMQ version 4.0

Expected behavior

Status of node rabbit@rabbitmq4-2 ...
Runtime

OS PID: 75325
OS: Linux
Uptime (seconds): 532
Is under maintenance?: false

Additional context

No response

[michaelklishin]

@hzsunzixiang I cannot reproduce with a 4.0.5 node.

According to the exception, rabbit_amqp1_0:list_local/0 is a function that does not exist. But it does, and as a part of the core, not the AMQP 1.0 plugin which is now a no-op.

I do not understand how you changed list_local/0 to list_local_my/0, that's not an argument.

We need a specific set of steps to reproduce.

[michaelklishin]

rabbitmq-diagnostics status (exactly the same command as rabbitmqctl status) calls
rabbit:status/0:

  def run([], %{node: node_name, timeout: timeout}) do
    :rabbit_misc.rpc_call(node_name, :rabbit, :status, [], timeout)
  end

rabbit:status/0 is defined in rabbit.erl.

It calls a lot of functions but the most relevant parts is rabbit_networking:local_connections/0
which looks like this:

-spec local_connections() -> [rabbit_types:connection()].
local_connections() ->
    Amqp091Pids = pg_local:get_members(rabbit_connections),
    Amqp10Pids = rabbit_amqp1_0:list_local(),
    Amqp10Pids ++ Amqp091Pids.

and rabbit_amqp1_0:list_loca/0 does exist in v4.0.x, and its exported:

-module(rabbit_amqp1_0).

-export([list_local/0,
         register_connection/1]).

%% (elided for brevity)

-spec list_local() -> [pid()].
list_local() ->
    pg:which_groups(pg_scope()).

Moreover, I do not understand how changing list_local/0 to list_local_my/0 would make an undef go away since that function clearly does not exist and indeed THAT would result in an undef.

@hzsunzixiang I have walked you through the entire code path with evidence that the function in question does exist, and it is just two function calls away from the status command:

rabbit:status/0 => rabbit_networking:list_local/0 => rabbit_amqp1_0:list_local/0, all of which exist in v4.0.x and 4.0.5.

Since no one else has reported something similar since 4.0.1 (shipped in September 2024), I conclude that something very specific about your cluster, such as some nodes not yet being upgraded, causes the exception. And I don't have any details to work with, so please take it from there. This is open source software and the code path explained above is straightforward.

[michaelklishin]

If this repo is any indication, the node could have been built from source using a custom build system.

In my case, the node I use is a generic binary build from v4.0.5.

Our team won't spend any time on troubleshooting custom RabbitMQ distributions. If you build it from source, you own it, you debug it, you maintain it from start to finish.