High server load caused by home-assistant-websocket node

[medicus07]

Describe the bug

I use Node Red (v4.08) in a Proxmox LXC Container with the home-assistant-websocket node (v0.74.2) installed. Home Assistant runs on its own server:
Core-2024.12.5
Supervisor-2024.12.0
Operating System-14.1
Frontend-20241127.8

For a few days now, perhaps since HA 2024.12, I have been noticing significant latency problems. That's why I went searching and was able to identify the problem at Nodered. The server had a very high load all the time. I reinstalled Nodered and then installed the individual nodes separately. The problem was in the home-assistant-websocket node. When I deactivated it, everything returned to normal.

I only use 12 HA nodes.

To Reproduce

active home-assistant-websocket node -> high server load

deactive home-assistant-websocket node -> normal, low server load

Expected behavior

No response

Screenshots

Example Flow

[
    {
        "id": "a386ab337d0a974f",
        "type": "mqtt out",
        "z": "3003f3c29df96849",
        "name": "",
        "topic": "FT55/EZ/innen",
        "qos": "",
        "retain": "",
        "respTopic": "",
        "contentType": "",
        "userProps": "",
        "correl": "",
        "expiry": "",
        "broker": "9ec0115c.8f7488",
        "x": 1140,
        "y": 420,
        "wires": []
    },
    {
        "id": "c0f02da063419dc8",
        "type": "join",
        "z": "3003f3c29df96849",
        "name": "",
        "mode": "custom",
        "build": "array",
        "property": "payload",
        "propertyType": "msg",
        "key": "topic",
        "joiner": "",
        "joinerType": "str",
        "accumulate": false,
        "timeout": "1",
        "count": "",
        "reduceRight": false,
        "reduceExp": "",
        "reduceInit": "",
        "reduceInitType": "",
        "reduceFixup": "",
        "x": 530,
        "y": 420,
        "wires": [
            [
                "c020b77f2c989387"
            ]
        ]
    },
    {
        "id": "c020b77f2c989387",
        "type": "function",
        "z": "3003f3c29df96849",
        "name": "Zähler",
        "func": "var eingang = msg.payload.length;\nif ( eingang === 1 ) {return {payload: 1 } }\nif ( eingang === 2 ) {return {payload: 2 } }\nif ( eingang === 3 ) {return {payload: 3 } }\nelse\nreturn{payload: 5 }",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 650,
        "y": 420,
        "wires": [
            [
                "71d63bcff43de0b9"
            ]
        ]
    },
    {
        "id": "6ca899ac64322255",
        "type": "function",
        "z": "3003f3c29df96849",
        "name": "",
        "func": "var FT55 = global.get(\"0101604d\");\nvar payload = msg.payload\nif ( payload === 1 && FT55 === 0) {return {payload: \"01-1\" } }\nelse\nreturn{payload: \"01-4\" }",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 920,
        "y": 420,
        "wires": [
            [
                "a386ab337d0a974f"
            ]
        ]
    },
    {
        "id": "71d63bcff43de0b9",
        "type": "switch",
        "z": "3003f3c29df96849",
        "name": "",
        "property": "payload",
        "propertyType": "msg",
        "rules": [
            {
                "t": "gt",
                "v": "1",
                "vt": "num"
            },
            {
                "t": "lte",
                "v": "1",
                "vt": "num"
            }
        ],
        "checkall": "true",
        "repair": false,
        "outputs": 2,
        "x": 770,
        "y": 420,
        "wires": [
            [],
            [
                "6ca899ac64322255"
            ]
        ]
    },
    {
        "id": "6f4b9646021e0717",
        "type": "function",
        "z": "3003f3c29df96849",
        "name": "function",
        "func": "if (msg.payload === \"on\"){\nreturn [{payload: 1}];}\nif (msg.payload === \"off\"){\nreturn [null,{payload: 0}];}",
        "outputs": 2,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 360,
        "y": 420,
        "wires": [
            [
                "9cb5b6597fa7acab",
                "c0f02da063419dc8"
            ],
            [
                "9cb5b6597fa7acab"
            ]
        ]
    },
    {
        "id": "9750cc3bd7c0888d",
        "type": "server-state-changed",
        "z": "3003f3c29df96849",
        "name": "FT55 - Esszimmer",
        "server": "e47ef4222ce3eae7",
        "version": 6,
        "outputs": 1,
        "exposeAsEntityConfig": "",
        "entities": {
            "entity": [
                "binary_sensor.e2m_ft55_ez_innen_ai_pressed"
            ],
            "substring": [],
            "regex": []
        },
        "outputInitially": false,
        "stateType": "str",
        "ifState": "",
        "ifStateType": "str",
        "ifStateOperator": "is",
        "outputOnlyOnStateChange": true,
        "for": 0,
        "forType": "num",
        "forUnits": "minutes",
        "ignorePrevStateNull": false,
        "ignorePrevStateUnknown": false,
        "ignorePrevStateUnavailable": false,
        "ignoreCurrentStateUnknown": false,
        "ignoreCurrentStateUnavailable": false,
        "outputProperties": [
            {
                "property": "payload",
                "propertyType": "msg",
                "value": "",
                "valueType": "entityState"
            },
            {
                "property": "data",
                "propertyType": "msg",
                "value": "",
                "valueType": "eventData"
            },
            {
                "property": "topic",
                "propertyType": "msg",
                "value": "",
                "valueType": "triggerId"
            }
        ],
        "x": 150,
        "y": 420,
        "wires": [
            [
                "6f4b9646021e0717"
            ]
        ]
    },
    {
        "id": "9ec0115c.8f7488",
        "type": "mqtt-broker",
        "name": "",
        "broker": "192.168.1.27",
        "port": "1883",
        "clientid": "",
        "autoConnect": true,
        "usetls": false,
        "compatmode": false,
        "protocolVersion": 4,
        "keepalive": "60",
        "cleansession": true,
        "autoUnsubscribe": true,
        "birthTopic": "",
        "birthQos": "0",
        "birthPayload": "",
        "closeTopic": "",
        "closeQos": "0",
        "closePayload": "",
        "willTopic": "",
        "willQos": "0",
        "willPayload": ""
    },
    {
        "id": "e47ef4222ce3eae7",
        "type": "server",
        "d": true,
        "name": "Home Assistant",
        "version": 5,
        "addon": false,
        "rejectUnauthorizedCerts": false,
        "ha_boolean": "y|yes|true|on|home|open",
        "connectionDelay": true,
        "cacheJson": true,
        "heartbeat": false,
        "heartbeatInterval": "30",
        "areaSelector": "friendlyName",
        "deviceSelector": "friendlyName",
        "entitySelector": "friendlyName",
        "statusSeparator": "at: ",
        "statusYear": "hidden",
        "statusMonth": "short",
        "statusDay": "numeric",
        "statusHourCycle": "h23",
        "statusTimeFormat": "h:m",
        "enableGlobalContextStore": true
    }
]

Environment Information

Version: 0.74.2

No Home Assistant server configured

Node-RED version: 4.0.8
Docker: no
Add-on: no

Node.js version: v20.18.1 x64 linux
OS: Linux 6.8.12-5-pve x64

Additional context

No response

[medicus07]

Faulty RAM in the HA server caused the problem with the home-assistant-websocket node in NR...

[medicus07]

Sorry, I was wrong... my server was not the problem.

The home-assistant-websocket node in version 0.74.2 causes massive traffic and crashes HA.

2024-12-29 08:23:57.890 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection] [547778350528] ... from 192.168.1.151: Client unable to keep up with pending messages. Reached 4096 pending messages. The system's load is too high or an integration is misbehaving; Last message was: b'{"type":"event","event":{"event_type":"state_changed","data":{"entity_id":"sensor.94a7f1bf_f653d29a_browser_height","old_state":null,"new_state":{"entity_id":"sensor.94a7f1bf_f653d29a_browser_height","state":"unavailable","attributes":{"restored":true,"icon":"mdi:arrow-up-down","friendly_name":"Browser height","supported_features":0,"unit_of_measurement":"px"},"last_changed":"2024-12-29T07:23:57.890369+00:00","last_reported":"2024-12-29T07:23:57.890369+00:00","last_updated":"2024-12-29T07:23:57.890369+00:00","context":{"id":"01JG8MSQJ20HY7XYJ5MA4V4Z8Z","parent_id":null,"user_id":null}}},"origin":"LOCAL","time_fired":"2024-12-29T07:23:57.890369+00:00","context":{"id":"01JG8MSQJ20HY7XYJ5MA4V4Z8Z","parent_id":null,"user_id":null}},"id":4}'

[zachowj]

Try connecting Node-RED directly to Home Assistant using its IP address and port instead of the supervisor proxy, and use a long-lived access token for authentication.

https://zachowj.github.io/node-red-contrib-home-assistant-websocket/guide/#configuration

[medicus07]

Now I have clearly identified the problem...
There is an interaction with another HA HASS plugin (https://github.com/thomasloven/hass-browser_mod)...
The Web API was creating more and more devices/entities for browser_mod, overloading the system. In the end, the browser_mod plugin had 25,000 devices and almost 100,000 entities.
After deleting the plugin with all entities, the problem was solved and has not reappeared.
In the end, I don't know if it was caused by the home-assistant-websocket node or by changes to the HA core in the Web API area.
I have reported the problem to the browser_mod developer. Thanks in any case.

[zachowj]

Thank you for following up with your findings!

This package, node-red-contrib-home-assistant-websocket, doesn’t request any more data than what is typically accessed when using the Home Assistant UI. The main difference is that, when using the Node-RED add-on, all requests are proxied through the supervisor proxy. From my experience, this proxying can slow down the data transfer between Home Assistant and Node-RED, which may lead to issues like the one you encountered when dealing with a large number of entities.

One solution that has worked for others in similar situations is connecting directly to Home Assistant rather than through the supervisor proxy. This bypasses the additional overhead and often improves performance.

Unfortunately, there isn’t much more that can be done from this package’s side to handle such scenarios.