Malformed UTF-8

[benbecke]

Description of the problem

From time to time when the esp restarts or reconnects to wifi i get the following message on mqtt broker when the client wants to connect. Seems that the client id is somehow malformed !?

1676553646: New connection from 192.168.195.60:56476 on port 1883.
1676553646: Bad socket read/write on client <unknown>: Malformed UTF-8

This is taking some time with this error and suddenly the error stops and everything is working fine.
Then suddently the problem again occurs

Versions

• ESPMQTTClient lib: 1.13.3
• PubSubClient lib: v2.8
• Arduino ESP32 / ESP8622 core: -2.5.0-

Hardware

• Board type : ESP8266
• Board model : Nodemcu 1.0

C++ code

const String boardNumber = "071"; 
const String clientStr = "Client"+boardNumber;

#include "EspMQTTClient.h"
EspMQTTClient client(
  "IOT",
  "XXXXXXX",
  "192.168.195.249",  // MQTT Broker server ip
  //"MQTTUsername",   // Can be omitted if not needed
  //"MQTTPassword",   // Can be omitted if not needed
  clientStr.c_str(),     // Client name that uniquely identify your device
  1883              // The MQTT port, default to 1883. this line can be omitted  
);

[EdJoPaTo]

Is this also an issue with the PubSubClient library directly? This library uses it for MQTT handling so I would assume this might be an issue over there.

---

Looking at your code I have some completely unrelated comment: I do like to use defines for strings as string concats are at compile time and a bit easier to write and handle compared to the String stuff that happens on execution:

#define BOARD_NUMBER "071"
#define CLIENT_NAME "Client" BOARD_NUMBER

EspMQTTClient client(
 …
 CLIENT_NAME,
 …
);

[benbecke]

i had a impementation before just using the pubsubclient lib. There i didn´t get this error :-/
But i had problems in handling the wifi therefore i switched to this one.
But maybe i am wrong and i only didn´t see the error at all before...

Ok thanks for this hint i will change that.
Is there a way to direktly trace what messages are sent at mqtt connection time to see where the malformed value should be ?

[EdJoPaTo]

you can enable client.enableDebuggingMessages() in the setup which shows the received and published MQTT messages via Serial

[benbecke]

ok i did that before but its not showing the broker connection stuff. Only the publish and subscribe messages.
Somehow i have to find out what is wrong in the initial connection message to the broker

[benbecke]

So interesting. The issue occurs for many hours and then suddenly stops. Then MQTT is perfectly working... After some time issue again and after some time suddenly stops...

[Sperryfreak01]

I am running into similar issues, I am still narrowing down the issue but I see the same issue even if I don't construct the string. client name ex: "sperryfreak01" -- I still get UTF 8 errors. I think it has to do with some kind of memory being freed while the library is still pointed at it. It only started for me when I switched over to the steps in #26

This code

EspMQTTClient* client;

char SSID[] =  WIFI_SSID;
char WIFIPASS[] = WIFI_PASSWORD;
char BROKER[] = MQTT_BROKER;
char MQTTu[] = MQTT_USER;   // Can be omitted if not needed
char MQTTp[] = MQTT_PASSWORD;
char mqttuser[] = "green01234567"; 
const char* deviceID;
const char* deviceChannel;  

void setup(){
Serial.begin(115200);
 //Retrieve and build the MAC strign so we can use it later as a MQTT device identifier
  macLow  = ESP.getEfuseMac() & 0xFFFFFFFF; 
  macHigh = ( ESP.getEfuseMac() >> 32 ) % 0xFFFFFFFF;
  fullMAC = ESP.getEfuseMac();
  deviceID = String(String(macLow) + String(macHigh)).c_str();

  Serial.println("mqttuser: ");
  strcpy (mqttuser,"");
  strcat (mqttuser, "green");
  strcat (mqttuser, String(macLow).c_str());
  
  String tmpdeviceChannel = String("greengame/events/" + String(macLow) + String(macHigh)); //MQTT channel for this specific device, used to post status msgs, logs, targeted OTAs, etc.
  deviceChannel = tmpdeviceChannel.c_str();
 
 
  client = new EspMQTTClient(
    SSID, // TODO #1 Change to allow user to set wifi password
    WIFIPASS,
    BROKER,  // MQTT Broker server ip
    MQTTu,   // Can be omitted if not needed
    MQTTp,   // Can be omitted if not needed
    mqttuser,     // Client name that uniquely identify your device  #TODO #2 make MQTT login client name dynamic somehow
    1883              // The MQTT port, default to 1883. this line can be omitted
    );

  // Optional functionalities of EspMQTTClient
  client->enableDebuggingMessages(); // Enable debugging messages sent to serial output
  client->enableHTTPWebUpdater(); // Enable the web updater. User and password default to values of MQTTUsername and MQTTPassword. These can be overridden with enableHTTPWebUpdater("user", "password").
  client->enableOTA(); // Enable OTA (Over The Air) updates. Password defaults to MQTTPassword. Port is the default OTA port. Can be overridden with enableOTA("password", port).
  client->enableLastWillMessage(deviceChannel, "Disconnected");  // You can activate the retain flag by setting the third parameter to true
 }

Nets me these messages

Debug Serial:

MQTT: Connecting to broker "mattlovett.com" with client name "green51938612" and username "[correct but removed]" ... (160.796000s)unable to connect (175.803000s), reason: MQTT_CONNECTION_TIMEOUT

Mosquito:

2023-04-02 11:30:56: New connection from 192.168.2.1:54575 on port 1883.
2023-04-02 11:30:56: Bad socket read/write on client <unknown>: Malformed UTF-8

[Sperryfreak01]

Well I figured it out, I was using "strings" (my python riddled brain isn't sure what is what anymore) as const char* and it was causing problems. After converting everything to char var[] or char var[x] it all started working flawlessly.