Negative 08-ssl-connect-cert-auth*.py tests fail because of exception race under high load

[narc-Ontakac2]

If you create high load with something like stress -c 16 -i 4 the negative 08-ssl-connect-cert-auth*.py tests often fail because the IO error 104 outruns the SSL error:

./08-ssl-connect-cert-auth-expired.py
1737794412: The 'port' option is now deprecated and will be removed in a future
version. Please use 'listener' instead.
1737794412: mosquitto version 2.0.20 starting
1737794412: Config loaded from 08-ssl-connect-cert-auth-expired.conf.
1737794412: Opening ipv4 listen socket on port 1888.
1737794412: Opening ipv6 listen socket on port 1888.
1737794412: Opening ipv4 listen socket on port 1889.
1737794412: Opening ipv6 listen socket on port 1889.
1737794412: mosquitto version 2.0.20 running
1737794413: New connection from ::1:46892 on port 1889.
1737794413: Client <unknown> closed its connection.
1737794413: New connection from 127.0.0.1:57692 on port 1888.
1737794413: OpenSSL Error[0]: error:0A000086:SSL routines::certificate verify
failed
1737794413: Client <unknown> disconnected: Protocol error.
1737794413: mosquitto version 2.0.20 terminating

Traceback (most recent call last):
  File "/tmp/autopkgtest.VIjXET/build.jf2/src/test/broker/./08-ssl-connect-
cert-auth-expired.py", line 40, in <module>
    mosq_test.do_send_receive(ssock, connect_packet, "", "connack")
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/autopkgtest.VIjXET/build.jf2/src/test/mosq_test.py", line 136, in
do_send_receive
    sent = sock.send(send_packet[total_sent:])
  File "/usr/lib/python3.13/ssl.py", line 1232, in send
    return self._sslobj.write(data)
           ~~~~~~~~~~~~~~~~~~^^^^^^
ConnectionResetError: [Errno 104] Connection reset by peer
make: *** [Makefile:154: 08] Error 1

I am still unsure where the race takes place, this might be a python issue. It might however also be a mosquitto server issue, which would make it more serious.

Any opinions on that are appreciated.

See also Debian bug #1094157.

[ralight]

Thanks for this. It's not a concern in itself. What's happening is we try and send a CONNECT packet. If that happens before the socket is closed then no exception is raised, if the socket is closed already because of the load then it produces this error. It's perfectly valid though. I've pushed a fix for the two tests where I could reproduce it. If you still get other failures can you please attach the logs for them?

[narc-Ontakac2]

If it not a mosquitto server issue I can simply fix it by also catching the errno 104 IO exception. You'll get a pull request within about a week. I didn't do this yet because I did not want to hide a potentially serious bug.