spurious handshake failed errors are actually expected, not errors

[qrkourier]

Some of the controller's TLS listeners accept client certs as well as clients without client certs, e.g., enrollees, and jwt or password authenticators, including ziti edge login.

Each time one of these legitimate clients negotiates server TLS (not mTLS) with the client or management APIs, the controller logs an error-level message reporting this completely expected and normal event, that the client did not present a client certificate.

This causes significant confusion for users who understandably expect error-level messages to indicate a problem, so they often report these handshake "errors" when asking for help using ziti.

Here's an example log message in the simplified text format:

ERROR transport/v2/tls.(*sharedListener).processConn [tls:0.0.0.0:8080]: {remote=[10.89.1.2:46914] error=[EOF]} handshake failed

Here's how it's presented in JSON format:

{"_context":"tls:0.0.0.0:1280","error":"remote error: tls: bad certificate","file":"github.com/openziti/transport/[email protected]/tls/listener.go:257","func":"github.com/openziti/transport/v2/tls.(*sharedListener).processConn","level":"error","msg":"handshake failed","remote":"[::1]:44242","time":"2024-10-15T17:44:12.823Z"}

[qrkourier]

Controllers and routers emit these misleading "errors" when any REST endpoint is accessed by a legitimate HTTP client, e.g., a readiness or liveness probe in the deployed environment.