Access log for multi-zone traffic (SNI)

[bcollard]

Description

Given service A in zone A, and in zone B the following services:

• service B, version 1
• service B, version 2

The traffic will leave service A, then go through service A sidecar (outbound TCP listener), then (no egress here) zone B ingress, then service B v1 or v2 sidecar

1st demand
I wish I could know where the traffic is heading to when looking at service A's sidecar logs.
The %UPSTREAM_CLUSTER% log command is missing. With that, I would know the upstream cluster name.

When the traffic is HTTP, we can also do weighted traffic, as shown in this excerpt, and knowing which cluster is targeted when looking at the access log would be helpful.

 "route": {
                  "weightedClusters": {
                   "clusters": [
                    {
                     "name": "backend_kuma-demo_svc_3001-_0_",
                     "weight": 20
                    },
                    {
                     "name": "backend_kuma-demo_svc_3001-_1_",
                     "weight": 40
                    },
                    {
                     "name": "backend_kuma-demo_svc_3001-_2_",
                     "weight": 40
                    }
                   ]

2nd demand
On the zone B ingress pod, we could log the SNI header. Example with a service having several versions of it: backend_kuma-demo_svc_3001{mesh=default,version=v1}

Suggested log format on the Ingress listener (access log is available at the Listener level, or at the TCP filter level)
[%START_TIME%] %REQUESTED_SERVER_NAME% %PROTOCOL% %RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %UPSTREAM_CLUSTER% \"%UPSTREAM_HOST%\"\n"

%REQUESTED_SERVER_NAME% holds the SNI

[jakubdyszkiewicz]

Triage:

1. Needs reproducing. Maybe it's just a matter of setting proper format
2. We don't have a way to target zone proxies by policies.

[michaelbeaumont]

@bcollard

What do you mean by

The %UPSTREAM_CLUSTER% log command is missing.
?

Have you tried changing the log format to include %UPSTREAM_CLUSTER%?

[michaelbeaumont]

Opened #8417

[slonka]

@bcollard re-posting Mike's question:

Have you tried changing the log format to include %UPSTREAM_CLUSTER%?

[bcollard]

yes, I found this snippet that works:

apiVersion: kuma.io/v1alpha1
kind: MeshAccessLog
metadata:
  name: default
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default
spec:
  targetRef:
    kind: Mesh
  from:
    - targetRef:
        kind: Mesh
      default:
        backends:
          - type: File
            file:
              path: /dev/stdout
  to:
    - targetRef:
        kind: Mesh
      default:
        backends:
          - type: File
            file:
              path: /dev/stdout
              format:
                type: Plain
                plain: '[%START_TIME%] %KUMA_MESH% "%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%" %RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% "%REQ(X-FORWARDED-FOR)%" "%REQ(USER-AGENT)%" "%REQ(X-REQUEST-ID)%" "%REQ(:AUTHORITY)%" "%KUMA_SOURCE_SERVICE%" "%KUMA_DESTINATION_SERVICE%" "%KUMA_SOURCE_ADDRESS_WITHOUT_PORT%" "%UPSTREAM_HOST%" - upstream cluster %UPSTREAM_CLUSTER%'

So ✅ for the 1st demand.

Any plan to allow customization of the log format on the zone ingress (and egress)?

[jakubdyszkiewicz]

Closing, because 1st is solved and 2nd will be covered in #8417