Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send events from pipeline to the listener #2245

Merged
merged 2 commits into from
Oct 10, 2024
Merged

Send events from pipeline to the listener #2245

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c55967e
Send events from pipeline to the listener
afrittoli Oct 10, 2024
8c60766
Updates the events docs for dogfooding
afrittoli Oct 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
180 changes: 3 additions & 177 deletions docs/dogfooding.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,187 +201,13 @@ spec:

Tekton Pipelines is configured in the `dogfooding` cluster to generate `CloudEvents`
which are sent every time a `TaskRun` or `PipelineRun` is executed.
`CloudEvents` are sent by Tekton Pipelines to an event broker. `Trigger` resources
can be defined to pick-up events from broken and have them delivered to consumers.

### CloudEvents Broker

The broker installed is based on Knative Eventing running on top of a Kafka backend.
Knative Eventing is installed following the [official guide](https://knative.dev/docs/install/eventing/install-eventing-with-yaml/)
from the Knative project:

```shell
# Install the CRDs
kubectl apply -f https://github.com/knative/eventing/releases/download/knative-v1.0.0/eventing-crds.yaml

# Install the core components
kubectl apply -f https://github.com/knative/eventing/releases/download/knative-v1.0.0/eventing-core.yaml

# Verify the installation
kubectl get pods -n knative-eventing
```

The Kafka backend is installed, as recommended in the Knative guide, using the [strimzi](https://strimzi.io/quickstarts/) operator:

```shell
# Create the namespace
kubectl create namespace kafka

# Install in the kafka namespace
kubectl create -f 'https://strimzi.io/install/latest?namespace=kafka' -n kafka

# Apply the `Kafka` Cluster CR file
kubectl apply -f https://strimzi.io/examples/latest/kafka/kafka-persistent-single.yaml -n kafka

# Verify the installation
kubectl wait kafka/my-cluster --for=condition=Ready --timeout=300s -n kafka
```

A [Knative Channel](https://github.com/knative-sandbox/eventing-kafka) is installed next:

```shell
# Install the Kafka "Consolidated" Channel
kubectl apply -f https://storage.googleapis.com/knative-nightly/eventing-kafka/latest/channel-consolidated.yaml

# Edit the "config-kafka" config-map in the "knative-eventing" namespace
# Replace "REPLACE_WITH_CLUSTER_URL" with my-cluster-kafka-bootstrap.kafka:9092/
kubectl edit cm/config-kafka -n knative-eventing
```

Install the [Knative Kafka Broker](https://knative.dev/docs/install/eventing/install-eventing-with-yaml/#optional-install-a-broker-layer)
following the official guide:

```shell
# Kafka Controller
kubectl apply -f https://github.com/knative-sandbox/eventing-kafka-broker/releases/download/knative-v1.0.0/eventing-kafka-controller.yaml

# Kafka Broken Data plane
kubectl apply -f https://github.com/knative-sandbox/eventing-kafka-broker/releases/download/knative-v1.0.0/eventing-kafka-broker.yaml
```

Create a broker resource:

```yaml
kind: Broker
metadata:
name: default
namespace: default
spec:
config:
apiVersion: v1
kind: ConfigMap
name: kafka-broker-config
namespace: knative-eventing
delivery:
retry: 0
```

The `retry: 0` part means that event delivery won't be retried on failure.
This is required because Tekton Triggers replies to CloudEvents with a JSON body
but no CloudEvents headers, which is interpreted by the message dispatcher as
a failure - see the [feature proposal](https://github.com/tektoncd/triggers/issues/1439)
on Triggers for more details.

### Kafka UI

The [Kafka UI](https://github.com/provectus/kafka-ui) allows viewing and searching for events stored by Kafka.
Events are retained by Kafka for some time (but not indefinitely), which helps when debugging event based integrations.
The Kafka UI allows managing channels and creating new events, so it is not publicly accessible. To access
the Kafka UI, port-forward the service port:

```shell
# Set up port forwarding
export POD_NAME=$(kubectl get pods --namespace kafka -l "app.kubernetes.io/name=kafka-ui,app.kubernetes.io/instance=kafka-ui" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace kafka port-forward $POD_NAME 8080:8080

# Point the browser to http://localhost:8080
```

The Kafka UI is installed via an helm chart as recommended in the [Kubernetes installation guide](https://github.com/provectus/kafka-ui#running-in-kubernetes).

```shell
helm install kafka-ui kafka-ui/kafka-ui --set envs.config.KAFKA_CLUSTERS_0_NAME=my-cluster --set envs.config.KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=my-cluster-kafka-bootstrap:9092 --set envs.config.KAFKA_CLUSTERS_0_ZOOKEEPER=my-cluster-zookeeper-nodes:2181 --namespace kafka
```
`CloudEvents` are sent by Tekton Pipelines an `EventListener` called `tekton-events`.

### CloudEvents Producer

Tekton Pipelines is the only `CloudEvents` producer in the cluster. It's [configured](../tekton/cd/pipeline/overlays/dogfooding/config-defaults.yaml) to send all events to the broker:
Tekton Pipelines is the only `CloudEvents` producer in the cluster. It's [configured](../tekton/cd/pipeline/overlays/dogfooding/config-defaults.yaml) to send all events to the event listener:

```yaml
data:
default-cloud-events-sink: http://kafka-broker-ingress.knative-eventing.svc.cluster.local/default/default
default-cloud-events-sink: http://el-tekton-events.default:8080
```

### CloudEvents Consumers

`CloudEvents` are consumed from the broker via a Knative Eventing CRD called `Trigger`.
The `dogfooding` cluster is setup so that all `TaskRun` start, running and finish events are forwarded from the
broker to the `tekton-events` event listener, in the `default` namespace.
This initial filtering of events allows to reduce the load on the event listener.

The following `Triggers` are defined in the cluster:

```yaml
apiVersion: eventing.knative.dev/v1
kind: Trigger
metadata:
name: taskrun-start-events-to-tekton-events-el
namespace: default
spec:
broker: default
filter:
attributes:
type: dev.tekton.event.taskrun.started.v1
subscriber:
uri: http://el-tekton-events.default.svc.cluster.local:8080
---
apiVersion: eventing.knative.dev/v1
kind: Trigger
metadata:
name: taskrun-running-events-to-tekton-events-el
namespace: default
spec:
broker: default
filter:
attributes:
type: dev.tekton.event.taskrun.running.v1
subscriber:
uri: http://el-tekton-events.default.svc.cluster.local:8080
---
apiVersion: eventing.knative.dev/v1
kind: Trigger
metadata:
name: taskrun-successful-events-to-tekton-events-el
namespace: default
spec:
broker: default
filter:
attributes:
type: dev.tekton.event.taskrun.successful.v1
subscriber:
uri: http://el-tekton-events.default.svc.cluster.local:8080
---
apiVersion: eventing.knative.dev/v1
kind: Trigger
metadata:
name: taskrun-failed-events-to-tekton-events-el
namespace: default
spec:
broker: default
filter:
attributes:
type: dev.tekton.event.taskrun.failed.v1
subscriber:
uri: http://el-tekton-events.default.svc.cluster.local:8080
```

### Troubleshooting Kafka

Occasionally, the Kafka cluster may stop working. Connecting via the Kafka UI
shows the cluster as down. The `el-tekton-events` deployment logs don't get
any new entry.

The kafka cluster logs show an error related to TLS certificates.
The solution in this case is to kill all `Pods` in the `kafka` namespace and
wait for things to start working again.

2 changes: 1 addition & 1 deletion tekton/cd/pipeline/overlays/dogfooding/config-defaults.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ metadata:
name: config-defaults
namespace: tekton-pipelines
data:
default-cloud-events-sink: http://kafka-broker-ingress.knative-eventing.svc.cluster.local/default/default
default-cloud-events-sink: http://el-tekton-events.default:8080