End-to-end Tests of Kyma Environment Broker

Overview

The following end-to-end (E2E) tests cover Kyma Environment Broker (KEB) and SAP BTP, Kyma runtime:

skr-tests for testing the following operations on different cloud service providers: Kyma provisioning, BTP Manager Secret reconciliation, updating OIDC, updating machine type, and Kyma runtime deprovisioning
keb-endpoints-test for checking if kyma-environment-broker endpoints require authorization
skr-aws-networking for checking if provisioning a Kyma runtime with custom networking parameters works as expected
skr-trial-suspension-dev for testing the following operations: Kyma provisioning, Kyma suspension, and Kyma runtime deprovisioning
skr-aws-binding for testing the following operations: Kyma provisioning, fetching Kyma Binding, using Kyma Binding, deleting Kyma Binding, and Kyma runtime deprovisioning
provisioning-service-aws-stage for checking if Cloud Management Service Provisioning API works as expected

E2E SKR Tests

Usage

You can use the SKR test in two modes - with or without provisioning.

With Provisioning

In this mode, the test executes the following steps:

Provisions a Kyma runtime cluster.
Runs the BTP Manager Secret reconciliation test.
Runs the OIDC test.
Runs the machine type update test.
Deprovisions the Kyma runtime instance and cleans up the resources.

Without Provisioning

In this mode the test additionally needs the following environment variables:

SKIP_PROVISIONING set to true
INSTANCE_ID - the UUID of the provisioned Kyma runtime instance

In this mode, the test executes the following steps:

Ensures the Kyma runtime exists.
Runs the OIDC test.
Cleans up the resources.

Test Execution

Before you run the test, prepare the .env file based on this .env.template.
To set up the environment variables in your system, run:
```
export $(xargs < .env)
```
Choose whether you want to run the test with or without provisioning.
- To run the test with provisioning, call the following target:
```
make skr
```
- To run the SKR test without provisioning, use the following command:
```
make skr SKIP_PROVISIONING=true
```

KEB Endpoints Test

Usage

The test executes the following steps:

Calls KEB endpoints without an authorization token.
Checks whether the call was rejected.

Test Execution

Before you run the test, prepare the .env file based on this .env.template.
To set up the environment variables in your system, run:
```
export $(xargs < .env)
```
Run the test scenario.
```
make keb-endpoints
```

Networking Parameter Tests

Usage

The test executes the following steps:

Calls KEB endpoints with invalid networking parameters.
Checks whether the call was rejected.
Provisions a cluster with custom networking parameters.
Deprovisions the cluster.

Test Execution

Before you run the test, prepare the .env file based on this .env.template.
To set up the environment variables in your system, run:
```
export $(xargs < .env)
```
Run the test scenario:
```
make skr-networking
```

E2E SKR Suspension Test

Usage

The test executes the following steps:

Provisions a Kyma runtime cluster.
Waits until Trial Cleanup CronJob triggers suspension.
Waits until suspension succeeds.
Deprovisions the Kyma runtime instance and cleans up the resources.

Test Execution

Before you run the test, prepare the .env file based on this .env.template.
To set up the environment variables in your system, run:
```
export $(xargs < .env)
```
Run the test scenario:
```
make skr-trial-suspension
```

Binding Tests

Usage

The test executes the following steps:

Provisions a Kyma runtime cluster.
Creates a Kyma Binding and saves the returned kubeconfig.
Initializes a Kubernetes client with the returned kubeconfig.
Fetches the sap-btp-manager Secret using the Kyma Binding.
Fetches the created Kyma Binding.
Deletes the created Kyma Binding.
Tries to fetch the sap-btp-manager Secret using the deleted Kyma Binding.
Tries to create a Kyma Binding using invalid parameters.
Tests response status codes.
Tries to create more than 10 Kyma Bindings.
Deprovisions the Kyma runtime instance and cleans up the resources.

Test Execution

Before you run the test, prepare the .env file based on this .env.template.
To set up the environment variables in your system, run:
```
export $(xargs < .env)
```
Run the test scenario:
```
make skr-binding
```

Provisioning Service Tests

Usage

The test executes the following steps:

Sends a call to Provisioning API to deprovision the remaining Kyma runtime and waits until the environment is deleted if the previous test run was not able to deprovision Kyma runtime.
Sends a call to Provisioning API to provision a Kyma runtime. The test waits until the environment is created.
Creates a Kyma Binding.
Fetches the sap-btp-manager Secret using the kubeconfig from the created Kyma Binding.
Fetches the created Kyma Biding.
Deletes the created Kyma Binding.
Tries to fetch the sap-btp-manager Secret using the invalidated kubeconfig.
Tries to fetch the deleted Kyma Binding.
Sends a call to Provisioning API to deprovision the Kyma runtime. The test waits until the environment is deleted.

Test Execution

Before you run the test, prepare the .env file based on this .env.template.
To set up the environment variables in your system, run:
```
export $(xargs < .env)
```
Run the test scenario:
```
make provisioning-service
```

CI Pipelines

The tests are run daily.

keb-endpoints-test - KEB endpoints test
skr-aws-integration-dev - SKR test
skr-aws-binding - Kyma Bindings test
skr-aws-networking - networking parameters test
skr-azure-integration-dev - SKR test
skr-azure-lite-integration-dev - SKR test
skr-free-aws-integration-dev - SKR test
skr-preview-dev - SKR test
skr-sap-converged-cloud-integration-dev - SKR test
skr-trial-integration-dev - SKR test
skr-trial-suspension-dev - SKR suspension test
provisioning-service-aws-stage - Provisioning API test