Making necessary changes in the running-functional-tests document. Also doing some cleanup. #8087

ytimocin · 2024-11-25T06:44:55Z

Description

Making necessary changes in the running-functional-tests document. Also doing some cleanup:

Removed unnecessary environment variables from the workflows.

Type of change

This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).
Fixes: #issue_number

Contributor checklist

Please verify that the PR meets the following requirements, where applicable:

An overview of proposed schema changes is included in a linked GitHub issue.
A design document PR is created in the design-notes repository, if new APIs are being introduced.
If applicable, design document has been reviewed and approved by Radius maintainers/approvers.
A PR for the samples repository is created, if existing samples are affected by the changes in this PR.
A PR for the documentation repository is created, if the changes in this PR affect the documentation or any user facing updates are made.
A PR for the recipes repository is created, if existing recipes are affected by the changes in this PR.

github-actions · 2024-11-25T06:59:41Z

Unit Tests

1 files ±0 289 suites ±0 5m 23s ⏱️ +9s
3 536 tests +1 3 531 ✅ ±0 4 💤 ±0 1 ❌ +1
3 535 runs ±0 3 530 ✅ - 1 4 💤 ±0 1 ❌ +1

For more details on these failures, see this check.

Results for commit 24a4e7b. ± Comparison against base commit 46dc40d.

♻️ This comment has been updated with latest results.

kachawla

Had left some comments on previous version of this PR here: #8069

ytimocin · 2024-11-25T18:32:57Z

What's the motivation for changing file name style?

I think the naming convention for sh files is underscores, dashes, or all words together. We use camel case for now. We use both underscores and dashes.

Ref: https://google.github.io/styleguide/shellguide.html#source-filenames.

I was creating new sh files for cloud and non-cloud, so I wanted to give them new names that would also be consistent with other names in our codebase.

That being said, we need to decide if we want to use underscores or dashes because we have both. We only had one camel-case and I changed that.

radius-functional-tests · 2024-11-26T18:24:39Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	radius-project/radius
Commit ref	`f8a1e10`
Unique ID	funcefc4ddb0b6
Image tag	pr-funcefc4ddb0b6

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr:
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funcefc4ddb0b6
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funcefc4ddb0b6
dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-funcefc4ddb0b6
controller test image location: ghcr.io/radius-project/dev/controller:pr-funcefc4ddb0b6
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funcefc4ddb0b6
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ corerp-cloud functional tests succeeded
✅ ucp-cloud functional tests succeeded

ytimocin · 2024-11-26T18:25:55Z

.github/workflows/functional-test-cloud.yaml

-          # AZURE_MSSQL_RESOURCE_ID
-          # AZURE_MSSQL_USERNAME
-          # AZURE_MSSQL_PASSWORD


I see these are not being used anymore.

ytimocin · 2024-11-26T18:26:27Z

.github/workflows/functional-test-cloud.yaml

@@ -729,9 +726,6 @@ jobs:
          AWS_ACCESS_KEY_ID: ${{ secrets.FUNCTEST_AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.FUNCTEST_AWS_SECRET_ACCESS_KEY }}
          AWS_REGION: ${{ env.AWS_REGION }}
-          RADIUS_SAMPLES_REPO_ROOT: ${{ github.workspace }}/samples


This is only being used in the noncloud tests.

ytimocin · 2024-11-26T18:26:50Z

.github/workflows/functional-test-noncloud.yaml

-          TEMP_CERT_DIR: ${{ steps.create-local-registry.outputs.temp-cert-dir }}
-          SSL_CERT_FILE: ${{ steps.create-local-registry.outputs.temp-cert-dir }}/certs/${{ env.LOCAL_REGISTRY_SERVER }}/client.crt


Not being used.

ytimocin · 2024-11-26T18:28:52Z

test/testrecipes/test-terraform-recipes/README.md

radius-functional-tests · 2024-12-03T14:00:14Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	radius-project/radius
Commit ref	`223c606`
Unique ID	funcec1695f50d
Image tag	pr-funcec1695f50d

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr:
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funcec1695f50d
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funcec1695f50d
dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-funcec1695f50d
controller test image location: ghcr.io/radius-project/dev/controller:pr-funcec1695f50d
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funcec1695f50d
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp-cloud functional tests...
⌛ Starting corerp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

brooke-hamilton

I was able to get all five services running with a debugger following these instructions. 🎆

A helpful note in troubleshooting would be to say that after launching the debugger, five items should be visible in the VS Code call stack panel (in the debugging tab):

Launch Deployment Engine
Launch Dynamic RP
Launch Applications RP
Launch Controller
Launch UCP

Initially, I had a failure because I was deploying a kind cluster using the instructions in our docs site to map ports, but those ports were conflicting with the ports the debugger was attaching to, causing 503 errors. This became apparent when I realized that the Launch Applications RP debugging configuration was failing and would disappear from the list of processes.

brooke-hamilton · 2024-12-12T14:44:04Z

...ntributing/contributing-code/contributing-code-control-plane/running-controlplane-locally.md

@@ -32,7 +32,7 @@ If you need to manually test APIs you can reach them at the following endpoints
 4. Install .NET 6.0 SDK - <https://dotnet.microsoft.com/en-us/download/dotnet/6.0>.
 5. Install C# VS Code extension - <https://marketplace.visualstudio.com/items?itemName=ms-dotnettools.csharp>.
 6. (Optional) Configure any cloud provider credentials you want to use for developing Radius.
-  
+


We should explain in step 1 that the Kubernetes cluster should be created with defaults, i.e., without the port mappings required in the docs because the debugger will attempt to attach to the same ports, and they will already be used by the cluster, resulting in 503 errors when running rad commands.

brooke-hamilton · 2024-12-12T14:49:35Z

...ntributing/contributing-code/contributing-code-control-plane/running-controlplane-locally.md


- This example adds a `dev` workspace:
+This example adds a `dev` workspace:

 ```yaml
 workspaces:


I was able to get the same result by adding the overrides: section to the existing default workspace, instead of adding a new workspace. It works either way, but do you think that modifying the existing workspace is a simpler way to understand it? In other words, override the ucp property of the current connection instead of creating a new workspace, like this. (I intend this comment as a discussion rather than a strong opinion on it.)

workspaces: default: default items: default: connection: context: kind-kind kind: kubernetes overrides: ucp: http://localhost:9000 environment: /planes/radius/local/resourceGroups/default/providers/Applications.Core/environments/default scope: /planes/radius/local/resourceGroups/default

brooke-hamilton · 2024-12-12T16:44:20Z

docs/contributing/contributing-code/contributing-code-tests/running-functional-tests.md

-### Run
+### Run Non-Cloud Functional Tests
+
+1. Required environment variables:


Where are the values for these found so that they can be set?

codecov · 2025-01-02T17:37:13Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 60.07%. Comparing base (710509a) to head (a617d63).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8087      +/-   ##
==========================================
- Coverage   60.09%   60.07%   -0.02%     
==========================================
  Files         584      584              
  Lines       38675    38675              
==========================================
- Hits        23240    23234       -6     
- Misses      13729    13732       +3     
- Partials     1706     1709       +3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

kachawla · 2025-01-02T17:36:25Z

.github/workflows/functional-test-cloud.yaml

@@ -86,16 +86,16 @@ env:
  # Private Git repository where terraform module for testing is stored.
  TF_RECIPE_PRIVATE_GIT_SOURCE: "git::https://github.com/radius-project/terraform-private-modules//kubernetes-redis"
  # bicep-types ACR url for uploading Radius Bicep types
-  BICEP_TYPES_REGISTRY: 'biceptypes.azurecr.io'
+  BICEP_TYPES_REGISTRY: "biceptypes.azurecr.io"


Can you give some context on why is this update needed?

This is a refactoring of the functional-test-cloud.yaml file. Not an extension but built-in feature ("shift + opt + F") in VS Code cleans it up.

kachawla · 2025-01-02T17:37:21Z

.github/workflows/functional-test-cloud.yaml

@@ -373,8 +373,8 @@ jobs:
          chmod +x ./bicep
          sudo mv ./bicep /usr/local/bin/bicep
          bicep --version
-
-      - name: Publish bicep types 
+


Are these tabs/spaces automatically being updated by an extension you are using?

This is a refactoring of the functional-test-cloud.yaml file. Not an extension but built-in feature ("shift + opt + F") in VS Code cleans it up.

kachawla · 2025-01-02T18:07:14Z

.github/workflows/functional-test-noncloud.yaml

@@ -364,7 +362,6 @@ jobs:
          RADIUS_SAMPLES_REPO_ROOT: ${{ github.workspace }}/samples
          BICEP_RECIPE_REGISTRY: "${{ env.LOCAL_REGISTRY_NAME }}:${{ env.LOCAL_REGISTRY_PORT }}"
          BICEP_RECIPE_TAG_VERSION: ${{ env.BICEP_RECIPE_TAG_VERSION }}
-          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}


Where does this gets set?

This yaml is for non-cloud tests. If you see the references of GH_TOKEN, you can see that it is only being used in a cloud test.

kachawla · 2025-01-02T18:16:58Z

test/execute-functional-tests-cloud.sh

+rad group create default
+rad env create kind-radius
+rad group create kind-radius
+
+# Check if TF_RECIPE_MODULE_SERVER_URL environment variable is set
+if [ -z "$TF_RECIPE_MODULE_SERVER_URL" ]; then
+  echo "Error: TF_RECIPE_MODULE_SERVER_URL environment variable is not set."
+  exit 1
+fi
+
+# Check if DOCKER_REGISTRY environment variable is set
+if [ -z "$DOCKER_REGISTRY" ]; then
+  echo "Error: DOCKER_REGISTRY environment variable is not set."
+  exit 1
+fi
+
+# Check if BICEP_RECIPE_REGISTRY environment variable is set
+if [ -z "$BICEP_RECIPE_REGISTRY" ]; then


Could you remind where is this handled currently?

This is not being handled anywhere currently in the helper script.

How's everyone running functional tests today? Is this documented somewhere for manual setup?

kachawla · 2025-01-02T18:18:09Z

test/execute-functional-tests-cloud.sh

+# Make sure that you have default environment in the default group.
+# The way to check that is to run the following commands:
+# `rad group switch default`
+# `rad env list` and check if there is an environment named `default`.
+# If not, create one using `rad env create default`.


This script seems to be creating group and env below L30-L32, so why is this manual check needed?

Just being extra defensive.

That seems unnecessary here.

kachawla · 2025-01-02T18:18:29Z

test/execute-functional-tests-cloud.sh

+# `rad env list` and check if there is an environment named `default`.
+# If not, create one using `rad env create default`.
+
+# To run functional tests, you have to be in `kind-radius` group and


Could you share context on why this is the case?

Because we have some hard-coded environment names in our functional tests: https://github.com/search?q=repo%3Aradius-project%2Fradius+%22kind-radius%22&type=code.

Can you update the comment to include the context?

kachawla · 2025-01-02T18:18:42Z

test/execute-functional-tests-cloud.sh

+rad group create default
+rad env create kind-radius
+rad group create kind-radius


Why does it need to create two groups?

Please see: https://github.com/search?q=repo%3Aradius-project%2Fradius+%22kind-radius%22&type=code.

The link you referenced points to all occurrences of kind-radius - can you help me understand how does that result into requiring two groups to be created?

kachawla · 2025-01-02T18:20:08Z

test/testrecipes/test-terraform-recipes/README.md

@@ -30,12 +30,12 @@ Then you can access the recipe download at:
 http://localhost:8999/<recipe_name>.zip
 ```

---- 
+---


What does this update do?

This is a refactoring of the README.md. That extra - was unnecessary and "shift + opt + F" in VS Code cleans it up.

…so doing some cleanup. Signed-off-by: ytimocin <[email protected]>

ytimocin had a problem deploying to functional-tests November 25, 2024 06:45 — with GitHub Actions Failure

ytimocin temporarily deployed to publish-bicep November 25, 2024 06:45 — with GitHub Actions Inactive

ytimocin requested review from brooke-hamilton and vishwahiremat November 25, 2024 06:45

ytimocin mentioned this pull request Nov 25, 2024

Updating the Running Cloud Functional Tests and adding the Running Non-cloud Functional Tests docs #8069

Closed

6 tasks

ytimocin force-pushed the ytimocin/test-doc-update branch from 67cae74 to 7aeb2b6 Compare November 25, 2024 06:47

ytimocin had a problem deploying to functional-tests November 25, 2024 06:47 — with GitHub Actions Failure

ytimocin temporarily deployed to publish-bicep November 25, 2024 06:48 — with GitHub Actions Inactive

kachawla reviewed Nov 25, 2024

View reviewed changes

ytimocin force-pushed the ytimocin/test-doc-update branch from 7aeb2b6 to f8a1e10 Compare November 26, 2024 18:23

ytimocin marked this pull request as ready for review November 26, 2024 18:23

ytimocin requested review from a team as code owners November 26, 2024 18:23

ytimocin temporarily deployed to functional-tests November 26, 2024 18:23 — with GitHub Actions Inactive

ytimocin temporarily deployed to publish-bicep November 26, 2024 18:23 — with GitHub Actions Inactive

ytimocin commented Nov 26, 2024

View reviewed changes

ytimocin force-pushed the ytimocin/test-doc-update branch from f8a1e10 to 223c606 Compare December 3, 2024 01:45

ytimocin temporarily deployed to functional-tests December 3, 2024 01:45 — with GitHub Actions Inactive

ytimocin temporarily deployed to publish-bicep December 3, 2024 01:45 — with GitHub Actions Inactive

brooke-hamilton suggested changes Dec 12, 2024

View reviewed changes

brooke-hamilton reviewed Dec 12, 2024

View reviewed changes

ytimocin force-pushed the ytimocin/test-doc-update branch from 223c606 to 10ec437 Compare January 2, 2025 17:03

ytimocin requested a deployment to functional-tests January 2, 2025 17:03 — with GitHub Actions Waiting

ytimocin had a problem deploying to publish-bicep January 2, 2025 17:03 — with GitHub Actions Error

ytimocin force-pushed the ytimocin/test-doc-update branch from 10ec437 to 8a17468 Compare January 2, 2025 17:04

ytimocin requested a deployment to functional-tests January 2, 2025 17:04 — with GitHub Actions Waiting

ytimocin temporarily deployed to publish-bicep January 2, 2025 17:05 — with GitHub Actions Inactive

ytimocin force-pushed the ytimocin/test-doc-update branch from 8a17468 to a617d63 Compare January 2, 2025 17:07

ytimocin requested a deployment to functional-tests January 2, 2025 17:07 — with GitHub Actions Waiting

ytimocin temporarily deployed to publish-bicep January 2, 2025 17:08 — with GitHub Actions Inactive

kachawla reviewed Jan 2, 2025

View reviewed changes

Making necessary changes in the running-functional-tests document. Al…

24a4e7b

…so doing some cleanup. Signed-off-by: ytimocin <[email protected]>

ytimocin force-pushed the ytimocin/test-doc-update branch from a617d63 to 24a4e7b Compare January 2, 2025 23:34

ytimocin requested a deployment to functional-tests January 2, 2025 23:34 — with GitHub Actions Waiting

ytimocin temporarily deployed to publish-bicep January 2, 2025 23:34 — with GitHub Actions Inactive

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Making necessary changes in the running-functional-tests document. Also doing some cleanup. #8087

Making necessary changes in the running-functional-tests document. Also doing some cleanup. #8087

ytimocin commented Nov 25, 2024

github-actions bot commented Nov 25, 2024 •

edited

Loading

kachawla left a comment

ytimocin commented Nov 25, 2024

radius-functional-tests bot commented Nov 26, 2024 •

edited

Loading

ytimocin Nov 26, 2024

ytimocin Nov 26, 2024

ytimocin Nov 26, 2024

ytimocin Nov 26, 2024

radius-functional-tests bot commented Dec 3, 2024 •

edited

Loading

brooke-hamilton left a comment

brooke-hamilton Dec 12, 2024

brooke-hamilton Dec 12, 2024

brooke-hamilton Dec 12, 2024

codecov bot commented Jan 2, 2025

kachawla Jan 2, 2025

ytimocin Jan 2, 2025

kachawla Jan 2, 2025

ytimocin Jan 2, 2025

kachawla Jan 2, 2025

ytimocin Jan 2, 2025

kachawla Jan 2, 2025

ytimocin Jan 2, 2025

kachawla Jan 3, 2025

kachawla Jan 2, 2025

ytimocin Jan 2, 2025

kachawla Jan 3, 2025

kachawla Jan 2, 2025

ytimocin Jan 2, 2025

kachawla Jan 3, 2025

kachawla Jan 2, 2025

ytimocin Jan 2, 2025

kachawla Jan 3, 2025

kachawla Jan 2, 2025

ytimocin Jan 2, 2025

		TEMP_CERT_DIR: ${{ steps.create-local-registry.outputs.temp-cert-dir }}
		SSL_CERT_FILE: ${{ steps.create-local-registry.outputs.temp-cert-dir }}/certs/${{ env.LOCAL_REGISTRY_SERVER }}/client.crt

Making necessary changes in the running-functional-tests document. Also doing some cleanup. #8087

Are you sure you want to change the base?

Making necessary changes in the running-functional-tests document. Also doing some cleanup. #8087

Conversation

ytimocin commented Nov 25, 2024

Description

Type of change

Contributor checklist

github-actions bot commented Nov 25, 2024 • edited Loading

Unit Tests

kachawla left a comment

Choose a reason for hiding this comment

ytimocin commented Nov 25, 2024

radius-functional-tests bot commented Nov 26, 2024 • edited Loading

Radius functional test overview

Test Status

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

radius-functional-tests bot commented Dec 3, 2024 • edited Loading

Radius functional test overview

Test Status

brooke-hamilton left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jan 2, 2025

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Nov 25, 2024 •

edited

Loading

radius-functional-tests bot commented Nov 26, 2024 •

edited

Loading

radius-functional-tests bot commented Dec 3, 2024 •

edited

Loading