[wip] add systemd services for configuration after start #980

anjannath · 2024-10-21T08:10:03Z

this adds 4 small systemd services that:

creates crc specific configurations for dnsmasq
sets a new uuid as cluster id
creates the pod for routes-controller
tries to grow the disk and filesystem
service that checks if all the cluster operators are ready

praveenkumar · 2024-10-21T08:49:38Z

systemd/crc-dnsmasq.sh

+address=/apps-crc.testing/$ip
+address=/api.crc.testing/$ip
+address=/api-int.crc.testing/$ip
+address=/$hostName.crc.testing/$iip


why we need this one for node name resolve (in OCP context) Also I think ip should be node name dummy IP which is 192.168.126.11?

yeah probably not needed now, i think this was needed before moving to SNO and in the crc code-base we still have it so added here as well

did we test it with dummy IP 192.168.126.11 ?

you mean using address=/$hostName.crc.testing/192.168.126.11? instead of use the result from hostname -i?

yes right, because this is used by internal service resolution so it might work.

systemd/crc-routes-controller.sh

systemd/crc-dnsmasq.sh

createdisk.sh

systemd/crc-dnsmasq.sh

systemd/crc-dnsmasq.service

cfergeau · 2024-11-20T14:27:32Z

systemd/ocp-cluster-status.sh

+done
+
+# need to set a marker to let `crc` know the cluster is ready
+# touch /tmp/.crc-cluster-ready


Instead of creating a file in the guest, you can also write a 'ready' notification message to a socket. podman does something similar: cfergeau@9e80530

systemd/ocp-clusterid.service

cfergeau · 2024-11-25T10:17:02Z

systemd/crc-pullsecret.sh

+    [ $retry == $max_retry ] && exit 1
+    sleep 5
+    ((retry++))
+done


I don't know if it is common/recommended to do this, but it might be possible to express "oc get secret ready" as a systemd unit

anjannath · 2024-11-26T12:43:16Z

/test e2e-snc

anjannath · 2024-11-26T15:54:35Z

/retest

systemd/ocp-userpasswords.service

systemd/ocp-cluster-ca.sh

systemd/ocp-custom-domain.sh

praveenkumar · 2024-12-13T15:03:48Z

systemd/ocp-cluster-ca.sh

+
+if [ -z $CRC_CLOUD ]; then
+    exit 1
+fi


same https://github.com/crc-org/snc/pull/980/files#r1884067998

openshift-ci · 2024-12-16T07:49:52Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from anjannath. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

anjannath · 2024-12-17T07:42:14Z

/retest

anjannath · 2024-12-17T09:26:05Z

/retest

anjannath · 2024-12-17T11:39:07Z

/retest

rohanKanojia · 2024-12-17T13:18:01Z

notes-self-sufficient.md

@@ -0,0 +1,24 @@
+# Self sufficient bundles
+
+Since release 4.19.0 of OpenShift Local, the bundles generated by `snc` contains additional systemd services to provision the cluster and removes the need of


Suggested change

Since release 4.19.0 of OpenShift Local, the bundles generated by `snc` contains additional systemd services to provision the cluster and removes the need of

Since release 4.19.0 of OpenShift Local, the bundles generated by `snc` contain additional systemd services to provision the cluster and remove the need for

rohanKanojia · 2024-12-17T13:18:34Z

notes-self-sufficient.md

+# Self sufficient bundles
+
+Since release 4.19.0 of OpenShift Local, the bundles generated by `snc` contains additional systemd services to provision the cluster and removes the need of
+an outside entity for provision the cluster, although an outside process needs to create some files on pre-defined locations inside the VM for the  systemd


Suggested change

an outside entity for provision the cluster, although an outside process needs to create some files on pre-defined locations inside the VM for the systemd

an outside entity to provision the cluster, although an outside process needs to create some files on pre-defined locations inside the VM for the systemd

rohanKanojia · 2024-12-17T13:18:57Z

notes-self-sufficient.md

+an outside entity for provision the cluster, although an outside process needs to create some files on pre-defined locations inside the VM for the  systemd
+services to do their work.
+
+## The following table lists the systemd services and the location of files they need to provision the cluster, users of SNC needs to create those files


Suggested change

## The following table lists the systemd services and the location of files they need to provision the cluster, users of SNC needs to create those files

## The following table lists the systemd services and the location of files they need to provision the cluster, users of SNC need to create those files

rohanKanojia · 2024-12-17T13:19:41Z

notes-self-sufficient.md

+
+> [!NOTE]
+> "Marker env variable" is set using an env file, if the required env variable is not set then unit is skipped
+> some units are ran only when CRC_CLOUD=1 is set, these are only needed when using the bundles with crc-cloud


Suggested change

> some units are ran only when CRC_CLOUD=1 is set, these are only needed when using the bundles with crc-cloud

> some units are run only when CRC_CLOUD=1 is set, these are only needed when using the bundles with crc-cloud

rohanKanojia · 2024-12-17T13:38:22Z

systemd/crc-cluster-status.sh

+done
+
+# need to set a marker to let `crc` know the cluster is ready
+# touch /tmp/.crc-cluster-ready


nit, Are these commented-out lines important? If not, shall we remove them?

rohanKanojia · 2024-12-17T13:38:41Z

systemd/crc-cluster-status.sh

+    return 1
+}
+
+# rm -rf /tmp/.crc-cluster-ready


https://github.com/crc-org/snc/pull/980/files#r1888535323

rohanKanojia · 2024-12-17T15:13:16Z

systemd/ocp-clusterid.sh

+retry=0
+max_retry=20
+until `oc get clusterversion > /dev/null 2>&1`
+do
+    [ $retry == $max_retry ] && exit 1
+    sleep 5
+    ((retry++))
+done


I see this block of code duplicated across various scripts where we are waiting for some resource to become available. I'm not 100% sure if it's possible to do it in SystemD setup but can we move this to a generic method that can handle various resource types? WDYT?

Maybe with https://fedoramagazine.org/systemd-template-unit-files/
Or just by using a shared script.

sure, I'll extract this into a function that we can re-use across scripts

praveenkumar · 2024-12-23T08:06:20Z

systemd/crc-pullsecret.sh

+echo "${existingPs}" | jq -e '.auths'
+
+if [[ $? != 0 ]]; then
+    pullSecretB64=$(cat /opt/crc/pull-secret)


pullSecretB64=$(cat /opt/crc/pull-secret | base64 -w0)

praveenkumar · 2024-12-23T08:07:18Z

systemd/crc-cluster-status.sh

+
+export KUBECONFIG=/opt/kubeconfig
+
+function check_cluster_unhealthy() {


name should be check_cluster_healthy and return value should be 0 in case successful and 1 in case failure.

praveenkumar · 2024-12-23T08:07:47Z

systemd/crc-cluster-status.sh

+CLUSTER_HEALTH_SLEEP=8
+CLUSTER_HEALTH_RETRIES=500
+
+while $(check_cluster_unhealthy)


this should change to while ! check_cluster_healthy

systemd/crc-pullsecret.sh

the services does the various needed tasks to setup the ocp or microshift cluster, these systemd units runs small shell scripts which are based on: https://github.com/crc-org/crc-cloud/blob/main/pkg/bundle/setup/clustersetup.sh and does the following tasks: - creates crc specific configurations for dnsmasq - sets a new uuid as cluster id - creates the pod for routes-controller - tries to grow the disk and filesystem - checks if the cluster operators are ready - adds the pull secret to the cluster - sets kubeadmin and developer user passwords - sets a custom ca for authentication - sets custom nip.io cluster domain

openshift-ci · 2025-01-08T14:59:20Z

@anjannath: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-microshift	`3881812`	link	true	`/test e2e-microshift`
ci/prow/e2e-snc	`3881812`	link	true	`/test e2e-snc`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci bot added the do-not-merge/work-in-progress label Oct 21, 2024

openshift-ci bot requested review from cfergeau and praveenkumar October 21, 2024 08:10

praveenkumar reviewed Oct 21, 2024

View reviewed changes

systemd/crc-routes-controller.sh Outdated Show resolved Hide resolved

anjannath force-pushed the issue_638 branch from 988c4df to fd280b2 Compare October 22, 2024 04:21

cfergeau reviewed Oct 22, 2024

View reviewed changes

systemd/crc-dnsmasq.sh Outdated Show resolved Hide resolved

createdisk.sh Outdated Show resolved Hide resolved

systemd/crc-dnsmasq.sh Outdated Show resolved Hide resolved

anjannath force-pushed the issue_638 branch 4 times, most recently from 8ccdb6e to bd2b783 Compare October 24, 2024 06:13

praveenkumar reviewed Oct 24, 2024

View reviewed changes

systemd/crc-dnsmasq.service Outdated Show resolved Hide resolved

anjannath force-pushed the issue_638 branch 5 times, most recently from 525b66d to 1f5383b Compare November 18, 2024 08:37

cfergeau reviewed Nov 20, 2024

View reviewed changes

systemd/ocp-clusterid.service Outdated Show resolved Hide resolved

anjannath force-pushed the issue_638 branch from 1f5383b to 4343a22 Compare November 25, 2024 08:11

cfergeau reviewed Nov 25, 2024

View reviewed changes

anjannath force-pushed the issue_638 branch from 4343a22 to c22ff87 Compare November 26, 2024 10:21

anjannath force-pushed the issue_638 branch 2 times, most recently from 9fd0545 to e6a4516 Compare November 27, 2024 09:13

praveenkumar reviewed Nov 27, 2024

View reviewed changes

systemd/ocp-userpasswords.service Outdated Show resolved Hide resolved

anjannath force-pushed the issue_638 branch from e6a4516 to 6a9aaf6 Compare November 28, 2024 09:45

praveenkumar reviewed Dec 10, 2024

View reviewed changes

systemd/ocp-cluster-ca.sh Outdated Show resolved Hide resolved

praveenkumar reviewed Dec 10, 2024

View reviewed changes

systemd/ocp-custom-domain.sh Show resolved Hide resolved

praveenkumar reviewed Dec 13, 2024

View reviewed changes

anjannath force-pushed the issue_638 branch from 47b297c to ffac727 Compare December 16, 2024 07:49

rohanKanojia reviewed Dec 17, 2024

View reviewed changes

systemd/crc-cluster-status.sh

return 1

}

# rm -rf /tmp/.crc-cluster-ready

Copy link

rohanKanojia Dec 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/crc-org/snc/pull/980/files#r1888535323

anjannath force-pushed the issue_638 branch from ffac727 to 76b364f Compare December 17, 2024 14:08

rohanKanojia reviewed Dec 17, 2024

View reviewed changes

anjannath force-pushed the issue_638 branch 3 times, most recently from 12df080 to c4d5572 Compare December 19, 2024 11:06

praveenkumar reviewed Dec 23, 2024

View reviewed changes

anjannath force-pushed the issue_638 branch from c4d5572 to 014695c Compare December 23, 2024 08:37

praveenkumar reviewed Dec 24, 2024

View reviewed changes

systemd/crc-pullsecret.sh Outdated Show resolved Hide resolved

anjannath force-pushed the issue_638 branch from 014695c to 69fdf29 Compare December 30, 2024 06:58

anjannath force-pushed the issue_638 branch 3 times, most recently from a62f49b to 0ed44f4 Compare January 7, 2025 15:59

anjannath force-pushed the issue_638 branch from 0ed44f4 to 3881812 Compare January 8, 2025 12:28

anjannath mentioned this pull request Jan 9, 2025

wip: disable post-start functions that are done in snc crc-org/crc#4560

Open

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[wip] add systemd services for configuration after start #980

[wip] add systemd services for configuration after start #980

anjannath commented Oct 21, 2024 •

edited

Loading

praveenkumar Oct 21, 2024

anjannath Oct 21, 2024

praveenkumar Dec 10, 2024

anjannath Dec 10, 2024

praveenkumar Dec 13, 2024

cfergeau Nov 20, 2024

cfergeau Nov 25, 2024

anjannath commented Nov 26, 2024

anjannath commented Nov 26, 2024

praveenkumar Dec 13, 2024

openshift-ci bot commented Dec 16, 2024

anjannath commented Dec 17, 2024

anjannath commented Dec 17, 2024

anjannath commented Dec 17, 2024

rohanKanojia Dec 17, 2024

rohanKanojia Dec 17, 2024

rohanKanojia Dec 17, 2024

rohanKanojia Dec 17, 2024

rohanKanojia Dec 17, 2024

rohanKanojia Dec 17, 2024

rohanKanojia Dec 17, 2024

cfergeau Dec 17, 2024

anjannath Dec 18, 2024

praveenkumar Dec 23, 2024

praveenkumar Dec 23, 2024

praveenkumar Dec 23, 2024

openshift-ci bot commented Jan 8, 2025

		@@ -0,0 +1,24 @@
		# Self sufficient bundles

		Since release 4.19.0 of OpenShift Local, the bundles generated by `snc` contains additional systemd services to provision the cluster and removes the need of

	an outside entity for provision the cluster, although an outside process needs to create some files on pre-defined locations inside the VM for the systemd
	an outside entity to provision the cluster, although an outside process needs to create some files on pre-defined locations inside the VM for the systemd

	## The following table lists the systemd services and the location of files they need to provision the cluster, users of SNC needs to create those files
	## The following table lists the systemd services and the location of files they need to provision the cluster, users of SNC need to create those files

	> some units are ran only when CRC_CLOUD=1 is set, these are only needed when using the bundles with crc-cloud
	> some units are run only when CRC_CLOUD=1 is set, these are only needed when using the bundles with crc-cloud


		export KUBECONFIG=/opt/kubeconfig

		function check_cluster_unhealthy() {

[wip] add systemd services for configuration after start #980

Are you sure you want to change the base?

[wip] add systemd services for configuration after start #980

Conversation

anjannath commented Oct 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anjannath commented Nov 26, 2024

anjannath commented Nov 26, 2024

Choose a reason for hiding this comment

openshift-ci bot commented Dec 16, 2024

anjannath commented Dec 17, 2024

anjannath commented Dec 17, 2024

anjannath commented Dec 17, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

openshift-ci bot commented Jan 8, 2025

anjannath commented Oct 21, 2024 •

edited

Loading