Cilium with servicemonitor on cluster bootstrap using Rancher fails

[fmiqbal]

When I bootstrap RKE2 cluster in vsphere, when initially deploying cilium, it fails because there is no servicemonitor crd, as expected, because the crd is available later after I install rancher-monitoring app. But I want to make this works because I think the servicemonitor component is essential to the cilium monitoring,

I also actually bootstrapping it via terraform using terraform rancher provider,

are there any alternative to do this ?, what works now is I disable the servicemonitor for all cilium component, and then maybe

• re add them using different tf apply (some kinda like there is file for initial bootstrap, and then there is file for post install, specifically for cilium component) ?
• get the current definition of servicemonitor from the source, and apply it either manually or using flux (but I'm afraid the definition will change/updated in the future)
• maybe, make the helm install for the cilium component ignore the missing definition for servicemonitor, and instead add them later ?
• maybe, some kind of post install apply ?
• install the servicemonitor crd specifically using "addtional manifests" method ?

or are there any alternative, better way to do this ?

[jmeza-xyz]

The way I've handled this in prod is by using a combo of your second bullet point and last bullet point. I've used flux and helmchartconfig, (the additional manifest does the same as kubectl apply -f to any yaml input there) to modify Cilium after cluster creation and manage it with flux. The servicemonitor in the values keys should not change, but if it does, its good practice to check the changelog between Cilium upgrades.