[Fleet] Update Fleet UI to handle agentless integrations + policies

[kpollich]

Blocked by elastic/package-spec#684

Today, the Cloud Security Posture Management (CSPM) integration supports agentless deployment through some hardcoded code paths, e.g.

kibana/x-pack/plugins/fleet/common/constants/agent_policy.ts

Line 39 in 6348ab3

export const AGENTLESS_POLICY_ID = 'agentless'; // the policy id defined here: https://github.com/elastic/project-controller/blob/main/internal/project/security/security_kibana_config.go#L86

kibana/x-pack/plugins/fleet/public/applications/fleet/sections/agent_policy/create_package_policy_page/single_page_layout/hooks/setup_technology.ts

Lines 17 to 29 in f7ebd29

	export const useAgentlessPolicy = () => {
	const { agentless: agentlessExperimentalFeatureEnabled } = ExperimentalFeaturesService.get();
	const { cloud } = useStartServices();
	const isServerless = !!cloud?.isServerlessEnabled;
	const isAgentlessEnabled = agentlessExperimentalFeatureEnabled && isServerless;
	const isAgentlessPolicyId = (id: string) => isAgentlessEnabled && id === AGENTLESS_POLICY_ID;
	return {
	isAgentlessEnabled,
	isAgentlessPolicyId,
	};
	};

These hardcoded checks need to be replaced with dynamic ones based on the above changes made to package-spec/integration. When creating an integration policy, we need to check whether the selected policy template has deployment_modes.agentless.enabled set to true. If a policy template supports agentless, we should show the "setup technology" selector UI that the cloud security team has built for CSPM today.

This logic can assume there will be a single agentless agent policy onto which these integration policies should be installed. When installing an agentless integration, the agent policy selector UI should not appear at all.

We'll also need to honor the hide_in_deployment_modes property for variables in order to hide/show certain variables in either the agentless or default UI.

Implementation

• Replace hardcoded checks for agentless support with dynamic ones based on policy template's deployment_modes.agentless.enabled value
• Ensure agentless integration policies are created on the hardcoded agentless policy when saved
• Honor hide_in_deployment_modes setting for variables that are explicitly hidden/shown in either the default or agentless UI

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[criamico]

This logic can assume there will be a single agentless agent policy onto which these integration policies should be installed.

Does that mean that we want to install all the integration policies that have deployment_modes.agentless.enabled: true to the same policy?
What would be the use of the supports_agentless property defined in #180377 in this case?

The way I pictured it is:

• An integration with deployment_modes.agentless.enabled: true is installed
• The integration policy gets added to the hardcoded agentless agent policy that will be marked with supports_agentless: true
• For now it would be limited only to the hardcoded policy, but in the future the integration policy could be added to any agent policy that would then be marked with supports_agentless: true as well

@kpollich does it makes sense to you?

[kpollich]

For now it would be limited only to the hardcoded policy, but in the future the integration policy could be added to any agent policy that would then be marked with supports_agentless: true as well

Yep this is exactly what I had in mind as well. For now, the only policy that will exist with supports_agentless: true will be the hardcoded policy with the ID of agentless. This is where all integration policies for an agentless integration will be installed (no agent policy selector for this case at all). In the future, users will be able to create more policies with supports_agentless: true to configure different settings for various types of agentless deployments.

[kpollich]

cc @eyalkraft @olegsu to make sure I'm not misrepresenting anything above ☝️

[criamico]

For now, the only policy that will exist with supports_agentless: true will be the hardcoded policy with the ID of agentless.

Sounds good, that means that for now we're keeping the hardcoded id check here:

kibana/x-pack/plugins/fleet/public/applications/fleet/sections/agent_policy/create_package_policy_page/single_page_layout/hooks/setup_technology.ts

Line 23 in f7ebd29

const isAgentlessPolicyId = (id: string) => isAgentlessEnabled && id === AGENTLESS_POLICY_ID;

Also, in the parent issue there is this task

Update serverless preconfiguration to provide is_agentless flag in addition to hardcoded agentless policy ID

I assume that this flag would be supports_agentless, unless we need a second one for some other reason?

[kpollich]

Yep - that's a typo good catch. Should be supports_agentless not is_agentless.

[kfirpeled]

This logic can assume there will be a single agentless agent policy onto which these integration policies should be installed. When installing an agentless integration, the agent policy selector UI should not appear at all.

Correct me if I'm wrong, but we plan to drop the hard-coded agentless agent-policy. And align to a dynamic approach and support multiple number of agent-policies of agentless.

One of these things is to stop using the hard-coded agentless agent policy ID as an identifier for agentless policy.

@olegsu keep me honest here

[olegsu]

The agentless hardcoded policy is created by the project-controller and does not yet mark with supports_agentless: true.

@maxcold please correct me if I am wrong:
In CSPM integration page, the selector is shown in case the policy exists and the integration is CSPM. Assuming there is a pre-configured policy (and elastic-agent deployed as agentless).

We actively working to introduce Agentless API (proposal draft) to support the creation of Agentless resources on demand, this will help us to reduce the cost and solve scalability issues. Once the API is deployed, the assumption to show/hide the selector will not work. Kibana needs first to create the policy and then call Agentless API with details.

Questions
Agentless API requires an enrollment token to deploy the agent and we want to continue to use the managed properly. Does the enrollment token for managed policy exist or do we need to create one?

[criamico]

The agentless hardcoded policy is created by the project-controller and does not yet mark with supports_agentless: true.

@oleg @kfirpeled since I merged the PR adding the support for supports_agentless to kibana, it's now possible to add the property to the hardcoded policy. I can open a PR to do it as I need it for the follow up work.

For this, what we need to do currently in Fleet UI is the following:

• Rely on the new property supports_agentless to mark a policy as enabled for agentless. These policies will also need to have managed: true so that users can't edit them.
• Support the new fields (deployment_modes.agentless.enabled: true and hide_in_deployment_modes specified in package-spec. Afaik CPSM doesn't supports them yet but there's a plan to extend it to other integrations as well.

Regarding the support to multiple policies:

Until the Agentless API are not ready we need to defined the flow for the creation of multiple policies from kibana.
To remove the limitation to the existing agentless policy we could allow the creation of policies (either though API or preconfiguration) marked with supports_agentless: true but without showing anything in the UI. Then, when the APIs will be ready, a new agentless onboarding and any additional support in the UI could be added. What do you think?

[olegsu]

@oleg @kfirpeled since I merged https://github.com/elastic/kibana/issues/180377adding the support for supports_agentless to kibana, it's now possible to add the property to the hardcoded policy. I can open a PR to do it as I need it for the follow up work.

@criamico it would be great. If am not mistaken, Kibana won't migrate existing policies to have the new property

[criamico]

When creating an integration policy, we need to check whether the selected policy template has deployment_modes.agentless.enabled set to true

I'm trying to clarify how this property should work in Fleet UI. Is it supposed to work like a visibility option for the whole policy template? i.e. a sample integration with following policy templates

   "policy_templates": [
            {
              "name": "sample",
              "title": "Agentless sample logs",
              "description": "Collect sample logs",
              "multiple": true,
              "inputs": [
                {
                  "title": "Collect sample logs from instances",
                  "vars": [],
                  "type": "logfile",
                  "description": "Collecting sample logs"
                }
              ],
              "deployment_modes": {
                "default": {
                  "enabled": false
                },
                "agentless": {
                  "enabled": true
                }
              }
            }
          ],

Then the policy template should visible in agentless but it will be hidden in stateful envs (or if it were false instead):

Is my assumption correct?

@kpollich @jsoriano

[kpollich]

Then the policy template should visible in agentless but it will be hidden in stateful envs (or if it were false instead):

No I think the policy template fields should all be visible in each mode, but if deployment_modes.agentless.enabled: true is set, then the "setup technology" UI should be shown.

Right now I think this is a UI Extension, which is probably fine to keep for now:

kibana/x-pack/plugins/fleet/public/applications/fleet/sections/agent_policy/create_package_policy_page/single_page_layout/index.tsx

Lines 321 to 348 in fb8ba21

	const { agentlessPolicy, handleSetupTechnologyChange, selectedSetupTechnology } =
	useSetupTechnology({
	newAgentPolicy,
	updateNewAgentPolicy,
	updateAgentPolicy,
	setSelectedPolicyTab,
	});
	const replaceStepConfigurePackagePolicy =
	replaceDefineStepView && packageInfo?.name ? (
	!isInitialized ? (
	<Loading />
	) : (
	<ExtensionWrapper>
	<replaceDefineStepView.Component
	agentPolicy={agentPolicy}
	packageInfo={packageInfo}
	newPolicy={packagePolicy}
	onChange={handleExtensionViewOnChange}
	validationResults={validationResults}
	isEditPage={false}
	handleSetupTechnologyChange={handleSetupTechnologyChange}
	agentlessPolicy={agentlessPolicy}
	/>
	</ExtensionWrapper>
	)
	) : undefined;

The component itself lives in the CSP plugin, e.g.

https://github.com/elastic/kibana/blob/fb8ba21e28fe0152eeb21f2a0d6622ed36787ebf/x-pack/plugins/cloud_security_posture/public/components/fleet_extensions/policy_template_form.tsx

So I think what we'd need to do is detect whether the current integration has the agentless deployment, then render this UI if that is the case.