Add a markdown with high level kubernetes metadata enrichment explanation

[MichaelKatsoulis]

• WHAT: Add a markdown which explains in high level how kubernetes metadata enrichment works in metrics modules
• WHY: the new implementation of [Metricbeat][Kubernetes] Share watchers between metricsets #37332 needs a high level explanation

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

[botelastic]

This pull request doesn't have a Team:<team> label.

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @MichaelKatsoulis? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Duration: 52 min 44 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[constanca-m]

I can't see the image in the preview, can you see it @MichaelKatsoulis ?

[constanca-m]

I also don't understand what the image is trying to say... Is it just an example using 3 metricsets on what they are doing?

[MichaelKatsoulis]

It is supposed to show an example of different metricsets having their own metadata enricher but sharing same watchers which use the k8s API to watch for a resource.
I am open to all kind of suggestions !!

[constanca-m]

Ok! Maybe you can just write a sentence below the image saying that?

[constanca-m]

Should this part:

beats/metricbeat/module/kubernetes/util/kubernetes.go

Lines 911 to 924 in 8efb366

	for key := range resourceMetaWatcher.metadataObjects {
	obj, exists, err := resourceMetaWatcher.watcher.Store().GetByKey(key)
	if err != nil {
	log.Errorf("Error trying to get the object from the store: %s", err)
	} else {
	if exists {
	newMetadataEvents := enricher.updateFunc(obj.(kubernetes.Resource))
	// add the new metadata to the watcher received metadata
	for id, metadata := range newMetadataEvents {
	enricher.metadata[id] = metadata
	}
	}
	}
	}

, be mentioned somehow or do you think it's too specific?

[gizas]

To differentiate with datasets. Just writ something same term is called datasets in elastic-agent.
https://www.elastic.co/blog/an-introduction-to-the-elastic-data-stream-naming-scheme

[gizas]

I think that the image is a little confusing. Maybe to put arrows to identify direction?
How about removing it and adding a table with what watchers are being created per metricset?

[MichaelKatsoulis]

Should this part:

beats/metricbeat/module/kubernetes/util/kubernetes.go

Lines 911 to 924 in 8efb366

	for key := range resourceMetaWatcher.metadataObjects {
	obj, exists, err := resourceMetaWatcher.watcher.Store().GetByKey(key)
	if err != nil {
	log.Errorf("Error trying to get the object from the store: %s", err)
	} else {
	if exists {
	newMetadataEvents := enricher.updateFunc(obj.(kubernetes.Resource))
	// add the new metadata to the watcher received metadata
	for id, metadata := range newMetadataEvents {
	enricher.metadata[id] = metadata
	}
	}
	}
	}

, be mentioned somehow or do you think it's too specific?

Yes, I will add more info.

[MichaelKatsoulis]

@constanca-m and @gizas I answered all your review comments. Can you give it another go?

[gizas]

Still dont explain why we have two terms. Users dont understand why this changes in beats vs agents. I think we should make this clear

[MichaelKatsoulis]

@gizas This markdown is in beats repo, under kubernetes module. There, only metricsets exist. Personally I would remove the dataset as it is confusing. What kind of sentence do you want me to add?

[gizas]

Missed that we are under beats. I had in my mind that this could be a more general doc.
So ok you can remove datasets overall

[gizas]

LGTM @MichaelKatsoulis ! Some last minor comments