[metricbeat] Add kube_daemonset_status_current_number_scheduled to kubernetes.state_daemonset

[barkbay]

This PR allows to ingest the kube_daemonset_status_current_number_schedule metric in the kubernetes.state_daemonset dataset:

# HELP kube_daemonset_status_current_number_scheduled [STABLE] The number of nodes running at least one daemon pod and are supposed to.
# TYPE kube_daemonset_status_current_number_scheduled gauge
kube_daemonset_status_current_number_scheduled{namespace="kube-system",daemonset="kube-proxy"} 1
kube_daemonset_status_current_number_scheduled{namespace="kube-system",daemonset="kindnet"} 1
kube_daemonset_status_current_number_scheduled{namespace="kube-system",daemonset="fluentd-elasticsearch"} 1

Refer to "Use cases" below for the motivation.

Proposed commit message

[metricbeat] Add kube_daemonset_status_current_number_scheduled to kubernetes.state_daemonset

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Use cases

Although the ready state may fluctuate frequently (e.g. because of DaemonSet upgrades which can take some time on large clusters), the number of desired Pods should always equal the number of scheduled Pods. These 2 values should be stable as they depend on the number of K8s nodes in cluster which is usually stable (or label changes, which is not supposed to happen often). Comparing the desired number of Pods with the scheduled can be useful to early detect scheduling issues with DaemonSets.

[botelastic]

This pull request doesn't have a Team:<team> label.

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @barkbay? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Duration: 54 min 11 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[MichaelKatsoulis]

@barkbay My only comment on this is that

beats/metricbeat/module/kubernetes/state_daemonset/_meta/data.json

Line 8 in 4b09b8b

"kubernetes": {

should also be updated with the new field.
Check this PR as reference

[gizas]

We should also create the issue for the kubernetes integration update under https://github.com/elastic/integrations in order to keep them in sync

[constanca-m]

@barkbay My only comment on this is that

beats/metricbeat/module/kubernetes/state_daemonset/_meta/data.json

Line 8 in 4b09b8b

"kubernetes": {

should also be updated with the new field.
Check this PR as reference

This is actually something that this PR broke. I created the fix just now here: #37333. After that, the file you mentioned should again be update with go test -data. @MichaelKatsoulis

Edit: the fix was merged, so you can just update your branch and run go test -data again and it should update the correct files @barkbay Sorry for this

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-12-10T19:39:02.938+0000

• Duration: 53 min 33 sec

Test stats 🧪

Test	Results
Failed	0
Passed	4509
Skipped	902
Total	5411

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @barkbay? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[mergify]

backport-8.x has been added to help with the transition to the new branch 8.x.
If you don't need it please use backport-skip label and remove the backport-8.x label.

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b daemonset-include-scheduled-replicas upstream/daemonset-include-scheduled-replicas
git merge upstream/main
git push upstream daemonset-include-scheduled-replicas