Skip to content

Commit

Permalink
docs: Prepare Changelog for 8.12.0 (#37655)
Browse files Browse the repository at this point in the history 
* docs: Close changelog for 8.12.0

* Update CHANGELOG.asciidoc

* Update CHANGELOG.asciidoc

* Apply suggestions from code review

Co-authored-by: David Kilfoyle <[email protected]>

---------

Co-authored-by: Pierre HILBERT <[email protected]>
Co-authored-by: David Kilfoyle <[email protected]>
  • Loading branch information
@elasticmachine @pierrehilbert @kilfoyle
3 people authored Jan 17, 2024
1 parent 750a13d commit 9ecb427
Show file tree
Hide file tree
Showing 3 changed files with 90 additions and 111 deletions.
86 changes: 86 additions & 0 deletions CHANGELOG.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,92 @@
:issue: https://github.com/elastic/beats/issues/
:pull: https://github.com/elastic/beats/pull/

[[release-notes-8.12.0]]
=== Beats version 8.12.0
https://github.com/elastic/beats/compare/v8.11.4\...v8.12.0[View commits]

==== Breaking changes

*Heartbeat*
- Decrease the ES default timeout to 10 for the load monitor state requests.

*Osquerybeat*

- Upgrade to osquery 5.10.2. {pull}37115[37115]

==== Bugfixes

*Filebeat*

- Add validation to the `http_endpoint` config for empty URL. {pull}36816[36816] {issue}36772[36772]
- Fix merging of array fields (processors, paths, parsers) in configurations generated from hints and default config. {issue}36838[36838] {pull}36857[36857]

==== Added

*Affecting all Beats*

- Allow `queue` configuration settings to be set under the output. {issue}35615[35615] {pull}36788[36788]
- Raise up logging level to warning when attempting to configure {beats} with unknown fields from autodiscovered events/environments.
- Elasticsearch output now supports `idle_connection_timeout`. {issue}35616[35615] {pull}36843[36843]
- Upgrade to Go 1.20.12. {pull}37350[37350]
- The Elasticsearch output can now configure performance presets with the `preset` configuration field. {pull}37259[37259]
- Upgrade `elastic-agent-system-metrics` to v0.9.1. See https://github.com/elastic/elastic-agent-system-metrics/releases/tag/v0.9.1. {pull}37353[37353]
- Upgrade to elastic-agent-libs v0.7.3 and golang.org/x/crypto v0.17.0. {pull}37544[37544]

*Auditbeat*

- Add `ignore_errors` option to audit module. {issue}15768[15768] {pull}36851[36851]
- Fix copy arguments for strict aligned architectures. {pull}36976[36976]

*Filebeat*

- Allow http_endpoint input to receive PUT and PATCH requests. {pull}36734[36734]
- Avoid unwanted publication of Azure entity records. {pull}36753[36753]
- Avoid unwanted publication of Okta entity records. {pull}36770[36770]
- Add support for Digest Authentication to CEL input. {issue}35514[35514] {pull}36932[36932]
- Use filestream input with `file_identity.fingerprint` as default for hints autodiscover. {issue}35984[35984] {pull}36950[36950]
- Add network processor in addition to interface based direction resolution. {pull}37023[37023]
- Make CEL input log current transaction ID when request tracing is turned on. {pull}37065[37065]
- Make Azure Blob Storage input GA and update docs accordingly. {pull}37128[37128]
- Add request trace logging to http_endpoint input. {issue}36951[36951] {pull}36957[36957]
- Make GCS input GA and update docs accordingly. {pull}37127[37127]
- Suppress and log max HTTP request retry errors in CEL input. {pull}37160[37160]
- Prevent CEL input from re-entering the eval loop when an evaluation failed. {pull}37161[37161]
- Update CEL extensions library to v1.7.0. {pull}37172[37172]

*Auditbeat*

- Upgrade go-libaudit to v2.4.0. {issue}36776[36776] {pull}36964[36964]
- Add a `/inputs/` route to the HTTP monitoring endpoint that exposes metrics for each dataset instance. {pull}36971[36971]

*Heartbeat*
- Capture and log the individual connection metrics for all the lightweight monitors.

*Metricbeat*

- Add metrics grouping by dimensions and time to Azure app insights. {pull}36634[36634]
- Align on the algorithm used to transform Prometheus histograms into Elasticsearch histograms. {pull}36647[36647]
- Enhance GCP billing with detailed tables identification, additional fields, and optimized data handling. {pull}36902[36902]
- Add a `/inputs/` route to the HTTP monitoring endpoint that exposes metrics for each metricset instance. {pull}36971[36971]
- Add Linux IO metrics to system/process. {pull}37213[37213]
- Add new memory/cgroup metrics to Kibana module. {pull}37232[37232]

*Packetbeat*

- Add metrics for TCP flags. {issue}36992[36992] {pull}36975[36975]

*Winlogbeat*

- Make ingest pipeline routing robust to letter case of channel names for forwarded events. {issue}36670[36670] {pull}36899[36899]
- Document minimum permissions required for local user account. {issue}15773[15773] {pull}37176[37176]

==== Deprecated

*Filebeat*

- Deprecate rsa2elk Filebeat modules. {issue}36125[36125] {pull}36887[36887]


[[release-notes-8.11.4]]
=== Beats version 8.11.4
https://github.com/elastic/beats/compare/v8.11.3\...v8.11.4[View commits]
Expand Down
114 changes: 3 additions & 111 deletions CHANGELOG.next.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,6 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
==== Breaking changes

*Affecting all Beats*
- The Elasticsearch output now enables compression by default. This decreases network data usage by an average of 70-80%, in exchange for 20-25% increased CPU use and ~10% increased ingestion time. The previous default can be restored by setting the flag `compression_level: 0` under `output.elasticsearch`. {pull}36681[36681]
- Elastic-agent-autodiscover library updated to version 0.6.4, disabling metadata for deployment and cronjob. Pods that will be created from deployments or cronjobs will not have the extra metadata field for kubernetes.deployment or kubernetes.cronjob, respectively. {pull}36877[36877]
- Defaults are changing for some options in the queue and Elasticsearch output, to improve typical performance based on current benchmark data. All changes can be overridden by setting them explicitly in the beat configuration. {pull}36990[36990] The changes are:
- `queue.mem.events` is changing from `4096` to `3200`.
- `queue.mem.flush.min_events` is changing from `2048` to `1600`.
- `queue.mem.flush.timeout` is changing from `1s` to `10s`.
- `output.elasticsearch.bulk_max_size` is changing from `50` to `1600`.
- `output.elasticsearch.idle_connection_timeout` is changing from `60s` to `3s`.
- Avoid logging fields values when handling Elasticsearch output errors except at the debug log level. The debug log level must now be used to see detailed errors, for example mapping errors and their cause. {pull}37229[37229]

*Auditbeat*

Expand All @@ -26,14 +17,12 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]


*Heartbeat*
- Decreases the ES default timeout to 10 for the load monitor state requests

*Metricbeat*


*Osquerybeat*

- Upgrade to osquery 5.10.2. {pull}37115[37115]

*Packetbeat*

Expand Down Expand Up @@ -62,12 +51,9 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
- Upgraded apache arrow library used in x-pack/libbeat/reader/parquet from v11 to v12.0.1 in order to fix cross-compilation issues {pull}35640[35640]
- Fix panic when MaxRetryInterval is specified, but RetryInterval is not {pull}35820[35820]
- Support build of projects outside of beats directory {pull}36126[36126]
- Fix memqueue producer blocking indefinitely even after being cancelled {issue}22813[22813] {pull}37077[37077]

*Auditbeat*

- Fix documentation regarding socket type selection. {issue}37174[37174] {pull}37175[37175]
- Fix guess trigger for system/socket creds on newer kernels. {issue}36905[36905] {pull}37136[37136]

*Filebeat*

Expand All @@ -82,24 +68,11 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
- Fix panic when sqs input metrics getter is invoked {pull}36101[36101] {issue}36077[36077]
- Fix handling of Juniper SRX structured data when there is no leading junos element. {issue}36270[36270] {pull}36308[36308]
- Fix Filebeat Cisco module with missing escape character {issue}36325[36325] {pull}36326[36326]
- Fix panic when redact option is not provided to CEL input. {issue}36387[36387] {pull}36388[36388]
- Remove 'onFilteredOut' and 'onDroppedOnPublish' callback logs {issue}36299[36299] {pull}36399[36399]
- Added a fix for Crowdstrike pipeline handling process arrays {pull}36496[36496]
- Ensure winlog input retains metric collection when handling recoverable errors. {issue}36479[36479] {pull}36483[36483]
- Revert error introduced in {pull}35734[35734] when symlinks can't be resolved in filestream. {pull}36557[36557]
- Fix ignoring external input configuration in `take_over: true` mode {issue}36378[36378] {pull}36395[36395]
- Add validation to http_endpoint config for empty URL {pull}36816[36816] {issue}36772[36772]
- Fix merging of array fields(processors, paths, parsers) in configurations generated from hints and default config. {issue}36838[36838] {pull}36857[36857]
- Fix handling of response errors in HTTPJSON and CEL request trace logging. {pull}36956[36956]
- Do not error when Okta API returns no data. {pull}37092[37092]
- Fix request body close behaviour in HTTP_Endpoint when handling GZIP compressed content. {pull}37091[37091]
- Make CEL input now global evaluate to a time in UTC. {pull}37159[37159]

*Heartbeat*

- Fix panics when parsing dereferencing invalid parsed url. {pull}34702[34702]
- Fix mapping errors with invalid urls. {pull}37255[37255]
- Added fix for formatting the logs from stateloader properly. {pull}37369[37369]

*Metricbeat*

Expand All @@ -115,18 +88,7 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
- Fix EC2 host.cpu.usage {pull}35717[35717]
- Add option in SQL module to execute queries for all dbs. {pull}35688[35688]
- Add remaining dimensions for azure storage account to make them available for tsdb enablement. {pull}36331[36331]
- Add missing 'TransactionType' dimension for Azure Storage Account. {pull}36413[36413]
- Add log error when statsd server fails to start {pull}36477[36477]
- Fix CassandraConnectionClosures metric configuration {pull}34742[34742]
- Fix event mapping implementation for statsd module {pull}36925[36925]
- The region and availability_zone ecs fields nested within the cloud field. {pull}37015[37015]
- Fix CPU and memory metrics collection from privileged process on Windows {issue}17314[17314]{pull}37027[37027]
- Enhanced Azure Metrics metricset with refined grouping logic and resolved duplication issues for TSDB compatibility {pull}36823[36823]
- Fix memory leak on Windows {issue}37142[37142] {pull}37171[37171]
- Fix unintended skip in metric collection on Azure Monitor {issue}37204[37204] {pull}37203[37203]
- Fix the "api-version query parameter (?api-version=) is required for all requests" error in Azure Billing. {pull}37158[37158]
- Add memory hard limit from container metadata and remove usage percentage in AWS Fargate. {pull}37194[37194]
- Fix the reference time rounding on Azure Metrics {issue}37204[37204] {pull}37365[37365]
- Fix Azure Resource Metrics missing metrics (min and max aggregations) after upgrade to 8.11.3 {issue}37642[37642] {pull}37643[37643]

*Osquerybeat*
Expand All @@ -137,7 +99,6 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]

*Winlogbeat*

- Fix dashboards under Kibana 8.x. {issue}37080[37080] {pull}37085[37085]


*Elastic Logging Plugin*
Expand All @@ -148,29 +109,11 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
*Affecting all Beats*

- Added append Processor which will append concrete values or values from a field to target. {issue}29934[29934] {pull}33364[33364]
- When running under Elastic-Agent the status is now reported per Unit instead of the whole Beat {issue}35874[35874] {pull}36183[36183]
- Add warning message to SysV init scripts for RPM-based systems that lack `/etc/rc.d/init.d/functions`. {issue}35708[35708] {pull}36188[36188]
- Mark `translate_sid` processor is GA. {issue}36279[36279] {pull}36280[36280]
- dns processor: Add support for forward lookups (`A`, `AAAA`, and `TXT`). {issue}11416[11416] {pull}36394[36394]
- Mark `syslog` processor as GA, improve docs about how processor handles syslog messages. {issue}36416[36416] {pull}36417[36417]
- Add support for AWS external IDs. {issue}36321[36321] {pull}36322[36322]
- [Enhanncement for host.ip and host.mac] Disabling netinfo.enabled option of add-host-metadata processor {pull}36506[36506]
Setting environmental variable ELASTIC_NETINFO:false in Elastic Agent pod will disable the netinfo.enabled option of add_host_metadata processor
- allow `queue` configuration settings to be set under the output. {issue}35615[35615] {pull}36788[36788]
- Beats will now connect to older Elasticsearch instances by default {pull}36884[36884]
- Raise up logging level to warning when attempting to configure beats with unknown fields from autodiscovered events/environments
- elasticsearch output now supports `idle_connection_timeout`. {issue}35616[35615] {pull}36843[36843]
- Upgrade golang/x/net to v0.17.0. Updates the publicsuffix table used by the registered_domain processor. {pull}36969[36969]
Setting environmental variable ELASTIC_NETINFO:false in Elastic Agent pod will disable the netinfo.enabled option of add_host_metadata processor
- Upgrade to Go 1.20.12. {pull}37350[37350]
- The Elasticsearch output can now configure performance presets with the `preset` configuration field. {pull}37259[37259]
- Upgrade elastic-agent-system-metrics to v0.9.1. See https://github.com/elastic/elastic-agent-system-metrics/releases/tag/v0.9.1. {pull}37353[pull]
- Upgrade to elastic-agent-libs v0.7.3 and golang.org/x/crypto v0.17.0. {pull}37544[37544]

*Auditbeat*

- Add `ignore_errors` option to audit module. {issue}15768[15768] {pull}36851[36851]
- Fix copy arguments for strict aligned architectures. {pull}36976[36976]

*Filebeat*

Expand All @@ -187,61 +130,18 @@ Setting environmental variable ELASTIC_NETINFO:false in Elastic Agent pod will d
- Add nginx ingress_controller parsing if one of upstreams fails to return response {pull}34787[34787]
- Add oracle authentication messages parsing {pull}35127[35127]
- Add `clean_session` configuration setting for MQTT input. {pull}35806[16204]
- Add fingerprint mode for the filestream scanner and new file identity based on it {issue}34419[34419] {pull}35734[35734]
- Add file system metadata to events ingested via filestream {issue}35801[35801] {pull}36065[36065]
- Add support for localstack based input integration testing {pull}35727[35727]
- Allow parsing bytes in and bytes out as long integer in CEF processor. {issue}36100[36100] {pull}36108[36108]
- Add support for registered owners and users to AzureAD entity analytics provider. {pull}36092[36092]
- Add support for endpoint resolver in AWS config {pull}36208[36208]
- Added support for Okta OAuth2 provider in the httpjson input. {pull}36273[36273]
- Add support of the interval parameter in Salesforce setupaudittrail-rest fileset. {issue}35917[35917] {pull}35938[35938]
- Add device handling to Okta input package for entity analytics. {pull}36049[36049]
- Add setup option `--force-enable-module-filesets`, that will act as if all filesets have been enabled in a module during setup. {issue}30916[30916] {pull}36286[36286]
- [Azure] Add input metrics to the azure-eventhub input. {pull}35739[35739]
- Reduce HTTPJSON metrics allocations. {pull}36282[36282]
- Add support for a simplified input configuraton when running under Elastic-Agent {pull}36390[36390]
- Make HTTPJSON response body decoding errors more informative. {pull}36481[36481]
- Allow fine-grained control of entity analytics API requests for Okta provider. {issue}36440[36440] {pull}36492[36492]
- Add support for expanding `journald.process.capabilities` into the human-readable effective capabilities in the ECS `process.thread.capabilities.effective` field. {issue}36454[36454] {pull}36470[36470]
- Allow fine-grained control of entity analytics API requests for AzureAD provider. {issue}36440[36440] {pull}36441[36441]
- For request tracer logging in CEL and httpjson the request and response body are no longer included in `event.original`. The body is still present in `http.{request,response}.body.content`. {pull}36531[36531]
- Added support for Okta OAuth2 provider in the CEL input. {issue}36336[36336] {pull}36521[36521]
- Improve error logging in HTTPJSON input. {pull}36529[36529]
- Disable warning message about ingest pipeline loading when running under Elastic Agent. {pull}36659[36659]
- Add input metrics to http_endpoint input. {issue}36402[36402] {pull}36427[36427]
- Remove Event Normalization from GCP PubSub Input. {pull}36716[36716]
- Update mito CEL extension library to v1.6.0. {pull}36651[36651]
- Added support for new features & removed partial save mechanism in the Azure Blob Storage input. {issue}35126[35126] {pull}36690[36690]
- Improve template evaluation logging for HTTPJSON input. {pull}36668[36668]
- Add CEL partial value debug function. {pull}36652[36652]
- Added support for new features and removed partial save mechanism in the GCS input. {issue}35847[35847] {pull}36713[36713]
- Re-use buffers to optimise memory allocation in fingerprint mode of filestream {pull}36736[36736]
- Allow http_endpoint input to receive PUT and PATCH requests. {pull}36734[36734]
- Add cache processor. {pull}36786[36786]
- Avoid unwanted publication of Azure entity records. {pull}36753[36753]
- Avoid unwanted publication of Okta entity records. {pull}36770[36770]
- Add support for Digest Authentication to CEL input. {issue}35514[35514] {pull}36932[36932]
- Use filestream input with file_identity.fingerprint as default for hints autodiscover. {issue}35984[35984] {pull}36950[36950]
- Add network processor in addition to interface based direction resolution. {pull}37023[37023]
- Add setup option `--force-enable-module-filesets`, that will act as if all filesets have been enabled in a module during setup. {issue}30915[30915] {pull}99999[99999]
- Make CEL input log current transaction ID when request tracing is turned on. {pull}37065[37065]
- Made Azure Blob Storage input GA and updated docs accordingly. {pull}37128[37128]
- Add request trace logging to http_endpoint input. {issue}36951[36951] {pull}36957[36957]
- Made GCS input GA and updated docs accordingly. {pull}37127[37127]
- Suppress and log max HTTP request retry errors in CEL input. {pull}37160[37160]
- Prevent CEL input from re-entering the eval loop when an evaluation failed. {pull}37161[37161]
- Update CEL extensions library to v1.7.0. {pull}37172[37172]

*Auditbeat*

- Upgrade go-libaudit to v2.4.0. {issue}36776[36776] {pull}36964[36964]
- Add a `/inputs/` route to the HTTP monitoring endpoint that exposes metrics for each dataset instance. {pull}36971[36971]

*Libbeat*

*Heartbeat*
- Added status to monitor run log report.
- Capture and log the individual connection metrics for all the lightweight monitors


*Metricbeat*
Expand All @@ -250,28 +150,18 @@ Setting environmental variable ELASTIC_NETINFO:false in Elastic Agent pod will d
- Add GCP CloudSQL metadata {pull}33066[33066]
- Add GCP Carbon Footprint metricbeat data {pull}34820[34820]
- Add event loop utilization metric to Kibana module {pull}35020[35020]
- Add metrics grouping by dimensions and time to Azure app insights {pull}36634[36634]
- Align on the algorithm used to transform Prometheus histograms into Elasticsearch histograms {pull}36647[36647]
- Enhance GCP billing with detailed tables identification, additional fields, and optimized data handling. {pull}36902[36902]
- Add a `/inputs/` route to the HTTP monitoring endpoint that exposes metrics for each metricset instance. {pull}36971[36971]
- Add linux IO metrics to system/process {pull}37213[37213]
- Add new memory/cgroup metrics to Kibana module {pull}37232[37232]

*Osquerybeat*


*Packetbeat*

- Add metrics for TCP flags. {issue}36992[36992] {pull}36975[36975]

*Packetbeat*


*Winlogbeat*

- Make ingest pipeline routing robust to letter case of channel names for forwarded events. {issue}36670[36670] {pull}36899[36899]
- Document minimum permissions required for local user account. {issue}15773[15773] {pull}37176[37176]
- Bump Windows Npcap version to v1.78. {issue}37300[37300] {pull}37370[37370]

*Functionbeat*

Expand All @@ -291,7 +181,6 @@ Setting environmental variable ELASTIC_NETINFO:false in Elastic Agent pod will d

*Filebeat*

- Deprecate rsa2elk Filebeat modules. {issue}36125[36125] {pull}36887[36887]

*Heartbeat*

Expand Down Expand Up @@ -351,6 +240,9 @@ Setting environmental variable ELASTIC_NETINFO:false in Elastic Agent pod will d









Expand Down
1 change: 1 addition & 0 deletions libbeat/docs/release.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ This section summarizes the changes in each release. Also read
<<breaking-changes>> for more detail about changes that affect
upgrade.

* <<release-notes-8.12.0>>
* <<release-notes-8.11.4>>
* <<release-notes-8.11.3>>
* <<release-notes-8.11.2>>
Expand Down

0 comments on commit 9ecb427

Please sign in to comment.