Merge branch 'main' into fix_bulk_executor_for_system_indices

docs-mdx/painless/painless-field-context.mdx

@@ -0,0 +1,136 @@
+---
+id: enElasticsearchPainlessPainlessFieldContext
+slug: /en/elasticsearch/painless/painless-field-context
+title: Field context
+description: Description to be written
+tags: []
+---
+
+<div id="painless-field-context"></div>
+
+Use a Painless script to create a
+[script field](((ref))/search-fields.html#script-fields) to return
+a customized value for each document in the results of a query.
+
+**Variables**
+
+`params` (`Map`, read-only)
+    : User-defined parameters passed in as part of the query.
+
+`doc` (`Map`, read-only)
+    : Contains the fields of the specified document where each field is a
+    `List` of values.
+
+[`params['_source']`](((ref))/mapping-source-field.html) (`Map`, read-only)
+    : Contains extracted JSON in a `Map` and `List` structure for the fields
+    existing in a stored document.
+
+**Return**
+
+`Object`
+    : The customized value for each document.
+
+**API**
+
+Both the standard <DocLink id="enElasticsearchPainlessPainlessApiReferenceShared">Painless API</DocLink> and
+<DocLink id="enElasticsearchPainlessPainlessApiReferenceField">Specialized Field API</DocLink> are available.
+
+**Example**
+
+To run this example, first follow the steps in
+<DocLink id="enElasticsearchPainlessPainlessContextExamples">context examples</DocLink>.
+
+You can then use these two example scripts to compute custom information
+for each search hit and output it to two new fields.
+
+The first script gets the doc value for the `datetime` field and calls
+the `getDayOfWeekEnum` function to determine the corresponding day of the week.
+
+```Painless
+doc['datetime'].value.getDayOfWeekEnum().getDisplayName(TextStyle.FULL, Locale.ROOT)
+```
+
+The second script calculates the number of actors. Actors' names are stored
+as a keyword array in the `actors` field.
+
+```Painless
+doc['actors'].size()   [^1]
+```
+[^1]: By default, doc values are not available for `text` fields. If `actors` was
+a `text` field, you could still calculate the number of actors by extracting
+values from `_source` with `params['_source']['actors'].size()`.
+
+The following request returns the calculated day of week and the number of
+actors that appear in each play:
+
+```console
+GET seats/_search
+{
+  "size": 2,
+  "query": {
+    "match_all": {}
+  },
+  "script_fields": {
+    "day-of-week": {
+      "script": {
+        "source": "doc['datetime'].value.getDayOfWeekEnum().getDisplayName(TextStyle.FULL, Locale.ROOT)"
+      }
+    },
+    "number-of-actors": {
+      "script": {
+        "source": "doc['actors'].size()"
+      }
+    }
+  }
+}
+```
+{/* TEST[setup:seats] */}
+
+```console-result
+{
+  "took" : 68,
+  "timed_out" : false,
+  "_shards" : {
+    "total" : 1,
+    "successful" : 1,
+    "skipped" : 0,
+    "failed" : 0
+  },
+  "hits" : {
+    "total" : {
+      "value" : 11,
+      "relation" : "eq"
+    },
+    "max_score" : 1.0,
+    "hits" : [
+      {
+        "_index" : "seats",
+        "_id" : "1",
+        "_score" : 1.0,
+        "fields" : {
+          "day-of-week" : [
+            "Thursday"
+          ],
+          "number-of-actors" : [
+            4
+          ]
+        }
+      },
+      {
+        "_index" : "seats",
+        "_id" : "2",
+        "_score" : 1.0,
+        "fields" : {
+          "day-of-week" : [
+            "Thursday"
+          ],
+          "number-of-actors" : [
+            1
+          ]
+        }
+      }
+    ]
+  }
+}
+```
+{/* TESTRESPONSE[s/"took" : 68/"took" : "$body.took"/] */}

docs/changelog/105894.yaml

@@ -0,0 +1,5 @@
+pr: 105894
+summary: Add allocation stats
+area: Allocation
+type: enhancement
+issues: []

docs/changelog/106102.yaml

@@ -0,0 +1,5 @@
+pr: 106102
+summary: Specialize serialization of array blocks
+area: ES|QL
+type: enhancement
+issues: []

docs/changelog/106156.yaml

@@ -0,0 +1,6 @@
+pr: 106156
+summary: Disable parallel collection for terms aggregation with `min_doc_count` equals
+  to 0
+area: Aggregations
+type: bug
+issues: []

docs/changelog/106172.yaml

@@ -0,0 +1,5 @@
+pr: 106172
+summary: "[Profiling] Allow to override index settings"
+area: Application
+type: enhancement
+issues: []

docs/reference/cluster/nodes-stats.asciidoc

@@ -50,6 +50,9 @@ using metrics.
   `adaptive_selection`::
       Statistics about <<search-adaptive-replica,adaptive replica selection>>.
 
+  `allocations`::
+      Statistics about allocated shards
+
   `breaker`::
       Statistics about the field data circuit breaker.
 
@@ -2802,6 +2805,44 @@ search requests on the keyed node.
 The rank of this node; used for shard selection when routing search
 requests.
 ======
+
+[[cluster-nodes-stats-api-response-body-allocations]]
+`allocations`::
+(object)
+Contains allocations statistics for the node.
++
+.Properties of `allocations`
+[%collapsible%open]
+======
+`shards`::
+(integer)
+The number of shards currently allocated to this node
+
+`undesired_shards`::
+(integer)
+The amount of shards that are scheduled to be moved elsewhere in the cluster
+or -1 other than desired balance allocator is used
+
+`forecasted_ingest_load`::
+(double)
+Total forecasted ingest load of all shards assigned to this node
+
+`forecasted_disk_usage`::
+(<<byte-units,byte value>>)
+Forecasted size of all shards assigned to the node
+
+`forecasted_disk_usage_bytes`::
+(integer)
+Forecasted size, in bytes, of all shards assigned to the node
+
+`current_disk_usage`::
+(<<byte-units,byte value>>)
+Current size of all shards assigned to the node
+
+`current_disk_usage_bytes`::
+(integer)
+Current size, in bytes, of all shards assigned to the node
+======
 =====
 ====
 

docs/reference/index-modules/slowlog.asciidoc

@@ -58,33 +58,56 @@ The search slow log file is configured in the `log4j2.properties` file.
 [discrete]
 ==== Identifying search slow log origin
 
-It is often useful to identify what triggered a slow running query. If a call was initiated with an `X-Opaque-ID` header, then the user ID
-is included in Search Slow logs as an additional **id** field
+It is often useful to identify what triggered a slow running query.
+To include information about the user that triggered a slow search,
+use the `index.search.slowlog.include.user` setting.
+
+[source,console]
+--------------------------------------------------
+PUT /my-index-000001/_settings
+{
+  "index.search.slowlog.include.user": true
+}
+--------------------------------------------------
+// TEST[setup:my_index]
+
+This will result in user information being included in the slow log.
 
 [source,js]
 ---------------------------
 {
-  "type": "index_search_slowlog",
-  "timestamp": "2030-08-30T11:59:37,786+02:00",
-  "level": "WARN",
-  "component": "i.s.s.query",
-  "cluster.name": "distribution_run",
-  "node.name": "node-0",
-  "message": "[index6][0]",
-  "took": "78.4micros",
-  "took_millis": "0",
-  "total_hits": "0 hits",
-  "stats": "[]",
-  "search_type": "QUERY_THEN_FETCH",
-  "total_shards": "1",
-  "source": "{\"query\":{\"match_all\":{\"boost\":1.0}}}",
-  "id": "MY_USER_ID",
-  "cluster.uuid": "Aq-c-PAeQiK3tfBYtig9Bw",
-  "node.id": "D7fUYfnfTLa2D7y-xw6tZg"
+  "@timestamp": "2024-02-21T12:42:37.255Z",
+  "log.level": "WARN",
+  "auth.type": "REALM",
+  "elasticsearch.slowlog.id": "tomcat-123",
+  "elasticsearch.slowlog.message": "[index6][0]",
+  "elasticsearch.slowlog.search_type": "QUERY_THEN_FETCH",
+  "elasticsearch.slowlog.source": "{\"query\":{\"match_all\":{\"boost\":1.0}}}",
+  "elasticsearch.slowlog.stats": "[]",
+  "elasticsearch.slowlog.took": "747.3micros",
+  "elasticsearch.slowlog.took_millis": 0,
+  "elasticsearch.slowlog.total_hits": "1 hits",
+  "elasticsearch.slowlog.total_shards": 1,
+  "user.name": "elastic",
+  "user.realm": "reserved",
+  "ecs.version": "1.2.0",
+  "service.name": "ES_ECS",
+  "event.dataset": "elasticsearch.index_search_slowlog",
+  "process.thread.name": "elasticsearch[runTask-0][search][T#5]",
+  "log.logger": "index.search.slowlog.query",
+  "elasticsearch.cluster.uuid": "Ui23kfF1SHKJwu_hI1iPPQ",
+  "elasticsearch.node.id": "JK-jn-XpQ3OsDUsq5ZtfGg",
+  "elasticsearch.node.name": "node-0",
+  "elasticsearch.cluster.name": "distribution_run"
 }
+
 ---------------------------
 // NOTCONSOLE
 
+If a call was initiated with an `X-Opaque-ID` header, then the ID is included
+in Search Slow logs in the **elasticsearch.slowlog.id** field. See
+<<x-opaque-id, X-Opaque-Id HTTP header>> for details and best practices.
+
 [discrete]
 [[index-slow-log]]
 === Index Slow log
@@ -119,6 +142,18 @@ PUT /my-index-000001/_settings
 --------------------------------------------------
 // TEST[setup:my_index]
 
+To include information about the user that triggered a slow indexing event,
+use the `index.indexing.slowlog.include.user` setting.
+
+[source,console]
+--------------------------------------------------
+PUT /my-index-000001/_settings
+{
+  "index.indexing.slowlog.include.user": true
+}
+--------------------------------------------------
+// TEST[setup:my_index]
+
 By default Elasticsearch will log the first 1000 characters of the _source in
 the slowlog. You can change that with `index.indexing.slowlog.source`. Setting
 it to `false` or `0` will skip logging the source entirely, while setting it to