Add the OTel quickstart (#4615)

(cherry picked from commit 756723f)

# Conflicts:
#	docs/en/serverless/index.asciidoc

docs/en/observability/index.asciidoc

@@ -22,6 +22,8 @@ include::quickstarts/monitor-hosts-with-elastic-agent.asciidoc[leveloffset=+2]
 
 include::quickstarts/monitor-k8s-logs-metrics.asciidoc[leveloffset=+2]
 
+include::quickstarts/monitor-hosts-with-otel.asciidoc[leveloffset=+2]
+
 include::quickstarts/monitor-k8s-otel.asciidoc[leveloffset=+2]
 
 include::quickstarts/collect-data-with-aws-firehose.asciidoc[leveloffset=+2]

docs/en/observability/quickstarts/monitor-hosts-with-otel.asciidoc

@@ -0,0 +1,70 @@
+[[quickstart-monitor-hosts-with-otel]]
+= Quickstart: Monitor hosts with OpenTelemetry
+
+preview::[]
+
+In this quickstart guide, you'll learn how to monitor your hosts using the Elastic Distribution of OpenTelemetry (EDOT) Collector.
+You'll also learn how to use {observability} features to gain deeper insight into your observability data after collecting it.
+
+[discrete]
+== Prerequisites
+
+* An {es} cluster for storing and searching your data, and {kib} for visualizing and managing your data. This quickstart is available for all Elastic deployment models. The quickest way to get started with this quickstart is using a trial project on {serverless-docs}/quickstart-monitor-hosts-with-otel.html[Elastic serverless].
+* This quickstart is only available for Linux and MacOS systems.
+* A user with the **Admin** role or higher—required to onboard system logs and metrics. To learn more, refer to {cloud}/ec-user-privileges.html[User roles and privileges].
+* Root privileges on the host—required to run the OpenTelemetry collector because of these components:
+** `hostmetrics` receiver to read all system metrics (all processes, memory, etc.).
+** `filelog` to allow the collector to read any user or application log files.
+
+
+[discrete]
+== Limitations
+Refer to https://github.com/elastic/opentelemetry/blob/main/docs/collector-limitations.md[Elastic OpenTelemetry Collector limitations] for known limitations when using the EDOT Collector.
+
+[discrete]
+== Collect your data
+
+Follow these steps to collect logs and metrics using the EDOT Collector:
+
+. In {kib}, go to the **Observability** UI and click **Add Data**.
+. Under **What do you want to monitor?** select **Host**, and then select **OpenTelemetry: Logs & Metrics**.
++
+[role="screenshot"]
+image::images/quickstart-monitor-hosts-otel-entry-point.png[Host monitoring entry point]
+. Select the appropriate platform.
+. Copy the command under step 1, open a terminal on your host, and run the command.
++
+This command downloads the {agent} package, extracts it in a EDOT directory. For example, `elastic-distro-8.16.0-linux-x86_64`.
+It also adds a sample `otel.yml` configuration file to the directory and updates the storage directory, Elastic endpoint, and API key in the file.
++
+The default log path is `/var/log/*.log`. To update the path, modify the `otel.yml` in the EDOT directory.
++
+Find additional sample `otel.yml` configuration files in the EDOT directory in the `otel_samples` folder.
+. Copy the command under Step 2 and run it in your terminal to start the EDOT Collector.
+
+NOTE: Logs are collected from setup onward, so you won't see logs that occurred before starting the EDOT Collector.
+
+Under **Visualize your data**, you'll see links to **Logs Explorer** to view your logs and **Hosts** to view your host metrics.
+
+[discrete]
+== Gain deeper insight into your host data
+
+After using the Hosts page and Discover to confirm you've ingested all the host logs and metrics you want to monitor,
+use Elastic {observability} to gain deeper insight into your host data with the following capabilities and features:
+
+* In the <<monitor-infrastructure-and-hosts,Infrastructure UI>>, analyze and compare data collected from your hosts.
+You can also:
+** <<inspect-metric-anomalies,Detect anomalies>> for memory usage and network traffic on hosts.
+** <<create-alerts,Create alerts>> that notify you when an anomaly is detected or a metric exceeds a given value.
+* In the <<explore-logs,Logs Explorer>>, search and filter your log data,
+get information about the structure of log fields, and display your findings in a visualization.
+You can also:
+** <<monitor-datasets,Monitor log data set quality>> to find degraded documents.
+** {kibana-ref}/xpack-ml-aiops.html#log-pattern-analysis[Run a pattern analysis] to find patterns in unstructured log messages.
+** <<create-alerts,Create alerts>> that notify you when an Observability data type reaches or exceeds a given value.
+* Use {kibana-ref}/xpack-ml.html[machine learning] to apply predictive analytics to your data:
+** {kibana-ref}/xpack-ml-anomalies.html[Detect anomalies] by comparing real-time and historical data from different sources to look for unusual, problematic patterns.
+** {kibana-ref}/xpack-ml-aiops.html#log-rate-analysis[Analyze log spikes and drops].
+** {kibana-ref}/xpack-ml-aiops.html#change-point-detection[Detect change points] in your time series data.
+
+Refer to the <<observability-introduction>> for a description of other useful features.

docs/en/serverless/index.asciidoc

@@ -0,0 +1,187 @@
+include::{asciidoc-dir}/../../shared/versions/stack/current.asciidoc[]
+include::{asciidoc-dir}/../../shared/attributes.asciidoc[]
+
+[[what-is-observability-serverless]]
+== {obs-serverless}
+
+++++
+<titleabbrev>Elastic Observability</titleabbrev>
+++++
+
+include::./what-is-observability-serverless.asciidoc[leveloffset=+2]
+
+// Group: Get started with Elastic Observability Serverless
+include::observability-get-started.asciidoc[leveloffset=+2]
+
+// What is Observability?
+include::./observability-overview.asciidoc[leveloffset=+3]
+
+// Observability billing dimensions
+include::./projects/billing.asciidoc[leveloffset=+3]
+
+// Create an Elastic Observability Serverless project
+include::./projects/create-an-observability-project.asciidoc[leveloffset=+3]
+
+// Quickstarts
+include::./quickstarts/monitor-hosts-with-elastic-agent.asciidoc[leveloffset=+3]
+include::./quickstarts/k8s-logs-metrics.asciidoc[leveloffset=+3]
+include::./quickstarts/monitor-hosts-with-otel.asciidoc[leveloffset=+3]
+include::./quickstarts/monitor-k8s-otel.asciidoc[leveloffset=+3]
+include::./quickstarts/collect-data-with-aws-firehose.asciidoc[leveloffset=+3]
+
+// Dashboards
+include::./dashboards/dashboards-and-visualizations.asciidoc[leveloffset=+3]
+
+// Group: Application and service monitoring
+include::./application-and-service-monitoring.asciidoc[leveloffset=+2]
+
+// APM
+include::./apm/apm.asciidoc[leveloffset=+3]
+
+// Synthetics
+include::./synthetics/synthetics-intro.asciidoc[leveloffset=+3]
+
+include::./synthetics/synthetics-get-started.asciidoc[leveloffset=+4]
+include::./synthetics/synthetics-get-started-project.asciidoc[leveloffset=+5]
+include::./synthetics/synthetics-get-started-ui.asciidoc[leveloffset=+5]
+
+include::./synthetics/synthetics-journeys.asciidoc[leveloffset=+4]
+include::./synthetics/synthetics-create-test.asciidoc[leveloffset=+5]
+include::./synthetics/synthetics-monitor-use.asciidoc[leveloffset=+5]
+include::./synthetics/synthetics-recorder.asciidoc[leveloffset=+5]
+
+include::./synthetics/synthetics-lightweight.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-manage-monitors.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-params-secrets.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-analyze.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-private-location.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-command-reference.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-configuration.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-mfa.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-settings.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-feature-roles.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-manage-retention.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-scale-and-architect.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-security-encryption.asciidoc[leveloffset=+4]
+
+include::./synthetics/synthetics-troubleshooting.asciidoc[leveloffset=+4]
+
+// Group: Infrastructure and hosts
+include::./infrastructure-and-host-monitoring-intro.asciidoc[leveloffset=+2]
+
+include::./infra-monitoring/infra-monitoring.asciidoc[leveloffset=+3]
+include::./infra-monitoring/get-started-with-metrics.asciidoc[leveloffset=+4]
+include::./infra-monitoring/view-infrastructure-metrics.asciidoc[leveloffset=4]
+include::./infra-monitoring/analyze-hosts.asciidoc[leveloffset=+4]
+include::./infra-monitoring/detect-metric-anomalies.asciidoc[leveloffset=+4]
+include::./infra-monitoring/configure-infra-settings.asciidoc[leveloffset=+4]
+
+include::./infra-monitoring/troubleshooting-infra.asciidoc[leveloffset=+3]
+include::./infra-monitoring/handle-no-results-found-message.asciidoc[leveloffset=+4]
+
+include::./infra-monitoring/metrics-reference.asciidoc[leveloffset=+3]
+include::./infra-monitoring/host-metrics.asciidoc[leveloffset=+4]
+include::./infra-monitoring/container-metrics.asciidoc[leveloffset=+4]
+include::./infra-monitoring/kubernetes-pod-metrics.asciidoc[leveloffset=+4]
+include::./infra-monitoring/aws-metrics.asciidoc[leveloffset=+4]
+
+// Group: Logs
+// TODO: Check the diff to see which changes need to be applied to these files. Also ask Mike if it's expected for serverless and stateful to be out of sync.
+
+include::./logging/log-monitoring.asciidoc[leveloffset=+2]
+
+include::./logging/get-started-with-logs.asciidoc[leveloffset=+3]
+include::./logging/stream-log-files.asciidoc[leveloffset=+3]
+include::./logging/correlate-application-logs.asciidoc[leveloffset=+3]
+include::./logging/plaintext-application-logs.asciidoc[leveloffset=+4]
+include::./logging/ecs-application-logs.asciidoc[leveloffset=+4]
+include::./logging/send-application-logs.asciidoc[leveloffset=+4]
+include::./logging/parse-log-data.asciidoc[leveloffset=+3]
+include::./logging/filter-and-aggregate-logs.asciidoc[leveloffset=+3]
+include::./logging/view-and-monitor-logs.asciidoc[leveloffset=+3]
+include::./logging/add-logs-service-name.asciidoc[leveloffset=+3]
+include::./logging/run-log-pattern-analysis.asciidoc[leveloffset=+3]
+include::./logging/troubleshoot-logs.asciidoc[leveloffset=+3]
+
+//TODO: Figure out where to put this. It's under "view and analyze data" in stateful, but that category doesn't exist in serverless yet.
+include::./inventory.asciidoc[leveloffset=+2]
+
+// Group: Incident management
+include::./incident-management.asciidoc[leveloffset=+2]
+
+// Alerting
+include::./alerting/alerting.asciidoc[leveloffset=+3]
+include::./alerting/create-manage-rules.asciidoc[leveloffset=+4]
+include::./alerting/aiops-generate-anomaly-alerts.asciidoc[leveloffset=+5]
+include::./alerting/create-anomaly-alert-rule.asciidoc[leveloffset=+5]
+include::./alerting/create-custom-threshold-alert-rule.asciidoc[leveloffset=+5]
+include::./alerting/create-elasticsearch-query-alert-rule.asciidoc[leveloffset=+5]
+include::./alerting/create-error-count-threshold-alert-rule.asciidoc[leveloffset=+5]
+include::./alerting/create-failed-transaction-rate-threshold-alert-rule.asciidoc[leveloffset=+5]
+include::./alerting/create-inventory-threshold-alert-rule.asciidoc[leveloffset=+5]
+include::./alerting/create-latency-threshold-alert-rule.asciidoc[leveloffset=+5]
+include::./alerting/create-slo-burn-rate-alert-rule.asciidoc[leveloffset=+5]
+include::./alerting/synthetic-monitor-status-alert.asciidoc[leveloffset=+5]
+include::./alerting/aggregation-options.asciidoc[leveloffset=+4]
+include::./alerting/rate-aggregation.asciidoc[leveloffset=+5]
+include::./alerting/view-alerts.asciidoc[leveloffset=+4]
+include::./alerting/triage-slo-burn-rate-breaches.asciidoc[leveloffset=+5]
+include::./alerting/triage-threshold-breaches.asciidoc[leveloffset=+5]
+
+// Cases
+include::./cases/cases.asciidoc[leveloffset=+3]
+
+include::./cases/create-manage-cases.asciidoc[leveloffset=+4]
+
+include::./cases/manage-cases-settings.asciidoc[leveloffset=+4]
+
+//SLOs
+include::./slos/slos.asciidoc[leveloffset=+3]
+
+include::./slos/create-an-slo.asciidoc[leveloffset=+4]
+
+//Data Set Quality
+include::./monitor-datasets.asciidoc[leveloffset=+2]
+
+//Observability AI Assistant
+include::./ai-assistant/ai-assistant.asciidoc[leveloffset=+2]
+
+//Machine learning
+
+include::./machine-learning/machine-learning.asciidoc[leveloffset=+2]
+include::./machine-learning/aiops-detect-anomalies.asciidoc[leveloffset=+3]
+include::./machine-learning/aiops-tune-anomaly-detection-job.asciidoc[leveloffset=+4]
+include::./machine-learning/aiops-forecast-anomaly.asciidoc[leveloffset=+4]
+include::./machine-learning/aiops-analyze-spikes.asciidoc[leveloffset=+3]
+include::./machine-learning/aiops-detect-change-points.asciidoc[leveloffset=+3]
+
+// Reference group
+
+include::./reference.asciidoc[leveloffset=+2]
+
+// Fields
+
+include::./reference/metrics-app-fields.asciidoc[leveloffset=+3]
+
+// Elastic Entity Model
+
+include::./reference/elastic-entity-model.asciidoc[leveloffset=+3]
+
+// Technical preview limitations
+
+include::./limitations.asciidoc[leveloffset=+2]
+
+// add redirects file
+include::redirects.asciidoc[]

docs/en/serverless/quickstarts/monitor-hosts-with-otel.asciidoc

@@ -0,0 +1,68 @@
+[[quickstart-monitor-hosts-with-otel]]
+= Quickstart: Monitor hosts with OpenTelemetry
+
+preview::[]
+
+In this quickstart guide, you'll learn how to monitor your hosts using the Elastic Distribution of OpenTelemetry (EDOT) Collector.
+You'll also learn how to use {observability} features to gain deeper insight into your observability data after collecting it.
+
+[discrete]
+== Prerequisites
+
+* An {observability} project. To learn more, refer to <<observability-create-an-observability-project>>.
+* This quickstart is only available for Linux and MacOS systems.
+* A user with the **Admin** role or higher—required to onboard system logs and metrics. To learn more, refer to <<general-assign-user-roles>>.
+* Root privileges on the host—required to run the OpenTelemetry collector because of these components:
+** `hostmetrics` receiver to read all system metrics (all processes, memory, etc.).
+** `filelog` to allow the collector to read any user or application log files.
+
+[discrete]
+== Limitations
+Refer to https://github.com/elastic/opentelemetry/blob/main/docs/collector-limitations.md[Elastic OpenTelemetry Collector limitations] for known limitations when using the EDOT Collector.
+
+[discrete]
+== Collect your data
+
+Follow these steps to collect logs and metrics using the EDOT Collector:
+
+. <<observability-create-an-observability-project,Create a new {obs-serverless} project>>, or open an existing one.
+. To open the quickstart, go to **Add Data**.
+. Select **Collect and analyze logs**, and then select **OpenTelemetry**.
+. Under **What do you want to monitor?** select **Host**, and then select **Elastic Agent: Logs & Metrics**.
++
+[role="screenshot"]
+image::images/quickstart-monitor-hosts-otel-entry-point.png[Host monitoring entry point]
+. Select the appropriate platform, and complete the following:
+   . For **MacOS and Linux**, copy the command, open a terminal on your host, and run the command to download and configure the OpenTelemetry collector.
+   . For **Kubernetes**, download the manifest.
+. Copy the command under Step 2:
+   . For **MacOS and Linux**, run the command in your terminal to start the EDOT Collector.
+   . For **Kubernetes**, run the command from the directory where you downloaded the manifest to install the EDOT Collector on every node of your cluster.
+
+Logs are collected from setup onward, so you won't see logs that occurred before starting the EDOT Collector.
+The default log path is `/var/log/*`. To update the path, modify `otel.yml`.
+
+Under **Visualize your data**, you'll see links to **Logs Explorer** to view your logs and **Hosts** to view your host metrics.
+
+[discrete]
+== Get value out of your data
+
+After using the Hosts page and Discover to confirm you've ingested all the host logs and metrics you want to monitor,
+use Elastic {observability} to gain deeper insight into your host data with the following capabilities and features:
+
+* In the <<observability-infrastructure-monitoring,Infrastructure UI>>, analyze and compare data collected from your hosts.
+You can also:
+** <<observability-detect-metric-anomalies,Detect anomalies>> for memory usage and network traffic on hosts.
+** <<observability-create-manage-rules,Create alerts>> that notify you when an anomaly is detected or a metric exceeds a given value.
+* In the <<observability-discover-and-explore-logs,Logs Explorer>>, search and filter your log data,
+get information about the structure of log fields, and display your findings in a visualization.
+You can also:
+** <<observability-monitor-datasets,Monitor log data set quality>> to find degraded documents.
+** {kibana-ref}/xpack-ml-aiops.html#log-pattern-analysis[Run a pattern analysis] to find patterns in unstructured log messages.
+** <<observability-create-manage-rules,Create alerts>> that notify you when an Observability data type reaches or exceeds a given value.
+* Use {kibana-ref}/xpack-ml.html[machine learning] to apply predictive analytics to your data:
+** {kibana-ref}/xpack-ml-anomalies.html[Detect anomalies] by comparing real-time and historical data from different sources to look for unusual, problematic patterns.
+** {kibana-ref}/xpack-ml-aiops.html#log-rate-analysis[Analyze log spikes and drops].
+** {kibana-ref}/xpack-ml-aiops.html#change-point-detection[Detect change points] in your time series data.
+
+Refer to the <<what-is-observability-serverless>> for a description of other useful features.