diff --git a/docs/en/observability/images/quickstart-monitor-hosts-otel-entry-point.png b/docs/en/observability/images/quickstart-monitor-hosts-otel-entry-point.png new file mode 100644 index 0000000000..fc5a000c9e Binary files /dev/null and b/docs/en/observability/images/quickstart-monitor-hosts-otel-entry-point.png differ diff --git a/docs/en/observability/index.asciidoc b/docs/en/observability/index.asciidoc index 4866d5393c..b288d07d09 100644 --- a/docs/en/observability/index.asciidoc +++ b/docs/en/observability/index.asciidoc @@ -22,6 +22,8 @@ include::quickstarts/monitor-hosts-with-elastic-agent.asciidoc[leveloffset=+2] include::quickstarts/monitor-k8s-logs-metrics.asciidoc[leveloffset=+2] +include::quickstarts/monitor-hosts-with-otel.asciidoc[leveloffset=+2] + include::quickstarts/monitor-k8s-otel.asciidoc[leveloffset=+2] include::quickstarts/collect-data-with-aws-firehose.asciidoc[leveloffset=+2] diff --git a/docs/en/observability/quickstarts/monitor-hosts-with-otel.asciidoc b/docs/en/observability/quickstarts/monitor-hosts-with-otel.asciidoc new file mode 100644 index 0000000000..47f3b75dce --- /dev/null +++ b/docs/en/observability/quickstarts/monitor-hosts-with-otel.asciidoc @@ -0,0 +1,70 @@ +[[quickstart-monitor-hosts-with-otel]] += Quickstart: Monitor hosts with OpenTelemetry + +preview::[] + +In this quickstart guide, you'll learn how to monitor your hosts using the Elastic Distribution of OpenTelemetry (EDOT) Collector. +You'll also learn how to use {observability} features to gain deeper insight into your observability data after collecting it. + +[discrete] +== Prerequisites + +* An {es} cluster for storing and searching your data, and {kib} for visualizing and managing your data. This quickstart is available for all Elastic deployment models. The quickest way to get started with this quickstart is using a trial project on {serverless-docs}/quickstart-monitor-hosts-with-otel.html[Elastic serverless]. +* This quickstart is only available for Linux and MacOS systems. +* A user with the **Admin** role or higher—required to onboard system logs and metrics. To learn more, refer to {cloud}/ec-user-privileges.html[User roles and privileges]. +* Root privileges on the host—required to run the OpenTelemetry collector because of these components: +** `hostmetrics` receiver to read all system metrics (all processes, memory, etc.). +** `filelog` to allow the collector to read any user or application log files. + + +[discrete] +== Limitations +Refer to https://github.com/elastic/opentelemetry/blob/main/docs/collector-limitations.md[Elastic OpenTelemetry Collector limitations] for known limitations when using the EDOT Collector. + +[discrete] +== Collect your data + +Follow these steps to collect logs and metrics using the EDOT Collector: + +. In {kib}, go to the **Observability** UI and click **Add Data**. +. Under **What do you want to monitor?** select **Host**, and then select **OpenTelemetry: Logs & Metrics**. ++ +[role="screenshot"] +image::images/quickstart-monitor-hosts-otel-entry-point.png[Host monitoring entry point] +. Select the appropriate platform. +. Copy the command under step 1, open a terminal on your host, and run the command. ++ +This command downloads the {agent} package, extracts it in a EDOT directory. For example, `elastic-distro-8.16.0-linux-x86_64`. +It also adds a sample `otel.yml` configuration file to the directory and updates the storage directory, Elastic endpoint, and API key in the file. ++ +The default log path is `/var/log/*.log`. To update the path, modify the `otel.yml` in the EDOT directory. ++ +Find additional sample `otel.yml` configuration files in the EDOT directory in the `otel_samples` folder. +. Copy the command under Step 2 and run it in your terminal to start the EDOT Collector. + +NOTE: Logs are collected from setup onward, so you won't see logs that occurred before starting the EDOT Collector. + +Under **Visualize your data**, you'll see links to **Logs Explorer** to view your logs and **Hosts** to view your host metrics. + +[discrete] +== Gain deeper insight into your host data + +After using the Hosts page and Discover to confirm you've ingested all the host logs and metrics you want to monitor, +use Elastic {observability} to gain deeper insight into your host data with the following capabilities and features: + +* In the <>, analyze and compare data collected from your hosts. +You can also: +** <> for memory usage and network traffic on hosts. +** <> that notify you when an anomaly is detected or a metric exceeds a given value. +* In the <>, search and filter your log data, +get information about the structure of log fields, and display your findings in a visualization. +You can also: +** <> to find degraded documents. +** {kibana-ref}/xpack-ml-aiops.html#log-pattern-analysis[Run a pattern analysis] to find patterns in unstructured log messages. +** <> that notify you when an Observability data type reaches or exceeds a given value. +* Use {kibana-ref}/xpack-ml.html[machine learning] to apply predictive analytics to your data: +** {kibana-ref}/xpack-ml-anomalies.html[Detect anomalies] by comparing real-time and historical data from different sources to look for unusual, problematic patterns. +** {kibana-ref}/xpack-ml-aiops.html#log-rate-analysis[Analyze log spikes and drops]. +** {kibana-ref}/xpack-ml-aiops.html#change-point-detection[Detect change points] in your time series data. + +Refer to the <> for a description of other useful features. diff --git a/docs/en/serverless/images/quickstart-monitor-hosts-otel-entry-point.png b/docs/en/serverless/images/quickstart-monitor-hosts-otel-entry-point.png new file mode 100644 index 0000000000..fc5a000c9e Binary files /dev/null and b/docs/en/serverless/images/quickstart-monitor-hosts-otel-entry-point.png differ diff --git a/docs/en/serverless/index.asciidoc b/docs/en/serverless/index.asciidoc index d0b4724ad6..14001a99ea 100644 --- a/docs/en/serverless/index.asciidoc +++ b/docs/en/serverless/index.asciidoc @@ -25,6 +25,7 @@ include::./projects/create-an-observability-project.asciidoc[leveloffset=+3] // Quickstarts include::./quickstarts/monitor-hosts-with-elastic-agent.asciidoc[leveloffset=+3] include::./quickstarts/k8s-logs-metrics.asciidoc[leveloffset=+3] +include::./quickstarts/monitor-hosts-with-otel.asciidoc[leveloffset=+3] include::./quickstarts/monitor-k8s-otel.asciidoc[leveloffset=+3] include::./quickstarts/collect-data-with-aws-firehose.asciidoc[leveloffset=+3] diff --git a/docs/en/serverless/quickstarts/monitor-hosts-with-otel.asciidoc b/docs/en/serverless/quickstarts/monitor-hosts-with-otel.asciidoc new file mode 100644 index 0000000000..47b52b9ab5 --- /dev/null +++ b/docs/en/serverless/quickstarts/monitor-hosts-with-otel.asciidoc @@ -0,0 +1,68 @@ +[[quickstart-monitor-hosts-with-otel]] += Quickstart: Monitor hosts with OpenTelemetry + +preview::[] + +In this quickstart guide, you'll learn how to monitor your hosts using the Elastic Distribution of OpenTelemetry (EDOT) Collector. +You'll also learn how to use {observability} features to gain deeper insight into your observability data after collecting it. + +[discrete] +== Prerequisites + +* An {observability} project. To learn more, refer to <>. +* This quickstart is only available for Linux and MacOS systems. +* A user with the **Admin** role or higher—required to onboard system logs and metrics. To learn more, refer to <>. +* Root privileges on the host—required to run the OpenTelemetry collector because of these components: +** `hostmetrics` receiver to read all system metrics (all processes, memory, etc.). +** `filelog` to allow the collector to read any user or application log files. + +[discrete] +== Limitations +Refer to https://github.com/elastic/opentelemetry/blob/main/docs/collector-limitations.md[Elastic OpenTelemetry Collector limitations] for known limitations when using the EDOT Collector. + +[discrete] +== Collect your data + +Follow these steps to collect logs and metrics using the EDOT Collector: + +. <>, or open an existing one. +. To open the quickstart, go to **Add Data**. +. Select **Collect and analyze logs**, and then select **OpenTelemetry**. +. Under **What do you want to monitor?** select **Host**, and then select **Elastic Agent: Logs & Metrics**. ++ +[role="screenshot"] +image::images/quickstart-monitor-hosts-otel-entry-point.png[Host monitoring entry point] +. Select the appropriate platform, and complete the following: + . For **MacOS and Linux**, copy the command, open a terminal on your host, and run the command to download and configure the OpenTelemetry collector. + . For **Kubernetes**, download the manifest. +. Copy the command under Step 2: + . For **MacOS and Linux**, run the command in your terminal to start the EDOT Collector. + . For **Kubernetes**, run the command from the directory where you downloaded the manifest to install the EDOT Collector on every node of your cluster. + +Logs are collected from setup onward, so you won't see logs that occurred before starting the EDOT Collector. +The default log path is `/var/log/*`. To update the path, modify `otel.yml`. + +Under **Visualize your data**, you'll see links to **Logs Explorer** to view your logs and **Hosts** to view your host metrics. + +[discrete] +== Get value out of your data + +After using the Hosts page and Discover to confirm you've ingested all the host logs and metrics you want to monitor, +use Elastic {observability} to gain deeper insight into your host data with the following capabilities and features: + +* In the <>, analyze and compare data collected from your hosts. +You can also: +** <> for memory usage and network traffic on hosts. +** <> that notify you when an anomaly is detected or a metric exceeds a given value. +* In the <>, search and filter your log data, +get information about the structure of log fields, and display your findings in a visualization. +You can also: +** <> to find degraded documents. +** {kibana-ref}/xpack-ml-aiops.html#log-pattern-analysis[Run a pattern analysis] to find patterns in unstructured log messages. +** <> that notify you when an Observability data type reaches or exceeds a given value. +* Use {kibana-ref}/xpack-ml.html[machine learning] to apply predictive analytics to your data: +** {kibana-ref}/xpack-ml-anomalies.html[Detect anomalies] by comparing real-time and historical data from different sources to look for unusual, problematic patterns. +** {kibana-ref}/xpack-ml-aiops.html#log-rate-analysis[Analyze log spikes and drops]. +** {kibana-ref}/xpack-ml-aiops.html#change-point-detection[Detect change points] in your time series data. + +Refer to the <> for a description of other useful features.