Docs second wave (#721)

* Move stuff around Signed-off-by: Annanay <[email protected]> * Fix troubleshooting section Signed-off-by: Annanay <[email protected]> * Add API documentation Signed-off-by: Annanay <[email protected]> * Address comments Signed-off-by: Annanay <[email protected]>
grafana · Jun 2, 2021 · 41b799c · 41b799c
1 parent c68b8b6
commit 41b799c
Show file tree

Hide file tree

Showing 22 changed files with 352 additions and 142 deletions.
diff --git a/docs/tempo/website/_index.md b/docs/tempo/website/_index.md
@@ -13,9 +13,10 @@ Grafana Tempo is an open source, easy-to-use and high-volume distributed tracing
 
 - [Getting Started](getting-started/)
 - [Configuration](configuration/)
-- [Monitoring](monitoring/)
+- [Deployment](deployment/)
+- [Operations](operations/)
+- [API](api_docs/) 
 - [Integration Guides/Trace Discovery](guides/)
-- [Tempo CLI](cli/)
 - [Architecture](architecture/)
 - [Troubleshooting](troubleshooting/)
 - [Community](community/)
diff --git a/docs/tempo/website/api_docs/_index.md b/docs/tempo/website/api_docs/_index.md
@@ -0,0 +1,186 @@
+---
+title: API documentation
+weight: 350
+---
+
+# Tempo's API
+
+Tempo exposes an API for pushing and querying traces, and operating the cluster itself.
+
+For the sake of clarity, in this document we have grouped API endpoints by service, but keep in mind that they're exposed both when running Tempo in microservices and singly-binary mode:
+- **Microservices**: each service exposes its own endpoints
+- **Single-binary**: the Tempo process exposes all API endpoints for the services running internally
+
+## Endpoints
+
+| API | Service | Type | Endpoint |
+| --- | ------- | ---- | -------- |
+| [Configuration](#configuration) | _All services_ |  HTTP | `GET /config` |
+| [Readiness probe](#readiness-probe) | _All services_ |  HTTP | `GET /ready` |
+| [Metrics](#metrics) | _All services_ |  HTTP | `GET /metrics` |
+| [Pprof](#pprof) | _All services_ |  HTTP | `GET /debug/pprof` |
+| [Ingest traces](#ingest) | Distributor |  - | See section for details |
+| [Querying traces](#query) | Query-frontend |  HTTP | `GET /api/traces/<traceID>` |
+| [Query Path Readiness Check](#query-path-readiness-check) | Query-frontend |  HTTP | `GET /api/echo` |
+| [Memberlist](#memberlist) | Distributor, Ingester, Querier, Compactor |  HTTP | `GET /memberlist` |
+| [Flush](#flush) | Ingester |  HTTP | `GET,POST /flush` |
+| [Shutdown](#shutdown) | Ingester |  HTTP | `GET,POST /shutdown` |
+| [Distributor ring status](#distributor-ring-status) | Distributor |  HTTP | `GET /distributor/ring` |
+| [Ingesters ring status](#ingesters-ring-status) | Distributor, Querier |  HTTP | `GET /ingester/ring` |
+| [Compactor ring status](#compactor-ring-status) | Compactor |  HTTP | `GET /compactor/ring` |
+
+
+### Configuration
+
+```
+GET /config
+```
+
+Displays the configuration currently applied to Tempo (in YAML format), including default values and settings via CLI flags.
+Sensitive data is masked. Please be aware that the exported configuration **doesn't include the per-tenant overrides**.
+
+
+### Readiness probe
+
+```
+GET /ready
+```
+
+Returns status code 200 when Tempo is ready to serve traffic.
+
+### Metrics
+
+```
+GET /metrics
+```
+
+Returns the metrics for the running Tempo service in the Prometheus exposition format.
+
+### Pprof
+
+```
+GET /debug/pprof/heap
+GET /debug/pprof/block
+GET /debug/pprof/profile
+GET /debug/pprof/trace
+GET /debug/pprof/goroutine
+GET /debug/pprof/mutex
+```
+
+Returns the runtime profiling data in the format expected by the pprof visualization tool.
+There are many things which can be profiled using this including heap, trace, goroutine, etc.
+
+_For more information, please check out the official documentation of [pprof](https://golang.org/pkg/net/http/pprof/)._
+
+### Ingest
+
+Tempo distributor uses the OpenTelemetry Receivers as a shim to ingest trace data.
+Note that these APIs are meant to be consumed by the corresponding client SDK or a pipeline service like Grafana 
+Agent / OpenTelemetry Collector / Jaeger Agent.
+
+|  Protocol | Type | Docs | 
+|  -------- | ---- | ---- |
+|  OpenTelemetry | GRPC | [Link](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/otlp.md) |
+|  OpenTelemetry | HTTP | [Link](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/otlp.md) |
+|  Jaeger | Thrift Compact | [Link](https://www.jaegertracing.io/docs/latest/apis/#span-reporting-apis) |
+|  Jaeger | Thrift Binary | [Link](https://www.jaegertracing.io/docs/latest/apis/#span-reporting-apis) |
+|  Jaeger | Thrift HTTP |  [Link](https://www.jaegertracing.io/docs/latest/apis/#span-reporting-apis) |
+|  Jaeger | GRPC | [Link](https://www.jaegertracing.io/docs/latest/apis/#span-reporting-apis) |
+|  Zipkin | HTTP | [Link](https://zipkin.io/zipkin-api/) |
+
+_For information on how to use the Zipkin endpoint with curl (for debugging purposes) check [here](pushing-spans-with-http)._ 
+
+### Query
+
+Tempo's Query API is simple. The following request is used to retrieve a trace from the query frontend service in 
+a microservices deployment, or the Tempo endpoint in a single binary deployment.
+
+```
+GET /api/traces/<traceid>
+```
+
+The following query API is also provided on the querier service for _debugging_ purposes.
+
+```
+GET /querier/api/traces/<traceid>?mode=xxxx&blockStart=0000&blockEnd=FFFF
+```
+Parameters:
+- `mode = (blocks|ingesters|all)`
+  Specifies whether the querier should look for the trace in blocks, ingesters or both (all).
+  Default = `all`
+- `blockStart = (GUID)`
+  Specifies the blockID start boundary. If specified, the querier will only search blocks with IDs > blockStart.
+  Default = `00000000-0000-0000-0000-000000000000`
+  Example: `blockStart=12345678-0000-0000-1235-000001240000`
+- `blockEnd = (GUID)`
+  Specifies the blockID finish boundary. If specified, the querier will only search blocks with IDs < blockEnd.
+  Default = `FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF`
+  Example: `blockStart=FFFFFFFF-FFFF-FFFF-FFFF-456787652341`
+
+Note that this API is not meant to be used directly unless for debugging the sharding functionality of the query 
+frontend.
+
+### Query Path Readiness Check
+
+```
+GET /api/echo
+```
+
+Returns status code 200 and body `echo` when the query frontend is up and ready to receive requests.
+
+**Note**: Meant to be used in a Query Visualization UI like Grafana to test that the Tempo datasource is working.
+
+
+### Flush
+
+```
+GET,POST /flush
+```
+
+Triggers a flush of all in-memory traces to the WAL. Useful at the time of rollout restarts and unexpected crashes.
+
+### Shutdown
+
+```
+GET,POST /shutdown
+```
+
+Flushes all in-memory traces and the WAL to the long term backend. Gracefully exits from the ring. Shuts down the 
+ingester service.
+
+**Note**: This is usually used at the time of scaling down a cluster.
+
+### Distributor ring status
+
+```
+GET /distributor/ring
+```
+
+Displays a web page with the distributor hash ring status, including the state, healthy and last heartbeat time of each
+distributor.
+
+_For more information, check the page on [consistent hash ring](../operations/consistent_hash_ring)._
+
+### Ingesters ring status
+
+```
+GET /ingester/ring
+```
+
+Displays a web page with the ingesters hash ring status, including the state, healthy and last heartbeat time of each ingester.
+
+_For more information, check the page on [consistent hash ring](../operations/consistent_hash_ring)._
+
+
+
+### Compactor ring status
+
+```
+GET /compactor/ring
+```
+
+Displays a web page with the compactor hash ring status, including the state, healthy and last heartbeat time of each
+compactor.
+
+_For more information, check the page on [consistent hash ring](../operations/consistent_hash_ring)._
+
diff --git a/docs/tempo/website/api_docs/pushing-spans-with-http.md b/docs/tempo/website/api_docs/pushing-spans-with-http.md
@@ -0,0 +1,104 @@
+---
+title: Pushing Spans with HTTP
+---
+
+Sometimes using a tracing system is intimidating because it seems like you need complex application instrumentation
+or a span ingestion pipeline in order to push spans.  This guide aims to show an extremely basic technique for
+pushing spans with http/json from a Bash script using the [Zipkin](https://zipkin.io/) receiver.
+
+## Starting Tempo
+
+Let's first start Tempo with the Zipkin receiver configured.  In order to do this create a config file like so:
+
+```yaml
+server:
+  http_listen_port: 3100
+
+distributor:
+  receivers:
+    zipkin:
+
+storage:
+  trace:
+    backend: local
+    local:
+      path: /tmp/tempo/blocks
+```
+
+and run Tempo using it:
+
+```bash
+docker run -p 9411:9411 -p 3100:3100 -v $(pwd)/config.yaml:/config.yaml grafana/tempo:latest -config.file /config.yaml
+```
+
+## Pushing Spans
+
+Now that Tempo is running and listening on port 9411 for [Zipkin spans](https://zipkin.io/zipkin-api/#/default/post_spans) let's push a span to it using `curl`.
+
+```bash
+curl -X POST http://localhost:9411 -H 'Content-Type: application/json' -d '[{
+ "id": "1234",
+ "traceId": "0123456789abcdef",
+ "timestamp": 1608239395286533,
+ "duration": 100000,
+ "name": "span from bash!",
+ "tags": {
+    "http.method": "GET",
+    "http.path": "/api"
+  },
+  "localEndpoint": {
+    "serviceName": "shell script"
+  }
+}]'
+```
+
+Note that the `timestamp` field is in microseconds and was obtained by running `date +%s%6N`.  The `duration` field is also in microseconds and so 100000 is 100 milliseconds.
+
+## Retrieving Traces
+
+The easiest way to get the trace is to execute a simple curl command to Tempo.  The returned format is [OTLP](https://github.com/open-telemetry/opentelemetry-proto/blob/main/opentelemetry/proto/trace/v1/trace.proto).
+
+```bash
+curl http://localhost:3100/api/traces/0123456789abcdef
+
+{"batches":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"shell script"}}]},"instrumentationLibrarySpans":[{"spans":[{"traceId":"AAAAAAAAAAABI0VniavN7w==","spanId":"AAAAAAAAEjQ=","name":"span from bash!","startTimeUnixNano":"1608239395286533000","endTimeUnixNano":"1608239395386533000","attributes":[{"key":"http.path","value":{"stringValue":"/api"}},{"key":"http.method","value":{"stringValue":"GET"}}]}]}]}]}
+```
+
+However, staring at a json blob in bash is not very fun.  Let's start up Tempo query so we can visualize our trace.  Tempo query is [Jaeger Query](https://hub.docker.com/r/jaegertracing/jaeger-query/) with a [GRPC Plugin](https://github.com/jaegertracing/jaeger/tree/master/plugin/storage/grpc) that allows it to query Tempo.
+
+```bash
+docker run --env BACKEND=localhost:3100 --net host grafana/tempo-query:latest
+```
+
+And open `http://localhost:16686/trace/0123456789abcdef` in the browser of your choice to see:
+
+<p align="center"><img src="../pushing-spans-with-http.png" alt="single span"></p>
+
+## More Spans!
+
+Now that we have the basics down it's easy to continue building our trace.  By specifying the same trace id and a parent span id we can start building a trace.
+
+```bash
+curl -X POST http://localhost:9411 -H 'Content-Type: application/json' -d '[{
+ "id": "5678",
+ "traceId": "0123456789abcdef",
+ "parentId": "1234",
+ "timestamp": 1608239395316533,
+ "duration": 100000,
+ "name": "child span from bash!",
+  "localEndpoint": {
+    "serviceName": "shell script"
+  }
+}]'
+```
+
+And now the UI shows:
+<p align="center"><img src="../pushing-spans-with-http2.png" alt="parent and child spans"></p>
+
+## Spans from everything!
+
+Tracing is not limited to enterprise languages with complex frameworks.  As you can see it's easy to store and track events from your js, python or bash scripts.
+You can use Tempo/distributed tracing today to trace CI pipelines, long running bash processes, python data processing flows or anything else
+you can think of.
+
+Happy tracing!
diff --git a/...ebsite/guides/pushing-spans-with-http.png → ...site/api_docs/pushing-spans-with-http.png b/...ebsite/guides/pushing-spans-with-http.png → ...site/api_docs/pushing-spans-with-http.png
diff --git a/...bsite/guides/pushing-spans-with-http2.png → ...ite/api_docs/pushing-spans-with-http2.png b/...bsite/guides/pushing-spans-with-http2.png → ...ite/api_docs/pushing-spans-with-http2.png
diff --git a/docs/tempo/website/community/_index.md b/docs/tempo/website/community/_index.md
@@ -3,7 +3,16 @@ title: Community
 weight: 600
 ---
 
-# Contribute
+## Communicate
+
+- [Grafana Slack](https://slack.grafana.com/) #tempo channel
+- [Community Forum](https://community.grafana.com/c/grafana-tempo/40) - for questions/feedback.
+- [Community Call](https://docs.google.com/document/d/1yGsI6ywU-PxZBjmq3p3vAXr9g5yBXSDk4NU8LGo8qeY/edit#) - Monthly
+  on the second thursday at 1630 UTC.
+  Recordings available [online](https://www.youtube.com/playlist?list=PLDGkOdUX1Ujqe8WZ8T1h2pNjpll0t-KLw).
+- [Google Groups](https://groups.google.com/forum/#!forum/tempo-users)
+
+## Contribute
 This page lists resources for developers who want to contribute to the Tempo software ecosystem.
 
 - [Governance](https://github.com/grafana/tempo/blob/main/GOVERNANCE.md)

diff --git a/docs/tempo/website/community/communication.md b/docs/tempo/website/community/communication.md
diff --git a/docs/tempo/website/guides/_index.md b/docs/tempo/website/guides/_index.md
@@ -6,5 +6,4 @@ weight: 400
 Because Tempo is a trace id only lookup it relies on integrations for trace discovery.  Common methods of discovery are through logs and exemplars. [The examples](https://github.com/grafana/tempo/tree/main/example) are also a good place to see how some of these discovery flows work.
 
 - [Loki Derived Fields](loki-derived-fields/)
-- [Pushing Spans with HTTP](pushing-spans-with-http/)
 - [Instrumentation Examples](instrumentation/)