diff --git a/docs/gitbook/.gitbook/assets/image (10).png b/docs/gitbook/.gitbook/assets/image (10).png new file mode 100644 index 0000000000..31a8a702b6 Binary files /dev/null and b/docs/gitbook/.gitbook/assets/image (10).png differ diff --git a/docs/gitbook/.gitbook/assets/image (11).png b/docs/gitbook/.gitbook/assets/image (11).png new file mode 100644 index 0000000000..656753225f Binary files /dev/null and b/docs/gitbook/.gitbook/assets/image (11).png differ diff --git a/docs/gitbook/.gitbook/assets/image (5).png b/docs/gitbook/.gitbook/assets/image (5).png new file mode 100644 index 0000000000..b6d84aedee Binary files /dev/null and b/docs/gitbook/.gitbook/assets/image (5).png differ diff --git a/docs/gitbook/.gitbook/assets/image (6).png b/docs/gitbook/.gitbook/assets/image (6).png new file mode 100644 index 0000000000..d7aa018eb4 Binary files /dev/null and b/docs/gitbook/.gitbook/assets/image (6).png differ diff --git a/docs/gitbook/.gitbook/assets/image (7).png b/docs/gitbook/.gitbook/assets/image (7).png new file mode 100644 index 0000000000..d7aa018eb4 Binary files /dev/null and b/docs/gitbook/.gitbook/assets/image (7).png differ diff --git a/docs/gitbook/.gitbook/assets/image (8).png b/docs/gitbook/.gitbook/assets/image (8).png new file mode 100644 index 0000000000..c7c1340ba5 Binary files /dev/null and b/docs/gitbook/.gitbook/assets/image (8).png differ diff --git a/docs/gitbook/.gitbook/assets/image (9).png b/docs/gitbook/.gitbook/assets/image (9).png new file mode 100644 index 0000000000..7d53b02677 Binary files /dev/null and b/docs/gitbook/.gitbook/assets/image (9).png differ diff --git a/docs/gitbook/SUMMARY.md b/docs/gitbook/SUMMARY.md index 639fafebc2..4dd39d5106 100644 --- a/docs/gitbook/SUMMARY.md +++ b/docs/gitbook/SUMMARY.md @@ -1,130 +1,134 @@ # Table of contents -- [What is BullMQ](README.md) -- [Quick Start]() -- [API Reference](https://api.docs.bullmq.io) -- [Changelogs](changelog.md) - - [v4](changelogs/changelog-v4.md) - - [v3](changelogs/changelog-v3.md) - - [v2](changelogs/changelog-v2.md) - - [v1](changelogs/changelog-v1.md) +* [What is BullMQ](README.md) +* [Quick Start]() +* [API Reference](https://api.docs.bullmq.io) +* [Changelogs](changelog.md) + * [v4](changelogs/changelog-v4.md) + * [v3](changelogs/changelog-v3.md) + * [v2](changelogs/changelog-v2.md) + * [v1](changelogs/changelog-v1.md) ## Guide -- [Introduction](guide/introduction.md) -- [Connections](guide/connections.md) -- [Queues](guide/queues/README.md) - - [Auto-removal of jobs](guide/queues/auto-removal-of-jobs.md) - - [Adding jobs in bulk](guide/queues/adding-bulks.md) - - [Global Concurrency](guide/queues/global-concurrency.md) - - [Removing Jobs](guide/queues/removing-jobs.md) -- [Workers](guide/workers/README.md) - - [Auto-removal of jobs](guide/workers/auto-removal-of-jobs.md) - - [Concurrency](guide/workers/concurrency.md) - - [Graceful shutdown](guide/workers/graceful-shutdown.md) - - [Stalled Jobs](guide/workers/stalled-jobs.md) - - [Sandboxed processors](guide/workers/sandboxed-processors.md) - - [Pausing queues](guide/workers/pausing-queues.md) -- [Jobs](guide/jobs/README.md) - - [FIFO](guide/jobs/fifo.md) - - [LIFO](guide/jobs/lifo.md) - - [Job Ids](guide/jobs/job-ids.md) - - [Job Data](guide/jobs/job-data.md) - - [Deduplication](guide/jobs/deduplication.md) - - [Delayed](guide/jobs/delayed.md) - - [Repeatable](guide/jobs/repeatable.md) - - [Prioritized](guide/jobs/prioritized.md) - - [Removing jobs](guide/jobs/removing-job.md) - - [Stalled](guide/jobs/stalled.md) - - [Getters](guide/jobs/getters.md) -- [Job Schedulers](guide/job-schedulers/README.md) - - [Repeat Strategies](guide/job-schedulers/repeat-strategies.md) - - [Repeat options](guide/job-schedulers/repeat-options.md) - - [Manage Job Schedulers](guide/job-schedulers/manage-job-schedulers.md) -- [Flows](guide/flows/README.md) - - [Adding flows in bulk](guide/flows/adding-bulks.md) - - [Get Flow Tree](guide/flows/get-flow-tree.md) - - [Fail Parent](guide/flows/fail-parent.md) - - [Remove Dependency](guide/flows/remove-dependency.md) - - [Ignore Dependency](guide/flows/ignore-dependency.md) - - [Remove Child Dependency](guide/flows/remove-child-dependency.md) -- [Metrics](guide/metrics/metrics.md) -- [Rate limiting](guide/rate-limiting.md) -- [Parallelism and Concurrency](guide/parallelism-and-concurrency.md) -- [Retrying failing jobs](guide/retrying-failing-jobs.md) -- [Returning job data](guide/returning-job-data.md) -- [Events](guide/events.md) -- [QueueScheduler](guide/queuescheduler.md) -- [Redis™ Compatibility](guide/redis-tm-compatibility/README.md) - - [Dragonfly](guide/redis-tm-compatibility/dragonfly.md) -- [Redis™ hosting](guide/redis-tm-hosting/README.md) - - [AWS MemoryDB](guide/redis-tm-hosting/aws-memorydb.md) - - [AWS Elasticache](guide/redis-tm-hosting/aws-elasticache.md) -- [Architecture](guide/architecture.md) -- [NestJs](guide/nestjs/README.md) - - [Producers](guide/nestjs/producers.md) - - [Queue Events Listeners](guide/nestjs/queue-events-listeners.md) -- [Going to production](guide/going-to-production.md) -- [Migration to newer versions](guide/migration-to-newer-versions.md) -- [Troubleshooting](guide/troubleshooting.md) +* [Introduction](guide/introduction.md) +* [Connections](guide/connections.md) +* [Queues](guide/queues/README.md) + * [Auto-removal of jobs](guide/queues/auto-removal-of-jobs.md) + * [Adding jobs in bulk](guide/queues/adding-bulks.md) + * [Global Concurrency](guide/queues/global-concurrency.md) + * [Removing Jobs](guide/queues/removing-jobs.md) +* [Workers](guide/workers/README.md) + * [Auto-removal of jobs](guide/workers/auto-removal-of-jobs.md) + * [Concurrency](guide/workers/concurrency.md) + * [Graceful shutdown](guide/workers/graceful-shutdown.md) + * [Stalled Jobs](guide/workers/stalled-jobs.md) + * [Sandboxed processors](guide/workers/sandboxed-processors.md) + * [Pausing queues](guide/workers/pausing-queues.md) +* [Jobs](guide/jobs/README.md) + * [FIFO](guide/jobs/fifo.md) + * [LIFO](guide/jobs/lifo.md) + * [Job Ids](guide/jobs/job-ids.md) + * [Job Data](guide/jobs/job-data.md) + * [Deduplication](guide/jobs/deduplication.md) + * [Delayed](guide/jobs/delayed.md) + * [Repeatable](guide/jobs/repeatable.md) + * [Prioritized](guide/jobs/prioritized.md) + * [Removing jobs](guide/jobs/removing-job.md) + * [Stalled](guide/jobs/stalled.md) + * [Getters](guide/jobs/getters.md) +* [Job Schedulers](guide/job-schedulers/README.md) + * [Repeat Strategies](guide/job-schedulers/repeat-strategies.md) + * [Repeat options](guide/job-schedulers/repeat-options.md) + * [Manage Job Schedulers](guide/job-schedulers/manage-job-schedulers.md) +* [Flows](guide/flows/README.md) + * [Adding flows in bulk](guide/flows/adding-bulks.md) + * [Get Flow Tree](guide/flows/get-flow-tree.md) + * [Fail Parent](guide/flows/fail-parent.md) + * [Remove Dependency](guide/flows/remove-dependency.md) + * [Ignore Dependency](guide/flows/ignore-dependency.md) + * [Remove Child Dependency](guide/flows/remove-child-dependency.md) +* [Metrics](guide/metrics/metrics.md) +* [Rate limiting](guide/rate-limiting.md) +* [Parallelism and Concurrency](guide/parallelism-and-concurrency.md) +* [Retrying failing jobs](guide/retrying-failing-jobs.md) +* [Returning job data](guide/returning-job-data.md) +* [Events](guide/events.md) +* [Telemetry](guide/telemetry/README.md) + * [Getting started](guide/telemetry/getting-started.md) + * [Running Jaeger](guide/telemetry/running-jaeger.md) + * [Running a simple example](guide/telemetry/running-a-simple-example.md) +* [QueueScheduler](guide/queuescheduler.md) +* [Redis™ Compatibility](guide/redis-tm-compatibility/README.md) + * [Dragonfly](guide/redis-tm-compatibility/dragonfly.md) +* [Redis™ hosting](guide/redis-tm-hosting/README.md) + * [AWS MemoryDB](guide/redis-tm-hosting/aws-memorydb.md) + * [AWS Elasticache](guide/redis-tm-hosting/aws-elasticache.md) +* [Architecture](guide/architecture.md) +* [NestJs](guide/nestjs/README.md) + * [Producers](guide/nestjs/producers.md) + * [Queue Events Listeners](guide/nestjs/queue-events-listeners.md) +* [Going to production](guide/going-to-production.md) +* [Migration to newer versions](guide/migration-to-newer-versions.md) +* [Troubleshooting](guide/troubleshooting.md) ## Patterns -- [Adding jobs in bulk across different queues](patterns/adding-bulks.md) -- [Manually processing jobs](patterns/manually-fetching-jobs.md) -- [Named Processor](patterns/named-processor.md) -- [Flows](patterns/flows.md) -- [Idempotent jobs](patterns/idempotent-jobs.md) -- [Throttle jobs](patterns/throttle-jobs.md) -- [Process Step Jobs](patterns/process-step-jobs.md) -- [Failing fast when Redis is down](patterns/failing-fast-when-redis-is-down.md) -- [Stop retrying jobs](patterns/stop-retrying-jobs.md) -- [Timeout jobs](patterns/timeout-jobs.md) -- [Redis Cluster](patterns/redis-cluster.md) +* [Adding jobs in bulk across different queues](patterns/adding-bulks.md) +* [Manually processing jobs](patterns/manually-fetching-jobs.md) +* [Named Processor](patterns/named-processor.md) +* [Flows](patterns/flows.md) +* [Idempotent jobs](patterns/idempotent-jobs.md) +* [Throttle jobs](patterns/throttle-jobs.md) +* [Process Step Jobs](patterns/process-step-jobs.md) +* [Failing fast when Redis is down](patterns/failing-fast-when-redis-is-down.md) +* [Stop retrying jobs](patterns/stop-retrying-jobs.md) +* [Timeout jobs](patterns/timeout-jobs.md) +* [Redis Cluster](patterns/redis-cluster.md) ## BullMQ Pro -- [Introduction](bullmq-pro/introduction.md) -- [Install](bullmq-pro/install.md) -- [Observables](bullmq-pro/observables/README.md) - - [Cancelation](bullmq-pro/observables/cancelation.md) -- [Groups](bullmq-pro/groups/README.md) - - [Getters](bullmq-pro/groups/getters.md) - - [Rate limiting](bullmq-pro/groups/rate-limiting.md) - - [Concurrency](bullmq-pro/groups/concurrency.md) - - [Local group concurrency](bullmq-pro/groups/local-group-concurrency.md) - - [Max group size](bullmq-pro/groups/max-group-size.md) - - [Pausing groups](bullmq-pro/groups/pausing-groups.md) - - [Prioritized intra-groups](bullmq-pro/groups/prioritized.md) - - [Sandboxes for groups](bullmq-pro/groups/sandboxes-for-groups.md) -- [Batches](bullmq-pro/batches.md) -- [NestJs](bullmq-pro/nestjs/README.md) - - [Producers](bullmq-pro/nestjs/producers.md) - - [Queue Events Listeners](bullmq-pro/nestjs/queue-events-listeners.md) - - [API Reference](https://nestjs.bullmq.pro/) - - [Changelog](bullmq-pro/nestjs/changelog.md) -- [API Reference](https://api.bullmq.pro) -- [Changelog](bullmq-pro/changelog.md) -- [Support](bullmq-pro/support.md) +* [Introduction](bullmq-pro/introduction.md) +* [Install](bullmq-pro/install.md) +* [Observables](bullmq-pro/observables/README.md) + * [Cancelation](bullmq-pro/observables/cancelation.md) +* [Groups](bullmq-pro/groups/README.md) + * [Getters](bullmq-pro/groups/getters.md) + * [Rate limiting](bullmq-pro/groups/rate-limiting.md) + * [Concurrency](bullmq-pro/groups/concurrency.md) + * [Local group concurrency](bullmq-pro/groups/local-group-concurrency.md) + * [Max group size](bullmq-pro/groups/max-group-size.md) + * [Pausing groups](bullmq-pro/groups/pausing-groups.md) + * [Prioritized intra-groups](bullmq-pro/groups/prioritized.md) + * [Sandboxes for groups](bullmq-pro/groups/sandboxes-for-groups.md) +* [Batches](bullmq-pro/batches.md) +* [NestJs](bullmq-pro/nestjs/README.md) + * [Producers](bullmq-pro/nestjs/producers.md) + * [Queue Events Listeners](bullmq-pro/nestjs/queue-events-listeners.md) + * [API Reference](https://nestjs.bullmq.pro/) + * [Changelog](bullmq-pro/nestjs/changelog.md) +* [API Reference](https://api.bullmq.pro) +* [Changelog](bullmq-pro/changelog.md) +* [Support](bullmq-pro/support.md) ## Bull -- [Introduction](bull/introduction.md) -- [Install](bull/install.md) -- [Quick Guide](bull/quick-guide.md) -- [Important Notes](bull/important-notes.md) -- [Reference](https://github.com/OptimalBits/bull/blob/develop/REFERENCE.md) -- [Patterns](bull/patterns/README.md) - - [Persistent connections](bull/patterns/persistent-connections.md) - - [Message queue](bull/patterns/message-queue.md) - - [Returning Job Completions](bull/patterns/returning-job-completions.md) - - [Reusing Redis Connections](bull/patterns/reusing-redis-connections.md) - - [Redis cluster](bull/patterns/redis-cluster.md) - - [Custom backoff strategy](bull/patterns/custom-backoff-strategy.md) - - [Debugging](bull/patterns/debugging.md) - - [Manually fetching jobs](bull/patterns/manually-fetching-jobs.md) +* [Introduction](bull/introduction.md) +* [Install](bull/install.md) +* [Quick Guide](bull/quick-guide.md) +* [Important Notes](bull/important-notes.md) +* [Reference](https://github.com/OptimalBits/bull/blob/develop/REFERENCE.md) +* [Patterns](bull/patterns/README.md) + * [Persistent connections](bull/patterns/persistent-connections.md) + * [Message queue](bull/patterns/message-queue.md) + * [Returning Job Completions](bull/patterns/returning-job-completions.md) + * [Reusing Redis Connections](bull/patterns/reusing-redis-connections.md) + * [Redis cluster](bull/patterns/redis-cluster.md) + * [Custom backoff strategy](bull/patterns/custom-backoff-strategy.md) + * [Debugging](bull/patterns/debugging.md) + * [Manually fetching jobs](bull/patterns/manually-fetching-jobs.md) ## Python -- [Introduction](python/introduction.md) -- [Changelog](python/changelog.md) +* [Introduction](python/introduction.md) +* [Changelog](python/changelog.md) diff --git a/docs/gitbook/guide/telemetry/README.md b/docs/gitbook/guide/telemetry/README.md new file mode 100644 index 0000000000..e89b38eabb --- /dev/null +++ b/docs/gitbook/guide/telemetry/README.md @@ -0,0 +1,10 @@ +--- +description: Enabling Telemetry for your BullMQ based applications +--- + +# Telemetry + +BullMQ provides a Telemetry interface that can be used to integrate it with any external telemetry backends. Currently we support the [OpenTelemetry](https://opentelemetry.io) specification, which is the new de-facto standard for telemetry purposes, however the interface if flexible enough to support any other backends in the future. + +Telemetry is very useful for large applications where you want to get a detailed and general overview of the system. For BullMQ it helps to gain insight in the different statuses a job may be during its complete lifecycle. In a large application it helpts tracking the source of the jobs and all the interactions the jobs or messages may perform with other parts of the system. + diff --git a/docs/gitbook/guide/telemetry/getting-started.md b/docs/gitbook/guide/telemetry/getting-started.md new file mode 100644 index 0000000000..f6c5a608c7 --- /dev/null +++ b/docs/gitbook/guide/telemetry/getting-started.md @@ -0,0 +1,46 @@ +# Getting started + +In this guide we will show how to setup a local telemetry facility for BullMQ that should serve you as a good basis for how to integrate it in larger applications. As OpenTelemetry is a well supported standard there are many third party UIs for visualizing the traces and spans generated when running an application, for this guide we will use [Jaeger](https://www.jaegertracing.io). + +We assume that you have a working BullMQ project that you want to add telemetry to it, so lets start by adding the `bullmq-otel` package to the project: + +``` +npm add --save bullmq-otel +``` + +This module provides a working implementation of BullMQ's telemetry interface for the OpenTelemetry standard. Adding it to your existing Queue's instances and Workers is quite straightforward: + +```typescript +import { Queue } from 'bullmq' +import { BullMQOtel } from "bullmq-otel"; + +const queue = new Queue("myQueue", { + connection: { + host: "127.0.0.1", + port: 6379, + }, + telemetry: new BullMQOtel("simple-guide"), +}); +``` + +```typescript +import { Worker } from "bullmq"; +import { BullMQOtel } from "bullmq-otel"; + +const worker = new Worker( + "myQueue", + async (job) => { + return 'some value' + }, + { + name: "myWorker", + connection: { + host: "127.0.0.1", + port: 6379, + }, + telemetry: new BullMQOtel("simple-guide"), + } +); +``` + +This is all that is needed in order to start producing traces and spans to observe your code. diff --git a/docs/gitbook/guide/telemetry/running-a-simple-example.md b/docs/gitbook/guide/telemetry/running-a-simple-example.md new file mode 100644 index 0000000000..9484c5ff75 --- /dev/null +++ b/docs/gitbook/guide/telemetry/running-a-simple-example.md @@ -0,0 +1,198 @@ +# Running a simple example + +### Creating a producer + +For this simple example we will create a producer that will add a couple of jobs, but it will add them in bulks instead of one by one, this will help us demonstrate how spans are linked between the consumers and the producers: + +{% code title="producer.ts" %} +```typescript +import { Queue } from "bullmq"; +import { BullMQOtel } from "bullmq-otel"; + +const queue = new Queue("myQueue", { + connection: { + host: "127.0.0.1", + port: 6379, + }, + telemetry: new BullMQOtel("simple-guide"), +}); + +const jobsBulk = Array.from({ length: 5 }, (_, i) => i); + +(async () => { + for (let i = 0; i < 10; i++) { + await queue.addBulk( + jobsBulk.map((j) => ({ + name: `myJob ${j}`, + data: { i: j }, + opts: { attempts: 2, backoff: 1000 }, + })) + ); + } +})(); +``` +{% endcode %} + +### Creating a consumer + +The consumer will be just a simple instance, we will use concurrency 10, so that jobs can be processed concurrently, and therefore create overlapping spans. We will also simulate jobs failures so that we can get retries, to show how spans are generated as the job gets failed, retried and finally completed: + +{% code title="consumer.ts" %} +```typescript +import { Worker } from "bullmq"; +import { BullMQOtel } from "bullmq-otel"; + +(async () => { + const worker = new Worker( + "myQueue", + async (job) => { + console.log("processing job", job.id, job.attemptsMade); + await new Promise(async (res) => { + setTimeout(() => res({}), 200); + }); + + if (job.attemptsMade < 1) { + throw new Error("This was an error"); + } + + return "my result value"; + }, + { + name: "myWorker", + connection: { + host: "127.0.0.1", + port: 6379, + }, + telemetry: new BullMQOtel("simple-guide"), + concurrency: 10, + } + ); +})(); +``` +{% endcode %} + +### Creating the instrumentation files + +To test the telemetry functionality we can run a simple example. For that we also need to instantiate the OpenTelemetry SDK using a so called OpenTelemetry Protocol (OTLP) exporter. + +We must install the following modules that are part of the OpenTelemetry SDK: + +``` +npm install @opentelemetry/exporter-trace-otlp-proto \ + @opentelemetry/exporter-metrics-otlp-proto +``` + +And now we must create so called "instrumentation" files. We will create one for our "producer" service, which is the service actually taking care of producing jobs, it will look like this. Note that we use localhost (127.0.0.1) where our jaeger service is running: + +{% code title="producer.inst.otlp.ts" %} +```typescript +import { NodeSDK } from '@opentelemetry/sdk-node'; +import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto'; +import { OTLPMetricExporter } from '@opentelemetry/exporter-metrics-otlp-proto'; +import { PeriodicExportingMetricReader } from '@opentelemetry/sdk-metrics'; + +const sdk = new NodeSDK({ + serviceName: 'producer', + traceExporter: new OTLPTraceExporter({ + url: 'http://127.0.0.1:4318/v1/traces' + }), + metricReader: new PeriodicExportingMetricReader({ + exporter: new OTLPMetricExporter({ + url: 'http://127.0.0.1:4318/v1/metrics' + }), + }), +}); + +sdk.start(); +``` +{% endcode %} + +Likewise we will create another instrumentation file for our "consumer" service, this is where the workers will run and consume the jobs produced by the "Queue" instance: + +{% code title="consumer.inst.otlp.ts" %} +```typescript +import { NodeSDK } from '@opentelemetry/sdk-node'; +import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto'; +import { OTLPMetricExporter } from '@opentelemetry/exporter-metrics-otlp-proto'; +import { PeriodicExportingMetricReader } from '@opentelemetry/sdk-metrics'; + +const sdk = new NodeSDK({ + serviceName: 'consumer', + traceExporter: new OTLPTraceExporter({ + url: 'http://127.0.0.1:4318/v1/traces' + }), + metricReader: new PeriodicExportingMetricReader({ + exporter: new OTLPMetricExporter({ + url: 'http://127.0.0.1:4318/v1/metrics' + }), + }), +}); + +sdk.start(); +``` +{% endcode %} + +Both services looks basically the same, just the service name will differ in this case. + +### Launching the services + +In order to guarantee that the OpenTelemetry instrumentation is run first, before everything else, and performs any required internal patching (even though BullMQ does not rely on patching other modules may do), we need to launch it like this (note that we use tsx in this example but Node runtime will do as well: + +``` +tsx --import producer.inst.otlp.ts producer.ts +tsx --import consumer.inst.otlp.ts consumer.ts +``` + +{% hint style="info" %} +You can also use Node runtime directly if you are using javascript (or building from Typescript to javascript): `node --import producer.inst.otlp.js producer.js` +{% endhint %} + +As the services are launched we will see that the consumers starts processing the jobs and produce some logs on the console: + +``` +> tsx --import consumer.inst.otlp.ts consumer.ts + +processing job 1 0 +processing job 2 0 +processing job 3 0 +processing job 4 0 +processing job 5 0 +processing job 6 0 +... +processing job 43 1 +processing job 44 1 +processing job 45 1 +processing job 46 1 +processing job 47 1 +processing job 48 1 +processing job 49 1 +processing job 50 1 +``` + +These are just the logs that we wrote ourselves on the "process" function in our worker, so nothing special here. However if we go to Jaeger we will find the following: + +
+ +We have now 2 services to choose from, consumer and producer. If we search for traces in the producer we will be able to see all the traces where the producer is involved: + +
+ +Here we can see as even though we are searching for the producer traces, we also get the consumer spans, and this is because jobs are linked between producers and consumers, so that we can trace all the way from the creation of a job to its final processing. + +If we look into the consumer spans for example, there are some interesting things to see: + +
+ +First of all, note how the producer span "addBulk myQueue", is the root of this trace. Since this was an addBulk, it means that several jobs were added to the queue in one go, 5 in this case. So the spans created by the consumer are therefore linked to this one producer span. The consumer spans "process myQueue" are generated for every job that is being processed, and since we had a concurrency factor larger than 5, all 5 jobs are processed concurrently, which we can see in the spans all starting at the same time. + +But we also forced the jobs to fail 1 time, so that they would be retried with a small backoff (delay), which is why we can see a "delay myQueue" span and then a final "process myQueue" span. + +If we open the spans we can find other useful information: + +
+ +We have some useful tags related to this particular job, and also logs that shows events that happened during the span lifetime, for instance here we can see that the job failed with the given error message. + +If we go to the last span of the trace we can see that the job was finally completed after being delayed a bit before its last retry: + +
diff --git a/docs/gitbook/guide/telemetry/running-jaeger.md b/docs/gitbook/guide/telemetry/running-jaeger.md new file mode 100644 index 0000000000..904af85abb --- /dev/null +++ b/docs/gitbook/guide/telemetry/running-jaeger.md @@ -0,0 +1,29 @@ +# Running Jaeger + +The easiest way to run Jaeger is by using Docker compose. If you have docker installed, it is a matter of running this docker-compose.yaml file: + +```yaml +services: + jaeger: + image: jaegertracing/all-in-one:latest + container_name: BullMQ_with_opentelemetry_jaeger + ports: + - '4318:4318' + - '16686:16686' + +``` + +Note that we need to expose 2 ports here, the first (4318) one is the endpoint to export our traces and the second one (16686) is our UI. + +You can now just run this service with: + +``` +docker-compose up +``` + +In a few seconds the image will be up and running. You can verify that is working by opening a browser window and pointing it to [http://localhost:16686](http://localhost:16686/search) + +As no traces has been created yet you will get a quite empty dashboard: + +
+