chore(langchain): auto-instrument with langgraph

[sabrenner]

Adds gated auto-instrumentation for LangChain with LangGraph, along with a couple small fixes for LangGraph.

LangGraph bug fixes

1. We were not properly recording errors from LangGraph spans, and not setting LLMObs tags in the case of failures. We add our set_llmobs_tags in the appropriate spots to resolve this.
2. LangGraph will sometimes have triggers for a step be something like ["__pregel_push"] in the case when a Send is enqueued as a trigger, and will not give hints to the actual invoker. To resolve this, we check that if for any given queued task, there is only one finished task. If so, set that task as the trigger.

Complications with LangChain linking

The primary obstacle was that, unlike with LangGraph, where we had an intermediary function to patch between tasks executed by the graph, we don't have that for LangChain LCEL chains. Their steps are executed in a loop inside their invoke method, which blocks us from jumping in between the steps to make the links.

Additionally, LangChain elements that we trace are sometimes embedded in other Runnable types:

• RunnableBinding, which binds an instance inside of it
• RunnableAssign, which embeds an instance of a RunnableParallel
• RunnableParallel, which can run multiple Runnables (some of which we might trace) in parallel, which have the parallel items in a steps__ attribute

Something I tried to overcome this was to flatten/flatmap the items of the list of steps to extract these.

To overcome linking between steps, I recorded the instance of each traced Runnable, mapping its ID to its span, and vice versa, to be able to grab instances if needed. Additionally, for each Runnable item in the chain, I marked them as a chain step by adding them to a set of steps to later check against.

Setting links

Link setting is split into setting the input links ("to": "input") and output links ("to": "output")

Input Links

Input links are set by:

1. Identifying if the span represents a step in a chain. If not, set its input link as input --> input from the parent span.
2. If it does represent a step in the chain, find the index of the previous traced step in the chain (the chain instance is grabbed from the span to instance mapping referenced above), and setting it as the output --> input link. If the step contains multiple spans (ie from a RunnableParallel), add all of those spans as links with the same output --> input attribute

The index of the previously traced step in the chain (-1 if not found or is not a chain step) is returned for use in output linkage.

Output Links

Output links are set by:

1. If the span does not represent a step in a chain, or the parent span isn't a chain (ie has a steps attribute), set the output --> output link from the current span onto the parent span.
2. If the span does represent a step a chain, remove all previous span links on the span from the previous traced step, and set the new span link from the current span. We do this overwriting every time because we don't know ahead of time which step in the chain will be the last one we trace, so we have to remove previous span links if we find we need to add a new one.

Checklist

• PR author has checked that all the criteria below are met
• The PR description includes an overview of the change
• The PR description articulates the motivation for the change
• The change includes tests OR the PR description describes a testing strategy
• The PR description notes risks associated with the change, if any
• Newly-added code is easy to change
• The change follows the library release note guidelines
• The change includes or references documentation updates if necessary
• Backport labels are set (if applicable)

Reviewer Checklist

• Reviewer has checked that all the criteria below are met
• Title is accurate
• All changes are related to the pull request's stated goal
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Newly-added code is easy to change
• Release note makes sense to a user of the library
• If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

[github-actions]

CODEOWNERS have been resolved as:

ddtrace/contrib/internal/langchain/patch.py                             @DataDog/ml-observability
ddtrace/contrib/internal/langgraph/patch.py                             @DataDog/ml-observability
ddtrace/llmobs/_integrations/base.py                                    @DataDog/ml-observability
ddtrace/llmobs/_integrations/langchain.py                               @DataDog/ml-observability
ddtrace/llmobs/_integrations/langgraph.py                               @DataDog/ml-observability

[pr-commenter]

Benchmarks

Benchmark execution time: 2025-02-12 20:42:40

Comparing candidate commit fe66852 in PR branch sabrenner/langchain-span-linking with baseline commit 0fcafa2 in branch main.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 393 metrics, 2 unstable metrics.

scenario:iast_aspects-ospathdirname_aspect

• 🟥 execution_time [+537.639ns; +595.766ns] or [+14.586%; +16.163%]

[datadog-dd-trace-py-rkomorn]

Datadog Report

Branch report: sabrenner/langchain-span-linking
Commit report: 98aca2a
Test service: dd-trace-py

✅ 0 Failed, 130 Passed, 1378 Skipped, 4m 44.97s Total duration (35m 12.56s time saved)

[Yun-Kim]

Some initial thoughts

[emmettbutler]

Needs a release note with a feature and two fixes. The change itself looks fine.