fix(llmobs): encode llm objects in utf-8 before sending

[jjxct]

This PR resolves an issue in the Python SDK where non-ascii/utf8 characters being annotated on spans resulted in span payloads being dropped due to encoding errors.

In #11330 we previously added the ensure_ascii=False option to our safe_json() helper's use of json.dumps(...) in order to keep non-ascii characters from being encoded multiple times into nonsense (as we were calling safe_json() multiple nested times while building the span event from the span tags. However this resulted in issues where non-latin1 characters (which is a subset of utf-8 and apparently the encoding scheme HTTP library relies on, which we in turn rely on to submit payloads) broke the encoding at payload submission time.

To fix this, we remove the ensure_ascii=False option at the final write time.

Also note that after #11543 we mostly centralized all of the times a span event is encoded, which is at write time and when encoding the span's input/output value fields (which can be a json dictionary format). Since we need to provide valid json formatting for the IO fields (which leads to a prettier UI display), we still need to call json.dumps(ensure_ascii=False) to avoid the same problem as fixed by #11330, i.e. keep the non-ascii characters unencoded until at the very end (i.e. write time)

This PR also adds minor test fixtures mocking out the LLMObs back end intake to make assertions on the payloads we should be submitting to LLMObs, since previous tests were all relying on the span events prior to encoding/submission and weren't able to cover this scenario.

Checklist

• PR author has checked that all the criteria below are met
• The PR description includes an overview of the change
• The PR description articulates the motivation for the change
• The change includes tests OR the PR description describes a testing strategy
• The PR description notes risks associated with the change, if any
• Newly-added code is easy to change
• The change follows the library release note guidelines
• The change includes or references documentation updates if necessary
• Backport labels are set (if applicable)

Reviewer Checklist

• Reviewer has checked that all the criteria below are met
• Title is accurate
• All changes are related to the pull request's stated goal
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Newly-added code is easy to change
• Release note makes sense to a user of the library
• If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

[github-actions]

CODEOWNERS have been resolved as:

releasenotes/notes/fix-llmobs-json-encoding-ascii-f8da77867a910de1.yaml  @DataDog/apm-python
ddtrace/llmobs/_llmobs.py                                               @DataDog/ml-observability
ddtrace/llmobs/_utils.py                                                @DataDog/ml-observability
ddtrace/llmobs/_writer.py                                               @DataDog/ml-observability
tests/contrib/langgraph/conftest.py                                     @DataDog/ml-observability
tests/llmobs/conftest.py                                                @DataDog/ml-observability
tests/llmobs/test_llmobs.py                                             @DataDog/ml-observability
tests/llmobs/test_llmobs_span_agent_writer.py                           @DataDog/ml-observability
tests/llmobs/test_llmobs_span_agentless_writer.py                       @DataDog/ml-observability

[pr-commenter]

Benchmarks

Benchmark execution time: 2025-01-21 23:08:04

Comparing candidate commit 09beaab in PR branch jonathan.chavez/encoding-llm with baseline commit 0003501 in branch main.

Found 1 performance improvements and 0 performance regressions! Performance is the same for 393 metrics, 2 unstable metrics.

scenario:iast_aspects-replace_aspect

• 🟩 execution_time [-673.386ns; -597.357ns] or [-9.227%; -8.185%]

[github-actions]

The backport to 2.18 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.18 2.18
# Navigate to the new working tree
cd .worktrees/backport-2.18
# Create a new branch
git switch --create backport-11961-to-2.18
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 e11a0a333ad959e0a13d3550bf1fe8cd030fb191
# Push it to GitHub
git push --set-upstream origin backport-11961-to-2.18
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.18

Then, create a pull request where the base branch is 2.18 and the compare/head branch is backport-11961-to-2.18.

[github-actions]

The backport to 2.19 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.19 2.19
# Navigate to the new working tree
cd .worktrees/backport-2.19
# Create a new branch
git switch --create backport-11961-to-2.19
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 e11a0a333ad959e0a13d3550bf1fe8cd030fb191
# Push it to GitHub
git push --set-upstream origin backport-11961-to-2.19
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.19

Then, create a pull request where the base branch is 2.19 and the compare/head branch is backport-11961-to-2.19.