fix(llmobs): fix content arg extraction for vertex ai integration #12034

ncybul · 2025-01-22T19:34:35Z

In MLOS-42 a customer was experiencing the following error:

ddtrace.internal.utils.ArgumentError: contents (at position 0)

where the content argument was not being extracted properly from the list of keyword arguments inputted into the chat.send_message method.

This is because in the Vertex AI integration, we look for the contents keyword argument. However, the content field is titled content in the send_message method and contents in the generate_content method, so it is necessary to differentiate between these two cases.

This PR is a small fix that corrects this error by differentiating between chat and completion requests in order to extract either content or contents respectively.

Testing

Automatic Testing

I edited some of the currently existing tests to use the keyword argument extraction rather than the positional argument extraction to get the content in order to confirm that this fix resolves the error.

Manual Testing

Running the following code reproduced the error; furthermore, I confirmed that with this fix, the error is no longer present and the request completes successfully.

llm = GenerativeModel("gemini-1.5-flash")
chat = llm.start_chat()
resp = chat.send_message(content="hello")

I also verified that the following code which uses the generate_content method is not impacted (continues to work as before) as a result of this fix.

llm = GenerativeModel("gemini-1.5-flash")
resp = llm.generate_content(contents="hello")

Checklist

PR author has checked that all the criteria below are met
The PR description includes an overview of the change
The PR description articulates the motivation for the change
The change includes tests OR the PR description describes a testing strategy
The PR description notes risks associated with the change, if any
Newly-added code is easy to change
The change follows the library release note guidelines
The change includes or references documentation updates if necessary
Backport labels are set (if applicable)

Reviewer Checklist

Reviewer has checked that all the criteria below are met
Title is accurate
All changes are related to the pull request's stated goal
Avoids breaking API changes
Testing strategy adequately addresses listed risks
Newly-added code is easy to change
Release note makes sense to a user of the library
If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
Backport labels are set in a manner that is consistent with the release branch maintenance policy

github-actions · 2025-01-22T19:35:09Z

CODEOWNERS have been resolved as:

releasenotes/notes/fix-vertexai-content-extraction-b216207bd8192e5f.yaml  @DataDog/apm-python
ddtrace/contrib/internal/vertexai/_utils.py                             @DataDog/ml-observability
ddtrace/contrib/internal/vertexai/patch.py                              @DataDog/ml-observability
ddtrace/llmobs/_integrations/vertexai.py                                @DataDog/ml-observability
tests/contrib/vertexai/test_vertexai.py                                 @DataDog/ml-observability
tests/contrib/vertexai/test_vertexai_llmobs.py                          @DataDog/ml-observability

datadog-dd-trace-py-rkomorn · 2025-01-22T19:49:24Z

Datadog Report

Branch report: nicole-cybul/fix-content-arg-extraction
Commit report: 4ebbf13
Test service: dd-trace-py

✅ 0 Failed, 130 Passed, 1378 Skipped, 4m 36.21s Total duration (35m 19.01s time saved)

pr-commenter · 2025-01-22T20:16:09Z

Benchmarks

Benchmark execution time: 2025-01-23 21:06:49

Comparing candidate commit ff2dfd7 in PR branch nicole-cybul/fix-content-arg-extraction with baseline commit 2f52abb in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 394 metrics, 2 unstable metrics.

Yun-Kim

Is keeping track of is_chat ultimately necessary/useful in this context?

ddtrace/contrib/internal/vertexai/_utils.py

datadog-dd-trace-py-rkomorn · 2025-01-23T16:32:29Z

Datadog Report

Branch report: nicole-cybul/fix-content-arg-extraction
Commit report: ff2dfd7
Test service: dd-trace-py

✅ 0 Failed, 130 Passed, 1378 Skipped, 4m 31.6s Total duration (35m 13.15s time saved)

ddtrace/llmobs/_integrations/vertexai.py

releasenotes/notes/fix-vertexai-content-extraction-b216207bd8192e5f.yaml

…92e5f.yaml Co-authored-by: Yun Kim <[email protected]>

…2034) In [MLOS-42](https://datadoghq.atlassian.net/browse/MLOS-42) a customer was experiencing the following error: ``` ddtrace.internal.utils.ArgumentError: contents (at position 0) ``` where the `content` argument was not being extracted properly from the list of keyword arguments inputted into the `chat.send_message` method. This is because in the Vertex AI integration, we look for the `contents` keyword argument. However, the content field is titled `content` in the [send_message](https://github.com/google-gemini/generative-ai-python/blob/main/google/generativeai/generative_models.py#L514) method and `contents` in the [generate_content](https://github.com/google-gemini/generative-ai-python/blob/main/google/generativeai/generative_models.py#L239) method, so it is necessary to differentiate between these two cases. This PR is a small fix that corrects this error by differentiating between chat and completion requests in order to extract either `content` or `contents` respectively. ## Testing ### Automatic Testing I edited some of the currently existing tests to use the keyword argument extraction rather than the positional argument extraction to get the content in order to confirm that this fix resolves the error. ### Manual Testing Running the following code reproduced the error; furthermore, I confirmed that with this fix, the error is no longer present and the request completes successfully. ``` llm = GenerativeModel("gemini-1.5-flash") chat = llm.start_chat() resp = chat.send_message(content="hello") ``` I also verified that the following code which uses the generate_content method is not impacted (continues to work as before) as a result of this fix. ``` llm = GenerativeModel("gemini-1.5-flash") resp = llm.generate_content(contents="hello") ``` ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) [MLOS-42]: https://datadoghq.atlassian.net/browse/MLOS-42?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: Yun Kim <[email protected]> (cherry picked from commit 67a5a9c)

github-actions · 2025-01-23T22:49:35Z

The backport to 2.19 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.19 2.19
# Navigate to the new working tree
cd .worktrees/backport-2.19
# Create a new branch
git switch --create backport-12034-to-2.19
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 67a5a9c66d7c0cfd114247ca277982df48752839
# Push it to GitHub
git push --set-upstream origin backport-12034-to-2.19
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.19

Then, create a pull request where the base branch is 2.19 and the compare/head branch is backport-12034-to-2.19.

…2034) In [MLOS-42](https://datadoghq.atlassian.net/browse/MLOS-42) a customer was experiencing the following error: ``` ddtrace.internal.utils.ArgumentError: contents (at position 0) ``` where the `content` argument was not being extracted properly from the list of keyword arguments inputted into the `chat.send_message` method. This is because in the Vertex AI integration, we look for the `contents` keyword argument. However, the content field is titled `content` in the [send_message](https://github.com/google-gemini/generative-ai-python/blob/main/google/generativeai/generative_models.py#L514) method and `contents` in the [generate_content](https://github.com/google-gemini/generative-ai-python/blob/main/google/generativeai/generative_models.py#L239) method, so it is necessary to differentiate between these two cases. This PR is a small fix that corrects this error by differentiating between chat and completion requests in order to extract either `content` or `contents` respectively. ## Testing ### Automatic Testing I edited some of the currently existing tests to use the keyword argument extraction rather than the positional argument extraction to get the content in order to confirm that this fix resolves the error. ### Manual Testing Running the following code reproduced the error; furthermore, I confirmed that with this fix, the error is no longer present and the request completes successfully. ``` llm = GenerativeModel("gemini-1.5-flash") chat = llm.start_chat() resp = chat.send_message(content="hello") ``` I also verified that the following code which uses the generate_content method is not impacted (continues to work as before) as a result of this fix. ``` llm = GenerativeModel("gemini-1.5-flash") resp = llm.generate_content(contents="hello") ``` ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) [MLOS-42]: https://datadoghq.atlassian.net/browse/MLOS-42?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: Yun Kim <[email protected]> (cherry picked from commit 67a5a9c)

…tion (#12034)" This reverts commit 67a5a9c.

…ckport #12034 to 2.19] (#12061) Backports #12034 to 2.19. In [MLOS-42](https://datadoghq.atlassian.net/browse/MLOS-42) a customer was experiencing the following error: ``` ddtrace.internal.utils.ArgumentError: contents (at position 0) ``` where the `content` argument was not being extracted properly from the list of keyword arguments inputted into the `chat.send_message` method. This is because in the Vertex AI integration, we look for the `contents` keyword argument. However, the content field is titled `content` in the [send_message](https://github.com/google-gemini/generative-ai-python/blob/main/google/generativeai/generative_models.py#L514) method and `contents` in the [generate_content](https://github.com/google-gemini/generative-ai-python/blob/main/google/generativeai/generative_models.py#L239) method, so it is necessary to differentiate between these two cases. This PR is a small fix that corrects this error by differentiating between chat and completion requests in order to extract either `content` or `contents` respectively. ## Testing ### Automatic Testing I edited some of the currently existing tests to use the keyword argument extraction rather than the positional argument extraction to get the content in order to confirm that this fix resolves the error. ### Manual Testing Running the following code reproduced the error; furthermore, I confirmed that with this fix, the error is no longer present and the request completes successfully. ``` llm = GenerativeModel("gemini-1.5-flash") chat = llm.start_chat() resp = chat.send_message(content="hello") ``` I also verified that the following code which uses the generate_content method is not impacted (continues to work as before) as a result of this fix. ``` llm = GenerativeModel("gemini-1.5-flash") resp = llm.generate_content(contents="hello") ``` ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) [MLOS-42]: https://datadoghq.atlassian.net/browse/MLOS-42?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: Yun Kim <[email protected]> (cherry picked from commit 67a5a9c) ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) [MLOS-42]: https://datadoghq.atlassian.net/browse/MLOS-42?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [MLOS-42]: https://datadoghq.atlassian.net/browse/MLOS-42?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

…2034) In [MLOS-42](https://datadoghq.atlassian.net/browse/MLOS-42) a customer was experiencing the following error: ``` ddtrace.internal.utils.ArgumentError: contents (at position 0) ``` where the `content` argument was not being extracted properly from the list of keyword arguments inputted into the `chat.send_message` method. This is because in the Vertex AI integration, we look for the `contents` keyword argument. However, the content field is titled `content` in the [send_message](https://github.com/google-gemini/generative-ai-python/blob/main/google/generativeai/generative_models.py#L514) method and `contents` in the [generate_content](https://github.com/google-gemini/generative-ai-python/blob/main/google/generativeai/generative_models.py#L239) method, so it is necessary to differentiate between these two cases. This PR is a small fix that corrects this error by differentiating between chat and completion requests in order to extract either `content` or `contents` respectively. ## Testing ### Automatic Testing I edited some of the currently existing tests to use the keyword argument extraction rather than the positional argument extraction to get the content in order to confirm that this fix resolves the error. ### Manual Testing Running the following code reproduced the error; furthermore, I confirmed that with this fix, the error is no longer present and the request completes successfully. ``` llm = GenerativeModel("gemini-1.5-flash") chat = llm.start_chat() resp = chat.send_message(content="hello") ``` I also verified that the following code which uses the generate_content method is not impacted (continues to work as before) as a result of this fix. ``` llm = GenerativeModel("gemini-1.5-flash") resp = llm.generate_content(contents="hello") ``` ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) [MLOS-42]: https://datadoghq.atlassian.net/browse/MLOS-42?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: Yun Kim <[email protected]> (cherry picked from commit 67a5a9c)

…ckport 2.18] (#12071) Backport 67a5a9c from #12034 to 2.18. In [MLOS-42](https://datadoghq.atlassian.net/browse/MLOS-42) a customer was experiencing the following error: ``` ddtrace.internal.utils.ArgumentError: contents (at position 0) ``` where the `content` argument was not being extracted properly from the list of keyword arguments inputted into the `chat.send_message` method. This is because in the Vertex AI integration, we look for the `contents` keyword argument. However, the content field is titled `content` in the [send_message](https://github.com/google-gemini/generative-ai-python/blob/main/google/generativeai/generative_models.py#L514) method and `contents` in the [generate_content](https://github.com/google-gemini/generative-ai-python/blob/main/google/generativeai/generative_models.py#L239) method, so it is necessary to differentiate between these two cases. This PR is a small fix that corrects this error by differentiating between chat and completion requests in order to extract either `content` or `contents` respectively. ## Testing ### Automatic Testing I edited some of the currently existing tests to use the keyword argument extraction rather than the positional argument extraction to get the content in order to confirm that this fix resolves the error. ### Manual Testing Running the following code reproduced the error; furthermore, I confirmed that with this fix, the error is no longer present and the request completes successfully. ``` llm = GenerativeModel("gemini-1.5-flash") chat = llm.start_chat() resp = chat.send_message(content="hello") ``` I also verified that the following code which uses the generate_content method is not impacted (continues to work as before) as a result of this fix. ``` llm = GenerativeModel("gemini-1.5-flash") resp = llm.generate_content(contents="hello") ``` ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) [MLOS-42]: https://datadoghq.atlassian.net/browse/MLOS-42?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Co-authored-by: ncybul <[email protected]>

…ckport 2.20] (#12072) Backport 67a5a9c from #12034 to 2.20. In [MLOS-42](https://datadoghq.atlassian.net/browse/MLOS-42) a customer was experiencing the following error: ``` ddtrace.internal.utils.ArgumentError: contents (at position 0) ``` where the `content` argument was not being extracted properly from the list of keyword arguments inputted into the `chat.send_message` method. This is because in the Vertex AI integration, we look for the `contents` keyword argument. However, the content field is titled `content` in the [send_message](https://github.com/google-gemini/generative-ai-python/blob/main/google/generativeai/generative_models.py#L514) method and `contents` in the [generate_content](https://github.com/google-gemini/generative-ai-python/blob/main/google/generativeai/generative_models.py#L239) method, so it is necessary to differentiate between these two cases. This PR is a small fix that corrects this error by differentiating between chat and completion requests in order to extract either `content` or `contents` respectively. ## Testing ### Automatic Testing I edited some of the currently existing tests to use the keyword argument extraction rather than the positional argument extraction to get the content in order to confirm that this fix resolves the error. ### Manual Testing Running the following code reproduced the error; furthermore, I confirmed that with this fix, the error is no longer present and the request completes successfully. ``` llm = GenerativeModel("gemini-1.5-flash") chat = llm.start_chat() resp = chat.send_message(content="hello") ``` I also verified that the following code which uses the generate_content method is not impacted (continues to work as before) as a result of this fix. ``` llm = GenerativeModel("gemini-1.5-flash") resp = llm.generate_content(contents="hello") ``` ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) [MLOS-42]: https://datadoghq.atlassian.net/browse/MLOS-42?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Co-authored-by: ncybul <[email protected]>

replace contents kw search with content

7841124

ncybul changed the title ~~replace contents kw search with content~~ fix(llmobs): replace contents kw search with content Jan 22, 2025

ncybul force-pushed the nicole-cybul/fix-content-arg-extraction branch from d124e97 to 7841124 Compare January 22, 2025 20:55

add tests; differentiate between chat and completion requests

6383258

ncybul changed the title ~~fix(llmobs): replace contents kw search with content~~ fix(llmobs): fix content arg extraction for vertex ai integration Jan 22, 2025

ncybul added 3 commits January 22, 2025 16:46

add release note

79583b0

add llmobs tests and pass is_chat to llmobs integration

ab03bcc

add more tests for streamed requests

4ebbf13

ncybul marked this pull request as ready for review January 22, 2025 22:20

ncybul requested review from a team as code owners January 22, 2025 22:20

ncybul requested review from avara1986 and sabrenner January 22, 2025 22:20

Yun-Kim reviewed Jan 22, 2025

View reviewed changes

ddtrace/contrib/internal/vertexai/_utils.py Outdated Show resolved Hide resolved

check both content and contents for tagging llmobs span

94d0c2f

Yun-Kim approved these changes Jan 23, 2025

View reviewed changes

ddtrace/llmobs/_integrations/vertexai.py Outdated Show resolved Hide resolved

releasenotes/notes/fix-vertexai-content-extraction-b216207bd8192e5f.yaml Outdated Show resolved Hide resolved

Kyle-Verhoog approved these changes Jan 23, 2025

View reviewed changes

ncybul and others added 2 commits January 23, 2025 14:31

Update releasenotes/notes/fix-vertexai-content-extraction-b216207bd81…

4f01415

…92e5f.yaml Co-authored-by: Yun Kim <[email protected]>

fix import list

ff2dfd7

ncybul added the backport 2.19 label Jan 23, 2025

ncybul merged commit 67a5a9c into main Jan 23, 2025
580 of 581 checks passed

ncybul deleted the nicole-cybul/fix-content-arg-extraction branch January 23, 2025 21:08

github-actions bot mentioned this pull request Jan 23, 2025

fix(llmobs): fix content arg extraction for vertex ai integration [backport 2.19] #12058

Closed

2 tasks

ncybul added backport 2.19 and removed backport 2.19 labels Jan 23, 2025

ncybul mentioned this pull request Jan 23, 2025

fix(llmobs): fix content arg extraction for vertex ai integration [backport #12034 to 2.19] #12061

Merged

4 tasks

wantsui added a commit that referenced this pull request Jan 23, 2025

Revert "fix(llmobs): fix content arg extraction for vertex ai integra…

183d5ed

…tion (#12034)" This reverts commit 67a5a9c.

Yun-Kim added backport 2.18 backport 2.20 labels Jan 24, 2025

This was referenced Jan 24, 2025

fix(llmobs): fix content arg extraction for vertex ai integration [backport 2.18] #12071

Merged

fix(llmobs): fix content arg extraction for vertex ai integration [backport 2.20] #12072

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(llmobs): fix content arg extraction for vertex ai integration #12034

fix(llmobs): fix content arg extraction for vertex ai integration #12034

ncybul commented Jan 22, 2025 •

edited

Loading

github-actions bot commented Jan 22, 2025 •

edited

Loading

datadog-dd-trace-py-rkomorn bot commented Jan 22, 2025 •

edited

Loading

pr-commenter bot commented Jan 22, 2025 •

edited

Loading

Yun-Kim left a comment

datadog-dd-trace-py-rkomorn bot commented Jan 23, 2025 •

edited

Loading

github-actions bot commented Jan 23, 2025

fix(llmobs): fix content arg extraction for vertex ai integration #12034

fix(llmobs): fix content arg extraction for vertex ai integration #12034

Conversation

ncybul commented Jan 22, 2025 • edited Loading

Testing

Automatic Testing

Manual Testing

Checklist

Reviewer Checklist

github-actions bot commented Jan 22, 2025 • edited Loading

datadog-dd-trace-py-rkomorn bot commented Jan 22, 2025 • edited Loading

Datadog Report

pr-commenter bot commented Jan 22, 2025 • edited Loading

Benchmarks

Yun-Kim left a comment

Choose a reason for hiding this comment

datadog-dd-trace-py-rkomorn bot commented Jan 23, 2025 • edited Loading

Datadog Report

github-actions bot commented Jan 23, 2025

ncybul commented Jan 22, 2025 •

edited

Loading

github-actions bot commented Jan 22, 2025 •

edited

Loading

datadog-dd-trace-py-rkomorn bot commented Jan 22, 2025 •

edited

Loading

pr-commenter bot commented Jan 22, 2025 •

edited

Loading

datadog-dd-trace-py-rkomorn bot commented Jan 23, 2025 •

edited

Loading