Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ai-proxy): query params in override.endpoint not being sent to LLMs #11863

Merged

Conversation

shreemaan-abhishek
Copy link
Contributor

Description

Although the ai-proxy plugin officially supports only openai as an LLM provider, the plugin can be used to proxy requests to azure-openai service as mentioned in this blog: https://docs.api7.ai/apisix/how-to-guide/ai-gateway/proxy-azure-openai-requests, by entering the correct URL in override.endpoint attribute of ai-proxy.

However, the azure-openai deployment URL contains some query params that wasn't being carried into the request (HTTP request made by ai-proxy plugin) to the LLM endpoint. This PR ensures that query params in override.endpoint are appropriately used.

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. bug Something isn't working labels Dec 26, 2024
@shreemaan-abhishek shreemaan-abhishek merged commit 9502bb4 into apache:master Dec 30, 2024
30 checks passed
@shreemaan-abhishek shreemaan-abhishek deleted the ai-proxy-query-params branch December 30, 2024 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working size:M This PR changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants