[Bug]: Azure OpenAI Service

[alancamillo]

Contact Details

[email protected]

What happened?

This is not exactly a bug, it's more of an improvement suggestion that I have already implemented for a need I had. However, I may have misunderstood the code and what I did may have been completely unnecessary. Here it goes:

I need the project to work exclusively via Azure endpoint. To do so, I filled in the session variables in the .env file. However, I noticed that the models used in the combo box of the interface follow the OpenAI session variable, even if they are not being filled in:

##########################
# OpenAI Endpoint: 
##########################

# Access key from OpenAI platform. 
# Leave it blank to disable this feature.
# Set to "user_provided" to allow the user to provide their API key from the UI.
OPENAI_KEY=

# Identify the available models, separated by commas *without spaces*. 
# The first will be default. 
# Leave it blank to use internal settings. 
OPENAI_MODELS=gpt-3.5-turbo-0301,gpt-4

Therefore, I ended up creating a logic with the AZURE_OPENAI_API_DEPLOYMENT_NAME variable so that, in the same way as with the OPENAI_MODELS variable, they are separated by commas and correspond in position to the OpenAI models. Example:

##########################
# AZURE Endpoint:   
##########################

# To use Azure with this project, set the following variables. These will be used to build the API URL.
# Chat completion: 
#     `https://{AZURE_OPENAI_API_INSTANCE_NAME}.openai.azure.com/openai/deployments/{AZURE_OPENAI_API_DEPLOYMENT_NAME}/chat/completions?api-version={AZURE_OPENAI_API_VERSION}`;
# You should also consider changing the `OPENAI_MODELS` variable above to the models available in your instance/deployment.
# Note: I've noticed that the Azure API is much faster than the OpenAI API, so the streaming looks almost instantaneous.

AZURE_OPENAI_API_KEY=xxxxxxxxx
AZURE_OPENAI_API_INSTANCE_NAME=gpt_xxx
AZURE_OPENAI_API_DEPLOYMENT_NAME=xxx-gpt-3_5-turbo-0301;xxx-gpt-4
AZURE_OPENAI_API_VERSION=2023-05-15

There is still a technical debt in that the names of the deployments in Azure must contain a reference to the name of the original OpenAI model for matching purposes.

But... as I mentioned, I may not have understood how to use the application correctly, and I may have done all this for nothing. What do you think?

Thank you!

What browsers are you seeing the problem on?

No response

Relevant log output

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

I'm not sure I completely follow. Does the deployment name determine which models are used?

[alancamillo]

Let me provide an example. Given the model names of the endpoints:

# OpenAI Endpoint:
OPENAI_MODELS=gpt-3.5-turbo-0301,gpt-4
# AZURE Endpoint: 
AZURE_OPENAI_API_DEPLOYMENT_NAME=xxx-gpt-3_5-turbo-0301;xxx-gpt-4

The OpenAI model "gpt-3.5-turbo-0301" will match the Azure model "xxx-gpt-3_5-turbo-0301" because it contains the name of the OpenAI model within the Azure deployment name.

[danny-avila]

From what I understand, deployment names will vary from organization. It might be convention to name after the model being used. It would help if you submit a PR to see what your logic is doing exactly, I've been given azure access for testing things like this but I only have one deployment name/model

[alancamillo]

Another possibility is to have another list of models in the Azure session that could be used, both for the chat menu and for invoking the Azure endpoint. I will explore these two possibilities and submit a pull request to the project.