Configurations for DeepSeek with Azure AI Foundry

[gitanna]

I get 401 status code error (no body) for DeepSeek model that is set up on Azure with Azure AI Foundry.
Here is my librechat.yaml:

endpoints:
  azureOpenAI:
    # Endpoint-level configuration
    titleModel: current_model
    plugins: false
    assistants: false
    groups:
    # Group-level configuration
    - group: "GROUP_NAME"
      apiKey: "API_KEY_1"
      instanceName: "INSTANCE_NAME"
      deploymentName: gpt-4o
      version: "2024-08-01-preview"
      # Model-level configuration
      models:
        gpt-4o:
          deploymentName: gpt-4o
  custom:
    - name: "AZUREFOUNDRY"
      apiKey: "API_KEY_2"
      baseURL:  "https://muram-m6sm2ac5-eastus2.services.ai.azure.com/models"
      models:
        default: ["deepseek-chat", "deepseek-coder", "deepseek-reasoner"]
        fetch: false
      titleConvo: true
      titleModel: "deepseek-chat"
      modelDisplayLabel: "Deepseek"
      

I made a change that has not changed the behavior and error, but looks better:

custom:
  - name: "AZUREFOUNDRY"
    apiKey: "API_KEY_2"
    baseURL:  "https://muram-m6sm2ac5-eastus2.services.ai.azure.com/models/"
    models:
      default : ["DeepSeek-R1"]
      fetch: false
    titleConvo: true
    titleModel: "DeepSeek-R1"
    modelDisplayLabel: "Deepseek"

I also tried serverless option, but it did not work at all. Not sure how serverless configurations should go together with other Azure OpenAI endpoints.

[hamletfg]

I'm also looking for a working example of using the Azure DeepSeek R1 model on Librechat, let me know if you firgure it out.

[gitanna]

I think the issue is in this parameter:

models:
      default : ["DeepSeek-R1"]

I tried different variations, but got the same error each time. If anyone knows the correct way, please post it here.

[danny-avila]

This works as a custom endpoint, I know it can be done with Azure Serverless config, too, but don't have that handy:

endpoints:
# other configs...
  custom:
# custom endpoints...
    - name: "Azure (DeepSeek)"
      apiKey: "${AZURE_DEEPSEEK_API_KEY}"
      baseURL: "https://DeepSeek-R1-YOUR-ENDPOINT-FROM-AZURE.models.ai.azure.com/v1/"
      models:
        default: [
          "DeepSeek-R1",
          ]
        fetch: false
      titleConvo: true
      titleModel: current_model
      dropParams: ["stop", "user", "frequency_penalty", "presence_penalty"]
      modelDisplayLabel: "DeepSeek-R1"

needs AZURE_DEEPSEEK_API_KEY defined in .env file, you can rename the variable if you like.

[danny-avila]

[gitanna]

The serverless solution works, but the custom endpoint above gives me the same error: a 401 status code (no body). I have everything the same as in your solution, except for the baseURL: 'https://****ai.azure.com/models/' (not /v1/). I'm not sure what needs to be done to make it work. The custom endpoint shows DeepSeek as a separate item in the menu, which is what we would like to see.

[danny-avila]

Make sure you have AZURE_DEEPSEEK_API_KEY (or whatever you name this variable as) defined in your .env, both solutions work for me.

[gitanna]

I have AZURE_DEEPSEEK_API_KEY in .env:
AZURE_DEEPSEEK_API_KEY=xxxxx

[gitanna]

Thank you very much, Danny! I just tested serverless configurations, and they worked too!
In case if anyone needs it:

    - group: "AIFOUNDRY" 
      apiKey: "API_KEY"
      baseURL:  "https://****ai.azure.com/models/"
      serverless: true
      models:
        DeepSeek-R1: true

[hamletfg]

@gitanna thanks for the assistance of the config and to Danny as well. I do have a question, @gitanna I used your config for my librechat.yaml file. First of all I want to make sure that I am using the .yaml file correctly, I created a copy of the librechat.example.yaml file and named it librechat.yaml, don't know if that is correct. Then I followed your config in there but added my api key and baseURL.

Next I then went to the .env file and updated the Azure Assistants API section with the following:

#==========================#
#   Azure Assistants API   #
#==========================#

AZURE_DEEPSEEK_API_KEY="API_KEY"
AZURE_MODELS=DeepSeek-R1

I'm not quite sure where to move from here, where will it appear on the frontend of LibreChat?

[jmartinezb3]

Hello :) can you please share a full example of your DeepSeek R1 AI Foundry working configuration? i'm trying to make it work too this way.

[hamletfg]

@jmartinezb3 Don't know if this helps but I did find a guide on configuring the .yaml file and having docker use it: https://www.librechat.ai/docs/configuration/docker_override#examples

[Passerby1011]

Hello :) can you please share a full example of your DeepSeek R1 AI Foundry working configuration? i'm trying to make it work too this way.

In this way, the Azure AI service version of Deepseek-R1 can engage in normal conversation.This can then be configured together with other models under the Azure OpenAI endpoint

    - group: "Azure AI Foundry"
      apiKey: "Fill in as you wish"
      serverless: true
      baseURL: "https://xxxxxxxxxxxxxxx.services.ai.azure.com/models"
      additionalHeaders:
        api-key: [api-key]
      models:
        DeepSeek-R1:
          deploymentName: "DeepSeek-R1"

[rubentalstra]

here is a serverless working endpoint from Azure AI Foundry

      - group: "DeepSeek-R1"
        apiKey: "${AZURE_DEEPSEEK_API_KEY}"
        baseURL: "https://xxxxxxxxxxxxxxx.services.ai.azure.com/models/"
        version: "2024-05-01-preview"
        serverless: true
        models:
          DeepSeek-R1: true