Does not work with fireworks.ai api

[dillfrescott]

Contact Details

[email protected]

What happened?

It throws the error:

chat-meilisearch  | 
chat-meilisearch  | 888b     d888          d8b 888 d8b                                            888
chat-meilisearch  | 8888b   d8888          Y8P 888 Y8P                                            888
chat-meilisearch  | 88888b.d88888              888                                                888
chat-meilisearch  | 888Y88888P888  .d88b.  888 888 888 .d8888b   .d88b.   8888b.  888d888 .d8888b 88888b.
chat-meilisearch  | 888 Y888P 888 d8P  Y8b 888 888 888 88K      d8P  Y8b     "88b 888P"  d88P"    888 "88b
chat-meilisearch  | 888  Y8P  888 88888888 888 888 888 "Y8888b. 88888888 .d888888 888    888      888  888
chat-meilisearch  | 888   "   888 Y8b.     888 888 888      X88 Y8b.     888  888 888    Y88b.    888  888
chat-meilisearch  | 888       888  "Y8888  888 888 888  88888P'  "Y8888  "Y888888 888     "Y8888P 888  888
chat-meilisearch  | 
chat-meilisearch  | Config file path:   "none"
chat-meilisearch  | Database path:              "./data.ms"
chat-meilisearch  | Server listening on:        "http://meilisearch:7700"
chat-meilisearch  | Environment:                "development"
chat-meilisearch  | Commit SHA:         "b11f85a635d41ff69a59d6b2068fd4b77fe04959"
chat-meilisearch  | Commit date:                "2023-11-13T10:36:01+00:00"
chat-meilisearch  | Package version:    "1.5.0"
chat-meilisearch  | Anonymous telemetry:        "Disabled"
chat-meilisearch  | 
chat-meilisearch  | A master key has been set. Requests to Meilisearch won't be authorized unless you provide an authentication key.
chat-meilisearch  | 
chat-meilisearch  | Check out Meilisearch Cloud!        https://www.meilisearch.com/cloud?utm_campaign=oss&utm_source=engine&utm_medium=cli
chat-meilisearch  | Documentation:                      https://www.meilisearch.com/docs
chat-meilisearch  | Source code:                        https://github.com/meilisearch/meilisearch
chat-meilisearch  | Discord:                    https://discord.meilisearch.com
chat-meilisearch  | 
chat-meilisearch  | [2023-12-13T00:25:12Z INFO  actix_server::builder] starting 4 workers
chat-meilisearch  | [2023-12-13T00:25:12Z INFO  actix_server::server] Actix runtime found; starting in Actix runtime
LibreChat         | 
LibreChat         | > [email protected] backend
LibreChat         | > cross-env NODE_ENV=production node api/server/index.js
LibreChat         | 
chat-meilisearch  | [2023-12-13T00:25:15Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes HTTP/1.1" 202 123 "-" "undici" 0.063740
chat-meilisearch  | [2023-12-13T00:25:15Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes HTTP/1.1" 202 125 "-" "undici" 0.141151
LibreChat         | Connected to MongoDB
chat-meilisearch  | [2023-12-13T00:25:15Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /health HTTP/1.1" 200 22 "-" "undici" 0.000134
chat-meilisearch  | [2023-12-13T00:25:16Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /indexes/messages/stats HTTP/1.1" 200 117 "-" "undici" 0.000241
chat-meilisearch  | [2023-12-13T00:25:16Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /indexes/convos/stats HTTP/1.1" 200 93 "-" "undici" 0.000180
LibreChat         | There are 2 messages in the database, 2 indexed
LibreChat         | There are 1 convos in the database, 1 indexed
LibreChat         | Server listening on all interfaces at port 3080. Use http://localhost:3080 to access it
chat-meilisearch  | [2023-12-13T00:25:37Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /health HTTP/1.1" 200 22 "-" "undici" 0.000172
LibreChat         | ask log
LibreChat         | {
LibreChat         |   text: 'hello',
LibreChat         |   conversationId: null,
LibreChat         |   endpointOption: {
LibreChat         |     endpoint: 'openAI',
LibreChat         |     chatGptLabel: null,
LibreChat         |     promptPrefix: null,
LibreChat         |     modelOptions: {
LibreChat         |       model: 'accounts/fireworks/models/mixtral-8x7b-fw-chat',
LibreChat         |       temperature: 1,
LibreChat         |       top_p: 1,
LibreChat         |       presence_penalty: 0,
LibreChat         |       frequency_penalty: 0
LibreChat         |     }
LibreChat         |   }
LibreChat         | }
LibreChat         | maxContextTokens 4095
LibreChat         | Loading history for conversation 6e94f051-0bc0-4a40-98af-5538ca9b7c3b 00000000-0000-0000-0000-000000000000
LibreChat         | remainingContextTokens, this.maxContextTokens (1/2) 4087 4095
LibreChat         | remainingContextTokens, this.maxContextTokens (2/2) 4087 4095
LibreChat         | <-------------------------PAYLOAD/TOKEN COUNT MAP------------------------->
LibreChat         | Payload: [ { role: 'user', content: 'hello' } ]
LibreChat         | Token Count Map: { 'd8504790-d718-49e8-a3f0-9c25aa9618b3': 5 }
LibreChat         | Prompt Tokens 8 remainingContextTokens 4087 this.maxContextTokens 4095
LibreChat         | { 'd8504790-d718-49e8-a3f0-9c25aa9618b3': 5, instructions: undefined }
LibreChat         | userMessage.tokenCount 5
LibreChat         | userMessage {
LibreChat         |   messageId: 'd8504790-d718-49e8-a3f0-9c25aa9618b3',
LibreChat         |   parentMessageId: '00000000-0000-0000-0000-000000000000',
LibreChat         |   conversationId: '6e94f051-0bc0-4a40-98af-5538ca9b7c3b',
LibreChat         |   sender: 'User',
LibreChat         |   text: 'hello',
LibreChat         |   isCreatedByUser: true,
LibreChat         |   tokenCount: 5
LibreChat         | }
chat-meilisearch  | [2023-12-13T00:25:41Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /indexes/convos/documents/6e94f051-0bc0-4a40-98af-5538ca9b7c3b HTTP/1.1" 404 189 "-" "undici" 0.000370
LibreChat         | [Meilisearch] Convo not found and will index 6e94f051-0bc0-4a40-98af-5538ca9b7c3b
LibreChat         | baseURL https://api.fireworks.ai/inference/v1
LibreChat         | modelOptions {
LibreChat         |   model: 'accounts/fireworks/models/mixtral-8x7b-fw-chat',
LibreChat         |   temperature: 1,
LibreChat         |   top_p: 1,
LibreChat         |   presence_penalty: 0,
LibreChat         |   frequency_penalty: 0,
LibreChat         |   stop: [ '||>', '\nUser:', '<|diff_marker|>' ],
LibreChat         |   user: '6578f39b41e77c180b2da0f5',
LibreChat         |   stream: true,
LibreChat         |   messages: [ { role: 'user', content: 'hello' } ]
LibreChat         | }
chat-meilisearch  | [2023-12-13T00:25:41Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes/messages/documents HTTP/1.1" 202 136 "-" "undici" 0.294749
chat-meilisearch  | [2023-12-13T00:25:41Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes/convos/documents HTTP/1.1" 202 134 "-" "undici" 0.239620
LibreChat         | [OpenAIClient.chatCompletion][stream] API Error
LibreChat         | [OpenAIClient.chatCompletion][finalChatCompletion] API Error
LibreChat         | [OpenAIClient.chatCompletion] Unhandled error type
LibreChat         | Error: Error: 400 [{'loc': ('body', 'user'), 'msg': 'extra fields not permitted', 'type': 'value_error.extra'}]
LibreChat         |     at OpenAIClient.chatCompletion (/app/api/app/clients/OpenAIClient.js:826:15)
LibreChat         |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
LibreChat         |     at async OpenAIClient.sendCompletion (/app/api/app/clients/OpenAIClient.js:442:15)
LibreChat         |     at async OpenAIClient.sendMessage (/app/api/app/clients/BaseClient.js:453:24)
LibreChat         |     at async /app/api/server/routes/ask/openAI.js:112:20
LibreChat         | Error: Error: 400 [{'loc': ('body', 'user'), 'msg': 'extra fields not permitted', 'type': 'value_error.extra'}]
LibreChat         |     at OpenAIClient.chatCompletion (/app/api/app/clients/OpenAIClient.js:826:15)
LibreChat         |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
LibreChat         |     at async OpenAIClient.sendCompletion (/app/api/app/clients/OpenAIClient.js:442:15)
LibreChat         |     at async OpenAIClient.sendMessage (/app/api/app/clients/BaseClient.js:453:24)
LibreChat         |     at async /app/api/server/routes/ask/openAI.js:112:20
chat-meilisearch  | [2023-12-13T00:25:41Z INFO  index_scheduler::batch] document addition done: DocumentAdditionResult { indexed_documents: 1, number_of_documents: 3 }
chat-meilisearch  | [2023-12-13T00:25:41Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes/messages/documents HTTP/1.1" 202 136 "-" "undici" 0.164542
chat-meilisearch  | [2023-12-13T00:25:41Z INFO  index_scheduler] A batch of tasks was successfully completed.
chat-meilisearch  | [2023-12-13T00:25:42Z INFO  index_scheduler::batch] document addition done: DocumentAdditionResult { indexed_documents: 1, number_of_documents: 2 }
chat-meilisearch  | [2023-12-13T00:25:42Z INFO  index_scheduler] A batch of tasks was successfully completed.
chat-meilisearch  | [2023-12-13T00:25:42Z INFO  index_scheduler::batch] document addition done: DocumentAdditionResult { indexed_documents: 1, number_of_documents: 4 }
chat-meilisearch  | [2023-12-13T00:25:42Z INFO  index_scheduler] A batch of tasks was successfully completed.

Steps to Reproduce

1. Try to connect to fireworks.ai api
2. Fails

What browsers are you seeing the problem on?

No response

Relevant log output

chat-meilisearch  | 
chat-meilisearch  | 888b     d888          d8b 888 d8b                                            888
chat-meilisearch  | 8888b   d8888          Y8P 888 Y8P                                            888
chat-meilisearch  | 88888b.d88888              888                                                888
chat-meilisearch  | 888Y88888P888  .d88b.  888 888 888 .d8888b   .d88b.   8888b.  888d888 .d8888b 88888b.
chat-meilisearch  | 888 Y888P 888 d8P  Y8b 888 888 888 88K      d8P  Y8b     "88b 888P"  d88P"    888 "88b
chat-meilisearch  | 888  Y8P  888 88888888 888 888 888 "Y8888b. 88888888 .d888888 888    888      888  888
chat-meilisearch  | 888   "   888 Y8b.     888 888 888      X88 Y8b.     888  888 888    Y88b.    888  888
chat-meilisearch  | 888       888  "Y8888  888 888 888  88888P'  "Y8888  "Y888888 888     "Y8888P 888  888
chat-meilisearch  | 
chat-meilisearch  | Config file path:   "none"
chat-meilisearch  | Database path:              "./data.ms"
chat-meilisearch  | Server listening on:        "http://meilisearch:7700"
chat-meilisearch  | Environment:                "development"
chat-meilisearch  | Commit SHA:         "b11f85a635d41ff69a59d6b2068fd4b77fe04959"
chat-meilisearch  | Commit date:                "2023-11-13T10:36:01+00:00"
chat-meilisearch  | Package version:    "1.5.0"
chat-meilisearch  | Anonymous telemetry:        "Disabled"
chat-meilisearch  | 
chat-meilisearch  | A master key has been set. Requests to Meilisearch won't be authorized unless you provide an authentication key.
chat-meilisearch  | 
chat-meilisearch  | Check out Meilisearch Cloud!        https://www.meilisearch.com/cloud?utm_campaign=oss&utm_source=engine&utm_medium=cli
chat-meilisearch  | Documentation:                      https://www.meilisearch.com/docs
chat-meilisearch  | Source code:                        https://github.com/meilisearch/meilisearch
chat-meilisearch  | Discord:                    https://discord.meilisearch.com
chat-meilisearch  | 
chat-meilisearch  | [2023-12-13T00:25:12Z INFO  actix_server::builder] starting 4 workers
chat-meilisearch  | [2023-12-13T00:25:12Z INFO  actix_server::server] Actix runtime found; starting in Actix runtime
LibreChat         | 
LibreChat         | > [email protected] backend
LibreChat         | > cross-env NODE_ENV=production node api/server/index.js
LibreChat         | 
chat-meilisearch  | [2023-12-13T00:25:15Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes HTTP/1.1" 202 123 "-" "undici" 0.063740
chat-meilisearch  | [2023-12-13T00:25:15Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes HTTP/1.1" 202 125 "-" "undici" 0.141151
LibreChat         | Connected to MongoDB
chat-meilisearch  | [2023-12-13T00:25:15Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /health HTTP/1.1" 200 22 "-" "undici" 0.000134
chat-meilisearch  | [2023-12-13T00:25:16Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /indexes/messages/stats HTTP/1.1" 200 117 "-" "undici" 0.000241
chat-meilisearch  | [2023-12-13T00:25:16Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /indexes/convos/stats HTTP/1.1" 200 93 "-" "undici" 0.000180
LibreChat         | There are 2 messages in the database, 2 indexed
LibreChat         | There are 1 convos in the database, 1 indexed
LibreChat         | Server listening on all interfaces at port 3080. Use http://localhost:3080 to access it
chat-meilisearch  | [2023-12-13T00:25:37Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /health HTTP/1.1" 200 22 "-" "undici" 0.000172
LibreChat         | ask log
LibreChat         | {
LibreChat         |   text: 'hello',
LibreChat         |   conversationId: null,
LibreChat         |   endpointOption: {
LibreChat         |     endpoint: 'openAI',
LibreChat         |     chatGptLabel: null,
LibreChat         |     promptPrefix: null,
LibreChat         |     modelOptions: {
LibreChat         |       model: 'accounts/fireworks/models/mixtral-8x7b-fw-chat',
LibreChat         |       temperature: 1,
LibreChat         |       top_p: 1,
LibreChat         |       presence_penalty: 0,
LibreChat         |       frequency_penalty: 0
LibreChat         |     }
LibreChat         |   }
LibreChat         | }
LibreChat         | maxContextTokens 4095
LibreChat         | Loading history for conversation 6e94f051-0bc0-4a40-98af-5538ca9b7c3b 00000000-0000-0000-0000-000000000000
LibreChat         | remainingContextTokens, this.maxContextTokens (1/2) 4087 4095
LibreChat         | remainingContextTokens, this.maxContextTokens (2/2) 4087 4095
LibreChat         | <-------------------------PAYLOAD/TOKEN COUNT MAP------------------------->
LibreChat         | Payload: [ { role: 'user', content: 'hello' } ]
LibreChat         | Token Count Map: { 'd8504790-d718-49e8-a3f0-9c25aa9618b3': 5 }
LibreChat         | Prompt Tokens 8 remainingContextTokens 4087 this.maxContextTokens 4095
LibreChat         | { 'd8504790-d718-49e8-a3f0-9c25aa9618b3': 5, instructions: undefined }
LibreChat         | userMessage.tokenCount 5
LibreChat         | userMessage {
LibreChat         |   messageId: 'd8504790-d718-49e8-a3f0-9c25aa9618b3',
LibreChat         |   parentMessageId: '00000000-0000-0000-0000-000000000000',
LibreChat         |   conversationId: '6e94f051-0bc0-4a40-98af-5538ca9b7c3b',
LibreChat         |   sender: 'User',
LibreChat         |   text: 'hello',
LibreChat         |   isCreatedByUser: true,
LibreChat         |   tokenCount: 5
LibreChat         | }
chat-meilisearch  | [2023-12-13T00:25:41Z INFO  actix_web::middleware::logger] 192.168.96.3 "GET /indexes/convos/documents/6e94f051-0bc0-4a40-98af-5538ca9b7c3b HTTP/1.1" 404 189 "-" "undici" 0.000370
LibreChat         | [Meilisearch] Convo not found and will index 6e94f051-0bc0-4a40-98af-5538ca9b7c3b
LibreChat         | baseURL https://api.fireworks.ai/inference/v1
LibreChat         | modelOptions {
LibreChat         |   model: 'accounts/fireworks/models/mixtral-8x7b-fw-chat',
LibreChat         |   temperature: 1,
LibreChat         |   top_p: 1,
LibreChat         |   presence_penalty: 0,
LibreChat         |   frequency_penalty: 0,
LibreChat         |   stop: [ '||>', '\nUser:', '<|diff_marker|>' ],
LibreChat         |   user: '6578f39b41e77c180b2da0f5',
LibreChat         |   stream: true,
LibreChat         |   messages: [ { role: 'user', content: 'hello' } ]
LibreChat         | }
chat-meilisearch  | [2023-12-13T00:25:41Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes/messages/documents HTTP/1.1" 202 136 "-" "undici" 0.294749
chat-meilisearch  | [2023-12-13T00:25:41Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes/convos/documents HTTP/1.1" 202 134 "-" "undici" 0.239620
LibreChat         | [OpenAIClient.chatCompletion][stream] API Error
LibreChat         | [OpenAIClient.chatCompletion][finalChatCompletion] API Error
LibreChat         | [OpenAIClient.chatCompletion] Unhandled error type
LibreChat         | Error: Error: 400 [{'loc': ('body', 'user'), 'msg': 'extra fields not permitted', 'type': 'value_error.extra'}]
LibreChat         |     at OpenAIClient.chatCompletion (/app/api/app/clients/OpenAIClient.js:826:15)
LibreChat         |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
LibreChat         |     at async OpenAIClient.sendCompletion (/app/api/app/clients/OpenAIClient.js:442:15)
LibreChat         |     at async OpenAIClient.sendMessage (/app/api/app/clients/BaseClient.js:453:24)
LibreChat         |     at async /app/api/server/routes/ask/openAI.js:112:20
LibreChat         | Error: Error: 400 [{'loc': ('body', 'user'), 'msg': 'extra fields not permitted', 'type': 'value_error.extra'}]
LibreChat         |     at OpenAIClient.chatCompletion (/app/api/app/clients/OpenAIClient.js:826:15)
LibreChat         |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
LibreChat         |     at async OpenAIClient.sendCompletion (/app/api/app/clients/OpenAIClient.js:442:15)
LibreChat         |     at async OpenAIClient.sendMessage (/app/api/app/clients/BaseClient.js:453:24)
LibreChat         |     at async /app/api/server/routes/ask/openAI.js:112:20
chat-meilisearch  | [2023-12-13T00:25:41Z INFO  index_scheduler::batch] document addition done: DocumentAdditionResult { indexed_documents: 1, number_of_documents: 3 }
chat-meilisearch  | [2023-12-13T00:25:41Z INFO  actix_web::middleware::logger] 192.168.96.3 "POST /indexes/messages/documents HTTP/1.1" 202 136 "-" "undici" 0.164542
chat-meilisearch  | [2023-12-13T00:25:41Z INFO  index_scheduler] A batch of tasks was successfully completed.
chat-meilisearch  | [2023-12-13T00:25:42Z INFO  index_scheduler::batch] document addition done: DocumentAdditionResult { indexed_documents: 1, number_of_documents: 2 }
chat-meilisearch  | [2023-12-13T00:25:42Z INFO  index_scheduler] A batch of tasks was successfully completed.
chat-meilisearch  | [2023-12-13T00:25:42Z INFO  index_scheduler::batch] document addition done: DocumentAdditionResult { indexed_documents: 1, number_of_documents: 4 }
chat-meilisearch  | [2023-12-13T00:25:42Z INFO  index_scheduler] A batch of tasks was successfully completed.

### Screenshots

_No response_

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct

[fuegovic]

I just tested and it's working fine for me

Make sure you have the proper config in the .env file:

OPENAI_API_KEY=t0BGpFZixxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxZw4dxkbx # <-Your fireworks API key goes here 
OPENAI_REVERSE_PROXY=https://api.fireworks.ai/inference/v1/completions

and run docker-compose up -d to apply the change made in the .env configuration

[dillfrescott]

Yup. I immediately tried to change the model name so it never even originally worked for me.

Odd glitch

[fuegovic]

@dillfrescott If what you really wanted is access to mixtral-8x7b-instruct, it looks like it's just been added to OpenRouter (which is working without issue in LibreChat)

[dillfrescott]

Oooo cool! Thank you!

[fuegovic]

And I was also able to test https://app.endpoints.anyscale.com/ directly with success yesterday

#1344 (comment)

[dillfrescott]

Interesting

[danny-avila]

Moved to troubleshooting because there is no guarantee to work with reverse proxies other than OpenRouter.

But looks like @fuegovic has access and was able to test it and it works. Thank you 🙏