You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I was also experiencing UND_ERR_HEADERS_TIMEOUT errors because the LLM was taking longer to respond than the fetch default of 30 seconds so I made the following changes:
Added the "undici" dependency to package.json and ran npm install:
Added the following to the beginning of ollama.ts: import { fetch, Agent } from 'undici';
And replaced:
// Using fetch to initiate a POST request that will return an SSE stream
const response = await fetch(`http://127.0.0.1:${port}/api/generate`, {
method: 'POST',
headers: headers,
body: requestBody,
});
With:
// Using fetch to initiate a POST request that will return an SSE stream
const response = await fetch(`http://127.0.0.1:${port}/api/generate`, {
method: 'POST',
headers: headers,
body: requestBody,
dispatcher: new Agent({
headersTimeout: 10 * 60e3, // 10 minutes
bodyTimeout: 10 * 60e3, // 10 minutes
}),
});
I also found that when the response from the LLM contained a curly brace, the json parsing would fail so I replaced the following in ollama.ts:
// Attempt to parse all complete JSON objects in the buffer
while (true) {
const openingBraceIndex = textBuffer.indexOf('{');
const closingBraceIndex = textBuffer.indexOf('}');
// Check if we have a complete object
if (openingBraceIndex !== -1 && closingBraceIndex !== -1) {
// Extract and parse the JSON object
const jsonString = textBuffer.slice(
openingBraceIndex,
closingBraceIndex + 1,
);
With (there may be better ways to achieve this):
const regex = /{.*?"done":(?:true|false).*?}/g;
const jsonStrings = textBuffer.match(regex);
if (jsonStrings == null) {
continue;
} else if (jsonStrings.length == 0) {
continue;
}
// Attempt to parse all complete JSON objects in the buffer
for (let i = 0; i < jsonStrings.length; i++) {
const currentJsonString = jsonStrings[i];
if (currentJsonString == null) {
break;
}
const closingBraceIndex = currentJsonString.length;
// Check if we have a complete object
if (closingBraceIndex !== -1) {
const jsonString = currentJsonString;
The text was updated successfully, but these errors were encountered:
I wasn't able to get the movies.run.ts example to work as is and had to change "template:" to "prompt:" here:
https://github.com/hrishioa/wishful-search/blob/master/src/ollama.ts#L24
I was also experiencing UND_ERR_HEADERS_TIMEOUT errors because the LLM was taking longer to respond than the fetch default of 30 seconds so I made the following changes:
Added the "undici" dependency to package.json and ran npm install:
Added the following to the beginning of ollama.ts:
import { fetch, Agent } from 'undici';
And replaced:
With:
I also found that when the response from the LLM contained a curly brace, the json parsing would fail so I replaced the following in ollama.ts:
With (there may be better ways to achieve this):
The text was updated successfully, but these errors were encountered: