Integration with Local LLMs like GPT4All, Jan.ai and H2OGpt

[GerryOnGithub]

Can OpenAI-DotNet integrate with GPT4All, Jan.ai and/or H2OGpt? These local models support OpenAI API standards so I'm curious if anyone has any experience integrating with them.

There is also Khoj but all I can get out of it is crashes.

Jan.ai is really nice as a local model host, but I haven't gotten local doc integration working yet.

[StephenHodgson]

This repo is only for OpenAI and OpenAI for Azure.

[uxmond]

I'm working on https://github.com/SpongeEngine/LocalAI.NET (just started today). I hope to finish in a day or two. Will try adding GPT4All, Jan.ai and/or H2OGpt if they make sense. May I ask why you are specifically interested in those? Why not KoboldCpp, Ollama, LM Studio or Oobabooga?

[GerryOnGithub]

I'm interested in local docs, which those three support. Don't know how well yet, or about others. jan.ai local doc support in beta. gpt4all was able to read one of two pdfs, but pdfs can be a total mess so I blame the pdf.

…

On Sun, Dec 29, 2024, 3:05 PM Dan Clipca ***@***.***> wrote: I'm working on https://github.com/SpongeEngine/LocalAI.NET (just started today). I hope to finish in a day or two. Will try adding GPT4All, Jan.ai and/or H2OGpt if they make sense. May I ask why you are specifically interested in those? Why not KoboldCpp, Ollama, LM Studio or Oobabooga? — Reply to this email directly, view it on GitHub <#393 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFS3SXTUABJRLHKT3AXJV2T2IB54NAVCNFSM6AAAAABUKFY4OKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNRZGI4DCMI> . You are receiving this because you authored the thread.Message ID: <RageAgainstThePixel/OpenAI-DotNet/repo-discussions/393/comments/11692811@ github.com>

[uxmond]

I see. I'll try adding them if that functionality is exposed via APIs. Will keep you updated here if that's alright.

[uxmond]

I'm interested in local docs, which those three support. Don't know how well yet, or about others. jan.ai local doc support in beta. gpt4all was able to read one of two pdfs, but pdfs can be a total mess so I blame the pdf.
…
On Sun, Dec 29, 2024, 3:05 PM Dan Clipca @.***> wrote: I'm working on https://github.com/SpongeEngine/LocalAI.NET (just started today). I hope to finish in a day or two. Will try adding GPT4All, Jan.ai and/or H2OGpt if they make sense. May I ask why you are specifically interested in those? Why not KoboldCpp, Ollama, LM Studio or Oobabooga? — Reply to this email directly, view it on GitHub <#393 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFS3SXTUABJRLHKT3AXJV2T2IB54NAVCNFSM6AAAAABUKFY4OKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNRZGI4DCMI . You are receiving this because you authored the thread.Message ID: <RageAgainstThePixel/OpenAI-DotNet/repo-discussions/393/comments/11692811@ github.com>

Are you still using GPT4All? KoboldCpp added document support as per https://github.com/LostRuins/koboldcpp/releases/tag/v1.81. I almost finished GPT4AllSharp but their server implementation is really bad compared to KoboldCpp. It's not as stable (crashes a lot) and doesn't even have features like streaming or stop tokens, as far as it seems.