-
-
Notifications
You must be signed in to change notification settings - Fork 65
Instances
Instances are AI providers, they manage their own models, text generation among other things.
No, all instances share the same chats, when generating a message the current chat is shared with the instance.
When using instances other than Ollama (Managed), Alpaca doesn't warranty the security of your data, for example if you generate a message using Gemini the whole chat is uploaded to Google's servers temporally.
Please review each AI provider's terms of service before using them with Alpaca.
PDF and ODT files are converted to plain text and appended to the message in the background, the files are not shared in their original state with third party instances.
Images are converted to Base64 before being sent, this is practically the same as just sending the image as is, please don't upload any sensitive information such as passports or contracts to third party instances.
This is an special case, Alpaca will make this instance type available in the case Ollama is detected in your system.
This type of instance manages the Ollama binary, meaning Alpaca has the capability of starting, stopping and modifying Ollama directly, this is the default and easiest way of using Alpaca.
Note
Since the default Flatpak and Snap packages include Ollama this also applies to them.
- Name
- Port (11435 default)
- Temperature
- Seed
- Model Directory
- Overrides (Optional)
This instance type is used to connect to existing Ollama instances:
- Remote Server
- Existing Installation of Ollama
- Containers (Docker, Podman, etc)
Please note that Alpaca won't be able to start, stop or modify the instance, all troubleshooting for Ollama will have to be done outside of Alpaca.
- Name
- Instance URL
- API Key / Bearer Token (Optional)
- Temperature
- Seed
When creating an Ollama instance outside of Alpaca you might add a bearer token, in the past Alpaca had full support for that but it has changed.
When managing models (listing, pulling, creating, etc) Alpaca uses requests to connect to the API and interact with the instance, in this case the bearer token is parsed in the header and it works normally.
When generating a message Alpaca parses the bearer token as the API Key in OpenAI.Client (library) instead of using requests, this means it gets handled in a different way and it might not work for your setup, if you have problems with generating messages whilst using a bearer token please open an issue detailing your setup and I'll take a look at it.
Alpaca uses the OpenAI library to generate content, most AI providers work with this library.
- Name
- API Key
- Max Tokens
- Temperature
- Seed (Not present in Google Gemini or Venice)
Alpaca is not responsible for how much you spent on generating messages nor does it have any warnings when you spend too many tokens, please exercise caution when using third party instances, friendly reminder that Alpaca is distributed with no warranty.
You can set the Max Tokens per instance, this limits how long the generated messages can be, when Alpaca generates a chat title the request's max_tokens parameter is always set to 100.
- OpenAI ChatGPT
- Google Gemini
- Together AI
- Venice AI
This type of instance isn't compatible with system messages, any system messages in a chat are converted internally to user messages when generating a new message.
You can use the option OpenAI Compatible Instance to add other instances like llama.cpp, Alpaca doesn't guarantee this type of instance to work, they get treated essentially the same as OpenAI ChatGPT with the option to change the URL.
- Name
- Instance URL
- API Key
- Max Tokens
- Temperature
- Seed