Skip to content

Commit

Permalink
chore: update README
Browse files Browse the repository at this point in the history
  • Loading branch information
mcharytoniuk committed Oct 19, 2024
1 parent c654b03 commit d9939f1
Showing 1 changed file with 17 additions and 0 deletions.
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,10 @@ Download the latest release for Linux, Mac, or Windows from the

On Linux, if you want Paddler to be accessible system-wide, rename the downloaded executable to `/usr/bin/paddler` (or `/usr/local/bin/paddler`).

### Running llama.cpp

Slots endpoint is required to be enabled in llama.cpp. To do so, run llama.cpp with the `--slots` flag.

### Running Agents

The next step is to run Paddler’s agents. Agents register your llama.cpp instances in Paddler and monitor the slots of llama.cpp instances.
Expand Down Expand Up @@ -81,6 +85,13 @@ Run the following to start a Paddler’s agent (replace the hosts and ports with
With the `--name` flag, you can assign each agent a custom name. This name will be displayed in the management dashboard and not used for any other purpose.

#### API Key

> [!NOTE]
> Available since v0.9.0
If your llama.cpp instance requires an API key, you can provide it with the `--local-llamacpp-api-key` flag.

### Running Load Balancer

Load balancer collects data from agents and exposes reverse proxy to the outside world.
Expand Down Expand Up @@ -199,6 +210,12 @@ If you want to keep the balancer management address predictable, I recommend usi

## Changelog

### v0.9.0

#### Features

- Add `--local-llamacpp-api-key` flag to balancer to support llama.cpp API keys (see: [#23](https://github.com/distantmagic/paddler/issues/23))

### v0.8.0

#### Features
Expand Down

0 comments on commit d9939f1

Please sign in to comment.