Skip to content

Commit

Permalink
Add api reference
Browse files Browse the repository at this point in the history
Signed-off-by: kerthcet <[email protected]>
  • Loading branch information
kerthcet committed Feb 7, 2025
1 parent 693a0f2 commit 045591d
Show file tree
Hide file tree
Showing 10 changed files with 1,365 additions and 5 deletions.
6 changes: 6 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
include Makefile-deps.mk

# ENVTEST_K8S_VERSION refers to the version of kubebuilder assets to be downloaded by envtest binary.
ENVTEST_K8S_VERSION = 1.28.3
ENVTEST_LWS_VERSION = v0.4.0
Expand Down Expand Up @@ -86,6 +88,10 @@ generate: controller-gen code-generator ## Generate code containing DeepCopy, De
$(CONTROLLER_GEN) object:headerFile="hack/boilerplate.go.txt" paths="./..."
./hack/update-codegen.sh go $(PROJECT_DIR)/bin

.PHONY: generate-apiref
generate-apiref: genref
cd $(PROJECT_DIR)/hack/genref/ && $(GENREF) -o $(PROJECT_DIR)/docs/reference

# Use same code-generator version as k8s.io/api
CODEGEN_VERSION := $(shell go list -m -f '{{.Version}}' k8s.io/api)
CODEGEN = $(shell pwd)/bin/code-generator
Expand Down
14 changes: 14 additions & 0 deletions Makefile-deps.mk
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
PROJECT_DIR := $(shell dirname $(abspath $(lastword $(MAKEFILE_LIST))))

ifeq (,$(shell go env GOBIN))
GOBIN=$(shell go env GOPATH)/bin
else
GOBIN=$(shell go env GOBIN)
endif
GO_CMD ?= go


GENREF = $(PROJECT_DIR)/bin/genref
.PHONY: genref
genref: ## Download genref locally if necessary.
@GOBIN=$(PROJECT_DIR)/bin $(GO_CMD) install github.com/kubernetes-sigs/reference-docs/[email protected]
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ spec:
modelName: opt-125m
```
### Test
### Verify
#### Expose the service
Expand Down
10 changes: 6 additions & 4 deletions docs/develop.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ A develop guidance for people who want to learn more about this project.

```structure
llmaz # root
├── bin # where the binaries locates, like the kustomize, ginkgo, etc.
├── chart # where the helm chart locates
├── cmd # where the main entry locates
├── docs # where all the documents locate, like examples, installation guidance, etc.
├── llmaz # where the model loader logic locates
├── pkg # where the main logic for Kubernetes controllers locates
```
Expand All @@ -14,10 +18,8 @@ llmaz # root

### Core APIs

**OpenModel**: `OpenModel` is mostly like to store the open sourced models as a cluster-scope object. We may need namespaced models in the future for tenant isolation. Usually, the cloud provider or model provider should set this object because they know models well, like the accelerators or the scaling primitives.
See the [API Reference](./reference/core.v1alpha1.md) for more details.

### Inference APIs

**Playground**: `Playground` is for easy usage, people who has little knowledge about cloud can quick deploy a large language model with minimal configurations. `Playground` is integrated with the SOTA inference engines already, like vLLM.

**Service**: `Service` is the real inference workload, people has advanced configuration requirements can deploy with `Service` directly if `Playground` can not meet their demands like they have a customized inference engine, which hasn't been integrated with llmaz yet. Or they have different topology requirements to align with the Pods.
See the [API Reference](./reference/inference.v1alpha1.md) for more details.
Loading

0 comments on commit 045591d

Please sign in to comment.