From adcbf3c054a5c86511b5560534e491f1f78c66d5 Mon Sep 17 00:00:00 2001 From: Daniel Bevenius Date: Thu, 2 Jan 2025 12:03:17 +0100 Subject: [PATCH] docs: add more details to the cuda-fp16 issue --- notes/llama.cpp/cuda-fp16-release-build-issue.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/notes/llama.cpp/cuda-fp16-release-build-issue.md b/notes/llama.cpp/cuda-fp16-release-build-issue.md index 6fd9dfb..71ab26c 100644 --- a/notes/llama.cpp/cuda-fp16-release-build-issue.md +++ b/notes/llama.cpp/cuda-fp16-release-build-issue.md @@ -124,3 +124,8 @@ Notice that my GPU has a compute capability of 8.9 which is greater than the reported 520. So there is something wrong with the way the compute capability is being detected. + +Using the following cmake options worked for me +```console +cmake -S . -B build -DGGML_CUDA=ON -DGGML_CUDA_F16=ON -DGGML_NATIVE=OFF -DCMAKE_BUILD_TYPE=Release +```