ollama cpu and memory profiling

[wassfila]

ollama run llama3.2-128k
model requires (58.0 GiB)

ollama run llama3.2-64k
model requires (44.5 GiB)

test sentence

"can you tell me the difference between async in python and javascript ?"

test with llama3.2-32k (3B)

total duration: 38.7275623s
load duration: 18.4363ms
prompt eval count: 38 token(s)
prompt eval duration: 302.583ms
prompt eval rate: 125.59 tokens/s
eval count: 535 token(s)
eval duration: 38.403926s
eval rate: 13.93 tokens/s

test with llama3-32k (8B)

total duration: 1m6.9071719s
load duration: 26.1333ms
prompt eval count: 23 token(s)
prompt eval duration: 593.725ms
prompt eval rate: 38.74 tokens/s
eval count: 485 token(s)
eval duration: 1m6.284375s
eval rate: 7.32 tokens/s

test summarizaition

markdown format of this page https://www.homesmartmesh.com/protocols/ultrawideband/
https://github.com/HomeSmartMesh/website/blob/main/content/protocols/ultrawideband/readme.md

Tokens : 10235

llama3.2-32k (3B)

llama3-32k (8B)

model fails two times outputting giberich "2020 {} The 2023 the 2020 {} of the 2020 {}" repetitively

llama3 (8B)