Jan
Running Alternative Engines (e.g. LlamaFile, PowerInfer) #3191
BradKML
started this conversation in
Feature Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Recently, Llamafile from Mozilla claimed that they managed to 10x CPU inference. Wondering if it is worth it to integrate Jan into it to speed things up for smaller models on portable hardware.
Beta Was this translation helpful? Give feedback.
All reactions