fix crash on non-AVX systems dynamically loading GGML CPU backends #11780
+9
−8
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Thanks for the awesome work by @slaren in #10469 (and a few follow up PRs) to enable dynamic GGML backend loading. This made supporting different CPU instructions in GGML much, much easier.
I noticed a small hitch with with the
llamafile
code where a machine with a non-AVX CPU would crash when trying todlopen
CPU libraries built withGGML_LLAMAFILE=ON
. This moves the AVX-dependent code to do a member variable, fixing the crash ondlopen
. I'm not sure howsgemm.cpp
is vendored, and so let me know the best way/place to suggest a change.