Support to latest MS Phi-2 model?

[Slavczays]

Hello. First of all thanks for your great app - fantastic work. Is there any chance to add support for newest small llm models Phi-2 2.7 from Microsoft? GGUF model (quantized by the Bloke) is available at HF, but isn't compatible with your app?
Maybe it will be helpful - it looks like Phi-2 support was just recently added to Llama cpp https://github.com/ggerganov/llama.cpp/pull/4490#pullrequestreview-1787346569

[danemadsen]

This build should work for you

https://github.com/Mobile-Artificial-Intelligence/maid/actions/runs/7268881043

will also be in 1.1.6 when its released

[Slavczays]

Thanks for so quick responce. I checked it and now loading phi 2 models doesn't provide to crashing app. Althought it (I mean phi-2) gives some strange responces on output. I guess it might be something wrong with model I downloaded (phi-2.Q5_K_M.gguf from theBloke) however with newest beta LMStudio this gguf works properly. Strange?

[danemadsen]

No problem. I'm currently away for Christmas so i cant really take on any major issues until i get back. Its possible the issue you have may be related to model parameters, but its more likely that its just another issue with my implementation of llama.cpp.

If you could write a detailed description of the problem, how it can be reproduced and what is happening that isn't expected and submit it as an issue that would really help me when I take this on.