Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Misc. bug: cannot convert GLM-4-9B-Chat (glm-4-9b-chat-hf) to GGUF format #11263

Open
MoonRide303 opened this issue Jan 16, 2025 · 1 comment

Comments

@MoonRide303
Copy link

MoonRide303 commented Jan 16, 2025

Name and Version

.\llama-cli.exe --version
version: 4491 (c67cc98)
built with MSVC 19.39.33523.0 for x64

Operating systems

Windows

Which llama.cpp modules do you know to be affected?

Python/Bash scripts

Command line

python convert_hf_to_gguf.py --outtype f16 ..\glm-4-9b-chat-hf\ --outfile glm-4-9b-chat-hf-F16.gguf

Problem description & steps to reproduce

Despite ChatGLM4-9b being marked as supported attempt to convert GLM-4-9B-Chat model (glm-4-9b-chat-hf, command above) to GGUF format fails, with the following error:

INFO:hf-to-gguf:Loading model: glm-4-9b-chat-hf
ERROR:hf-to-gguf:Model GlmForCausalLM is not supported

First Bad Commit

No response

Relevant log output

@arch-btw
Copy link
Contributor

The transformers/hf-only version hasn't been added yet, please see last 2 comments here: #10573

For now the only one that works is the same model but slightly different in that it isn't a pure transformers implementation:

https://huggingface.co/THUDM/glm-4-9b-chat

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants