Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

llm-llm needs json file for llama3.2-1B-prefill-ax630c #9

Closed
nyasu3w opened this issue Feb 3, 2025 · 3 comments
Closed

llm-llm needs json file for llama3.2-1B-prefill-ax630c #9

nyasu3w opened this issue Feb 3, 2025 · 3 comments

Comments

@nyasu3w
Copy link

nyasu3w commented Feb 3, 2025

Environment: StackFlow v1.4.0 and M5Module-LLM dev branch

When I tried to use llama3.2-1B-prefill-ax630c on llm-llm, it failed because of missing json file.
The code of M5Module-llm side is as below.

    m5_module_llm::ApiLlmSetupConfig_t config;
    config.max_token_len = 1023; 
    config.model = "llama3.2-1B-prefill-ax630c";
    llm_work_id = module_llm.llm.setup(config);

It seemed to need llama3.2-1B-prefill-ax630c.json. It succeeded when I copied (symlinked) mode_llama3.2-1B-prefill-ax630c.json
to llama3.2-1B-prefill-ax630c.json in /opt/m5stack/data/llama3.2-1B-prefill-ax630c .

@Abandon-ht
Copy link
Contributor

Thanks for your feedback. If your llama3.2-1B-prefill-ax630c installation method is through https://github.com/m5stack/StackFlow/releases/download/v1.3.0/llm-llama3.2-1B-prefill-ax630c_0.2-m5stack1_arm64.deb. The file mode_llama3.2-1B-prefill-ax630c.json will exist in /opt/m5stack/data/models/. llama3.2-1B-prefill-ax630c can be loaded normally.

@nyasu3w
Copy link
Author

nyasu3w commented Feb 6, 2025

Thanks!

@nyasu3w
Copy link
Author

nyasu3w commented Feb 8, 2025

I confirmed that it work fine without my symlink and the json exists at the path.
I am not sure what went wrong before, but it should be my mistake.
Let me close this issue.

@nyasu3w nyasu3w closed this as completed Feb 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants