: Unknown vision tower: /share/junjie/shuyan/clip-vit-large-patch14-336

[wwpu]

Traceback (most recent call last):
File "/home/LLM/videoxl/videoxl/infer.py", line 17, in
tokenizer, model, image_processor, _ = load_pretrained_model(model_path, None, "llava_qwen", device_map="cuda:0")
File "/home/LLM/videoxl/videoxl/videoxl/model/builder.py", line 215, in load_pretrained_model
model = LlavaQwenForCausalLM.from_pretrained(model_path, low_cpu_mem_usage=True, attn_implementation=attn_implementation, **kwargs)
File "/home/LLM/videoxl/videoxl/videoxl/model/language_model/llava_qwen.py", line 1498, in from_pretrained
model, loading_info = super().from_pretrained(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3404, in from_pretrained
model = cls(config, *model_args, **model_kwargs)
File "/home/LLM/videoxl/videoxl/videoxl/model/language_model/llava_qwen.py", line 1466, in init
self.model = LlavaQwenModel(config)
File "/home/LLM/videoxl/videoxl/videoxl/model/language_model/llava_qwen.py", line 1454, in init
super(LlavaQwenModel, self).init(config)
File "/home/LLM/videoxl/videoxl/videoxl/model/llava_arch.py", line 40, in init
self.vision_tower = build_vision_tower(config, delay_load=delay_load)
File "/home/LLM/videoxl/videoxl/videoxl/model/multimodal_encoder/builder.py", line 23, in build_vision_tower
raise ValueError(f"Unknown vision tower: {vision_tower}")

[shuyansy]

Hi, you can download the weight here, and change the path to yours.
https://huggingface.co/openai/clip-vit-large-patch14-336

[wwpu]

Hi, you can download the weight here, and change the path to yours. https://huggingface.co/openai/clip-vit-large-patch14-336

Thank you. I have downloaded it, but I can't find the path where this model is placed.

[zzz130981]

hello, could you find how to change the path?

[zzz130981]

Hi, you can download the weight here, and change the path to yours. https://huggingface.co/openai/clip-vit-large-patch14-336

Thank you. I have downloaded it, but I can't find the path where this model is placed.

may you can create /share/junjie/shuyan directory manually, although it is not elegent.

[wwpu]

Hi, you can download the weight here, and change the path to yours. https://huggingface.co/openai/clip-vit-large-patch14-336

Thank you. I have downloaded it, but I can't find the path where this model is placed.

may you can create /share/junjie/shuyan directory manually, although it is not elegent.
Thank u. I manually modified the path of the vision tower in "multimodal_encoder\builder.py"

[DaozeZhang]

hello, could you find how to change the path?

you may add config.mm_vision_tower = 'local_path' to line 1468 in the llava_qwen.py