Inference Speed

[vefalun]

Hello, I would like to ask what the approximate inference speed of the model can reach after deployment?

Thanks for your great work!

[XMHZZ2018]

@vefalun

Thanks for your interest in our work! The inference speed largely depends on the data (e.g., image resolution) and the available computing resources (for batching the inputs). Additionally, we have several versions of VLM2Vec, each with a different number of parameters. For reference, on MMEB-eval, a 7B model takes about 10 ~ 20 GPU hours on an H100. Our model is also integrated into vLLM, which I believe further enhances inference speed. Let me know if this answers your question!