How to run the model on another machine and send the answer to another machine.

[ixn3rd3mxn]

System Info

transformers 4.31.0 , window os , python 3.10.12

Who can help?

vision models: @amyeroberts, @qubvel

I have tried using this model on my machine myself, and it works normally, but the processing is very slow because the GPU on my machine is not that powerful. However, I have a server with a strong GPU. If I install this model on the server and run the code on my machine, when it reaches the video processing stage, it sends the task to the server, and the server sends back the result. Then my machine will print the answer and display the result. Is this possible? If so, how can I do it?

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

.

Expected behavior

I expect it to work in a hybrid way between my computer and the server to achieve faster results.

[david-waterworth]

You want a model/inference server. Perhaps start with gradio as it's relatively simple. NVIDIA Triton is another example if you're looking for something "industrial strength)

[qubvel]

Hi @ixn3rd3mxn, we are trying to keep issues on github for bug reports, feature requests, and issues related to the library code. For your question, it's better to use the forum https://discuss.huggingface.co/. But in general, I agree with @david-waterworth, it's possible and gradio or any framework like Flask or FastAPI might be useful for your purpose.