[Question]:Concurrent inference is stuck using AltDiffusion

[6Roy]

Description

I deployed this part of the code on the web side, but in the concurrent inference stuck, that is, two users at the same time inference graph stuck, do you know how to solve it? Here's the code: I moved the model into cuda to inference, is that the reason? Should I put it on the cpu?
我把这部分代码部署在了web端，但是在进行并发推理中卡死，也就是2个用户同时打开web端，进行推理生图卡住，请问你知道怎么解决么？以下是代码：我把模型移入到了cuda中然后进行inference，难道是这个原因？应该放到cpu上？
load model.py
self.pipe = AltDiffusionPipeline.from_pretrained("/home/yuxiang/pretrain_modals/AltDiffusion-m18",revision='fp16',torch_dtype=torch.float16)
device = torch.device("cuda:2") # 指定使用第一个可用的CUDA设备
self.pipe = self.pipe.to(device)
self.pipe.scheduler = DPMSolverMultistepScheduler.from_config(self.pipe.scheduler.config)
infer.py
news = self.split_text(news, num_parts=4)
prompts=news
imagelist=[]
option_seed=100
start_time = time.time() # 获取当前时间
for num,i in enumerate(prompts):#为4个不同的prompt生成图片
i = f"{i}:{model}"
image= self.pipe(i, num_inference_steps=15, guidance_scale=9.0,width=512,height=512).images[0]
imagelist.append(image)
end_time = time.time() # 获取当前时间
print("Total time taken: ", end_time - start_time, "secondss") # 打印运行时间
return imagelist
Here is a screenshot of the runtime jam:
以下是运行时卡死的截图：

Alternatives

No response