Coqui and multithreading? #2898

FlorianEagox · 2023-08-28T02:27:25Z

FlorianEagox
Aug 28, 2023

Hello, I'm new to ML and working on a relatively complex piece of software that incorporates TTS, and I was wondering if it was possible to run it in parallel? This software often needs to perform hundreds of tts operations, and while it runs extremely well, I'd like to squeeze as much performance out of it as possible.

Simply running tts_to_file as part of a ProcessPoolExecutor results in a torch serialization error, and even if I try generation and then output with regular tts.tts, I still get a halting issue with no debug info.

If it's simply not possible right now to perform such functions, I'm content with its single-process performance.

Examples:

tts = TTS('tts_models/en/vctk/vits')

tests = ["this is the beepis", "give me a squeepis", "where is my peepis?", "give me a deepis"]
with concurrent.futures.ProcessPoolExecutor() as pool:
		tasks = [pool.submit(lambda: tts.tts_to_file(test, speaker="p326", file_path=f"output/test-{i}.wav")) for i, test in enumerate(tests)]		
		for future in concurrent.futures.as_completed(tasks):
			print(future.__dict__)

results in:

Cowardly refusing to serialize non-leaf tensor which requires_grad, since autograd does not support crossing process boundaries.  If you just want to transfer the data, call detach() on the tensor before serializing (e.g., putting it on the queue).'

I then tried doing something like this:

def tts_detach(test, output_file):
    tts_output = tts.tts(test, speaker="p326")
    tts_output = tts_output.detach()  # Detach the tensor
    tts_output.save(output_file)

But this results in the program hanging with no errors or logging. Coqui will list the splitted sentances and then top.