How do I run server.py with xtts_v2 model?

[MooTheCow]

I'm trying to run the server.py with the xtts_v2 model because I think that is the only way I can use "speaker_wav" to set a vocal to clone. (Certainly it ignores speaker.wav if I use the ljspeech/vits model).
I can write a simple python script to call tts.tts_to_file() and use the speaker_wav correctly. But I want to modify the server to allow me to do this remotely.

If I run: python server.py --model_name tts_models/en/ljspeech/vits
then I get the server running and can access the http form with that model.

If I run: python server.py --model_name tts_models/multilingual/multi-dataset/xtts_v2
then I get this error:

tts_models/multilingual/multi-dataset/xtts_v2 is already downloaded.
Traceback (most recent call last):
File "C:\Users\bob\AppData\Roaming\Python\Python310\site-packages\TTS\server\server.py", line 104, in
synthesizer = Synthesizer(
File "C:\Users\bob\AppData\Roaming\Python\Python310\site-packages\TTS\utils\synthesizer.py", line 93, in init
self._load_tts(tts_checkpoint, tts_config_path, use_cuda)
File "C:\Users\bob\AppData\Roaming\Python\Python310\site-packages\TTS\utils\synthesizer.py", line 185, in load_tts
self.tts_config = load_config(tts_config_path)
File "C:\Users\bob\AppData\Roaming\Python\Python310\site-packages\TTS\config_init.py", line 86, in load_config
ext = os.path.splitext(config_path)[1]
File "C:\Program Files\Python310\lib\ntpath.py", line 230, in splitext
p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

When I list models I have both models downloaded:
1: tts_models/multilingual/multi-dataset/xtts_v2 [already downloaded]
16: tts_models/en/ljspeech/vits [already downloaded]

But, as I mentioned, my simple python one-shot call works great, so I'm not sure what is up with server.py

from TTS.api import TTS
tts = TTS("tts_models/multilingual/multi-dataset/xtts_v2").to("cuda")
tts.tts_to_file(
    text="The quick brown fox jumped over the lazy dog.",
    speaker_wav="VoiceToClone.wav",
    file_path="Output.wav",
    language="en"
)

[MooTheCow]

I've managed to get the tts-server working with xtts_v2 model, and also use speaker.wav so you can clone a voice.

Command to use to point to your xtts_v2 model-

python server.py --use_cuda true --model_path C:\Users\bob\AppData\Local\tts\tts_models--multilingual--multi-dataset--xtts_v2 --config_path C:\Users\bob\AppData\Local\tts\tts_models--multilingual--multi-dataset--xtts_v2\config.json

Note that, weirdly, the model_path does not include the actual model file, but the config path includes the config file. I think this is the error that this bug refers to: it needs the config file to be directly referenced.

But there were some more changes I made to get things working.
Fix "AssertionError: Language is not supported." and "TypeError: Invalid file: None"
It seems that xtts is multilingual so needs a language, which the UI does not set, so I have hardcoded it to "en" here.
And also xtts requires speaker.wav, which is also not set, so again I hardcoded that. In the future I plan to send this along with the request.

server.py [Line 203-ish] -

def tts():
    with lock:
        text = request.headers.get("text") or request.values.get("text", "")
        speaker_idx = request.headers.get("speaker-id") or request.values.get("speaker_id", "")
        language_idx = request.headers.get("language-id") or request.values.get("language_id", "")
        style_wav = request.headers.get("style-wav") or request.values.get("style_wav", "")
        style_wav = style_wav_uri_to_dict(style_wav)
        print(f" > Model input: {text}")
        print(f" > Speaker Idx: {speaker_idx}")
        print(f" > Language Idx: {language_idx}")
        #CHANGE THIS LINE: wavs = synthesizer.tts(text, speaker_name=speaker_idx, language_name=language_idx, style_wav=style_wav)
        #TO THIS-
        wavs = synthesizer.tts(text, language_name="en", style_wav=style_wav, speaker_wav="YOUR_SPEAKER_WAV.wav")
        out = io.BytesIO()
        synthesizer.save_wav(wavs, out)
    return send_file(out, mimetype="audio/wav")

Fix error "The following model_kwargs are not used by the model: ['speaker_id'] "
It seems that xtts does not like speaker_id for some reason.

synthesizer.py [Line 384-ish] -

            if hasattr(self.tts_model, "synthesize"):
                outputs = self.tts_model.synthesize(
                    text=sen,
                    config=self.tts_config,
                    #COMMENT_THIS_OUT speaker_id=speaker_name,
                    voice_dirs=self.voice_dir,
                    d_vector=speaker_embedding,
                    speaker_wav=speaker_wav,
                    language=language_name,
                    **kwargs,
                )

That's hopefully everything.

[krougeau]

FIXED: SEE UPDATE

Hi there! Thanks ever so much for sharing your solution. It's working great as it is, but I was wondering if you ever managed to get the speaker_wav working as part of the request? I've been fiddling with it but can't seem to get it to switch voices. I'm just hacking about willy nilly, lol, but here's what I was trying in server.py:
I added this line:
wav_file = request.headers.get("speaker_wav") or request.values.get("speaker_wav", "")
and then altered this line to suit:
wavs = synthesizer.tts(text, language_name="en", style_wav=style_wav, speaker_wav=wav_file)
I then send a request along the lines of:
http://ServerIP:Port/api/tts?speaker.wav=PATH_TO_MY_SPEAKER_WAV_FILE&text=TEXT_TO_SAY
which works fine, but uses the default voice rather than the one I've specified.
I'm building a custom TTS for a friend who was hoping they could use custom voices for certain people in her chat, so it would be great to get this going. That said, I'm doing it for kicks, so if not, it's not the end of the world either haha.
Thanks again for getting me this far along with it! If need be, I can hack something together that renames the files themselves in the background as they need to change. It's not an elegant solution, but hey, whatever works in the end, right?

UPDATE:
Alright, so I missed something basic & was making life difficult for myself, haha! Turns out an instance of the server hadn't shut down properly in the background (took me way too long to realize it, too). I'd been hacking away at tying this all into Unity & Twitch all day so it's only so surprising & I should probably sleep. Anyway, after a fresh reboot, it all seems to work with the code I'm currently using which I'll include below. The earlier code may well work for all I know, but here's what it looked like by the time I was done.

In the server.py file I added this line:
speaker_wav = request.headers.get("speaker-wav") or request.values.get("speaker_wav", "")

and altered this line to suit:
wavs = synthesizer.tts(text, language_name="en", style_wav=style_wav, speaker_wav=speaker_wav)

and it seems to work! Now I've just gotta code it all in to recognize people and switch to their files. Joy!
Thanks again & again for your post as you're the only one I've seen out there with this information, and I was digging, haha!
Next I'll see if there's a way to use multiple speaker wavs, but first, some much needed rest.

[unoexperto]

I still get the same error. What should be inside config.json ? Could you please paste yours ?

[krougeau]

Hello. I didn't make any modifications to the default config.json file, so it's still default. The changes I made were to the server.py file and the synthesizer.py file, and that is all. Sorry, I wish I could be of more help, but that really is all I altered.