Better support for Microsoft TTS?

[Gvolten]

Hi. I would like to use this integration with the Microsoft Azure TTS service. But I have a few problems with that.

What problems have I encountered?

First of all, I would like to be able to announce in several different languages. When calling the Microsoft TTS service directly, I can just specify the options like this:

language: en-us
options:
  gender: Female
  type: AriaNeural

But I have found no equivalent here.

Secondly, is it just me, or does the sound from this integration seem worse than from the standard one?
Microsoft supports downloading different qualities from their service. Could the user maybe specify one here?

How am I imagining the improvements?

Remove the language, TLD and gender fields. Add just one field, maybe call it options and let the user define what will be passed directly to the integration.
Do not compress the audio in any way and use the standard quality, or let the user specify.

---

I may be missing some crucial setting somewhere, so please let me know if you have figured out a solution for these issues.
Thanks in advance 👍

[nimroddolev]

Hi @Gvolten, thank you for message, I appreciate your input.

I agree that having an options field to support additional service options as you suggested would be beneficial.

I wanted to make the chime_tts.say service as user friendly as possible by exposing the various common & platform-specific configuration options as separate fields. I think the type option for Microsoft TTS could be added as an additional field, but to your point: I think an options field should be included, which would override the other fields, for users who would rather define the options that way instead of definng each field separately. This would allow for easier conversion of existing scripts/automations given the shared options parameter.
I feel this solution would be the best of both worlds.

Regarding the sound quality, I have not noticed any quality issues, that being said the audio generated is indeed a new MP3 so it is a possibility. The only intentional manipulations done are to the TTS audio speed when set to any value above 100%.

Thanks again for the great feedback!

[nimroddolev]

Hi @Gvolten, I have released a new beta version which adds a new options field.

To install the new version:

1. Open the Chime TTS repository in HACS
2. Click on the ⋮ button in the top right
3. Select the ↻ Redownload option
4. Enable the Show beta versions option
5. Select version v0.7.4-beta2 from the updated version drop-down list

Please let me know if you try the new version and if it is working for you 👍

[nimroddolev]

Thank you. Can you please also provide the full debug logs for the service calls to chime_tts.say?

To enable debug logging:

1. Turn on debug logging

service: logger.set_level
data:
  custom_components.chime_tts: debug

2. Call the service chime_tts.say again.

3. Check your log messages in Home Assistant:

https://{YOUR_HOME_ASSISTANT_ADDRESS}:8123/config/logs?filter=chime_tts

4. Click LOAD FULL LOGS

[Gvolten]

I'd rather not share the entire log (because of security concerns), but these are the most important messages that appear:

ERROR (SyncWorker_5) [pycsspeechtts.pycsspeechtts] Text synthesis failed, statuscode 400, reason:
WARNING (MainThread) [custom_components.chime_tts] - Error calling tts.async_get_media_source_audio: No TTS from Microsoft for 'Test'
WARNING (MainThread) [custom_components.chime_tts] Unable to generate TTS audio

I think the problem is, how the language is passed to the actual Microsoft service.

[nimroddolev]

I appreciate your security concerns, could you please provide a redacted log without sensitive information?
I do not use the Microsoft Azure TTS service so I am quite in the dark.

[Gvolten]

INFO (MainThread) [custom_components.chime_tts] Media player entity ***** is turned off. Turning on...
DEBUG (MainThread) [custom_components.chime_tts] async_get_playback_audio_path
DEBUG (MainThread) [custom_components.chime_tts] get_audio_from_path("custom_components/chime_tts/mp3s/bells_2.mp3", 0, audio)
DEBUG (MainThread) [custom_components.chime_tts] - Retrieving audio from path: "custom_components/chime_tts/mp3s/bells_2.mp3"...
DEBUG (MainThread) [custom_components.chime_tts] - ...audio with duration 1.279s retrieved successfully
DEBUG (MainThread) [custom_components.chime_tts] async_request_tts_audio(hass, tts_platform = microsoft, message = Test, tts_playback_speed = 100.0, cache = False, language = en-us, type = AriaNeural)
DEBUG (MainThread) [custom_components.chime_tts] - Generating TTS audio...
ERROR (SyncWorker_5) [pycsspeechtts.pycsspeechtts] Text synthesis failed, statuscode 400, reason:
WARNING (MainThread) [custom_components.chime_tts] - Error calling tts.async_get_media_source_audio: No TTS from Microsoft for 'Test'
WARNING (MainThread) [custom_components.chime_tts] Unable to generate TTS audio
DEBUG (MainThread) [custom_components.chime_tts] - Final audio created:
DEBUG (MainThread) [custom_components.chime_tts] - Duration = 1.279s
DEBUG (MainThread) [custom_components.chime_tts] Creating final audio file
DEBUG (MainThread) [custom_components.chime_tts] - Temp folder exists: /media/sounds/temp/chime_tts/
DEBUG (MainThread) [custom_components.chime_tts] - Creating temporary mp3 file...
DEBUG (MainThread) [custom_components.chime_tts] - temp_obj = <tempfile._TemporaryFileWrapper object at *****>
DEBUG (MainThread) [custom_components.chime_tts] - Filepath = '/media/sounds/temp/chime_tts/*****.mp3'
DEBUG (MainThread) [custom_components.chime_tts] - File saved successfully
DEBUG (MainThread) [custom_components.chime_tts] - audio_dict = {'audio_path': '/media/sounds/temp/chime_tts/*****.mp3', 'audio_duration': 1.279}
DEBUG (MainThread) [custom_components.chime_tts] Calling media_player.play_media service with data:
DEBUG (MainThread) [custom_components.chime_tts] - media_content_type: music
DEBUG (MainThread) [custom_components.chime_tts] - media_content_id: media-source://media_source/local/sounds/temp/chime_tts/*****.mp3
DEBUG (MainThread) [custom_components.chime_tts] - entity_id: ['*****']
DEBUG (MainThread) [custom_components.chime_tts] ...media_player.play_media completed.
DEBUG (MainThread) [custom_components.chime_tts] Waiting 1.279s for audio playback to complete...
DEBUG (MainThread) [custom_components.chime_tts] ----- Chime TTS Say Completed in ***** ms -----
DEBUG (MainThread) [custom_components.chime_tts] Removing current queued service call.
DEBUG (MainThread) [custom_components.chime_tts] Queue emptied. Reinitializing values.

[nimroddolev]

Thank you. I will work on a fix.

[dbullendsl]

Add my vote to having an overriding Options field. I've used Amazon Polly features for various text announcements. You could also have a drop down list of options to use based on the choice of TTS platforms.

[nimroddolev]

There already is an overriding Options field:

I also thought about adding addition drop down lists / checkboxes for the options of each tts platform, but the list is quite long and would likely clutters the UI.

[Gvolten]

Hi. After I while, I decided to check out this project again.
It looks great now that you can fully customize the whole TTS message (add as many segments as you want)
But I still have an issue with Microsoft TTS. It says that the language field is not supported, but that is not the case. Check out the documentation here: https://www.home-assistant.io/integrations/microsoft/
Because I am a beginner developer, I decided to try and add MICROSOFT_TTS to the list of supported integrations for this field, but this still failed. Here is a redacted debug log:

2024-06-16 15:04:02.199 DEBUG (MainThread) [custom_components.chime_tts] ----- Chime TTS Say Called. Version v1.0.0 -----
2024-06-16 15:04:02.199 DEBUG (MainThread) [custom_components.chime_tts.queue_manager] Adding service call to queue
2024-06-16 15:04:02.200 INFO (MainThread) [custom_components.chime_tts.helpers.media_player] Media player entity "*****REDACTED*****" is turned off. Turning on...
2024-06-16 15:04:05.773 DEBUG (MainThread) [custom_components.chime_tts.helpers.media_player] ```Media players found: [{'entity_id': '*****REDACTED*****', 'should_change_volume': True, 'initial_volume_level': -1.0, 'group_members_supported': False, 'announce_supported': False, 'resume_media_player': False}]
2024-06-16 15:04:05.773 DEBUG (MainThread) [custom_components.chime_tts.helpers.helpers] ----- General Parameters -----
2024-06-16 15:04:05.774 DEBUG (MainThread) [custom_components.chime_tts.helpers.helpers] * entity_ids = ['*****REDACTED*****']
2024-06-16 15:04:05.774 DEBUG (MainThread) [custom_components.chime_tts.helpers.helpers] * cache = False
2024-06-16 15:04:05.774 DEBUG (MainThread) [custom_components.chime_tts.helpers.helpers] * offset = 0.0
2024-06-16 15:04:05.774 DEBUG (MainThread) [custom_components.chime_tts.helpers.helpers] * final_delay = 0.0
2024-06-16 15:04:05.774 DEBUG (MainThread) [custom_components.chime_tts.helpers.helpers] * message = [{'chime': '*****REDACTED*****'}, {'tts': 'Announcement Test', 'language': 'en-us'}, {'chime': '*****REDACTED*****'}]
2024-06-16 15:04:05.774 DEBUG (MainThread) [custom_components.chime_tts.helpers.helpers] * tts_platform = microsoft
2024-06-16 15:04:05.775 DEBUG (MainThread) [custom_components.chime_tts.helpers.helpers] * tts_speed = 100.0
2024-06-16 15:04:05.775 DEBUG (MainThread) [custom_components.chime_tts.helpers.helpers] * tts_pitch = 0
2024-06-16 15:04:05.775 DEBUG (MainThread) [custom_components.chime_tts.helpers.helpers] * announce = False
2024-06-16 15:04:05.775 DEBUG (MainThread) [custom_components.chime_tts.helpers.helpers] * volume_level = 0.5
2024-06-16 15:04:05.775 DEBUG (MainThread) [custom_components.chime_tts.helpers.helpers] * join_players = False
2024-06-16 15:04:05.775 DEBUG (MainThread) [custom_components.chime_tts.helpers.helpers] * unjoin_players = False
2024-06-16 15:04:05.778 DEBUG (MainThread) [custom_components.chime_tts] - Retrieving audio from path: "*****REDACTED*****"...
2024-06-16 15:04:06.068 DEBUG (MainThread) [custom_components.chime_tts] ...audio retrieved. Duration: 3.152s
2024-06-16 15:04:06.068 DEBUG (MainThread) [custom_components.chime_tts] - Generating new TTS audio with parameters:
2024-06-16 15:04:06.068 DEBUG (MainThread) [custom_components.chime_tts] * tts_platform = 'microsoft'
2024-06-16 15:04:06.069 DEBUG (MainThread) [custom_components.chime_tts] * message = 'Announcement Test'
2024-06-16 15:04:06.069 DEBUG (MainThread) [custom_components.chime_tts] * cache = False
2024-06-16 15:04:06.069 DEBUG (MainThread) [custom_components.chime_tts] * language = "en-us"
2024-06-16 15:04:06.069 DEBUG (MainThread) [custom_components.chime_tts] * options = {}
2024-06-16 15:04:06.721 ERROR (SyncWorker_12) [pycsspeechtts.pycsspeechtts] Text synthesis failed, statuscode 400, reason:
2024-06-16 15:04:06.730 ERROR (MainThread) [custom_components.chime_tts] - Error calling tts.async_get_media_source_audio: No TTS from Microsoft for 'Announcement Test'
2024-06-16 15:04:06.730 WARNING (MainThread) [custom_components.chime_tts] Error generating TTS audio from messsage segment #2: {'language': 'en-us', 'type': 'tts', 'message': 'Announcement Test'}
2024-06-16 15:04:06.731 DEBUG (MainThread) [custom_components.chime_tts] - Retrieving audio from path: "*****REDACTED*****"...
2024-06-16 15:04:07.028 DEBUG (MainThread) [custom_components.chime_tts] ...audio retrieved. Duration: 4.597s
2024-06-16 15:04:07.030 DEBUG (MainThread) [custom_components.chime_tts.helpers.helpers] Combining audio files with no delay or overlay
2024-06-16 15:04:07.031 DEBUG (MainThread) [custom_components.chime_tts] - Saving mp3 file to folder: /media/sounds/temp/chime_tts/...
2024-06-16 15:04:07.494 DEBUG (MainThread) [custom_components.chime_tts.helpers.filesystem] - File saved to path: /media/sounds/temp/chime_tts/66wajn87.mp3
2024-06-16 15:04:07.807 DEBUG (MainThread) [custom_components.chime_tts] - Chime TTS audio generated:
2024-06-16 15:04:07.807 DEBUG (MainThread) [custom_components.chime_tts] - audio_duration = 7.749
2024-06-16 15:04:07.807 DEBUG (MainThread) [custom_components.chime_tts] - local_path = "/media/sounds/temp/chime_tts/66wajn87.mp3"
2024-06-16 15:04:07.807 DEBUG (MainThread) [custom_components.chime_tts] - public_path = None
2024-06-16 15:04:07.807 DEBUG (MainThread) [custom_components.chime_tts] - media_content_id = "media-source://media_source/local/sounds/temp/chime_tts/66wajn87.mp3"
2024-06-16 15:04:07.807 DEBUG (MainThread) [custom_components.chime_tts] *** Pre-Playback Actions ***
2024-06-16 15:04:07.807 DEBUG (MainThread) [custom_components.chime_tts] - Setting volume level to 0.5 for 1 non-fade in/out media_player
2024-06-16 15:04:07.808 DEBUG (MainThread) [custom_components.chime_tts.helpers.media_player] - Setting *****REDACTED*****'s volume to 0.5
2024-06-16 15:04:08.838 DEBUG (MainThread) [custom_components.chime_tts] *** Chime TTS playback ***
2024-06-16 15:04:08.838 DEBUG (MainThread) [custom_components.chime_tts] 1 Standard media player detected:
2024-06-16 15:04:08.839 DEBUG (MainThread) [custom_components.chime_tts] - *****REDACTED*****
2024-06-16 15:04:08.839 DEBUG (MainThread) [custom_components.chime_tts] Calling media_player.play_media with data:
2024-06-16 15:04:08.839 DEBUG (MainThread) [custom_components.chime_tts] - media_content_type: music
2024-06-16 15:04:08.839 DEBUG (MainThread) [custom_components.chime_tts] - media_content_id: media-source://media_source/local/sounds/temp/chime_tts/66wajn87.mp3
2024-06-16 15:04:08.839 DEBUG (MainThread) [custom_components.chime_tts] - entity_id: ['*****REDACTED*****']
2024-06-16 15:04:08.843 DEBUG (MainThread) [custom_components.chime_tts] - Waiting 7.749s for audio playback to complete...
2024-06-16 15:04:16.597 DEBUG (MainThread) [custom_components.chime_tts.helpers.media_player] - Waiting until 1 media_player isn't playing...
2024-06-16 15:04:18.006 DEBUG (MainThread) [custom_components.chime_tts.helpers.media_player] ✔ *****REDACTED*****
2024-06-16 15:04:18.007 DEBUG (MainThread) [custom_components.chime_tts] *** Post-Playback Actions ***
2024-06-16 15:04:18.011 DEBUG (MainThread) [custom_components.chime_tts] ----- Chime TTS Say Completed in 15.81 s -----

Could you add the support for changing language? I use it (the official way) in my automations with no problems.

[Gvolten]

Oh, I made it work in some way. Don't know why it didn't work, but it works now. So yes, Microsoft does support languages.
Btw the quality still seems worse then the official way, but it's good enough.