Questions regarding using it for tts

[thewh1teagle]

Hi there! :)

I’m using nakdimon in nakdimon-ort to create a Hebrew TTS model. I plan to train the model using phonemes (IPA) rather than diacritical text (though it will be layered on top) for simplicity and consistency with other models.

Currently, I use espeak-ng phonemizer to generate the phonemes, but it lacks many letters with niqqud.
I’d like to add every possible letter and niqqud output from nakdimon. How can I go about this?

Additionally, I noticed that some of the letters generated by nakdimon don’t have niqqud (when not needed). If there’s no niqqud, does it make sense to ignore it, as it typically doesn’t produce any sound?

Thanks!

[elazarg]

You probaly want to use dicta: https://nakdan.dicta.org.il/
It has an API that can be queried for all possible diacrtitizations. It's also more accurate.

[elazarg]

Improving it amounts to finding more and better data.
I am looking into using Claude Opus to diacritize data - it looks like it beats Dicta hands down, except for being inconsistent with generation of כתיב מלא.

[elazarg]

Very nice work on the TTS :)

[thewh1teagle]

I am looking into using Claude Opus to diacritize data - it looks like it beats Dicta hands down, except for being inconsistent with generation of כתיב מלא.

Claude add very good diacritics and seems like much better than Dicta.

היי, מתי אתה הולך לסופר?
הֵיי, מָתַי אַתָּה הוֹלֵךְ לַסּוּפֶּר?


שלום, מתי אתה הולך לספר?
שָׁלוֹם, מָתַי אַתָּה הוֹלֵךְ לַסַּפָּר?


שלום, מתי אתה הולך לספר לי?
שָׁלוֹם, מָתַי אַתָּה הוֹלֵךְ לְסַפֵּר לִי?

Does Nakdimon is context aware? because without context such words will never be 100% accurate because ספר has 10 meanings.
Can you show example of the issue with כתיב מלא?

How much data is needed in your opinion? Claude can generate 1M tokens / words in the API for ~1$

[elazarg]

All diacritizers are context aware in some way...

Take any text with words like אישה, Claude will (correctly) remove the יו"ד, but is likely to do so even if asked explicitly not to (it tries, but with mixed results).

[thewh1teagle]

All diacritizers are context aware in some way...

Take any text with words like אישה, Claude will (correctly) remove the יו"ד, but is likely to do so even if asked explicitly not to (it tries, but with mixed results).

Maybe if you have already data from the previous training that already handles כתיב מלא then it will still be preserved