Output [begin -> end] timestamps for each word

[ronyfadel]

Hello! Is that feasible?

[jianfch]

In short, no.

The model was trained to predict the start and end of a segment. It also tends to predict the start of the next segment as the end of the current segment. Thus, treating the word token predictions as the end timestamp of the segment, in theory, gives the start timestamp of next word token. This script puts that into practice and uses the most probable predicted timestamp tokens of the predicted word tokens as timestamps for those word tokens. But in the current state of the model, it simply cannot predict word timestamps that can be meaningful start and end timestamps of each word. This is why the script gives one timestamp for a word and interprets it as the end of the current word / start of the following word if there is one.

[catalwaysright]

@jianfch Then is it feasible that we don't omit the blank tokens, which means we can get the start timestamps of blank tokens and use them as the end of the last token? Thus, if a one word token's next token is blank, we can still get a relatively accurate end time.

[jianfch]

The current approach of stable-ts chooses timestamp irrespective of the token chosen by the decoder. And the blank token is not suppressed (by default only symbols are suppressed ) so it is not omitted in this sense if this is what you meant.