glowTTS | gradTTS | VITS (Monotonic Alignment Search MAS)

[p0p4k]

We discussed how to calculate logp in previous discussion.
This article is explanation for the following MAS code: (finding alignment/path that maximizes logp)

TTS/TTS/tts/utils/helpers.py

Lines 199 to 238 in d6ad9a0

	def maximum_path_numpy(value, mask, max_neg_val=None):
	"""
	Monotonic alignment search algorithm
	Numpy-friendly version. It's about 4 times faster than torch version.
	value: [b, t_x, t_y]
	mask: [b, t_x, t_y]
	"""
	if max_neg_val is None:
	max_neg_val = -np.inf # Patch for Sphinx complaint
	value = value * mask
	device = value.device
	dtype = value.dtype
	value = value.cpu().detach().numpy()
	mask = mask.cpu().detach().numpy().astype(np.bool)
	b, t_x, t_y = value.shape
	direction = np.zeros(value.shape, dtype=np.int64)
	v = np.zeros((b, t_x), dtype=np.float32)
	x_range = np.arange(t_x, dtype=np.float32).reshape(1, -1)
	for j in range(t_y):
	v0 = np.pad(v, [[0, 0], [1, 0]], mode="constant", constant_values=max_neg_val)[:, :-1]
	v1 = v
	max_mask = v1 >= v0
	v_max = np.where(max_mask, v1, v0)
	direction[:, :, j] = max_mask
	index_mask = x_range <= j
	v = np.where(index_mask, v_max + value[:, :, j], max_neg_val)
	direction = np.where(mask, direction, 1)
	path = np.zeros(value.shape, dtype=np.float32)
	index = mask[:, :, 0].sum(1).astype(np.int64) - 1
	index_range = np.arange(b)
	for j in reversed(range(t_y)):
	path[index_range, index, j] = 1
	index = index + direction[index_range, index, j] - 1
	path = path * mask.astype(np.float32)
	path = torch.from_numpy(path).to(device=device, dtype=dtype)
	return path

In this example I assume integer values, but it will be logp values in float (since p is takes values in [0,1]).

(Previous paper discussions : #1494, #1501, #1506, #1799, #1984)

[p0p4k]

PS. Ignore the 48% in the last image.

[erogol]

Awesome content!! Thanks!!