Out of memory with T4 GPU

[dgoryeo]

I am doing some tests with stable-ts, by running batch transribe on 10 mp3 files --in a for loop. I have couple of times given out of memory error when on T4 GPU. I wonder what causes it. Below is the error message.

I swtched to running on A100 40GB GPU VRAM and I can see on resource monitor that everytime stable-ts is called on a new file, the memory usage jumps --see the graph below.

I wonder if I am doing something silly or am missing some optimisation technique.

/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py in convert(t)
983 return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None,
984 non_blocking, memory_format=convert_to_format)
--> 985 return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
986
987 return self._apply(convert)

OutOfMemoryError: CUDA out of memory. Tried to allocate 26.00 MiB (GPU 0; 14.76 GiB total capacity; 13.19 GiB already allocated; 5.75 MiB free; 13.71 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF