Exoplayer has higher latency when starting Multicast/UDP videos compared to MediaPlayer

[GentianB]

We have been trying to convert a video application from using internal MediaPlayer/Videoplayer to Exoplayer library so we can benefit from its advantages,

we are using exoplayer r2.10.5.

one of the issues we came across during the adaptation, is when playing Multicast/UDP videos,

it has a higher latency (higher zapping time) when you change the video streams on exoplayer, because it keeps static frames for both previous video and new video, so first it freeze the last video frame, keeps it for 1-2 seconds, then shows the first frame of the new video and keeps it for another 1-2 second until it finally starts the video.

for the same setup ( same box, same streams, same network, same apk) just changing to Mediaplayer it has a shorter time, because it only keeps the last frame of previous video and starts immediately with the next video, sot it does not show the first frame and then load/buffer, it starts immediately, most probably the android mediaplayer does the buffering during the first stage when it keeps the last frame from previous video. This behaviour is better in terms of user experience and also is lower latency.
When using exoplayer channel changes happen with two freezed frames (last and first video frames) which is not very good experience. You can only observe this with Multicast Videos, when doing HLS the start times are so fast that you can see that.

we have played with buffertimes by lowering bufferForPlaybackMs to 1000ms, and it improves startuptime of the video but you can still see the first frame as static, then loading/buffering and then it starts..
it would be better to do this loading in the first step while it keeps the last frame video and then start immediately with the second/next video.

[LRTNZ]

Just to add to this, I am using Exoplayer to do a similar thing, playing back different multicast streams, that the user can change between with channel up/down.
We are also suffering an issue where the Zap time for getting the media stream the user changes to, is excessively high. What does not help, is that the playbackState will be changed to READY, when the video stream is still frozen, but the audio stream will have begun playing.
We have the buffer time set to 1000ms, or lower, and changing it from the default ~2.5k down makes no perceivable difference in the starting of the video stream playback.

[ojw28]

The problem here is most likely that we just play what's in the source media, and that in these cases the source media contains several seconds of audio before the first video keyframe.

It's not really "wrong" that we do this, and READY is the correct state because we're playing the media. It's not the behaviour you want though. I think the behaviour you want is for the player to discard all audio before the first video keyframe, and start then start from that position (which is tracked by #5024). Does that sound right?

[ojw28]

One other question: Are these streams MPEG-TS?

[GentianB]

Hi,

sample streams below, with HD streams this issue is more noticeable,
in our case the delay is not a big issue, is almost the same as with Mediaplayer, and it think is related to GOP in multicast video streams, but we need to find a solution how to make the next video start immediately and do the caching while keeping the last videos frame.

Stream #0:0[0x3f3]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 50 tbr, 90k tbn, 50 tbc
Stream #0:10x3f4: Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 128 kb/s
Stream #0:20x3f7: Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Stream #0:30x3f8: Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)

or:

Stream #0:0[0x3fd]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt470bg), 704x576 [SAR 16:11 DAR 16:9], 25 fps, 50 tbr, 90k tbn, 50 tbc
Stream #0:10x3ff: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 128 kb/s
Stream #0:20x3fe: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, mono, s16p, 128 kb/s

thnx

[fallenhaivision]

Hi,
Any guidance or hints on how to do this? I've been going through the code for the last few hours and have thought of this -

1. In ExoPlayerImplInternal:startRenderers get the keyframe timeUs (time in uS) from MediaCodecVideoRenderer.java somehow.
2. In MediaCodecAudioRenderer.java discard all audio older, somehow.

It's the somehow that I'm still trying to figure out as I dig through the code.

Thanks

[ojw28]

I suspect that solving #5024 would also fix this problem, so closing as a duplicate of that.