ConcatenatingMediaSource - momentary frame freeze while switching between clipped media items

[AradiPatrik]

When using ConcatenatingMediaSource the video freezes for a moment when the player is transitioning between MediaItems.
Using ClippingConfiguration.setStartsAtKeyframe solves this. I want our users to be able to set precise clipping for their edited videos thus I can't use this solution.

Below is a screen recording of this behavior:

att.jY6I83zhjK9iXswWv1nqAE9aAVxg7Ba40W5MD2_nHv0.mp4

Notice when the bunny cracks it's neck the frame freezes for around half a second. In my app I don't want this to happen.

I have a repository which contains a minimal project that reproduces this issue: https://github.com/AradiPatrik/SmoothTransitionIssue

I have been searching github issues and have found this issue

• Can ConcatenatingMediaSource connect multiple video sources to achieve smooth transition at the connection? google/ExoPlayer#10408

@marcbaechinger suggested manually disabling codec flushing. I did as suggested but that seem to have just break things (the next video was blank, video player stuck). I couldn't find anything in the related issues either that would point me in the direction of where to start implementing a solution.

I see there are couple ways to customize the behavior of the exo player: https://developer.android.com/media/media3/exoplayer/customization

Could you please give me some pointers on which of these to start with: MediaSource, MediaSource.Factory, Renderer or LoadControl to somehow do some work in advance to avoid the jank?

I also discovered there's a PreloadMediaSource, can I use this class to solve this issue?

[tianyif]

Hi @AradiPatrik,

I think the prerequisite of disabling codec flushing approach is the media item clipped exactly at the key frame, as my colleague previously commented. Please also see google/ExoPlayer#5501 for reference. Could you please confirm if you have done so?

PreloadMediaSource is probably not an ideal solution very suitable for this case, as it can get the upstream media samples available in the sample queue before being played though, it cannot get the codec ready unless it is prepared by the player.

[AradiPatrik]

Hi @tianyif,

As far as I understand, the time between keyframes can be quite long (sometimes several seconds).

I tried clipping exactly at the keyframe, and while the transition was seamless, this approach doesn’t work for my application. I need to allow users to adjust the clip points of each media item precisely.

From this ExoPlayer GitHub issue, it seems there isn’t an out-of-the-box solution for this. Could you provide some guidance on how to implement smooth transitions between media items when they aren’t clipped to start at a keyframe?

In this video:

att.Nlh5mBxxWbYBE-PZOrN5RugfY4WTHBCTArSlFOxUKm4.mp4

you can see that the user is manually setting the clip points, likely not on keyframes, but the transitions remain smooth. How might this be achieved?

Thank you for your help!

[microkatz]

Hello @AradiPatrik,

The delay you are seeing with a playlist of ClippingMediaSources has to do with time it takes to initialize a decoder and process decode-only frames during the media item transition process. The team is currently working on a pre-warming renderer feature that addresses this latency and should help your use case. I will update this thread when the feature is launched so that you can try it. We appreciate your patience!