Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clock, renderer and player customization #2034

Open
roshanpatel6040 opened this issue Jan 11, 2025 · 3 comments
Open

Clock, renderer and player customization #2034

roshanpatel6040 opened this issue Jan 11, 2025 · 3 comments
Assignees
Labels

Comments

@roshanpatel6040
Copy link

Currently, I have a use case where I’m playing fmp4 with exoplayer. This fmp4 chunks are modified and have basemediadecodetime which specifies when that video is recorded. As expected, player current position returns epoch time in millis which is correct.

Now my question is I have this videos at different time so there could be gap between the videos so instead of player jumping to the next video. I want to customize player and instead lf its own clock, I want to handle clock with my timeline. Timeline is basically a ruler in minutes, hours and seconds. I’m planning to move ruler every millis and based on that player should decode frames and render frames.

Second question, when player doesn’t have frames at time basically a gap between two videos then to notify there is no video for particular time.

@roshanpatel6040 roshanpatel6040 changed the title Clock, rendered and player customization Clock, renderer and player customization Jan 11, 2025
@tonihei
Copy link
Collaborator

tonihei commented Jan 14, 2025

The player automatically synchronizes the video output to the audio position, similar to what you describe.
I don't fully understand the set up you'd like to achieve, but it sounds like the main problem is that we you have multiple video files (or just a single video file) and need to find a way to tell the player which time offset these videos have to your audio clock time? ExoPlayer has a few utility classes to clip and merge sources with optional timestamp adjustments, see https://developer.android.com/media/media3/exoplayer/media-sources#advanced-media-source-composition, which may help.

I want to handle clock with my timeline

If it helps. you can provide your own clock by adding a NoSampleRenderer to the player that also implements MediaClock.

when player doesn’t have frames at time basically a gap between two videos then to notify there is no video for particular time.

Video frames don't necessarily have a duration, so it's unclear when a gap starts in most cases. You could handle it with any logic you like though by adding a FrameMetadataListener to ExoPlayer that gives you a callback for every single frame. In there you could start a timer that notifies a "gap" if there is no further frame within a certain amount of time or so.

@roshanpatel6040
Copy link
Author

roshanpatel6040 commented Jan 15, 2025

There are multiple video files and as per fmp4 structure, it have init.mp4 before segments.

I don’t need to offsets video to audio.

l understood that NoSampleRenderer can be used as clock.

Let me give you an example:

1 video from 12:00:00 am to 12:02:00 am - 2 minute duration
2 video from 12:05:00 am 12:07:00 am - 2 minute duration

As I early mentioned, these video have basemediadecodetime which represents recording time and its in epoch millis. This modification are correct and player currentPosition returns epoch millis as video plays further.

Now, timeline is at 12:00 am, player will continue to play video but when player finished playing first video my ruler is at 12:02:00. Now instead of jumping to the next video, ruler will increment every second untill it reaches 12:05 am and now player should decode frame which is at 12:05:00 and play untill 12:07:00.

I’m looking to customize renderer where player should play frame as per current time of my timeline so when my timeline is between 12:02:00 to 12:05:00, player shoudn’t play next video and instead wait until my timeline reaches to 12:05:00.

My video files are in bytes transmitted over internet and i’m using customized progressive media source.

Note: Timeline basically returns epoch time in millis. It will be incremented every second.

@tonihei
Copy link
Collaborator

tonihei commented Jan 15, 2025

From what you describe, I'd recommend to try and amend the input media according to your needs. The samples inside the fmp4 files have timestamps and they should be set according to your intended playback timestamp. Then I assume you wouldn't need to modify the clock at all.

In any case, this is a very custom solution and we can't help you debug your app in detail I'm afraid. If you think there is some functionality missing that would be good to have in the player, please let us know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants