Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve video render algorithm to avoid dropped frames #4549

Closed
wants to merge 1 commit into from

Conversation

MarekLuxoft
Copy link
Collaborator

@MarekLuxoft MarekLuxoft commented Dec 10, 2024

The reason of creating this PR is problem with huge amount of dropped frames observed on refresh rate 120Hz for 60fps with viewport different than 4K. For lower refresh rate and/or different fps problem didn't occur also for viewport 4K we didn't observe issue like that.

Video frames are synced to the audio timestamp. However, the audio timestamp is not queried at a consistent interval. For example, if the query intervals are 16ms, 17ms, 16ms, 17ms, etc., then a 60fps video may display even frames twice and drop odd frames.

To improve video render algorithm to avoid dropped frames we:

  1. Favor advancing the frame sooner. This addresses the situation where the audio timestamp query interval is a little shorter than a frame. These favors displaying the next frame over displaying the current frame twice (or more).
  2. Favor displaying the frame for a little longer. This addresses the situation where the audio timestamp query interval is a little longer than a frame.

Copy link

google-cla bot commented Dec 10, 2024

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@victorpasoshnikov
Copy link
Collaborator

Could you explain why rendering algorithm work correctly in one case (so called "Old TV" and becomes wrong for "New TV" ?
The renderer has 2 queue (audio and video) as input and their timestamps don't depend on anything except ones in the source stream. Why in the second case we get problem? What changed?

@MarekLuxoft
Copy link
Collaborator Author

Could you explain why rendering algorithm work correctly in one case (so called "Old TV" and becomes wrong for "New TV" ?

if we are talking about 120Hz, then it doesn't work correctly for all TV (for viewport 1k, 2k with 60fps), as described here:

huge amount of dropped frames observed on refresh rate 120Hz for 60fps with viewport different than 4K. For lower refresh rate and/or different fps problem didn't occur also for viewport 4K we didn't observe issue like that.

@MarekLuxoft
Copy link
Collaborator Author

During collection data for this PR I notice strange behave around samples. I will temporary close this ticket until I finish investigation this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants