What is the expected behavior for backgrounded tabs?

[aosmond]

I have a simple test case which just embeds a video, and repeatedly calls requestVideoFrameCallback on the element, which simply dumps the metadata to the console.

In Safari 17.6, when the tab is in the background, it continues to issue the callbacks. There does not appear to be any discontinuity when returning the tab to the foreground.

In Chrome 127, when the tab is in the background, it stops issuing the callbacks. The video timeline continues to advance, so that when one returns the tab to the foreground, the next callback jumps to the present timeline.

Both Safari and Chrome continue issuing callbacks regardless of whether the element is visible or not, as long as the tab is in the foreground.

In Firefox, we don't issue the callbacks if the element is invisible in a foreground tab, but I intend to correct this as that is a bug given both Safari and Chrome issue the callbacks in this scenario. We also don't issue the callbacks if the tab has been backgrounded, because 1) while we advance the video timeline, we do not produce the frames for consumption, and 2) we throttle the requestAnimationFrame callbacks as well, and there are specific expectations about requesting an rAF from an rVFC callback. That I intend to leave as is for now.

Could the specification provide some clarity regarding visibility and backgrounded tabs/windows?

[WesselKroos]

As a webdeveloper that utilizes the rFVC's I would like to additionally ask if it is possible to take into account what influence such an offscreen video's rFVC has on the values in the VideoPlaybackQuality interface.

Because currently in Chromium an offscreen video does still fire rVFC's, but this currently also has the side effect that it increases the getVideoPlaybackQuality().droppedVideoFrames value. Video quality control algorithms use these values to drop the video resolution when required to keep the playback smooth. Because as the Mozilla documentation describes:
"This information (in getVideoPlaybackQuality) can be used to determine whether or not to downgrade the video stream to avoid dropping frames."

So, this behavior in Chromium currently requires me, as a webdeveloper, to write a workaround to prevent the video resolution from dropping (and false-positive reports of playback problems in analytics tools).

A real-world example is YouTube, which drops the video resolution for a long time when this happens. I have reported this issue to Chromium developers, which are currently wondering if this is the expected behavior: https://issues.chromium.org/issues/40727525#comment29
I think it would be beneficial to clarify this behavior in the specification to make this behavior consistent in all browsers.

[chrisn]

We may want to address the integration with VideoPlaybackQuality in https://w3c.github.io/media-playback-quality/ (along with other integration points such as w3c/media-playback-quality#19).