WIP: Add the VK_EXT_present_timing extension

appendices/VK_EXT_present_timing.adoc

@@ -0,0 +1,236 @@
+// Copyright (c) 2017-2024 Khronos Group.
+//
+// SPDX-License-Identifier: CC-BY-4.0
+
+include::{generated}/meta/{refprefix}VK_EXT_present_timing.adoc[]
+
+=== Other Extension Metadata
+
+*Last Modified Date*::
+    2024-10-09
+*IP Status*::
+    No known IP claims.
+*Contributors*::
+  - Ian Elliott, Google
+  - James Jones, NVIDIA
+  - Jeff Juliano, NVIDIA
+  - Daniel Rakos, AMD
+  - Daniel Stone, Collabora
+  - Daniel Vetter, Intel
+  - Aric Cyr, AMD
+  - Faith Ekstrand, Intel
+  - Nicolai Hähnle, AMD
+  - Alon Or-Bach, Samsung
+  - Niklas Smedberg, Unity Technologies
+  - Tobias Hector, AMD
+  - Lionel Duc, NVIDIA
+  - Lina Versace, Google
+  - Sebastian Wick, Red Hat
+  - Jakob Bornecrantz, Collabora
+
+=== Description
+
+This device extension allows an application that uses the
+`apiext:VK_KHR_swapchain` extension to obtain information about the
+presentation engine's display, to obtain timing information about each
+present, and to schedule a present to happen no earlier than a desired time.
+An application can use this to minimize various visual anomalies (e.g.
+stuttering).
+
+Traditional game and real-time animation applications need to correctly
+position their geometry for when the presentable image will be presented to
+the user.
+To accomplish this, applications need various timing information about the
+presentation engine's display.
+They need to know when presentable images were actually presented, and when
+they could have been presented.
+Applications also need to tell the presentation engine to display an image
+no sooner than a given time.
+This allows the application to avoid stuttering, so the animation looks
+smooth to the user.
+
+include::{generated}/interfaces/VK_EXT_present_timing.adoc[]
+
+=== Issues
+
+1) How does the application determine refresh duration, quanta for change,
+whether FRR vs. VRR, etc.
+
+*PROPOSED*: the query returns two values: 1) a refresh-cycle duration
+(pname:refreshDuration), and 2) an indication whether the timing is
+currently fixed (FRR) or variable (VRR).
+If pname:refreshDuration is zero, the platform cannot supply these
+values until after at least one flink:vkQueuePresentKHR has been done,
+from this time (e.g. if flink:vkQueuePresentKHR has been previously
+called for this swapchain, at least one additional call must be made).
+After calling flink:vkQueuePresentKHR, the query can be repeated until
+pname:refreshDuration is non-zero, at which point the FRR vs. VRR
+indication will also be valid.
+
+If the presentation engine's pname:refreshDuration is a fixed value,
+the application's image present duration (IPD) should be a multiple of
+pname:refreshDuration.
+That is, the quanta for changing the IPD is pname:refreshDuration.
+For example, if pname:refreshDuration is 16.67ms, the IPD can be
+16.67ms, 33.33ms, 50.0ms, etc.
+
+If the presentation engine's pname:refreshDuration is variable,
+pname:refreshDuration is the minimum value of the application's IPD, and
+the IPD can be larger by any quanta that is meaningful to the application.
+For example, if the pname:refreshDuration is 10ms (i.e. the maximum
+refresh rate is 100Hz), the application can choose an IPD of 11ms,
+13.33ms, 13.5ms, or 66.0ms; any value greater than or equal to 10ms is
+valid.
+There may be negative consequences for choosing an IPD that is too
+high, as the presentation engine may actually have a practical maximum
+pname:refreshDuration, where it needs to display the previous image
+again, and during this time the presentation engine might delay
+displaying a newly-presented image.
+
+FRR displays on at least one platform (Wayland) are not necessarily
+fixed; but can change over time.
+For example, if a full-screen video player application is visible, the display
+may operate at a 24Hz refresh cycle; and then later switch to 60Hz when
+multiple windows are visible.
+
+VRR displays on some platforms can also be seen as having different
+characteristics over time.
+For example, if an application's window is full-screen-exclusive (i.e. no other
+window or window system component is visible), the display can look like a VRR
+display (however that is defined).
+If the application's window is not full-screen-exclusive (e.g. a normal
+multi-window case), the display can look like an FRR display (i.e. because the
+compositor is trying to treat all windows in a consistent manner).
+A different issue will deal with how the timing characteristics can change
+over time.
+
+
+2) Do we return min/max values for refresh duration for VRR?
+
+*PROPOSED*: return only the minimum value of refreshDuration for a VRR.
+
+VRR displays have a minimum and maximum refresh rate, and therefore a minimum
+and maximum refreshDuration.
+It has been asserted that the display effectively does not have a minimum
+refresh rate.
+That is because if an application does not present soon enough, the display
+hardware will automatically re-display the previous image.
+However, when the display does that, an application cannot present a new image
+for a certain period of time.
+It is unclear about whether that period is large enough to cause visual
+artifacts.
+
+
+3) How to deal with changes in timing properties?
+
+*RESOLVED*: The slink:VkPastPresentationTimingEXT structure that is
+returned by flink:vkGetPastPresentationTimingEXT contains
+pname:timeDomainChanged, which is ename:VK_TRUE if the time
+domain enabled for the swapchain is not currently available.
+
+An example of why display timing properties can change is if a surface
+changes from being a window that’s a subset of the display size, to
+becoming full-screen-exclusive.
+While the surface was a subset of the display, a compositor might
+enforce fixed timings on the surface (e.g. FRR of 60Hz), where the
+presentation engine might be free to allow VRR behavior of a
+full-screen-exclusive surface.
+
+It is possible that a full-screen-exclusive window can become
+temporarily obscured (e.g. when a short-term dialog pops up).
+In this case, the surface might use FRR timings while the dialog is
+visible and VRR otherwise.
+
+
+4) One Query for all Timing info vs. an initial query to determine FRR vs. VRR,
+and then FRR-specific vs VRR-specific queries?
+
+*RESOLVED*: Have one query, as described in issue 1, that can be
+called whenever the application needs to obtain the timing properties
+of the surface.
+
+
+5) Query to determine time domain?
+
+*RESOLVED*: Have a query to determine the time domain.
+This extension defines a basic swapchain-local time domain.
+Other extensions can add other platform-specific time domains.
+
+
+6) What time to use for targetPresentTime for early images?
+
+*RESOLVED*: Have no query for determining the current time in the PE’s time
+domain; and do allow the special value of zero for targetPresentTime,
+meaning that there is no target.
+
+On some platforms, there is no way to determine the current time, nor
+to determine surface timing properties until after at least one image
+has been presented.
+
+In such cases, the special value of zero allows the application to
+indicate that timing feedback is desired, but that no
+targetPresentTime is requested.
+Later, once the application has obtained feedback, it can specify
+targetPresentTime by using the result's actualPresentTime.
+
+
+7) How long before an application’s request for new image duration is honored?
+
+*UNRESOLVED*: Apparently, changes to some vendors' display hardware settings do
+not take effect immediately.
+It is not clear what settings, and therefore, it is not clear how to
+address this issue.
+
+
+8) Do we have a query for the anticipated latency from present to feedback?
+
+*RESOLVED*: Do not provide a query for the feedback latency.
+
+There is some amount of latency from when an application calls
+vkQueuePresentKHR to when the image is displayed to the user, to when feedback
+is available to the application on when the image was actually displayed to the
+user.
+The first time (from the call till the image is presented) generally doesn’t
+matter, because the application will likely be providing a targetPresentTime
+(i.e. the application may have some indication for how long this will be).
+However, the latency between targetPresentTime until feedback is available may
+be much longer.
+For example, on Android on the 1st-generation Pixel phone (60Hz FRR display),
+the latency was approximately 5 refresh cycles (83.33ms).
+For higher-frequency displays, the latency may have a larger number of refresh
+cycles.
+
+
+9) Do we have a query(s) about the number of VkPastPresentationTimingEXT
+structs to keep?
+
+*RESOLVED*: Do not provide a query for the number of results the swapchain
+is allowed to store before querying them with
+vkGetPastPresentationTimingEXT. Let the application specify that value with
+a dedicated API.
+
+
+10) How is the SWAPCHAIN_LOCAL and STAGE_LOCAL time domain used with the
+calibrated timestamps extension?
+
+*RESOLVED*: Define a struct to chain into
+VkCalibratedTimestampInfoEXT::pNext that specifies a swapchain and present
+stage.
+
+
+11) Should VK_PRESENT_MODE_FIFO_LATEST_READY_EXT be part of this extension,
+or split out into its own extension?
+
+*RESOLVED*: It is only tangentially related.
+Split it out into its own extension and define the interaction here.
+
+=== Version History
+
+  * Revision 1, 2018-05-11 (Ian Elliott)
+  ** Internal revisions.
+
+  * Revision 2, 2022-11-30 (Lionel Duc)
+  ** Rebase for public discussions.
+
+  * Revision 3, 2024-10-09 (Lionel Duc)
+  ** Public revisions.

appendices/glossary.adoc

@@ -770,6 +770,11 @@ Framebuffer Region::
     A framebuffer region is a set of sample (x, y, layer, sample)
     coordinates that is a subset of the entire framebuffer.
 
+ifdef::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
+Frame Rate::
+     A non-Vulkan term for Image Present Rate (IPR).
+endif::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
+
 Front-Facing::
     See Facingness.
 
@@ -862,6 +867,19 @@ Image::
     of device memory.
     Represented by a slink:VkImage object.
 
+ifdef::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
+Image Present Duration::
+    The amount of time the application intends for each
+    newly-presented image to be visible to the user.
+    This value should: be a multiple of the refresh cycle duration.
+
+Image Present Rate::
+    The number of newly-presented images the application intends to present
+    each second (a.k.a. frame rate).
+    On fixed refresh rate displays, this value should: be a multiple of
+    the refresh rate.
+endif::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
+
 Image Subresource::
     A specific mipmap level, layer, and set of aspects of an image.
 
@@ -1068,6 +1086,11 @@ Invocation Repack Instruction::
     implementation may: change the set of invocations that are executing.
 endif::VK_KHR_ray_tracing_pipeline,VK_NV_ray_tracing[]
 
+ifdef::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
+IPD::
+     Image Present Duration.
+endif::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
+
 ifdef::VK_KHR_deferred_host_operations[]
 Join (Deferred Host Operations)::
     The act of instructing a thread to participate in the execution of a
@@ -1596,6 +1619,11 @@ ifdef::VK_KHR_ray_tracing_pipeline[flink:vkCmdTraceRaysKHR, and flink:vkCmdTrace
     .
 endif::VK_KHR_ray_tracing_pipeline,VK_NV_ray_tracing[]
 
+ifdef::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
+RC::
+    Refresh Cycle.
+endif::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
+
 ifdef::VK_KHR_video_queue[]
 Reconstructed Picture::
     A video picture resource reconstructed from a compressed bitstream using
@@ -1618,6 +1646,17 @@ Reference Picture Metadata::
     Opaque state associated with a DPB slot, maintained by a video session.
 endif::VK_KHR_video_queue[]
 
+ifdef::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
+Refresh Cycle::
+    The periodic process for updating the contents of the Presentation Engine's display.
+
+Refresh Cycle Duration::
+    The amount of time from the start of one refresh cycle to the next.
+
+Refresh Rate::
+    The number of refresh cycles per second.
+endif::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
+
 Release Operation (Resource)::
     An operation that releases ownership of an image subresource or buffer
     range.