From 9fb6b0a679abedf6ad8c1bd2f124bfa1280cf318 Mon Sep 17 00:00:00 2001 From: "S. Bharadwaj Yadavalli" Date: Fri, 1 Dec 2023 15:06:18 -0500 Subject: [PATCH 01/20] Add Acronym and Glossary to TOC; fix Glossary reference in Introduction. (#130) * Add Acronym and Glossary to TOC; fix Glossary reference in introduction. * Fix wording. --- specs/language/hlsl.tex | 4 ++-- specs/language/introduction.tex | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/specs/language/hlsl.tex b/specs/language/hlsl.tex index 355e5798..4345cfe8 100644 --- a/specs/language/hlsl.tex +++ b/specs/language/hlsl.tex @@ -1,7 +1,8 @@ \documentclass[oneside]{book} \usepackage[utf8]{inputenc} \usepackage[english]{babel} -\usepackage[acronym]{glossaries} +\usepackage{nameref} +\usepackage[acronym,numberedsection=nameref,toc]{glossaries} \usepackage{multicol} \usepackage[marginparwidth=75pt, margin=2cm]{geometry} \usepackage{listings} @@ -78,7 +79,6 @@ \clearpage -\label{glossaries} \printglossary[type=\acronymtype] \printglossary diff --git a/specs/language/introduction.tex b/specs/language/introduction.tex index dfa80ebf..405b546e 100644 --- a/specs/language/introduction.tex +++ b/specs/language/introduction.tex @@ -51,8 +51,8 @@ \p This document aims to use terms consistent with their definitions in \gls{isoC} and \gls{isoCPP}. In cases where the definitions are unclear, or -where this document diverges this section, the remaining sections in this -chapter, and the attached \ref{glossaries}. +where this document diverges, refer to the remaining sections in this +chapter and the attached \nameref{main}. \Sec{Runtime Targeting}{Intro.Runtime} From 05d21c519d08a2e00904901a50e1f6b654f8b486 Mon Sep 17 00:00:00 2001 From: Steven Perron Date: Mon, 4 Dec 2023 09:35:30 -0800 Subject: [PATCH 02/20] Remove statement disallowing unions (#132) This statement explicitly contradicts a later statement that says: > While buffer pointer types are allowed in unions I believe we landed on allowed buffer pointers in unions, so we will keep that statement. --- proposals/0010-vk-buffer-ref.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0010-vk-buffer-ref.md b/proposals/0010-vk-buffer-ref.md index 9f098c26..89e1cd2b 100644 --- a/proposals/0010-vk-buffer-ref.md +++ b/proposals/0010-vk-buffer-ref.md @@ -172,7 +172,7 @@ The pointee of a vk::BufferPointer is considered to be a buffer and will be laid ### Buffer Pointer Usage -vk::BufferPointer cannot be used in Input and Output variables. It also cannot be used in Unions, when those appear in HLSL. +vk::BufferPointer cannot be used in Input and Output variables. A vk::BufferPointer can otherwise be used whereever the HLSL spec does not otherwise disallow it through listing of allowed types. Specifically, buffer members, local and static variables, function argument and return types can be vk::BufferPointer. Ray tracing payloads and shader buffer table records may also contain vk::BufferPointer. From 7e3d166625d5e0fe87d4938f8bfa925c71903994 Mon Sep 17 00:00:00 2001 From: Steven Perron Date: Mon, 4 Dec 2023 10:34:12 -0800 Subject: [PATCH 03/20] Reformat the buffer pointer proposal (#133) --- proposals/0010-vk-buffer-ref.md | 188 +++++++++++++++++++++++--------- 1 file changed, 135 insertions(+), 53 deletions(-) diff --git a/proposals/0010-vk-buffer-ref.md b/proposals/0010-vk-buffer-ref.md index 89e1cd2b..cf1536b7 100644 --- a/proposals/0010-vk-buffer-ref.md +++ b/proposals/0010-vk-buffer-ref.md @@ -1,20 +1,32 @@ + # Buffer Pointers in HLSL With vk::BufferPointer -* Author(s): [Greg Fischer](https://github.com/greg-lunarg) -* Sponsor(s): [Chris Bieneman](https://github.com/llvm-beanz), [Steven Perron](https://github.com/s-perron), [Diego Novillo](https://github.com/dnovillo) -* Status: **Under Review** -* Planned Version: Retroactive addition to Vulkan 1.2 (requires SPIR-V 1.3. Some language details require HLSL 202x +* Author(s): [Greg Fischer](https://github.com/greg-lunarg) +* Sponsor(s): [Chris Bieneman](https://github.com/llvm-beanz), + [Steven Perron](https://github.com/s-perron), + [Diego Novillo](https://github.com/dnovillo) +* Status: **Under Review** +* Planned Version: Retroactive addition to Vulkan 1.2 (requires SPIR-V 1.3. + Some language details require HLSL 202x ## Introduction -This proposal seeks to improve tool support for Vulkan shaders doing buffer device addressing by adding the vk::BufferPointer type to HLSL. +This proposal seeks to improve tool support for Vulkan shaders doing buffer +device addressing by adding the vk::BufferPointer type to HLSL. ## Motivation -vk::RawBufferLoad() and vk::RawBufferStore are currently used to reference physical storage buffer space. Unfortunately, use of these functions has a number of shortcomings. One is that they generate low-level SPIR-V so that tools such as spirv-reflect, spirv-opt and renderdoc do not have the context to analyze and report on which members of a buffer are used in a logical manner. A bigger problem is that the HLSL programmer must compute the physical offsets of the members of a buffer which is error prone and difficult to maintain. +vk::RawBufferLoad() and vk::RawBufferStore are currently used to reference +physical storage buffer space. Unfortunately, use of these functions has a +number of shortcomings. One is that they generate low-level SPIR-V so that tools +such as spirv-reflect, spirv-opt and renderdoc do not have the context to +analyze and report on which members of a buffer are used in a logical manner. A +bigger problem is that the HLSL programmer must compute the physical offsets of +the members of a buffer which is error prone and difficult to maintain. -For example, here is a shader using vk::RawBufferLoad(). Note the physical offset 16 hard-coded into the shader: +For example, here is a shader using vk::RawBufferLoad(). Note the physical +offset 16 hard-coded into the shader: ```c++ // struct GlobalsTest_t @@ -34,24 +46,38 @@ struct TestPushConstant_t float4 MainPs(void) : SV_Target0 { float4 vTest = vk::RawBufferLoad(g_PushConstants.m_nBufferDeviceAddress + 16); - + return vTest; } ``` -The SPIR-V for this shader can be seen in Appendix A. Note the lack of logical context for the accessed buffer i.e. no declaration for the underlying structure GlobalsTest_t as is generated for other buffers. - -There is another way to use RawBufferLoad which does allow logical selection of the buffer fields, but it inefficiently loads the entire buffer to do it. See https://github.com/microsoft/DirectXShaderCompiler/issues/4986. - -The goal of this proposal is to have a solution that meets the following requirements: - -* Removes the need for having to manually or automatically generate offsets to load structured data with BufferDeviceAddress. -* Enables equivalent tooling functionality as is provided by the buffer reference feature in GLSL. Namely, tools like RenderDoc are able to introspect the type information such that its buffer inspection and shader debugger are able to properly understand and represent the type of the data. -* Make it possible through SPIR-V reflection to determine which members of a struct accessed by BufferDeviceAddress are statically referenced and at what offset. This is already possible for other data like cbuffers in order for shader tooling to be able to identify which elements are used and where to put them. +The SPIR-V for this shader can be seen in Appendix A. Note the lack of logical +context for the accessed buffer i.e. no declaration for the underlying structure +GlobalsTest_t as is generated for other buffers. + +There is another way to use RawBufferLoad which does allow logical selection of +the buffer fields, but it inefficiently loads the entire buffer to do it. See +https://github.com/microsoft/DirectXShaderCompiler/issues/4986. + +The goal of this proposal is to have a solution that meets the following +requirements: + +* Removes the need for having to manually or automatically generate offsets to + load structured data with BufferDeviceAddress. +* Enables equivalent tooling functionality as is provided by the buffer + reference feature in GLSL. Namely, tools like RenderDoc are able to + introspect the type information such that its buffer inspection and shader + debugger are able to properly understand and represent the type of the data. +* Make it possible through SPIR-V reflection to determine which members of a + struct accessed by BufferDeviceAddress are statically referenced and at what + offset. This is already possible for other data like cbuffers in order for + shader tooling to be able to identify which elements are used and where to + put them. ## Proposed solution -Our solution is to add a new builtin type in the vk namespace that is a pointer to a buffer of a given type: +Our solution is to add a new builtin type in the vk namespace that is a pointer +to a buffer of a given type: ```c++ template @@ -63,27 +89,48 @@ class vk::BufferPointer { } ``` -This class represents a pointer to a buffer of type struct `S`. `align` is the alignment in bytes of the pointer. If `align` is not specified, the alignment is assumed to be alignof(S). +This class represents a pointer to a buffer of type struct `S`. `align` is the +alignment in bytes of the pointer. If `align` is not specified, the alignment is +assumed to be alignof(S). This new type will have the following operations -* Copy assignment and copy construction - These copy the value of the pointer from one variable to another. -* Dereference Method - The Get() method represents the struct lvalue reference of the pointer to which it is applied. The selection . operator can be applied to the Get() to further select a member from the referenced struct. -* Two new cast operators are introduced. vk::static_pointer_cast allows casting any vk::BufferPointer to vk::BufferPointer only if SrcType is a type derived from DstType. vk::reinterpret_pointer_cast allows casting for all other BufferPointer types. For both casts, DstAlign <= SrcAlign must be true. -* A buffer pointer can be constructed from a uint64_t u using the constructor syntax vk::BufferPointer(u). -* A buffer pointer can be cast to a bool. If so, it returns FALSE if the pointer is null, TRUE otherwise. +* Copy assignment and copy construction - These copy the value of the pointer + from one variable to another. +* Dereference Method - The Get() method represents the struct lvalue reference + of the pointer to which it is applied. The selection . operator can be + applied to the Get() to further select a member from the referenced struct. +* Two new cast operators are introduced. vk::static_pointer_cast allows + casting any vk::BufferPointer to + vk::BufferPointer only if SrcType is a type derived from + DstType. vk::reinterpret_pointer_cast allows casting for all other + BufferPointer types. For both casts, DstAlign <= SrcAlign must be true. +* A buffer pointer can be constructed from a uint64_t u using the constructor + syntax vk::BufferPointer(u). +* A buffer pointer can be cast to a bool. If so, it returns FALSE if the + pointer is null, TRUE otherwise. Note the operations that are not allowed: -* There is no default construction. Every vk::BufferPointer is either contained in a global resource (like a cbuffer, ubo, or ssbo), or it must be constructed using the copy constructor. -* There is no explicit pointer arithmetic. All addressing is implicitly done using the `.` operator, or indexing an array in the struct T. -* The comparison operators == and != are not supported for buffer pointers. +* There is no default construction. Every vk::BufferPointer is either + contained in a global resource (like a cbuffer, ubo, or ssbo), or it must be + constructed using the copy constructor. +* There is no explicit pointer arithmetic. All addressing is implicitly done + using the `.` operator, or indexing an array in the struct T. +* The comparison operators == and != are not supported for buffer pointers. -Most of these restrictions are there for safety. They minimize the possibility of getting an invalid pointer. If the Get() method is used on a null or invalid pointer, the behaviour is undefined. +Most of these restrictions are there for safety. They minimize the possibility +of getting an invalid pointer. If the Get() method is used on a null or invalid +pointer, the behaviour is undefined. -When used as a member in a buffer, vk::BufferPointer can be used to pass physical buffer addresses into a shader, and address and access buffer space with logical addressing, which allows tools such as spirv-opt, spirv-reflect and renderdoc to be able to better work with these shaders. +When used as a member in a buffer, vk::BufferPointer can be used to pass +physical buffer addresses into a shader, and address and access buffer space +with logical addressing, which allows tools such as spirv-opt, spirv-reflect and +renderdoc to be able to better work with these shaders. -For example, here is a shader using vk::BufferPointer to do the same thing as the shader above using vk::RawBufferLoad. Note the natural, logical syntax of the reference: +For example, here is a shader using vk::BufferPointer to do the same thing as +the shader above using vk::RawBufferLoad. Note the natural, logical syntax of +the reference: ```c++ @@ -111,7 +158,11 @@ float4 MainPs(void) : SV_Target0 ``` -In SPIR-V, Globals_p would be a pointer to the physical buffer storage class. The struct type of the push constant would contain one of those pointers. The SPIR-V for this shader can be seen in Appendix B. Note the logical context of the declaration and addressing of underlying struct Globals_s including Offset decorations all Globals_s members. +In SPIR-V, Globals_p would be a pointer to the physical buffer storage class. +The struct type of the push constant would contain one of those pointers. The +SPIR-V for this shader can be seen in Appendix B. Note the logical context of +the declaration and addressing of underlying struct Globals_s including Offset +decorations all Globals_s members. ## Linked Lists and Local Variables @@ -146,69 +197,101 @@ float4 MainPs(void) : SV_Target0 ``` -Note also the ability to create local variables of type vk::BufferPointer such as g_p which can be read, written and dereferenced. +Note also the ability to create local variables of type vk::BufferPointer such +as g_p which can be read, written and dereferenced. ## Design Details ### Writing Buffer Pointer Pointees -The pointees of vk::BufferPointer objects can be written as well as read. See Appendix C for example HLSL. See Appendix D for the SPIR-V. +The pointees of vk::BufferPointer objects can be written as well as read. See +Appendix C for example HLSL. See Appendix D for the SPIR-V. ### Differences from C++ Pointers -vk::BufferPointer is different from a C++ pointer in that the method Get() can and must be applied to de-reference it. +vk::BufferPointer is different from a C++ pointer in that the method Get() can +and must be applied to de-reference it. ### Buffer Pointer Target Alignment -The target alignment `A` of `vk::BufferPointer` must be at least as large as the largest component type in the buffer pointer's pointee struct type `T` or the compiler may issue an error. +The target alignment `A` of `vk::BufferPointer` must be at least as large +as the largest component type in the buffer pointer's pointee struct type `T` or +the compiler may issue an error. ### Buffer Pointer Data Size and Alignment -For the purpose of laying out a buffer containing a vk::BufferPointer, the data size and alignment is that of a uint64_t. +For the purpose of laying out a buffer containing a vk::BufferPointer, the data +size and alignment is that of a uint64_t. ### Buffer Pointer Pointee Buffer Layout -The pointee of a vk::BufferPointer is considered to be a buffer and will be laid out as the user directs all buffers to be laid out through the dxc compiler. All layouts that are supported by dxc are supported for vk::BufferPointer pointee buffers. +The pointee of a vk::BufferPointer is considered to be a buffer and will be laid +out as the user directs all buffers to be laid out through the dxc compiler. All +layouts that are supported by dxc are supported for vk::BufferPointer pointee +buffers. ### Buffer Pointer Usage vk::BufferPointer cannot be used in Input and Output variables. -A vk::BufferPointer can otherwise be used whereever the HLSL spec does not otherwise disallow it through listing of allowed types. Specifically, buffer members, local and static variables, function argument and return types can be vk::BufferPointer. Ray tracing payloads and shader buffer table records may also contain vk::BufferPointer. +A vk::BufferPointer can otherwise be used whereever the HLSL spec does not +otherwise disallow it through listing of allowed types. Specifically, buffer +members, local and static variables, function argument and return types can be +vk::BufferPointer. Ray tracing payloads and shader buffer table records may also +contain vk::BufferPointer. ### Buffer Pointer and Semantic Annotations -Applying HLSL semantic annotations to objects of type vk::BufferPointer is disallowed. +Applying HLSL semantic annotations to objects of type vk::BufferPointer is +disallowed. ### Buffer Pointers and Aliasing -By default, buffer pointers are assumed to be restrict pointers as defined by the C99 standard. +By default, buffer pointers are assumed to be restrict pointers as defined by +the C99 standard. -An attribute vk::aliased_pointer can be attached to a variable, function parameter or a struct member of BufferPointer type. It is assumed that the pointee of a BufferPointer with this attribute can overlap with the pointee of any other BufferPointer with this attribute if they have the same pointee type and their scopes intersect. This also means that the pointee of a BufferPointer with this attribute does not overlap with the pointee of a default (restrict) BufferPointer. +An attribute vk::aliased_pointer can be attached to a variable, function +parameter or a struct member of BufferPointer type. It is assumed that the +pointee of a BufferPointer with this attribute can overlap with the pointee of +any other BufferPointer with this attribute if they have the same pointee type +and their scopes intersect. This also means that the pointee of a BufferPointer +with this attribute does not overlap with the pointee of a default (restrict) +BufferPointer. -The result of vk::static_pointer_cast and vk::reinterpret_pointer_cast as well as all constructors is restrict. +The result of vk::static_pointer_cast and vk::reinterpret_pointer_cast as well +as all constructors is restrict. -A pointer value can be assigned to a variable, function parameter or struct member entity, even if the aliasing disagrees. Such an assignment is an implicit cast of this property. +A pointer value can be assigned to a variable, function parameter or struct +member entity, even if the aliasing disagrees. Such an assignment is an implicit +cast of this property. See Appendix E for example of aliasing casting. ### Buffer Pointers and Address Space -All buffer pointers are presumed to point into the buffer device address space as defined by the Vulkan type VkDeviceAddress. See the following link for additional detail: https://registry.khronos.org/vulkan/specs/1.3-khr-extensions/html/vkspec.html#VkDeviceAddress. +All buffer pointers are presumed to point into the buffer device address space +as defined by the Vulkan type VkDeviceAddress. See the following link for +additional detail: +https://registry.khronos.org/vulkan/specs/1.3-khr-extensions/html/vkspec.html#VkDeviceAddress. ### Buffer Pointer Availability -The following can be used at pre-processor time to determine if the current compiler supports vk::BufferPointer: __has_feature(hlsl_vk_buffer_pointer). +The following can be used at pre-processor time to determine if the current +compiler supports vk::BufferPointer: __has_feature(hlsl_vk_buffer_pointer). ### Buffer Pointers and Type Punning Through Unions -While buffer pointer types are allowed in unions, type punning with buffer pointer types is disallowed as it is with all other types in HLSL. Specifically, when a member of a union is defined, all other members become undefined, no matter the types. +While buffer pointer types are allowed in unions, type punning with buffer +pointer types is disallowed as it is with all other types in HLSL. Specifically, +when a member of a union is defined, all other members become undefined, no +matter the types. ## SPIR-V Appendices ### Appendix A: SPIR-V for RawBufferLoad -Note the lack of logical context for the accessed buffer i.e. no declaration for the underlying structure GlobalsTest_t as is generated for other buffers. +Note the lack of logical context for the accessed buffer i.e. no declaration for +the underlying structure GlobalsTest_t as is generated for other buffers. ``` @@ -264,7 +347,9 @@ Note the lack of logical context for the accessed buffer i.e. no declaration for ### Appendix B: SPIR-V for vk::buffer_ref -Here is the SPIR-V for this shader. Note the logical context of the declaration and addressing of underlying struct Globals_s including Offset decorations all Globals_s members: +Here is the SPIR-V for this shader. Note the logical context of the declaration +and addressing of underlying struct Globals_s including Offset decorations all +Globals_s members: ``` OpCapability Shader @@ -321,7 +406,6 @@ Here is the SPIR-V for this shader. Note the logical context of the declaration ### Appendix C: HLSL for Write through vk::BufferPointer - ```c++ struct Globals_s @@ -351,15 +435,14 @@ float4 MainPs(void) : SV_Target0 ### Appendix D: SPIR-V for Write through vk::BufferPointer - ``` OpCapability Shader OpCapability PhysicalStorageBufferAddresses OpExtension "SPV_KHR_physical_storage_buffer" OpMemoryModel PhysicalStorageBuffer64 GLSL450 OpEntryPoint Fragment %MainPs "MainPs" %out_var_SV_Target0 %g_PushConstants - OpExecutionMode %MainPs OriginUpperLeft - OpSource HLSL 600 + OpExecutionMode %MainPs OriginUpperLeft + OpSource HLSL 600 OpName %type_PushConstant_TestPushConstant_t "type.PushConstant.TestPushConstant_t" OpMemberName %type_PushConstant_TestPushConstant_t 0 "m_nBufferDeviceAddress" OpName %Globals_s "Globals_s" @@ -410,7 +493,6 @@ float4 MainPs(void) : SV_Target0 ### Appendix E: HLSL for Implicit Cast of Restrict to Aliased - ```c++ struct Globals_s From d657efcb5d573bafc89c3cad616b90b63a338c2e Mon Sep 17 00:00:00 2001 From: Greg Fischer Date: Thu, 7 Dec 2023 09:03:23 -0700 Subject: [PATCH 04/20] [0010] Update linked list example to match latest rules (#108) --- proposals/0010-vk-buffer-ref.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/0010-vk-buffer-ref.md b/proposals/0010-vk-buffer-ref.md index cf1536b7..34991f22 100644 --- a/proposals/0010-vk-buffer-ref.md +++ b/proposals/0010-vk-buffer-ref.md @@ -189,9 +189,10 @@ struct TestPushConstant_t float4 MainPs(void) : SV_Target0 { - block_p g_p = g_PushConstants.root; + block_p g_p(g_PushConstants.root); g_p = g_p.Get().next; - if (uint64_t(g_p) == 0) return float4(0.0,0.0,0.0,0.0); + if (!(bool)g_p) // Null pointer test + return float4(0.0,0.0,0.0,0.0); return g_p.Get().x } From 7098ff0f420be4ca676dd3dff058370d5d2cccaa Mon Sep 17 00:00:00 2001 From: Steven Perron Date: Fri, 8 Dec 2023 12:10:09 -0800 Subject: [PATCH 05/20] [0010] Allow casting to uint64_t (#134) * [0010] Allow casting to uint64_t We have decided to allow casting to uint64_t. It will make create more opportunities for developers to make mistakes, but they do not have to use it. The implementation cost does not seem too high. Fixes https://github.com/microsoft/hlsl-specs/issues/93 * Remove cast to bool As Greg suggested, the cast to bool is no longer needed if we can cast to an int, so we removed it. This commit also adds the uint64_t cast to the pseudo class definition for BufferPointers. * Fix linked list example after rebaseing. --- proposals/0010-vk-buffer-ref.md | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/proposals/0010-vk-buffer-ref.md b/proposals/0010-vk-buffer-ref.md index 34991f22..9c083c3e 100644 --- a/proposals/0010-vk-buffer-ref.md +++ b/proposals/0010-vk-buffer-ref.md @@ -86,6 +86,7 @@ class vk::BufferPointer { vk::BufferPointer& operator=(const vk::BufferPointer&); vk::BufferPointer(const uint64_t); S& Get() const; + operator uint64_t() const; } ``` @@ -105,10 +106,10 @@ This new type will have the following operations vk::BufferPointer only if SrcType is a type derived from DstType. vk::reinterpret_pointer_cast allows casting for all other BufferPointer types. For both casts, DstAlign <= SrcAlign must be true. -* A buffer pointer can be constructed from a uint64_t u using the constructor +* A buffer pointer can be constructed from a uint64_t using the constructor syntax vk::BufferPointer(u). -* A buffer pointer can be cast to a bool. If so, it returns FALSE if the - pointer is null, TRUE otherwise. +* A buffer pointer can be cast to a uint64_t. The cast will return the 64-bit + address that the pointer points to. Note the operations that are not allowed: @@ -120,8 +121,12 @@ Note the operations that are not allowed: * The comparison operators == and != are not supported for buffer pointers. Most of these restrictions are there for safety. They minimize the possibility -of getting an invalid pointer. If the Get() method is used on a null or invalid -pointer, the behaviour is undefined. +of getting an invalid pointer. If a buffer pointer is cast to and from a +uint64_t, then it is the responsibility of the user to make sure that a valid +pointer is generated, and that aliasing rules are followed. + +If the Get() method is used on a null or invalid pointer, the behaviour is +undefined. When used as a member in a buffer, vk::BufferPointer can be used to pass physical buffer addresses into a shader, and address and access buffer space @@ -191,7 +196,7 @@ float4 MainPs(void) : SV_Target0 { block_p g_p(g_PushConstants.root); g_p = g_p.Get().next; - if (!(bool)g_p) // Null pointer test + if ((uint64_t)g_pi == 0) // Null pointer test return float4(0.0,0.0,0.0,0.0); return g_p.Get().x } From 1315feb5ea1234752b35106366284afe17b3e49e Mon Sep 17 00:00:00 2001 From: Chris B Date: Wed, 13 Dec 2023 15:18:33 -0600 Subject: [PATCH 06/20] Add stub grammar definitions and sections (#138) These are just empty placeholder sections with some grammar definitions to fill in the ID expression grammars. Properly documenting this will require going back and defining scopes and other lexical constructs. Since all of this is more or less inherited from C/C++ I'm going to leave them as stubs for now and focus on the HLSL-specific sections. --- specs/language/expressions.tex | 37 ++++++++++++++++++++++++++++++++++ specs/language/hlsl.tex | 13 ++++++++++-- specs/language/macros.tex | 1 + 3 files changed, 49 insertions(+), 2 deletions(-) diff --git a/specs/language/expressions.tex b/specs/language/expressions.tex index 007b5296..b1308214 100644 --- a/specs/language/expressions.tex +++ b/specs/language/expressions.tex @@ -96,3 +96,40 @@ and value category as \textit{E} without the enclosing parenthesis. A parenthesized expression may be used in the same contexts with the same meaning as the same non-parenthesized expression. + +\Sub{Names}{Expr.Primary.ID} + +\begin{note} + The grammar and behaviors of this section are almost identical to C/C++ with + some subtractions (notably lambdas and destructors). +\end{note} + +\begin{grammar} + \define{id-expression}\br + unqualified-id\br + qualified-id +\end{grammar} + +\SubSub{Unqualified Identifiers}{Expr.Primary.ID.Unqual} + +\begin{grammar} + \define{unqualified-id}\br + identifier\br + operator-function-id\br + conversion-function-id\br + template-id\br +\end{grammar} + +\SubSub{Qualified Identifiers}{Expr.Primary.ID.Qual} + +\begin{grammar} + \define{qualified-id}\br + nested-name-specifier \opt{\keyword{template}} unqualified-id\br + \define{nested-name-specifier}\br + \terminal{::}\br + type-name \terminal{::}\br + namespace-name \terminal{::}\br + nested-name-specifier identifier \terminal{::}\br + nested-name-specifier \opt{\keyword{template}} simple-template-id \terminal{::} +\end{grammar} + diff --git a/specs/language/hlsl.tex b/specs/language/hlsl.tex index 4345cfe8..2d0faa5f 100644 --- a/specs/language/hlsl.tex +++ b/specs/language/hlsl.tex @@ -18,9 +18,11 @@ \renewcommand{\familydefault}{\sfdefault} +\setcounter{secnumdepth}{3} % number subsections \newcommand{\Ch}[2]{\chapter[#1]{#1\hfill[#2]}\label{#2}} \newcommand{\Sec}[2]{\section[#1]{#1\hfill[#2]}\label{#2}} \newcommand{\Sub}[2]{\subsection[#1]{#1\hfill[#2]}\label{#2}} +\newcommand{\SubSub}[2]{\subsubsection[#1]{#1\hfill[#2]}\label{#2}} \input{glossary} @@ -50,8 +52,15 @@ \setlength\parindent{0cm} -\newcounter{parcount} -\counterwithin{parcount}{subsection} +\newcommand{\newparcounter}[1]{ +\newcounter{#1} +\counterwithin{#1}{chapter} +\counterwithin{#1}{section} +\counterwithin{#1}{subsection} +\counterwithin{#1}{subsubsection} +} + +\newparcounter{parcount} \newcommand\p{% \stepcounter{parcount}% \parnum \hspace{1em}% diff --git a/specs/language/macros.tex b/specs/language/macros.tex index 22fb97ef..236d8887 100644 --- a/specs/language/macros.tex +++ b/specs/language/macros.tex @@ -19,6 +19,7 @@ \newcommand{\define}[1]{{\textit{##1}\textnormal{:}}} \newcommand{\terminal}[1]{{\texttt{##1}}} \newcommand{\br}{\hfill\\*} + \newcommand{\opt}[1]{{##1\textsubscript{\textit{opt}}}} \renewcommand{\texttt}[1]{{\small\ttfamily\upshape ##1}} From e64a254a4107d4f476abfceb963a88150de6c152 Mon Sep 17 00:00:00 2001 From: Chris B Date: Wed, 13 Dec 2023 16:50:50 -0600 Subject: [PATCH 07/20] [Expr] Add Expr.Post.Subscript (#139) This adds definitions for HLSL array subscripting. HLSL deviates from C/C++ in part because of the lack of pointers. --- specs/language/expressions.tex | 27 +++++++++++++++++++++++++++ specs/language/placeholders.tex | 1 + 2 files changed, 28 insertions(+) diff --git a/specs/language/expressions.tex b/specs/language/expressions.tex index b1308214..c388d6fb 100644 --- a/specs/language/expressions.tex +++ b/specs/language/expressions.tex @@ -133,3 +133,30 @@ nested-name-specifier \opt{\keyword{template}} simple-template-id \terminal{::} \end{grammar} +\Sec{Postfix Expressions}{Expr.Post} + +\begin{grammar} + \define{postfix-expression}\br + primary-expression\br + postfix-expression \terminal{[} expression \terminal{]}\br + postfix-expression \terminal{[} braced-init-list \terminal{]}\br + postfix-expression \terminal{(} \opt{expression-list} \terminal{)}\br + simple-type-specifier \terminal{(} \opt{expression-list} \terminal{)}\br + typename-specifier \terminal{(} \opt{expression} \terminal{)}\br + simple-type-specifier braced-init-list\br + typename-specifier braced-init-list\br + postfix-expression \terminal{.} \opt{\terminal{template}} id-expression\br + postfix-expression \terminal{->} \opt{\terminal{template}} id-expression\br + postfix-expression \terminal{++}\br + postfix-expression \terminal{--} +\end{grammar} + +\Sec{Subscript}{Expr.Post.Subscript} + +\p A \textit{postfix-expression} followed by an expression in square brackets +(\texttt{[ ]}) is a subscript expression. In an array subscript expression of +the form \texttt{E1[E2]}, \texttt{E1} must either be a variable of array of +\texttt{T[]}, or an object of type \texttt{T} where \texttt{T} provides an +overloaded implementation of \texttt{operator[]} (\ref{Overload}).\footnote{HLSL +does not support the base address of a subscript operator being the expression +inside the braces, which is valid in C and C++.} diff --git a/specs/language/placeholders.tex b/specs/language/placeholders.tex index b51bca4b..be3ecd52 100644 --- a/specs/language/placeholders.tex +++ b/specs/language/placeholders.tex @@ -1,4 +1,5 @@ \Ch{Declarations}{Decl} \Sec{Attributes}{Decl.Attr} \Sub{Entry Attributes}{Decl.Attr.Entry} +\Ch{Overloading}{Overload} \Ch{Runtime}{Runtime} From 07f16ba8e59eda53059bcc3a063eab2b05453252 Mon Sep 17 00:00:00 2001 From: Chris B Date: Thu, 14 Dec 2023 16:17:11 -0600 Subject: [PATCH 08/20] Fix failing GitHub action to deploy docs (#142) Our doc build started failing due to a change in the azure images. The failure is the same as what is reported in this issue: https://github.com/actions/runner-images/issues/7603. The fix described there is to call `sudo apt-get update`, which is the change here. --- .github/workflows/jekyll-gh-pages.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/jekyll-gh-pages.yml b/.github/workflows/jekyll-gh-pages.yml index 302b6886..3610d0bb 100644 --- a/.github/workflows/jekyll-gh-pages.yml +++ b/.github/workflows/jekyll-gh-pages.yml @@ -31,6 +31,7 @@ jobs: uses: actions/configure-pages@v2 - name: Install dependencies run: | + sudo apt-get update sudo apt -y install texlive sudo apt -y install texlive-latex-extra curl -fsSL https://github.com/jgm/pandoc/releases/download/3.1.9/pandoc-3.1.9-1-amd64.deb -o pandoc.deb From 802d4568e5d2690e2675d5d2eeb7e219e1f5bc9f Mon Sep 17 00:00:00 2001 From: Chris B Date: Thu, 14 Dec 2023 17:15:37 -0600 Subject: [PATCH 09/20] Fix Pandoc build (#144) For some reason pandoc doesn't like this formatting. --- specs/language/expressions.tex | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/specs/language/expressions.tex b/specs/language/expressions.tex index c388d6fb..52655199 100644 --- a/specs/language/expressions.tex +++ b/specs/language/expressions.tex @@ -138,8 +138,10 @@ \begin{grammar} \define{postfix-expression}\br primary-expression\br - postfix-expression \terminal{[} expression \terminal{]}\br - postfix-expression \terminal{[} braced-init-list \terminal{]}\br + % The [] characters on the two lines below should be inside \terminal, however + % pandoc doesn't seem to like that. + postfix-expression [ expression ]\br + postfix-expression [ braced-init-list ]\br % postfix-expression \terminal{(} \opt{expression-list} \terminal{)}\br simple-type-specifier \terminal{(} \opt{expression-list} \terminal{)}\br typename-specifier \terminal{(} \opt{expression} \terminal{)}\br From 86a90d7aa17b9a3dd823161a4c5378eb1313d34b Mon Sep 17 00:00:00 2001 From: Chris B Date: Mon, 18 Dec 2023 11:44:43 -0600 Subject: [PATCH 10/20] [Expr] Add Expr.Post.Call (#140) This section specifies how HLSL function calls work as well as how argument and parameters are represented, their lifetimes and semantics. Co-authored-by: S. Bharadwaj Yadavalli --- specs/language/expressions.tex | 84 +++++++++++++++++++++++++++++++++ specs/language/placeholders.tex | 1 + 2 files changed, 85 insertions(+) diff --git a/specs/language/expressions.tex b/specs/language/expressions.tex index 52655199..015464e9 100644 --- a/specs/language/expressions.tex +++ b/specs/language/expressions.tex @@ -162,3 +162,87 @@ overloaded implementation of \texttt{operator[]} (\ref{Overload}).\footnote{HLSL does not support the base address of a subscript operator being the expression inside the braces, which is valid in C and C++.} + +\Sec{Function Calls}{Expr.Post.Call} + +\p A function call may be an \textit{ordinary function}, or a \textit{member +function}. In a function call to an \textit{ordinary function}, the +\textit{postfix-expression} must be an lvalue that refers to a function. In a +function call to a \textit{member function}, the \textit{postfix-expression} +will be an implicit or explicit class member access whose \textit{id-expression} +is a member function name. + +\p When a function is called, each parameter shall be initialized with its +corresponding argument. The order in which parameters are initialized is +unspecified. \footnote{Today in DXC targeting DXIL matches the Microsoft C++ ABI +and evaluates argument expressions right-to-left, while SPIR-V generation +matches the Itanium ABI evaluating parameters left-to-right. There are good +arguments for unifying these behaviors, and arguments for keeping them +different.} + +\p If the function is a non-static member function the \texttt{this} argument +shall be initialized to a reference to the object of the call as if casted by an +explicit cast expression to an lvalue reference of the type that the function is +declared as a member of. + +\p Parameters are either \textit{input parameters}, \textit{output parameters}, +or \textit{input/output parameters} as denoted in the called function's +declaration (\ref{Decl.Function}). + +\p \textit{Input parameters} are passed by-value into a function. If an argument +to an \textit{input parameter} is of constant-sized array type, the array is +copied to a temporary and the temporary value is converted to an address via +array-to-pointer decay. If an argument is an unsized array type, the array +lvalue directly decays via array-to-pointer decay. \footnote{This results in +\textit{input} parameters of unsized arrays being modifiable by a function.} + +\p Arguments to \textit{output} and \textit{input/output parameters} must be +lvalues. \textit{Output parameters} are not initialized prior to the call; they +are passed as an uninitialized cxvalue (\ref{Basic.lval}). An \textit{output +parameter} is only initialized explicitly inside the called function. It is +undefined behavior to not explicitly initialize an \textit{output parameter} +before returning from the function in which it is defined. The cxvalue created +from an argument to an \textit{input/output parameter} is initialized through +copy-initialization from the lvalue argument expression. In both cases, the +cxvalue shall have the type of the parameter and the argument can be converted +to that type through implicit or explicit conversion. + +\p If an argument to an \textit{output} or \textit{input/output parameter} is a +constant sized array, the array is copied to a temporary cxvalue following the +same rules for any other data type. If an argument to an \textit{output} or +\textit{input/output parameter} is an unsized array type, the array lvalue +directly decays via array-to-pointer decay. An argument of a constant sized +array of type \texttt{T[N]} can be converted to a cxvalue of an unsized array +of type \texttt{T[]} through array to pointer decay. An unsized array of type +\texttt{T[]}, cannot be implicitly converted to a a constant sized array of type +\texttt{T[N]}. + +\p On expiration of the cxvalue, the value is assigned back to the argument +lvalue expression following an inverted conversion if applicable. The argument +expression must be of a type or able to convert to a type that has defined +copy-initialization to and from the parameter type. The lifetime of the cxvalue +begins at argument expression evaluation, and ends after the function returns. A +cxvalue argument is passed by-address to the caller. + +\p If the lvalue passed to an \textit{output} or \textit{input/output parameter} +does not alias any other parameter passed to that function, an implementation +may avoid the creation of excess temporaries by passing the address of the +lvalue instead of creating the cxvalue. + +\p When a function is called, any parameter of object type must have completely +defined type, and any parameter of array of object type must have completely +defined element type.\footnote{HLSL \textit{output} and \textit{input/output +parameters} are passed by value, so they must also have complete type.} The +lifetime of a parameter ends on return of the function in which it is +defined.\footnote{As stated above cxvalue parameters are passed-by-address, so +the expiring parameter is the reference to the address, not the cxvalue. The +cxvalue expires in the caller.} Initialization and destruction of each +parameter occurs within the context of the calling function. + +\p The value of a function call is the value returned by the called function. + +\p A function call is an lvalue if the result type is an lvalue reference type; +otherwise it is a prvalue. + +\p If a function call is a prvalue of object type, the type of the prvalue must +be complete. diff --git a/specs/language/placeholders.tex b/specs/language/placeholders.tex index be3ecd52..3b8e2721 100644 --- a/specs/language/placeholders.tex +++ b/specs/language/placeholders.tex @@ -1,4 +1,5 @@ \Ch{Declarations}{Decl} +\Sec{Function Definitions}{Decl.Function} \Sec{Attributes}{Decl.Attr} \Sub{Entry Attributes}{Decl.Attr.Entry} \Ch{Overloading}{Overload} From 19491266edee2839d59c2d6e5d525886e2c5323c Mon Sep 17 00:00:00 2001 From: Chris B Date: Mon, 18 Dec 2023 12:02:16 -0600 Subject: [PATCH 11/20] Explicitly state lvalue-reference semantics (#136) This clarifies any ambiguity around atomic operations. Fixes #88 --- proposals/0010-vk-buffer-ref.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/proposals/0010-vk-buffer-ref.md b/proposals/0010-vk-buffer-ref.md index 9c083c3e..c1abf158 100644 --- a/proposals/0010-vk-buffer-ref.md +++ b/proposals/0010-vk-buffer-ref.md @@ -101,6 +101,9 @@ This new type will have the following operations * Dereference Method - The Get() method represents the struct lvalue reference of the pointer to which it is applied. The selection . operator can be applied to the Get() to further select a member from the referenced struct. + The reference returned by the Get() method is supported in all APIs that + take reference, `inout` or `out` parameters, and can be converted to an + rvalue following standard conversion rules. * Two new cast operators are introduced. vk::static_pointer_cast allows casting any vk::BufferPointer to vk::BufferPointer only if SrcType is a type derived from From 25d2e8852c09099bf0b2ac393c0cede8f6843174 Mon Sep 17 00:00:00 2001 From: Chris B Date: Tue, 9 Jan 2024 12:30:50 -0600 Subject: [PATCH 12/20] Add pre-merge build (#145) There is a separate artifact containing the PDF build to make it easier to grab the PDF. --- .github/workflows/jekyll-gh-pages.yml | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/.github/workflows/jekyll-gh-pages.yml b/.github/workflows/jekyll-gh-pages.yml index 3610d0bb..7a749646 100644 --- a/.github/workflows/jekyll-gh-pages.yml +++ b/.github/workflows/jekyll-gh-pages.yml @@ -5,6 +5,10 @@ on: # Runs on pushes targeting the default branch push: branches: ["main"] + pull_request: + types: [opened,synchronize] + paths: + - specs/language/** # Allows you to run this workflow manually from the Actions tab workflow_dispatch: @@ -50,9 +54,15 @@ jobs: destination: ./_site - name: Upload artifact uses: actions/upload-pages-artifact@v1 + - if: ${{ github.event_name == 'pull_request'}} + uses: actions/upload-artifact@v4 + with: + name: PDF + path: ${{github.workspace}}/specs/hlsl.pdf # Deployment job deploy: + if: ${{ github.event_name == 'push'}} environment: name: github-pages url: ${{ steps.deployment.outputs.page_url }} From db6623f9bcb37e18b7d791be2b9185e502947891 Mon Sep 17 00:00:00 2001 From: Chris B Date: Tue, 9 Jan 2024 16:33:44 -0600 Subject: [PATCH 13/20] Update process for infrastructural proposals (#131) This updates the process docmentation and adds a new template for describing infrastructural proposals. This will allow us to use this process format for desigining project infrastructure and implementation details within the tooling that may not have language or other user exposure. --- docs/Process.md | 32 +++++--- proposals/infra/index.md | 8 ++ .../templates/infrastructure-template.md | 76 +++++++++++++++++++ 3 files changed, 106 insertions(+), 10 deletions(-) create mode 100644 proposals/infra/index.md create mode 100644 proposals/templates/infrastructure-template.md diff --git a/docs/Process.md b/docs/Process.md index 7d2f2c42..428b1ba0 100644 --- a/docs/Process.md +++ b/docs/Process.md @@ -26,18 +26,13 @@ This process draws heavily from [Swift's Evolution process](https://github.com/apple/swift-evolution/), and is further tweaked to align with the HLSL team's goals and priorities. -## Proposing a Feature +Significant project infrastructure or implementation details will also use this +process to refine and document the design process. -By far the best way for an external contributor to propose a feature is through -GitHub issues (See the section below on "Filing Issues"). Issues in this -repository will be used to publicly track feature requests for the HLSL language -and HLSL runtime interfaces. Direct tooling feature requests to the -[DirectXShaderCompiler](https://github.com/microsoft/DirectXShaderCompiler/issues/new). +## Making a Proposal -> Note: a tooling feature would be a feature that does not impact HLSL source -> representations in any way (no added syntax, APIs, or altered -> interpretations), and instead exposes new ways to use the DXC compiler or -> library. +The best way for an external contributor to propose a feature is through GitHub +issues (See the section below on "Filing Issues"). If you write a proposal yourself you must find a member of the HLSL team to act as a _Sponsor_ for the change. The _Sponsor_ is responsible for tracking and @@ -58,6 +53,10 @@ HLSL [Design Considerations](DesignConsiderations.md). Draft proposals are first provided as pull requests. They should be written following one of the templates in the `proposals/templates` directory. +Add new proposals for language or runtime features directly in the `proposals` +directory. Add new proposals for project infrastructure or implementation +details of the compilers in the `proposals/infra` directory. + Proposals that follow the most simplified path from idea to feature will move through the following states in order: @@ -94,6 +93,19 @@ requirements for justification are not high and could be as simple as ## Filing Issues +Issues in this repository publicly tracks feature requests for the HLSL language +and HLSL runtime interfaces as well as issues with proposals and specifications +contained within the repository. + +Please direct tooling feature requests to the +[DirectXShaderCompiler](https://github.com/microsoft/DirectXShaderCompiler/issues/new), +or [Clang](https://github.com/llvm/llvm-project/issues/new) as appropriate. + +> Note: a tooling feature would be a feature that does not impact HLSL source +> representations in any way (no added syntax, APIs, or altered +> interpretations), and instead exposes new ways to use the DXC compiler or +> library. + This repository provides three custom issue templates: 1. Feature Request diff --git a/proposals/infra/index.md b/proposals/infra/index.md new file mode 100644 index 00000000..ab7a1654 --- /dev/null +++ b/proposals/infra/index.md @@ -0,0 +1,8 @@ +# Current Active Proposals + +{% assign doclist = site.pages | sort: 'url' %} +{% for doc in doclist %} +{% if doc.name contains '.md' and doc.dir == '/proposals/' and doc.name != 'index.md' %} +* [{{ doc.name }}]({{ doc.url | relative_url }}) +{% endif %} +{% endfor %} diff --git a/proposals/templates/infrastructure-template.md b/proposals/templates/infrastructure-template.md new file mode 100644 index 00000000..7ce93118 --- /dev/null +++ b/proposals/templates/infrastructure-template.md @@ -0,0 +1,76 @@ + + +# Feature name + +## Instructions + +> This template wraps at 80-columns. You don't need to match that wrapping, but +> having some consistent column wrapping makes it easier to view diffs on +> GitHub's review UI. Please wrap your lines to make it easier to review. + +> When filling out the template below for a new feature proposal, please do the +> following first: + +> 1. exclude the "Planned Version", "PRs" and "Issues" from the header. +> 2. Do not spend time writing the "Detailed design" until the feature has been +> merged in the "Under Consideration" phase. +> 3. Delete this Instructions section including the line below. + +--- + +* Proposal: [NNNN](NNNN-filename.md) +* Author(s): [Author 1](https://github.com/author_username) +* Sponsor: TBD +* Status: **Under Consideration** +* Impacted Project(s): (DXC, Clang, etc) + +*During the review process, add the following fields as needed:* + +* PRs: [#NNNN](https://github.com/microsoft/DirectXShaderCompiler/pull/NNNN) +* Issues: + [#NNNN](https://github.com/microsoft/DirectXShaderCompiler/issues/NNNN) + +## Introduction + +10,000 ft view of the change being proposed. Try to keep to one paragraph and +less than 10 sentences. + +## Motivation + +Describe the problems users are currently facing that this feature addresses. +Include concrete examples, links to related issues, and any relevant background. + +The point of this section is not to convince reviewers that you have a solution, +but rather that a problem needs to be resolved. + +## Proposed solution + +Describe your solution to the problem. Provide examples and describe how they +work. Show how your solution is better than current workarounds: is it cleaner, +safer, or more efficient? + +## Detailed design + +_The detailed design is not required until the feature is under review._ + +This section should grow into a specification that will live in the +specifications directory once complete. Each feature will need different levels +of detail here, but some common things to think through are: + +* Is there any potential for changed behavior? +* Will this expose new interfaces that will have support burden? +* How will this proposal be tested? +* Does this require additional hardware/software/human resources? + +## Alternatives considered (Optional) + +If alternative solutions were considered, please provide a brief overview. This +section can also be populated based on conversations that occur during +reviewing. + +## Acknowledgments (Optional) + +Take a moment to acknowledge the contributions of people other than the author +and sponsor. + + From f68908fa1e3140eb642c616de9b57804fc6aa9d7 Mon Sep 17 00:00:00 2001 From: Chris B Date: Tue, 9 Jan 2024 18:37:34 -0600 Subject: [PATCH 14/20] [0010] Mark proposal as accepted! (#152) All outstanding issues for the vk::BufferPointer proposal are resolved. This proposal is accepted and ready for implementation. --- proposals/0010-vk-buffer-ref.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0010-vk-buffer-ref.md b/proposals/0010-vk-buffer-ref.md index c1abf158..21ed83d3 100644 --- a/proposals/0010-vk-buffer-ref.md +++ b/proposals/0010-vk-buffer-ref.md @@ -6,7 +6,7 @@ * Sponsor(s): [Chris Bieneman](https://github.com/llvm-beanz), [Steven Perron](https://github.com/s-perron), [Diego Novillo](https://github.com/dnovillo) -* Status: **Under Review** +* Status: **Accepted** * Planned Version: Retroactive addition to Vulkan 1.2 (requires SPIR-V 1.3. Some language details require HLSL 202x From e733c90eb14c62cb11da98640ba2b5df32f9a7e7 Mon Sep 17 00:00:00 2001 From: Chris B Date: Tue, 16 Jan 2024 14:12:32 -0600 Subject: [PATCH 15/20] Apply feedback from #130 (#151) This applies the feedback from PR #130 in this comment: https://github.com/microsoft/hlsl-specs/pull/130#discussion_r1412549773 --- specs/language/introduction.tex | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/specs/language/introduction.tex b/specs/language/introduction.tex index 405b546e..cfe7b857 100644 --- a/specs/language/introduction.tex +++ b/specs/language/introduction.tex @@ -51,8 +51,9 @@ \p This document aims to use terms consistent with their definitions in \gls{isoC} and \gls{isoCPP}. In cases where the definitions are unclear, or -where this document diverges, refer to the remaining sections in this -chapter and the attached \nameref{main}. +where this document diverges from \gls{isoC} and \gls{isoCPP}, the definitions +in this section, the remaining sections in this chapter, and the attached +glossary (\ref{main}) supersede other sources. \Sec{Runtime Targeting}{Intro.Runtime} From 2676ae5ade30c4a9f6e63f37b950c7869bd9b48e Mon Sep 17 00:00:00 2001 From: Cassandra Beckley Date: Thu, 18 Jan 2024 12:00:52 -0800 Subject: [PATCH 16/20] [0011] Clarify behavior of SpirvType parameters and add SpirvOpaqueType (#150) Fixes #128 --- proposals/0011-inline-spirv.md | 56 +++++++++++++++++++++++++++++----- 1 file changed, 49 insertions(+), 7 deletions(-) diff --git a/proposals/0011-inline-spirv.md b/proposals/0011-inline-spirv.md index 7875eb9f..cca1d5ae 100644 --- a/proposals/0011-inline-spirv.md +++ b/proposals/0011-inline-spirv.md @@ -157,11 +157,37 @@ test. Some of the difficulties are: they cannot use the same id for two different types. This can become hard to manage. -This proposal deprecates the old mechanism, and replaces it with a new type -`vk::SpirvType`. The template on the type contains the opcode -and all of the parameters necessary for that opcode. The difficulty with this is -that the operands are not just literal integer values. Sometimes they are -another type. +This proposal deprecates the old mechanism, and replaces it with two new types +`vk::SpirvOpaqueType` and +`vk::SpirvType`. For +`SpirvOpaqueType`, the template on the type contains the opcode and all of the +parameters necessary for that opcode. Each parameter may be one of three kinds +of values: + +1. Any expression that can be evaluated to a constant scalar value at compile + time. This value will be passed in to the type-declaration instruction as + the id of an `OpConstant*` instruction. +1. An expression as described above, wrapped in a call to `vk::ext_literal`. + This value will be passed in to the type-declaration instruction as an + immediate literal value. +1. Any type. The id of the lowered type will be passed in to the + type-declaration instruction. + +For example, [`OpTypeArray`](https://registry.khronos.org/SPIR-V/specs/unified1/ +SPIRV.html#OpTypeArray) takes an id for the element type and an id for the +element length, so an array of 16 integers could be declared as + +``` +vk::SpirvOpaqueType +``` + +[`OpTypeVector`](https://registry.khronos.org/SPIR-V/specs/unified1/SPIRV.html# +OpTypeVector) takes an id for the component type and a literal for the component +count, so a 4-integer vector could be declared as + +``` +vk::SpirvOpaqueType +``` The header file could create a partial instantiation with a more meaningful name. For example, if you wanted to declare the types from the @@ -169,11 +195,11 @@ name. For example, if you wanted to declare the types from the you could have ``` -typedef vk::SprivType AvcMcePayloadINTEL; +typedef vk::SpirvOpaqueType AvcMcePayloadINTEL; // Requires HLSL2021 template -using VmeImageINTEL = vk::SpirvType; +using VmeImageINTEL = vk::SpirvOpaqueType; ``` Then the user could simply use the types: @@ -183,6 +209,22 @@ VmeImageINTEL image; AvcMcePayloadINTEL payload; ``` +If you want to use an inline SPIR-V type in a context where the size and +alignment matter, for example as an interface type or in a push constant, you +should use `SpirvType` instead of `SpirvOpaqueType`. + +`SpirvType` additionally takes a `size` parameter, specifying the number of +bytes a single value of the type occupies, and an `alignment` parameter, +specifying a power of two that the value will be aligned to in memory. For +example, an unsigned 8-bit integer type could be declared as + +``` +typedef vk::SpirvType uint8_t; +``` + +Neither `SpirvType` nor `SpirvOpaqueType` may be used as the component type for +an HLSL vector or matrix. + ### Decorations The current inline SPIR-V includes the `vk::ext_decorate` attribute. This works From 627ae31a2fc9835a7752645f42759ae30e4ee7f6 Mon Sep 17 00:00:00 2001 From: Greg Roth Date: Tue, 23 Jan 2024 14:00:04 -0700 Subject: [PATCH 17/20] Wave size range proposal (#149) * Initial wave size range proposal Adds the ability to specify a range of wave sizes that a shader can use instead of being limited to just one value --- proposals/0014-wave-size-range.md | 320 ++++++++++++++++++++++++++++++ 1 file changed, 320 insertions(+) create mode 100644 proposals/0014-wave-size-range.md diff --git a/proposals/0014-wave-size-range.md b/proposals/0014-wave-size-range.md new file mode 100644 index 00000000..2d632c37 --- /dev/null +++ b/proposals/0014-wave-size-range.md @@ -0,0 +1,320 @@ +# Wave Size Range + +* Proposal: [0014](0014-wave-size-range.md) +* Author(s): [Greg Roth](https://github.com/pow2clk) +* Sponsor: [Greg Roth](https://github.com/pow2clk) +* Status: **Under Consideration** +* Planned Version: Shader Model 6.8 + +## Introduction + +Shader Model 6.6 included the ability to specify a wave size on a shader entry + point in order to indicate either that a shader depends on or strongly prefers + a specific wave size. +If that size of wave isn't available to the current hardware, + shader will fail to load. +If that wave size is among those supported by the platform, + it must be the one used when the shader is executed. +This enables creating custom shaders optimized for the wave size of a + particularly interesting platform and loading them when possible. +Existing queries provide the developer with all the information necessary + to determine what a platform supports, + which should let them choose shaders accordingly. + +## Motivation + +Shader Model 6.6 provides no mechanism for specifying multiple wave sizes that a + single shader might be able to support. +Consequently, if more than one target wave size is of interest, separate entry + points will have to be created for each wave size. +These will each have to be compiled and shipped separately and, at run time, + the appropriate shaders will have to be selected. +This increases the size of the shipped product and potentially slows down + runtime shader loading. + +Ranges of wave sizes are already present in areas of the D3D API. +The query to determine what wave sizes are supported takes a range and similar + information is transmitted from the shader to the runtime, though not the + driver. + +## Proposed solution + +Modifying the parameters of the `WaveSize` attribute that was introduced by + Shader Model 6.6 to take additional parameters will allow the shader author to + specify fully what the shader supports and what it prefers. + +By adding an optional second `WaveSize` parameter, shader authors can provide + two values that represent the range of wave sizes that the shader can support. +This allows the same shader to be used in the event of the availability of a + wave size that the shader has optimized for specifically as well as other sizes + that are supported, but might not be so optimal. + +Some platforms that support a range of wave sizes might overlap with the range + specified as supported by the shader in more than one value. +In these cases, the graphics driver has a choice of wave sizes, + but it probably doesn't have the information needed to choose the best wave size. + +To provide the needed information, the shader author could specify the + preferred wave size in addition to the full range of acceptable values. +By adding an optional third `WaveSize` parameter, shader authors can specify the + optimal wave size for a given shader in addition to the range of acceptable + sizes. +This also preserves the ability to force the driver to choose your preferred + wave size when multiples are available just as was done with the single value + `WaveSize` attribute. + +## Detailed Design + +### HLSL additions + +The existing `WaveSize` attribute gains a second overload: + +```HLSL +[WaveSize(, , [prefWaveSize])] +void main() ... +``` + +Where `minWaveSize` is the minimum wave size supported by the shader + representing the beginning of the allowed range, + `maxWaveSize` is the maximum wave size supported by the shader + representing the end of the allowed range, + and `prefWaveSize` is the optional preferred wave size representing the size + expected to be the most optimal for this shader. + +### DXIL Additions + +The existing `WaveSize` value is stored as a metadata 32-bit integer constant. +To store the additional values, an additional metadata tag indicating a tuple of + three 32-bit integers representing the given parameters to `WaveSize`. +In the case where no preferred wave size is specified, the value zero will + indicate that nothing was specified. + +| Tag | Constant | Value | +|-------------------------|----------|-------------------------| +|kDxilRangedWaveSizeTag | 23 |MD list: (i32, i32, i32) | + +### SPIR-V Additions + +To represent the wave size range and preferred value in SPIR-V, + a new `ExecutionMode` value would need to be defined for the `OpExecutionMode` + instruction to allow specifying `SubgroupSize` with three operands instead of + one. +Like the other formats, the location of the operands would indicate + what each value represents, the first two being the min and max wave sizes and + the third being the preferred wave size. + +### Diagnostic Changes + +#### New Errors + +These are where new or slightly altered errors are produced: + +* If any of the parameters of `WaveSize` are not literal power of two integers + between 4 and 128 inclusive. +* If the minimum wave size value is greater than or equal to the max wave size + as respectively specified by the first and second parameters in the `WaveSize` + attribute. +* If the preferred wave size value indicated by the third parameter of + `WaveSize` is not in the range specified by the first two parameters. +* If multiple `WaveSize` attributes are applied to the same entry point + with different numbers or values of parameters, + the existing error indicating an attribute conflict is produced. +* If more than three or fewer than one parameter is applied to `WaveSize`. + +#### Validation Changes + +Validation should confirm: + +* Each element in that is a power of two integer between 4 and 128 inclusive. + The third parameter may also be zero. +* The minimum wave size is less than the maximum wave size. +* The preferred wave size is zero or lies between the minimum and maximum. +* Where validator version is less than 1.8, fail on shaders that use + `RangedWaveSize`. +* Validation should fail in an entry has both `RangedWaveSize` and `WaveSize` + metadata. +* Where validator version is greater than or equal to 1.8, fail when The tuple + that the wave size range tag points to does nto have exactly three elements. + +### Runtime Additions + +#### Runtime information + +No additions are needed here. +The PSV0 runtime data structure already contains both + `MinimumExpectedWaveLaneCount` and `MaximumExpectedWaveLaneCount` members that + can be used to transmit the minimum and maximum values to the runtime. + +#### Device Capability + +As a required feature, devices supporting the shader model it ships with are + required to respect the wave size restrictions indicated by the wave size range + metadata. + +## Testing + +### Correct Behavior Testing + +Verify this compiler output: + +1. The two parameter overload of `WaveSize` correctly transmits those values to a + metadata tuple with a zero third value that is pointed to by the correct tag + in the entry point attribute list. +2. The three parameter overload of `WaveSize` transmits those values as well as + the third in the same tuple. +3. That the PSV0 `MinimumExpectedWaveLaneCount` and `MaximumExpectedWaveLaneCount` + values reflect those provided for the wave size range. + +#### Diagnostics Testing + +1. Use the following invalid parameters each parameter location to `WaveSize` + and ensure that an appropriate error is produced: + 1. Negative power of two integer + 2. Floating point value + 3. non-literal integer + 4. Integer less than 4 + 5. Integer greater than 128 + 6. A non-power-of-two integer between 4 and 128 +2. Add the following invalid `WaveSize` attributes to an compute shader entry + point and ensure that an appropriate error is produced: + 1. no parameter list + 2. an empty parameter list "()" + 3. four parameters +3. Try the following invalid `WaveSize` parameter value combinations and ensure + that an appropriate error is produced: + 1. Set the minimum wave size equal to the maximum + 2. Set the minimum wave size greater than the maximum + 3. Set the preferred wave size to a value outside of the specified range +4. Combine multiples of the 1, 2 and 3 parameter `WaveSize` attribute overloads + with different values on the same entry point and ensure that an attribute + conflict error is produced. + +### Validation Testing + +Test that the following produce validation errors: + +1. The wave size range tag pointing to anything but a tuple of 3 +2. A tuple value is not an integer +3. A range tuple value is -4, 0, 1, 2, 3, 127, 129, or 256 +4. A preferred tuple value is -4, 1, 2, 3, 127, 129, or 256 +5. The minimum wave size value is equal to the maximum +6. The minimum wave size value is greater than the maximum +7. The preferred wave size is outside the specified range, but otherwise valid +8. Multiple metadata `kDxilRangedWaveSizeTag`s are in the same compiled shader. +9. A metadata `kDxilRangedWaveSizeTag` is included with a `kDxilWaveSizeTag` in + the same compiled shader. +10. Explicit validator versions before 1.8 used with `kDxilRangedWaveSizeTag`s + +### Execution Testing + +The runtime responsibilities are to reject shaders that have wave size + requirements that can't be supported and to use the preferred value when + possible even if another size is available. + +The platform's supported wave size range should first be queried using + `WaveLaneCountMin` and `WaveLaneCountMax` from the + `D3D12_FEATURE_DATA_D3D12_OPTIONS1` D3D structure. +This platform range should be used to craft a shader requested range that + verifies that the platform accepts and rejects all the shaders it should. + +Ensure that the following shader range with platform range combinations are + accepted: + +* The shader range minimum is the same as the platform range maximum. +* The shader range maximum is the same as the platform range minimum. +* The shader range is a superset of the platform range, but only by one power of + two value if possible. +* The shader range is the full 4-128. + +Ensure that the following shader range with platform combinations are rejected: + +* The shader range minimum is one power of two greater than the platform + range maximum. +* The shader range maximum is one power of two less than the platform range + minimum. + +For platforms that support more than one wave size, platform treatment of the + preferred wave size is needed. +When available to the platform, the preferred value must be used. +When not available, but others in the range are, the shader should still be + accepted. +The wave size used can be queried using `WaveGetLaneCount` and fed back to the + test to determine that the preferred wave size was used. + +For each wave size in the platform range, + ensure that the following shader range, preferred value, and platform range + combinations are accepted and use the preferred value: + +* The preferred value and shader maximum is equal to the current platform + range value and the shader minimum is one power of two less than the platform + minimum. +* The preferred value and shader minimum is equal to the current platform + range value and the shader maximum is one power of two more than the platform + maximum. +* The shader range is the full 4-128 and the preferred value is the current + platform range value. + +For each wave size in the platform range, + ensure that the following shader range, preferred value, and platform range + combinations are accepted: + +* The shader maximum is equal to the current platform range value and the + preferred value and shader minimum is one power of two less than the platform + minimum. +* The shader minimum is equal to the current platform range value and the + preferred value and shader maximum is one power of two more than the platform + maximum. +* The shader range is the full 4-128 and the preferred value is one power of two + less than the current platform range value. +* The shader range is the full 4-128 and the preferred value is one power of two + greater than the current platform range value. + +## Alternatives considered + +Useful as it is, the preferred wave size parameter adds some level of testing, + diagnostic, and other implementation complexity. +It wasn't part of the original discussions that motivated this feature, + but it is necessary to maintain one aspect of the original `WaveSize` behavior. +In addition to allowing the platform to reject wave sizes that are unsupported + at all, the single-value `WaveSize` attribute tells the platform which wave + size to choose among however many options the platform offers. +Without the preferred wave size, there wouldn't be a way to specify this + preference and the platform might choose arbitrarily among the supported values. + +Instead of modifying the existing attribute, an additional range attribute taking + only two parameters might be provided to specify the range while keeping the + existing `WaveSize` attribute to indicate the preferred wave size. +This has the advantage of keeping the syntax and semantics of the existing + attribute unaltered. +However, the name of the attribute is not really consistent with a preferred + value. +If we could do it over and replace them with attributes named `WaveSizeRange` and + `WaveSizeOptimal` or similar, the two attribute approach would be clean and + of clear intent. +Introducing new values like that and deprecating recently added old ones is + more likely to cause confusion than employing the broadly named `WaveSize` to + include all the information about wave sizing that a shader might need to + specify to the runtime system. + +Additionally, the existing `WaveSize` could be kept unaltered and a new + attribute with new spelling such as `WaveSizeRange` could take either two or + three parameters and represent the range with the optional preferred size. +The only issue here is that it complicates correctness checking a bit where + shaders that might employ both are concerned. +The simple solution would be to allow only one of either `WaveSize` or + `WaveSizeRange`. +Mild aesthetic preference ultimately opted to reuse the existing attribute. + +## Acknowledgements + +This document received invaluable contributions and feedback from various Microsoft + team members and our partners. + +Special thanks: + +* Alan Baker +* Chris Bieneman +* Martin Fuller +* Amar Patel +* Damyan Pepper +* Tex Riddell From 4cc2497813449a55e25cf6f58690aecc0719060a Mon Sep 17 00:00:00 2001 From: Greg Roth Date: Tue, 23 Jan 2024 16:30:25 -0700 Subject: [PATCH 18/20] [Wave Size Range] Renumber spec (#162) Miscommunication left this with an explicit number and circumstances put it in before the one currently (also inappropriately) tentatively numbered 13. This renumbers it to 13 to be sequential. What I should have done was create it as NNNN- until just before merging. That's what I'll do in the future. --- proposals/{0014-wave-size-range.md => 0013-wave-size-range.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename proposals/{0014-wave-size-range.md => 0013-wave-size-range.md} (99%) diff --git a/proposals/0014-wave-size-range.md b/proposals/0013-wave-size-range.md similarity index 99% rename from proposals/0014-wave-size-range.md rename to proposals/0013-wave-size-range.md index 2d632c37..50307c30 100644 --- a/proposals/0014-wave-size-range.md +++ b/proposals/0013-wave-size-range.md @@ -1,6 +1,6 @@ # Wave Size Range -* Proposal: [0014](0014-wave-size-range.md) +* Proposal: [0013](0013-wave-size-range.md) * Author(s): [Greg Roth](https://github.com/pow2clk) * Sponsor: [Greg Roth](https://github.com/pow2clk) * Status: **Under Consideration** From 04f6ad5d184a1e6070bd7aa6d32ca7c19c41f006 Mon Sep 17 00:00:00 2001 From: Greg Roth Date: Wed, 24 Jan 2024 13:09:05 -0700 Subject: [PATCH 19/20] [Wave Size Range] Allow compile-time constant parameters (#163) Replaces all mention of literal values with "compile-time constant" values per our decision to expand the allowed values. Note that DXC currently doesn't allow many expected compile-time values to work here, but that's a separate bug. Additionally corrected references to "power-of-two" as an adjective to use hyphens. Fixes #161 --- proposals/0013-wave-size-range.md | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/proposals/0013-wave-size-range.md b/proposals/0013-wave-size-range.md index 50307c30..bf9f7c05 100644 --- a/proposals/0013-wave-size-range.md +++ b/proposals/0013-wave-size-range.md @@ -109,8 +109,8 @@ Like the other formats, the location of the operands would indicate These are where new or slightly altered errors are produced: -* If any of the parameters of `WaveSize` are not literal power of two integers - between 4 and 128 inclusive. +* If any of the parameters of `WaveSize` are not compile-time constant + power-of-two integers between 4 and 128 inclusive. * If the minimum wave size value is greater than or equal to the max wave size as respectively specified by the first and second parameters in the `WaveSize` attribute. @@ -125,7 +125,7 @@ These are where new or slightly altered errors are produced: Validation should confirm: -* Each element in that is a power of two integer between 4 and 128 inclusive. +* Each element in that is a power-of-two integer between 4 and 128 inclusive. The third parameter may also be zero. * The minimum wave size is less than the maximum wave size. * The preferred wave size is zero or lies between the minimum and maximum. @@ -134,7 +134,7 @@ Validation should confirm: * Validation should fail in an entry has both `RangedWaveSize` and `WaveSize` metadata. * Where validator version is greater than or equal to 1.8, fail when The tuple - that the wave size range tag points to does nto have exactly three elements. + that the wave size range tag points to does not have exactly three elements. ### Runtime Additions @@ -165,13 +165,17 @@ Verify this compiler output: 3. That the PSV0 `MinimumExpectedWaveLaneCount` and `MaximumExpectedWaveLaneCount` values reflect those provided for the wave size range. +Note that the above must all use literal values for the parameters on account of + other compile-time contants being broken due to DXC bug + [#2188](https://github.com/microsoft/DirectXShaderCompiler/issues/2188). + #### Diagnostics Testing 1. Use the following invalid parameters each parameter location to `WaveSize` and ensure that an appropriate error is produced: - 1. Negative power of two integer + 1. Negative power-of-two integer 2. Floating point value - 3. non-literal integer + 3. non-compile-time constant integer 4. Integer less than 4 5. Integer greater than 128 6. A non-power-of-two integer between 4 and 128 From 51f19b723e2dc1252c253d6823f4baec261631e8 Mon Sep 17 00:00:00 2001 From: Cassandra Beckley Date: Wed, 7 Feb 2024 10:44:22 -0800 Subject: [PATCH 20/20] [0011] Fix invalid example code (#166) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fix a couple of the examples – global constants must be `static`, attr lists should go before the function declaration, not the template declaration. --- proposals/0011-inline-spirv.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/0011-inline-spirv.md b/proposals/0011-inline-spirv.md index cca1d5ae..443df262 100644 --- a/proposals/0011-inline-spirv.md +++ b/proposals/0011-inline-spirv.md @@ -102,8 +102,8 @@ extension in a header file. The header file could be something like // spv_khr_post_depth_coverage.h // It would be nice to have this live in a namespace spv::khr::, but not possible with attributes. -const uint32_t SampleMaskPostDepthCoverageCapabilityId = 4447; -const uint32_t PostDepthCoverageExecutionModeId = 4446; +static const uint32_t SampleMaskPostDepthCoverageCapabilityId = 4447; +static const uint32_t PostDepthCoverageExecutionModeId = 4446; #define SPV_KHR_PostDepthCoverageExecutionMode vk::ext_extension("SPV_KHR_post_depth_coverage"), vk::ext_capability(SampleMaskPostDepthCoverageCapabilityId), vk::spvexecutionmode(PostDepthCoverageExecutionModeId) ``` @@ -128,10 +128,10 @@ defined in the header file as a function, and then the function can be called by the users. For example, ``` +template [[vk::ext_capability(5568)]] [[vk::ext_extension("SPV_INTEL_subgroups")]] [[vk::ext_instruction(/* OpSubgroupShuffleINTEL */ 5571)]] -template T SubgroupShuffleINTEL(T data, uint32 invocationId); ```