Got layouts and shaders working better (#60).

music/ICSC-2024/cloud-5.tex

@@ -75,18 +75,20 @@
 
 \section{Introduction}
 
-The World Wide Web was invented for the purpose of publishing scientific information by any scientist, to any scientist. It was then co-opted by American business interests for the purpose of selling to consumers. Along the way, it became a conduit for the wholesale theft of intellectual property in the form of illegal downloads of recorded music, films, and computer games. Then it became the major platform for social media, another (disguised) form of selling to consumers while taking their personal data. Indeed, most people's use of the Web has been increasingly funneled through Google search and various social media platforms, which are highly proprietary and far from open. Yet, at every step along this tortuous path, the original inventions that created the World Wide Web, including packet-switched networking (especially TCP/IP) and the Web browser itself, based on  Web standards (including HTTP, URLs, HTML, MIME types, JavaScript, and now WebAssembly), have remained non-proprietary, decentralized, backwards compatible, and more or less open. In fact, driven by both entrepreneurship and competitive pressures to show more appealing ads, the power of Web browsers has increased to the point of providing the equivalent of a game engine and an operating system, running only about 1.5 to 2.5  times as slowly as native C code.
+The World Wide Web was invented for the purpose of publishing scientific information by any scientist, to any scientist. It was then co-opted by American business interests for the purpose of selling to consumers. Along the way, it became a conduit for the wholesale theft of intellectual property in the form of illegal downloads of recorded music, films, and computer games. Then it became the major platform for social media, another (disguised) form of selling to consumers while taking their personal data. Indeed, most people's use of the Web has been increasingly funneled through Google search and various social media platforms, which are highly proprietary and far from open. Yet, at every step along this tortuous path, the original inventions that created the World Wide Web, including packet-switched networking (especially TCP/IP) and the Web browser itself, based on  Web standards (including HTTP, URLs, HTML, MIME types, JavaScript, and now WebAssembly), have remained non-proprietary, decentralized, backwards compatible, and more or less open. In fact, driven by both entrepreneurship and the competitive pressure to show ever more appealing ads, the power of Web browsers has increased to the point of providing the equivalent of a game engine and an operating system, running only about 1.5 to 2.5  times as slowly as native C code.
 
 I would go so far as to say that the establishment of Web standards is one of the most fortunate things that has ever happened, because it preserves essential freedoms in face of a rather remarkable level of skilled greed (I remain wary, however, that private interests will end up hijacking these standards and manipulating them to be less open).
 
-Now to music. Music now has, in spite of everything, thanks to Web standards and the growing capabilities of browsers, a new platform that provides both novel power to composers and novel availability to audiences. This is what I call \emph{cloud music}: computer music that runs in Web browsers. Not downloads, not streams, but autonomous programs that synthesize music and visuals in real time with the option of interacting with the audience, performing endlessly, producing endless variations, and communicating across the world. Such music can be free, or secured for paid subscriptions, or supported by advertising.
+Now to music. Music now has, in spite of everything, thanks to Web standards and the growing capabilities of browsers, a new platform that provides both novel power to composers and novel availability to audiences. This is what I call \emph{cloud music}: computer music that runs in Web browsers. Not downloads, not streams, but autonomous programs that synthesize music and visuals in real time with the option of interacting with the audience, performing endlessly, producing endless variations, and communicating across the world. Such music can be free, or secured for paid subscriptions, or supported by advertising. 
+
+The audio resolution in cloud-5 is the same as that of WebAudio: 48,000 frames per second of floating-point samples in 128 frame buffers. Although studio software often offers even higher resolution, WebAudio does operate within the recognized range of professional audio production quality. Furthermore, Csound can interpolate samples within the 128 frame buffers,
 
 For examples of cloud music, one may look to online computer games with procedural audio, the Strudel live coding system, or my own compositions produced using the subject of this paper, cloud-5.
 
 \section{Use Cases and Examples}
 
 \begin{description}
-\item[Composition] The cloud-5 system is useful for the composition of electroacoustic music because it contains very powerful facilities. These include all the astonishing capabilities already built into every standard Web browser, a complete WebAssembly build of Csound, a complete WebAssembly build of my CsoundAC algorithmic composition library, the complete Strudel live coding system which can render audio using either Csound or its own built-in sampler and synthesizer, and any other module that will run in a Web browser. All of these facilities are completely cross-platform. That makes cloud-5 far and away the easiest computer music system of comparable power to install and configure: unzip it somewhere and run a local Web server in that directory. The audio resolution in cloud-5 is the same as that of WebAudio, namely 48,000 frame per second in 128 frame buffers. Note that this a professional level of quality (although professional gear can have considerably higher resolution, nobody hears the difference), and that Csound can interpolate within the 128 frame chunks.
+\item[Composition] The cloud-5 system is useful for the composition of electroacoustic music because it contains very powerful facilities. These include all the astonishing capabilities already built into every standard Web browser, a complete WebAssembly build of Csound, a complete WebAssembly build of my CsoundAC algorithmic composition library, the complete Strudel live coding system which can render audio using either Csound or its own built-in sampler and synthesizer, and any other module that will run in a Web browser. All of these facilities are completely cross-platform. That makes cloud-5 far and away the easiest computer music system of comparable power to install and configure: unzip it somewhere and run a local Web server in that directory. 
 \item[Fixed Pieces] These are similar to fixed medium pieces of electroacoustic music. When the user starts the piece, it plays audio until the score ends. \item[Always-On Pieces] Similar to fixed pieces; but once started, always-on pieces play indefinitely. By using randomization or fractals, always-on pieces can play indefinitely without any repetition.
 \item[Interactive Pieces] Similar to fixed pieces or always-on pieces; but once the piece is started, the user interface provides controls with which the user may steer some aspects of the composition or rendering.
 \item[Live Coding] Similar to interactive pieces; but the user has \emph{complete} control over the composition in the Strudel REPL, and can create entirely new compositions or engage in lengthy improvisations.
@@ -106,20 +108,38 @@ \subsection{Components}
 Currently, cloud-5 consists of the following custom elements and user addons:
 
 \begin{description}
-\item[\texttt{<cloud5-piece>}] This custom element defines the main menu of the piece and its event handlers, instantiates Csound and/or Strudel as required, starts and stops performances, hosts the controls submenu, and contains some helper code.
-\item[\texttt{<cloud5-piece>.csound\_code\_addon}]  This is user-defined code as a multi-line JavaScript string that contains the complete text of a Csound .csd file (which can be quite large).
-\item[\texttt{<cloud5-piece>.score\_generator\_function\_addon}]  This is user-defined code in the form of a JavaScript function that will be called at the start of performance to generate a CsoudAC score, which will be performed by Csound. 
+\item[\texttt{<cloud5-piece>}] This custom element defines the main menu of the piece and its event handlers, instantiates Csound and/or Strudel as required, starts and stops performances, hosts a controls submenu, and defines some helper JavaScript code.
+\item[\texttt{<cloud5-piece>.csound\_code\_addon}]  This is user-defined code as a multi-line JavaScript string containing the complete text of a Csound .csd file (which can be quite large).
+\item[\texttt{<cloud5-piece>.control\_parameters\_addon}]  This a user-defined JavaScript object whose fields have the names and initial values of Csound control channels.
+\item[\texttt{<cloud5-piece>.menu\_folder\_addon}]  The user calls this with the name of a new folder to be added to the controls menu of the piece.
+\item[\texttt{<cloud5-piece>.menu\_slider\_addon}]  The user calls this with the name of a new slider control to be added to the controls menu, along with its Csound channel name, lowest value, highest value, and the folder to which the slider will be added.
+\item[\texttt{<cloud5-piece>.score\_generator\_function\_addon}]  This is user-defined code in the form of a JavaScript function that will be called at the start of performance to generate a CsoudAC score to be performed by Csound. 
 \item[\texttt{<cloud5-piano-roll>}] This custom element is an overlay that draws a three-dimensional piano roll display of a generated CsoundAC Score. During performance, the piano roll shows the current position in the Score with a moving ball. It is possible to zoom in and out of the piano roll, drag it around, rotate it, and so on.
-\item[\texttt{<cloud5-strudel>}] This custom element is an overlay that shows the Strudel REPL, in which the user can do live coding of the Strudel patch during performance. The Strudel REPL has its own real-time piano roll score, and outlines the currently active functions in the patch.
-\item[Strudel Patch] This is user-defined code as a multi-line JavaScript string, defined in a \texttt{<script>} element of the piece, that contains a complete Strudel patch to be performed by the piece. This variable must be assigned to the \texttt{strudel\_patch} property of the \texttt{<cloud5-strudel>} element.
-\item[\texttt{<cloud5-shader>}] This  custom element is an overlay that shows a canvas displaying a GLSL shader. This shader can be used either to visualize audio, or to produce notes for Csound to perform.
-\item[Fragment Shader] This is user-defined code as a multi-line JavaScript string, defined in a \texttt{<script>} element of the piece, that contains the complete code of a GLSL fragment shader (which can be quite large). This variable must be assigned to the \texttt{fragment\_patch} property of the \texttt{<cloud5-shader>} element.
-\item[Shader Sampler] This is user-defined code as a JavaScript function, defined in a \texttt{<script>} element of the piece, that will be called on a timer interval to downsample the video buffer and translate it to a fragment of a Csound score to be played in real time. This function must be assigned to the \texttt{shader\_sampler\_hook} property of the \texttt{<cloud5-shader>} element.
-\item[Audio Visualizer] This user-defined code as a JavaScript function, defined in a \texttt{<script>} element of the piece, that will be called on a timer interval to make an FFT analysis of the current audio buffer, and to translate this analysis to uniform variables that will be sent to the shader and that should control the appearance of the visuals. This function must be assigned to the \texttt{audio\_visualizer\_hook} property of the \texttt{<cloud5-shader>} element.
+\item[\texttt{<cloud5-strudel>}] This custom element is a popup IFrame that shows the Strudel REPL, in which the user can do live coding of the Strudel patch during performance. The Strudel REPL has its own real-time piano roll display, and highlights the currently active functions in the Strudel code.
+\item[\texttt{<cloud5-strudel>.strudel\_code\_addon}] This is user-defined code as a multi-line JavaScript string containing a complete Strudel patch to be performed by the piece.
+\item[\texttt{<cloud5-shadertoy>}] This  custom element is an overlay that shows a canvas displaying a GLSL shader. This shader can be used to visualize audio, to produce notes for Csound to perform, and so on. The element is designed to support the easy adaptation of shaders developed in the ShaderToy Web site.
+\item[\texttt{<cloud5-shadertoy>.shader\_parameters\_addon}] This is a user-defined Javascript object with the following fields:
+\begin{description}
+\item[\texttt{fragment\_shader\_code\_addon}] Contains user-defined GLSL code to be compiled for display on the canvas of the shader overlay.
+\item[\texttt{vertex\_shader\_code\_addon}] May contain user-defined GLSL code to be compiled for display on the canvas of the shader overlay; there is a default value that includes the entire canvas.
+\item[\texttt{set\_uniforms\_function\_addon}] This may be set to user-defined code in the form of a JavaScript function that will be called before every shader animation frame to set uniforms that can be used to control the shader; commonly used to implement an audio visualizer.
+\item[\texttt{get\_attributes\_function\_addon}] This may be set to user-defined code in the form of a JavaScript function that will be called after every shader animation frame to read attributes that can be used by the piece; for example, to sample the visuals and translate them to Csound notes to be played.
+\end{description}
 \item[\texttt{<cloud5-log>}] This custom element is an overlay that presents a scrolling list of runtime messages from Csound and/or other sources.
 \item[\texttt{<cloud5-about>}] This custom element is an overlay that presents license information, authorship, credits, program notes, and so on.
 \end{description}
 
+\section{An Example Piece}
+
+The following example is a simple piece that nevertheless uses many of the custom elements presented above. The code is folded to show only the high-level elements and how they are connected together.
+
+\subsection{Score Generator}
+\subsection{Csound Orchestra}
+\subsection{Strudel Patch}
+\subsection{Controls}
+\subsection{Audio Visualizer}
+\subsection{About}
+
 \section{Best Practices}
 
 The cloud-5 system is designed for making permanent works of music -- pieces that will always play, even in the far future (assuming that the Web standards continue to be versionless and backwards-compatible, as they have been for 35 years). In other words, cloud-5 is not at all a general purpose Web development system, and therefore pieces should not be developed in the standard way.