From 9eea774ed15a1e75a81b783808bd514780a9c075 Mon Sep 17 00:00:00 2001 From: Peter Harris Date: Sat, 11 Jan 2025 00:32:49 +0000 Subject: [PATCH 1/2] Improve docs --- README.md | 64 ++++++++++++++++++------------ docs/running_android.md | 88 +++++++++++++++++++++++++++++++++++------ lgl_android_install.py | 6 +-- 3 files changed, 117 insertions(+), 41 deletions(-) diff --git a/README.md b/README.md index f7e22f7..3b0a3ca 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,13 @@ # About -libGPULayers provides tooling for creating a skeleton implementation of a -graphics API layer driver. Over time, we plan to add a set of useful -ready-to-use diagnostic layers that have been built using the tooling. +libGPULayers provides tooling to rapidly create new Vulkan layer drivers, +allowing developers to quickly generate new layers that can be used for +ad hoc experiments during development. + +In addition, we provide a number of pre-built layers that have been built +using these tools. These layers can be used as standalone tools in their +own right, and some can be used alongside other Arm tools such as Arm +Performance Studio. ## What are layer drivers? @@ -15,35 +20,34 @@ change, everything that the native driver sees makes layers an exceptionally powerful tool for debugging functional and performance issues. The Vulkan API defines a standard layer driver mechanism. The API uses layers -to implement API parameter validation and error checking, but they are a +to implement API parameter validation and error checking, but they are also a general purpose mechanism for all kinds of developer tooling. -The OpenGL ES API does not have layer drivers as a standard feature. However, -since Android 10, the Android OpenGL ES loader has implemented support for -OpenGL ES layers, so we can access the same methodology benefits as Vulkan. - ## What is the purpose of this project? We support many application developers during their development cycle. We -rarely get access to application source code, so layer drivers provide us a -convenient way for us to make modifications to API behavior. We use this -capability to triage bugs and to test the impact of application API usage fixes -and optimizations, and wanted to share the capability with our wider developer -ecosystem. - -This project has two main goals: - -* Firstly, it provides developers with access to the same easy-to-use layer - generation tooling that we use internally, allowing them to perform their - own experiments using layers. -* Secondly, we will use this layer generation to provide a number of pre-built - layers than can be used during development to debug and optimize application - rendering. +rarely get access to application source code, so layer drivers provide us with +an invaluable mechanism to make modifications to application API usage. The +`GPU Support` layer in this project is a a tool we use during technical support +investigations to quickly triage developers problems. + +We also use layer drivers as a way to develop new API-aware debug and profiling +capabilities. The performance layers in this repository, such as the +`GPU Timeline` layer, are often early prototypes that we want to share with +developers to test new ideas and gather feedback. Some are designed to be used +as standalone development tools, others can also be used alongside other Arm +tools such as the Arm Streamline profiler in [Arm Performance Studio][2]. + +As you can tell, we find layers exceptionally useful. However, creating a new +layer from scratch requires a lot of boilerplate code and is fiddly to get +right. We therefore also wanted to take this opportunity to share our layer +generation tools which make it trivial to create a complete bare-bones layer +that is ready to extend and use. ## Supported devices -This library is currently tested on devices running Android and using Arm® -Immortalis™ and Arm Mali™ GPUs. Contributions adding support for other +This library is currently tested on devices running Android or Linux, and using +Arm® Immortalis™ and Arm Mali™ GPUs. Contributions adding support for other platforms is welcome. # License @@ -54,22 +58,30 @@ from this repository you acknowledge that you accept terms specified in the # Documentation +Common documentation + * [Building a new layer](./docs/building.md) * [Running using a layer on Android](./docs/running_android.md) * [Running using a layer on Linux](./docs/running_linux.md) * [Frequently asked questions](./docs/faq.md) +Layer documentation + +* [Layer: GPU Support](./layer_gpu_support/LAYER_README.md) +* [Layer: GPU Timeline](./layer_gpu_support/LAYER_README.md) + # Support If you have issues with the layer framework, or the prebuilt layers, please raise them in the project's GitHub issue tracker. If you have any questions about Arm GPUs, application development for Arm GPUs, -or general mobile graphics development or technology please submit them on the +or general mobile graphics development or technology, please submit them on the [Arm Community graphics forums][1]. - - - -_Copyright © 2024, Arm Limited and contributors._ +_Copyright © 2024-2025, Arm Limited and contributors._ [1]: https://community.arm.com/support-forums/f/graphics-gaming-and-vr-forum/ +[2]: https://developer.arm.com/Tools%20and%20Software/Arm%20Performance%20Studio diff --git a/docs/running_android.md b/docs/running_android.md index c9f2bed..2b79ac5 100644 --- a/docs/running_android.md +++ b/docs/running_android.md @@ -1,23 +1,85 @@ # Running using a layer on Android -There are multiple ways to install layer drivers for Android. For our use case -we cannot usually modify the application binary, so we install the layer into -the target application data directory on the file system. +To make it easy to install and configure layers for Android, we provide an +installation script which can automatically configure one or more layers. -## Script configuration +These instruction assume that you have already built the layers that you want +to install. See the [Building a new layer](./building.md) page for build +instructions. -When generating a new layer, the generator also creates an installation -script for Android with the correct settings to use. +## Script configuration -To install and configure a layer for use, run this script with the name of -the package that you want to instrument: +From the root directory of the GitHub project run the Android installation +utility, specifying the directory containing the layer that you want to +install: ```sh -py -3 android_install.py --package +python3 lgl_android_install.py --layer layer_gpu_example ``` -... run your test scenario, and then press any key to wake the script and -remove the layer from the device. +By default the script will automatically search to find connected Android +devices, and debuggable packages on the chosen device. If there are multiple +options the script will present a menu and prompt you for a selection. You +can avoid this by manually specifying the device (`--device`/`-D`) and package +(`--package`/`-P`) to instrument. + +Wait for the layer to be installed and configured. The script will notify you +when this has been done. You can now perform your development work. When you +are finished, return to the script and press a key so it can clean up the +device and remove the layers. + +### Layer configuration + +Some layers require a configuration file to control their behavior. Most +layers that need a configuration file ship with a default config, +`layer_config.json`, in their layer directory. Users can override this with +a custom config by using the `--config`/`-C` option to specify a custom +config file. + +**NOTE:** The layer that each config file applies to is specified in the config +file itself, and is not implied by command line order. + +### Multi-layer installation + +The script can install multiple layers in a stack. Specify the `--layer`/`-L` +option multiple times, once per layer. Layers are stacked in command line +order, with the first layer specified being the top of the stack closest to the +application. + +### Khronos validation layer installation + +The script can install the Khronos validation layer. A dummy layer directory +is provided as the `layer_khronos_validation` directory. Download the the +latest binary release from [Vulkan-ValidationLayers/releases][1] GitHub, and +place the binaries into dummy build tree at the correct location. + +Once this is done you can install the validation layer like any other. + +**NOTE:** When installing the Khronos validation layer you need to decide where +to install it in the layer stack. If you install it as the first layer in the +stack it will only report issues from the application. If you install it last, +it will report errors from the application and all layers above it in the +stack. Both of these might be useful, depending on what you are trying to do. + +### Capturing logcat + +During development it is often useful to capture the Android log, using +`logcat` to save it to a file on the host. + +If you specify the `--logcat ` option the script will automatically clear +the logcat log after installing the layers, and start recording logcat to the +specified file. Logcat recording will end during device clean up. + +### Capturing Perfetto traces + +The Mali Timeline tool is designed to work alongside the Android Perfetto +render stages trace, providing additional semantic information that can be +combined with the Perfetto data to provide a richer visualization with more API +awareness. + +If you specify the `--perfetto ` option the script will automatically +configure Perfetto to capture render stages information for the target +application and save it to the specified file. ## Manual configuration @@ -60,4 +122,6 @@ adb shell settings delete global gpu_debug_layer_app - - - -_Copyright © 2024, Arm Limited and contributors._ +_Copyright © 2024-2025, Arm Limited and contributors._ + +[1]: https://github.com/KhronosGroup/Vulkan-ValidationLayers/releases/ \ No newline at end of file diff --git a/lgl_android_install.py b/lgl_android_install.py index 16d9830..7f18dbd 100755 --- a/lgl_android_install.py +++ b/lgl_android_install.py @@ -702,15 +702,15 @@ def parse_cli() -> argparse.Namespace: parser.add_argument( '--symbols', '-S', action='store_true', default=False, - help='use to install layers with unstripped symbols') + help='install layers with unstripped symbols') parser.add_argument( '--logcat', type=str, default=None, - help='file path to save logcat to after a run') + help='save logcat to this file during the run') parser.add_argument( '--perfetto', type=str, default=None, - help='file path to save Perfetto trace to after run') + help='save Perfetto trace to this file during the run') return parser.parse_args() From b99d1af17a4b7fadf1eb057fcee6da98d6e58b17 Mon Sep 17 00:00:00 2001 From: Peter Harris Date: Sat, 11 Jan 2025 00:42:21 +0000 Subject: [PATCH 2/2] Update after review --- docs/running_android.md | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/docs/running_android.md b/docs/running_android.md index 2b79ac5..b052e19 100644 --- a/docs/running_android.md +++ b/docs/running_android.md @@ -25,8 +25,8 @@ can avoid this by manually specifying the device (`--device`/`-D`) and package Wait for the layer to be installed and configured. The script will notify you when this has been done. You can now perform your development work. When you -are finished, return to the script and press a key so it can clean up the -device and remove the layers. +are finished, return to the script and press a key to notify it that it can +clean up the device and remove the layers. ### Layer configuration @@ -48,10 +48,10 @@ application. ### Khronos validation layer installation -The script can install the Khronos validation layer. A dummy layer directory -is provided as the `layer_khronos_validation` directory. Download the the -latest binary release from [Vulkan-ValidationLayers/releases][1] GitHub, and -place the binaries into dummy build tree at the correct location. +The script can install the Khronos validation layer. A dummy layer directory , +`layer_khronos_validation`, is provided. Download the the latest binary release +from the [Vulkan-ValidationLayers/releases][1] GitHub, and place the binaries +into dummy build tree at the correct location. Once this is done you can install the validation layer like any other. @@ -59,9 +59,9 @@ Once this is done you can install the validation layer like any other. to install it in the layer stack. If you install it as the first layer in the stack it will only report issues from the application. If you install it last, it will report errors from the application and all layers above it in the -stack. Both of these might be useful, depending on what you are trying to do. +stack. -### Capturing logcat +### Capturing Android logcat During development it is often useful to capture the Android log, using `logcat` to save it to a file on the host. @@ -70,12 +70,12 @@ If you specify the `--logcat ` option the script will automatically clear the logcat log after installing the layers, and start recording logcat to the specified file. Logcat recording will end during device clean up. -### Capturing Perfetto traces +### Capturing Android Perfetto traces -The Mali Timeline tool is designed to work alongside the Android Perfetto -render stages trace, providing additional semantic information that can be -combined with the Perfetto data to provide a richer visualization with more API -awareness. +The Timeline layer is designed to provide semantic metadata that can be used +to annotate an Android Perfetto render stages trace. This provides profiling +tools with API information that Perfetto alone cannot provide, making it a +much more data-rich visualization . If you specify the `--perfetto ` option the script will automatically configure Perfetto to capture render stages information for the target @@ -86,6 +86,10 @@ application and save it to the specified file. For users with existing configuration flows, Vulkan layers can be manually configured using `adb` to copy the files and set the system properties. +For layers with configuration files, the configuration file must be named +after the layer (e.g. `VK_LAYER_LGL_GPUSUPPORT.json`) and then pushed to +the `/data/local/tmp` directory with global (`0666`) access permissions. + **Note:** Android will fail to start an application if it has a layer enabled, but the layer library cannot be found. Ensure that you disable your layer when you have finished using it.