From b8d70d2418de60576e133ee723acf037fecd4a52 Mon Sep 17 00:00:00 2001 From: Altynbek Orumbayev Date: Mon, 23 Dec 2024 01:04:25 +0100 Subject: [PATCH] docs: initial rewrites for appmanager; clients and debugger docs --- docs/source/capabilities/app-manager.md | 212 ++++++++++++++++++++++++ docs/source/capabilities/client.md | 106 ++++++++++-- docs/source/capabilities/debugger.md | 70 +++++--- 3 files changed, 355 insertions(+), 33 deletions(-) create mode 100644 docs/source/capabilities/app-manager.md diff --git a/docs/source/capabilities/app-manager.md b/docs/source/capabilities/app-manager.md new file mode 100644 index 00000000..2c59ec47 --- /dev/null +++ b/docs/source/capabilities/app-manager.md @@ -0,0 +1,212 @@ +# App management + +App management is a higher-order use case capability provided by AlgoKit Utils that builds on top of the core capabilities. It allows you to create, update, delete, call (ABI and otherwise) smart contract apps and the metadata associated with them (including state and boxes). + +## `AppManager` + +The `AppManager` is a class that is used to manage app information. + +To get an instance of `AppManager` you need to instantiate it with an algod client: + +```python +from algokit_utils import AppManager + +app_manager = AppManager(algod_client) +``` + +## Calling apps + +### App Clients + +The recommended way of interacting with apps is via [Typed app clients](./typed-app-clients.md) or if you can't use a typed app client then an [untyped app client](./app-client.md). The methods shown on this page are the underlying mechanisms that app clients use and are for advanced use cases when you want more control. + +### Creation + +To create an app you can use the following parameters: + +- `sender: str` - The address of the account that will create the app +- `approval_program: bytes | str` - The program to execute for all OnCompletes other than ClearState as raw TEAL that will be compiled (string) or compiled TEAL (encoded as bytes) +- `clear_state_program: bytes | str` - The program to execute for ClearState OnComplete as raw TEAL that will be compiled (string) or compiled TEAL (encoded as bytes) +- `schema: dict | None` - The storage schema to request for the created app. This is immutable once the app is created. It is a dictionary with: + - `global_ints: int` - The number of integers saved in global state + - `global_byte_slices: int` - The number of byte slices saved in global state + - `local_ints: int` - The number of integers saved in local state + - `local_byte_slices: int` - The number of byte slices saved in local state +- `extra_program_pages: int | None` - Number of extra pages required for the programs. This is immutable once the app is created. + +If you pass in `approval_program` or `clear_state_program` as a string then it will automatically be compiled using Algod and the compilation result will be available via `app_manager.get_compilation_result` (including the source map). To skip this behaviour you can pass in the compiled TEAL as bytes. + +### Updating + +To update an app you can use the following parameters: + +- `sender: str` - The address of the account that will update the app +- `app_id: int` - The ID of the app to update +- `approval_program: bytes | str` - The new program to execute for all OnCompletes other than ClearState +- `clear_state_program: bytes | str` - The new program to execute for ClearState OnComplete + +### Deleting + +To delete an app you can use the following parameters: + +- `sender: str` - The address of the account that will delete the app +- `app_id: int` - The ID of the app to delete + +### Calling + +To call an app you can use the following parameters: + +- `sender: str` - The address of the account that will call the app +- `app_id: int` - The ID of the app to call +- `on_complete: OnComplete | None` - The on-completion action to specify for the call +- `args: list[bytes | str] | None` - Any arguments to pass to the smart contract call +- `accounts: list[str] | None` - Any account addresses to add to the accounts array +- `foreign_apps: list[int] | None` - The ID of any apps to load to the foreign apps array +- `foreign_assets: list[int] | None` - The ID of any assets to load to the foreign assets array +- `boxes: list[BoxReference | BoxIdentifier] | None` - Any boxes to load to the boxes array + +### ABI Return Values + +The `AppManager` provides a static method to parse ABI return values from transaction confirmations: + +```python +abi_return = AppManager.get_abi_return(confirmation, abi_method) +if abi_return: + raw_return_value = abi_return.raw_return_value + return_value = abi_return.return_value +``` + +## Accessing state + +### Global state + +To access global state you can use the following method from an `AppManager` instance: + +- `app_manager.get_global_state(app_id)` - Returns the current global state for the given app ID decoded into a dictionary keyed by the UTF-8 representation of the state key with various parsed versions of the value (base64, UTF-8 and raw binary) + +```python +global_state = app_manager.get_global_state(12345) +``` + +Global state is parsed from the underlying algod response via the following static method from `AppManager`: + +- `AppManager.decode_app_state(state)` - Takes the raw response from the algod API for global state and returns a friendly dictionary keyed by the UTF-8 value of the key + +```python +app_state = AppManager.decode_app_state(global_app_state) + +key_as_binary = app_state['value1'].key_raw +key_as_base64 = app_state['value1'].key_base64 +if isinstance(app_state['value1'].value, str): + value_as_string = app_state['value1'].value + value_as_binary = app_state['value1'].value_raw + value_as_base64 = app_state['value1'].value_base64 +else: + value_as_number = app_state['value1'].value +``` + +### Local state + +To access local state you can use the following method from an `AppManager` instance: + +- `app_manager.get_local_state(app_id, address)` - Returns the current local state for the given app ID and account address decoded into a dictionary keyed by the UTF-8 representation of the state key with various parsed versions of the value (base64, UTF-8 and raw binary) + +```python +local_state = app_manager.get_local_state(12345, 'ACCOUNTADDRESS') +``` + +### Boxes + +To access and parse box values and names for an app you can use the following methods from an `AppManager` instance: + +- `app_manager.get_box_names(app_id)` - Returns the current box names for the given app ID +- `app_manager.get_box_value(app_id, box_name)` - Returns the binary value of the given box name for the given app ID +- `app_manager.get_box_values(app_id, box_names)` - Returns the binary values of the given box names for the given app ID +- `app_manager.get_box_value_from_abi_type(app_id, box_name, abi_type)` - Returns the parsed ABI value of the given box name for the given app ID for the provided ABI type +- `app_manager.get_box_values_from_abi_type(app_id, box_names, abi_type)` - Returns the parsed ABI values of the given box names for the given app ID for the provided ABI type +- `AppManager.get_box_reference(box_id)` - Returns a tuple of `(app_id, name_bytes)` representation of the given box identifier/reference + +```python +app_id = 12345 +box_name = 'my-box' +box_name2 = 'my-box2' + +box_names = app_manager.get_box_names(app_id) +box_value = app_manager.get_box_value(app_id, box_name) +box_values = app_manager.get_box_values(app_id, [box_name, box_name2]) +box_abi_value = app_manager.get_box_value_from_abi_type(app_id, box_name, algosdk.abi.StringType()) +box_abi_values = app_manager.get_box_values_from_abi_type(app_id, [box_name, box_name2], algosdk.abi.StringType()) +``` + +## Getting app information + +To get reference information and metadata about an existing app you can use: + +- `app_manager.get_by_id(app_id)` - Returns current app information by app ID including approval program, clear state program, creator, schemas, and global state + +## Common app parameters + +When interacting with apps (creating, updating, deleting, calling), there are some common parameters that you will be able to pass in to all calls: + +- `app_id: int` - ID of the application; only specified if the application is not being created +- `on_complete: OnComplete | None` - The on-complete action of the call +- `args: list[bytes | str] | None` - Any arguments to pass to the smart contract call +- `accounts: list[str] | None` - Any account addresses to add to the accounts array +- `foreign_apps: list[int] | None` - The ID of any apps to load to the foreign apps array +- `foreign_assets: list[int] | None` - The ID of any assets to load to the foreign assets array +- `boxes: list[BoxReference | BoxIdentifier] | None` - Any boxes to load to the boxes array + +When making an ABI call, the `args` parameter is replaced with ABI-specific arguments and there is also a `method` parameter: + +- `method: ABIMethod` +- `args: list[ABIArgument]` - The arguments to pass to the ABI call, which can be one of: + - `ABIValue` - Which can be one of: + - `bool` + - `int` + - `str` + - `bytes` + - A list of one of the above types + - `Transaction` + - `TransactionWithSigner` + +## Box references + +A box can be referenced by either a `BoxIdentifier` (which identifies the name of the box and app ID `0` will be used - i.e. the current app) or `BoxReference`: + +```python +# BoxIdentifier can be: +# * bytes (the actual binary of the box name) +# * str (that will be encoded to bytes) +# * AccountTransactionSigner (that will be encoded into the public key address) +BoxIdentifier = str | bytes | AccountTransactionSigner + +# BoxReference is a class with: +# * app_id: int - A unique application id +# * name: BoxIdentifier - Identifier for a box name +BoxReference = BoxReference +``` + +## Compilation + +The `AppManager` class allows you to compile TEAL code with caching semantics that allows you to avoid duplicate compilation and keep track of source maps from compiled code. + +If you call `app_manager.compile_teal(teal_code)` then the compilation result will be stored and retrievable from `app_manager.get_compilation_result(teal_code)`. + +```python +teal_code = 'return 1' +compilation_result = app_manager.compile_teal(teal_code) +# ... +previous_compilation_result = app_manager.get_compilation_result(teal_code) +``` + +### Template compilation + +The `AppManager` also supports compiling TEAL templates with variables and deployment metadata: + +```python +compilation_result = app_manager.compile_teal_template( + teal_template_code, + template_params={"VAR1": "value1"}, + deployment_metadata={"updatable": True, "deletable": True} +) +``` diff --git a/docs/source/capabilities/client.md b/docs/source/capabilities/client.md index 851c66ff..08614682 100644 --- a/docs/source/capabilities/client.md +++ b/docs/source/capabilities/client.md @@ -1,29 +1,107 @@ # Client management -Client management is one of the core capabilities provided by AlgoKit Utils. -It allows you to create [algod](https://developer.algorand.org/docs/rest-apis/algod), [indexer](https://developer.algorand.org/docs/rest-apis/indexer) -and [kmd](https://developer.algorand.org/docs/rest-apis/kmd) clients against various networks resolved from environment or specified configuration. +Client management is one of the core capabilities provided by AlgoKit Utils. It allows you to create (auto-retry) [algod](https://developer.algorand.org/docs/rest-apis/algod), [indexer](https://developer.algorand.org/docs/rest-apis/indexer) and [kmd](https://developer.algorand.org/docs/rest-apis/kmd) clients against various networks resolved from environment or specified configuration. -Any AlgoKit Utils function that needs one of these clients will take the underlying `algosdk` classes (`algosdk.v2client.algod.AlgodClient`, `algosdk.v2client.indexer.IndexerClient`, -`algosdk.kmd.KMDClient`) so inline with the [Modularity](../index.md#core-principles) principle you can use existing logic to get instances of these clients without needing to use the -Client management capability if you prefer. +Any AlgoKit Utils function that needs one of these clients will take the underlying algosdk classes (`algosdk.v2client.algod.AlgodClient`, `algosdk.v2client.indexer.IndexerClient`, `algosdk.kmd.KMDClient`) so inline with the [Modularity](../index.md#core-principles) principle you can use existing logic to get instances of these clients without needing to use the Client management capability if you prefer. To see some usage examples check out the [automated tests](https://github.com/algorandfoundation/algokit-utils-py/blob/main/tests/test_network_clients.py). +## `ClientManager` + +The `ClientManager` is a class that is used to manage client instances. + +To get an instance of `ClientManager` you can instantiate it directly: + +```python +from algokit_utils import ClientManager + +# Algod client only +client_manager = ClientManager(algod=algod_client) +# All clients +client_manager = ClientManager(algod=algod_client, indexer=indexer_client, kmd=kmd_client) +# Algod config only +client_manager = ClientManager(algod_config=algod_config) +# All client configs +client_manager = ClientManager(algod_config=algod_config, indexer_config=indexer_config, kmd_config=kmd_config) +``` + ## Network configuration -The network configuration is specified using the `AlgoClientConfig` class. This same interface is used to specify the config for algod, indexer and kmd clients. +The network configuration is specified using the `AlgoClientConfig` type. This same type is used to specify the config for [algod](https://developer.algorand.org/docs/sdks/python/), [indexer](https://developer.algorand.org/docs/sdks/python/) and [kmd](https://developer.algorand.org/docs/sdks/python/) SDK clients. There are a number of ways to produce one of these configuration objects: -- Manually creating the object, e.g. `AlgoClientConfig(server="https://myalgodnode.com", token="SECRET_TOKEN")` -- `algokit_utils.get_algonode_config(network, config, token)`: Loads an Algod or indexer config against [Nodely](https://nodely.io/docs/free/start) to either MainNet or TestNet -- `algokit_utils.get_default_localnet_config(configOrPort)`: Loads an Algod, Indexer or Kmd config against [LocalNet](https://github.com/algorandfoundation/algokit-cli/blob/main/docs/features/localnet.md) using the default configuration +- Manually specifying a dictionary that conforms with the type, e.g. + ```python + { + "server": "https://myalgodnode.com" + } + # Or with the optional values: + { + "server": "https://myalgodnode.com", + "port": 443, + "token": "SECRET_TOKEN" + } + ``` +- `ClientManager.get_config_from_environment_or_localnet()` - Loads the Algod client config, the Indexer client config and the Kmd config from well-known environment variables or if not found then default LocalNet; this is useful to have code that can work across multiple blockchain environments (including LocalNet), without having to change +- `ClientManager.get_algod_config_from_environment()` - Loads an Algod client config from well-known environment variables +- `ClientManager.get_indexer_config_from_environment()` - Loads an Indexer client config from well-known environment variables; useful to have code that can work across multiple blockchain environments (including LocalNet), without having to change +- `ClientManager.get_algonode_config(network)` - Loads an Algod or indexer config against [AlgoNode free tier](https://nodely.io/docs/free/start) to either MainNet or TestNet +- `ClientManager.get_default_localnet_config()` - Loads an Algod, Indexer or Kmd config against [LocalNet](https://github.com/algorandfoundation/algokit-cli/blob/main/docs/features/localnet.md) using the default configuration ## Clients -Once you have the configuration for a client, to get the client you can use the following functions: +### Creating an SDK client instance + +Once you have the configuration for a client, to get a new client you can use the following functions: + +- `ClientManager.get_algod_client(config)` - Returns an Algod client for the given configuration; the client automatically retries on transient HTTP errors +- `ClientManager.get_indexer_client(config)` - Returns an Indexer client for given configuration +- `ClientManager.get_kmd_client(config)` - Returns a Kmd client for the given configuration + +You can also shortcut needing to write the likes of `ClientManager.get_algod_client(ClientManager.get_algod_config_from_environment())` with environment shortcut methods: + +- `ClientManager.get_algod_client_from_environment()` - Returns an Algod client by loading the config from environment variables +- `ClientManager.get_indexer_client_from_environment()` - Returns an indexer client by loading the config from environment variables +- `ClientManager.get_kmd_client_from_environment()` - Returns a kmd client by loading the config from environment variables + +### Accessing SDK clients via ClientManager instance + +Once you have a `ClientManager` instance, you can access the SDK clients: + +```python +client_manager = ClientManager(algod=algod_client, indexer=indexer_client, kmd=kmd_client) + +algod_client = client_manager.algod +indexer_client = client_manager.indexer +kmd_client = client_manager.kmd +``` + +If the method to create the `ClientManager` doesn't configure indexer or kmd (both of which are optional), then accessing those clients will trigger an error. + +### Creating a TestNet dispenser API client instance + +You can also create a [TestNet dispenser API client instance](./dispenser-client.md) from `ClientManager` too. + +## Automatic retry + +When receiving an Algod or Indexer client from AlgoKit Utils, it will be a special wrapper client that handles retrying transient failures. + +## Network information + +You can get information about the current network you are connected to: + +```python +# Get network information +network = client_manager.network() +print(f"Connected to: {network.name}") # e.g., "mainnet", "testnet", "localnet" +print(f"Genesis ID: {network.genesis_id}") +print(f"Genesis hash: {network.genesis_hash}") + +# Check specific network types +is_mainnet = client_manager.is_mainnet() +is_testnet = client_manager.is_testnet() +is_localnet = client_manager.is_localnet() +``` -- `algokit_utils.get_algod_client(config)`: Returns an Algod client for the given configuration or if none is provided retrieves a configuration from the environment using `ALGOD_SERVER`, `ALGOD_TOKEN` and optionally `ALGOD_PORT`. -- `algokit_utils.get_indexer_client(config)`: Returns an Indexer client for given configuration or if none is provided retrieves a configuration from the environment using `INDEXER_SERVER`, `INDEXER_TOKEN` and optionally `INDEXER_PORT` -- `algokit_utils.get_kmd_client_from_algod_client(config)`: - Returns a Kmd client based on the provided algod client configuration, with the assumption the KMD services is running on the same host but a different port (either `KMD_PORT` environment variable or `4002` by default) +The first time `network()` is called it will make a HTTP call to algod to get the network parameters, but from then on it will be cached within that `ClientManager` instance for subsequent calls. diff --git a/docs/source/capabilities/debugger.md b/docs/source/capabilities/debugger.md index 96ff7d22..85b87c22 100644 --- a/docs/source/capabilities/debugger.md +++ b/docs/source/capabilities/debugger.md @@ -4,43 +4,75 @@ The AlgoKit Python Utilities package provides a set of debugging tools that can ## Configuration -The `config.py` file contains the `UpdatableConfig` class which manages and updates configuration settings for the AlgoKit project. The class has the following attributes: - -- `debug`: Indicates whether debug mode is enabled. -- `project_root`: The path to the project root directory. Can be ignored if you are using `algokit_utils` inside an `algokit` compliant project (containing `.algokit.toml` file). For non algokit compliant projects, simply provide the path to the folder where you want to store sourcemaps and traces to be used with [`AlgoKit AVM Debugger`](https://github.com/algorandfoundation/algokit-avm-vscode-debugger). Alternatively you can also set the value via the `ALGOKIT_PROJECT_ROOT` environment variable. -- `trace_all`: Indicates whether to trace all operations. Defaults to false, this means that when debug mode is enabled, any (or all) application client calls performed via `algokit_utils` will store responses from `simulate` endpoint. These files are called traces, and can be used with `AlgoKit AVM Debugger` to debug TEAL source codes, transactions in the atomic group and etc. -- `trace_buffer_size_mb`: The size of the trace buffer in megabytes. By default uses 256 megabytes. When output folder containing debug trace files exceedes the size, oldest files are removed to optimize for storage consumption. -- `max_search_depth`: The maximum depth to search for a an `algokit` config file. By default it will traverse at most 10 folders searching for `.algokit.toml` file which will be used to assume algokit compliant project root path. - -The `configure` method can be used to set these attributes. +The `config.py` file contains the `UpdatableConfig` class which manages and updates configuration settings for the AlgoKit project. To enable debug mode in your project you can configure it as follows: -```py +```python from algokit_utils.config import config config.configure(debug=True) ``` -## Debugging Utilities +## Configuration Options + +The `UpdatableConfig` class provides several configuration options that affect debugging behavior: + +- `debug` (bool): Indicates whether debug mode is enabled. +- `project_root` (Path | None): The path to the project root directory. Can be ignored if you are using `algokit_utils` inside an `algokit` compliant project (containing `.algokit.toml` file). For non algokit compliant projects, simply provide the path to the folder where you want to store sourcemaps and traces to be used with AlgoKit AVM Debugger. Alternatively you can also set the value via the `ALGOKIT_PROJECT_ROOT` environment variable. +- `trace_all` (bool): Indicates whether to trace all operations. Defaults to false, this means that when debug mode is enabled, any (or all) application client calls performed via `algokit_utils` will store responses from `simulate` endpoint. +- `trace_buffer_size_mb` (float): The size of the trace buffer in megabytes. By default uses 256 megabytes. When output folder containing debug trace files exceedes the size, oldest files are removed to optimize for storage consumption. +- `max_search_depth` (int): The maximum depth to search for an `algokit` config file. By default it will traverse at most 10 folders searching for `.algokit.toml` file which will be used to assume algokit compliant project root path. -Debugging utilities can be used to simplify gathering artifacts to be used with [AlgoKit AVM Debugger](https://github.com/algorandfoundation/algokit-avm-vscode-debugger) in non algokit compliant projects. The following methods are provided: +You can configure these options as follows: -- `persist_sourcemaps`: This method persists the sourcemaps for the given sources as AVM Debugger compliant artifacts. It takes a list of `PersistSourceMapInput` objects, a `Path` object representing the root directory of the project, an `AlgodClient` object for interacting with the Algorand blockchain, and a boolean indicating whether to dump teal source files along with sourcemaps. -- `simulate_and_persist_response`: This method simulates the atomic transactions using the provided `AtomicTransactionComposer` object and `AlgodClient` object, and persists the simulation response to an AVM Debugger compliant JSON file. It takes an `AtomicTransactionComposer` object representing the atomic transactions to be simulated and persisted, a `Path` object representing the root directory of the project, an `AlgodClient` object representing the Algorand client, and a float representing the size of the trace buffer in megabytes. +```python +config.configure( + debug=True, + project_root=Path("./my-project"), + trace_all=True, + trace_buffer_size_mb=512, + max_search_depth=15 +) +``` + +## Debugging Utilities + +When debug mode is enabled, AlgoKit Utils will automatically: + +1. Store simulation traces for transactions that fail (by default) or all transactions (if `trace_all=True`) +2. Save these traces in the `debug_traces` folder within your project root +3. Manage the trace buffer size to prevent excessive disk usage + +The following methods are provided for scenarios where you want to manually persist sourcemaps and traces: + +- `persist_sourcemaps`: This method persists the sourcemaps for the + given sources as AVM Debugger compliant artifacts. It takes a list of + `PersistSourceMapInput` objects, a `Path` object representing the root + directory of the project, an `AlgodClient` object for interacting with the + Algorand blockchain, and a boolean indicating whether to dump teal source + files along with sourcemaps. +- `simulate_and_persist_response`: This method simulates the atomic + transactions using the provided `AtomicTransactionComposer` object and + `AlgodClient` object, and persists the simulation response to an AVM + Debugger compliant JSON file. It takes an `AtomicTransactionComposer` + object representing the atomic transactions to be simulated and persisted, + a `Path` object representing the root directory of the project, an + `AlgodClient` object representing the Algorand client, and a float + representing the size of the trace buffer in megabytes. ### Trace filename format The trace files are named in a specific format to provide useful information about the transactions they contain. The format is as follows: -```ts -`${timestamp}_lr${last_round}_${transaction_types}.trace.avm.json`; +``` +${timestamp}_lr${last_round}_${transaction_types}.trace.avm.json ``` Where: -- `timestamp`: The time when the trace file was created, in ISO 8601 format, with colons and periods removed. -- `last_round`: The last round when the simulation was performed. -- `transaction_types`: A string representing the types and counts of transactions in the atomic group. Each transaction type is represented as `${count}#${type}`, and different transaction types are separated by underscores. +- `timestamp`: The time when the trace file was created, in ISO 8601 format, with colons and periods removed +- `last_round`: The last round when the simulation was performed +- `transaction_types`: A string representing the types and counts of transactions in the atomic group. Each transaction type is represented as `${count}#${type}`, and different transaction types are separated by underscores For example, a trace file might be named `20220301T123456Z_lr1000_2#pay_1#axfer.trace.avm.json`, indicating that the trace file was created at `2022-03-01T12:34:56Z`, the last round was `1000`, and the atomic group contained 2 payment transactions and 1 asset transfer transaction.