Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add gas handling doc, benchmark updates, few fixes #188

Merged
merged 2 commits into from
May 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 13 additions & 33 deletions doc/final-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,11 @@ Scripts and modules have limited access to the balance transfer functionality vi
```rust
/// Execute Move script transaction sent by the user.
#[pallet::call_index(0)]
#[pallet::weight(T::WeightInfo::execute())]
#[pallet::weight(T::WeightInfo::execute(*gas_limit))]
pub fn execute(
origin: OriginFor<T>,
transaction_bc: Vec<u8>,
gas_limit: u64,
gas_limit: u32,
cheque_limit: BalanceOf<T>,
) -> DispatchResultWithPostInfo;
```
Expand All @@ -34,31 +34,33 @@ Scripts and modules have limited access to the balance transfer functionality vi
/// Publish a Move module sent by the user.
/// Module is published under its sender's address.
#[pallet::call_index(1)]
#[pallet::weight(T::WeightInfo::publish_module())]
#[pallet::weight(T::WeightInfo::publish_module(*gas_limit))]
pub fn publish_module(
origin: OriginFor<T>,
bytecode: Vec<u8>,
gas_limit: u64,
gas_limit: u32,
) -> DispatchResultWithPostInfo;
```

```rust
/// Publish a Move bundle sent by the user.
///
/// Bundle is just a set of multiple modules.
#[pallet::call_index(2)]
#[pallet::weight(T::WeightInfo::publish_module_bundle())]
#[pallet::weight(T::WeightInfo::publish_module_bundle(*gas_limit))]
pub fn publish_module_bundle(
origin: OriginFor<T>,
bundle: Vec<u8>,
gas_limit: u64,
gas_limit: u32,
) -> DispatchResultWithPostInfo;
```

```rust
/// Publish a standard library bundle, e.g. Move-Stdlib or Substrate-Stdlib. Sudo user only.
///
/// It should be used carefully - and should not introduce backward, incompatible changes.
/// All standard libraries are published at their default address 0x1.
#[pallet::call_index(3)]
#[pallet::weight(T::WeightInfo::update_stdlib())]
#[pallet::weight(T::WeightInfo::update_stdlib_bundle())]
pub fn update_stdlib_bundle(
origin: OriginFor<T>,
stdlib: Vec<u8>,
Expand All @@ -67,30 +69,8 @@ Scripts and modules have limited access to the balance transfer functionality vi

## RPC

### Method `mvm_gasToWeight`
Convert gas to weight.

**Parameters**

`gas: u64` - Amount of gas.

`at: Option<BlockHash>` - Optional block.

----------------------------------------------------------------

### Method `mvm_weightToGas`
Convert weight to gas.

**Parameters**

`weight: Weight` - Amount of weigth.

`at: Option<BlockHash>` - Optional block.

----------------------------------------------------------------

### Method `mvm_estimateGasPublishModule`
Estimate gas for publishing a module.
Estimate gas and weight cost for publishing a module.

**Parameters**

Expand All @@ -103,7 +83,7 @@ Estimate gas for publishing a module.
----------------------------------------------------------------

### Method `mvm_estimateGasPublishBundle`
Estimate gas for publishing a bundle.
Estimate gas and weight cost for publishing a bundle.

**Parameters**

Expand All @@ -116,7 +96,7 @@ Estimate gas for publishing a bundle.
----------------------------------------------------------------

### Method `mvm_estimateGasExecuteScript`
Estimate gas for executing a Move script.
Estimate gas and weight cost for executing a Move script.

**Parameters**

Expand Down
53 changes: 53 additions & 0 deletions doc/gas-handling.md
Rqnsom marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Gas Handling and Weight Costs in Pallet-Move

The gas unit is an internal processing fee for publishing modules and executing scripts within the MoveVM.

In Substrate-based blockchains, another unit, `Weight`, measures and manages the time it takes to validate a block.
Substrate defines one unit of weight as one picosecond of execution time on reference hardware.

## Gas Handling in Move Layer

From within the MoveVM, there are two different sources for gas costs:
- Cost for script execution,
- Cost for publishing modules and bundles.

Script execution can broken down into:
- Gas cost per bytecode instruction - bytecode gas cost table can be found [here](https://github.com/eigerco/substrate-move/blob/d33762e0ec9ee82ca71d7c5991247eec135666d1/move-vm-backend-common/src/gas_schedule.rs#L39)
- Gas cost per native function call with variable gas cost depending on the input function arguments - native gas cost table can be found [here](https://github.com/eigerco/substrate-move/blob/d33762e0ec9ee82ca71d7c5991247eec135666d1/move-vm-backend-common/src/gas_schedule.rs#L166)

It is possible to modify the table values to control the cost of script execution.
It is recommended that both tables be scaled by the same factor to preserve the proportional cost relationship between native functions and bytecode instructions.
These table values are less dependent on storage access and are more related to raw processing time.

The cost for publishing modules and bundles is independently defined from the script execution costs and it can be controlled by modifying [gas-cost-per-byte constant](https://github.com/eigerco/substrate-move/blob/d33762e0ec9ee82ca71d7c5991247eec135666d1/move-vm-backend-common/src/gas_schedule.rs#L23).
This value was selected during the testing and benchmarks and it is an arbitrary choice specifically for the MoveVM instance running within the Substrate runtime.

All internal MoveVM gas handling costs are defined in the same [gas schedule module](https://github.com/eigerco/substrate-move/blob/main/move-vm-backend-common/src/gas_schedule.rs) so that tweaking any gas-related factor can be done from within that module.

## Extrinsic Weight Cost in Pallet Layer

Three main extrinsics interact with MoveVM, which stores its state within the Substrate storage:
- `publish_module`
- `publish_module_bundle`
- `execute`

All above extrinsics have a `gas_limit` argument which is used as an input to the `MoveVM`.
Since the required minimum `gas_limit` can vary a lot between different extrinisic calls, the extrinsic weight cost formula also heavily depends on the `gas_limit` on top of the fixed base extrinisic cost.
That means that using more than the necessary amount of `gas_limit` leads to an unnecessarily more costly extrinsic.
For that reason, it is recommended to use `smove node rpc` estimation RPC commands which provide data about:
- the minimum required `gas_limit` to execute/publish a given script/module,
- an estimated weight cost for the extrinsic call for the given `gas_limit`.

Previously executed Substrate benchmarks define conversion between gas limit and weight and can be found in the auto-generated file here.
To better understand the costs, it's best to use the estimation RPC methods.

> [!NOTE]
> The same amount of `gas_limit` between different extrinsic doesn't necessarily add the equal weight cost.

Future pallet users should monitor and observe gas handling costs once the pallet gets integrated into the actual blockchain and then recalibrate it according to their needs, if necessary, according to the info above.

[move-stdlib]: https://github.com/eigerco/substrate-move/tree/main/language/move-stdlib
[substrate-move]: https://github.com/eigerco/substrate-move
[substrate-stdlib]: https://github.com/eigerco/substrate-stdlib
[native-fn]: https://move-language.github.io/move/functions.html?highlight=native#native-functions
[smove]: https://github.com/eigerco/smove
16 changes: 10 additions & 6 deletions doc/stdlib-doc.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Move Standard Library In Pallet-Move

The standard library provides a set of common modules that should be available to every Move developer from the genesis block.
All modules within the standard library are published under the special account address 0x01 in the Move storage.
All modules within the standard library are published under the special account address `0x01` in the Move storage.

The standard library is contained in two repositories:
- [`move-stdlib`](move-stdlib) - commonly known set of standard library modules that can be found across all Move compatible ecosystems.
Expand All @@ -15,13 +15,13 @@ Any new modules can be easily added once the end goal for the pallet user is cle
In the following chapters, one can find a set of instructions that can be used to update the standard library when there is a clear need to do so.

Before we delve into the next two chapters, we should mention the special kind of functions called [native functions](native-fn).
These functions dont have a regular function body specified in Move source files like normal functions have, but instead, their implementation is integrated within the MoveVM itself.
These functions don't have a regular function body specified in Move source files like normal functions have, but instead, their implementation is integrated within the MoveVM itself.
Native functions are only allowed in standard libraries, as updating them requires updating the MoveVM source code itself.

## Updates adding new Move source code modules

Adding new modules to the standard library is pretty simple.
Add a module in the wanted repository and then build it as a bundle with `smove bundle` and also generate doc files with `smove docgen`.
Add a module to the wanted repository and then build it as a bundle with `smove bundle` and also generate doc files with `smove docgen`.

If the standard library needs to be updated in post-genesis block settings, the `update_stdlib_bundle` extrinsic can achieve this.
Note: This extrinsic can be used only by the root user.
Expand All @@ -32,12 +32,16 @@ A simple example can be found in this [pull request](https://github.com/eigerco/

Adding new native functions to the standard library requires the implementation of a function body in Rust within the MoveVM code.

Feel free to use this PR as an example of how that can be done.
- TODO here
You can use these PRs as an example of how that can be done:
- in [`substrate-move`](substrate-move): [feat: add native functions for substrate_hash module](https://github.com/eigerco/substrate-move/pull/83)
- in [`substrate-stdlib`](substrate-stdlib): [feat: add new hash module with native functions](https://github.com/eigerco/substrate-stdlib/pull/4)

Hopefully, pallet users won’t need to add new native functions after the genesis block, otherwise the node's runtime will require and update containing the new version of the MoveVM instance with the latest code that includes added native functions.
Whenever native functions are added, [smove](smove) needs to update dependencies so it fetches the latest MoveVM changes and then a new version of `smove` should be published.

Hopefully, pallet users won't need to add new native functions after the genesis block, otherwise the node's runtime will require an update containing the new version of the MoveVM instance with the latest code that includes added native functions.

[move-stdlib]: https://github.com/eigerco/substrate-move/tree/main/language/move-stdlib
[substrate-move]: https://github.com/eigerco/substrate-move
[substrate-stdlib]: https://github.com/eigerco/substrate-stdlib
[native-fn]: https://move-language.github.io/move/functions.html?highlight=native#native-functions
[smove]: https://github.com/eigerco/smove
10 changes: 10 additions & 0 deletions pallet/src/assets/move-projects/basic_coin/Move.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[package]
name = "basic_coin"
version = "0.0.0"

[dependencies]
MoveStdlib = { git = "https://github.com/eigerco/move-stdlib", rev = "main" }

[addresses]
std = "0x1"
CafeAccount = "5FHneW46xGXgs5mUiveU4sbTyGBzmstUspZC92UhjJM694ty"
68 changes: 68 additions & 0 deletions pallet/src/assets/move-projects/basic_coin/sources/BasicCoin.move
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
/// This module defines a minimal Coin and Balance.
module CafeAccount::BasicCoin {
use std::signer;

/// Address of the owner of this module
const MODULE_OWNER: address = @CafeAccount;

/// Error codes
const ENOT_MODULE_OWNER: u64 = 0;
const EINSUFFICIENT_BALANCE: u64 = 1;
const EALREADY_HAS_BALANCE: u64 = 2;

struct Coin has store {
value: u64
}

/// Struct representing the balance of each address.
struct Balance has key {
coin: Coin
}

/// Publish an empty balance resource under `account`'s address. This function must be called before
/// minting or transferring to the account.
/// Making this function entry allows to call it directly in the transactions
entry public fun publish_balance(account: &signer) {
let empty_coin = Coin { value: 0 };
assert!(!exists<Balance>(signer::address_of(account)), EALREADY_HAS_BALANCE);
move_to(account, Balance { coin: empty_coin });
}

/// Mint `amount` tokens to `mint_addr`. Mint must be approved by the module owner.
public fun mint(module_owner: &signer, mint_addr: address, amount: u64) acquires Balance {
// Only the owner of the module can initialize this module
assert!(signer::address_of(module_owner) == MODULE_OWNER, ENOT_MODULE_OWNER);

// Deposit `amount` of tokens to `mint_addr`'s balance
deposit(mint_addr, Coin { value: amount });
}

/// Returns the balance of `owner`.
public fun balance_of(owner: address): u64 acquires Balance {
borrow_global<Balance>(owner).coin.value
}

/// Transfers `amount` of tokens from `from` to `to`.
public fun transfer(from: &signer, to: address, amount: u64) acquires Balance {
let check = withdraw(signer::address_of(from), amount);
deposit(to, check);
}

/// Withdraw `amount` number of tokens from the balance under `addr`.
fun withdraw(addr: address, amount: u64) : Coin acquires Balance {
let balance = balance_of(addr);
// balance must be greater than the withdraw amount
assert!(balance >= amount, EINSUFFICIENT_BALANCE);
let balance_ref = &mut borrow_global_mut<Balance>(addr).coin.value;
*balance_ref = balance - amount;
Coin { value: amount }
}

/// Deposit `amount` number of tokens to the balance under `addr`.
fun deposit(addr: address, check: Coin) acquires Balance {
let balance = balance_of(addr);
let balance_ref = &mut borrow_global_mut<Balance>(addr).coin.value;
let Coin { value } = check;
*balance_ref = balance + value;
}
}
15 changes: 15 additions & 0 deletions pallet/src/assets/move-projects/basic_coin/sources/GetCoin.move
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
script {
use CafeAccount::BasicCoin;

fun publish_balance(s: signer) {
BasicCoin::publish_balance(&s);
}
}

script {
use CafeAccount::BasicCoin;

fun mint_some(module_owner: signer, rx_addr: address, amount: u64) {
BasicCoin::mint(&module_owner, rx_addr, amount);
}
}
2 changes: 1 addition & 1 deletion pallet/src/assets/move-projects/car-wash-example/build.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/bin/sh
cd $(dirname $0)
# Build the project
smove build
smove bundle
# Create all Script-Transactions
smove create-transaction --compiled-script-path build/car-wash-example/bytecode_scripts/initial_coin_minting.mv --args signer:5FHneW46xGXgs5mUiveU4sbTyGBzmstUspZC92UhjJM694ty
smove create-transaction --compiled-script-path build/car-wash-example/bytecode_scripts/register_new_user.mv --args signer:5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY
Expand Down
2 changes: 1 addition & 1 deletion pallet/src/assets/move-projects/multiple-signers/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ ALICE=5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY
DAVE=5DAAnrj7VHTznn2AWBemMuyBwZWs6FNFjdyVXUeYum3PTXFy
EVE=5HGjWAeFDfFCWPsjFQdVV2Msvz2XtMktvgocEZcCj68kUMaw
# Build the Move sources.
smove build
smove bundle
# Generate script-transactions.
# 1. init_module(Bob)
smove create-transaction --compiled-script-path build/multiple-signers/bytecode_scripts/init_module.mv --args signer:$BOB
Expand Down
18 changes: 16 additions & 2 deletions pallet/src/assets/move-projects/smove-build-all.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,21 @@ build_dir=(
"signer-scripts"
)
bundle_dir=(
"developer-bundle",
"prohibited-bundle"
"testing-move-stdlib"
"testing-substrate-stdlib"
"using_stdlib_natives"

"car-wash-example"
"multiple-signers"
"base58_smove_build"
"basic_coin"
)


# Info for the below actions:
# If the script needs to be bundled, it should be done in build.sh script if one exists.

# Build simple packages
for dir in "${build_dir[@]}"; do
echo $dir
Expand All @@ -36,5 +44,11 @@ done
# Build bundles
for dir in "${bundle_dir[@]}"; do
echo $dir
smove bundle -p $dir
build_script=$dir"/build.sh"
if [ -f "$build_script" ];
then
sh $build_script
else
smove bundle -p $dir
fi
done
7 changes: 6 additions & 1 deletion pallet/src/balance.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
//! TODO Short introduction of this module
//! Module for handling balance functionality related to the MoveVM balance module.
//!
//! Using the MoveVM it's possible to execute balance changes, but only if the execution script
//! succeeds.
//! The possible balance changes are capped with each user's `cheque_limit` value.
use core::{cmp::Ordering, marker::PhantomData};

use codec::{Decode, Encode};
Expand Down
Loading
Loading