Skip to content

Commit

Permalink
Merge pull request #877 from ton-blockchain/testnet
Browse files Browse the repository at this point in the history
Merge developer branch
  • Loading branch information
EmelyanenkoK authored Jan 25, 2024
2 parents 9728bc6 + e459aea commit 8a9ff33
Show file tree
Hide file tree
Showing 31 changed files with 211 additions and 85 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/ton-x86-64-linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
run: |
cp assembly/nix/build-linux-x86-64-nix.sh .
chmod +x build-linux-x86-64-nix.sh
./build-linux-x86-64-nix.sh
./build-linux-x86-64-nix.sh -t
- name: Simple binaries test
run: |
Expand All @@ -38,4 +38,4 @@ jobs:
uses: actions/upload-artifact@master
with:
name: ton-x86_64-linux-binaries
path: artifacts
path: artifacts
4 changes: 2 additions & 2 deletions .github/workflows/ton-x86-64-macos.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
run: |
cp assembly/nix/build-macos-nix.sh .
chmod +x build-macos-nix.sh
./build-macos-nix.sh
./build-macos-nix.sh -t
- name: Simple binaries test
run: |
Expand All @@ -34,4 +34,4 @@ jobs:
uses: actions/upload-artifact@master
with:
name: ton-x86_64-macos-binaries
path: artifacts
path: artifacts
10 changes: 8 additions & 2 deletions Changelog.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,20 @@
## 2024.01 Update

1. Fixes in how gas in transactions on special accounts is accounted in block limit. Previously, gas was counted as usual, so to conduct elections that costs >30m gas block limit in masterchain was set to 37m gas. To lower the limit for safety reasons it is proposed to not count gas on special accounts. Besides `gas_max` is set to `special_gas_limit` for all types of transactions on special accounts. New behavior is activated through setting `gas_prices_v3` in `ConfigParam 20;`.
1. Fixes in how gas in transactions on special accounts is accounted in block limit. Previously, gas was counted as usual, so to conduct elections that costs >30m gas block limit in masterchain was set to 37m gas. To lower the limit for safety reasons it is proposed to caunt gas on special accounts separately. Besides `gas_max` is set to `special_gas_limit` for all types of transactions on special accounts. New behavior is activated through setting `version >= 5` in `ConfigParam 8;`.
* Besides update of config temporally increases gas limit on `EQD_v9j1rlsuHHw2FIhcsCFFSD367ldfDdCKcsNmNpIRzUlu` to `special_gas_limit`, see [details](https://t.me/tonstatus/88).
2. Improvements in LS behavior
* Improved detection of the state with all shards applied to decrease rate of `Block is not applied` error
* Better error logs: `block not in db` and `block is not applied` separation
* Fix error in proof generation for blocks after merge
* Fix most of `block is not applied` issues related to sending too recent block in Proofs
* LS now check external messages till `accept_message` (`set_gas`).
3. Improvements in DHT work and storage, CellDb, config.json ammendment, peer misbehavior detection, validator session stats collection, emulator.
4. Change in CTOS and XLOAD behavior activated through setting `version >= 5` in `ConfigParam 8;`:
* Loading "nested libraries" (i.e. a library cell that points to another library cell) throws an exception.
* Loading a library consumes gas for cell load only once (for the library cell), not twice (both for the library cell and the cell in the library).
* `XLOAD` now works differently. When it takes a library cell, it returns the cell that it points to. This allows loading "nested libraries", if needed.

Besides the work of the core team, this update is based on the efforts of @XaBbl4 (peer misbehavior detection).
Besides the work of the Core team, this update is based on the efforts of @XaBbl4 (peer misbehavior detection) and @akifoq (CTOS behavior and gas limit scheme for special accounts).

## 2023.12 Update

Expand Down
10 changes: 5 additions & 5 deletions assembly/cicd/jenkins/test-builds.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ pipeline {
sh '''
cp assembly/nix/build-linux-x86-64-nix.sh .
chmod +x build-linux-x86-64-nix.sh
./build-linux-x86-64-nix.sh
./build-linux-x86-64-nix.sh -t
'''
sh '''
cd artifacts
Expand Down Expand Up @@ -69,7 +69,7 @@ pipeline {
sh '''
cp assembly/nix/build-linux-arm64-nix.sh .
chmod +x build-linux-arm64-nix.sh
./build-linux-arm64-nix.sh
./build-linux-arm64-nix.sh -t
'''
sh '''
cd artifacts
Expand Down Expand Up @@ -107,7 +107,7 @@ pipeline {
sh '''
cp assembly/nix/build-macos-nix.sh .
chmod +x build-macos-nix.sh
./build-macos-nix.sh
./build-macos-nix.sh -t
'''
sh '''
cd artifacts
Expand Down Expand Up @@ -145,7 +145,7 @@ pipeline {
sh '''
cp assembly/nix/build-macos-nix.sh .
chmod +x build-macos-nix.sh
./build-macos-nix.sh
./build-macos-nix.sh -t
'''
sh '''
cd artifacts
Expand Down Expand Up @@ -233,4 +233,4 @@ pipeline {
}
}
}
}
}
19 changes: 17 additions & 2 deletions assembly/nix/build-linux-arm64-nix.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,27 @@
nix-build --version
test $? -eq 0 || { echo "Nix is not installed!"; exit 1; }

with_tests=false


while getopts 't' flag; do
case "${flag}" in
t) with_tests=true ;;
*) break
;;
esac
done

cp assembly/nix/linux-arm64* .
cp assembly/nix/microhttpd.nix .
cp assembly/nix/openssl.nix .
export NIX_PATH=nixpkgs=https://github.com/nixOS/nixpkgs/archive/23.05.tar.gz

nix-build linux-arm64-static.nix
if [ "$with_tests" = true ]; then
nix-build linux-arm64-static.nix --arg testing true
else
nix-build linux-arm64-static.nix
fi
mkdir artifacts
cp ./result/bin/* artifacts/
chmod +x artifacts/*
Expand All @@ -17,4 +32,4 @@ nix-build linux-arm64-tonlib.nix
cp ./result/lib/libtonlibjson.so.0.5 artifacts/libtonlibjson.so
cp ./result/lib/libemulator.so artifacts/
cp -r crypto/fift/lib artifacts/
cp -r crypto/smartcont artifacts/
cp -r crypto/smartcont artifacts/
20 changes: 18 additions & 2 deletions assembly/nix/build-linux-x86-64-nix.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,28 @@
nix-build --version
test $? -eq 0 || { echo "Nix is not installed!"; exit 1; }

with_tests=false


while getopts 't' flag; do
case "${flag}" in
t) with_tests=true ;;
*) break
;;
esac
done

cp assembly/nix/linux-x86-64* .
cp assembly/nix/microhttpd.nix .
cp assembly/nix/openssl.nix .
export NIX_PATH=nixpkgs=https://github.com/nixOS/nixpkgs/archive/23.05.tar.gz

nix-build linux-x86-64-static.nix
if [ "$with_tests" = true ]; then
nix-build linux-x86-64-static.nix --arg testing true
else
nix-build linux-x86-64-static.nix
fi

mkdir artifacts
cp ./result/bin/* artifacts/
chmod +x artifacts/*
Expand All @@ -17,4 +33,4 @@ nix-build linux-x86-64-tonlib.nix
cp ./result/lib/libtonlibjson.so.0.5 artifacts/libtonlibjson.so
cp ./result/lib/libemulator.so artifacts/
cp -r crypto/fift/lib artifacts/
cp -r crypto/smartcont artifacts/
cp -r crypto/smartcont artifacts/
20 changes: 18 additions & 2 deletions assembly/nix/build-macos-nix.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,25 @@
nix-build --version
test $? -eq 0 || { echo "Nix is not installed!"; exit 1; }

with_tests=false


while getopts 't' flag; do
case "${flag}" in
t) with_tests=true ;;
*) break
;;
esac
done

cp assembly/nix/macos-* .
export NIX_PATH=nixpkgs=https://github.com/nixOS/nixpkgs/archive/23.05.tar.gz
nix-build macos-static.nix

if [ "$with_tests" = true ]; then
nix-build macos-static.nix --arg testing true
else
nix-build macos-static.nix
fi
mkdir artifacts
cp ./result-bin/bin/* artifacts/
chmod +x artifacts/*
Expand All @@ -14,4 +30,4 @@ nix-build macos-tonlib.nix
cp ./result/lib/libtonlibjson.dylib artifacts/
cp ./result/lib/libemulator.dylib artifacts/
cp -r crypto/fift/lib artifacts/
cp -r crypto/smartcont artifacts/
cp -r crypto/smartcont artifacts/
3 changes: 2 additions & 1 deletion assembly/nix/linux-arm64-static.nix
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
{ pkgs ? import <nixpkgs> { system = builtins.currentSystem; }
, lib ? pkgs.lib
, stdenv ? pkgs.stdenv
, testing ? false
}:
let
microhttpdmy = (import ./microhttpd.nix) {};
Expand All @@ -25,7 +26,7 @@ stdenv.mkDerivation {
];

makeStatic = true;
doCheck = true;
doCheck = testing;

cmakeFlags = [
"-DTON_USE_ABSEIL=OFF"
Expand Down
3 changes: 2 additions & 1 deletion assembly/nix/linux-x86-64-static.nix
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
{ pkgs ? import <nixpkgs> { system = builtins.currentSystem; }
, lib ? pkgs.lib
, stdenv ? pkgs.stdenv
, testing ? false
}:
let
microhttpdmy = (import ./microhttpd.nix) {};
Expand All @@ -25,7 +26,7 @@ stdenv.mkDerivation {
];

makeStatic = true;
doCheck = true;
doCheck = testing;

cmakeFlags = [
"-DTON_USE_ABSEIL=OFF"
Expand Down
5 changes: 3 additions & 2 deletions assembly/nix/macos-static.nix
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
{ pkgs ? import <nixpkgs> { system = builtins.currentSystem; }
, lib ? pkgs.lib
, stdenv ? pkgs.stdenv
, testing ? false
}:

pkgs.llvmPackages_14.stdenv.mkDerivation {
Expand All @@ -29,7 +30,7 @@ pkgs.llvmPackages_14.stdenv.mkDerivation {

dontAddStaticConfigureFlags = true;
makeStatic = true;
doCheck = true;
doCheck = testing;

configureFlags = [];

Expand Down Expand Up @@ -62,4 +63,4 @@ pkgs.llvmPackages_14.stdenv.mkDerivation {
done
'';
outputs = [ "bin" "out" ];
}
}
2 changes: 1 addition & 1 deletion common/global-version.h
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,6 @@
namespace ton {

// See doc/GlobalVersions.md
const int SUPPORTED_VERSION = 4;
const int SUPPORTED_VERSION = 5;

}
5 changes: 0 additions & 5 deletions crypto/block/block.tlb
Original file line number Diff line number Diff line change
Expand Up @@ -696,11 +696,6 @@ gas_prices_ext#de gas_price:uint64 gas_limit:uint64 special_gas_limit:uint64 gas
block_gas_limit:uint64 freeze_due_limit:uint64 delete_due_limit:uint64
= GasLimitsPrices;

// same fields as gas_prices_ext; behavior differs
gas_prices_v3#df gas_price:uint64 gas_limit:uint64 special_gas_limit:uint64 gas_credit:uint64
block_gas_limit:uint64 freeze_due_limit:uint64 delete_due_limit:uint64
= GasLimitsPrices;

gas_flat_pfx#d1 flat_gas_limit:uint64 flat_gas_price:uint64 other:GasLimitsPrices
= GasLimitsPrices;

Expand Down
4 changes: 0 additions & 4 deletions crypto/block/mc-config.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -654,13 +654,9 @@ td::Result<GasLimitsPrices> Config::do_get_gas_limits_prices(td::Ref<vm::Cell> c
res.delete_due_limit = r.delete_due_limit;
};
block::gen::GasLimitsPrices::Record_gas_prices_ext rec;
block::gen::GasLimitsPrices::Record_gas_prices_v3 rec_v3;
vm::CellSlice cs0 = cs;
if (tlb::unpack(cs, rec)) {
f(rec, rec.special_gas_limit);
} else if (tlb::unpack(cs = cs0, rec_v3)) {
f(rec_v3, rec_v3.special_gas_limit);
res.special_full_limit = true;
} else {
block::gen::GasLimitsPrices::Record_gas_prices rec0;
if (tlb::unpack(cs = cs0, rec0)) {
Expand Down
1 change: 0 additions & 1 deletion crypto/block/mc-config.h
Original file line number Diff line number Diff line change
Expand Up @@ -349,7 +349,6 @@ struct GasLimitsPrices {
td::uint64 block_gas_limit{0};
td::uint64 freeze_due_limit{0};
td::uint64 delete_due_limit{0};
bool special_full_limit{false};

td::RefInt256 compute_gas_price(td::uint64 gas_used) const;
};
Expand Down
14 changes: 6 additions & 8 deletions crypto/block/transaction.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1039,12 +1039,8 @@ bool ComputePhaseConfig::parse_GasLimitsPrices_internal(Ref<vm::CellSlice> cs, t
delete_due_limit = td::make_refint(r.delete_due_limit);
};
block::gen::GasLimitsPrices::Record_gas_prices_ext rec;
block::gen::GasLimitsPrices::Record_gas_prices_v3 rec_v3;
if (tlb::csr_unpack(cs, rec)) {
f(rec, rec.special_gas_limit);
} else if (tlb::csr_unpack(cs, rec_v3)) {
f(rec_v3, rec_v3.special_gas_limit);
special_gas_full = true;
} else {
block::gen::GasLimitsPrices::Record_gas_prices rec0;
if (tlb::csr_unpack(std::move(cs), rec0)) {
Expand Down Expand Up @@ -1153,8 +1149,8 @@ namespace transaction {
* not enough to clean up old queires, thus locking funds inside.
* See comment in crypto/smartcont/highload-wallet-v2-code.fc for details on why this happened.
* Account address: EQD_v9j1rlsuHHw2FIhcsCFFSD367ldfDdCKcsNmNpIRzUlu
* It was proposed to validators to increase gas limit for this account for a limited amount of time (until 2024-02-16).
* It is activated by setting gas_prices_v3 in ConfigParam 20 (config_mc_gas_prices).
* It was proposed to validators to increase gas limit for this account for a limited amount of time (until 2024-02-29).
* It is activated by setting global version to 5 in ConfigParam 8.
* This config change also activates new behavior for special accounts in masterchain.
*
* @param cfg The compute phase configuration.
Expand All @@ -1164,10 +1160,10 @@ namespace transaction {
* @returns True if gas_limit override is required, false otherwise
*/
static bool override_gas_limit(const ComputePhaseConfig& cfg, ton::UnixTime now, const Account& account) {
if (!cfg.mc_gas_prices.special_full_limit) {
if (!cfg.special_gas_full) {
return false;
}
ton::UnixTime until = 1708041600; // 2024-02-16 00:00:00 UTC
ton::UnixTime until = 1709164800; // 2024-02-29 00:00:00 UTC
ton::WorkchainId wc = 0;
const char* addr_hex = "FFBFD8F5AE5B2E1C7C3614885CB02145483DFAEE575F0DD08A72C366369211CD";
return now < until && account.workchain == wc && account.addr.to_hex() == addr_hex;
Expand Down Expand Up @@ -1563,6 +1559,7 @@ bool Transaction::prepare_compute_phase(const ComputePhaseConfig& cfg) {
vm.set_global_version(cfg.global_version);
vm.set_c7(prepare_vm_c7(cfg)); // tuple with SmartContractInfo
vm.set_chksig_always_succeed(cfg.ignore_chksig);
vm.set_stop_on_accept_message(cfg.stop_on_accept_message);
// vm.incr_stack_trace(1); // enable stack dump after each step

LOG(DEBUG) << "starting VM";
Expand Down Expand Up @@ -3533,6 +3530,7 @@ td::Status FetchConfigParams::fetch_config_params(
TRY_RESULT_PREFIX(mc_gas_prices, config.get_gas_limits_prices(true),
"cannot unpack masterchain gas prices and limits: ");
compute_phase_cfg->mc_gas_prices = std::move(mc_gas_prices);
compute_phase_cfg->special_gas_full = config.get_global_version() >= 5;
storage_phase_cfg->enable_due_payment = config.get_global_version() >= 4;
compute_phase_cfg->block_rand_seed = *rand_seed;
compute_phase_cfg->max_vm_data_depth = size_limits.max_vm_data_depth;
Expand Down
1 change: 1 addition & 0 deletions crypto/block/transaction.h
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ struct ComputePhaseConfig {
std::unique_ptr<vm::Dictionary> suspended_addresses;
SizeLimitsConfig size_limits;
int vm_log_verbosity = 0;
bool stop_on_accept_message = false;

ComputePhaseConfig() : gas_price(0), gas_limit(0), special_gas_limit(0), gas_credit(0) {
compute_threshold();
Expand Down
Loading

0 comments on commit 8a9ff33

Please sign in to comment.