feat: Support injection of ethereum logs for transactions.

[fairax]

Ethereum logs could be injected for currently running Ethereum transactions or without running an Ethereum transaction.
It could be used to emit ERC721 events while executing substrate transaction for example.

[tgmichel]

Switching from on memory to storage based evm logs has impact on transaction throughput, both on execution time and pov size. Also, logs data is unbounded, we cannot store it on events (or CurrentLogs for that matter, if that happens to be read somewhere in the STF, that unbounded data needs to be fully available in the merkle proof).

[tgmichel]

This is not good, for 375 gas you can do 2 db writes.

[fairax]

There is an assertion in on_finalize that asserts that CurrentLogs value is not stored at the end of the block; it never leaves memory.
It also affects execution time only a little, as DbWeight measures actual writes, which don't happen there.

The only real overhead is: checking CurrentLogs storage the first time it is accessed, wasm-to-host memory copy on append call, and host-to-wasm copy on the transaction end.

Also, unbounded log data is a problem with the current approach too.

[tgmichel]

There is an assertion in on_finalize that asserts that CurrentLogs value is not stored at the end of the block; it never leaves memory.

Ah I see missed that part, sounds good then. Still, there are projects not using pallet-ethereum that will pull this change from the Runner, so the CurrentLogs assertion needs to happen in pallet-evm's on_finalize.

[tgmichel]

Impl wise LGTM but needs to be thoroughly tested

[sorpaas]

Is this supposed to go into production runtime? If not, please add a feature gate.

[fairax]

It is for production. You can see use case here https://github.com/UniqueNetwork/unique-chain/blob/b86b27d80045514c2aa82c2d28b4251b17fc71b2/pallets/nonfungible/src/lib.rs#LL524C22-L524C22. Substrate call is emitting both a substrate and an ethereum events.

[sorpaas]

Have you considered implementing the burn function as a precompile? In that way you can avoid all those complicated and intrusive changes to inject fake transactions.

[CertainLach]

This function is callable from both precompile
https://github.com/UniqueNetwork/unique-chain/blob/develop/pallets/nonfungible/src/erc.rs#L555
And from extrinsic
https://github.com/UniqueNetwork/unique-chain/blob/develop/pallets/unique/src/lib.rs#L926
It is not always run in the precompile context.

When it is called from substrate side, we still want to see this Log on ethereum side, so we need to inject transaction anyway.

[boundless-forest]

@CertainLach Have you ever considered constructing a standalone ethereum transaction and invoke Pallet::<T>::apply_validated_transaction(source, transaction), for example: https://github.com/darwinia-network/darwinia/blob/main/pallet/message-transact/src/lib.rs#L125-L167

You can invoke apply inside your burn substrate call.

[CertainLach]

@boundless-forest
I don't understand how apply_validated_transaction will help us in this case.

In our chain, NFT/Fungible/Refungible collections are implemented in pallets, and those pallets are exposed as extrinsics.
However, we provide an EVM compatibility layer: every native collection is also represented as a evm contract.

Thus, both substrate users and ethereum users/contracts may manipulate tokens.

We want all token manipulations to be represented as logs on ethereum side, both in the case of ethereum call and in the case of substrate extrinsic call.
The problem is, during plain extrinsic execution, there is no transaction receipt available on ethereum side to attach logs to, and we need to create one.
Or, in case this method was called during ethereum transaction execution, we want to attach this Log to the currently executing transaction.

The other problem this PR solves is substrate event order in case of precompile call.
Let's imagine this precompile:

    #[pallet::event]
    enum Event {
        TestEvent(i32),
    }

    fn precompile_test() {
        deposit_event(Event::TestEvent(2));
    }

   event TestEvent(int value);
   function test() {
       emit TestEvent(1);
       precompile.precompileTest();
       emit TestEvent(3);
   }

The expected order of events on the substrate side is

PalletEvm::Log(encoded(TestEvent(1)));
Precompile::TestEvent(2);
PalletEvm::Log(encoded(TestEvent(3)));

However, because of current event processing (They are stored in memory and only stored after evm execution: https://github.com/paritytech/frontier/blob/master/frame/evm/src/runner/stack.rs#L264), the resulting event order is

Precompile::TestEvent(2);
PalletEvm::Log(encoded(TestEvent(1)));
PalletEvm::Log(encoded(TestEvent(3)));

Also, this PR exposes events emitted during the pallet_evm::call extrinsic call as a fake transaction (Currently, they are only emitted as substrate events and not available on the ethereum side).

[boundless-forest]

I don't understand how apply_validated_transaction will help us in this case.

Your goal is to create Ethereum logs when the users call the paricular substrate extrinsics, and the log can be indexed later by ethereum receipt. Within the apply_validated_transaction, you can create an internal ethereum transaction, which has transaction receipt, will be executed like normal ethereum transactions. The internal transaction can call the dedicated precompile to record the Ethererum logs.

pallet call {
    let tx = Transaction {
        sender: xxx,
        to: 0x0000000000000000000000000123, // dedicated precompile
        value: 0,
        input: your log paramss,
    }
    Ethereum::apply_validated_transaction(sender, tx)?;
}

precompile {
    record_logs(input) {
        let logs = input.logs;
        record_logs(logs);
    }
}

[sorpaas]

If you don't want the overhead of running EVM, you can also just insert a fake transaction into Pending, like how it did in apply_validated_transaction. I would be more happy to merge a PR adding a function apply_fake_validated_transaction than this PR.

[sorpaas]

To fix the log order issue, you'll want to modify this function: https://github.com/paritytech/frontier/blob/master/frame/evm/src/runner/stack.rs#L565

Instead of inserting into the intermediate logs, simply log event directly.

Note that this only works in our specific case, as we use sp_io::transaction. In Ethereum, logs are always supposed to only get appended at the end of the transaction, because anywhere in the middle it can be rolled back.

[CertainLach]

Note that this only works in our specific case, as we use sp_io::transaction. In Ethereum, logs are always supposed to only get appended at the end of the transaction, because anywhere in the middle it can be rolled back.

They are handled the same way as storage access, so this PR unifies storage and log manipulations, they are now both reverted at the same time.

If you don't want the overhead of running EVM, you can also just insert a fake transaction into Pending, like how it did in apply_validated_transaction. I would be more happy to merge a PR adding a function apply_fake_validated_transaction than this PR.

Sometimes we are already in the context of precompile execution, as this function can be called from both substrate extrinsics and ethereum transactions. We can have Option<impl PrecompileHandle>, and either create a new transactions, or append logs to the existing handle, but this becomes a mess, quick.

You don't like automatic flushing done by SignedExtenstion, or the problem is in storing current logs in in-memory storage?

[icodezjb]

Will this be merged into the master branch? We are experiencing the same situation, any suggestions or reference code?