Reorg Async Backing

docs/learn/learn-async-backing.md

@@ -7,103 +7,179 @@ keywords: [parachains, slots, backing, parablock]
 slug: ../learn-async-backing
 ---
 
+import RPC from "./../../components/RPC-Connection";
+
+:::info Learn about Parachain Consensus
+
+To fully follow the material on this page, it is recommended to be familiar with the primary stages
+of the [Parachain Protocol](./learn-parachains-protocol.md).
+
+:::
+
+## Synchronous Backing
+
+Before diving into asynchronous backing it is important to understand what synchronous backing is
+and what its main limitations are.
+
+:::info What is backing?
+
+**Backing** refers to the process in which a parablock is verified by a subset of validators who
+determine the blocks validity. It is an important step in the validation process for parablocks, as
+it is the first line of defense in ensuring valid state transitions are taking place. Validators who
+back the parablock put their stake at risk, and if the block is found to be malicious, those
+validators are slashed.
+
+:::
+
+In synchronous backing, parablock validation is tightly coupled to the relay chain's progression on
+a one-to-one basis. Every parablock must be generated and backed within a relay-chain block
+(six-second window), and (if successfully backed) it will be included in a relay-chain block (often
+referred to as the **parent** block, as the parablock anchors itself to it) after an additional six
+seconds. Thus, a parablock can be produced every 12 seconds because a new parablock can be produced
+after the inclusion of the previous one.
+
+The **contextual execution** of new parablocks are parablock ancestors included in the relay chain
+after 12 seconds, where contextual execution refers to the context built by included parablock
+ancestors used by the parachains to generate new parablocks.
+
+:::info It is important to note the clarification below
+
+- Not the entire parablock, but the **paraheader** is placed on the parent block on the relay chain.
+- The relay chain does not access the entire state of a parachain but only the values that changed
+  during that block and the merkelized hashes of the unchanged values.
+
+:::
+
+### Synchronous Backing Diagram
+
+FIGURE HERE?
+
+A particular parablock, `P`, would only be valid for backing at relay chain parent `R` and
+subsequently included at `R + 1` should it be backed successfully. Thus, a parablock is rushing to
+be backed and included within a two-relay-chain-blocks window due to the inherent requirement for
+synchrony between the parachain and relay chain. The next parablock `P + 1` will be valid for
+backing at realy-chain block `R + 2`, i.e. two relay-chain blocks after the previously included
+parablock.
+
+The parablock generation and backing are bound together within a six-second window that limits the
+amount of data a collator can add to each parablock. Essentially, a parablock is limited to the
+requirement of being backed in six seconds, leaving little time for its generation and its
+blockspace to be properly filled.
+
+## Asynchronous Backing
+
+:::warning Disclaimer: Performance Measurements
+
+Due to asynchronous backing not being fully implemented in a running production network, each
+performance metric is not fully tested.
+
+:::
+
+### Pipelining
+
 Asynchronous backing is a feature that introduces
 [pipelining](https://www.techtarget.com/whatis/definition/pipelining) to the parachain block
-generation and validation process. It is analogous to the logical pipelining of processor
+[generation](./learn-parachains-protocol.md), [backing](./learn-parachains-protocol.md) and
+[inclusion](./learn-parachains-protocol.md). It is analogous to the logical pipelining of processor
 instruction in "traditional" architectures, where some instructions may be executed before others
 are complete. Instructions may also be executed in parallel, enabling multiple parts of the
-processor to be working on potentially different instructions at the same time.
+processor to work on potentially different instructions simultaneously.
 
 Bundles of state transitions represented as blocks may be processed similarly. In the context of
-Polkadot, it aims to increase the throughput of the entire network.
+{{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }}, pipelining aims to increase the
+throughput of the entire network.
 
-:::info
+### Synchronous vs. Asynchronous Backing
 
-In order to realize which exact processes this upgrade improves, it is recommended to be familiar
-with the stages of parablock validation, which you may [read here](./learn-parachains-protocol.md).
+Asynchronous Backing has three overarching goals:
 
-:::
+1. Decrease parachain block (parablock) validation time to **6 seconds** from **12 seconds**
+2. **Increase the usable blockspace**, allowing more state changes per relay chain block.
+3. **Allow parablocks to be re-proposed** to the network if they are not included successfully on
+   the first attempt.
 
-Asynchronous backing enables logical pipelining over the parablock
-[**generation**](https://wiki.polkadot.network/docs/learn-parachains-protocol#collators),
-[**backing**](https://wiki.polkadot.network/docs/learn-parachains-protocol#parachain-phase), and
-[**inclusion**](https://wiki.polkadot.network/docs/learn-parachains-protocol#inclusion-pipeline)
-processes. A parablock may be at different stages, but multiple parablocks can be processed
-simultaneously (in parallel) if needed. Most notably, parablock N + 1 can be generated and backed
-while its predecessor, parablock N, is undergoing inclusion on the relay chain. Processes can occur
-while their ancestors are included on the relay chain.
+In synchronous backing, parablock generation relies on the most recent relay-chain block. Each
+parablock must be generated and go through the entire backing process in a single relay block. It
+then proceeds to the inclusion process during the generation of the next relay-chain block.
 
-This pipeline will allow [collators](./learn-parachains-protocol.md#collators) to include an
-estimated ~3-5x more transactions/data while speeding up parachain block times from 12 to 6 seconds.
-In short, Polkadot with asynchronous backing will deliver an estimated ~6-10x more blockspace to its
-parachains.
+Asynchronous backing brings the following changes to the parachain protocol:
 
-Asynchronous Backing has three overarching goals:
+- Parablock generation is now decoupled from the backing process, avoiding the previous 6-second
+  deadline through the backing process. This gives more time for collators to properly use
+  blockspace, decreasing the probability of generating empty parablocks.
+- Parachains can generate blocks and have them placed into **unincluded segments** of parablock
+  ancestors (i.e. parablocks that are seconded but not yet backed by paravalidators and included in
+  the relay chain) rather than ancestors included in the relay chain state.
 
-1. Decrease parachain block (parablock) validation time to **6 seconds** from **12 seconds**
-2. Increase the amount of usable blockspace, allowing more state changes per relay chain block.
-3. Allow for parablocks to be re-proposed to the network if they are not included successfully on
-   the first attempt.
+### Unincluded Segments
 
-Asynchronous backing provides a form of **contextual execution**, which allows for more time for
-parachain collators to fit more transactions and prepare block candidates for backing and inclusion.
-**Contextual execution** refers to how a parablock can be built earlier using the context provided
-by an _unincluded segment_ of recent block ancestors.
+Unincluded segments are chains of candidate parablocks that are seconded but not yet backed and
+included in the relay chain. Parablocks can be added to this unincluded segment without waiting for
+the latest included parent block of the relay chain. The core functionality that asynchronous
+backing brings is the ability to build on these unincluded segments of block ancestors rather than
+ancestors included in the relay chain state.
 
-**Unincluded segments** are chains of candidate blocks that are not yet included in the relay chain.
-Parablocks can be added to this unincluded segment without waiting for the latest included parent
-block of the relay chain. The core functionality that asynchronous backing brings is the ability to
-build on these unincluded segments of block ancestors rather than ancestors included in the relay
-chain state.
+Compared to synchronous backing, contextual execution shift from being the parablock ancestors
+included in the relay chain, to being being the latest ancestor parablock pushed into the unincluded
+segment. This allows to start building parablocks earlier, giving plenty of time to parachain
+collators to fit more transactions and prepare block candidates for backing and eventually
+inclusion.
 
-Currently, parablocks rely on the most recent relay chain block (often referred to as the **parent**
-block, as the parablock anchors itself to it). Each parablock must be generated and go through the
-entire backing process in a single relay block. It then proceeds to the availability + inclusion
-process during the next block.
+### Asynchronous Backing Diagram
 
-:::info
+FIGURE HERE?
 
-It is important to note the following clarifications:
+In asynchronous backing, parablock `P + 1` can begin the generation process soon after parablock `P`
+has been generated and appended into the [unincluded segment](#unincluded-segments). Backing of
+parablock ancestor `P - 1` (pushed to the unincluded segment before `P`) can happen as soon as `P`
+is in the unincluded segment, available as execution context for the generation of parablock
+`P + 1`. Inclusion of `P - 1` and backing of `P` can happen within the span of one relay chain block
+(i.e. six-second time window) thanks to [pipelining](#pipelining).
 
-1. The parablock is not placed on the relay chain in its entirety, but rather the **paraheader**
-2. The relay chain does not access the entire state of a parachain but only the values that changed
-   during that block and the merkelized hashes of the unchanged values.
+The execution context for the latest parablock is derived from the unincluded segment of block
+ancestors upon which the newest parablock is built. These blocks can be prepared in anticipation of
+being included later rather than keeping in sync with the relay chain's progress one-to-one. A
+parablock can be in a different stage from another one being built if it abides by the parameters
+set forth by the asynchronous backing configuration.
 
-:::
+Asynchronous backing will allow [collators](./learn-parachains-protocol.md#collators) to include an
+estimated ~3-5x more data into parablocks while speeding up parachain block times from 12 to 6
+seconds. Due to the 2x decrease in block time and the possibility of building blocks in advance to
+fit more data, Polkadot with asynchronous backing will deliver an estimated ~6-10x more blockspace
+to its parachains.
 
-For more information on the validity and availability process, be sure to visit the
-[parachain protocol](../learn/learn-parachains-protocol.md) page.
+This combination of lower latency, higher storage per block, and a logical pipeline spanning
+Polkadot's networking, runtime, and collation aspects will allow for higher, more robust throughput.
 
-## Synchronous Backing on Polkadot
+### Prospective Parachains
 
-With synchronous backing, there was only about a single relay chain block (6-second window), to
-complete the parablock candidate generation and backing process. This was tightly coupled to the
-relay chain's progress, where blocks had to be created within this window.
+Prospective parachains are essentially all unincluded segments from all parachains.
 
-The main limitation of synchronous backing is that parablock validation is tightly coupled to the
-relay chain's progression on a 1-1 basis, meaning every parablock must be generated and backed
-within six seconds. This time limit reduces the amount of data a collator has time to add to each
-block.
+The
+[prospective Parachains subsystem](https://paritytech.github.io/polkadot/book/node/backing/prospective-parachains.html),
+is responsible for coordinating the state of various prospective parachain fragments. A fragment is
+a prospective/potential parablock. More than one of these make up a
+[**fragment tree**](https://paritytech.github.io/polkadot/book/node/backing/prospective-parachains.html#fragment-trees),
+and represent the potential states of a parachain. Each of these fragments are anchored to a past
+relay chain parent. This does not have to be the latest parent, as was before, but can be one in the
+past.
 
-A particular parablock, `P1`, would **only** be valid for backing at relay chain parent `R1 + 1` and
-subsequently, be included at `R1 + 2` should it be backed successfully. Essentially, a parablock is
-rushing to be backed and included within this two-block window due to the inherent requirement for
-synchrony between the parachain and relay chain.
+This subsystem also communicates with other subsystems in the validation process, such as the
+Backing subsystem, once a candidate block has been seconded.
 
-## Asynchronous Backing on Polkadot
+### Further Benefits of Async Backing
 
-With asynchronous backing, the window of time is customizable and will most likely sit around the
-6-18 second range. It also introduces a parameter to aid in defining the maximum amount of ancestor
-blocks, which allows for a parablock to be backed later in the future, enabling more computational
-and storage time per block. The context for the latest parablock is derived from the unincluded
-segment of block ancestors upon which the newest parablock is built.
+Asynchronous backing also introduces a parameter to define the maximum number of ancestor blocks
+within the unincluded segment. This allows for a parablock to be backed later, enabling more
+computational and storage time per block. The decoupling of collation (candidate generation) and
+backing also allows for more execution time, while citing a lower validation time.
 
-These blocks can be prepared in anticipation of being included later rather than keeping in sync
-with the relay chain's progress 1-1. The block can be in a different stage from another block being
-built, as long as it abides by the parameters set forth by the asynchronous backing configuration.
+There are two parameters that are controlled can be controlled by governance:
 
-This combination of lower latency, higher storage per block, and a logical pipeline spanning
-Polkadot's networking, runtime, and collation aspects will allow for higher, more robust throughput.
+- [`allowed_ancestry_len`](https://github.com/paritytech/polkadot-sdk/blob/f204e3264f945c33b4cea18a49f7232c180b07c5/polkadot/primitives/src/vstaging/mod.rs#L54) -
+  How many ancestors of a relay parent are allowed to build candidates on top of.
+- [`max_candidate_depth`](https://github.com/paritytech/polkadot-sdk/blob/f204e3264f945c33b4cea18a49f7232c180b07c5/polkadot/primitives/src/vstaging/mod.rs#L49) -
+  The maximum amount of candidates between the latest parablock and relay parent ancestor.
 
 ## Learn More
 
@@ -114,3 +190,4 @@ resources:
   Rob Habermeier, Polkadot founder, details the plans for Polkadot for 2023.
 - [Asynchronous Backing Spec & Tracking Issue](https://github.com/paritytech/polkadot/issues/3779) -
   The implementation tracking issue for asynchronous backing
+- [Prospective Parachains Subsystem - The Polkadot Parachain Host Implementers' Guide](https://paritytech.github.io/polkadot/book/node/backing/prospective-parachains.html)