## Description
Consensus, or establishing total order across transactions, pose…s a major scalability bottleneck in blockchain networks. In short, the main challenge with consensus is that it requires all nodes (often called validators or miners) to process all transactions. Regardless of the specific consensus protocol implementation used, this makes blockchain performance limited to that of a single miner at best.
Borrowing ideas from traditional distributed databases, one possible approach to overcoming this limitation is to resort to the partitioning, or **sharding**, of state processing and transaction ordering. In a sharded system, the blockchain stack is divided into different groups called shards. Each shard is operated by its own set of miners, keeps a subset of the state, and is responsible for processing a part of the transactions sent to the system. The rationale behind sharding is that by dividing the whole blockchain network into different groups, the load of the system is balanced among them, increasing the overall transaction throughput of the system. Instead of every node having to process all transactions sent to the system, each shard processes and handles the state of a subset of transactions and objects.
Existing sharded designs (e.g. [OmniLedger](https://eprint.iacr.org/2017/406.pdf) and [Shard Scheduler](https://arxiv.org/pdf/2107.07297.pdf)) often follow a similar approach to the one traditionally used in distributed databases, where the system is treated monotonically, and a sharded system acts as a distributed controller which assigns miners to different shards, and attempts to distribute the state evenly across shards to balance the load of the system. Many of these designs use a static hash-based assignment to deterministically select what state needs to be kept and what transactions are to be processed by each shard.
The main challenge with applying traditional sharding to the Byzantine fault-tolerant context of the blockchain lies in the security/performance tradeoff. As miners are assigned to shards, there is a danger of dilution of security compared to the original single-chain (single-shard) solution. For instance, in both Proof-of-Work and Proof-of-Stake blockchains sharding may lead to the ability of the attacker to compromise a single shard with only a fraction of the mining power, potentially leading to compromising the system as a whole. Such attacks are often referred to as 1% attacks. To circumvent such attacks, sharding systems need to re-assign randomly and periodically miners to shards in an unpredictable way to cope with a semi-dynamic adversary. In the following, we refer to this approach as traditional sharding.
We believe that the traditional sharding approach to scaling, which considers the system as a monolith, is not suitable for decentralized blockchains. Instead, in this project, we depart from the traditional sharding approach to build **hierarchical consensus**. In hierarchical consensus, instead of algorithmically assigning node membership and evenly distributing the state, we follow a ``sharding-like'' approach where users and miners are grouped into **subnets** and where they can freely choose the subnets they want to belong to. What is more, miners can spawn new child subnets from the one they are operating in at will, according to their needs. Even users can spawn a new child subnet, provided they operate as miners in that subnet.
We refer to the state of a subnet as a **chain**. Each subnet keeps its state in an independent chain and processes transactions that involve objects that are stored in the chain. Every subnet can run its own independent consensus algorithm and have its own security and performance guarantees.
All subnets in the system are organized hierarchically where each of them will have one parent subnet and can have any number of child subnets, except root subnets which have no parent and are hence called **root networks**, or **rootnets**. As a major difference compared to traditional sharding, subnets in hierarchical consensus are firewalled, in the sense that a security violation in a given subnet is limited in effect to that particular subnet and its child subnets, without affecting its ancestor subnets. Moreover, ancestor subnets help secure their descendant subnets --- for instance, checkpointing a Proof-of-Stake subnet into its parent may help alleviate notorious long-range and similar attacks. In addition, rootnets in **hierarchical consensus** are also able to commit in parallel into other blockchains/rootnets with better or complementary security guarantees. For instance, the rootnet in Filecoin's hierarchical consensus can leverage the very high security of the Bitcoin network by periodically committing a checkpoint of its state (see #5).
At a high level, hierarchical consensus allows for incremental, on-demand scaling and simplifies the deployment of new use cases on a blockchain network. Our design is inspired by the Proof-of-Stake sidechain design, to our knowledge first proposed [here](https://eprint.iacr.org/2018/1239.pdf). Hierarchical consensus generalizes this approach, minding the specifics of Filecoin, which does not use Proof-of-Stake as a Sybil attack protection on the root chain, but rather copes with sybils in a way specifically tailored to data storage (Proof-of-SpaceTime ([PoST](https://spec.filecoin.io/algorithms/pos/post/)) and Proofs-of-Replication ([PoRep](https://spec.filecoin.io/algorithms/pos/porep/)).
## Resources
* [Project spec](https://github.com/protocol/ConsensusLab/blob/main/docs/b1.pdf) (WIP / living document)
* [Implementation](https://github.com/filecoin-project/eudico/)
## Papers
* A. De Rocha, L. Kokoris-Kogias, J. M. Soares, and M. Vukolić, “[Hierarchical consensus : A horizontal scaling framework for blockchains](https://research.protocol.ai/publications/hierarchical-consensus-a-horizontal-scaling-framework-for-blockchains/)," 2022. (preprint)
## Talks
#### 2022-02-03 PL Research Seminar: Filecoin Hierarchical Consensus Specification
[](https://youtu.be/G7d5KNRZdp0)
#### 2022-02-07 ConsensusLab 22Q2 Team Week: B1 update
[](https://youtu.be/4-MNI_3VG5k)
## Demos
#### 2021-10-14, hierarchical consensus MVP
[](https://youtu.be/ZkPyfXYRtb0)
#### 2021-12-09, new actor architecture
[](https://youtu.be/nz68CJtWAY4)
#### 2022-02-10, cross-net transactions
[](https://youtu.be/qPDYOa8Tnp0)
#### 2022-03-10, atomic execution between subnets
[](https://youtu.be/MwR3JKEFOSk)
