8 Innovations that Make Solana the First Web-Scale Blockchain

If a blockchain network as a whole is going to match the performance of a single node, that implies that bandwidth cannot be the bottleneck, but rather computation. To achieve this, we need to first optimize how the nodes in the network communicate.

Wireless cellular networks offer many similarities to blockchain-based networks, and have long been focused on optimizing network communication. At scale, no single radio tower has enough bandwidth to give each cell phone its own radio frequency to transmit on, so telecom companies needed “multiple access technologies” to cram multiple phone calls on the same frequency.

Time Division Multiple Access (TDMA) is one of the major technologies that enabled massive scalability in cellular networks. TDMA specifies that towers divide each radio frequency into time slots, and allocate these time slots to each phone call. In this manner, the cell tower provides a globally available clock for the network. This massively increases scalability of limited bandwidth by letting each frequency support multiple, simultaneous data channels, and decreasing interference from multiple phones broadcasting on the same frequency at the same time.

Today’s blockchain-based networks have a clock problem. Their clocks “tick” whenever a new block is produced. For Ethereum, that happens once every fifteen seconds, and there’s only so much information that can be fit into a single block. The TDMA-equivalent for blockchain-based networks would be a clock with sub-second granularity that all validating nodes agree on, so that they can more efficiently process transactions.

The core Solana innovation is Proof of History (POH), a globally-available, permissionless source of time in the network that works before consensus. POH is not a consensus protocol or anti-Sybil mechanism. Rather, POH is a solution to the clock problem.

Whereas other blockchains require validators to talk to one another in order to agree that time has passed, each Solana validator maintains its own clock by encoding the passage of time in a simple SHA-256, sequential-hashing verifiable delay function (VDF). Solana does not use a VDF for randomness. Instead, each validator uses the VDF to maintain its own clock. Because each validator maintains its own clock, leader selection is scheduled ahead of time for an entire epoch. Like Tendermint, the schedule for an epoch lasts for thousands of blocks. However, unlike Tendermint, the network never waits for a failed node. Each validator runs the VDF to prove that it has acquired its slot to transmit a block and validators. Each validator is compensated for doing so because the block producer receives a reward for producing a block.

Enabled by Proof of History, leaders continue to rotate and the network as a whole makes progress regardless of network conditions. This means that the network never stops. The network can make a decision to rotate validators without any of the validators talking to one another. This is a subtle but profound shift. No other blockchain has a comparable mechanism. In every other chain, validators must communicate in order to make a decision. In Solana, leader rotation decisions are made asynchronously.

This core innovation opened up the design space going up the stack. In addition to providing a clock that can be used for timestamping, POH allows Solana to optimize for block time (800ms), block propagation (log200(n)), throughput (50K-80=K TPS)), and ledger storage (petabytes) available on the network.

Tower BFT

On top of Proof of History, Solana runs Tower Consensus, a PBFT-like consensus algorithm specifically designed to take advantage of the synchronized clock. Unlike PBFT, Tower Consensus prefers liveness over consistency. Like PBFT, nodes exponentially increase their timeouts to come to an agreement, but because the ledger is also a trustless source of time, nodes can observe and examine timeouts of all the other validators in the network. To better understand, let’s consider an example:

Imagine that you are on an island, and a bottle floats by with a thumb drive. Inside the drive is a Solana ledger. If you just look at the ledger itself, you will see that each node can compute the number of validators present, the state of each validator, and critically, the timeout each validator has committed to any block in the network. Based on the data structure alone, without any peer-to-peer messages, a validator can make the decision to vote, and the network can come to consensus.

Turbine

Given that the Solana consensus layer has no dependencies on peer-to-peer messages, Solana is able to optimize how blocks are transmitted through the network independently of consensus. Turbine, Solana’s block-propagation technique, borrows heavily from BitTorrent. As a block is streamed, it is broken up into small packets along with erasure codes, and then fanned out across a large set of random peers. With a fan-out of 200, the second layer of the network can cover 40,000 validators. Thus, validators are able to propagate blocks with a log200(n) impact to finality. For all practical purposes, if each connection is 100 ms, replication can be achieved in 400 ms, and finality in 500 ms for a 40,000 node network.

The fanout mechanism must be resistant to faults. As such, validators encode data using Reed-Solomon erasure codes, providing a degree of fault tolerance.

Gulf Stream

In a high performance network, mempool management is a new class of problem that other chains don’t really have to address. Gulf Stream functions by pushing transaction caching and forwarding to the edge of the network. Since every validator knows the order of upcoming leaders in Solana architecture, clients and validators forward transactions to the expected leader ahead of time. This allows validators to execute transactions ahead of time, reduce confirmation times, switch leaders faster, and reduce the memory pressure on validators from the unconfirmed transaction pool.

Clients, such as wallets, sign transactions that reference a specific block-hash. Clients select a fairly recent block-hash that has been fully confirmed by the network. Blocks are proposed roughly every 800ms, and require an exponentially increasing timeout to unroll with every additional block. Using our default timeout curve, a fully confirmed block-hashes are, in the worst case, 32 blocks old. Assuming block times of 800 ms, that equates to 25.6 seconds.

Once a transaction is forwarded to any validator, the validator forwards it to one of the upcoming leaders. Clients can subscribe to transaction confirmations from validators. Clients know that a block-hash expires in a finite period of time, or the transaction is confirmed by the network. This allows clients to sign transactions that are guaranteed to execute or fail. Once the network moves past the rollback point such that the blockhash the transaction reference has expired, clients have a guarantee that the transaction is now invalid and will never be executed on chain.

Sealevel

To take advantage of Solana’s high-performance network, we built Sealevel, a hyper parallelized transaction processing engine designed to scale horizontally across GPUs and SSDs. Note that all other blockchains are single-threaded computers. Solana is the only chain to support parallel transaction execution (not just signature verification) in a single shard.

The solution to this problem borrows heavily from an operating system driver technique called scatter-gather. Transactions specify up front what state they will read and write while executing. The runtime is able to find all the non-overlapping state transition functions occurring in a block and execute them in parallel — what is called parallel execution — while optimizing how read and writes to the state are scheduled across an array of RAID 0 SSDs.

Although Sealevel itself is a VM that schedules transactions, Sealevel doesn’t actually execute transactions in the VM. Instead, Sealevel hands off transactions to be executed on hardware natively using an industry-proven bytecode called the Berkeley Packet Filter (BPF), which is designed for high-performance packet filters. This bytecode has been optimized since the early 90s, and has been deployed in production in millions of switches worldwide to handle 60 million packets per second on a 40-gigabit network in a single switch.

Every time Nvidia doubles the number of SIMD lanes available, our network will double in computational capacity. Virtually all other blockchains, which are single-threaded computers by design, can never scale in this way.

Using LLVM, the same compiler that targets WASM, we provide a great set of tools for developers to write high-performance smart contracts in C/C++ and Rust that execute contracts on GPUs. Although Solana isn’t using WASM, developers can re-compile C and Rust code written for WASM compilers in the Solana compiler with minimal changes. Thus, developers can easily migrate their applications from other major WASM chains like Dfinity, EOS, Polkadot and Ethereum 2.0.

Ethereum has had a history of bugs resulting from the software architecture. Two relevant examples:

• Multiple parity hacks through Delegate Call
• The DAO rentrancy bug through ‘call’

It’s definitely possible to write safe Solidity code, just like it’s possible to write complex software in C without memory protection. But as long as unsafe behavior is easy to add and hard to detect, it becomes geometrically harder to verify the behavior of complex software. Both Solana and the Libra team recognized this problem early on and developed architectures that maintain a strict separation of state between different modules.

The Move language introduced Resources and Scripts as high level concepts. Both fit naturally into the Solana Sealevel Runtime, and how we have been designing our native programs. Our goal is to support Move as a first level language, such that Resources behave as native Solana programs, and can be developed and composed through Move, or through our own native Rust ABI without any compromises to performance or security.

Pipelining — a Transaction Processing Unit for validation optimization

The process of transaction validation on the Solana network makes extensive use of an optimization common in CPU design called pipelining. Pipelining is appropriate process when there’s a stream of input data that needs to be processed by a sequence of steps and there’s different hardware responsible for each.

On the Solana network, the pipeline mechanism — Transaction Processing Unit — progresses through Data Fetching at the kernel level, Signature Verification at the GPU level, Banking at the CPU level, and Writing at the kernel space. By the time the TPU starts to send blocks out to the validators, it’s already fetched in the next set of packets, verified their signatures, and begun crediting tokens.