Summary
The Solana validator network continues to grow and thrive, as measured by metrics including node count, Nakamoto Coefficient, and node distribution and diversity. Notably, since the last validator health report:
- There has been rapid progress in Solana’s evolution as a multi-client network, with over 31% of stake running through the Jito Labs client: This is up from 0% just over one year ago. Additionally, two more validator clients are in development.
- The network has experienced 100% uptime: Multiple new practices in software upgrade procedures have been implemented since a February 2023 performance degradation. Since February 26, 2023, the network has experienced 100% uptime.
As of Sept. 6, 2023, Solana continues to be one of the largest proof of stake networks in the world by node count, and one of the most distributed, as measured by Nakamoto Coefficient and now, validator software clients.
The Solana Foundation has also seen a noticeable uptick in the engagement of the validator network in recent months. These are hard metrics to put a number to, but some indicators of validator community engagement include:
- Regular community led validator calls: In March 2023, the validator ecosystem began planning and convening consistent calls for the validator community to share notes and best practices.
- Block 0, a conference for Solana validators: The validator community is hosting the first ever Block 0 in Amsterdam on Oct. 30, 2023. This is an entirely community-run event to discuss the evolution of the Solana network and reinforce social ties.
Solana mainnet beta launched in March 2020, three and a half years ago. In that time, the ecosystem has matured substantially. The Solana Foundation strives to be rigorous and intellectually honest as we assess the network’s health and opportunities to make it even more resilient, and we encourage the community to share their thoughts here.
Core Client Development
In the previous Validator Health Report, the Foundation discussed changes in thinking about the best way to measure and evaluate the health of the network. In particular, the Foundation has spent significant effort recently reinforcing the validator network’s health at the software level. In this vein, the Foundation has focused on encouraging the development of new software clients and reinforcing a network of core contributing developers from multiple organizations.
Validators are computers that run a Solana validator client, which is the operating system of the Solana network. It’s important for the resiliency and decentralization of any blockchain network to have more than one software client; this helps ensure that there’s no single failure point in the network’s software. One of the biggest victories for the ecosystem is Solana becoming a multi-client network, which means validators have a choice of clients to run.
How Solana is doing:
There are four different validator client implementations for the Solana network in active development, built on three independent codebases. Notably, the Jito Labs client is run by over 31% of Solana validators, up from 16% since March 2023 (the previous Validator Health Report) and up from 0% since August 2022, when the client first launched to mainnet. 9
Validator client diversity is important for the long term health and functioning of the network. With multiple validator clients, the risk of one bug or harmful piece of code in a single client is mitigated by the existence of other independent clients, which are unlikely to have the same bug or malware attack, making a total network outage less likely.
The first validator client on Solana was originally developed by Solana Labs. Since then, there’s been several independent efforts to create additional full or light validator clients on the Solana network:
- Jito Labs: In August 2022, Jito Labs released a second validator client to mainnet. This is a fork of Solana Labs code that Jito is independently building and is responsible for maintaining, changing, and deploying. However, because this is a fork of the existing client, a bug in the Solana Labs client would likely be present in this client.
- Firedancer: Also in August 2022, Jump Crypto announced plans to build a completely new validator client on Solana. This validator client is being developed from the ground up in C++ and has shown significant performance improvements. In testing environments, Firedancer has processed up to one million transactions per second (in comparison, the original Solana Labs client processes closer to 55,000 transactions per second in similar testing environments.).
- Sig: In July 2023, Syndica announced the development of Sig, a validator client for the Solana network written in the Zig programming language. In September 2023, the validator team at Syndica unveiled the initial implementation of the gossip protocol for Sig.
- TinyDancer: In addition to these four validator clients, TinyDancer, a light client for Solana, is in active development. Light clients like TinyDancer do not build blocks and participate in consensus, but instead make it easier for users to verify the state of a blockchain without having to run a full node themselves.
Total Validator Count
Blockchains with more validators tend to be more resilient. When a user executes a contract on a blockchain, they need to be confident that their transmission will be recorded. Ideally, each addition to a blockchain is recorded on every validator on that chain, which is why a higher number of validators is important; a large number of diverse validators protects against catastrophic events like a data center outage.
There are two types of validators:
- Consensus nodes: Consensus nodes are central to the functioning of the network by providing two essential functions: (1) creating and proposing new blocks to the rest of the network and (2) voting on the validity of new blocks proposed by other nodes on the network.
- RPC nodes: Remote Procedure Call (RPC) nodes are an application’s gateway to the Solana infrastructure. These nodes, like consensus nodes, independently verify all new blocks and changes to the network. However, they do not vote.
See the appendix for more details on total validator count and why it’s important.
How Solana is doing:
In March 2023, total consensus nodes dropped from around 2200 to roughly 1700: this drop was due to a large amount of stake that was redelegated away from nodes that were charging 100% commission. The stakeholder recognized the issue and reallocated their delegation to more active validators. After this drop, and as of Sept. 13, consensus nodes have slowly and steadily grown upwards, reaching 1,961 consensus nodes and 2,874 validator nodes total.
The absolute number of nodes on Solana is quite high relative to other proof of stake blockchains. The Foundation expects to undertake changes to its programs in the coming months to encourage node quality – not just node quantity.
What makes a “high-quality validator” is subjective, but some examples include uptime, hardware performance, service levels in the case of a user issue, or how active the validator operator is in the broader validator community. The Foundation has identified a few opportunities to encourage validators to meet these standards and will begin sharing these and rolling them out to the community in the coming months.
Nakamoto Coefficient for Voting Power
The Nakamoto Coefficient for voting power is defined as the minimum number of nodes that would need to be compromised to censor blocks or stop consensus in a network, thereby preventing some or all new blocks (and the transactions within them) from being confirmed. For most proof of stake networks, this is the minimum number of nodes required to represent at least 33.4% of voting power.
When stake distribution is highly centralized, a smaller number of validators may represent 33.4% (a superminority) of total stake delegations. In a more decentralized distribution of stake and consensus power, this set is larger, making it more difficult for a business, rogue actor, or other entity to manipulate the blockchain through censorship.
See the appendix for more background on evaluating decentralization with respect to the Nakamoto Coefficient.
How Solana is doing:
When pulled on Sept. 6, 2023, the Nakamoto Coefficient on Solana was 31. This means the lowest number of validators that would have to collude to censor the network is 31. The Nakamoto Coefficient is unchanged from the last Validator Health Report (March 2023), when it was also 31.
Solana’s Nakamoto Coefficient grew steadily from the chain’s launch in March 2020 through September 2022 and has remained relatively stable since then. A Nakamoto Coefficient of 31 is robust. While the Foundation would prefer to see this increase over time, seeing the number grow is not a leading indicator of the network’s decentralization, given that the Solana network is fairly distributed from the perspective of stake distribution.
Listed below are the Nakamoto Coefficient of several other proof of stake blockchains, for the sake of benchmarking. 10
Even a large, highly distributed network with multiple software clients is vulnerable to several exogenous factors that could impact the resilience of a blockchain. These are discussed in the next and final section.
Distribution
The Nakamoto Coefficient and client diversity are critical metrics, but don’t capture the human element involved in running a blockchain. One of the least appreciated aspects of validator network health is the role of exogenous factors such as geopolitics, natural disasters, and corporate incentives.
In this final section, we look at the Solana network’s resilience in the context of a few exogenous factors and how they could potentially affect a proof of stake network like Solana.
Stake Distribution by Data Center Provider:
Anyone can run a Solana node. Because the Solana protocol requires highly performant hardware, validator operators will often rent server space from third-party data centers to run their nodes. This is not unusual; the majority of the computing power on most blockchains is done on third-party owned servers in large data centers.
The risk of using third-party data centers to run validators means that the owners of data centers have disproportionate power over the functioning of a blockchain. Stake should be relatively distributed among private companies that rent server space, in order to minimize the risk that a single company can compromise a chain.
This risk reared its head in November 2022, when server provider Hetzner blocked Solana nodes. Notably, the network continued performing through this time. This was the equivalent of a 20% attack on the network and demonstrates why distribution of stake across multiple server providers is so critical.
How Solana is doing:
Data centers providers often operate multiple data centers and ASNs (Autonomous System Number). The data below is split out based on ASNs of major data centers, based on data that’s publicly available.
An Autonomous System (AS) is a network of servers with a single routing number. Different Autonomous Systems are identified by a unique number, known as the ASN. Depending on how the internal networking or routing is configured, a single ASN could span multiple physical locations in different geographies.
Source (last updated 9/7/23)
Stake on Solana is relatively distributed among ASNs, with no one autonomous system hosting anything close to 33.3% of active stake. At the moment, at least three data centers would have to collude to assemble more than 33.3% of stake and halt the network.
Stake Distribution by Geography:
A global, resilient blockchain has to continue operating, no matter the events in a given part of the world. Consider:
- One government carries out an attack on underwater fiber cables that deliver internet, and knocks out internet to an entire region.
- A dissident facing retribution from a dictatorial regime has to feel confident she can access funds, even if that regime chooses to shut down servers running a chain in-country.
- A natural disaster disrupts all the nodes in a particular region. Users of a blockchain in any part of the world still need to feel confident that the chain will keep running, even when many validators are unexpectedly knocked offline.
How Solana is doing:
Here’s a snapshot of the geographic distribution of the network, organized based on the percentage of stake in each country.
Source (last updated 9/7/23)
The network is well distributed geographically, with no one country having 33.3% of active stake, although recent gains in the United States suggest they could eventually achieve that percentage. Notably, the United States plus Canada have a combined active stake of 34.3%. The stake percentages by country here differ significantly from the absolute number of data centers by country.
Stake running through the United States has increased substantially since the last report, going from 23.5% to 29.2%. The Foundation is monitoring this change closely, and taking actions to address it, for example, by assisting stake pools in optimizing decentralization by geography in their scoring algorithms.
Looking to the future
The Solana Foundation is continuously working to improve the health of the validator network by providing tools and education to our global community of validators and stakers, and by encouraging the community to be thoughtful participants in securing the network.
We continue to celebrate growth in the validator community and Nakamoto Coefficient, but the Foundation's focus has broadened to include less easily measurable ways to improve the network health.
The Solana validator community has created several new initiatives to improve the health of the network, including organizing Block 0 (a conference for validators), community led validator calls, and new governance practices.
Over the coming months, the Foundation will be adding support to the community’s efforts with its own initiatives designed to improve validator network health, including changes to the Solana Foundation Delegation program that will help validators become more self-reliant and sustainable and reinforce the network’s community orientation.
The Foundation welcomes feedback and questions on the Validator Health Report.
Special thanks to:
- Joseph Burgess, for his countless hours on this report.
- The team at Chainflow, who always offers incredibly valuable feedback.
- Elias and the team Rated, for their excellent data and willingness to walk through Ethereum Nakamoto Coefficient calculations; and, @larry0x on Threads, whose excellent thread explains the various ways one might calculate Ethereum’s Nakamoto Coefficient.
Footnotes and Appendices
- Source: Nakaflow (as of 9-6-2023)
- Source: Solana command line (as of 9-13-2023)
- Source: Avalanche (as of 9-5-2023)
- Source: Cosmos (as of 9-5-2023)
- Source: Ethernodes (as of 9-5-2023), filtered for unsynced nodes
- Source: The Nakamoto Coefficient of Ethereum is manually calculated using data on stake distribution from Rated as of 9-19-23. We made several assumptions: 1) Lido nodes should be treated as separate operators, 2) The Nakamoto Coefficient of Ethereum is reached when 50% of all stake is compromised, which could result in a real-time halt to the Ethereum network (as opposed to 33% on most Proof of Stake networks).
- Source: NEAR (as of 9-5-2023)
- Source: Polygon (as of 9-5-2023)
- Source: Jito Labs (as of 9-11-2023)
- Source: Nakaflow (as of 9-6-2023 and 3-7-2023)
Blockchains with more validators tend to be more resilient. When a user executes on a contract on a blockchain, they need to be confident that their transmission will be recorded. Ideally, each transmission on a blockchain is recorded on every validator on that chain, which is why a higher number of validators is important: The more times that a message is recorded, the more confident a user can be that their message is accurately recorded and won’t be tampered with.
There are two types of validators on the Solana network:
Consensus nodes: Consensus nodes are central to the functioning of the network by providing two essential functions: (1) creating and proposing new blocks to the rest of the network and (2) voting on the validity of new blocks proposed by other nodes on the network.
Each block contains many messages that are submitted by various users and applications on the network. Every consensus node independently verifies all new messages in a proposed block before voting on its validity. The more nodes that participate in this consensus process, the more confidence a user or third party has that a change to the network or a transaction was verified by a large population.
RPC nodes: Remote Procedure Call (RPC) nodes are an application’s gateway to the Solana infrastructure. RPC node operators can offer API, indexing, or other services to provide a convenient interface for users and applications to the core Solana network. These are often commissioned or run by individual applications and are dedicated to that program’s particular task, rather than maintaining consensus on the blockchain. RPC nodes, like consensus nodes, all independently verify all new blocks and changes to the network. They do not vote.
A large number of nodes is critical for the health of the network. There’s no clear number for how many nodes is enough. What’s important is that:
- Users feel confident that their submission will be recorded, no matter what. This is why it’s important to have a large number of copies of the current state available on many nodes, and that they exist in a broad distribution across the world. Solana’s single global state tracks the latest contents of each account, updated in real time with the latest information.
- Nodes operate independently of each other. A failure of a single node or set of nodes (say, run by a single entity or in a particular geography) should not impact the functioning of the network.
- Users can verify the accuracy of submissions by looking at other nodes. If a single node goes down or has an issue recording a message, users can rely on other nodes to verify the accuracy of the blockchain.
- Having 100% uptime in the period since the last Validator Health Report intersects with the above points. Having nodes or sets of nodes act independently helps ensure the functioning of the network, while having a high uptime percentage ensures users are confident their submissions will be recorded and they can verify the accuracy of submissions.
Users of a blockchain must be confident that any valid message they submit will be included in a block and then confirmed through consensus. If a group of consensus nodes becomes compromised or acts maliciously in a coordinated manner, it can attempt to alter or prevent the network from achieving consensus on new blocks. The Nakamoto Coefficient is a common way to measure a blockchain’s resilience against such behavior. The Nakamoto Coefficient, a metric first proposed by Balaji Srinivasan, is defined as the minimum number of nodes that would need to be compromised to alter or stop consensus in a network, thereby preventing some or all new blocks (and therefore the transactions within them) from being confirmed. This process is known as censorship, and could impact the entire network or some subset of users or applications. In proof of stake networks, the Nakamoto Coefficient is the minimum number of nodes required to represent at least 33.4% of voting power.
This set of validators with a cumulative 33% of the network’s stake delegation is known as a superminority. A compromise of a superminority would impact the blockchain’s real-time ability to guarantee that new blocks be voted on and added to the chain. In the event that a superminority is compromised, the blockchain could recover by excising the affected validators and restarting consensus without them. The Nakamoto Coefficient for stake distribution represents the size of the smallest population of individual validators which comprise a superminority of stake delegations. When stake distribution is highly centralized, a smaller number of validators may represent 33% (superminority) of total stake delegations. In a more decentralized distribution of stake and consensus power, this set is larger, making it more difficult for a business, rogue actor, or other entity to manipulate the blockchain through censorship.
Join the community
Sign up for the Solana Community Update newsletter and hear the latest from ecosystem teams around the world.
