Headline: One Misconfigured Route Nearly Broke Solana — Network Came 86% of the Way to Losing Finality A single routing misconfiguration at one hosting provider on Wednesday morning knocked 28.83% of staked SOL offline and pushed Solana roughly 86% of the way to losing transaction finality. Finality on Solana stops when 33.34% of stake is delinquent; the outage peaked just 4.51 percentage points shy of that threshold. What happened - Teraswitch, the provider at the center of the incident, says the problem began when a default route from its Miami site was propagated with its metric and BGP communities stripped. A route reflector in Amsterdam then distributed that route into Europe and Asia-Pacific. - Edge routers in several regions accepted the route as if it had been locally originated and preferred it over their native routes. Those routers then handed the bogus route up to the data center core, which rejected it as invalid. - The result: twelve data-center sites across London, Amsterdam, Dublin, Frankfurt, Singapore and Tokyo had no valid route to forward traffic. North American sites were not affected. - Engineers identified the fault within ten minutes and service was restored at 04:16:15 UTC. Scope and stake concentration - Marinade Finance’s post-incident analysis found the outage concentrated in a single autonomous system, AS20326, which carries 118,890,767 SOL — more than a quarter of all staked SOL. About 94% of that AS’s stake went offline during the event. - That exceeds the Solana Foundation’s delegation cap of 25% per autonomous system: AS20326 sits at 27.34%. - Marinade reports that 59 validators holding 80.2 million SOL came back online in the same narrow window (Amsterdam, Frankfurt and Tokyo), having waited for routing to reconverge rather than switching to failover options. Helius, Solana’s second-largest validator, remained down for the full 33 minutes. - Of 74 operators Marinade measured, only three — Laine and Cogent Crypto (both run by Sol Strategies) and Lion3d — “came back clean.” - The exposure was wider than one provider: an additional 14.1 million SOL went offline simultaneously across latitude.sh, Limestone, Butterfly Research and Allnodes in ways Marinade could not explain from available data. Marinade warns that counting stake by hosting provider may understate how much fails together. Local introspection and remediation - Marinade ran the same analysis on its own allocations and found an internal concentration problem: four autonomous systems hold two-thirds of the stake its allocation model distributes, and one AS (AS395201) accounts for 36.94%. The firm said it will review concentration limits per network and per data center and begin publishing whether validators support hot-swap and automatic failover — things that can’t be determined externally today. Consequences and risk - The outage caused validators to miss about 333 SOL in rewards; those missed rewards will be covered by validator bonds at the end of the epoch. - Had delinquency crossed the 33.34% finality threshold, no transactions would have finalized for any SOL holder and there would have been no bond mechanism to cover that outcome. By comparison, Solana’s last full halt in February 2024 took nearly five hours to recover. Why this matters - The incident highlights how a single routing mistake — and concentration of stake across a small number of autonomous systems and hosting providers — can quickly threaten a high-throughput proof-of-stake network’s ability to finalize transactions. - Operators, delegators and governance programs may need stronger safeguards: enforced diversification limits by AS and data center, clearer public disclosure of failover capabilities, and better operational practices across hosting providers and validators. Teraswitch has published a technical writeup of the routing failure; Marinade and other ecosystem participants say they will re-evaluate concentration policies and transparency measures to reduce the chance of a repeat. Read more AI-generated news on: undefined/news
