Author: Cryptonaitive
On June 12, 2023, Gensyn, a blockchain-based AGI computing power market protocol, announced the completion of a $43 million Series A financing round, led by a16z, with participation from Eden Block, CoinFund, Galaxy, Protocol Labs and others.
What is Gensyn? Why did it receive huge investment from top VCs? Golden Finance will help you understand it in one article.
a16z: Why we led Gensyn’s $43 million Series A round
a16z published an article explaining why it led Gensyn's $43 million Series A financing. a16z said that the recent progress in artificial intelligence is incredible and has the power to save the world (see Golden Finance's previous report "a16z founder's 10,000-word long article: Why AI will save the world"). But building AI systems requires deploying greater computing power to train and reason about today's largest and most powerful models. This means that large technology companies have an advantage over startups in the competition to extract value from artificial intelligence, thanks to privileged access to computing power and economies of scale in large data centers. In order to compete on a level playing field, startups need to be able to affordably use their own large-scale computing power.
What makes blockchain unique as a new type of computer is that developers can write code and make firm commitments about how it will behave in the future. This permissionless component of blockchain can create a marketplace for buyers and sellers of computing power — or any other type of digital resource like data or algorithms — to trade globally without middlemen.
Gensyn is a blockchain-based AGI computing power market protocol that connects developers (anyone who can train machine learning models) with solvers (anyone who wants to train machine learning models with their own machines). By leveraging idle, long-tail computing devices with machine learning capabilities around the world, such as small data centers, personal gaming computers, M1 and M2 Macs, and even smartphones, Gensyn can increase available computing power for machine learning by 10-100 times.
The problem facing AGI (artificial general intelligence): high degree of centralization
After nearly half a year of development, the market generally recognizes that AGI is the future. However, the AGI industry currently seems to be highly monopolized. The trade and talent war between countries is between China and the United States, and the game between companies is between large technology companies (Microsoft, Google, Meta). This is because the three key resources of AGI (computing power, knowledge and data) are currently highly centralized.
Computing power: Increasingly large and complex models require high-computing processors to train. Between countries: The chip war between China and the United States, the United States has been actively preventing China from obtaining high-computing chips. Between companies: Insufficient production capacity, Nvidia's latest AI chips are all purchased by certain large customers, and other companies cannot buy them at all. In the technology stack: Some companies have even created their own dedicated hardware for deep learning, such as Google's TPU cluster. These perform better than standard GPUs for deep learning and are not sold, only for rent.
Knowledge: Many public breakthroughs stem from new large model architectures developed by researchers, but there is a battle over the underlying intellectual property and talent. For example, the United States has attracted more than 50% of China's AI talent, and large companies that use this talent to develop large models are increasingly reducing the accessibility of this technology; OpenAI's GPT-3.5 or 4 is nominally publicly available, but it is behind an API and only Microsoft can access its source code.
Data: AGI deep learning models require a lot of data — both labeled and unlabeled — and generally improve as the amount of data increases. GPT-3 was trained on 300 billion words. Labeled data is particularly important, and the datasets needed to train AGI are concentrated in the hands of a few large companies. For example, a little-known fact: every time you solve a reCaptcha to visit a website, you are labeling training data to improve Google Maps.
Difficulties in decentralized AGI computing
Decentralized computing can create a cheaper and freer foundation for researching and developing artificial intelligence. But decentralized AGI has a work verification problem. How do you know that a third party has completed the calculation you requested?
There are two factors that make proof of work difficult: state dependency and high computational cost.
State dependency: Each layer in a neural network is connected to all nodes in the layer before it. This means it needs the state of the previous layer. Even worse, all weights in each layer are determined by the previous time step. So if you want to verify if someone trained a model — say, by picking a random point in the network and seeing if you get the same state — you need to keep training the model up to that point, which is computationally expensive.
High computational expense: The cost of training a single GPT-3 in 2020 was approximately $12 million, more than 270 times higher than the estimated cost of training a GPT-2 in 2019 of approximately $43,000. In general, model complexity (size) of the best neural networks currently doubles every three months. If neural networks were cheaper, and/or if training represented less of a model development process, then perhaps the validation overhead from state dependency would be acceptable.
If we want to make deep learning training cheaper and decentralize control, we need a system that can trustlessly manage state-related verification while also being cheap in terms of overhead and rewarding those who contribute computation.
How Gensyn Decentralizes AGI Computing
The Gensyn protocol unites all the world’s computation into a global machine learning supercluster that is available to anyone at any time. It does this by combining two things to enable trustless training of neural networks at massive scale and low cost:
1. Innovative verification system
A verification system that efficiently solves the state dependency problem in training neural networks at any scale. The system combines model training checkpoints with probabilistic checks that terminate on-chain. It does all of this in a trustless manner with overhead that scales linearly with model size (keeping verification costs constant).
According to the Gensyn Litepaper, Gensyn solves the verification problem mainly through three concepts: probabilistic proof-of-learning (using metadata from the gradient-based optimization process to build certificates of work performed and quickly verify through replication of certain stages), graph-based precise positioning protocol (using multi-granularity, graph-based precise positioning protocol and cross-evaluator consistent execution to allow verification work to be re-run and compared for consistency, and ultimately confirmed by the chain itself), Truebit-style incentive game (using staking and slashing to build an incentive game to ensure that every financially rational participant acts honestly and performs their intended tasks)
The system consists of four main participants: submitters, solvers, verifiers, and whistleblowers. Submitters: end users of the system, providing tasks to be calculated and paying for completed work units; solvers: the main working part of the system, performing model training and generating proofs for verifiers to check; verifiers: linking the non-deterministic training process to deterministic linear computations, copying part of the solver's proof and comparing the distance to the expected threshold; whistleblowers: the last line of defense, checking the work of the verifier and challenging it in the hope of winning the cumulative bonus.
2. New computing power supply
Leverage underutilized and underutilized/unoptimized computing device resources. These devices range from currently unused gaming GPUs to GPU miners from the previous Ethereum PoW era. And the protocol’s decentralization means that it will ultimately be managed by the community majority and cannot be “shut down” without community consent; unlike its web2 counterpart, this makes it censorship-resistant.
Large scale + low cost: Gensyn protocol provides similar cost to data center owned GPUs, and its scale can exceed AWS