Written by Maggie, Foresight Research
ZKML (Zero knowledge machine learning) is a technology that uses zero-knowledge proofs for machine learning. ZKML is a bridge between AI and blockchain. ZKML can solve the privacy protection issues of AI models/inputs and the verifiability of the reasoning process, so that small models or reasoning ZKPs can be put on the chain. The significance of putting models/reasoning proofs on the chain is:
Let the blockchain perceive the physical world. For example, a face recognition model running on the blockchain can perceive faces for the blockchain. Through the on-chain AI model, the blockchain can understand that the face may be a woman, her approximate age, etc.
Allow smart contracts to make decisions. For example, the WETH price prediction model on the chain can help smart contracts make trading decisions.
Run AI models with privacy protection. For example, an enterprise has spent a lot of computing power to train a model and hopes to provide inference services to the outside world in a privacy-protected manner, or the user's input needs to be kept private. Using ZKML can not only ensure the privacy of the model/input, but also prove to the user that the inference is correct, thus achieving trustless inference.
Applications of ZKML
On-chain AI: Put AI models/AI reasoning proofs on-chain, so that smart contracts can use AI to make decisions. For example, on-chain trading systems are used for on-chain investment decisions.
Self-improving blockchain: Allow blockchain to use AI capabilities to continuously improve and modify strategies based on historical data, such as an AI-based on-chain reputation system.
AIGC on-chain: Content/artwork generated by AIGC is minted on-chain into NFT. ZK can prove the correctness of the process and that no copyrighted images are used in the data set.
Biometric authentication (KYC) of the wallet: The proof of facial recognition is uploaded to the chain, and the wallet completes KYC.
AI security: Use AI for fraud detection, Sybil attack prevention, etc.
On-chain ZKML games: on-chain AI chess players, neural network-driven NFT characters, etc.
ZKML technically
Objective: Convert neural networks into ZK circuits. Difficulties: 1. ZK circuits do not support floating point numbers. 2. Neural networks that are too large are difficult to convert.
Current Progress:
The earliest ZKML library was created 2 years ago, and the entire technology has a short history of development. Currently, the latest ZKML library supports some simple neural network ZKization and is applied to blockchain. It is said that it can put basic linear regression models on the chain, and other types of smaller neural network models can support proof on the chain. However, there are very few demos, only one for handwritten number recognition.
Some tools are said to support 100M parameters, and others claim to be able to convert GPT2 into ZK circuits and generate ZK proofs.
Direction of development:
Network Quantization converts floating-point numbers in neural networks into fixed-point numbers and lightweight neural networks (ZK-friendly).
Try to convert neural networks with large parameters into ZK circuits and improve proof efficiency (expand ZK capabilities).
Summarize:
ZKML is a bridge between AI and blockchain. Its significance lies in enabling blockchain to perceive the physical world, enabling smart contracts to make decisions, and running AI models with privacy protection. It is a very promising technology.
This technology has a short history but has developed rapidly. At present, some simple neural network models can be converted into ZK circuits, and models can be put on the chain or reasoning proofs can be put on the chain. However, the language is relatively difficult. At present, Ddkang/zkml claims to be able to generate ZK versions of GPT2, Bert and Diffusion natural language processing models, but the actual effect is unclear. It can run but may not be put on the chain. I believe that with the development of network volume technology, ZK technology, and blockchain expansion technology, ZKML of language models will soon become available.
1. Background
(If you are familiar with ZK and ML, you can skip this chapter).
Zero-knowledge proof (ZK): Zero-knowledge proof means that the prover can convince the verifier that a certain statement is correct without providing any useful information to the verifier. ZK is mainly used to prove that the calculation process is executed correctly and to protect privacy.
Prove the correctness of the calculation process: Taking ZK-rollup as an example, the operation of ZK-rollup is simply to package multiple transactions together, publish them to L1, and publish a proof (using zero-knowledge proof technology) to claim that these transactions are valid. Once verified on L1, the status of zk-rollup will be updated.
Privacy protection: Taking the Aztec protocol as an example, the assets on Aztec's zk.money exist in the form of bills, similar to Bitcoin's UTXO. The amount of the bill is encrypted. When a user needs to transfer money, the bill needs to be destroyed and a new bill needs to be created for the payee and himself (change). Zero-knowledge proof is used to protect privacy and prove that the amount of the destroyed and newly created bills is the same, and the user has the right to control the bill.
Machine Learning: Machine learning is a branch of artificial intelligence. The theory of machine learning is mainly about designing and analyzing algorithms that allow computers to "learn" automatically. A type of machine learning algorithm automatically analyzes patterns from data and uses these patterns to make predictions about unknown data. Machine learning has been widely used in computer vision, natural language processing, biometrics, search engines, medical diagnosis, detection of credit card fraud, securities market analysis, DNA sequencing, speech and handwriting recognition, games and robotics.
2. What problem does ZKML solve?
ZKML is a research and development field that has caused a sensation in the cryptography community in the past two years.
, the main goal of this technology is to use zero-knowledge proof to solve the privacy protection and verifiability problems of machine learning. This enables small models or inference ZKPs to be put on the chain, becoming a bridge between AI and blockchain:
Model on-chain: ML models can be converted into ZK circuits, and small ZKML models can be stored in smart contracts on the blockchain. Users can use the model by calling smart contract methods. For example, RockyBot from Modulus Labs has built an on-chain AI model to predict the price of WETH for trading decisions.
Model reasoning proof and other on-chain: Convert the ML model into a ZK circuit, perform reasoning off-chain, and generate a ZK proof. The ZK proof can prove that the reasoning process is executed correctly. The reasoning result and ZK proof are submitted to the chain for the caller's reference and the smart contract to verify the proof.
What is the significance of putting models/reasoning proofs on the chain?
Let the blockchain perceive the physical world. For example, a face recognition model running on the blockchain can perceive faces for the blockchain. Through the on-chain AI model, the blockchain can understand that the face may be a woman, her approximate age, etc.
Allow smart contracts to make decisions. For example, the WETH price prediction model on the chain can help smart contracts make trading decisions.
Run AI models with privacy protection. For example, an enterprise has spent a lot of computing power to train a model and hopes to provide inference services to the outside world in a privacy-protected manner, or the user's input needs to be kept private. Using ZKML can not only ensure the privacy of the model/input, but also prove to the user that the inference is correct, thus achieving trustless inference.
The role of zero-knowledge proof in ZKML:
Privacy protection: Protect the privacy of input data during ML model or prediction process.
Data privacy (Public Model + Private Data): I have some sensitive data, such as medical data, facial images, etc. I can use ZKML to protect the privacy of the input data, run a public neural network model on this data, and get the results. For example, a face recognition model,
Model privacy (Private Model + Public Data): For example, I have trained a model at great cost, and I do not want to expose my model, so I need to protect the privacy of the model. I can use ZKML to run a privacy-protected private neural network model, which can infer the public input to get the output.
Verifiability: ZKP is used to prove the correct execution of the ML reasoning process, making the machine learning process verifiable.
If the model is not executed on my server, but I need to ensure that the inference is executed correctly, I can use ZKML to perform an inference on a certain input and model, and it produces an output. ZKP can prove that the process is executed correctly. Even if the running process is not on my computer, I can verify ZKP to know whether the inference is executed correctly, so I can trust the result.
3. Use Cases of ZKML
Computational integrity
On-chain AI: Deploy AI models on the blockchain so that smart contracts can make decisions through AI models.
Modulus Labs: RockyBot On-chain verifiable ML trading bot
Self-improving blockchain: Enable blockchain to use the power of AI to continuously improve and modify strategies based on historical data.
Enhancing Lyra Finance’s AMM with AI.
Creating an AI-based reputation system for Astraly.
Creating AI-based compliance capabilities at the smart contract level for the Aztec Protocol
Modulus Labs:Blockchains that self-improve (link):
AIGC on-chain: Content/artwork generated by AIGC is minted on-chain into NFT. ZK can prove the correctness of the process and that no copyrighted images are used in the data set.
ML as a Service (MLaaS) transparency (link)
AI security: Use AI for fraud detection, Sybil attack prevention, etc. Train AI anomaly detection models based on smart contract data, suspend contracts if indicators are abnormal, and use ZK to upload anomaly detection proofs to the chain.
On-chain ZKML games: on-chain AI chess players, neural network-driven NFT characters, etc.
Verifiable AI model benchmarking: Use ZK to provide benchmarking proof of the model and provide verifiability for the test results of the model's performance and effectiveness.
Proof of correctness of model training: Since model training is very resource-intensive, using ZK to prove the correctness of model training is not currently available. However, many people believe that the technology is feasible and are trying to use ZK to prove that the model uses certain data/does not use certain data to resolve AIGC's copyright issues.
privacy protection
Biometric authentication/digital identity for wallets
WordCoin is providing users with a unique and verifiable digital identity by scanning their irises with the Orb biometric device. WorldCoin is working on zkml, which it plans to use to upgrade World ID. After the upgrade, users will be able to autonomously store their signature biometrics in encrypted storage on their mobile devices, download the ML model for iris code generation, and create a zero-knowledge proof locally to prove that their iris code was indeed generated from the signature image using the correct model.
Blockchain-based machine learning bounty platform
The company publishes a bounty and provides public and private data. Public data is used to train the model, and private data is used for prediction. Some AI service providers train the model and convert it into a ZK circuit. The model is encrypted and submitted to the contract for verification. For private data, predictions are made, results are obtained, and ZK proofs are generated. The ZK proofs are submitted to the contract for verification. After completing a series of operations, the AI service provider receives the bounty. zkML: Demo for circomlib-ml on Goerli testnet
Privacy-preserving reasoning: For example, using private patient data for medical diagnosis and then sending sensitive inference results (such as cancer test results) to the patient. (vCNN paper, page 2/16)
4. ZKML’s Landscape
Take a look at the ZKML map compiled by SevenX Ventures.
Hardware acceleration: Many organizations are actively developing hardware acceleration for ZKP, which is also conducive to the development of ZKML. Generally, FPGA, GPU and ASIC chips are used to accelerate the generation of ZKP. For example, Accseal is developing ASIC chips for ZKP hardware acceleration, and Ingonyama is building a ZK acceleration library ICICE, which is designed for GPUs that support CUDA. Supranational focuses on GPU acceleration, and Cysic and Ulvetanna focus on FPGA acceleration.
Input: To use on-chain data input, Axiom, Herodotus, Hyper Oracle, Lagrange will improve user access to blockchain data and provide more sophisticated on-chain data views. ML input data can then be extracted from imported historical data
Reasoning: ModulusLabs is developing a new zkSNARK system specifically for ZKML. This part can be merged with the ZKML toolset, mainly focusing on ZK-ing the model and the toolset required in the ZK-ing process. Giza is a machine learning platform based on StarkNet, focusing on fully on-chain model deployment and expansion.
Computing: Focus on building decentralized computing networks for training AI models that are accessible to everyone. They allow people to use edge computing resources to train AI models at a lower cost.
Decentralized training/computing power: Focus on building decentralized computing networks for training AI models that are accessible to everyone. They allow people to use edge computing resources to train AI models at a lower cost.
ZKML toolset: See Chapter 5 for the history of technology development. The ZAMA in the figure mainly uses fully homomorphic encryption (FHE) for privacy protection in machine learning. Compared with ZKML, FHEML only provides privacy protection but not trustless verification.
Use cases: Worldcoin, using ZKML for digital identity authentication. Encrypted biometric signatures in user devices, ZK-based machine learning models for iris recognition, running models during identity recognition to verify whether biometrics match. Using ZKP to prove the correctness of the running process. Modulars Labs makes on-chain AI trading robots. Cathie's EIP7007, zkML AIGC-NFT standard. On-chain AI chess players, neural network-driven NFT characters, etc.
5. ZKML’s technical development history
The main challenges in converting neural networks into ZK circuits are:
The circuit requires fixed-point operations, but floating-point numbers are used extensively in neural networks.
The problem of model size is that large models are difficult to convert and the circuits are large.
The development history of the ZKML library is as follows:
2021, zk-ml/linear-regression-demo, Peiyuan Liao
A linear regression circuit is implemented. Linear regression is a very basic prediction algorithm that assumes a linear relationship between the output variable and the input variable. It is suitable for predicting numerical variables and studying the relationship between two or more variables. For example: predicting house prices based on house area and other features, or predicting future sales based on historical sales data, etc.
2022年, 0xZKML/zk-mnist, 0xZKML
Based on the MNIST dataset, a neural network ZK circuit was built to recognize handwritten numbers. For example, if you write the number 2, the handwriting is recognized as 2 and a proof of the reasoning process is generated. The proof can be uploaded to the chain and can be verified using ethers + snarkjs.
In fact, the zk-mnist library currently only converts the last layer into a circuit, and does not convert the complete neural network into a circuit.
2022, socathie/zkML, Cathie
Compared to zk-mnist, ZKML converts the complete neural network into a circuit. Cathie's zkMachineLearning provides multiple ZKML toolkits such as cirocmlib-ml and keras2circom to help ML engineers convert models into circuits.
November 2022, zk-ml/uchikoma, Peiyuan Liao
Convert floating point operations in neural networks to fixed point operations. Created and open sourced a general tool and framework that can convert almost any machine learning algorithm into a zero-knowledge proof circuit that is easily integrated with the blockchain.
Visual Model -> AIGC
Language Model -> Chatbot, Writing Assistant
Linear models and decision trees -> Fraud detection, Sybil attack prevention
Multimodal Model -> Recommender System
A blockchain-friendly content generation machine learning model (AIGC) was trained and converted into a ZK circuit. It can be used to generate artworks, generate concise ZK proofs, and finally mint the artworks into NFTs.
July 2022, March 2023 update, zkonduit/ezkl
ezkl
A library and command-line tool for reasoning about deep learning models and other computational graphs in zk-snark (ZKML). It uses Halo2 as the proof system.
You can define a computational graph, such as a neural network, and then use ezkl to generate a ZK-SNARK circuit. The ZKP generated for the reasoning can be verified with a smart contract.
It is said to be able to support models with 100M parameters, but it may be very resource-intensive.
May 2023, Ddkang/zkml (Link)
zkml claims that it can use ZK to implement GPT2, Bert and Diffusion models. However, it may require a lot of memory, and it is unclear whether the proof can be stored in the smart contract.
zkml can verify the execution of a model to 92.4% accuracy on ImageNet, and can also prove an MNIST model with 99% accuracy in four seconds.
May 2023, zkp-gravity/0g
Lightweight neural network, supporting private data + public models.
In general, we can see the current exploration direction of ZKML technology:
Network Quantization converts floating-point numbers in neural networks into fixed-point numbers and lightweight neural networks (ZK-friendly).
Try to convert neural networks with large parameters into ZK circuits and improve proof efficiency (expand ZK capabilities).
6. Conclusion
ZKML is a bridge between AI and blockchain. Its significance lies in enabling blockchain to perceive the physical world, enabling smart contracts to make decisions, and running AI models with privacy protection. It is a very promising technology.
ZKML has a short history and has developed rapidly. At present, some simple neural network models can be converted into ZK circuits, and models can be put on the chain or reasoning proofs can be put on the chain. Language models are relatively difficult. At present, Ddkang/zkml claims to be able to generate ZK versions of GPT2, Bert and Diffusion models. I believe that with the development of network technology, ZK technology, and blockchain expansion technology, ZKML of language models will soon become available.