Author: Muhammad Yusuf, researcher at Delphi Digital; Translation: Golden Finance 0xjs
This article is divided into 8 parts: 1. Dark pools in the 1960s; 2. Mixers, pools, and intermediate states; 3. Privacy architecture; 4. Internal operations of dark pools: wallet settings, order discovery and matching, liquidity sources; 5. Business model; 6. MEV and censorship resistance; 7. SEC compliance; 8. Unfinished thoughts.
Before we delve into the inner workings of blockchain-based dark pools, let’s first understand how dark pools came to be, why they continue to work, and to what extent they are a game changer.
A 15th-century Japanese painting shows a man wearing a mask throwing a dollar into a well filled with black ink. AI rendering
1. Dark Pool in the 1960s
In 1969, when computers were still big enough to take up a room or two and traders were shouting orders around the trading floor, institutional investors needed a better way to buy and sell stocks without setting off a domino effect in the market. At the time, Jerome Pustilnik founded Instinet, pioneering electronic trading on Wall Street. Institutional investors placed orders, and Instinet would match buy and sell orders and execute them. For something like Instinet to work, you need a lot of volume on both the demand and supply sides of the order. So what makes institutional investors willing to use it? Confidentiality. Instinet allows large companies to operate anonymously by hiding their identities and orders from other participants and the broader market. While this prevents their trades from affecting the market, it also means they are less at risk of being front-runners by other traders.
As of 2022, there are more than 60 dark pools registered with the U.S. Securities and Exchange Commission (SEC). Some are operated by exchanges, such as the New York Stock Exchange (NYSE) or NASDAQ, some are operated by brokers, such as Morgan Stanley's MS Pool, Goldman Sachs' SigmaX, and some are independently operated, such as Liquidnet or MatchNow.
The problem with operator-run dark pools is that the operators may have more incentive to misbehave than to comply with regulations. The reason is simple: the profits from corruption > the costs of corruption. The operators may make more profit from corruption than having to pay fines. In 2018, the SEC fined Citigroup $12 million for misleading investors about operating their dark pools while leaking confidential order information to high-frequency traders who executed over $9 billion worth of orders and profited from them.
Traditional dark pool users face the risk of being gamed by operators. It’s a heavy price they have to pay for participating in a broken trust model. Since 2011, dark pool operators have paid more than $340 million in fines to settle charges. That’s a relatively small price to pay compared to the profits they could have amassed. Blockchain-based dark pools attempt to abstract away the need to trust operators who may act maliciously. But there are still problems.
2. Mixers, Pools, and Intermediate States
Blockchain was originally designed to be completely transparent. While it facilitates auditability, this is a double-edged sword. If you receive a salary payment on-chain, anyone with your wallet address can see how much you have received over the years, and how much you will receive in the future.
Decentralized exchanges (DEXs) and wallets may be tracked by wallets and copy trading platforms, the latter of which even allow anonymous people to sniff out well-performing traders and complicate their trading strategies. Large orders also face the risk of being front-run by searchers in public mempools.
Before we get into dark pools, let’s first differentiate between mixers and dark pools. Mixers are a subset of dark pools. Mixers like Tornado Cash mix tokens to break the connection between wallets and assets, making it difficult to trace the source of funds. Dark pools, on the other hand, not only break the connection between wallets and tokens, but also help users trade with each other without revealing any information about the parties involved.
Now that we’ve looked at the origins and motivations behind traditional dark pools and differentiated mixers from dark pools, let’s dive into blockchain-based dark pools that introduce privacy into their infrastructure through balance models and PETs (privacy-enhancing technologies) such as zero-knowledge proofs, MPC, and FHE.
Not all privacy or privacy-focused DeFi companies are included
As of the time of writing, dark pools such as Portal Gate, Renegade, and Tristero are under development. Privacy protocols such as Panther and Railgun are already online, facilitating privacy DeFi transactions other than swaps, such as staking, providing liquidity, and lending.
3. Privacy Architecture
Balance Model
Blockchains are state machines. The state consists of accounts and transactions. As transactions are made to accounts, a new set of updates are added to the chain each time the state transitions. The balance of the account is also updated. Balance models are different ways that blockchains track and manage user account balances. Blockchains like Ethereum, Solana, and Polygon use the account model, while blockchains like Bitcoin, Zcash, and Monero use the UTXO model. These models differ in how they manage and present the blockchain state.
The account model is similar to a bank account, showing the current state as a set of addresses and their balances. Whenever you make a payment, you send exactly the amount, no change is required, and no new accounts are created. But the UTXO (Unspent Transaction Output) model is not that simple. When you make a payment, your entire balance is transferred and your existing balance is destroyed. The remaining amount you have is added to the new balance. The UTXO model shows the current state as a graph of all spent and unspent transaction outputs.
To illustrate this, consider the following scenario:
Alice's wallet contains two UTXOs of 6 coins and 5 coins respectively, while BobBob's wallet contains one UTXO of 5 coins. When Alice sends 5 coins to Bob, she combines her two UTXOs into one transaction, making it difficult for observers to figure out the source of the 5 coins she sent to Bob and what happened to the remaining balance in her UTXO. This privacy protection mechanism effectively masks the flow of funds and maintains the confidentiality of transaction participants.
Newly generated UTXOs have no known owners and require advanced on-chain analysis to be linked to a single user. The account model is fungible. It encourages address reuse, so the history of a single user can be easily tracked.
With the UTXO model in place, double spending is also fairly difficult. Each UTXO has a unique invalidator that is derived from itself. When a transaction is validated, i.e. when a UTXO is spent, its invalidator is also spent. This prevents the transaction from being spent again.
Several versions of the UTXO model have been used in privacy-focused L1 and L2. Aleo uses the Record model, which is a variation of the UTXO model. Aztec and Polygon Miden use a mix of the UTXO and Account models.
The UTXO model used by Zcash has greatly inspired the UTXO model used in dark pools. Individual protocols have modified it to support multiple assets and accounts. Protocols such as Portal Gate, Penumbra, Railgun, and Renegade use a note-based mechanism where each UTXO is a note. These notes, like UTXOs, contain information about the asset, order, and value, but are only visible to the owner/user. These notes are stored in a Merkle tree that stores the state of the chain, called the state commitment tree. These notes may be created as a result of internal transfers or settlements within a dark pool, or by paying fees to relayers. Renegade uses MPC to match UTXOs (orders) and update private state. Multiple notes can also be stored in a single note.
In Panther Protocol, each asset or account is represented as a zAsset or zNFT, and the UTXO model operates through a Merkle tree called the "Bus", which processes UTXOs in batches of up to 64 UTXOs per batch, optimizing cost efficiency for users. "Bus Operators" are responsible for processing, and users incentivize them with rewards denominated in ZKP. When executing a single UTXO, Panther has the so-called "Taxi" Merkle tree. To support multi-chain UTXOs, the "Ferry" Merkle tree.
Proof System
This is where PET (Privacy Enhancing Technology) comes into play. Zero-knowledge proofs are used to prove possession of correct information without revealing it. In the context of dark pools, zero-knowledge proofs can be issued to prove that a user has sufficient funds and permissions to place an order and have it matched and settled, while hiding these details from the dark pool operator, other traders, and external third parties.
Proof systems are cryptographic structures designed for generating these proofs. There are many types of them, each of which produces proofs of different sizes, requires different time frames, and consumes different computational resources to generate and verify. Here, we will take a deep dive into two well-known proof systems: Groth16 and UltraPlonk, which have found applications in the dark pools we have discussed.
Panther, Penumbra, and Railgun utilize the Groth16 proof system to generate proofs. Groth16 is known for generating proofs of consistent size, which makes it a popular choice since proof size can affect transaction size, gas costs, and throughput. Portal Gate and Renegade, on the other hand, use the UltraPlonk proof system.
Both Groth16 and UltraPlonK require a trusted setup, where a group of trusted parties collaborate to establish a common set of parameters. These parameters are used to verify proofs in a way that the verifier does not need to trust the prover and the prover does not need to trust the verifier. It is worth noting that a trusted setup implies trust assumptions on the parties involved. Secure multi-party computation can be used to prevent any single party from gaining access to the underlying setup structure. Groth16's trusted setup relies on the input of multiple parties, while UltraPlonk only requires the participation of a single party.
4. The inner workings of dark pools
Now let’s go through the inner workings of dark pools, how they preserve privacy while conducting trades, provide liquidity, prevent MEV, their censorship resistance, and regulatory compliance in the following sections.
Wallet Settings
In most cases, you will need to create new dedicated wallets to interact with dark pools. Railgun has a third-party wallet provider called Railway Wallet, which allows you to create an encrypted EOA that allows you to hold a balance of shielded tokens/NFTs and privately call any public smart contract you want through the Relay Adapt mechanism.
To Renegade, a wallet is like a UTXO that has a private key and can sign transactions. You will need to generate a new UTXO and commit it with a ZK proof. This proof proves that the new UTXO is actually new and that you have not yet spent the funds it contains. Most operations such as deposits, orders, and trades will invalidate this wallet, causing you to create a new wallet to ensure that double spending does not occur. The user experience is the same as any other Ethereum wallet.
To use Penumbra, you need to create a new wallet, just like when you first launch any other Cosmos chain. Afterwards, the wallet can be funded with another IBC transfer.
With Panther and Portal Gate, KYC processes are in place. Ethereum wallets can be used, but connected wallet users must verify their passports to obtain credentials to interact with the protocol. Business entities require KYB to share their business details. Verification is done off-chain through registered compliance providers. These credentials expire periodically, after which verification must be repeated.
Order discovery and matching
In terms of order discovery and matching, general decentralized exchanges such as Uniswap and Curve use an automated market maker approach to match orders with liquidity pools. This approach is not the best choice for privacy protection because when orders are matched with public liquidity pools, the order details are publicly visible on the blockchain. This is not ideal for traders who trade in large quantities because this may reveal their intentions to other participants in the market who may exploit this information. Some dark pools that ultimately match with liquidity pools or use other external public contracts for order matching utilize relayers, which act as a long wall between traders who must protect privacy and external liquidity sources. In the next section, we will examine how dark pools can obtain liquidity while maintaining privacy.
Remember, traders have deposited funds into the wallet of the dark pool. We will now review what happens when an order is placed. Some dark pools take a peer-to-peer approach to order matching, while others take a peer-to-pool approach, using PETs such as zero-knowledge or multi-party computation, while their internal contracts interact with external public contracts. Taking a peer-to-peer approach or crossing orders between traders can help better execution without slippage. Because if an order is matched with a liquidity pool, the price of the liquidity pool may change between the time the order is placed and the time the order is executed. When orders are crossed between traders, the order will be executed at the price agreed between them.
Renegade uses a peer-to-peer approach, and when a trader submits an order, they also select a relayer who can bid (match the order) for a fee. The trader generates a ZK proof called a "valid commitment" that shows that they own their wallet and their order. The "handshake" is then sent to the network to match the handshakes of other traders. The handshake is essentially an immutable list of valid commitment ZK proofs, hashed order details, invalidators, and key pairs. When a trader's handshake is found to match the handshake of another trader, they proceed to multi-party computation. Renegade uses collaborative ZK-SNARKs to prove that the trader's order truly matches the counterparty's order. This ensures that when order matching is performed, the order details are hidden from other parties. Once the order is matched with all the required proofs, a new wallet is created, and after verifying that the wallet is indeed new, the exchanged tokens are deposited into it to prevent double spending. The user experience is the same as using any Ethereum wallet.
For Portal Gate, when a trader submits an order, the API node receives the order, encrypts it and generates a ZK proof, and forwards the order to Book, which is a set of off-chain nodes that collect orders and run matching algorithms in an FHE (fully homomorphic encryption) environment. This means that encrypted orders can be matched without decryption. After the order is matched, the total result of all matching orders is displayed, without showing the details of a single matching order. This again helps protect privacy.
Railgun Railgun, when a trader places an exchange order through Railgun, a smart contract called Adapt Module performs multiple operations, namely obtaining the specified private balance in the trader's wallet that they want to exchange and unshielding it. The Adapt Module will also verify that the order (UTXO) has not been canceled/spent. The 0x API then swaps the assets across aggregated DEX liquidity, looking for the best interest rate. The Adapt Module then shields the exchanged assets back to private balances so that their activity or addresses are not revealed. This workflow can also be applied to other DeFi transaction types.
Panther Protocol operates in a similar way, using a module called Zswap to connect Panther's MASP (Multi-Asset Shielded Pool) with other DeFi protocols. MASP is not so much a "pool" as a collection of "append-only" Merkle trees, where each leaf is a commitment to a UTXO, representing an IOU of the assets deposited with the MASP. When a trader creates an order, Zswap aggregates quotes from other DeFi protocols for the trader to choose from. Once the order is placed, Zswap creates a cryptographic time-based escrow contract that privately facilitates the swap between users without revealing any details. Once the assets are swapped, the user receives a tokenized IOU in the form of a ZAsset (ZNFT) from Panther's MASP.
Penumbra’s swap module is also called Zswap, but it works very differently. When a trader creates an order, the assets in the trader’s private balance are burned and the encrypted order value is sent to the validator. There is also a fee for processing the order. The user is given a “swap” NFT to preserve a record of the event. The NFT is identified by a combination of various parameters, including the asset traded, the fee, the input amount, and the encryption key. The validator aggregates the order inputs from multiple traders into a batch and then executes those inputs against a pooled liquidity position. Once executed, the output is delivered to the trader in proportion to each trader’s input. In this case, individual data remains private, but the aggregated data is exposed when the validator aggregates orders to execute against a pooled liquidity position.
Finding Liquidity
Dark pools like Renegade and Portal Gate work by anonymously crossing buy and sell orders directly between users, but liquidity on the other side of the order may not always be available. With a regular DEX, you are almost guaranteed liquidity for your order, but you make the tradeoff at the expense of privacy and price impact. Privacy, because anyone can view what you are selling or buying. Price impact, because guaranteed liquidity comes with a premium, and you end up paying that premium along with your order. With a dark pool, order information is hidden both before and after the trade.
Renegade follows this design philosophy, matching order flow with counter order flow without going through any intermediary liquidity providers. In the absence of liquidity, there is a mechanism called "Indication of Interests" where traders can reveal certain details of their orders to relayers who match orders. Details such as price, size, asset, order type, etc. are effectively proven through zero knowledge. If a trader chooses to reveal such details, other traders will be able to view these orders on the order book, which may lead to faster execution of these orders.
Panther Protocol is more than just a dark pool. It has many other DeFi activities that can be done privately. Users first deposit into a multi-asset shielded pool, where the deposited assets are represented as ZkAssets (ZkNFTs). Essentially an IOU token, similar to the stETH you get when you stake Ethereum. A plugin called DeFi Adaptor connects MASP privately with existing DeFi protocols. Zswap and Ztrade are adapters that connect MASP with DEXs like Uniswap, Quickswap, Curve, etc.
Penumbra, like Panther or Railgun, is an avenue for diverse DeFi activity. In Penumbra, users end up creating their own small AMM when they start an LP position. So you have a collection of tens of thousands of pooled liquidity positions created by users who set their own desired fees. This fragments liquidity, but the Zswap/DEX engine takes into account and synthesizes all of these individual liquidity positions into a single AMM. This makes it possible to route incoming trades across the entire liquidity graph.
Automaton is used to facilitate liquidity as a backup to the ledger through Portal Gate, an anonymous AMM-DEX, where relayers run the order book used to cross-users trade with each other. If the order book cannot facilitate a trade due to lack of liquidity, the order will be routed to Automaton. Liquidity on Automaton will be bootstrapped like other AMMs.
Railgun’s Railway DEX uses 0xAPI, which is built on 0x. This means that orders originating from the Railway DEX are routed to find the best price to execute on the 0xAPI DEX aggregator. Once a trade route is found, the wallet generates proofs to exchange and shield incoming tokens from 0xAPI to the trader’s balance using funds from the trader’s balance.
5. Business Model
Tradfi dark pools are more or less seen as an elite club where people with large balances trade. They pay a membership fee to the dark pool and can place orders anonymously, as well as receive relevant over-the-counter order flow from the other party. Trading fees are usually lower than on tradfi exchanges. Tradfi dark pools can also act as brokers, buying securities from sellers at a lower price and then selling them to another buyer at a higher price.
Dark pools in crypto move away from this “operator”-centric design and act more like “facilitators.” They aim to generate revenue through two main components. Protocol fees for successfully matched orders and a portion of the relayer fees that traders pay to relayers for processing their trades. A portion of the relayer fees can also go to relay operators. Traders can also choose to run their own relayers for better privacy and to avoid paying relayer fees.
Decentralized dark pools in the crypto space may face a cold start problem. Unlike DEXs that trade with liquidity pools, most dark pools let users trade with each other by matching buy and sell orders. For such a system, it is crucial to let traders on both sides of the market place orders. It is also important to note that dark pools in tradfi are hosted by multi-billion dollar banks such as Goldman Sachs and exchanges such as the New York Stock Exchange. This makes it relatively easier to provide order flow to users. There are also independent dark pools like Liquidnet, who estimated that they would need at least 100 buy-side firms to operate on their platform to reach critical mass as of today, but ended up with only 38 firms. Liquidnet is currently ranked among the top 3 dark pools in the world.
Image credits @yusufxzy, Source: Coingecko
Centralized exchanges have an average of 10 times more trading volume than decentralized exchanges, as their broader user base is attracted by better user experience and regulatory compliance, which in turn increases supply to meet demand. OTC liquidity is relatively difficult to obtain.
Most crypto OTC trading is done through OTC desks and escrow smart contracts and even in some cases through Telegram groups. It’s really shocking. People are getting scammed all the time. It’s easy for scammers to impersonate Telegram, edit messages and trade details. Using a dark pool may not guarantee you instant settlement of illiquid tokens or NFTs, but it ensures a higher level of security and privacy. Since OTC trades are done through dark pools, clients of OTC desks also don’t have to pay broker fees.
Volume is critical for dark pools to operate effectively. It helps match orders efficiently and maintain the anonymity of traders. The more anonymous funds there are in a pool or the more orders there are in an order book, the more difficult it is to connect depositors and traders to their assets. This is because there are more potential depositors who can try to link assets to the dark pool. It is also important to consider the type of token when protecting assets. Shielding a common stablecoin like USDC or DAI provides greater anonymity than shielding an unknown meme token with very few depositors.
6. MEV and censorship resistance
Most DEXs running on public chains act as an open, transparent distributed ledger that allows the existence of MEV. Order details on DEXs such as Uniswap or Curve are not confidential. This allows searchers and builders to rearrange transactions in bundles and blocks accordingly to extract MEV.
Privacy is great, but it’s a subjective question. Tornado Cash is great at allowing people to conduct financial activities anonymously on-chain. Since it’s open source and free for anyone to use, there are also criminals who use it to launder stolen funds. North Korean hacker group Lazarus has stolen nearly $1 billion ($958 million) to date. OFAC (Office of Foreign Assets Control) maintains a list of Specially Designated Nationals and Sanctioned Individuals. Malicious individuals and entities, such as Lazarus, and even Tornado Cash’s smart contract addresses, are listed on OFAC’s list of violations. Protocols that are not OFAC compliant have a high chance of having their transactions excluded from blocks added to the chain. This is because the top 5 of the 6 largest block builders are OFAC compliant. Is this a serious censorship resistance issue? Yes. It doesn’t stop with builders. Validators and relayers can also choose to omit transactions or blocks.
In dark pools, since ZKPs are published for each order, order details are mostly hidden from all third parties, including validators, eliminating the need to reveal order matching or even settlement details. This architecture can be MEV and censorship-resistant, or at least it is so as long as the trading workflow remains within the cryptographic premise of the dark pool. In cases where public contracts are called or shared sorters are used, MEV capture opportunities may arise again.
7. SEC Compliance
Whether you are operating within the protocol or as an individual user, the last thing you want is to see your transaction hanging in a pending state, not processed within a block, thus risking the state transition to succeed, or having the SEC call you for tea. These concerns became a harsh reality for the Tornado Cash founders and developers, who were arrested on money laundering charges. The TC frontend is also illegal/defunct in many countries and is not supported by multiple CEXs. The TC smart contract itself is still functional and can be used by people with some technical knowledge.
To try and avoid this, dark pools and other privacy-focused protocols have found ways to allow users to remain compliant at their own discretion.
Renegade can allow each trader to select a set of counterparties to trade with. An individual trader may only want to perform some basic AML/sanctions checks on their counterparties, while an institution may want to perform KYB/KYC checks on their counterparties. This counterparty selection logic using ZKPs for compliance checks can be set up before the MPC begins.
Portal Gate and Panther Protocol work with compliance oracles, which in turn work with traditional compliance service providers to verify off-chain user accounts for KYC/KYB. Once verified, users are issued a ZK compliance certificate. This certificate allows their wallet to be used with the dark pool protocol. Verification needs to be done regularly.
With Penumbra, users can use a “transaction view” to demonstrate off-chain compliance to disclose their activity, including selective sources of funds. Similarly, with Railgun, users have a “view key” that allows them to see transactions and balances from X to Y block times.
Railgun uses “Proofs of Innocence” discussed in the Privacy Pool 2.0 paper developed by Chainway. Proofs of Innocence help you prove that your transactions are legitimate without revealing any transaction details. The Merkle Tree acts as an aggregation of all previous UTXOs (transactions, balances). In this way, it is possible to determine that a specific balance is part of a specific set of transactions. But in order to prove that a specific balance is not part of a certain set of transactions, a Sparse Merkle Tree that records NULL values is required. With this proof, it is possible to generate proofs that verify that the balance is not part of a certain set of transactions. In this case, recursive SNARKs are used to prove that the chain of proofs from the initial deposit to the final withdrawal is accurately calculated to prove that the user's balance is not part of a certain set of transactions.
8. Unfinished Thoughts
Dark pool users rely on public prices as a benchmark for traders, but asset prices on public exchanges may not reflect movements occurring within dark pools. Until 2014, price discovery was a one-way relationship between dark pools and public exchanges. FINRA and the SEC implemented an initiative that mandates that post-settlement dark pool trade information be publicly released on a security-by-security basis with a four-week delay. It is unclear whether similar public disclosure standards could be applied to cryptocurrency dark pools in the future to promote better price discovery for all market participants.
Pure dark pools must channel order flow on both sides of the market to reach critical mass. We may see many OTC trades move to dark pools as they are more secure, private, and cost-effective than OTC desks and Telegram groups. In the case of private DEXs interacting with public liquidity pools or external smart contracts, disguising leaked information may cause some trades and traders to unhook.