Inside the ‘Private Mempools’ Where Ethereum Traders Hide From Front-Running Bots

Ethereum is swarming with bots that are programmed to front-run transactions. The bots exploit the brief window of time between when transactions are submitted, and when they’re officially finalized, to copy trades from other users, quickly execute them, and in doing so eat into any would-be profits.

It’s a practice called maximal extractable value (MEV), and it’s a huge nuisance to novice crypto traders and to veterans alike.

But Ethereum’s transaction pipeline has undergone a quiet shift over the past two years as more of the chain’s users have embraced “private mempools” to execute their trades – bypassing the blockchain’s “public” transaction lobby to avoid broadcasting trades to the whole world before they’re finalized. This helps to prevent MEV and help users get better settlement for their transactions.

While there are obvious benefits to this stealthier mode of using Ethereum, experts say private mempools carry risks of their own.

“I think most everyone, including myself, expects there to be more private transactions moving forward, not less,” Matt Cutler, CEO of MEV firm Blocknative, told CoinDesk. “I think the big question in my mind is, would more private transactions be a good thing or a bad thing for the network?”

What is MEV?

Understanding transaction privatization requires understanding some quirks with how the second-largest blockchain network works today.

Submitting a transaction to Ethereum (and similar blockchains) generally means sending it to the chain’s “public” mempool, which is a giant waiting area for transactions that are still waiting to get executed.

The thousands of validators that run Ethereum behind the scenes scoop those mempool transactions into blocks – usually with help from third-party “block builders” who organize them according to certain criteria, including how much they pay to validators in fees. Once they’re added to a block, the transactions are officially written to the blockchain, where they are cemented permanently.

With this system comes a clear issue: Transactions in Ethereum’s public mempool are like sitting ducks. The seconds (or minutes) of queue time leaves enough for quick-witted trading bots, sometimes called “searchers,” to front-run transactions or execute other strategies that eat into the profits of regular traders.

“Private” mempools are presented as a stealthier alternative, a way for decentralized finance (DeFi) traders to transact without exposing their trades to the prying eyes of MEV (maximal extractable value) bots. Those bots preview mempool transactions to ink a profit.

On average, roughly 10% of Ethereum transactions are routed through private mempools each day, which is double the share of private transactions the chain recorded in 2022, according to Blocknative. While the proportion of private transactions on Ethereum has oscillated a fair bit in recent months (private transactions peaked above 20% some days in 2023 before stabilizing closer to 10%), experts expect the trend toward mempool privatization to increase in the coming months.

Why go private?

The benefits of private mempools are clear.

Private mempool services from firms like CoW Swap, bloXroute and Blocknative offer to hide transactions from MEV bots.

These setups are useful for large organizations and individuals who want higher security and privacy for their transactions. They’re also used by sophisticated trading firms that want quick, guaranteed transaction settlement and can’t afford to broadcast their trades to competitors before they’re filled.

Mempools aren’t just for big-time traders and privacy geeks, though.

Some private mempool services, like CoWSwap, will pay direct kickbacks (sometimes called “refunds”) to users whose transactions have the potential to net block builders their own MEV profits.

There’s also a growing field of products that use private mempools to guarantee better settlement for DeFi traders. UniswapX, which is run by Uniswap, the biggest decentralized exchange on Ethereum, uses a kind of private mempool to help retail traders get better prices for their token swaps.

UniswapX’s private mempool connects traders directly with market-makers, with the idea being that this direct connection can net traders better strike prices than they’d get on the open market.

What are the risks?

There are some risks, though.

Most pressingly, there’s the worry that private mempools might cement new middlemen at key areas in Ethereum’s transaction pipeline: “I expect these to be centralizing in their nature,” Cutler said.

MetaMask, the most popular Ethereum wallet, is poised to introduce a transaction-routing feature in 2024 that could catalyze the biggest yet shift away from Ethereum’s public mempool. But in a telling email exchange with CoinDesk when the feature was first reported, officials at Consensys, MetaMask’s parent company, pushed back against the “private mempool” label – hinting at some of the term’s baggage.

The new feature from MetaMask dodges Ethereum’s public mempool – ostensibly as a way to help users transact more cheaply and with better ease-of-use. MetaMask’s specially-built sidetrack to the public Ethereum mempool is similar to the private mempool concept described in this article, but Consensys shies away from the “private mempool” moniker because it’s associated with certain risks that MetaMask claims it’s tech doesn’t have.

Private mempools frequently ask users to place their implicit trust into individual third parties, rather than the broader Ethereum network, to guarantee their transactions are executed. Unless private mempools are engineered carefully (and the details of MetaMask’s system are not altogether clear), private mempools these third parties could upcharge users or front-run them just like a normal MEV bot would.

Ethereum’s public transactions lobby comes with downsides, but is also one of the main ways the network stays decentralized, and it provides users a clear window into the status of their transactions.

Toni Wahrstätter, a researcher at the Ethereum Foundation, told CoinDesk via a direct message on X that “The impact of private mempools on Ethereum’s network is a nuanced issue.”

On the positive end, Wharstätter noted that “more companies are now open-sourcing their data,” meaning Ethereum’s research community has been able to conduct more analyses into private mempool traffic.

Also, “while they might lead to more centralization among builders and searchers, they are unlikely to affect the crucial aspect of validator decentralization,” Wharstätter added.

However, there are still some risks. “Looking ahead, I anticipate a rise in private order flow,” Wahrstätter continued. “It’s important to monitor and address any potential centralization issues among builders, as this could threaten key features like censorship resistance. If such centralization becomes significant, we’ll need to take steps to mitigate its impact.”