Waiting For Confirmation: Bitcoin Optech’s Series On Mempool And Policy

Much of the discussion about Bitcoin technology lately revolves around various soft-fork consensus changes to the Bitcoin protocol including new opcodes or sighashes or around layer 2 technologies like Lightning. Discussion of mempool or policy (the non-consensus rules around Bitcoin nodes communicating with one another) often falls to the background. However, with recent high transaction volume and launch of various NFT and token projects/platforms and the accompanying feerate spike, mempool and policy came to the forefront.

User of Inscriptions and other protocols ran into issues with standardness policy rules, leading many to question their purpose and look for ways to remove or subvert them. Regular Bitcoin users ran into issues with fees or bumping fees. In an effort to educate and engage the community, Bitcoin developers Gloria Zhao and Murch authored a 10-week series about mempool and relay policy called ‘Waiting for confirmation’ on the Bitcoin Optech website.

The series starts with an overview of what this cache of unconfirmed transactions we call a mempool is and why we have one. Having a mempool of unconfirmed transactions allows better fee estimation for wallets, faster downloading of new blocks, and supports a decentralized transaction and block relay network.

However, Bitcoin miners are under no obligation to include these unconfirmed transactions in a block. As block space is limited, miners select transactions with the highest feerate by transaction weight to maximize their profits. The post on incentives details some nuances around feerates including the fact that onchain fees are paid not in proportion to the transaction amount but by the size of the transaction and complications that arise with relationships between different transactions.

But what should a transaction’s feerate be? That is the goal of feerate estimation: to translate a user’s urgency into a minimal feerate a transaction should pay. Transactions in the mempool and transactions in recent blocks can help provide a good start for estimating transaction fees.

In ‘Bidding for block space’ Gloria and Murch discuss practical strategies to get the most for your transaction fees. When creating a transaction, consider coin selection, using newer output types like taproot’s P2TR that allow for fee savings, or batching. After broadcasting a transaction, techniques like Child Pays For Parent (CPFP) and Replace By Fee (RBF) can be used to increase the feerate of a transaction that is taking too long to confirm.

With the goal of a robust and decentralized network of Bitcoin nodes in mind, we want it to be as cheap and accessible as possible for anyone to run a node. Not only that, but a node’s resources must be protected from DoS attacks. Transaction policy rules that are more restrictive than Bitcoin’s consensus rules help protect node resources (including memory, computational resources, and bandwidth) by enforcing limits on its untrusted peers on the Bitcoin P2P network.

Likewise, network-wide resources including the UTXO set, protocol upgrade hooks, the size of the block chain and the computational effort required to process it, also need to be protected. A series of other policy rules, including limits on arbitrary data publishing to the block chain, minimum feerates, and limits on low value outputs all help safeguard these network resources.

While policy is optional, Bitcoin Core doesn’t offer many ways to configure them. In ‘Policy Consistency’ Gloria and Murch outline potential ramifications of altering some of the default policies and why Bitcoin Core has historically been conservative with the configurability of policies.

It is not just individuals running nodes that should be aware of transaction policy rules. Wallets, services, and layer 2 protocols that broadcast transactions must be designed with policy rules in mind to avoid creating transactions that are rejected and to ensure they can get confirmed, even during times of fluctuating feerates. For example, different types of pinning attacks are possible on L2 settlement transactions like Lightning that take advantage of limitations in mempool policy to prevent incentive-compatible transactions from entering mempools or getting confirmed.

Just as policy rules have been changed or added in the past, there are a series of proposals to improve policy as well. Package relay, cluster mempool, version 3 transaction relay, and ephemeral anchors are a few that are under development currently.

However, since transaction relay policy changes to Bitcoin Core can impact many ecosystem participants, they require collaboration, socialization, feedback, and testing from the wider Bitcoin community prior to consideration. The authors note: “Decentralized decision-making is a challenging process, but necessary to support the diverse ecosystem of protocols and applications that use Bitcoin’s transaction relay network.”

Readers should consider getting involved in the different avenues of discussion and participation.

Bitcoin Optech’s also has a podcast special that highlights all of the 10 weeks of our discussion with Murch and Gloria, including comments by guest speakers and questions from the audience.

Thank you to Gloria Zhao and Mark “Murch” Erhardt for authoring the series as well as explaining each weekly article on the Bitcoin Optech Podcast.