Zcash’s Halo Breakthrough Is a Big Deal – Not Just For Cryptocurrencies

Michael J. Casey is the chairman of CoinDesk’s advisory board and a senior advisor for blockchain research at MIT’s Digital Currency Initiative.

The following article originally appeared in CoinDesk Weekly, a custom-curated newsletter delivered every Sunday exclusively to our subscribers.

An underappreciated, sideline payoff from cryptocurrency R&D is that it also generates advances within the sector’s component technologies.

The most important are occurring within the field from which the term “cryptocurrency” derives. Cryptography – essentially, the study of mathematical secrets – is as old as the exploration of ciphers in ancient times. But in the past 10 years, thanks largely to the invention of bitcoin and censorship-resistant money, it’s seen an explosion of activity.

That’s especially in the sub-field of zero-knowledge proofs, which enable the verification of facts that are derived from a secret the verifier cannot access. These advances matter because zero-knowledge proofs offer the tantalizing prospect of people transacting in confidence without accessing potentially compromising information about each other. Its potential goes beyond the narrow realm of cryptocurrencies to face the ultimate challenge of the Internet age: achieving security with privacy.

This is why a breakthrough by the Electric Coin Company, the startup behind zcash, is rich with potential. ECC had already been an engine of progress for cryptography by advancing the use of zk-SNARKS, another cryptocurrency-inspired addition to the zero-knowledge proof toolkit, with which zcash produces a provably auditable blockchain without revealing users’ addresses (a disclosure note: Digital Currency Group, CoinDesk’s parent company, is an ECC investor).

But the company’s recent announcement of Halo, a “trustless recursive” version of zero-knowledge proofs that provides a massively scalable solution to the field’s unwieldy reliance on “trusted setups,” is arguably bigger. If the discovery by ECC researcher Sean Bowe holds up to scientific scrutiny, it could one day unleash a host of powerful, real-world applications for the digital age that go far beyond cryptocurrency.

Might it even achieve the impossible: lowering  the heat that zcash CEO Zooko Wilcox and his cofounders relentlessly receive  for the 20% founder fee built into the cryptocurrency’s protocol, a deal that has delivered them millions of dollars’ worth of tokens since the launch in 2016? The founders justify the fee on the grounds that it both pays for maintenance and rewards research and development to strengthen the protocol. For now at least, this looks like a discovery that ECC can flag as money well spent – not just for zcash, but for the entire crypto ecosystem.

A proof of proofs

Halo allows a user to both prove that no one involved in the initial establishment of a large-scale zero-knowledge proof system has created a secret backdoor with which to later amend the code and that that secure state has existed over the course of ongoing updates and changes to the system. Until now the risk of fraud at setup meant that zero knowledge proofs often required elaborate, costly procedures at the outset to instill confidence in users. (A prime example was the zcash genesis “ceremony” –  recorded live on YouTube and documented in an entertaining episode for NPR’s Radiolab  – when various founders and outside particiapants based in multiple locations went to extraordinary lengths to jointly and securely create the initial key pair and then demonstrate that none of them would ever have access to the private key.)

As such, zero-knowledge proofs were too cumbersome for anything other than privately proving small one-off facts. Repeating the inefficient, time-consuming trusted setup over and over again was costly. To be sure, one-off trustless solutions known as “bulletproofs” have been around since 2017, but they lack the recursive quality needed to verify the ever-accumulating information within a large, growing changing database.

Halo gets around this problem by establishing an accumulated “proof of proofs,” such that the latest mathematical output contains within it a proof that all prior claims to the relevant secret knowledge have themselves been sufficiently proven through a similar process. In a dramatic compression in computational requirements, all that’s now needed to verify the veracity of the entire database’s current state is a single mathematical proof. (The way Wilcox explained it to me, the process sounded similar to the efficiency gains of Merkle tree structures, which aggregate previously hashed information into a single root hash output.)

Cheap full nodes

The scaling benefits of this lightweight proofing system were illustrated with a mid-September demonstration by the EEC team using the bitcoin blockchain.  They generated a proof of the current block’s proof-of-work integrity that also contained proofs of the integrity of every preceding block, all the way down the chain to Satoshi Nakamoto’s genesis block of January 3, 2009.

In light of the  fraught debates  in the bitcoin community over full nodes, decentralization and block sizes, this sounds like game-changer material. While there will still need to be nodes that read the full blockchain to identify transactions, the overall task of verifying the integrity of a blockchain could become a much less costly problem for the network as a whole. Ordinary users could achieve the ease-of-use and efficiency they need but do so with their own full verification nodes. It would thus negate the need for so-called SPV wallets, which rely on others to verify on the user’s behalf and so create a trust problem. For the network, the result could be greater decentralization at a lower cost.

The ECC is planning to integrate Halo into the zcash blockchain as a Layer 1 scaling solution. If it works, the zcash network might much more cheaply handle significantly larger amounts of on-chain data. This is a markedly different approach to the scaling problem from the Layer 2 model favored by bitcoin supporters of the Lightning Network, where scale is achieved by taking transactions off chain. If it works for zcash, one wonders whether bitcoin cash developers will be tempted to integrate it into their protocol to lower the cost of maintaining the larger blocks they adopted in the contentious 2017 fork from Bitcoin Core.

Bigger visions

But it’s the potential for non-cryptocurrency solutions that makes Halo an especially exciting prospect. Wilcox even claims Halo “may turn out to be a building block for the next generation of the Internet and other such social infrastructure.”

In a conversation, he pointed to the vulnerabilities of large, ever-changing centralized databases such as that of the famously hacked credit scorer Equifax, as well as those of different states’ DMV outlets and of siloed medical record custodians. All must share information with other parties but struggle with the risks of doing so. “Now instead of them spitting out copies of a full report of the data, they keep the only copy but spit out zero knowledge proofs,” Wilcox said.

The ideal, however, would be to dispense with the centralized record-keeper entirely. Wilcox thinks Halo-like zero-knowledge proofs will pave the way. Taking the prior example one step further, he said, “What if instead of me saying ‘here is a proof that Equifax says I haven’t had any defaults over the last 10 years,’ I can say ‘here is a proof from all the 100 people that have lent to me over the past 10 years and each of them attests to me not having defaulted?”

Getting to such a utopia won’t happen quickly. Regulation, corporate incumbency and behavioral inertia will continue to pose resistance. And, to be clear, Bowe’s mathematical proof still needs to be subject to rigorous peer review.

But even if holes are found in the current iteration, they will be patched. Better versions will emerge.

The process of follow-on research that this discovery will unleash in all areas of the digital economy is undeniable. And if the world isn’t ready for such a radical reorganization of how we manage sensitive information, it will eventually be moved to adopt such changes by the relentless buildup of vulnerable databases and the ongoing attacks against them by increasingly sophisticated hackers. That’s a trend that led Juniper Research to recently assert that  cybercrime will cost the global economy a stunning $5 trillion a year by 2024.

The world badly needs fixes for these giant challenges. Cryptocurrency developers are doing as much as anybody to find them.

Zooko Wilcox image via CoinDesk archives