Digging Deeper on Crypto Exchange Data and Transparency

Clay Collins is CEO and co-founder at Nomics, an API-first cryptoasset data company delivering market data to institutional crypto investors and exchanges. The opinions expressed in this article are his own, and do not reflect CoinDesk’s position.​ 

The following article originally appeared in “Institutional Crypto” by CoinDesk, a free newsletter for the institutional market with news and views on crypto infrastructure delivered every Tuesday. Sign up here or at the link below. 

On March 19th, Bitwise came out with a report to the U.S. Securities and Exchange Commission (SEC) that made two claims of particular interest to us at Nomics.

These claims were:

1. Only 10 exchanges had actual volume
2. 95 percent of reported exchange volume is fake

As a data company, the first thing we did was look at these exchanges to see what the commonalities were:

What our examination of the Bitwise report has revealed

Eight of the 10 cryptocurrency exchanges identified by Bitwise as good actors provide historical trade level data (i.e. the most granular and audible form of pricing data available). That is, the one thing we found in common among the good actors identified by Bitwise, is they were very transparent about trading activity.

In contrast, we found that every single one of the exchanges explicitly called out by Bitwise as bad actors provides limited trading history and virtually no granularity around trading activity.

Transparency of Exchanges Identified As Trusted vs Suspect, According To Bitwise’s Report

In other words, the good actors from Bitwise’s report had highly transparent data practices, and the bad actors were largely not transparent. (Source: Nomics)

It makes sense that opacity around exchange data would be correlated with fake volume, toxic activity, and wash trading. Indeed, just like an IRS audit, the more data history and granularity provided by an exchange engaging in nefarious activities, the more likely they are to be caught.

We’re excited about the Bitwise report (and the prospect of a Bitwise ETF). But, we also believe the Bitwise report left some questions unanswered (which we assume will be addressed in due time, given the cadence and thoroughness of the regulatory process they’re undergoing).

Two things about the Bitwise report surprised us:

• First, the lack of scrutiny around the report, particularly by Crypto Twitter, which has a reputation for being an all-out war zone.
• Second, the degree to which the public took a document meant to market the approval of Bitwise’s ETF to the SEC, and overgeneralized the findings to apply to all exchanges (even ones not analyzed by Bitwise) and all cryptoassets. Indeed, Bitwise only demonstrated explorations of BTC/USD and BTC/USDT pairs.

I believe the crypto community’s response to the report lacked criticism and nuance. Don’t get me wrong, as a bitcoin hodler I want this ETF to be approved. But I believe the claims made by the report are just too large and global to take at face value, and that a more nuanced conversation needs to be had.

Our seven critiques of the Bitwise report and the public’s response:

1. The report’s primary purpose is to persuade the SEC to allow a Bitwise ETF. For this reason, it has an inherent bias. That is, the document is a marketing document first, and a research document second.
2. The report seems to have a “base 10 bias.” The number 10 seems very curious: why not 9 or 11?
3. Their conclusions are not falsifiable. Bitwise is looking at the data and giving individual exchanges a thumbs up/down. There is no stated independent test that stands apart from Bitwise that a third party can independently apply. You either have to trust their conclusions or not. (Note: Bitwise has informed me that they will be sharing a more falsifiable methodology in the near future).
4. The cryptosphere has responded to the report as if its conclusions are stuck in time; there is far too much over-extrapolation. Because Bitwise does not state a formal methodology that can be used to independently rate an exchange, a third party cannot update the list. So today, some of the exchanges identified by Bitwise as being good actors might have flipped and be engaging in wash trading. And exchanges that Bitwise did not indicate as having “actual volume” may have cleaned up their practices.
5. People have over-extrapolated the report to apply to cryptoassets other than bitcoin, to markets other than BTC/USD & BTC/USDT, and to exchanges that don’t list bitcoin. This report, as it stands right now, does not apply to other cryptoassets, to pairs other than with USD & USDT, and exchanges other than the 80 they examined (all of which list bitcoin). While Bitwise’s analysis was perfect for a bitcoin ETF, it’s a mistake to apply their findings broadly to all crypto markets.
6. There is no stated time frame for the most important data point in the report. For example, they assert that 95 percent of the volume on global crypto exchanges is fake. Is this volume on one day of analysis? Is this for 2019 YTD? 2018? The lack of detail is important to note.
7. The public’s response to the report has been unfair to upstanding exchanges not examined by Bitwise. Too many readers have given the report a cursory glance and concluded that there are only 10 exchanges with true volume: this is an incorrect assumption.

Conclusion

Much of the response to Bitwise’s report has involved restricting volume analysis (and market data in general) to the 10 exchanges identified as having “actual volume.” However, this reduction in scope results in analysis that’s stuck in time; it also over-extends Bitwise’s findings to cryptoassets other than bitcoin, and exchanges that don’t have BTC/USD or BTC/USDT markets.

But the real problem with restricting global cryptosphere analysis to just 10 exchanges is that it shifts our reliance from one centralized provider (e.g. CoinMarketCap) to another (e.g. Bitwise). Instead, as an industry we need to get iteratively better at spotting suspect exchange behavior on a near real-time basis. And we need formal open source methodologies that can be used by any independent third party to evaluate exchange activity in real time.

Dollars under magnifying glass image via Shutterstock

Interested in receiving a weekly email with updates on market infrastructure, regulation and institutional crypto products? Sign up for our free Institutional Crypto newsletter here.