The Enablers of Decentralized AI

Decentralized AI is regaining prominence in recent months as more people start to explore the intersection of Web3 and generative AI. While most people agree that decentralization can have positive benefits for AI, the specific technical paths run into major roadblocks.

For instance, while we all think that decentralized compute networks (DePIN) can help counterbalance the GPU accumulation games dominated by big tech platforms, the reality is that the pre-training and fine-tuning requirements of large foundation models require concentrated GPU topologies with massive communication buses.

Similarly, decentralized data networks seem like a clear idea to mitigate the increasing data concentration in large companies. But, in practice, the attempts to solve that problem have seen very mediocre traction.

When thinking about decentralizing AI, it is important to realize that AI tends to increasingly evolve more towards centralized architectures, which makes any decentralization effort incredibly challenging. For decentralized generative AI to overcome those natural challenges, it needs to rely on or influence four key trends:

1. For decentralized AI to become feasible, open-source generative AI needs to become more mainstream

2. Decentralized AI is an inference game and nothing else

3. Web3 compute and data infrastructures need to scale to meet the demands of foundation models

4. Foundation models need to be adapted to run in decentralized infrastructures. Mostly every path to decentralized AI is dependent on flavors of these four ideas.

AI as a centralization force

In the history of technology, there are trends that fundamentally evolve towards centralized markets, while others are more open and balanced markets. Mobile is the canonical example of a technology that evolved towards a centralized market with two main players.

This is mostly given to the complex requirements in terms of hardware, software, research, supply chain, and distribution, which only a few companies can master. Other trends, such as databases, started as centralization forces with companies like Oracle, IBM, and Microsoft, and evolved into more competitive markets with a lot of open-source players.

AI has one of the most centralized DNAs of any tech in history. This is obviously due to the dependencies in areas such as compute or data, which are also fundamentally centralized. From that perspective, it is natural to expect that the AI market will be dominated by a handful of companies.

Any centralized market can use a counterbalancing force. In that sense, it is tempting to think that blockchains could be that element. While that conclusion is directionally correct, in order to materialize, it will require the push of external market forces as well as the mainstream adoption of open source generative AI and the evolution of Web3 infrastructure and foundation models.

Open-Source Generative AI Needs to Become Mainstream

There is no decentralized generative AI without open source generative AI. The wave of innovation in open-source generative AI is certainly breathtaking, but only a handful of companies, such as HuggingFace, Meta, Mistral, or Stability, are viable alternatives to OpenAI, Anthropic or Google.

Enterprise applications with strong security and privacy constraints, as well as AI startups in regulated industries, seem to be strong growth vectors for open-source generative AI. In those scenarios, Web3 AI infrastructure can become a viable alternative. If we assume that open-source generative AI will gain some level of mainstream adoption, the next challenge is to understand which use cases within the lifecycle of generative AI solutions are well-suited for decentralized infrastructures.

Decentralized AI is an inference game

In simplistic terms, the lifecycle of generative AI models can be divided into three main phases: pre-training, fine-tuning and inference.

The first two have to do with getting models to generalize knowledge based on data, while inference is related to the predictions outputted by models. The extremely large size of foundation models and their complex compute requirements make decentralized infrastructures completely impractical for pre-training and fine-tuning scenarios.

Those workflows require hundreds or thousands of GPUs connected by high-performance communication buses that can ingest data and recalculate weights in a model for weeks or months. Not to mention that the data required for pre-training and fine-tuning is often stored in cloud data centers, and it results in being completely cost-prohibitive to move it to a decentralized infrastructure.

Inference is a much more practical scenario for decentralized generative AI and, arguably, the only one that could work in the current state of technologies. From a market perspective, inference workloads also account for a larger percentage of generative AI revenues. Now that we know that decentralized AI will focus on inference workflows with open-source foundation models, we need to figure out which type of Web3 infrastructure can support those use cases.

Web3 infrastructures need to grow

The current generation of blockchain runtimes has not been designed to run large foundation models, even for inference use cases. To address that challenge, new blockchain runtimes optimized for larger and more complex compute workloads are definitely needed. Off-chain inference computation is a nice middle ground, but one that doesn’t fully address the centralization concerns with generative AI.

Foundation models need to become smaller

Last year, Microsoft coined the term “small language models” based on its work on a foundation model called Phi and the iconic paper “Textbooks is All You Need”. The small Phi was only 3B parameters and was pre-trained on a series of computer science textbooks, and it was able to outperform 70B models in math and computer science tasks.

The work on Phi signaled that smaller and more specialized models are one of the most important steps towards the adoption of generative AI. In the same way that Web3 infrastructures need to scale to adopt foundation models, the SLM trend can make models more practical to run on Web3 infrastructure. It is unlikely we will see a Web3 infrastructure running a trillion parameter model in the near future, but 2B-3B is definitely possible.

Difficult but possible path to decentralized AI

The idea of decentralized generative AI is conceptually trivial but practically very difficult. AI naturally evolves as an increasingly centralized technology, and any decentralization efforts are an uphill battle. The mainstream adoption of open source generative AI models is essential for the viability of decentralized AI infrastructures. Similarly, the current state of generative AI suggests that most of the initial use cases of decentralized AI will focus on inference rather than pre-training or fine-tuning. Finally, to make decentralized AI practical, Web3 infrastructures need to scale on several orders of magnitude, while foundation models need to become smaller and more adaptable to decentralized environments.

That combination of factors represents the best path towards decentralized generative AI. That path will be extremely difficult but, at least for now, certainly possible.