IBM’s new AI chip offers 22X speedup with ‘mind-blowing’ energy efficiency

IBM recently debuted a new prototype artificial intelligence (AI) chip purported to be both faster and far more energy efficient than any chip currently available. 

According to research published in Science Magazine on Oct. 19, the new chip, dubbed NorthPole, “achieves a 25 times higher energy metric” on a relevant benchmark, “and a 22 times lower time metric of latency.”

Ostensibly, this translates to the potential for post-GPU performance at a fraction of the cost in energy requirements.

Damien Querlioz, a nanoelectronics researcher at the University of Paris-Saclay in Palaiseau, described NorthPole’s energy efficiency as “mind-blowing,” in an article published on Nature.

Per the IBM Research team’s paper:

“NorthPole outperforms all prevalent architectures, even those that use more-advanced technology processes.”

One of the major impediments to improving AI processing is called the “von Neumann bottleneck.” Using currently available architecture, AI chips tend to have faster processing capabilities than the memory they require to run processes. As a result, latency is introduced whenever information is sent between the processing unit and random access memory.

This is especially true at “the edge,” where chips and data are stored together. Removing this bottleneck has long been considered by many experts to be the key to running powerful neural networks locally on devices.

Publishing this week in @ScienceMagazine, IBM Research’s newest prototype AI chip, NorthPole, could help us move toward more energy-efficient AI. Learn more: https://t.co/qyKIUGg5ld pic.twitter.com/eFklLNSqkW

— IBM Research (@IBMResearch) October 19, 2023

According to IBM Research, the new prototype chip built in the company’s Alamaden, California laboratory bypasses the von Neumann bottleneck by, essentially, integrating the memory component onto the processing chip itself.

As the chip’s lead developer, Dharmendra Modha, puts it, NorthPole is “an entire network on a chip” that “forges a completely different path from the von Neumann architecture.”

The benchmark used to demonstrate the chip’s effectiveness, ResNet50, is a 50-layer neural network primarily used to test computer vision tasks such as image classification.

The NorthPole hardware’s reported results on this benchmark indicate that it could perform exceptionally well at associated tasks such as autonomous surgery, operation of self-driving cars and other vehicles, and numerous robotics-related endeavors.

IBM Research is already years into research on the next chip using the NorthPole architecture. According to the company blog, “this is just the start of the work for Modha on NorthPole.”