Google Partners with Marvell on Two-Chip TPU Strategy Aiming to Transform AI Inference for ASICs

Recent reports indicate that Google is collaborating with Marvell to develop two innovative chips, one dedicated to enhancing existing Tensor Processing Units (TPUs), and the other representing a next-generation TPU design.

Google and Marvell Collaborate on Next-Gen AI Chips

According to a report from The Information, discussions between Google and Marvell have commenced, focusing on the creation of two specialized chips aimed at optimizing AI inference.

While details on the progress of these discussions remain scarce, it is clear that Google has proposed two distinct chips: one for enhancing current TPUs and the other as an entirely new TPU architecture. This suggests that there’s a foundational plan in place for their development.

The purpose of the two chips diverges significantly. The first chip, related to the TPU, will not be a custom silicon but rather a memory processing unit (MPU) designed to work in conjunction with a TPU. This innovative MPU is expected to relieve some of the memory demands typically placed on the chip or system, thereby facilitating more efficient in-memory processing.

A reflective close-up of a square chip labeled Ironwood with a blurred bokeh background.

The second chip under discussion represents a next-generation TPU, tailored specifically for AI inference models. Currently, Google’s flagship AI accelerator is the TPU v7 or Ironwood series, which boasts impressive specifications including 192 GB of high-bandwidth memory (HBM) and a peak performance of 4614 TFLOPs. This powerful TPU is integrated within the Superpod, which consists of 9216 chips.

As these developments unfold, we anticipate that the forthcoming Google TPUs, in conjunction with the new MPUs, will significantly enhance the memory subsystem. This improvement is expected to lead to faster and more efficient AI model performance, particularly in the realm of AI inference.

Source & Images