NVIDIA Cuts Token Costs by 10x with New Blackwell Platform Thanks to Team Green’s Extreme Codesign Strategy

NVIDIA’s latest Blackwell platform has set a new benchmark in token optimization for AI inference, marking a significant achievement in the field of tokenomics.

NVIDIA’s GB200 NVL72 Surpasses Hopper with 10x Enhanced Tokenomics, Noted for “Expert-Level”Parallelism

In the fast-paced landscape of AI technology, NVIDIA has prioritized enhancing the efficiency of its hardware. With the introduction of Blackwell-trained frontier AI models, remarkable advancements in token output and associated costs have come to light. Recently, NVIDIA shared insights about their collaboration with various businesses to elevate Blackwell’s performance, boasting a tenfold improvement compared to the previous Hopper generation.

Leading inference service providers such as Baseten, DeepInfra, Fireworks AI, and Together AI are leveraging the NVIDIA Blackwell platform, achieving a reduction in cost per token by as much as 10 times compared to the NVIDIA Hopper platform. These companies host sophisticated open-source models that have reached frontier-level intelligence.

By merging open-source frontier intelligence with NVIDIA Blackwell’s robust hardware-software codesign and tailored inference stacks, these providers are facilitating substantial token cost savings for businesses in diverse sectors.

– NVIDIA

NVIDIA has recognized organizations such as Baseten, Sully.ai, DeepInfra, and Latitude for their commitment to optimizing tokenomics with Blackwell. These companies benefit from reduced latency, lower inference costs, and reliable output, establishing Blackwell as the preferred technology stack for contemporary AI enterprises. Notably, Sentient Labs reported achieving a “25-50% better cost efficiency”in comparison to the Hopper platform, particularly in multi-agent and specialized AI agent deployments.

The image shows a comparison of system costs and costs associated with each token using diagrams — Image Credits: NVIDIA

The success of the Blackwell architecture can be attributed to NVIDIA’s innovative “extreme co-design”strategy, which is especially compatible with modern Mixture of Experts (MoE) architectures. The GB200 NVL72 employs a 72-chip configuration complemented by 30TB of high-speed shared memory, thus elevating expert parallelism to unprecedented heights. This architecture enables continuous batch splitting and distribution across GPUs, resulting in a non-linear increase in communication volume, a crucial factor for achieving optimal tokenomics.

Looking forward, NVIDIA aims to enhance infrastructure efficiency even further with its Vera Rubin project, focusing on architectural innovations and specialized tools, such as CPX for prefill functionality. Given the rapid evolution of AI technology, it is imperative to understand that optimizing existing hardware is just as vital as developing new systems.

Source & Images