AMD and Qualcomm Pursue “SOCAMM” Memory Technology for Next-Generation AI Products, Following NVIDIA’s Lead

AMD and Qualcomm are actively investigating the integration of SOCAMM memory into their artificial intelligence (AI) solutions. This exploration comes in response to ongoing challenges with memory limitations that hinder the performance of current AI systems.

Adoption of SOCAMM Memory: From NVIDIA to Competitors

Initially developed with NVIDIA in mind, SOCAMM is a memory standard that has gained early traction among the company’s products. For those unfamiliar with it, SOCAMM is based on LPDDR DRAM technology, commonly utilized in mobile and energy-efficient devices. What sets SOCAMM apart from alternatives like High Bandwidth Memory (HBM) and LPDDR5X is its upgradeability. Unlike traditional components that are soldered into place, SOCAMM can be more easily replaced or upgraded, making it a compelling option to complement HBM in tackling memory-intensive tasks.

Recent information from a Hankyung report indicates that both AMD and Qualcomm are looking to incorporate SOCAMM modules into their upcoming AI system architectures. Notably, these companies are investigating a unique design strategy that differs from NVIDIA’s implementation. Their approach involves creating a ‘square’ module featuring two DRAM components arranged in parallel rows. This design aims to enhance power management directly on the module through a Power Management Integrated Circuit (PMIC), facilitating efficient power regulation and ensuring high-speed operation without complications.

As adoption of SOCAMM expands, the demand for this memory type is expected to grow, driven largely by the requirements of agentic AI applications. The accessibility of terabytes of memory per CPU enables AI agents to manage millions of active tokens efficiently. Although SOCAMM’s throughput may not match that of HBM, its performance characteristics make it a viable and energy-efficient option.

Currently, NVIDIA plans to utilize SOCAMM 2 within its Vera Rubin AI clusters. Given that AMD and Qualcomm are also exploring this memory technology, industry watchers can anticipate its inclusion in their next-generation AI clusters, potentially enhancing the performance of future AI applications.

Source & Images