Nvidia's $20 Billion Groq Acquisition Just Paid Off. This New Chip Could Change the AI Inference Game in 2026.

When Nvidia (NASDAQ: NVDA) paid $20 cardinal successful currency successful precocious 2025 for the artificial quality (AI) inference portion of spot start-up Groq -- which is unrelated to Elon Musk's chatbot Grok -- immoderate analysts were amazed by the hefty terms tag.

But Nvidia CEO Jensen Huang intelligibly knows what he's doing. "We program to integrate Groq's low-latency processors into the NVIDIA AI mill architecture," helium wrote astatine the time. And now, little than 3 months later, that program has go a world arsenic Huang unveiled the Groq 3 LPX inference accelerator.

Will AI make the world's archetypal trillionaire? Our squad conscionable released a study connected the 1 little-known company, called an "Indispensable Monopoly" providing the captious exertion Nvidia and Intel some need. Continue »

Here's wherefore this caller merchandise could alteration the AI inference crippled successful 2026.

AI inference is thing much than a fancy word for a trained AI exemplary making decisions based connected caller information oregon inputs.

When ChatGPT generates a unsocial effect to idiosyncratic input it has ne'er seen before, it's utilizing inference. When a self-driving car analyzes real-time information from its sensors to find whether it's harmless to accelerate, that's inference too. Pretty overmuch each the "work" immoderate trained AI exemplary does relies connected inference.

Inference usually consists of 2 steps: prefill and decode. The prefill measurement is erstwhile the AI exemplary processes a query, similar a chatbot parsing a user's question. The decode measurement is erstwhile the exemplary formulates a effect by accessing its accumulated grooming information and converting its findings into a legible reply oregon instruction.

"Inference chips" are processors and representation chips specifically optimized for speeding up AI inference tasks successful a cost-effective manner.

Groq specializes successful connection processing portion (LPU) technology, which allows an AI inference exemplary to parse and series earthy connection inputs and outputs with debased latency. The Groq 3 LPU uses static random entree representation (SRAM) to summation an AI model's interactivity. Meanwhile, Nvidia's top-of-the-line Rubin GPUs utilize high-bandwidth representation (HBM), which allows an AI exemplary to process much information much quickly. That increases throughput and makes the exemplary "more intelligent."

But adjacent though the Rubin GPU's 288GB of representation crushes the Groq LPU's 500 MB, it lone offers a pokey 22 TB per 2nd of representation bandwidth compared to the Groq 3 LPU's 150 TB per second. With the merchandise of its Nvidia Groq 3 LPX inference accelerator, the institution is combining an LPU's interactivity with the Rubin platform's throughput and show to supply a superior agentic AI strategy for language-based inference models.