AMD Expands AI/HPC Product Lineup With Flagship GPU-only Intuition MI300X with 192GB Reminiscence

Alongside their EPYC server CPU updates, as a part of as we speak’s AMD Information Middle occasion, the corporate can be providing an replace on the standing of their nearly-finished AMD Intuition MI300 accelerator household. The corporate’s next-generation HPC-class processors, which use each Zen 4 CPU cores and CDNA 3 GPU cores on a single package deal, have now grow to be a multi-SKU household of XPUs.

Becoming a member of the beforehand introduced 128GB MI300 APU, which is now being known as the MI300A, AMD can be producing a pure GPU half utilizing the identical design. This chip, dubbed the MI300X, makes use of simply CDNA 3 GPU tiles relatively than a mixture of CPU and GPU tiles within the MI300A, making it a pure, high-performance GPU that will get paired with 192GB of HBM3 reminiscence. Aimed squarely on the massive language mannequin market, the MI300X is designed for patrons who want all of the reminiscence capability they will get to run the biggest of fashions.

First announced back in June of last year, and detailed in greater depth back at CES 2023, the AMD Intuition MI300 is AMD’s massive play into the AI and HPC market. The distinctive, server-grade APU packs each Zen 4 CPU cores and CDNA 3 GPU cores on to a single, chiplet-based chip. None of AMD’s rivals have (or can have) a mixed CPU+GPU product just like the MI300 collection this yr, so it offers AMD an fascinating answer with a very united reminiscence structure, and loads of bandwidth between the CPU and GPU tiles.

MI300 additionally contains on-chip reminiscence by way of HBM3, utilizing 8 stacks of the stuff. On the time of the CES reveal, the best capability HBM3 stacks had been 16GB, yielding a chip design with a most native reminiscence pool of 128GB. Nonetheless, due to the latest introduction of 24GB HBM3 stacks, AMD is now going to have the ability to provide a model of the MI300 with 50% extra reminiscence – or 192GB. Which, together with the extra GPU chiplets discovered on the MI300X, are supposed to make it a powerhouse for processing the biggest and most complicated of LLMs.

Underneath the hood, MI300X is definitely a barely easier chip than MI300A. AMD has changed MI300A’s trio of CPU chiplets with simply two CDNA 3 GPU chiplets, leading to a 12 chiplet design total – 8 GPU chiplets and what seems to be one other 4 IO reminiscence chiplets. In any other case, regardless of excising the CPU cores (and de-APUing the APU), the GPU-only MI300X seems lots just like the MI300A. And clearly, AMD is aiming to reap the benefits of the synergy in providing each an APU and a flagship CPU in the identical package deal.

Uncooked GPU efficiency apart (we have no onerous numbers to talk of proper now), a bit a part of AMD’s story with the MI300X goes to be reminiscence capability. Simply providing a 192GB chip by itself is a giant deal, on condition that reminiscence capability is the constraining issue for the present era of huge language fashions (LLMs) for AI. As we’ve seen with latest developments from NVIDIA and others, AI prospects are snapping up GPUs and different accelerators as rapidly as they will get them, all of the whereas demanding extra reminiscence to run even bigger fashions. So with the ability to provide a large, 192GB GPU that makes use of 8 channels of HBM3 reminiscence goes to be a large benefit for AMD within the present market – at the very least, as soon as MI300X begins delivery.

The MI300 household stays on observe to ship sooner or later later this yr. Based on AMD, the 128GB MI300A APU is already sampling to prospects now. In the meantime the 192GB MI300X GPU will likely be sampling to prospects in Q3 of this yr.

It additionally goes with out saying that, with this announcement, AMD has solidified that they are doing a versatile XPU design at the very least 3 years earlier than rival Intel. Whereas Intel scrapped their mixed CPU+GPU Falcon Shores product for a pure GPU Falcon Shores, AMD is now slated to supply a versatile CPU+GPU/GPU-only product as quickly as the tip of this yr. On this timeframe, it will likely be going up in opposition to merchandise reminiscent of NVIDIA’s Grace Hopper superchip, which though is not an APU/XPU both, comes very shut by linking up NVIDIA’s Grace CPU with a Hopper GPU by way of a excessive bandwidth NVLink. So whereas we’re ready on additional particulars on MI300X, it ought to make for a really fascinating battle between the 2 GPU titans.

General, the strain on AMD as regards to the MI300 household is critical. Demand for AI accelerators has been by means of the roof for a lot of the previous yr, and MI300 will likely be AMD’s first alternative to make a major play for the market. MI300 is not going to fairly be a make-or-break product for the corporate, however moreover getting the technical benefit of being the primary to ship a single-chip server APU (and the bragging rights that include it), it is going to additionally give them a contemporary product to promote right into a market that’s shopping for up all of the {hardware} it could possibly get. Briefly, MI300 is anticipated to be AMD’s license to print cash (ala NVIDIA’s H100), or so AMD’s keen buyers hope.

AMD Infinity Structure Platform

Alongside as we speak’s 192GB MI300X information, AMD can be briefly asserting what they’re calling the AMD Infinity Structure Platform. That is an 8-way MI300X design, permitting for as much as 8 of AMD’s top-end GPUs to be interlinked collectively to work on bigger workloads.

As we’ve seen with NVIDIA’s 8-way HGX boards and Intel’s own x8 UBB for Ponte Vecchio, an 8-way processor configuration is at the moment the candy spot for high-end servers. That is each for bodily design causes – room to position the chips and room to route cooling by means of them – in addition to one of the best topologies which are accessible to hyperlink up numerous chips with out placing too many hops between them. If AMD is to go toe-to-toe with NVIDIA and to seize a part of the HPC GPU market, then that is yet another space the place they’re going to want to match NVIDIA’s {hardware} choices

AMD is looking the Infinity Structure Platform an “industry-standard” design. Accoding to AMD, they’re utilizing an OCP server platform as their base right here; and whereas this suggests that MI300X is utilizing an OAM type issue, we’re nonetheless ready to get express affirmation of this.