Microsoft Plans to Make Maia Its Primary Inference System. Maia 300 Adds 36 Percent More Compute and 2.5 Times the Memory Bandwidth.

On Monday, October 12, 2026, Microsoft announced Maia 300, the third generation of its in-house AI accelerator. The chip itself is a 2027 story. The sentence that matters today is in Scott Guthrie's post: Microsoft will scale the Maia family to multi-gigawatt deployments starting in 2027, "making Maia the primary inference system for Microsoft products and services."

That is a bigger claim than a spec sheet. Microsoft is saying in writing that the default home for serving its own AI products will be its own silicon, not merchant GPUs.

Read the numbers, though, and the chip tells you something specific about where inference is heading. The headline gain is not compute. It is memory.

What Microsoft Announced

The architecture deep dive, by Rani Borkar, president of Azure Hardware Systems and Infrastructure, and Saurabh Dighe, lays out the part. Compute sits on chiplets built on TSMC's N2P process, with memory controller and I/O functions moved to separate dies. Each accelerator carries eight HBM4 stacks, 288 GB in total, at 18 TB/s, inside a 950-watt TDP.

The rack is the real product. Ninety-six accelerators sit in 24 four-accelerator trays inside a 167-kilowatt envelope, with 27.6 TB of HBM and 1.7 PB/s of aggregate HBM bandwidth per rack. A second tier of optical switches links up to 128 racks into what Microsoft calls a superpod of 12,288 accelerators. Liquid cooling removes more than 80 percent of the rack's heat.

Microsoft's performance claims come in two forms. Against its own previous chip, it says a full 96-accelerator rack delivers up to 7x the rack-scale token throughput of Maia 200. Against the market, Guthrie writes that Microsoft expects Maia 300 "to deliver 40% more tokens per megawatt compared to leading GPU-based systems." Neither post names the GPU system behind that comparison.

The Spec Sheet, Side by Side

Maia 200 was announced on Monday, January 26, 2026, 259 days before Maia 300. Here is how the two compare on the figures Microsoft has published.

Maia 200 figures from Microsoft's January announcement as reported by Thurrott and Pulse 2. The Maia 200 rack count is implied by Microsoft's statement that Maia 300 doubles the accelerators per rack. Percent changes are our arithmetic.

Why the Memory Number Is the Story

Compute goes up 36 percent. HBM bandwidth goes up roughly 2.5 times. Per watt of chip TDP, HBM bandwidth roughly doubles: by our arithmetic, 18 TB/s over 950 W against 7 TB/s over 750 W comes out at about 2.03x. Microsoft separately claims up to 2.5x more performance per watt than Maia 200.

That ratio is a design choice. Generating tokens one at a time, the decode phase, is usually limited by how fast weights and the KV cache can be pulled out of memory, not by raw arithmetic. Microsoft says as much: it calls Maia 300 an inference-first architecture for "long-context, memory-intensive and decode-heavy AI workloads," and the support for MXFP4 and sub-3-bit storage formats is another way of moving fewer bytes per token.

The 7x rack claim fits the same pattern. Twice the accelerators per rack accounts for 2x. That leaves roughly 3.5x per accelerator, by our arithmetic, far more than a 36 percent compute bump can explain on its own. The rest has to come from bandwidth, the new scale-up fabric, lower precision formats and software, in proportions Microsoft has not broken out.

Who Runs on It, and Who Does Not

Maia 200 is already doing real work. Guthrie writes that it "is now serving Microsoft production workloads, including Microsoft Copilot," with GitHub Copilot and Microsoft Foundry to follow by the end of the year, and that it is deploying across the United States, Europe and Australia. For built-in AI features in Teams, Copilot and Cowork, Microsoft reports up to 1.7x more tokens per watt than leading GPU infrastructure.

On models, the post says Maia 200 "runs OpenAI models, Microsoft's own MAI models and leading open-source model families." Anthropic is not among the model families Microsoft named. In June we wrote about reported early-stage talks to put Claude inference on Maia 200; nothing in Monday's posts says whether those went anywhere.

Maia 300's planned workloads are all Microsoft's own: multi-step agentic coding for GitHub, model serving for Foundry, inference and data generation for the MAI team, and latency-sensitive work for Dragon Copilot. Neither post describes a Maia instance that outside customers can rent directly, the way Google sells TPU capacity and AWS sells Trainium.

Training is a separate lane. Alistair Speirs, a Microsoft general manager, told CNBC that Microsoft partners with Nvidia and AMD for training models, and added: "We reserve a little bit of optionality there, because this is an industry that's changing very fast." CNBC also cited Evercore ISI analysts estimating that Maia will be about 10 percent of Microsoft's AI chip spending by the first quarter of 2027.

What We Cannot Check Yet

Every performance figure in this story comes from Microsoft. The 40 percent tokens-per-megawatt claim has no named GPU baseline, so it cannot be compared with Nvidia's own numbers. It is also worth noticing that Maia 200's GPU claim, 1.7x tokens per watt, is larger than Maia 300's 1.4x per megawatt. The two are measured differently and probably against different GPU generations, so this is not a regression. It does show how much each number depends on what it is measured against.

Production deployments begin in 2027, with no month given. Volume is unclear too. A September report citing people close to Microsoft put the 2027 target above 300,000 chips; Notebookcheck, citing Reuters, says Microsoft could deploy more than a million next year. Microsoft's posts give no unit count. Microsoft says it will share more at Ignite next month.

Our Take

The phrase "primary inference system" is the news, and it is a measured one. Microsoft is not leaving Nvidia. It is moving the workload that repeats on every request, serving tokens for Copilot, GitHub and Foundry, onto a chip it controls, and keeping the lumpy, capability-chasing training work on merchant GPUs, at least for now. Google, Amazon and Meta all run custom accelerators of their own; Microsoft has now put a plain word, primary, on what its chip is for.

The design tells you which bottleneck Microsoft expects to matter. A chip that buys 2.5x the bandwidth for 36 percent more compute is built for agents that hold long contexts and generate many tokens per task. We think that is the right bet on where 2027 traffic goes.

The weak spot is verification. Maia 300 has no outside benchmark, no named GPU baseline and no date more precise than a year. Maia 200 was announced in January, and Foundry is still on its "by end of year" list. Until a third party can measure tokens per megawatt on a Maia rack, the 40 percent is a target, and we would treat it that way when pricing anything built on Azure.

Three things to watch over the next 60 days:

• Whether GitHub Copilot and Foundry actually move onto Maia 200 by the end of the year, as Microsoft says they will.
• Whether Microsoft names the GPU system behind the 40 percent claim, or publishes any Maia result on a third-party benchmark, at Ignite.
• Whether a non-OpenAI frontier lab, Anthropic in particular, shows up on the list of model families Maia serves.

Sources: Microsoft (Scott Guthrie), Azure Maia 300 architecture, Microsoft Maia 200 announcement, CNBC, Notebookcheck, Digital Today, Thurrott, Pulse 2, Analytics Insight.