Maia 200 is born, Microsoft's microchip for Ai that wants to challenge Google and Aws

The Redmond company's spokesmen christened it thus: a new accelerator designed to make artificial intelligence faster, more efficient and more convenient in everyday use. By announcing Maia 200, Microsoft is sending a clear signal to the market, promising more performance in inference (the process of running an AI model trained to make predictions on new data) and speed of deployment at the data centre level, and clearly putting Google's TPU (Tensor Processor Unit) and Amazon's Trainium chips in its sights. Scott Guthrie, Microsoft's Executive Vice President Cloud and AI, explained in a lengthy blog post how Maia was born to 'dramatically improve the economics of AI token generation', reiterating how this solution is a key step towards making artificial intelligence not only more advanced but also more easily applicable to real, everyday cases and scenarios. Maia, in other words, is meant to be a 'boost' of the process that allows AI to respond to prompts, generate content or support decisions in real time) and a further signal that the real game on this technology is played on the silicon platforms, and thus on the infrastructure that allows models to run on a large scale and at a sustainable cost.

Microsoft explicitly focused on the inference factor and avoided 'uncomfortable' comparisons with Nvidia because the dominance of Jensen Huang's company in the field of training is indisputable.

What is Maia 200 and why inference really matters

Maia 200 is therefore a strategic choice because it is precisely inference that accounts for the bulk of the load over time. The Redmond company's turning point is explained by its desire to go beyond the limits of hardware (the same hardware used for training models is expensive and complex is training) that is powerful but complex and inefficient in terms of cost, latency and energy consumption, by focusing on a component that can significantly improve these parameters in the execution of large-scale generative and reasoning models. Maia's technical specifications give a good idea of this concept: the accelerator is manufactured by TSMC using a 3-nanometer production process, integrates over 140 billion transistors and is optimised for the use of low-precision numerical formats such as FP4 and FP8. In operational terms, its processing capabilities make it possible to reduce the number of chips required for the execution of the largest models available today and the burden per single response; Microsoft, to be clearer, is officially talking about a 30 per cent improvement in performance per dollar compared to the latest generation hardware already present in its data centres. And if so, this is no small improvement.

Another expression coined by Guthrie to present Maia 200, whose first systems are already being deployed in the Azure US Central region (in the coming months, the deployment will touch other regions in the US), is 'inference powerhouse', referring to the fact that the superchip does not only respond to the need for computing power but works for the overall balance of the system. "FLOPS aren't the only ingredient for faster AI," noted the Microsoft manager, pointing out in essence how efficiently powering models is just as crucial as mere computing power if we want to have AI that truly works in a scalable and sustainable manner. Alongside the native FP8 and FP4 tensor cores, the architecture integrates a redesigned memory subsystem, with 216 GBytes of HBM3e and a bandwidth of 7 Terabits per second, plus a significant amount of on-chip SRAM memory. At the system level, Maia 200 adopts a two-tier scale-up network based on standard Ethernet, while each accelerator offers 2.8 TB/s of dedicated bi-directional bandwidth and can be aggregated in clusters of up to 6,144 units. Technicalities aside, the ultimate goal is to keep data 'close' to the computing system, reducing bottlenecks that slow down token generation.