Microsoft: Maia 200 goes into production; the new Maia 300 chips are due in 2027
From next year, Maia will become the main inference system for Microsoft products, with installations on a multi-gigawatt scale. The aim is to handle more requests using the same amount of energy and to control a larger proportion of the costs associated with Copilot and the agents.
Microsoft has announced that, from 2027, it will scale up deployments of its Azure Maia family of chips to a multi-gigawatt scale, making it the primary inference system for its products and services. The aim is to reduce the cost of the operations that Copilot performs every time it summarises a meeting, suggests a line of code or reads a document. To respond, the model must be run, data must be moved and memory must be populated: across millions of requests, the efficiency of these operations has a significant impact on the cost of artificial intelligence.
What is Maia and what is inference used for?
Let’s put it this way: every time Copilot summarises a meeting, suggests a line of code or reads a document, Microsoft incurs a cost. The model has to be run, data has to be moved, and memory has to be allocated. With millions of requests, these operations become a matter of industrial scale. Inference is the work that a pre-trained model carries out to respond to a request. During training, it acquires the capabilities it then uses in its interactions with users. With agents, the cost can rise: to carry out a task, a system may query the model several times, use external tools, examine the results and correct its course. A single request can therefore involve numerous processing steps. Essentially, the new Azure Maia plan addresses this aspect of the artificial intelligence bill. According to Microsoft, it runs Microsoft 365 Copilot’s production workloads and supports the Microsoft AI team in model development. GitHub Copilot and Microsoft Foundry will follow by the end of the year. The hardware runs OpenAI models, internal MAI models and open-source families. The programme has moved beyond the demonstration phase and has become part of the services from which the company must generate revenue and margins. This is why it is so important for Microsoft’s finances and for the AI market.
What is the Maia 300 like?
Microsoft presents it as a comprehensive system, developed in tandem with the software, memory and networks. The reason is straightforward: during inference – particularly when generating responses and when the model has to handle long contexts – the computing units may end up waiting for data. Being able to perform multiple operations is of little use if the information arrives too slowly.
Maia 300 aims to reduce these delays. The software determines where to store data and how to transfer it between memory units, processing units and accelerators. A local on-chip memory keeps the information close to the units that use it, reducing traffic to the main memory. In this way, a greater proportion of the energy can be allocated to processing.
The published specifications outline the design’s dimensions: eight stacks of HBM4 memory, with a total peak bandwidth of 18 terabytes per second and a TDP of 950 watts per accelerator. Computing, memory control and interconnects are distributed across several chiplets – small integrated components housed within the same package. Those dedicated to computation will be manufactured using TSMC’s N2P process.
