Microsoft Unveils Custom Chips to Boost AI Workloads and Enhance Data Center Efficiency

Today at the Ignite developer conference, Microsoft announced two new chips for its data center infrastructure: the Azure Integrated HSM and the Azure Boost DPU.

These custom-designed chips will be released in the coming months. They aim to tackle security and efficiency issues in existing data centers. Their main goal? To optimize servers for large-scale AI workloads.

This announcement follows the launch of Microsoft’s Maia AI accelerators and Cobalt CPUs. It’s a significant step in the company’s plan to rethink and improve every layer of its technology stack—from silicon to software—to support advanced AI.

Microsoft, led by Satya Nadella, also shared new strategies to manage power usage and heat emissions in data centers. Many people are concerned about the environmental impact of AI operations.

Recently, Goldman Sachs published research predicting that advanced AI workloads could drive a 160% increase in data center power demand by 2030. By the end of the decade, these facilities might consume 3-4% of global power.

While Microsoft continues to use top-notch hardware from companies like Nvidia and AMD, it is pushing forward with its custom chips. Last year, at Ignite, Microsoft made waves with the Azure Maia AI accelerator and the Azure Cobalt CPU, which is an Arm-based processor designed for general-purpose compute workloads on the Microsoft Cloud.

Now, Microsoft has expanded its custom silicon portfolio, focusing on security and efficiency. The Azure Integrated HSM features a dedicated hardware security module that meets FIPS 140-3 Level 3 security standards.

Omar Khan, vice president for Azure Infrastructure marketing, explained that this module strengthens key management. It ensures that encryption and signing keys remain secure within the chip. Importantly, this doesn’t compromise performance or increase latency.

The Azure Integrated HSM uses specialized hardware cryptographic accelerators. This allows secure, high-performance cryptographic operations right within the chip’s isolated environment. Unlike traditional HSM architectures that require network round-trips or key extraction, this chip handles encryption, decryption, signing, and verification entirely within its dedicated hardware.

While the Integrated HSM enhances data protection, the Azure Boost DPU optimizes data centers for high-volume data streams. It focuses on power efficiency and can handle millions of network connections.

This is Microsoft’s first offering in this category. The Boost DPU works alongside CPUs and GPUs, combining multiple server components into one piece of silicon. This includes high-speed Ethernet and PCIe interfaces, network and storage engines, data accelerators, and security features.

The Boost DPU operates through a sophisticated hardware-software co-design. A custom, lightweight data-flow operating system enables higher performance and lower power consumption compared to traditional setups. Microsoft expects this chip to run cloud storage workloads at three times less power and four times the performance of existing CPU-based servers.

In addition to the new chips, Microsoft shared updates on improving data center cooling and optimizing power consumption. For cooling, the company introduced an advanced version of its heat exchanger unit—a liquid cooling “sidekick” rack.

While specific performance gains weren’t disclosed, it can be retrofitted into Azure data centers. This helps manage heat emissions from large-scale AI systems using AI accelerators and power-hungry GPUs like those from Nvidia.

On the energy management front, Microsoft announced a collaboration with Meta to create a new disaggregated power rack. This aims to enhance flexibility and scalability. Each disaggregated power rack will feature 400-volt DC power, allowing for up to 35% more AI accelerators in each server rack. This enables dynamic power adjustments to meet varying AI workload demands.

Microsoft is open-sourcing the cooling and power rack specifications through the Open Compute Project. Regarding the new chips, the company plans to install Azure Integrated HSMs in every new data center server starting next year. However, the timeline for the DPU rollout is still unclear.