AI Server Liquid Cooling: The Complete Guide

AI servers run dangerously hot. When you pack eight high-performance GPUs or next-generation ASICs into a single chassis, the thermal design power (TDP) easily exceeds 10kW per server. Relying on fans and airflow is no longer a viable engineering strategy. You need liquid. If you design hardware or manage procurement for high-performance data centers, you face a critical transition. You must choose the right liquid cooling components to protect million-dollar server racks from thermal throttling and catastrophic leaks.

This guide provides everything you need to specify, size, and source your cooling architecture. It serves as your comprehensive AI liquid cooling overview. We strip away the marketing noise and look directly at the engineering facts. You will learn exactly when to use friction-stir-welded cold plates, how to properly size a coolant distribution unit (CDU), and how to future-proof your facilities for extreme 2026 density levels.

1. Why AI Density Forces the Switch to Liquid Cooling

The transition from air to liquid is a matter of basic physics. As AI workloads grow more complex, silicon manufacturers pack more transistors into smaller package areas. This creates massive heat fluxes that air simply cannot carry away.

1.1 Why air cooling hits a physical wall

Water possesses a volumetric heat capacity nearly 4,000 times greater than air. To remove the same amount of heat, you must move a massive volume of air, which requires large fans running at extreme RPMs. This creates unacceptable acoustic noise and consumes up to 30% of a data center’s total power budget just to spin the fans.

Traditional air-cooled heat sinks for AI servers also run into spatial constraints. A 1000W chip requires a massive copper and aluminum fin stack. In a dense 1U or 2U server chassis, there is simply no physical room for heat sinks large enough to dissipate this thermal load. Standard data center configurations max out at around 30kW to 40kW per rack with optimized air containment.

1.2 How liquid cooling works vs air

Liquid cooling brings a high-capacity heat transfer fluid directly to the hottest components. Instead of blowing cold air across a fin stack, liquid routes through micro-channels inside a metal plate bolted directly over the processor. The fluid absorbs the heat, carries it out of the server, and rejects it to the facility water system.

When configuring your facility, workloads matter. The hardware utilized in liquid cooling for LLM training vs inference varies in density. Training racks run continuous, steady-state high loads, while inference racks experience burst loads. Liquid cooling easily absorbs these thermal spikes due to the thermal mass of the fluid in the loop.

To help you understand how to cool high-density AI racks, review this basic comparison between air and liquid capabilities:

Decision Matrix:

· Use air cooling when: Your rack densities remain strictly below 30kW, chips are under 400W TDP, and legacy facility infrastructure cannot be upgraded.

· Use liquid cooling when: You deploy any modern AI accelerator, rack densities exceed 40kW, or your power usage effectiveness (PUE) mandates strictly prohibit heavy fan usage.

2. Cold-Plate Technologies and When to Use Them

The cold plate is the heart of the system. It sits between the silicon die and the coolant loop. Selecting the right manufacturing method for your cold plate determines your thermal resistance, pressure drop, and leak safety. You must match the technology to your specific server architecture.

2.1 Skived-fin cold plates

Skiving involves shaving a solid block of copper or aluminum to create ultra-thin, densely packed fins. A cover is then sealed over the fins to create a fluid channel.

· Pros: High surface area for heat transfer; excellent thermal performance; low tooling cost for custom sizes.

· Cons: Can result in higher pressure drops across the plate if the fin pitch is too tight; sealing the cover requires O-rings or brazing, introducing minor leak risks if poorly manufactured.

· Use this when: You need rapid prototyping, your budget is tight, and you have strong pump pressure available to overcome the internal resistance.

· Avoid this when: Your pump system requires ultra-low pressure drops across the server loop.

2.2 Brazed liquid cold plates

Vacuum brazing fuses multiple layers of CNC-machined metal together inside a high-temperature vacuum furnace. Engineers design complex, optimized internal fluid channels that balance thermal extraction with fluid dynamics.

· Pros: Highly customizable internal flow paths (e.g., U-shape, parallel channels); excellent structural integrity; can cover multiple chips (CPU, GPU, memory) with a single large plate.

· Cons: Higher initial tooling costs; longer lead times for manufacturing.

· Use this when: You need a single, large cold plate to cool multiple components on a dense AI motherboard simultaneously.

· Avoid this when: You are doing low-volume test runs where tooling costs outpace the component budget.

2.3 Friction-stir-welded (FSW) cold plates

Friction stir welding uses a rotating tool to generate frictional heat, plastically mixing two metal pieces without melting them. This creates a solid-state joint with no filler material. The resulting seal is as strong as the parent metal itself. Read our AI server rack cold plate guide for deep specifications on FSW performance.

· Pros: Absolute highest reliability; virtually zero risk of leaks; exceptional burst-pressure tolerance; no flux or filler materials to contaminate the coolant.

· Cons: Requires specialized, expensive manufacturing equipment; design geometries are slightly restricted by the path of the welding tool.

· Use this when: Leak prevention is your absolute highest priority, you are deploying in mission-critical environments, and you use high-pressure liquid loops.

· Avoid this when: You require highly complex, non-linear, tight-radius sealing paths that an FSW tool head cannot easily navigate.

2.4 Direct-to-chip cold plates

As we approach limits of standard thermal interface materials (TIMs), engineers are removing the chip's integrated heat spreader (IHS). Direct-to-chip cold plates for AI accelerators place the micro-channel cooling surface directly against the bare silicon die.

· Pros: Eliminates a layer of thermal resistance; necessary for the absolute highest TDPs (1200W+).

· Cons: Extremely fragile assembly process; high risk of cracking the silicon during mounting.

· Use this when: Standard cold plates fail to keep your next-generation ASIC below its maximum junction temperature (TjMax).

· Avoid this when: You lack a highly controlled cleanroom assembly environment, or you are cooling standard commercial off-the-shelf (COTS) CPUs.

3. Coolant Distribution Units (CDU) and Liquid Loops

You cannot pump raw facility chilled water directly through server components. Facility water is often dirty, heavily chemically treated, and operates at pressures too high for delicate IT equipment. You must separate the building loop from the server loop.

3.1 What a CDU does in an AI facility

A Coolant Distribution Unit (CDU) sits between the facility water (primary loop) and the IT equipment (secondary loop). It acts as the heart of the cooling system. Inside every CDU for AI data centers, you will find a liquid-to-liquid heat exchanger, redundant pumps, micro-filters, and smart control logic.

The CDU monitors the server temperatures and modulates its pump speed to provide exactly enough secondary coolant to keep the hardware stable. It also tightly controls the secondary fluid quality, usually using treated water mixed with propylene glycol (PG) and corrosion inhibitors, ensuring the cold plates never clog or corrode.

3.2 Sizing a CDU basics

When specifying a CDU, you must calculate both the required heat rejection capacity (kW) and the required fluid flow rate (Liters Per Minute, LPM). CDUs generally come in two form factors:

1. In-Rack CDUs: Installed directly inside the server rack, usually occupying 2U to 4U of space. They handle 30kW to 80kW of heat.

· Use this when: You are retrofitting a few AI racks into a legacy air-cooled data center and want to keep the liquid loop contained to a single cabinet.

2. In-Row / End-of-Row CDUs: Large, cabinet-sized units that sit alongside the server racks. They can manage 300kW to 1MW+ of heat and distribute fluid to multiple racks via headers.

· Use this when: You are building a greenfield AI data center and deploying rows of high-density compute simultaneously.

4. Two-Phase and Immersion Cooling Overview

While single-phase water cooling (where the water stays liquid) dominates today's market, extreme heat fluxes are pushing the industry toward phase-change technologies. These leverage the latent heat of vaporization—the massive amount of energy absorbed when a fluid turns from liquid to gas.

4.1 Two-phase cold plates

In a two-phase cold plate, a specialized dielectric fluid is pumped over the chip. The heat of the processor boils the fluid. The resulting vapor travels to a condenser (either in the rack or in the CDU), rejects its heat to the facility water, turns back into a liquid, and returns to the server.

Because boiling absorbs tremendous energy without raising the fluid temperature, two-phase systems offer incredibly uniform temperature profiles across large silicon dies. If you are struggling with hot spots on massive GPU packages, you must evaluate two-phase and immersion cooling for AI servers.

· Use this when: Your chip TDP exceeds 1500W, or you need absolute temperature uniformity across a massive silicon wafer.

· Avoid this when: Your facility cannot handle specialized, highly engineered dielectric fluids (which are strictly regulated due to global warming potential).

4.2 Immersion cooling

Instead of routing fluid through cold plates, immersion cooling submerges the entire server chassis in a tank of engineered dielectric fluid.

· Single-phase immersion: The fluid heats up, rises, and is cycled through a heat exchanger.

· Two-phase immersion: The fluid boils directly off the bare components, and vapor condenses on water-cooled coils at the top of the tank.

· Use this when: You want to eliminate all fans entirely, optimize facility floor space, and cool every single component on the board (memory, VRMs, switches) without designing custom cold plates for each.

· Avoid this when: You require frequent physical access to swap out hard drives or DIMMs, as handling wet servers complicates maintenance.

5. System Architecture: From Chip to Rear-Door

A successful deployment requires visualizing the entire AI data center cooling architecture. A failure at any single connection point compromises the whole system. The architecture is broken into three distinct levels.

5.1 The server level

Inside the chassis, copper or aluminum cold plates bolt to the GPUs and CPUs. Secondary plates or specialized thermal pads may cover the voltage regulator modules (VRMs) and memory. These plates are plumbed together using flexible fluoropolymer tubing (FEP/PFA). The internal tubing routes to the back of the server chassis and terminates in quick-disconnect (QD) fittings.

· Specify this: Always use non-spill, flat-face quick disconnects. This allows technicians to hot-swap a failed server without draining the loop.

5.2 The rack level

At the back of the rack, a vertical manifold distributes fluid. The manifold features matching quick-disconnects for every server slot. Cool coolant flows up the supply side of the manifold, enters the servers, absorbs heat, exits into the return side of the manifold, and heads to the CDU.

· Specify this: Use stainless steel manifolds with welded connections. Avoid threaded joints where possible, as temperature cycling can cause threads to back out and leak over time.

5.3 The row and facility level

The rack manifolds connect to the CDU via under-floor or overhead piping. The CDU then interfaces with the primary facility chilled water system. If the facility water is warm (e.g., 30°C / 86°F or higher), you are utilizing "warm-water cooling." This is highly desirable because it allows the data center to reject heat to the outside atmosphere using dry coolers, completely bypassing power-hungry chillers.

6. How to Size and Calculate a Liquid Loop

Specifying the correct pump and tube diameters requires basic thermodynamic math. Do not guess; calculate your exact requirements.

6.1 Heat load and flow guidance

To determine how much liquid flow you need, you use the standard heat transfer equation:

Q = ṁ × Cp × ΔT

Where:

· Q: Total heat load (Watts or kW)

· ṁ: Mass flow rate (converted to Liters Per Minute)

· Cp: Specific heat capacity of your fluid (Water is approx 4.18 kJ/kg·K; PG mixes are lower).

· ΔT: Allowed temperature rise of the fluid from server inlet to server outlet.

Rule of thumb for single-phase water: You need approximately 1 LPM of flow for every 1kW of heat if you are targeting a 15°C temperature rise across the cold plate.

Therefore, an AI server with 10kW of total hardware heat requires roughly 10 LPM of flow. A rack with 8 of these servers requires an 80 LPM manifold and CDU flow rate, plus a 20% safety margin for pump redundancy.

6.2 The engineering decision checklist

Before contacting a manufacturer, complete this ordered checklist:

1. Map the TDP: Calculate the maximum wattage of CPUs, GPUs, and high-power memory.

2. Define the fluid: Decide between pure water, PG-mix, or dielectric fluid. (This dictates whether you need copper, aluminum, or stainless steel cold plates to prevent galvanic corrosion).

3. Set the approach temperature: Define the facility inlet water temperature.

4. Calculate pressure drop: Ensure your target flow rate does not create a pressure drop exceeding your CDU pump capabilities.

5. Select the QD sizes: Match your flow requirements to standard quick-disconnect inner diameters (e.g., 1/4", 3/8", 1/2").

7. 2026 Power-Density Trends and Next-Gen Accelerators

If you design systems today, they must handle the silicon of tomorrow. Current architectures like Nvidia’s H100 run at roughly 700W TDP. However, AI accelerator power-density trends 2026 indicate that the GB200, GB300, and equivalent custom ASICs will push far beyond this.

By 2026, single package TDPs will exceed 1200W. Server racks will routinely demand 100kW to 120kW of power.

To survive this density:

· You must abandon aluminum skived plates for high-performance copper brazed or FSW cold plates.

· You must prepare for direct-to-chip micro-channel architectures.

· You must size your rack manifolds wider. 1-inch diameter manifolds will cause too much pressure drop for 100kW loads; plan to move to 1.5-inch or 2-inch internal diameters.

· You must evaluate facility loop pressures. Higher rack densities require higher fluid velocities, pushing system pressures up to 60-80 PSI.

Frequently Asked Questions

What is the best coolant for single-phase AI server cooling?

For most deployments, a mixture of 25% to 30% Propylene Glycol (PG) and pure deionized water, combined with industrial corrosion inhibitors, is standard. It provides excellent heat transfer while preventing organic growth and metal corrosion.

How do I prevent galvanic corrosion in my liquid loop?

Never mix incompatible metals in the same wetted loop. If your cold plates are copper, ensure your manifold and CDU heat exchangers use copper, brass, or stainless steel. Do not mix bare aluminum with copper in a water-based system, as the aluminum will rapidly corrode.

What is the lifespan of a friction-stir-welded cold plate?

Because FSW is a solid-state joint with no dissimilar filler metals, the weld is as strong as the base metal. Assuming proper fluid chemistry is maintained, an FSW cold plate will easily outlast the 5-to-7-year operational lifecycle of the IT equipment itself.

How does facility altitude affect liquid cooling?

Unlike air cooling, which loses massive efficiency at high altitudes due to thinner air density, liquid cooling remains almost entirely unaffected by altitude. This makes liquid cooling ideal for data centers located in high-elevation regions.

What is the maximum TDP a single-phase cold plate can handle?

With highly optimized copper micro-channels, direct-to-chip mounting, and high-velocity fluid flow, advanced single-phase cold plates can successfully cool silicon exceeding 1500W. Beyond 1500W-2000W, two-phase cooling becomes structurally and economically more viable.

How do I choose between vacuum-brazed and friction-stir-welded plates?

Choose vacuum brazing if you need complex, multi-layered internal fluid routing (like cooling a GPU, its surrounding memory modules, and VRMs simultaneously). Choose FSW if you have a simpler internal flow path but require the absolute maximum burst pressure and leak-proof reliability.

Do liquid-cooled servers still need fans?

Yes, in most architectures. While the cold plate removes 70% to 85% of the heat (from the CPU/GPU), the remaining ambient heat from motherboard components, storage drives, and PCIe switches still requires low-speed fans to pull air through the chassis. Only full immersion cooling eliminates fans entirely.