What Is Direct-to-Chip Liquid Cooling? – Why AI Racks Need It

A few years ago, a 15 kilowatt rack was the hot one in the room. Now operators get asked to host racks that pull 100 kW or more, and the people who built the air systems are quietly doing the math and wincing. Fans can only move so much air through a box before the noise, the power draw and the physics win. What is direct-to-chip liquid cooling, and why do AI racks need it?

Direct-to-chip liquid cooling pumps coolant through metal cold plates mounted right on top of processors, so liquid carries the heat away instead of air. AI racks need it because each accelerator draws hundreds of watts to over a kilowatt, packed eight or more to a server, which pushes rack loads past what air can remove sensibly. The warmed coolant flows through a rack manifold to a coolant distribution unit, which hands the heat to the building's water loop. Air still cools the rest of the server.

How Does Direct-to-Chip Liquid Cooling Work?

It works the way a car's radiator loop works, scaled down to a circuit board and multiplied by thousands. A copper or aluminum cold plate with tiny internal channels sits on the chip, with thermal paste between them. Coolant, usually treated water or a water and propylene glycol mix, flows through those channels and picks up heat.

From there, the path looks like this:

1. Cold plates on each GPU and CPU connect by small hoses to the server's inlet and outlet.
2. Drip-free quick disconnects link each server to a vertical manifold at the back of the rack, so a tray can be pulled for service.
3. The manifold feeds a coolant distribution unit (CDU), either inside the rack or on the floor serving a row.
4. The CDU's heat exchanger passes heat to the facility water system, which carries it to dry coolers, cooling towers or chillers outside.

ASHRAE calls the server-side loop the technology cooling system and the building side the facility water system. Keeping them separate matters. The server loop stays small, clean and tightly controlled, while the building loop can be big and less fussy. Vertiv's primer on direct-to-chip cooling in high-performance computing lays out this cold plate to CDU arrangement and its single-phase and two-phase variants.

Why Can't Air Cool AI Racks Anymore?

Air can, up to a point, but the point keeps moving out of reach. An NVIDIA H100 in its SXM form is rated at up to 700 watts, and newer accelerators run higher. Put eight in a server with CPUs, memory and networking, and one box can draw around 10 kW. NVIDIA's own DGX H100 system is rated around 10.2 kW at maximum. Four of those in a rack is already about 41 kW.

Now look at what air has to do. A handy rule says airflow in cubic feet per minute (CFM) equals heat in Btu/h divided by 1.08 times the temperature rise in °F. For a 100 kW rack with a 20°F rise across the servers:

• Heat: 100 kW × 3,412 = 341,200 Btu/h
• Airflow: 341,200 ÷ (1.08 × 20) ≈ 15,800 CFM through one rack

That is a small hurricane forced through a cabinet about two feet wide. The fans to do it burn real power, get loud and still struggle to keep each chip below its limit. Water does the same job with a trickle by comparison. Using the water rule of thumb, gallons per minute (gpm) equals Btu/h divided by 500 times the temperature rise: 341,200 ÷ (500 × 18) ≈ 38 gpm for a 10°C (18°F) rise. That flows through hoses you can hold in one hand.

The newest rack-scale systems are designed around liquid from the start. NVIDIA's GB200 NVL72 is a liquid-cooled rack with 72 Blackwell GPUs and 36 Grace CPUs working as one system, and it is not meant to run on air at all. If you want the bigger picture on how rack loads have climbed, our guide to rack power density walks through the numbers from typical enterprise racks to today's GPU cabinets.

Single-Phase vs Two-Phase Cold Plates

Most direct-to-chip deployments today are single-phase, meaning the coolant stays liquid the whole time and simply warms up. It is the simpler, more familiar design, and it uses water-based coolant that is cheap and carries heat well.

Two-phase cold plates use a refrigerant-like dielectric fluid that boils inside the plate. Boiling absorbs a lot of heat at a nearly constant temperature, which helps with extreme hot spots, and a leak of a dielectric fluid will not short out electronics the way water could. The trade-offs are higher fluid cost, more complex controls and a smaller pool of experience in the field.

What Changes in the Data Center Building?

Quite a lot, and this is where projects get surprised. Direct-to-chip liquid cooling does not just swap a part inside the server. It brings plumbing onto the data hall floor.

• Piping. Facility water has to reach every row, either overhead or under a raised floor, with valves and isolation points for maintenance.
• CDUs. They need floor space, power and redundancy, because a stopped pump means a hot rack within seconds to minutes.
• Water quality. The server loop needs fine filtration and treated coolant to keep tiny cold plate channels from fouling or corroding.
• Leak detection. Sensor cables under racks and in drip trays, tied into alarms and sometimes automatic shutoffs.
• Structure. Liquid-cooled racks full of dense gear can weigh well over a ton each, so floor loading needs a fresh look.

There is also a skills shift. Your team now handles fluid sampling, glycol mixes and fittings alongside breakers and fans. In my experience the plumbing is rarely the hard part; the hard part is writing procedures so nobody unhooks the wrong hose during a maintenance window.

Is Water in the Rack a Leak Risk?

Yes, but a manageable one, and the industry has gotten good at it. The server loop runs at low pressure, fittings are designed to seal when disconnected and some systems run the loop under slight negative pressure so a crack pulls air in rather than pushing coolant out. Leak detection then catches the rare drip before it reaches anything energized.

The bigger risk is people. Most leaks I would worry about come from a rushed maintenance job, a fitting that was not fully seated or a hose bent past its limit. Good procedures, training and a second set of eyes on every connection do more than any sensor.

Air Does Not Go Away

Cold plates usually cover the processors and sometimes memory, but power supplies, drives, network cards and voltage regulators still shed heat into the air. Depending on the server design, cold plates often capture somewhere around 70 to 80 percent of the heat, with the rest left for room air. So a liquid-cooled hall still needs air handlers, just smaller ones. Plan for both.

Does Direct-to-Chip Cooling Save Energy and Water?

It can save a lot of both, mostly because it works with warm water. Chips are happy with coolant temperatures that would feel like a hot bath, so the facility water can often run warm enough to reject heat through dry coolers for most or all of the year, without chillers and without evaporating water. ASHRAE's liquid-cooling classes, with names like W32 and W45, describe facility water supply up to those temperatures in °C.

Warmer loops mean less compressor energy, smaller or no cooling towers and server fans that spin slower. That shows up as a lower power usage effectiveness (PUE). Warm return water, sometimes above 120°F, is also more useful for heat reuse than lukewarm exhaust air.

The savings are not automatic. If a site runs its liquid loop cold out of caution, or ties it to evaporative towers in a hot climate, it will use more energy and water than it needs to. Design temperature is the lever that matters.

What About Behind-the-Meter and Off-Grid AI Sites?

For sites running on their own gas generation or other on-site power, liquid cooling has a direct payoff: lower cooling overhead means fewer megawatts to build and fuel. Say a campus has 50 MW of IT load. At a facility PUE of 1.4 it needs 70 MW of generation before redundancy. At 1.15 it needs 57.5 MW. That 12.5 MW gap is a couple of big reciprocating engines you do not have to buy, run or maintain.

Water-scarce sites benefit too. In West Texas or similar country, a closed liquid loop on dry coolers can keep on-site water use close to zero, which matters both for permits and for neighbors.

There is a reliability catch. Liquid-cooled racks have very little thermal mass, so if pumps stop, temperatures climb fast. CDUs and facility pumps belong on uninterruptible power alongside the servers, with redundant pumps and a clear plan for what happens during a generator transfer.

Direct-to-Chip Liquid Cooling Is Becoming the Default for Dense Racks

The practical lesson is about timing. Liquid costs far less to draw on a blank sheet than to bolt onto a hall built for air, so the decision belongs at the start of design, not after the first GPU order.

For a facility meant to host GPU workloads, commit to liquid in the first design package. Size the CDUs and piping for the densest racks you expect, pick the warmest facility water temperature your hardware allows and put the pumps on protected power. Then write the maintenance procedures before the first rack arrives, because the hose someone unhooks by mistake is a bigger risk than the hardware.

← All guides