Quick answer: Direct Liquid Cooling (DLC) — cold plates on CPUs, GPUs, and increasingly on the NIC — reduces junction temperatures by 25–40 °C versus traditional hot-aisle air cooling. That thermal headroom lets you push ConnectX-7 / ConnectX-8 cards and Quantum-2 / Spectrum-4 switches to their full 400G/800G envelope with much lower failure rates. But the optics and the breakout DAC cables are still the weak link: anything that ends up outside the cold-plate loop must survive a hotter microenvironment, and immersion-cooled tanks need sealed, dielectric-rated optical modules and AOCs.
Why liquid cooling is no longer optional in modern AI training clusters
When NVIDIA published the reference design for the DGX H100 / H200 / B200 racks, the spec sheet quietly moved the heat budget from the CPU socket to the entire rack bus. A fully populated 8-way H200 rack now dumps 80–120 kW into ~60 cm of rack depth. Traditional Computer Room Air Handlers (CRAHs) cannot push that heat out of the hot aisle fast enough — the delta-T across the cold aisle collapses and the GPUs throttle. That is why hyperscalers (Meta, Microsoft, Oracle, ByteDance) all moved to direct-to-chip or full-immersion cooling for new AI buildouts in 2024–2026.
For datacenter operators building or refreshing a 1–10 MW AI training pod, the question has shifted from “should we adopt liquid cooling?” to “what does liquid cooling do to the network stack we just bought from Zhongxin Data?”
Three thermal zones inside one rack
| Zone | Typical ΔT vs ambient | Components | Cooling strategy |
|---|---|---|---|
| Zone A — Cold plate | +5 to +15 °C above coolant inlet | CPU, GPU, HBM stacks, increasingly the NIC SoC | Direct Liquid Cooling (DLC) cold plate |
| Zone B — Warm air pocket | +20 to +35 °C above cold-aisle air | DIMM DDR5 modules, VRMs, internal M.2 boot drives | High-velocity air / rear-door heat exchanger |
| Zone C — Patch panel & cabling | +15 to +25 °C above cold-aisle air | AOCs, DAC/ACC breakouts, QSFP-DD/OSFP faceplates, patch-panel transceivers | Sealed LC patches / routed coolant on the spine switch |
The networking gear never sits in Zone A in current-generation designs — even when the GPU cold plate gets the benefit of DLC, the NIC and the faceplate pluggable optics stay in Zone B/C. That asymmetry is the source of most optical-link failures we see in customer support tickets.
What changes for the NIC itself
ConnectX-7 / ConnectX-8 in DLC racks
Mellanox ConnectX-7 NDR (400G) and ConnectX-8 XDR (800G) cards are spec’d for a Tj of 105 °C and an ambient operating window of 0–55 °C. In a hot-aisle air-cooled rack, the NIC ASIC typically runs at 90–100 °C under sustained 400G load with traffic pushing P99. After you bolt a cold plate onto the card (which NVIDIA, Dell, and HPE all offer as a factory SKU), the junction drops to 60–70 °C. The win isn’t just lower temps — it is the headroom to:
- Hold sustained 400G line rate on RoCE without thermal throttling
- Run deeper SERDES pre-emphasis for longer-reach DAC and ACC breakouts
- Drop the ASIC failure rate meaningfully (every 10 °C roughly doubles silicon lifetime)
What about the optics?
Pluggable optics — the QSFP-DD800 and OSFP faceplates — are not cold-plated. They live in the patch panel, and they pull up to ~14 W each at 800G. In a fully populated rack that is 8–16 kW of optical heat *per rack* that has to be moved by air cooling alone. The realistic operating window for a sealed-top OSFP in this position is +5 to +70 °C ambient, with a long-term drift at the hot end.
| Ambient temp (faceplate inlet) | Predicted BER @ 800G PAM4 | Notes |
|---|---|---|
| 25 °C (cold aisle) | 1e-9 (clean) | Spec sheet baseline |
| 35 °C | 1e-8 | Marginal — bit-error-correction working hard |
| 45 °C (typical hot-aisle pocket) | 1e-7 to 1e-6 | FEC correcting 2–3 dB of margin loss |
| 55 °C + sustained load | FEC uncorrectable events begin | Replace with sealed OSFP, or relocate patch panel |
The cabling fork: DAC, ACC, AOCs, and optical line systems
As the speed per lane climbed from 25G (NRZ, 100G) to 50G (PAM4, 200/400G) to 100G (PAM4, 800G) and now to 200G-per-lane for 1.6T, the copper reach shrank dramatically. What used to be a 5 m passive DAC at 100G is now 1.5 m at 800G before the eye diagram collapses. That changes the cabling topology of a DLC rack in three ways:
- Inside the rack: — short ACC (Active Copper Cable) or near-active AECs for ≤ 2 m GPU↔leaf-switch runs.
- Across the row: — AOCs (Active Optical Cable) with sealed boots, because the coolant manifold cannot tolerate hot spots from passive copper at 800G.
- Between rows: — full line-system optical transceivers + single-mode patch, often routed through overhead trays.
| Cable type | Reach at 400G (PAM4 50G/lane) | Reach at 800G (PAM4 100G/lane) | Liquid-cooling compatibility |
|---|---|---|---|
| DAC (passive copper) | ≤ 3 m | ≤ 1.5 m | OK in Zone C (above cold plate) |
| ACC (linear-drive copper) | ≤ 5 m | ≤ 3 m | OK; check boot temp rating |
| AOC (active optical) | ≤ 30 m | ≤ 30 m | Sealed-rated required above coolant loop |
| SR4 / SR8 (parallel MMF) | ≤ 100 m (OM4) | ≤ 50 m (OM4, 800G) | Faceplate-only; never immersed |
| DR4 / DR8 (single-mode) | ≤ 500 m (WDM mux) | ≤ 500 m | Standard patch panel; OK |
| FR4 / 2×FR4 | ≤ 2 km | ≤ 2 km | Standard patch panel; OK |
The practical guidance from the field: at 800G, AOC starts at 3 m and replaces DAC almost everywhere outside of one-server-to-adjacent-switch topologies. We see DAC purchases drop by ~60% when a customer moves from air-cooled to DLC racks at the same speed per port.
Three procurement pitfalls to avoid on a first DLC deployment
We have walked through enough first-time DLC rollouts to flag the three mistakes that always come back to bite the team that spec’d the rack.
Pitfall #1 — buying optics off the same BOM as the air-cooled version. When the rack architecture changes, the optics SKU often needs to change with it. A standard QSFP-DD800 module rated for 0–70 °C is technically legal in a DLC rack, but the cooler racks live at 25–30 °C, which is well inside its spec, whereas pockets behind the patch panel can hit 55 °C on a hot day. Ask your supplier (us, in this case) for the “sealed-top” variant; it costs ~$30 more per module and pays back in the first summer.
Pitfall #2 — underprovisioning the CDU bypass loop. A 1 MW DLC pod needs 1.5 MW cooling capacity in steady state, but it also needs a fast bypass to absorb power spikes from GPUs going into and out of all-reduce rings. A CDU rated for exactly 1 MW will let coolant temperature spike 6–8 °C during a 90-second all-reduce event, which translates to a 0.5 °C drift on the NIC junction — enough to soft-throttle the ConnectX. We always recommend 1.5× peak sizing, which leaves thermal margin for one (1) failed pump in the secondary loop.
Pitfall #3 — ignoring dew-point and condensation during downtime. When you take a DLC rack offline for service, the cold plates stay cold. If you open the rack to swap a NIC in humid ambient air, condensation will form on the cold plate and on the NIC ASIC and on the optical faceplate. Worse, the moisture that wicks into the optical connector is invisible until the laser fails three weeks later. The fix is procedural: bring rack inlet air to < 30% humidity before opening, and never open a cold plate within 60 minutes of a power-down.
Software-side tuning that pays off on DLC
Hardware is only half the story. Three configuration changes on the NIC bring out the most of a DLC deployment:
- Set the thermal policy to “maximum performance” instead of “balanced.” The throttle ceiling jumps from 95 °C to 105 °C; you bought the headroom, use it.
- Disable the IR3 (interrupt moderation 3) preset when running all-reduce heavy workloads. DLC lowers the interrupt latency floor; you can move moderation from 20 µs to 5 µs without paying a CPU cost.
- Enable PFC + ECN on every lossless queue, even on single-tenant AI fabrics. Without it, a single congestion event drops a flow that needs 5+ retries to recover, and those retries propagate head-of-line blocking across the whole all-reduce ring.
These three settings, all in mlxconfig or the UFM/Spectrum-X manager, are the cheapest 5–8% per-iteration improvement you will find in 2026 — and they only became safe once DLC dropped the NIC junction out of the throttle band.
Migration playbook: 18-month timeline we recommend
For a customer running an existing 5 MW air-cooled AI cluster who wants to migrate to DLC without rebuilding from scratch, our typical playbook is:
- Months 0–3: Pilot on one rack. Replace one air-cooled compute rack with a DLC rack. Confirm the in-rack thermal map with rack-mounted sensors. Run shadow training traffic for 30 days before declaring success.
- Months 3–6: Roll out the leaf/spine network into the existing cold-aisle containment. You can mix DLC compute with air-cooled switches for the pilot — that’s the design we recommend.
- Months 6–12: Standardize the BOM across the fleet. Lock in the cold plate SKU per OEM server. Lock in the AOC cable BOM. Lock in the immersion-rated OSFP SKU for any immersion tiers.
- Months 12–18: Decommission the legacy CRAH units, switch the secondary loop to free-cooling, hit the < 1.2 PUE target.
This is roughly the path our customers at zhongxindata.hk have walked twice in the last 24 months. The articles above (in particular our Technical Guides series) document the failure modes and the recovery patterns in more depth.
Immersion vs cold-plate — different rules for optics
Single-phase immersion with dielectric fluorinert or mineral oil is enjoying a comeback for very-high-density AI training. Immersion gets the entire NIC, including the heat sinks, into the dielectric. It also gets the faceplate. Pluggable optics that are not immersion-rated will fail within weeks: the optical lubricant outgases, the laser wavelength drifts, and the housing plastic absorbs the fluid.
The fix is straightforward but procurement-specific: ask for immersion-rated OSFP/QSFP-DD modules. Mellanox/NVIDIA sells them with a sealed top (closed-cell foam gasket) and an inert laser package; the SKU is different. The bad news is that immersion-rated 800G optics currently carry ~25% price premium and longer lead times because the volumes are still niche. Many operators avoid the issue by using direct-to-chip DLC for the server side and air cooling for the spine switch — this is the dominant 2025/2026 design we ship to customers.
Switch-side thermal: Quantum-2 and Spectrum-4 in DLC pods
The leaf and spine switches — NVIDIA Quantum-2 QM9700 (64-port 400G NDR) and Spectrum-4 SN5600 (64-port 800G XDR) — pull ~1.5–2.2 kW per switch under full load with FEC on. Cold-plating an entire switch chassis is non-trivial (different thermal interface than a CPU), so the standard approach is to keep the switch on cold-aisle air but in a sealed cold-aisle containment with a higher ΔT allowance. The next-generation SN6000 series (2026) brings a factory DLC option — direct manifold into the switch mid-plane — which addresses the heat density issue head-on.
For operators sizing a new AI pod in 2026, the rough budget per 1 MW of training compute is:
- ~16 racks of DLC compute (each ~60–65 kW)
- ~3 racks of network gear (spine + leaf), partly DLC on the SN6000
- Coolant distribution unit (CDU) sized at 1.5× peak heat (for transient spikes)
FAQ — liquid cooling & networking hardware
Do I need to re-spec every NIC and switch when migrating to liquid cooling?
No, but you do need to confirm two things: (1) the cold plate is the right cold-block geometry for your SKU (Dell, HPE, Lenovo, Supermicro each have their own heatsink offset), and (2) the optics at the patch panel are within their ambient spec. Most Mellanox / Broadcom / Intel NICs ship with a DLC cold plate option from the OEM. Browse our Mellanox ConnectX catalog.
Are DAC cables still usable inside a DLC rack?
Yes, for short GPU-to-leaf runs up to ~1.5 m at 800G (3 m at 400G). Above that, switch to ACC or AOC. We do not recommend passive DAC at 800G in any topology that exceeds a single rack.
How does liquid cooling affect Mean Time Between Failures for optics?
When the optics move from a 55 °C hot-aisle pocket down to a 25 °C cold-aisle pocket (because the surrounding rack is DLC-cooled), the laser package failure rate roughly halves for every 10 °C drop, following the Arrhenius model. In our 2024–2025 customer fleet, MTBF on QSFP-DD800 optics improved from ~9 years to ~14 years after a DLC retrofit.
Should I use immersion or cold-plate?
For new AI training pods, direct-to-chip cold-plate is the default in 2026 — it has a clear O&M model, an existing CDU supply chain, and it leaves the optics in air (cheaper and faster to procure). Immersion makes sense only when the desired density exceeds ~80 kW per rack, which is rare outside of the very largest hyperscalers.
What should I buy from Zhongxin Data to start a DLC retrofit?
We recommend starting with the NVIDIA ConnectX-7 NDR 400G NIC with factory DLC plate, the Quantum-2 QM9700 leaf switch for the pod, and AOC breakout cables (5 m, 30 m, 100 m) for the cross-rack runs. Our team can pre-size the cold-block manifold to match the OEM server you already have.
Closing note
Liquid cooling is now an assumption for any 2026 AI cluster buildout, not an option. The networking stack you put inside that rack has to be selected with the three thermal zones in mind — and the optics, surprisingly, end up being the deciding factor for whether the rack actually runs reliably. We’ve shipped more than 200 DLC-ready AI pods in the last 18 months; if you want help sizing your topology, talk to our solutions team.