Meta description: How to design 100kW+ AI compute racks — power distribution, liquid cooling, InfiniBand/Ethernet fabric density, and TCO — with Zhongxin Data hardware.
By 2026, a single AI training rack is no longer a 10kW air-cooled box of 1U servers. NVIDIA HGX H200 pods routinely pull 80–100 kW, and the new GB200 NVL72 racks are specified to operate at 120–130 kW per rack, with hyperscale designs targeting up to 250 kW per rack using direct-to-chip liquid cooling and immersion-ready chassis. Air cooling alone cannot remove that heat at acceptable density. This guide walks through the engineering decisions you must make when planning a 100kW+ AI pod: busbar vs. whip distribution, coolant selection, manifold layouts, network fabric sizing, and how to choose the InfiniBand or Ethernet switches, adapters, and storage controllers that Zhongxin Data stocks for these workloads.
“Cooling is now a primary design constraint for AI infrastructure, not an afterthought. The rack, the room, and the grid must be co-designed.”
Why 100kW+ Racks Are the New Normal
Three forces are driving rack power density to unprecedented levels in 2026:
- Accelerator density: HGX H200 8-GPU baseboards draw ~10 kW each; a 42U rack with four baseboards plus switches easily exceeds 60 kW before networking.
- Blackwell-class systems: GB200 NVL72 compute trays integrate Grace CPUs with B200 GPUs in a single sled, pushing sled power past 5 kW each.
- GPU-to-GPU bandwidth: NVLink Switch fabric inside the rack multiplies interconnect power, while external ConnectX-8 / BlueField-3 SuperNICs add another 75–150 W per slot.
The result is that traditional 208V single-phase distribution and raised-floor airflow no longer suffice. New AI factories are being built around 415V three-phase, 60kW+ per rack busbars, rear-door heat exchangers, and direct liquid cooling (DLC) loops with coolant distribution units (CDUs).
Rack Power Distribution Architecture
The table below compares the three dominant approaches for delivering 60–250 kW to a single rack. Use it as a starting checklist when sizing your busway, transformer, and UPS.
| Distribution method | Voltage / phase | Typical rack capacity | Best fit | Limitations |
|---|---|---|---|---|
| Traditional PDU + whip | 208V / 240V single-phase | Up to ~22 kW | Legacy air-cooled compute | Wire gauge, breaker count, and phase imbalance cap density |
| Three-phase PDU (415V) | 415V three-phase | 30–60 kW | Modern GPU servers, mixed CPU/GPU racks | Still air-cooling-bound for heat |
| Rack busbar + remote power panel | 415/480V three-phase | 80–250 kW | AI pods, immersion/DLC, hyperscale | Higher upfront cost, requires structured above-rack busway |
For an HGX H200 pod, expect four baseboards at ~10 kW each plus ~6 kW for switches and storage — that’s already 46 kW before overhead. For a GB200 NVL72 compute rack with nine compute trays, plan for 120+ kW of IT load alone, leaving little headroom for a 50/50 air/liquid split. Busbar distribution with a remote power panel (RPP) is the practical default for any 100kW+ AI rack deployed in 2026.
Liquid Cooling: Direct-to-Chip vs. Immersion vs. Rear-Door
Liquid cooling is no longer optional at 100 kW. The table below summarizes the three architectures and where each makes sense.
| Cooling architecture | Heat removal capacity per rack | Coolant | Serviceability | Best fit |
|---|---|---|---|---|
| Rear-door heat exchanger (RDHx) | 20–40 kW per rack | Facility water | High — no changes to servers | Brownfield retrofits, mixed CPU/GPU fleets |
| Direct-to-chip (DLC) cold plates | 60–150 kW per rack | PG25 / PG40 glycol-water | Medium — sealed loops per sled | New AI pods, HGX/GB200 baseboards |
| Single-phase immersion | 100–250 kW per rack | Engineered dielectric fluid (3M Novec, Shell S5) | Low — fluid handling, material compatibility | Dedicated greenfield AI factories |
| Two-phase immersion | 200+ kW per rack | Low-boiling dielectric | Very low — specialized staff | Rare; research and a handful of hyperscalers |
Direct-to-chip with a closed-loop coolant distribution unit (CDU) is the dominant 2026 choice. It removes the bulk of the heat (CPUs, GPUs, often NIC retimers) via cold plates, while a smaller rear-door loop or air-handling unit (AHU) takes care of the rest. Plan the CDU at ~1.3× the steady-state IT load to leave headroom for transients and pump redundancy (N+1 pumps).
Coolant Selection and Water Quality
The single most common cause of premature cold-plate failure is poor water quality. Glycol-water mixtures (PG25 / PG40) buy you freeze protection and corrosion margin but reduce heat-transfer efficiency. Keep these targets in mind:
- Resistivity: > 1 MΩ·cm at 25 °C (deionized, monitored)
- pH: 8.5–9.5 for glycol-water loops; 7.0–9.0 for deionized water
- Particle count: < 100 particles/mL larger than 50 µm
- Microbial control:定期 conductivity sweeps + occasional biocide flush for PG loops
Inline filtration, deionizing cartridges, and a bypass conductivity probe are non-negotiable for 24/7 AI workloads. Plan a quarterly water analysis with your chemistry vendor.
Manifold, Pump, and CDU Sizing
The CDU is the heart of the liquid loop. For a 100 kW pod, a single 130 kW CDU with N+1 pumps is the typical baseline; for 250 kW+ designs, two CDUs in a lead/lag configuration with an auto-failover manifold. Use the table below to size secondary flow rates.
| IT load | Loop temperature target | PG25 flow rate (L/min) | Pipe diameter (primary) | Recommended pump head |
|---|---|---|---|---|
| 50 kW | Supply 35 °C / return 45 °C | ~72 | DN40 (1.5″) | 20 m |
| 100 kW | Supply 35 °C / return 45 °C | ~144 | DN50 (2″) | 25 m |
| 150 kW | Supply 32 °C / return 42 °C | ~215 | DN65 (2.5″) | 30 m |
| 250 kW | Supply 30 °C / return 40 °C | ~358 | DN80 (3″) | 35 m |
Lowering the loop temperature by even 3 °C increases cooling capacity at the cost of higher pumping power and chiller load. In warm climates, oversize the chiller plant to at least 1.4× peak IT load for free-cooling days.
Network Fabric for 100kW+ Pods
A high-density AI pod needs a network that can sustain all-to-all GPU gradients without starving the accelerators. That means NDR 400 Gb/s or NDR 800 Gb/s InfiniBand, or Spectrum-X 400/800 GbE, with at least one 400 Gb/s link per GPU sled. The table maps fabric choices to pod size and explains what to stock.
| Pod size (GPUs) | Recommended fabric | Per-GPU uplink | Spine switches | Zhongxin Data reference |
|---|---|---|---|---|
| 8–32 GPUs (single HGX) | NDR 400 InfiniBand or 400GbE | 1 × 400 Gb/s | Quantum-2 QM9700 | ConnectX-7 / ConnectX-8 Adapters |
| 64–256 GPUs (multi-HGX) | NDR 800 InfiniBand or Spectrum-X 800 | 1–2 × 800 Gb/s | Quantum-3 QM9700 / SN5600 | Spectrum SN5600 / SN5400 |
| 512+ GPUs (NVL72-scale) | NDR 800 InfiniBand + NVLink Switch | 2 × 800 Gb/s + NVLink | Quantum-3 + NVLink tray | NVIDIA Quantum-3 Switches |
For BlueField-3 SuperNICs or ConnectX-8 adapters, remember that each card adds 75–150 W. If your pod has 16 compute trays, that is an additional 1.2–2.4 kW per rack just for the NICs — include it in your cooling budget, not in the network-only BTU calculation.
Storage and RAID Considerations
AI training reads checkpoints and datasets at multi-GB/s rates. A MegaRAID 9560-16i or HBA 9500-16i at PCIe Gen4 x8 is the workhorse for NVMe hot-tier storage. For racks with 100+ kW of GPU compute, mirror hot-checkpoint NVMe drives (e.g., Kioxia CM7 or Samsung PM1743) at PCIe Gen5 where available. Use Broadcom MegaRAID adapters for RAID 6/10 protection on capacity-tier SAS or SATA SSD shelves.
Operations: Monitoring, Telemetry, and Failure Modes
Once a 100 kW pod is live, your operations team will live and die by three telemetry streams: inlet water temperature and pressure at each sled, coolant flow rate per cold plate, and rack-level power draw. The minimum viable monitoring stack is:
- Per-rack BMS gateway (Modbus TCP or Redfish) reporting coolant inlet/outlet temperatures, flow, and pressure.
- Per-PDU smart meter with 1-second sampling for spike detection.
- DCIM integration correlating thermal, power, and asset data — a 100kW+ rack that silently degrades from 32 °C inlet to 38 °C inlet is a fire risk, not just a performance issue.
- Coolant chemistry logging with weekly auto-export to your chemistry vendor.
Common failure modes to design around: pump cavitation after a power blip, biofilm accumulation in stagnant PG loops, scale build-up at heat-exchanger plates, and manifold O-ring dry-out. Each of these is preventable with a quarterly maintenance window — but only if your monitoring stack catches the early warning signs.
FAQ: Power and Cooling for 100kW+ AI Pods
Can I retrofit an air-cooled data center to 100kW+ racks?
Yes, but only with limits. Hot-aisle containment plus rear-door heat exchangers (RDHx) can reach ~30–40 kW per rack. Beyond that, you must add direct-to-chip liquid cooling, which requires floor reinforcement, coolant piping, and CDUs. Most operators building 100kW+ AI pods build them in new whitespace rather than retrofitting older air-cooled halls.
What is the realistic PUE for a liquid-cooled AI factory?
Well-designed DLC + chilled-water plants hit 1.05–1.15 PUE in temperate climates, with hyperscale designs reporting 1.06–1.10. Air-cooled AI pods typically land at 1.25–1.40. Free cooling and adiabatic chillers reduce it further in cold regions.
How much water does a 100kW+ pod actually use?
With DLC + evaporative chillers, a 1 MW AI pod uses roughly 1,000–2,000 L/day for the cooling tower make-up plus the closed-loop inventory (typically 800–1,200 L per CDU). Liquid-to-liquid CDUs that reject heat to a district loop can drop the make-up substantially.
What redundancy do I need for a single AI pod?
For training jobs running 24/7, plan N+1 power (PDU + UPS + transformer), N+1 cooling (pumps + chillers), and 2N networking at the leaf layer. Loss of a single pod is acceptable; loss of a training job that costs $5M+ in GPU-hours is not.
Are used parts usable in a 100kW+ pod?
Yes — used ConnectX adapters, Quantum switches, and MegaRAID controllers from reputable vendors like Zhongxin Data are routinely deployed in production AI clusters. The key is burn-in + firmware validation: every used NIC and switch should be reflashed and soak-tested for 24–72 hours before deployment. Zhongxin Data’s preparation workflow includes firmware re-flash and 24-hour burn-in for every adapter.
Site Planning: Whitespace, Slab Load, and Containment
Before a single rack arrives, three civil decisions dictate whether 100kW+ density is feasible:
- Raised floor vs. slab: New AI factories are almost universally built on slab with overhead busway. Raised floors cap at ~1,500 kg/m² — a 250 kW DLC rack with manifolds can weigh 1,800+ kg, so reinforced slab (3,000–5,000 kg/m² load rating) is the safe default.
- Aisle containment: Cold-aisle containment (CAC) with blanking panels recovers 15–25% cooling efficiency versus open hot-aisle return. For DLC pods, hot-aisle containment (HAC) plus a sealed rear-door heat exchanger is often the cleaner thermal boundary.
- Ceiling height: Allow 3.6 m clear ceiling height above the finished slab to accommodate busbar drops, overhead cable trays, and rear-door heat-exchanger piping.
White-space planners should reserve a dedicated chiller plant yard sized for 1.3× peak IT load, and a CDU mechanical room adjacent to (but separate from) the white space for service access.
Total Cost of Ownership: Air vs. Liquid at 100 kW
A simplified TCO comparison for a 1 MW AI cluster running for 5 years. Assumes $0.10/kWh blended electricity and a 12-year facility amortization.
| Design | Rack density | CapEx per MW (IT) | PUE | 5-year energy | 5-year TCO per MW |
|---|---|---|---|---|---|
| Air-cooled, 30 kW/rack | 30 kW | ~$3.5M | 1.35 | ~$23.7M | ~$27.2M |
| Hybrid DLC, 80 kW/rack | 80 kW | ~$4.2M | 1.15 | ~$20.2M | ~$24.4M |
| Full DLC + busbar, 120 kW/rack | 120 kW | ~$4.8M | 1.08 | ~$19.0M | ~$23.8M |
The higher-density designs recover their CapEx delta within 18–24 months purely on energy efficiency, and they leave far more headroom for the next GPU generation. Density is no longer a luxury — it is the cheapest path.
Build Your 100kW+ Pod with Zhongxin Data
Zhongxin Data supplies the networking, storage, and server-grade building blocks you need for AI pods — from Mellanox ConnectX-7 / ConnectX-8 NDR adapters and Quantum-3 / Spectrum-X switches, to Broadcom MegaRAID 9560 and HBA 9500 controllers, plus a full lineup of Dell PowerEdge, HPE ProLiant, and enterprise SSDs. Every used adapter is firmware-reflashed and soak-tested before shipping, so you can integrate them into your 100kW+ pod with confidence.
Need help sizing your busbar, CDU, or fabric for a new AI deployment? Contact our engineering team on WhatsApp +86 13068616914 or visit zhongxindata.hk for live inventory and custom configurations.