A joint effort by immersion cooling company Iceotope and Facebook parent company Meta found that cooling hard drives with dielectric liquid is safe and a more efficient way to cool down than using fans.
Hyperscalers like Meta deploy thousands of HDDs in their data centers, and while the amount of heat emitted in a single case is small, it adds up, especially since the drives are in constant use and close together. The drives are stored in server racks that can only hold a few dozen hard drives, called JBODs (just a bunch of disks).
The JBOD can overheat without cooling, so far the fans have done the trick, but some drives are farther from the fans than others, causing uneven cooling.
One thing that helped the experiment were changes in disk drive manufacturing. Newer high-capacity drives are filled with helium to reduce vibration during rotation, and to retain the helium, the drives are sealed. The same thing that prevents helium leaks will also prevent liquid leaks.
The test used a standard air-cooled high-density JBOD storage system that houses 72 3.5-inch hard drives in a 40U rack, along with two single-socket nodes, two SAS expansion cards, a NIC, and a power distribution board , and the transformation of single-phase immersion cooling was carried out.
Iceotope doesn’t submerge gears in a tank full of coolant. Instead, it works at the chassis level, delivering small amounts of dielectric fluid precisely to hot spots before cascading across the chassis to remove heat from other components. Think of it as partial immersion using a fraction of the liquid required for tank immersion.
Iceotope uses fluorinated dielectric fluids with very high boiling points, but since conducting this research it has successfully experimented with synthetic hydrocarbon dielectric fluids. Iceotope says its technology is liquid-agnostic and the company uses a range of dielectric fluid types and suppliers.
Meta found that regardless of the position of the 72 drives within the rack, the liquid-cooled system had a 5.4°F (3°C) temperature difference, more uniform results than air cooling.
Iceotope says that at low loads, the power consumption of air cooling and liquid cooling is very similar, but as the load increases, immersion consumes less power than air cooling.
Meta is evaluating the benefits of Iceotope’s immersion cooling in its IT infrastructure. According to Iceotope, server racks can be retrofitted to support liquid cooling with minimal cost and effort, often a negligible percentage of the server’s cost, but liquid cooling still requires sealed drives.