Artificial intelligence is increasingly a part of our everyday lives, and tech companies are building more data centers to accommodate the spike in computer processing.
Meeting the demands from thousands of new machines plugged into the power grid can be tricky, so promoting energy efficiency has become more important than ever.
Through his research at Binghamton University, Associate Professor Pritam Das has developed technology that could help. Now, thanks to a program supported by the National Science Foundation, he’s hoping to get it out to the marketplace.
Das — a faculty member at the Thomas J. Watson College of Engineering and Applied Science’s Department of Electrical and Computer Engineering — has one patent issued and another one pending. He recently received $100,000 from the University’s Excellence in Entrepreneurship and Discovery (EXCEED) program, supported by an NSF Accelerating Research Translation (ART) grant. He plans to use these funds for prototyping, data collection, and the evaluation of the Binghamton University inventions, to encourage a startup company to build a commercial model for industry partners.
As AI data centers proliferate and scale, and AI hyperscalers like Amazon Web Services, Microsoft Azure, and Google Cloud push for increased performance, challenges have emerged in powering critical AI graphics processing units (GPUs). Moore’s law — which for many years promised that the number of transistors on a microchip would double every two years — has reached its limit. Also, physical CMOS (complementary metal-oxide-semiconductor) transistors — which are the building blocks for modern electronics — measuring smaller than 4 to 7 nanometers are not currently viable.
As a result, GPU and AI processor designers aim for higher computing ability for all usable area, but managing heat and power becomes paramount. Modern chips are said to be power-limited, not area-limited, and designers are aggressively working to push transistor power to 1 volt and even as low as 0.5 volt.
“Because we cannot make the transistor any smaller, we need to reduce the voltage that these chips handle or operate from,” Das said. “Traditionally, it has been 5 volts or 3.3 volts, and now chipmakers are looking below 1 volt. As the voltage is reduced, it allows electronics packaging in a more dense way.”
However, Ohm’s law — the fundamental principle outlined by physicist Georg Ohm in the early 19th century — dictates that lowering the voltage requires more current for the same amount of computing power, and AI’s ever-increasing computing demands only make those equations more complex.
There are plans to increase the voltage on the DC distribution bus from 12 to 48 volts, which leads to 16 times less ohmic power loss and significantly reduces a data center’s thermal stress and energy consumption.
These two changes require a fundamental technology shift for point-of-load converters (POLc), which are the small DC-DC converters that sit closest to the ultra-low voltage GPUs and power them from the 48-volt DC current.
Traditional point-of-load converters for GPUs rely on multiple stages of DC-DC power conversion, reducing their efficiencies to 80%. That means for every 100 watts delivered, about 20 watts is lost as heat.
“These point-of-load converters are placed very close together, and those processors produce a lot of heat while they are computing,” he said. “You don’t want to add to that thermal problem with less efficiency from the power conversion, because wherever efficiency drops, it also becomes heat, and that heat needs to be managed properly by a thermal cooling system.”
Das’ solution is a new kind of point-of-load converter to better step down the power in a single stage. A laboratory prototype achieved 10-12% higher efficiency for all load conditions and doubled the slew rate, which measures how fast the power conversion happens.
“You need to deliver that current at a very fast pace, just like our brain needs a lot of brain food when it is working,” he said. “AI needs that food very quickly as it is computing.”
Das and PhD student Tuhin Sasmal’s current patent covers this single-stage power conversion from 48 volts to 1 volt, and the pending patent would allow manufacturers to package the converter as close as 5 millimeters to the microchip — about the thickness of a bar of chocolate.
NSF’s ART grant supports programs like EXCEED to help institutions of higher education in building capacity, infrastructure, and training to accelerate research translation, strengthen technology transfer, and create sustained economic and societal impacts across the U.S.
“It is incredible to be able to support technologies like Professor Das’ POLc patents through EXCEED, so these projects can more quickly address real-world problems,” said Kathryn Cherny, senior program manager for the University’s Office of Entrepreneurship and Innovation Partnerships. “These problems are not theoretical, and neither are the impacts of Binghamton innovation.”
Leave a Reply