Views: 0 Author: Site Editor Publish Time: 2026-09-16 Origin: Site
When an AI chip glows red-hot and liquid cooling rushes in around it, that is the physical reality the industry can no longer ignore.
For years, the AI chip industry answered one question: how do we pack more computing power into silicon? Transistors shrank. Chiplets appeared. CoWoS bundled multiple dies together. HBM stacked memory ever higher. NVLink, UALink, and even CPO moved data between chips faster.
Every one of those answers eventually runs into the same physical wall: heat.
At SEMICON Taiwan 2026, TSMC's advanced packaging research and development lead James Chen laid out numbers that put the problem in perspective. Between 2024 and 2029, TSMC expects CoWoS packages to grow from about 3.3 times a reticle size to beyond 14 times. Compute transistors in a single package could rise roughly 48-fold, and HBM bandwidth more than 34-fold. Alongside all that, AI system power could climb about 6x in five years, with advanced AI packages moving from hundreds of watts toward the 4.1kW level.
Chips can keep stacking, HBM can keep growing, and packages can keep expanding. But the heat those transistors generate has to go somewhere. Cooling is no longer a supporting detail of the AI server — it is becoming the fundamental constraint on whether the next generation of AI chips can keep scaling at all.
The semiconductor industry has always been a story of scaling, and the first era was transistor scaling. Smaller nodes — 28nm, 16nm, 7nm, 5nm, 3nm, 2nm, and into angstroms — meant more transistors per square millimeter.
When advanced nodes became too expensive and single-die size hit reticle limits, the industry moved to the second era: packaging scaling. Chiplet, 2.5D, 3D IC, CoWoS, HBM, and hybrid bonding all solve the same problem — since one die cannot grow forever, put more silicon together. TSMC's disclosed roadmap shows 14x-reticle CoWoS integrating about 10 large compute dies and 20 HBM stacks by 2028, and beyond that, a roughly 40x-reticle SoW-X system-level wafer technology in 2029.
But a package can hold 10 compute dies and 20 HBM stacks only if those chips can run flat-out at the same time. The real limit is no longer how much silicon you can fit — it is how many watts per square centimeter you can safely run. That is why AI chips are entering a third scaling era: thermal scaling.
This changes the metrics of the AI chip race. Alongside transistor counts, TB/s bandwidth, bump pitch, and die-to-die bandwidth, a metric borrowed from the thermal-engineering world — W/cm², watts per unit area — becomes increasingly decisive.
The numbers show why. At a liquid-cooling session at NVIDIA GTC 2026, Supermicro showed rack power climbing from about 45kW in recent years, with the Vera Rubin reference architecture reaching about 227kW per rack. Its roadmap points to 400-500kW racks next, and toward megawatt racks in the long run. Vera Rubin requires 100% liquid cooling. A 600W GPU is manageable with a large heatsink or a cold plate; a package integrating ten large compute dies and 20 HBM stacks — running at thousands of watts — is an entirely different problem. A study published in Applied Thermal Engineering in July 2026 built a realistic 2.5D package test structure with epoxy molding compound and RDL, running at about 948W total power and 400W/cm² heat flux, and it listed suppressing thermal crosstalk between chiplets as a key goal of microchannel cooling. [1]
For decades, the basic recipe for chip cooling barely changed: move heat from the silicon to the outside, then let air or liquid carry it away.
Early data centers relied on air cooling. Heat moved from the die through thermal interface material (TIM), a heat spreader, and a large heatsink into the air, with fans and room air conditioning doing the rest.
When GPUs entered the hundreds of watts, direct liquid cooling took over. A typical cold plate path looks like: silicon → TIM → heat spreader → TIM → cold plate → coolant. Liquid handles far higher heat flux than air, but there is an unavoidable penalty: every interface between the chip and the coolant adds thermal resistance.
IBM's research on advanced chip thermal management makes the same point. Conventional flip-chip packages cool through the die backside via a copper lid and TIM layers; removing one TIM layer, or bringing the cold plate closer to the bare die, cuts thermal resistance. IBM argues that interlayer cooling is the only volume-scalable solution that grows with the number of dies in a stack, enabling very high 3D integration — though it demands sealing structures, because electrical interconnects sit close to the coolant flow, and the package must hold structural integrity even under high pressure. [2]
The engineering logic is simple: if the external cold plate is too far from the transistors, move the cooling structure closer to the silicon. That is exactly what microchannel cooling does.
Coolant is being pushed ever closer to the silicon — the microchannels live where the heat is.
Microchannel cooling works by machining micron-scale channels close to the die, letting coolant pass directly through the highest-heat-flux regions and shortening the path from silicon to liquid.
The idea is not new. Academic research on silicon microchannels for high-heat-flux chips dates back to 1981. In 2012, DARPA launched the ICECool program, explicitly pushing microfluidic cooling into substrates, dies, and packages. TSMC has studied it for years: in 2021, it disclosed a direct silicon water-cooling scheme for 3D ICs that bonded a silicon cap with grooved, grid-like cooling structures to the logic die via low-temperature bonding. Its test structures reached more than 2600W total cooling for a single SoC, about 4.8W/mm², and with direct silicon backside water cooling, demonstrated power density beyond 7W/mm².
At SEMICON Taiwan 2026, James Chen named microchannel cooling a key direction for future high-power AI advanced packaging. TrendForce reported that the research structure has demonstrated cooling capacity above 5kW, aimed at future packages drawing 4kW or more.
What makes microchannels genuinely new is not "smaller channels" — it is moving cooling from outside the chip into the package, and eventually into the silicon itself. And there is a real design split over how close to the silicon the channels should go.
One middle path, published in 2026 in Communications Engineering (Nature Portfolio), embeds microchannels in the packaging substrate rather than the semiconductor itself — "direct-to-package" cooling. The prototype handled about 625W/cm² with only 2-4mL of coolant; junction temperature and thermal resistance fell roughly 6-7x versus ambient air cooling and 2-3x versus traditional liquid cooling with TIM. [3]
The more aggressive route pushes coolant further inward. In 2026, researchers from KAIST and Georgia Tech demonstrated a manifold microchannel cooler with microfluidics embedded directly in a silicon thermal test chip, fabricated with a CMOS-compatible process below 350°C. Under room-temperature single-phase water cooling, it demonstrated chip-level heat flux above 2000W/cm², keeping junction temperature under 100°C with a total pressure drop of just 7.8kPa. [4]
And microchannels are leaving the lab. Microsoft has tested an in-chip microfluidic cooling system that etches channels into the silicon backside, bringing coolant as close as possible to the hot silicon. Depending on workload and configuration, lab tests showed up to 3x better performance than a conventional cold plate, and a 65% reduction in the maximum temperature rise of the GPU's silicon. Microsoft even used AI to identify hotspot distributions and design the channels so coolant flows to the hottest regions. It is now studying how to bring the technology into chip manufacturing and data-center deployment, and plans to work with foundry and chip partners to push microchannels into future self-designed chips.
Put together, microchannels are crossing a threshold: from a "cooling technology" into a "chip technology." That shift carries a real tension — the closer the coolant gets to the transistors, the greater the cooling potential; but each step toward the silicon raises manufacturing difficulty, sealing requirements, and reliability risk. The trajectory, in rough form: air cooling → cold plate liquid cooling → in-package microchannels → direct silicon cooling.
The heat problem is not limited to compute chips. In the 2.5D era, the memory sitting next to the GPU — HBM — can become the whole package's thermal bottleneck.
Stacked memory brings more active silicon into a tiny 3D volume — and heat follows.
HBM is not a flat DRAM. It stacks DRAM dies vertically with TSVs — from 4 and 8 layers to 12 and even 16 — packing more active silicon into a small 3D space. Its thermal problem has two dimensions: the heat it generates itself, plus the heat conducted over from the neighboring compute die. That is the thermal coupling introduced by 2.5D packaging.
A 2026 study from Vinci on 2.5D AI accelerator packaging notes that power density of modern AI accelerators and HPC devices regularly exceeds 1W/mm², and advanced 2.5D packages integrate high-power AI ASICs with 8 to 16 HBM stacks. The resulting coupling: local hotspots inside the ASIC conduct heat sideways into adjacent HBM, raising HBM junction temperature, which can force frequency reduction, performance throttling, and accelerated memory aging. In the representative package studied — two AI accelerator ASICs plus eight HBM stacks — HBM junction temperature approached 100°C at maximum power. [5]
There is a structural conflict underneath all this. Chip layout has traditionally optimized for signal distance, bandwidth, routing, and package area — put HBM close to the GPU to shorten paths and raise interconnect efficiency. But from a thermal standpoint, two high-power chips sitting closer together worsen each other's heat. Electrical design wants chips near; thermal design wants them apart.
So memory makers have started changing HBM internals to shed heat. In May 2026, SK hynix unveiled iHBM, which adds an integrated cooling element (ICE) made of electrically insulating, high-thermal-conductivity silicon-based material inside the HBM package, creating extra heat-conduction paths. SK hynix says the scheme cuts thermal resistance by more than 30%, targeting next-generation high-performance HBM.
Samsung is doing something similar. At COMPUTEX 2026, its HBM4E and HBM5 technologies included HPB — heat path blocks. Samsung notes that the die-to-die PHY responsible for high-speed GPU-HBM communication has become a major heat source inside the base die: the faster the data transfer, the worse the local heating. HPB builds extra heat-conduction paths right around that high-speed interface. Samsung is validating the approach on HBM4E and plans to adopt it in HBM5.
Every one of these moves points the same way: coolant — or at least heat-conduction structure — is getting closer to the transistor. Once it was cold air in the data center; then it entered the server; then the cold plate; now the package; and in the future, possibly the silicon itself. For the first time, the boundary of cooling has moved inside semiconductor packaging.
Whether microchannels become a real industry, though, does not depend on cooling a chip in a lab. The hard question is manufacturing them reliably.
That is why another SEMICON Taiwan 2026 development mattered. German laser and semiconductor equipment maker TRUMPF showed, for the first time, an ultrafast-pulse laser processing solution for integrated cooling of AI chips. According to TRUMPF's September 1 release, the system can machine ultrafine cooling microstructures inside chip stacks, with the goal of industrial, low-cost manufacturing — not lab prototypes.
A technology becomes a semiconductor industry only through a chain: architecture → process development → equipment → materials → yield → volume production. When dedicated equipment vendors start building industrial tools for chip-integrated cooling, cooling has moved from a system component to a manufacturing process.
Today's liquid-cooling supply chain revolves around system parts: cold plates, CDUs, pumps, manifolds, and heat exchangers. If microchannels push into packages and even silicon, the chain changes in at least four ways.
First, microstructure machining. As channels shrink, they can be made by laser, silicon etching, or deep etching. Channel dimensions, surface roughness, and consistency directly affect flow and cooling performance.
Second, bonding and sealing. Etching channels is only step one. Coolant must be reliably sealed very close to expensive AI chips. How package layers bond to microchannel structures, whether they leak over thermal cycling, and whether interface materials corrode all become new process challenges.
Third, fluid distribution. Not every spot inside a large AI package consumes the same power. Future cooling systems will need different flow allocation for compute dies, HBM, and I/O chiplets. Too little local flow creates new hotspots; channels that are too fine raise pressure drop and pumping power.
Fourth, inspection and reliability. Advanced packaging already relies on X-ray, acoustic, and optical inspection for bumps, voids, and bonds. Add liquid channels inside the package, and you must also verify channel integrity, sealing, leak detection, particle contamination, and coolant-material compatibility. A 2026 review of microchannel cooling flags coolant selection, channel structure, substrate materials, and manufacturing complexity as the open problems for industrialization — different coolants trade off thermal conductivity, electrical insulation, viscosity, and chemical stability.
The closer water gets to the chip, the higher the reliability bar. For an AI accelerator worth tens of thousands of dollars or more, any small leak, blockage, or material-compatibility failure costs far more than it would in an ordinary server. So embedded cooling will not replace cold plates overnight. The realistic path is gradual migration: cold plates stay common but get finer inside; microchannels enter heat spreaders or packages; and on the highest-power-density products, cooling moves toward direct silicon contact.
What matters is not which technology wins. It is that cooling is acquiring an entire set of process requirements closely resembling semiconductor manufacturing itself.
HBM makers are adding dedicated heat-conduction paths. Advanced packaging is researching microchannels. Equipment vendors are building microstructure tools. These scattered moves point to one trend: thermal management is becoming part of AI compute scaling.
For tomorrow's AI chips, water routing will be designed, etched, and bonded at the wafer and packaging stage just like electrical routing — RDL, TSV, power delivery networks. Cooling is no longer a support system for AI compute; it has become part of semiconductor manufacturing itself, and the ultimate physical boundary for how far the next generation of AI compute can go.
