One of the recurring mistakes in discussions about AI infrastructure is to look for the bottleneck.
GPUs are scarce.
Then HBM is scarce.
Then advanced packaging.
Then cooling.
Then transformers and switchgear.
Then grid connections.
This can look like a sequence of unrelated shortages.
It is not.
It is the characteristic behaviour of a constrained system under rapidly increasing demand.
A particularly good interactive visualization of this process is “How Electricity Becomes Intelligence” by Manas Bihani. It follows an AI accelerator from model demand through compute, memory, packaging, thermal management, rack architecture, power delivery, the data centre and finally the electricity grid.
Its central idea can be summarized in one sentence:
Nothing was fixed. The constraint moved.
That is a much more useful way to think about the current AI infrastructure build-out.
A bottleneck is not a component
Consider a system with capacities
$$
C_1,C_2,\ldots,C_n.
$$
To first approximation, its achievable throughput is constrained by
$$
C_{\mathrm{system}}=\min_i C_i.
$$
If demand increases until one capacity becomes binding, industry invests in that capacity.
The interesting part comes next.
Once
$$
C_k
$$
has been expanded sufficiently, it is no longer necessarily the limiting factor. Another capacity becomes binding:
$$
C_j<C_k.
$$
The bottleneck has moved.
This sounds trivial when written as an equation. In a real technological system it can be surprisingly difficult to see, because each constraint belongs to a different engineering discipline, industrial sector and investment cycle.
AI hardware provides an unusually clear example.
From compute to memory
For decades, semiconductor scaling allowed more computation to be obtained without proportionally increasing power.
That stopped being true.
Higher throughput increasingly required more silicon and more electrical power. Accelerator performance therefore became an important constraint.
But improving accelerator arithmetic capability creates another problem: the processors need to be supplied with data fast enough to use that capability.
The constraint moves toward memory bandwidth.
HBM becomes important.
Solve memory bandwidth, however, and another problem appears. Large amounts of high-bandwidth memory and several compute dies have to be integrated into increasingly complex packages.
The constraint moves again:
compute → memory → packaging
None of the previous technologies disappears. They simply stop being the marginal constraint.
Then physics becomes increasingly visible
A modern high-performance accelerator can dissipate around a kilowatt or more through an area of only a few square centimetres.
At this point thermal engineering stops being a secondary packaging question.
The thermal path can be represented schematically as
$$
\text{silicon}
\rightarrow
\text{interface}
\rightarrow
\text{heat spreader}
\rightarrow
\text{cold plate}
\rightarrow
\text{coolant}.
$$
The relevant quantity is no longer merely total power.
It is heat flux.
$$
q^{\prime \prime}=\frac{P}{A}
$$
Increasing (P) while the package area remains approximately fixed causes every thermal resistance close to the die to matter more.
This explains why materials such as diamond suddenly become interesting.
Diamond can conduct heat several times better than copper and therefore potentially spread local hotspots before the heat reaches the cold plate.
But this introduces another systems lesson.
A material with excellent bulk properties does not automatically create an excellent component.
The complete resistance is closer to
$$
R_{\mathrm{interface}}
+
R_{\mathrm{spreader}}
+
R_{\mathrm{interface}}
+
R_{\mathrm{coldplate}}.
$$
If the diamond-to-semiconductor interface dominates, the extraordinary thermal conductivity of bulk diamond provides little benefit.
The bottleneck can therefore move from the material itself to its interface.
Why air cooling eventually loses
Traditional copper heat spreaders and fin stacks are mature technology.
They have not become useless.
They have become commoditized.
The problem is that transferring large amounts of heat from a fin stack into air requires increasingly large surfaces and increasingly large air flows.
For convective cooling,
$$
\dot Q=hA\Delta T.
$$
When heat density rises, there are only a few variables left to increase: (h), (A), or (\Delta T).
There are practical limits to all three.
Liquid cooling provides much larger heat-transfer coefficients and transports far more thermal energy per unit volume.
So the constraint moves again:
thermal spreading → air interface → liquid cooling
But liquid cooling does not merely replace a fan.
It changes the infrastructure.
Pipes, pumps, coolant distribution units, heat exchangers and facility water systems now become part of the computing architecture.
A solution at one scale creates requirements at the next scale.
The rack becomes a machine
As accelerators become more tightly interconnected, the server is no longer necessarily the natural unit of the system.
The rack starts behaving like a single machine:
- common power distribution,
- common cooling,
- high-bandwidth interconnect,
- coordinated accelerators.
Now rack-level power can increase from tens of kilowatts toward hundreds of kilowatts.
At that point electrical conversion losses, bus bars, cabling, transformers and switchgear stop being peripheral details.
Again:
cooling → rack → power delivery
And once racks consume that much power, the existing data-centre building itself may become the constraint.
A facility designed for 10 kW racks cannot necessarily accommodate 100–150 kW racks simply by replacing the servers.
Floor loading, cooling infrastructure, electrical feeders and power distribution were designed around another regime.
The bottleneck has escaped the computer.
Eventually, the computer reaches the electricity grid
This is perhaps the most interesting transition.
Electricity itself may not represent the largest operating cost of an accelerator compared with the hardware investment.
Yet access to electrical capacity can still determine whether the infrastructure can be built.
That distinction matters.
A resource can be relatively inexpensive per unit and still be scarce because access to it is rationed through physical capacity and long lead times.
At sufficiently large data-centre scale, the relevant question is no longer:
What does one kilowatt-hour cost?
It becomes:
Can another 300 MW or 1 GW be connected here, and when?
At that point the planning horizon of the electricity system collides with the development cycle of AI.
Model generations can change within months.
Grid connections can take years.
The final constraint is therefore no longer inside the accelerator, rack or even data centre.
It is part of the surrounding infrastructure.
This is a complex-systems problem
The important lesson extends far beyond AI.
The same pattern occurs in manufacturing systems, logistics networks, energy systems and supply chains.
Optimizing one subsystem changes the state of the whole system.
The relevant question is therefore not simply:
How do we remove the current bottleneck?
A better question is:
Where will the bottleneck move after we remove it?
That distinction separates local optimization from system understanding.
A useful model must therefore represent not only component performance but also dependencies between capacities, timescales and investment decisions.
In abstract form:
$$
(C_1(t),C_2(t),\ldots,C_n(t))
$$
with interventions changing one or several capacities,
$$
F(\mathbf C(t),\mathbf u(t)),
$$
while rising demand continuously changes which constraint becomes active.
The bottleneck is therefore not an object.
It is a state of the system.
And that state moves.
Follow the constraint
Manas Bihani’s interactive visualization makes this idea unusually tangible.
It traces ten stages from model demand through accelerator, HBM, packaging, thermal density, liquid cooling, racks, power delivery, data centres and finally the grid. It also looks at technologies beside the main path, including conventional copper/air cooling and possible future diamond heat spreading.
The value of the visualization is not primarily the individual numbers.
It is the structure of the argument:
Every solution changes what becomes scarce next.
For anyone interested in AI infrastructure, industrial systems, technology strategy or complex systems, it is worth exploring:
“How Electricity Becomes Intelligence” — Manas Bihani
https://manasbihani-com-kappa.vercel.app/bottleneck
A useful principle to keep from it:
Do not only optimize the bottleneck. Understand where it moves next.