There’s a conversation we still aren’t having honestly enough: AI isn’t just a software challenge anymore.
It’s a power problem. And many of the software abstractions we’ve spent decades celebrating are now quietly making that problem worse.
Data centers are being built as fast as utilities will allow. In many regions, the limiting factor is no longer capital or demand, but grid capacity, cooling infrastructure, and permitting. New builds are being delayed or outright denied because the power simply isn’t there. This isn’t speculative. It’s already happening.
Yet the industry keeps acting as if “software efficiency” was solved years ago because CPUs and GPUs got fast.
They did get fast. We’re just wasting it.
Abstraction Has a Physical Cost
Modern software stacks exist for one primary reason: to make large groups of humans productive. Object-oriented design, layered frameworks, dynamic allocation, runtime polymorphism, containers, and deep abstractions all reduce cognitive load. They help teams move faster and make fewer mistakes.
What they were never designed to do is minimize power consumption.
Every layer of abstraction has a physical footprint. Extra indirection means extra memory accesses. Cache misses translate directly into energy. Pointer chasing becomes heat. Memory bloat becomes power draw. At the level of a single program, this barely registers. At the scale of global infrastructure, it adds up to terawatts.
That tradeoff made sense for a long time. Compute was cheap. Power was cheap. Software complexity was exploding, and humans were the bottleneck. Optimizing for developer productivity was the right call.
That world no longer exists.
Compute is expensive again. GPUs are capital equipment. Power is constrained by politics and physics. Cooling is a first-order engineering problem. Small inefficiencies don’t stay small when multiplied across millions of servers running continuously.
“But C++ Can Be As Fast As C”
In theory, sure. In practice, almost never.
Real systems aren’t microbenchmarks or hand-tuned kernels. They’re built by large teams, under deadlines, using abstractions that are considered reasonable and maintainable. The result is code that is heavier than it needs to be, more indirect than it needs to be, and far more memory-hungry than the hardware would prefer.
That isn’t a failure of engineers. It’s the predictable outcome of a culture that optimizes for human scalability rather than machine efficiency.
Until recently, that cost was mostly hidden. Now it shows up on the power bill.
AI Changes the Tradeoff
Here’s the shift many people haven’t fully internalized yet.
We invented abstraction to protect humans from complexity. Now we’re increasingly handing that complexity to machines.
AI systems don’t need inheritance hierarchies to understand relationships. They don’t need design patterns to manage cognitive load. They don’t need layered architectures to stay sane. Those tools exist for us.
If machines are increasingly writing and maintaining code, then the original justification for heavy abstraction starts to weaken. That opens the door to a different kind of software design: flatter, more explicit, more data-oriented, more locality-aware. Not because it’s prettier or more elegant, but because it’s cheaper to run.
When power is the constraint, efficiency stops being optional.
The Real Bottleneck Isn’t Compute
We’re not going to run out of clever algorithms. We’re not going to run out of GPUs. What we’re going to run out of is grid capacity, permits, cooling water https://www.waterwaysjournal.net/2025/10/10/data-centers-raise-questions-about-water-usage/, and political tolerance for 500-megawatt facilities and methane assisted data centers. https://www.msn.com/en-us/weather/topstories/epa-accuses-elon-s-methane-xai-datacenter-of-secretly-making-power/ar-AA1UBLHO
The bottleneck is becoming physical, and in a physically constrained world, inefficiency stops being an aesthetic problem and becomes an economic one.
The Point
This isn’t an attack on C++, object-oriented programming, or modern tooling. Those choices made sense for the world we were in. We optimized software for a time when humans were the bottleneck and compute was cheap.
We’re entering a different world. Machines are increasingly the programmers. Power is scarce. Heat is expensive. Infrastructure is the constraint.
In that world, abstraction is no longer free so we should probably stop pretending that it is.
If you need help shipping an embedded product—or rescuing one that’s stuck—talk to an engineer https://endvr.com/contact/