The Race for AI Infrastructure

Recently, President Trump announced the Stargate Initiative, a staggering $500 billion investment aimed at accelerating the creation of AI datacenters. With major players like Microsoft already committing $80 billion in capital spending this year, Meta investing over $20 billion for AI infrastructure alone, and similar investments expected from Google and AWS, the AI infrastructure race is undeniably on. In 2024, nearly $300 billion was spent globally on datacenters. But this explosive investment brings with it a significant question: Are we creating a sustainable, efficient ecosystem—or are we falling into a cobweb cycle?

Understanding the Cobweb Cycle

The cobweb cycle, a concept seen in the early semiconductor industry, is a pattern of overproduction driven by market participants reacting to current prices rather than anticipating future trends. In the context of AI infrastructure, it could lead to a situation where companies invest heavily based on current demand projections—only to find that by the time their datacenters are up and running, the market conditions have shifted, causing overproduction and price instability. The result? A boom followed by a bust.  This cycle is shown in the chart below.

                                 A Cobweb growing around the supply and demand curves

In the early days of semiconductors, this cycle was notorious for pushing companies to overbuild fabrication plants (FABs) in response to short-term demand. However, since then, the industry has consolidated and become more efficient at balancing supply and demand. Will the AI infrastructure market fall into the same trap?

Training vs. Inference: The Two Sides of AI Compute Demand

When we talk about AI, it’s crucial to differentiate between training and inference. Training refers to the process of creating or modifying models, which requires massive amounts of computational power. Inference, on the other hand, refers to the use of those models once they’re trained—and it’s where things get interesting.

While training is currently driving much of the demand for compute power, we expect inference to eventually take the lead. However, there’s a surprising twist: inference doesn’t necessarily require vast amounts of computing power. For example, NVIDIA’s Digits 3000 offers a $3,000 solution that can run significant models on a personal computer. Furthermore, the introduction of models like Deepseek and Mistral, which utilize Mixture of Expert (MoE) architectures, has demonstrated that much less computational power is needed to run complex AI tasks than previously thought.

In fact, some researchers have already successfully used 8 Mac Minis to run the full Deepseek V3 model with impressive results—less than 3 seconds to the first token and nearly 6 tokens per second. This suggests that, with the right optimizations, even consumer-level hardware could handle complex AI tasks. A laptop with 512GB of RAM and silicon acceleration could run many applications just as efficiently as the massive datacenters we’re building today.

The Economics of Private AI Inference: A New Paradigm?

Let’s take a hypothetical scenario: If the industry were to invest $400 billion in AI infrastructure this year, that capital could fund the production of 100 million high-performance inference computers, each capable of running AI models privately and on-demand. With this setup, these 100 million devices could process an astounding 15 quadrillion tokens annually—equivalent to generating 36 million words per person in the U.S.

This raises an important question: What happens to the massive datacenter investments if distributed models become more efficient, cheaper to run, and more easily accessible? Will datacenters become obsolete, or will they shift focus toward more specialized needs?

The Risk for Datacenter Providers

The risk faced by datacenter providers is significant. The structure and efficiency of AI models could change dramatically in the next 3 to 5 years. Today’s investments are based on the assumption that traditional architectures will dominate, but if innovations like MoE models continue to evolve, the demand for centralized datacenter compute may not materialize as expected. If smaller, more efficient inference models take over, we could see the current datacenter boom morph into a costly misstep, with infrastructure that no longer serves the market’s needs.

At smartR AI, we often work with clients to optimize their AI workloads, specifically focusing on reducing inference costs. For many organizations, inference is a major budgetary challenge, and we’ve seen a growing market pressure to move to models that are cheaper to run. As AI continues to evolve, I suspect this pressure will only intensify.

Are Datacenters Built for the Right Future?

Technologies like multimodal LLMs (large language models) could shift the landscape, increasing demand for compute. However, it’s important to consider whether current datacenters are optimized for these new architectures. Deepseek has already demonstrated that AI models can be far more efficient to run than we once thought. But as we’ve seen in the past, hardware development (like designing new silicon or building out new datacenters) is a slow-moving process, while AI model development happens at a rapid pace. The risk is that the hardware we’re building today may not align with the needs of tomorrow’s AI technologies.

Conclusion: The Future of AI Infrastructure

We are on the cusp of a major transformation in AI infrastructure, driven by massive investments and rapid innovation in both hardware and software. However, the speed of AI model evolution—coupled with the slow pace of hardware development—means there’s a significant risk of overinvestment in infrastructure that may not meet the future needs of the industry.

As the demand for AI continues to grow, we must remain mindful of the possibility that the datacenters being built today could become expensive white elephants in just a few years. At smartR AI, we’re focused on helping clients navigate this uncertain landscape, optimizing their AI operations to stay ahead of the curve. The question is: Will the infrastructure boom be a sustainable foundation for AI’s future, or just another overblown cycle in the making?

Written by Oliver King-Smith, CEO of smartR AI.