Data Centers Part I: The Next Decade Driven by Acceleration & GenAI

The TD Cowen Insight

The concurrent paradigm shifts of generative AI and accelerated computing are driving fundamental changes in the data center across central processing, accelerators, networking, memory & storage, and power. In this report we break down what is changing and how it will impact key companies across the ecosystem (semis in particular) as the industry reacts to these tectonic shifts.

Our Thesis

The paradigm shifts towards accelerated computing and generative AI will both drive and enable profound changes in architecture, especially at data center scale. While we recognize much has been written on AI, we take a more granular view analyzing these paradigm shifts and trace the impact on data center architecture across compute, memory, networking and power. As Moore's Law has slowed, the need for greater offload of tasks to tailored, workload-specific hardware accelerators (of which AI accelerators are a subset) has increased. We see this trend continuing and accelerating, with hardware heterogeneity and tightly coupled accelerators driving efficiency and TCO advantages.

Similarly, the rise of GenAI has driven an explosion of demand for AI acceleration hardware, both for training ever larger models and running at-scale, production-level inference of large language models − with inference itself necessitating acceleration (for the first time). In our view, a rising tide is likely to lift many boats, but the true winners will be the companies that can deliver (or enable) fully optimized systems from silicon to servers through to software. Further, we believe these trends are likely more durable and long lasting than many investors appreciate and could drive above-trend growth through our forecast period (and likely beyond).

What Is Proprietary

In this report, we develop a bottom-up, company-specific framework for data center silicon spending to 2030E across Central Processing Unit, Graphics Processing Unit, Tensor Processing Unit, Application-Specific Integrated Circuit (including both units and average selling price estimates). Further, we discuss in-depth key industry trends including low-level software infrastructure, networking architecture, and power management.

Financial & Industry Model Implications

Overall, we anticipate (at least) high-teens (%) annual growth for data center semis from 2024E to 2030E, with acceleration and networking unsurprisingly growing much faster than the overall market. We also include domain-specific forecasts for Central Processing Unit (including Arm-based processors), AI accelerators (with Tensor Processing Unit and others included), and networking spend (including the Smart Network Interface Card and Data Processing Unit market).

What to Watch

Ongoing investment in GenAI capabilities, software and market penetration; hyperscale, sovereign, enterprise data center CapEx trends and commentary; data center physical infrastructure impacts on power delivery, cooling and efficiency; and durability of demand across data center spending verticals.