Technology

Data Centers Part I: The Next Decade Driven by Acceleration & GenAI

By: Matthew D. Ramsay, Ethan Potasnick, Sean O'Loughlin, George Kuhle, Krish Sankar, Joshua Buchalter, Michael Elias, John Blackledge, Derrick Wood, Steven Chin, Eddy Orabi, Robert Mertens, Lannie Trieu, Sam Reiff

Jun. 19, 2024 - 2 minutes 30 seconds

Engineers talking in a data center server room.

The TD Cowen Insight

The concurrent paradigm shifts of generative AI and accelerated computing are driving fundamental changes in the data center across central processing, accelerators, networking, memory & storage, and power. In this report we break down what is changing and how it will impact key companies across the ecosystem (semis in particular) as the industry reacts to these tectonic shifts.

Our Thesis

The paradigm shifts towards accelerated computing and generative AI will both drive and enable profound changes in architecture, especially at data center scale. While we recognize much has been written on AI, we take a more granular view analyzing these paradigm shifts and trace the impact on data center architecture across compute, memory, networking and power. As Moore's Law has slowed, the need for greater offload of tasks to tailored, workload-specific hardware accelerators (of which AI accelerators are a subset) has increased. We see this trend continuing and accelerating, with hardware heterogeneity and tightly coupled accelerators driving efficiency and TCO advantages.

Similarly, the rise of GenAI has driven an explosion of demand for AI acceleration hardware, both for training ever larger models and running at-scale, production-level inference of large language models − with inference itself necessitating acceleration (for the first time). In our view, a rising tide is likely to lift many boats, but the true winners will be the companies that can deliver (or enable) fully optimized systems from silicon to servers through to software. Further, we believe these trends are likely more durable and long lasting than many investors appreciate and could drive above-trend growth through our forecast period (and likely beyond).

What Is Proprietary

In this report, we develop a bottom-up, company-specific framework for data center silicon spending to 2030E across Central Processing Unit, Graphics Processing Unit, Tensor Processing Unit, Application-Specific Integrated Circuit (including both units and average selling price estimates). Further, we discuss in-depth key industry trends including low-level software infrastructure, networking architecture, and power management.

Financial & Industry Model Implications

Overall, we anticipate (at least) high-teens (%) annual growth for data center semis from 2024E to 2030E, with acceleration and networking unsurprisingly growing much faster than the overall market. We also include domain-specific forecasts for Central Processing Unit (including Arm-based processors), AI accelerators (with Tensor Processing Unit and others included), and networking spend (including the Smart Network Interface Card and Data Processing Unit market).

What to Watch

Ongoing investment in GenAI capabilities, software and market penetration; hyperscale, sovereign, enterprise data center CapEx trends and commentary; data center physical infrastructure impacts on power delivery, cooling and efficiency; and durability of demand across data center spending verticals.

Subscribing clients can read the full report, The Next Datacenter Decade, Driven By Acceleration & GenAI - Ahead Of The Curve, on the TD One Portal

Disclaimer

The views or opinions expressed herein represent the personal views of the writer and do not necessarily reflect the views of TD Securities or its affiliates.

This material is intended to provide commentary on the market for commodities discussed herein.

Not Advice: The information contained in this material is for informational purposes only and is not intended to provide professional, investment or any other type of advice or recommendation, or to create a fiduciary relationship. Neither TD Securities (USA) LLC (“TD Securities USA”) nor any of its affiliates (collectively, “TD”) makes any representation or warranty, express or implied, regarding the accuracy, reliability, completeness, appropriateness or sufficiency for any purpose of any information included in this material. Certain information may have been provided by third-party sources and, while believed to be reliable, has not been independently verified by TD, and its accuracy or completeness acannot be guaranteed. You should not make an investment decision in reliance on this material, which is intended to provide only brief comments on the topics addressed, and is based on information that is likely to change without notice.

Not Securities or Derivatives Research: This material has not been produced, reviewed or approved by TD’s securities or derivatives research departments. The views of the author may differ from others at TD, including TD securities or derivatives research analysts.

Not Independent: The views expressed in this material may not be independent of the interests of TD. TD may engage in conflicting activities, including principal trading before or after posting this material, or other services involving commodities discussed in this material, or related financial products. TD may have a financial interest in the commodities discussed in this material, including, without limitation, a financial product that references such commodities.

Not An Offer or Solicitation: Nothing contained in this material is, or should be construed as, an offer, a solicitation of an offer or an invitation to buy or sell any commodity, or any financial product that references such commodity, and it is not intended for distribution in any jurisdiction where such distribution would be contrary to law.

Risk of Loss. Transactions in commodities, and financial instruments that reference commodities, involve risk of loss, and are subject to the risks of fluctuating prices. You should weigh potential benefits against the risks. Past performance is no indicator of future performance and the Materials are not intended to forecast or predict future events.

Matthew D. Ramsay

Managing Director, TMT – Semiconductors Research Analyst, TD Cowen

Matthew D. Ramsay
Managing Director, TMT – Semiconductors Research Analyst, TD Cowen

Biography

Matthew D. Ramsay is a Managing Director and senior research analyst covering the semiconductor sector. Prior to joining TD Cowen, Mr. Ramsay worked as an analyst at Canaccord Genuity, where he also covered semiconductor stocks. He previously worked as a senior design engineer at AMD and held senior performance engineering roles at Sun Microsystems and Spansion.

Mr. Ramsay holds a BS in computer engineering from the University of Evansville, where he was a NCAA Division I Men’s Soccer athlete. He holds an MS in electrical engineering & computer architecture from the University of Wisconsin, and an MBA from Indiana University’s Kelley School of Business.

Krish Sankar

Managing Director, TMT - Semiconductor Capital Equipment, IT Hardware Research Analyst, TD Cowen

Krish Sankar
Managing Director, TMT - Semiconductor Capital Equipment, IT Hardware Research Analyst, TD Cowen

Biography

Krish Sankar is a Managing Director and Senior Research Analyst covering the semiconductor capital equipment and IT hardware sectors. He joined TD Cowen’s Technology, Media & Telecom team in February 2018. Prior to joining TD Cowen, Mr. Sankar worked as a top-ranked analyst at Bank of America Merrill Lynch for almost 14 years, where he covered the semiconductor capital equipment, EDA software, and alternative energy sectors. Before joining Bank of America-Merrill Lynch, Mr. Sankar was a product manager at Applied Materials. Mr. Sankar is ranked among the top semiconductor capital equipment analysts in Institutional Investor polls.

Mr. Sankar holds a B.Tech (with Honors) in ceramic engineering from IIT (BHU), Varanasi, and an MS in materials science and engineering from the University of Minnesota.

Joshua Buchalter, CFA

Director, TMT – Semiconductors Research Analyst, TD Cowen

Joshua Buchalter, CFA
Director, TMT – Semiconductors Research Analyst, TD Cowen

Biography

Joshua Buchalter is a director covering the semiconductor and mobility technology sectors. He has been working on the semiconductors team at TD Cowen since 2018.

Prior to joining TD Cowen, Mr. Buchalter covered the semiconductor sector as an equity research associate at Oppenheimer & Co and Needham & Co.

Mr. Buchalter received a BSc in economics from Tufts University, an MBA in finance and accounting from Boston College, and is a CFA® charterholder.

Michael Elias

Vice President, TMT - Communications Infrastructure Research Analyst, TD Cowen

Michael Elias
Vice President, TMT - Communications Infrastructure Research Analyst, TD Cowen

Biography

Michael Elias is a Vice President covering the Communications Infrastructure sector including Data Centers and Content Delivery Networks (CDN’s) and has been a member of the Communications Infrastructure team at TD Cowen since 2017.

Prior to joining TD Cowen, Mr. Elias worked as an equity analyst at Xanthus Capital Management. Mr. Elias received his B.S. in Industrial Engineering and Operations Research: Engineering Management Systems at Columbia University’s School of Engineering and Applied Sciences.

John Blackledge

Managing Director, TMT – Internet Research Analyst, TD Cowen

John Blackledge
Managing Director, TMT – Internet Research Analyst, TD Cowen

Biography

John Blackledge is a managing director and senior research analyst who covers the Internet and new media sector. Prior to joining TD Cowen in August 2012, he was a vice president covering Internet stocks at Credit Suisse for more than three years. Before joining Credit Suisse, Mr. Blackledge spent six years at JPMorgan, where he was a vice president covering the entertainment, broadcasting, and cable/DBS sectors before assuming research coverage of radio, TV, and outdoor stocks. Earlier in his career, he held positions at ABN AMRO, CIBC, and Arthur Andersen. Mr. Blackledge has a BS degree with a major in accounting from Georgetown University.

Derrick Wood

Managing Director, TMT – Software Research Analyst, TD Cowen

Derrick Wood
Managing Director, TMT – Software Research Analyst, TD Cowen

Biography

Derrick Wood joined TD Cowen in April 2016 and covers the enterprise software sector, including SaaS and big data/analytics. Before joining TD Cowen, Mr. Wood was an analyst with Susquehanna International Group, Wedbush Securities, and Pacific Growth Equities. He started covering enterprise software in 2000.

Mr. Wood received a bachelor’s degree in economics from UC Davis. He is a CFA charterholder.