Recently, DeLL’ORO GROUP released a forecast report, reviewing the outstanding trends in the 23-year data center infrastructure report and looking forward to the development of data center infrastructure in 2024. The following is the report content.

Outstanding Trends Highlighted In 2023 Forecasts

• Data center capex growth in 2023 has decelerated noticeably, as projected after a surge of spending growth in 2022. The top 4 US cloud service providers (SPs) in aggregate slowed their capex significantly in 2023, with Amazon and Meta undergoing a digestion cycle, while Microsoft and Google are on track to increase their greenfield spending on accelerated computing deployments and data centers. The China Cloud SP market remains depressed as cloud demand remains soft from macroeconomic and regulatory headwinds, although there are signs of a turnaround in second-half 2023 from AI-related investments. The enterprise server and storage system market performed worse than expected, as most of the OEMs are on track to experience a double-digit decline in revenue growth in 2023 from a combination of inventory correction, and lower end-demand given the economic uncertainties. However, network and physical infrastructure OEMs have fared better in 2023 because strong backlog shipments fulfilled in the first half of 2023 which lifted revenue growth.
• We underestimated the impact of accelerated computing investments to enable AI applications in 2023. During that year, we saw a pronounced shift in spending from general-purpose computing to accelerated computing and complementary equipment for network and physical infrastructure. AI training models have become larger and more sophisticated, demanding the latest advances in accelerators such as GPUs and network connectivity. The high cost of AI-related infrastructure that was deployed helped to offset the sharp decline in the general-purpose computing market. However, supplies on accelerators have remained tight, given strong demand from new hyperscale.
• General-purpose computing has taken a backseat to accelerated computing in 2023, despite significant CPU refreshes from Intel and AMD with their fourth-generation processors. These new server platforms feature the latest in server interconnect technology, such as PCIe 5, DDR5, and more importantly CXL. CXL has the ability to aggregate memory usage across servers, improving overall utilization. However, general-purpose server demand has been soft, and the transition to the fourth-generation CPU platforms has been slower than expected (although AMD made significant progress in 3Q23). Furthermore, CXL adoption is limited to the hyperscale market, with limited use cases.
• Server connectivity is advancing faster than what we had expected a year ago. In particular, accelerated computing is on a speed transition cycle at least a generation ahead of the mainstream market. Currently, accelerated servers with NVIDIA H100 GPUs feature network adapters at up 400 Gbps with 112 Gbps SerDes, and bandwidth will double in the next generation of GPUs a year from now. Furthermore, Smart NIC adoption continues to gain adoption, though, mostly limited to the hyperscale market. According to our Ethernet Adapter and Smart NIC report, Smart NIC revenues increased by more than 50% in 2023.
• The edge computing market has been slow to materialize, and we reduced our forecast in the recent Telecom Server report, given that the ecosystem and more compelling use cases need to be developed, and that additional adopters beyond the early adopters have been limited.

According to our Data Center IT Capex report, we project data center capex to return to double-digit growth in 2024 as market conditions normalize. Accelerated computing will remain at the forefront of capex plans for the hyperscalers and enterprise market to enable AI-related and other domain specific workloads. Given the high cost of accelerated servers and their specialized networking and infrastructure requirements, the end-users will need to be more selective in their capex priorities. While deployments of general-purpose servers are expected to rebound in 2024, we believe greater emphasis will be made to increase server efficiency and utilization, while curtailing cost increases.

Below, we highlight key trends that can enhance the optimization of the overall server footprint and decrease the total cost of ownership for end-users:

Accelerated Computing Maintains Momentum

We estimate that 11% of the server unit shipments are accelerated in 2023, and are forecast to grow at a five-year compound annual growth rate approaching 30%. Accelerated Servers contain accelerators such as GPUs, FPGAs, or custom ASICs, and are more efficient than general-purpose servers when matched to domain-specific workloads. GPUs will likely remain as the primary choice for training large AI models, as well as running inference applications. While NVIDIA currently has a dominant share in the GPU market, we anticipate that other vendors such as AMD and Intel will gain some share over time as customers seek greater vendor diversity. Greater choices in the supply chain could translate to much-needed supply availability and cost reduction to enable sustainable growth of accelerated computing.

Advancements in Next-Generation Server Platform

General-purpose servers have been increasing in compute density, as evolution in the CPUs is enabling servers with more processor cores per CPU, memory, and bandwidth. Ampere Computing Altra Max, AMD’s Bergamo are offered with up to 128 cores per processor, and Intel’s Granite Rapids (available later this year), will have a similar number of cores per processor. Less than seven years ago, Intel’s Skylake CPUs were offered with a maximum of 28 cores. The latest generation of CPUs also contains onboard accelerators that are optimized for AI inference workloads.

Lengthening of the Server Replacement Cycle

The hyperscale cloud SPs have lengthened the replacement cycle of general-purpose servers. This measure has the impact of reducing the replacement cost of general-purpose servers over time, enabling more capex to be allocated to accelerated systems.

Disaggregation of Compute, Memory, and Storage

Compute and storage have been disaggregated in recent years to improve server and storage system utilization. We believe that next-generation rack-scale architectures based on CXL will enable a greater degree of disaggregation, benefiting the utilization of compute cores, memory, and storage.

forward from：Data Center Infrastructure—a Look into 2024, Baron Fung , Dell’Oro Group