13Aug, 2024
     It doesn’t matter where you run, what matters is the right direction

    Optical communications field – the main application scenarios of silicon photonic chips

    With the rapid development of artificial intelligence (AI) and machine learning (ML) applications, the demand is for components to exchange data quickly and consume as little power as possible while maintaining high computing density, especially in terms of short-distance connections between XPUs (CPU, GPU, and memory).

    The AI computing era we are facing has the following characteristics:

    • Require faster network speeds – rapidly advancing towards 800G/1.6T
    • Require network products to have lower latency – improve computing efficiency, and increase the response speed and training speed of AI systems
    • Require network products to have lower power consumption – reduce energy consumption and develop sustainable AI computing power
    • Require higher reliability of system hardware and software – reduce failure rate, improve availability and user experience of AI services

    In this context, traditional pluggable optical modules are already “stretched” in terms of cost performance and power consumption, while highly integrated high-speed silicon photonic chips have become a more superior option due to their obvious advantages in potential price reduction and power consumption. The market’s choice is the best proof. Silicon photonic modules have achieved partial commercial success, and the application of higher-speed silicon photonic modules will also increase significantly.

    Silicon photonics is emerging as a promising technology compared to traditional methods. Silicon photonics can enable better communication between computing units such as CPUs and GPUs. Memory units can also be improved to increase the computing power and efficiency of AI applications.

    Gigalight Silicon Photonics Transceiver

    In response to the comprehensive goal of large-scale commercial use and technological advancement of 800G architecture in AI data centers at home and abroad, Gigalight launched a low-power 800G series silicon photonic modules based on 7nm DSP with a power consumption of 16W, including 800G DR8\DR8+, 800G 2×FR4/2×LR4 data center optical modules, supporting OSFP and QSFP-DD packaging forms, laying a solid delivery foundation for the company to become a pioneer in the field of silicon photonics.

    800G OSFP 2×FR4/ 2×LR4

    Gigalight’s 800G OSFP 2×FR4 and 800G OSFP 2×LR4 optical module ports use 2× Dual LC interfaces. The transmitter uses a monolithic integrated silicon photonic modulation chip solution, using 4 CWDM4 light sources. The integrated silicon photonic chip realizes 1-to-2 light source and CWDM wavelength division and MZ modulation. The receiver is 2 discrete free-space optical POSAs and adopts a low-power direct-drive 7nm DSP chip solution. The product has the advantages of low power consumption and high performance.

    800G DR8/DR8+/DR8++

    Gigalight’s 800G OSFP DR8/DR8+/DR8++ optical module port uses a 2×MPO-12/APC interface. The transmitter uses a monolithic integrated silicon photonic modulation chip solution, uses two CWL light sources, and implements 1-to-4 and MZ modulation of the light source in the integrated silicon photonic chip. The receiver uses two discrete FA solutions and a low-power direct-drive 7nm DSP chip solution. This product has the advantages of low power consumption and high performance.

    At the same time, Gigalight also provides 800G silicon photonic modules in QSFP-DD package to meet the needs of different users.

    800G QSFP-DD DR8/DR8+/DR8++

    • 1310nm CW+MZ+PIN
    • 500m/2km/10km
    • <14.5W/16W

    800G QSFP-DD 2×FR4/ 2×LR4

    • CWDM4 CW+MZ+PIN
    • 2km/10km
    • <16W

    400G DR4

    As early as 2023, Gigalight has mass-produced 400G OSFP-RHS DR4 and 400G QSFP112 DR4 silicon photonic modules. The addition of the latest low-power version 800G silicon photonic series has strongly promoted the company’s confidence in the growth of next-generation optical communication performance. Gigalight has substantially entered the research and development of silicon photonic chips and silicon photonic modules since 2018, and has completed the mass production or sample stage of at least 10 silicon photonic modules. Thanks to the strong collaboration of the global supply chain, more innovative and differentiated silicon photonic modules will be launched in Gigalight in the future.

    In the 800G AI computing center, the 800G silicon photonic module can branch two 400G optical modules through single-mode optical fiber to achieve a smooth upgrade of the 400G network.

    5Jul, 2024
    Silicon photonics is the key to unlocking the full potential of AI

    As artificial intelligence (AI) and machine learning (ML) applications grow rapidly, we can foresee that industries and companies will use these systems and tools in their daily processes. As the complexity of these data-intensive applications continues to increase, the need for high-speed transmission and efficient communication between computing units becomes critical.

    This demand has sparked interest in optical interconnects, especially for short-distance connections between XPUs (CPUs, GPUs, and memory). Silicon photonics is emerging as a promising technology that can improve performance, cost-effectiveness, and thermal management capabilities compared to traditional approaches, ultimately improving the functionality of AI/ML applications.

    Advantages of Silicon Photonics in Artificial Intelligence

    Silicon photonics interconnects play a critical and specialized role in managing the growing demand for AI/ML applications across industries. These components need to exchange data quickly and consume as little power as possible while maintaining high computing density. Silicon photonics enables good communication between computing units such as CPUs and GPUs, and memory units can also be improved to increase the computing power and efficiency of AI applications.

    Silicon photonics example layout (Source: OpenLight)

    Laser integration is essential for generating, modulating, and manipulating optical signals and introducing them into various systems. However, this has always been a major challenge in the field of silicon photonics.

    On-chip optical interconnect technology

    To meet market demands, companies have begun investing in on-chip optical interconnects to enable scalability from one laser to hundreds of lasers, surpassing the challenges posed by traditional electrical interconnects.

    Short-reach optical interconnects using silicon photonics offer a solution that enables high-speed data transmission while reducing power consumption and increasing thermal efficiency (pJ/bit). This is important to reduce heat generation and keep systems running efficiently.

    In addition, silicon photonic integration can create smaller and denser photonic integrated circuits (PiCs), facilitating high-density bandwidth connections that are critical to AI/ML workloads. Heterogeneous integration can more efficiently connect lasers and waveguides, resulting in better coupling and reduced power consumption. In addition, with the development of new lasers, thermal efficiency has been improved, and the number of channels and the number of potential wavelengths per channel have been expanded.

    Overcoming the challenges of high-density bandwidth connections

    In the case of back-end manufacturing, companies can save significant operating and capital expenditures without having to use lasers externally coupled to passive silicon photonic chips. By using more channels per square inch of silicon and combining different active components together, it is possible to use less power and significantly increase the total bandwidth of each PIC.

    Gigalight Silicon Photonics 800G Optical Transceiver

    Silicon photonics allows for efficient data transmission in AI/ML applications using short-reach optical interconnects. In the case of AI applications such as natural language processing, image recognition, and autonomous driving, large data sets are processed and analyzed in real time.

    Fast and efficient data transmission is critical for real-time decision making and optimal system performance. Silicon photonics can provide high-speed data transmission and efficient communication between components, helping to improve the overall efficiency and performance of AI systems in these areas.

    By leveraging silicon photonics technology, companies can optimize their AI/ML systems and unleash more powerful computing power for more accurate and responsive results.

    The future of silicon photonics in artificial intelligence

    The road ahead is full of hope. Silicon photonic technology has the potential to revolutionize artificial intelligence algorithms and further enhance the capabilities of artificial intelligence systems. The use of silicon photonic technology in artificial intelligence can develop smarter systems that handle complex tasks with higher performance and efficiency.

    As architects further develop AI networks, silicon photonics and heterogeneous integration will transform the switching layer, replacing traditional packet switching, which will enable lower latency and lower power consumption at the required interconnect density.

    It is believed that silicon photonics technology will become a disruptive technology for future AI/ML systems and has significant advantages over traditional electrical signal solutions. This, in turn, could push the boundaries of what is possible in the field of AI.

    23Jan, 2024
    Exploring the Advantages of 200G (8x25G NRZ) Optical Technology in Data Centers

    GIGALIGHT, which has focused on optical communication for eight years, directs your attention to the 200G (8x25G NRZ) technology, delving into its advantages such as low power consumption, low latency, and easy deployment. This provides a deeper understanding of cost-effective optical interconnection solutions within data centers.

    There are two technological directions for 200G optical modules: one utilizes the QSFP-DD packaging for an economical 8x25G NRZ network, while the other employs QSFP56 packaging for a 4x50G PAM4 network (to be discussed in a subsequent article).

    Compared to PAM4 technology, 200G (8x25G NRZ) boasts advantages in low power consumption, low latency, and easy deployment.

    Technical details:

    QSFP-DD packaging for 8x25G NRZ network: The 200G optical module uses QSFP-DD packaging, transmitting 25G NRZ signals through 8 channels. NRZ, representing non-return-to-zero signals, involves signal switching between high and low levels. Compared to traditional PAM4 signals, NRZ signals are easier to modulate and demodulate.

    Low power consumption: Utilizing 25G NRZ optical components, the module’s power consumption is reduced by 2–3W compared to modules based on PAM4 DSP CDR technology. This offers significant advantages for data center operational costs (OPEX) and cooling efficiency.

    Low latency: 200G NRZ excels in special application scenarios with stringent network latency requirements, such as high-frequency trading networks. The transmission method of NRZ signals provides an advantage in terms of latency.

    Deployment ease:

    Mature industry chain: The technology of 25G NRZ optical chips and components is relatively mature, providing a reliable foundation for the production of 200G NRZ. This makes the development of related 200G switch ports and optical module products relatively easy, facilitating rapid commercial deployment.

    Low-cost optical interconnection: Due to mature technology and industry chains, 200G NRZ achieves low-cost optical interconnection within data centers. This offers an economically effective solution for the construction and expansion of large-scale data centers.

    200G QSFP-DD SR8

    The 200G QSFP-DD SR8 is a multimode parallel product, utilizing an 8-channel 850nm VCSEL array, compliant with the 100GBASE-SR4 protocol standard. Leveraging the advantages of traditional VCSEL platforms, GIGALIGHT has adopted a simple, efficient, and reliable fiber coupling process. By introducing a 45° prism between the laser and the fiber, along with special material treatment on the fiber surface, the fiber coupling efficiency has been increased to over 80%.

    200G QSFP-DD PSM8

    The 200G QSFP-DD PSM8 is a high-speed product utilizing single-mode parallel technology. In comparison to wavelength division multiplexing (WDM) technology, PSM technology offers cost advantages. GIGALIGHT PSM optical module enables error-free long-distance transmission up to 10km, five times the distance achieved by traditional PSM optical modules.

    200G QSFP-DD LR8

    The 200G QSFP-DD LR8 optical module is based on an 8-channel LAN-WDM wavelength division multiplexing design, achieving a 1E-12 bit error rate over a transmission distance of 20km. In the context of high-reliability 5G middle-backhaul transmission systems, the long-term transmission of PAM4 signals may pose challenges such as insufficient bandwidth. The 200G QSFP-DD LR8, however, offers a high-performance, low-latency, and highly reliable channel solution.

    GIGALIGHT has recently made significant strides in optical design, particularly achieving notable success in the development of 800G optical modules. The design includes the completion of 800G SR8/DR8. For more details, please refer to here. Moreover, GIGALIGHT has successfully mass-produced and deployed 8-channel optical products in niche markets globally, encompassing various fields such as 200G, 400G, and 800G. Visit our official website for further information.

    29Dec, 2023
    Global AI Server Market Surges to $50 Billion in 2023, Expected to Exceed 50% Share by 2027

    Recently, market research organization IDC released its latest research report on the global server market. The report indicates that in 2022, the global server market size experienced a year-on-year growth of 20.1%, reaching $123.224 billion. It is anticipated that the market size will see a slight increase to $128.471 billion in 2023. The annual growth rates for the following four years are estimated to be 11.8%, 10.2%, 9.7%, and 8.9%, respectively. By 2027, the market size is projected to reach $189.139 billion.

    Specifically, in 2022, global x86 server sales amounted to $110.955 billion, marking an 18.8% year-on-year increase. In contrast, non-x86 server sales surged by 32.8% year-on-year to $12.269 billion. In 2023, the growth in the global server market slowed down, with x86 server sales experiencing a mere 2.3% year-on-year increase to $110.955 billion. However, non-x86 server sales maintained a high year-on-year growth of 22.1%, reaching $14.984 billion. According to IDC’s projections, by 2027, x86 server sales will escalate to $165.568 billion, while non-x86 server sales will also increase to $23.571 billion.

    Looking at the broad pattern, although non-x86 servers have sustained continuous rapid growth over the coming years, x86 servers remain the mainstream in the server market, maintaining a sales-based share of over 87.5%. Even by 2027, the market share of non-x86 servers will still be slightly under 12.5%, barely up from 11.66% in 2023, an increase of less than 1 percentage point.

    However, significant shifts are occurring within the non-x86 server market segment. Over the past decade, around half of the non-x86 servers were comprised of IBM’s Power Systems and System z mainframes, with the rest being a combination of other proprietary servers and Arm servers. But with the rise of Arm servers among hyperscale vendors and cloud builders, the portion led by Arm in the non-x86 server segment is growing. Furthermore, with the emergence of RISC-V servers, this market segment is expected to undergo further changes.

    Another dataset from IDC reveals that in 2020, IBM’s System z and Power revenues amounted to $4.98 billion. This suggests that in 2020, non-x86 server product sales, including Arm servers and other processor architectures, totaled $3.87 billion. Assuming certain scenarios for future IBM products, such as the upgrade cycles for Power10 and z16 machines in 2021 and 2022, and Power11 and z17 machines in 2025, it’s possible to anticipate a gradual decline in IBM server hardware sales from slightly below $5 billion in 2020 to $3.5 billion in 2026, potentially reaching $3.3 billion by 2027. Based on IDC’s data, other types of servers within the non-x86 segment, like Arm/RISC-V, are expected to grow at a robust pace, subject to investment cycles of hyperscale enterprises and cloud builders, maintaining a benchmark level. Inevitably, if adoption occurs, the annual sales of Arm and RISC-V servers (mostly Arm servers) could reach around $20 billion.

    Moreover, due to numerous hyperscale enterprises and cloud builders focusing on customizing Arm server CPUs and AI coprocessors, there’s a wide range of choices available. This trend faces pressure not only from using X86 server CPUs and Nvidia GPUs for AI and other compute-intensive workloads but also from the widespread adoption of customized Arm servers.

    Considering this, how AI server sales (primarily for training but also for inference) differ from sales of other types of servers?

    IDC mentioned in its report: “In each subsequent quarter of 2022, inflation had a stronger direct impact on servers, with the average selling price (ASP) year-on-year growth rate rising to 29% in the second quarter of 2023, while server shipments mostly hovered around low double-digit year-on-year growth for most of 2022 but declined by 1.4% year-on-year in the fourth quarter of 2022 and by 10% in the first quarter of 2023, and now in the second quarter of 2023, it has again fallen by 19.9% year-on-year.”

    Compared to the significant decrease in shipment volumes, the continuous rise in average selling price is primarily driven by very expensive AI training and inference nodes (each node costing hundreds of thousands of dollars with four or eight GPUs). AI servers and non-AI servers indeed need to be segregated as they represent very distinct segments in the market. Therefore, The Next Platform, based on IDC’s revenue forecast for servers from 2020 to 2027 (IDC data indicates the global AI server market size was $11.2 billion in 2021), attempted to classify them as follows:

    The Next Platform believes that unless something slows down the growth of AI models or makes AI training and inference computations cheaper, it’s reasonable to expect that by 2026 or 2027, AI servers will account for approximately half of the entire server market revenue.

    The model assumes a steep decline in non-AI server revenue by 11.2% starting in 2022, followed by moderate growth similar to GDP every year. By 2023, this situation is expected to improve, with only a 5% decline. Between 2022 and 2023, AI server revenue will experience an astonishing nearly fivefold leap, reaching around $50 billion in the global AI server market size by 2023. Subsequently, there will be a healthy and stable growth of around 20% in 2024, tapering off to 15% by 2027. According to this predictive model, AI server sales will surpass traditional server sales by 2027.

    Currently, AI applications are experiencing explosive growth. With the increase in Nvidia GPU supply and price reductions, as well as the entry of GPUs from other brands and other types of AI accelerators gaining traction in the market, everything is expected to stabilize and normalize, possibly reaching an entirely new level.

    One question remains: how much supply does the world’s AI need? Predicting the situation four or five years down the line is indeed very challenging. If the demand for AI accelerators continues to outpace supply, and prices remain high, revenue will stay elevated. If production doubles or triples and prices decrease by half or two-thirds, then revenue might stay unchanged.

    Clearly, no manufacturer would want to lower profit margins through excessive sales, but intense competition might force companies into that position.

    From 1985 to 2000, it took fifteen years for RISC/Unix machines and internet technology to capture a 45% share of server market revenue through active upgrades in mainframes and proprietary small-scale machines. It might take a similar fifteen-year span — from 2010 to 2025, or from 2011 to 2026 — for AI servers to account for around 45% of global server revenue, with AI workloads replacing or expanding into various applications you can imagine.

    18Dec, 2023
    High-Speed Optical Module Demand Soars: AI Computing and Market Projections Drive Innovations

    Discovering the intersection of AI computing and escalating market trends, the reliance on optical modules has surged. From high-scale computational scenarios in AI-powered systems to market forecasts propelling technological advancements, the landscape is evolving. Let’s delve into the interplay of AI, market projections, and groundbreaking optical innovations, reshaping the future of computational networking

    Section 1, AI Computing Scenario Optical Module Application

    In the usage of NVIDIA’s A100 systems, such as within the NVIDIA DGX SuperPOD, concerning the application of optical modules required for computation and storage sides, all cables for both computation and storage sides utilize NVIDIA MF1S00-HxxxE series AOC active optical cables. Consequently, each port requires an optical module, meaning each cable corresponds to two optical modules. Therefore, the computation and storage sides total required amounts to (1120+1124+1120)*2 + (280+92+288)*2 = 8048 optical modules. In other words, the approximate quantity of 200G optical modules needed per GPU is about 1:7.2.

    DGX GH200 supercomputer, equipped with 256 NVIDIA GH200 Grace Hopper super chips, each of which can be regarded as a server, interconnected through NVLINK SWITCH. Structurally, the DGX GH200 adopts a two-tiered fat tree topology, with the first and second tiers employing 96 and 36 switches, respectively. Each NVLINK SWITCH has 32 ports operating at a speed of 800G. Additionally, the DGX GH200 is equipped with 24 NVIDIA Quantum-2 QM9700 IB switches for the IB network.

    For estimation based on ports, given the closer proximity of L1 tier, it’s assumed that copper cables are used for connectivity without involving optical modules. For the L2 tier with 36 switches in a non-converging fat tree architecture, each switch’s ports connect downwards to the uplink ports of L1 tier switches. Therefore, a total of 36 * 32 * 2 = 1152 units of 800G optical modules are needed. In the IB network architecture, the 24 switches require 24 * 32 = 768 units of 800G optical modules. Thus, the DGX GH200 supercomputer requires a total of 1152 + 768 = 1920 units of 800G optical modules, corresponding to 12 units of 800G optical modules per GH200 chip.

    Section 2, Optical Module Market Demand Forecast Driven by Computational Networking

    The significant surge in computing power demand driven by new-generation information technologies like cloud computing, artificial intelligence, and big data has hastened the development of cloud computing infrastructure. Concurrently, with the comprehensive launch of projects such as the “Eastern Data and Western Computing,” China’s computational infrastructure construction is rapidly advancing. Optical modules, as pivotal components in cloud computing data centers, will experience a continuous surge in market demand due to the substantial increase in data transmission volume.

    According to LightCounting’s projections, the global adoption rate of 800G technology is expected to be merely 0.62% by 2023. Large AI models like ChatGPT impose new demands on data flows within and outside data centers, potentially accelerating the transition of optical modules towards 800G. It’s anticipated that 800G optical modules will start dominating the market by the end of 2025.

    Based on LightCounting’s data, the global market size for optical modules increased from $58.6 billion in 2016 to $66.7 billion in 2020. Projections suggest that by 2025, the global optical module market will reach $11.3 billion, 1.7 times the size in 2020. Structurally, the data communication market is expected to hold around 60%, while the telecommunications market will encompass approximately 40% of the market share.

    GIGALIGHT Optical Modules for Computational Applications

    The core technology of GIGALIGHT’s silicon optical modules lies in its innovative packaging design with high-coupling efficiency in the free-space COB and the MZI software locking algorithm. Additionally, in terms of silicon waveguide cores, the company has engaged in collaborative design work for multiple silicon chip designs with partners.

    23Sep, 2023
    Silicon Photonics 400G DR4 Optical Modules : Paving the Way for 400G Transmission

    The continuous growth of data centers and the demand for higher bandwidth and lower power consumption are driving constant innovations in optical communication technology. In this context, 400G DR4 silicon photonics products have emerged as a standout, thanks to their outstanding performance and groundbreaking technologies. Let’s delve into the overall characteristics of silicon photonics technology, its advantages in data center applications, and the key technological breakthroughs behind this achievement.

    Characteristics of 400G DR4 Silicon Photonics Products

    1. High Speed
    400G DR4 silicon photonics products stand out with their incredibly high single-port transmission bandwidth. This high-speed capability offers greater flexibility and performance for data center applications, ensuring fast data transmission and processing.

    2. Small Form Factor
    Complementing their high-speed capabilities, 400G DR4 silicon photonics products come in compact form factors. They boast high panel density, providing more space for data center architecture, while accommodating the growing data demands with larger overall capacity.

    3. Low Power Consumption
    Despite their exceptional performance, 400G DR4 silicon photonics products prioritize energy efficiency. Their low-power design contributes to energy savings and reduces cooling requirements, a critical aspect for large data centers that helps lower operational costs.

    Advantages of Matching 400G QSFP-DD DR4 Silicon Photonics Modules with 32-Port High-Capacity Switches

    The perfect compatibility between 400G QSFP-DD DR4 silicon photonics modules and 32-port high-capacity switches brings a range of advantages to data centers:

    1. Smallest Form Factor
    With QSFP-DD packaging compliant with MSA standards, 400G QSFP-DD DR4 silicon photonics modules are currently the smallest in size among 400G optical modules. This provides 1U-high switches with exceptional port density, offering a total capacity of up to 1.28TBPS.

    2. Silicon Photonics Integration Technology
    These modules feature integrated components such as 7nm DSP, CW light sources, MZ modulators, silicon waveguides, detectors, and more. This internal integration significantly reduces chip size and lowers power consumption. It enhances performance while reducing energy consumption.

    3. 100G PAM4 Technology
    400G QSFP-DD DR4 silicon photonics modules adopt 100G PAM4 technology, including four parallel channels with a total data rate of up to 425Gbps, four times that of 100G optical modules. This delivers exceptional bandwidth performance, meeting the demands of high-speed data transmission.

    Key Technological Breakthroughs in 400G QSFP-DD DR4 Silicon Photonics Modules

    The success of 400G QSFP-DD DR4 silicon photonics modules relies on key technological breakthroughs, with one of the most significant being:

    Silicon Lens Technology
    Efficient coupling of silicon photonics is achieved through silicon lens technology. Under three temperatures, the response rate reaches as high as 0.8A/W, overcoming adverse effects like crosstalk. This technology enables perfect commercial production, ensuring customers have easy access to the superior performance of silicon photonics at 400G.

    P1. Silicon Photonics Optical Module in Manufacture
    P2. Silicon Photonics Optical Module in Manufacture

    In conclusion, silicon photonics technology is paving the way for the future of data centers. With its high speed, small form factor, low power consumption, and compatibility with high-capacity switches, it has become a vital innovation in the field of optical communication. As data demands continue to grow, silicon photonics technology will evolve and strengthen, supporting high-performance and sustainable development in data centers. For more information on silicon photonics at 400G, please feel free to contact Gigalight.

    30Aug, 2023
    Development Trends and Analysis of Liquid Cooling Technology in Data Centers — Chapter 3. Trends and Opportunities

    Review from our first chapter, our discussion commences with the question, “Why Liquid Cooling?”This section covered various aspects, including the differentiation between cooling solutions like air and liquid cooling, and their applications in data centers. Moving forward, we examined the factors contributing to the explosive growth of liquid cooling technology. We delved into the challenges and considerations associated with its implementation, exploring elements such as cost, compatibility, and adherence to environmental regulations. The narrative also highlighted the potential benefits of liquid cooling in enhancing IT output and driving business growth.

    In the second chapter, we explored the intricacies linked to large-scale adoption. The discourse underscored the significance of safety compliance, operational challenges, and standardization. We also touched upon the potential shift toward single-phase immersion cooling in the medium term. The progress made by telecommunication operators and internet giants in adopting these technologies was noted. Lastly, we discussed the prerequisites for widespread liquid cooling adoption, particularly its rising prominence in various industries, especially in the realm of AI.

    Now, in the third chapter. let’s look ahead to the data center liquid cooling technology opportunities.

    The rapid advancement of liquid cooling technology using cold plate solutions is driven by its robust technical compatibility and seamless integration into existing data center infrastructure, especially for completed data center projects.

    However, looking ahead to the medium term, single-phase immersion cooling technology is poised to become mainstream. This projection is based on several factors, including its superior heat dissipation capabilities and simplified structure. If issues like fire safety and flammability can be addressed for oil-based cooling fluids, their widespread adoption becomes viable.

    Currently, telecommunication operators are predominantly promoting both cold plate and single-phase immersion cooling techniques. Conversely, internet giants might opt for divergent paths based on their individual stances. Some may even choose a combination of both, while others might skip certain intermediate stages.

    For data center manufacturers, my personal recommendation is to establish a foundational cooling capability, whether cold plate or immersion. This encompasses essential infrastructure factors such as load-bearing capacity and rack height to cater to customer preferences. Furthermore, offering integrated, readily deployable cold plate solutions in the short term is advised to meet market demands.

    Looking ahead, it’s evident that AI training is a pivotal domain where large-scale liquid cooling adoption is being explored. Additionally, from training to inference stages, powerful computing capabilities are required for inference servers, necessitating the incorporation of liquid cooling technology. Ultimately, liquid cooling technology is poised to permeate the data center industry.

    However, the widespread application of liquid cooling technology necessitates several prerequisites. Firstly, the overall cost of liquid cooling systems must be reduced to a level where users perceive liquid cooling to be more cost-effective than air cooling, without requiring special configurations for the latter. Secondly, a comprehensive ecosystem of liquid cooling technology must be established, offering a range of supporting equipment and solutions. It’s only under these conditions that users will universally opt for liquid cooling technology.

    In the context of telecommunication operators, I believe substantial progress has already been made in the realm of liquid cooling technology. As per plans, by 2025 and beyond, over 50% of projects are projected to adopt scaled liquid cooling technology applications — a notably ambitious target.

    Regarding internet giants and other enterprises, I am confident that as the industry matures, they will gradually embrace liquid cooling technology. Whether embarking from the AI sector or comprehensively adopting liquid cooling technology across various domains, these entities will progressively venture into this landscape.

    10Aug, 2023
    Development Trends and Analysis of Liquid Cooling Technology in Data Centers — First Chapter

    As energy scarcity and environmental protection become increasingly prominent issues, liquid cooling technology, as an efficient and energy-saving solution, is continuously adopted by data centers to achieve sustainable development. In this first chapter, we explore the background and advantages of liquid cooling technology, as well as the reasons why liquid cooling technology is widely applied in the current era.

    Table of Contents
    Part 1/ Why Do We Need Liquid Cooling?
    – Heat dissipation is crucial for data center stability.
    – Liquid cooling technology serves as an external supplementary heat dissipation method.
    – Liquid cooling technology involves transferring the heat generated by IT equipment to an external cold source circulation system.

    Part 2/ Why Has Liquid Cooling Become a Explosive Growth Point?
    – Technological bottlenecks, costs, and sustainability drive the demand for liquid cooling.
    – Liquid cooling technology addresses the heat dissipation challenges of high-power devices.
    – Liquid cooling technology enhances energy efficiency, reducing operational costs and meeting energy efficiency requirements.

    Drivers of Liquid Cooling Demand — Demand-Driven and Inflection Point — AI
    – The development of AI technology has led to the emergence of large-scale models.
    – Liquid cooling technology caters to the heat dissipation needs of high-power devices.
    – The development of liquid cooling technology aligns with energy scarcity and environmental protection demands.
    – AI’s application in training and inference stages necessitates substantial liquid cooling support.

    Part 1: Why Liquid Cooling?

    The reason humans are considered the “King of the creatures” is due to our powerful brains and robust muscles, allowing us to walk upright. What is often overlooked is that humans also possess the most potent heat dissipation capability among animals.

    Imagine, for instance, a cheetah that can run at speeds of 120 kilometers per hour but can only sustain this speed for 60 seconds. If the cheetah fails to catch its prey within these 60 seconds, it must abandon the chase. If it repeatedly sprints at such high speeds within a day without securing food, it might eventually die. How do humans manage this feat? Nowadays, many individuals participate in marathon races, with the faster ones completing the race in around two to three hours, and the slower ones perhaps taking six hours. However, they all manage to persist until the finish line. This endurance is possible due to humans possessing excellent heat dissipation capabilities, providing us with impressive stamina.

    In the context of data centers, heat dissipation is vital for the stability of the entire data center and its infrastructure. Simultaneously, we aim to ensure the reliable operation of the entire IT system using the least amount of electricity.

    Using a similar analogy, humans employ various techniques for heat dissipation. For example, we use our internal mechanisms such as breathing, skin, well-developed sweat glands, and blood circulation to dissipate heat. Apart from these intrinsic heat dissipation mechanisms, as the “crown of creation,” humans can also utilize external methods to aid heat dissipation. For instance, if I find the air conditioning provided at an event not sufficiently cool, I can take a shower to cool down through water spray, or I can choose to go swimming.

    As we all knows, swimming represents the ultimate form of heat dissipation. While discussing liquid cooling technology, many people refer to the portion that involves using liquid for heat dissipation as liquid cooling technology. However, genuine liquid cooling technology refers to a part of the data center’s internal circulation system, specifically how to transfer the heat generated by IT equipment to the internal circulation section of the external cold source loop.

    Key aspects related to liquid cooling technology, such as the heat dissipation per square centimeter, present significant challenges when discussing liquid cooling technology. Similarly, the human brain consumes approximately 20% of the body’s energy, about 24 watts. Although this might not seem like much, when the brain operates at high speeds, one might feel the head heating up, and the head is prone to sweating. Moreover, when the brain overheats, thinking speed decreases. This actually illustrates that the human brain, as the most critical component of the body, also possesses a critical role in maintaining the stable operation of the entire system.

    Part 2. Why Liquid Cooling Has Become an Explosive Growth Point

    Reasons Driving the Demand for Liquid Cooling — Technological Bottlenecks, Costs, Sustainability.

    The reasons behind the surge in demand for liquid cooling are multi-fold. Firstly, the development of IT technology has led to a stagnation in air cooling efficiency. Moreover, in the case of CPUs and GPUs, the power consumption of CPUs has escalated from 205 watts in 2017 to 350 watts today, with a rapid growth rate. It’s even possible for future power levels to reach unimaginable numbers like 500 or 600 watts.

    Furthermore, the latest GPU version, H100, consumes up to 700 watts. In many scenarios, air cooling is insufficient to maintain chip stability. In addition, NVIDIA’s test results have demonstrated that under similar performance conditions, liquid cooling can reduce energy consumption by 30%.

    But what does a 30% energy savings really mean? Primarily, for the entire data center, a substantial reduction in energy consumption is crucial. This meets PUE requirements and lowers energy benchmarks. For businesses, it signifies the potential for more robust computing capabilities within limited resources. Approaching the critical threshold of 20%-30%, many factors become pivotal in determining the sustainability of operations for enterprises.

    Furthermore, liquid cooling technology boasts energy efficiency advantages. Energy efficiency translates to lower operational costs and cost savings when meeting energy benchmarks. It also implies more effective utilization of power metrics in the realm of power supply.

    At present, PUE (Power Usage Effectiveness) guidance in European and American countries can vary due to diverse standards, organizations, and regions. For instance:

    The Green Grid: A nonprofit organization dedicated to advancing data center energy efficiency, The Green Grid commonly proposes a PUE requirement of below 1.5. This standard is widely recognized and applicable across many countries.

    U.S. Department of Energy: Providing PUE recommendations, the U.S. Department of Energy typically suggests a PUE range from 1.2 to 1.5. The specific range depends on the scale and nature of the respective data center.

    European Code of Conduct for Data Centres: An initiative spanning Europe, this code encourages energy efficiency in data centers. The code recommends a PUE below 2.0, while also promoting the use of additional metrics for a comprehensive assessment of energy efficiency.

    ASHRAE: The American Society of Heating, Refrigerating and Air-Conditioning Engineers offers multiple standards for data center energy efficiency, including ASHRAE 90.4–2019. According to this standard, efficient data centers should maintain a PUE of 1.3 or lower.

    These requirements not only indicate that liquid cooling technology is suitable for these scenarios but also drive its rapid advancement.

    Reasons Driving Liquid Cooling Demand — Demand-Driven and Inflection Point — AI

    Since the end of last year, ChatGPT has become a highly anticipated technological innovation, sparking widespread attention and discussions. The emergence of ChatGPT signifies a significant breakthrough in the field of artificial intelligence. Various representative models have been introduced, each aiming to lead the era, fostering optimistic expectations within the industry. To a certain extent, the First Industrial Revolution transitioned from steam engines to internal combustion engines to electrification. But what does Industry 4.0 represent? People have been contemplating what could represent Industry 4.0 or whether there are significant events in the new information revolution. In my opinion, today’s representation, exemplified by large models like ChatGPT, might truly usher us into the age of intelligence, signifying the onset of the next industrial revolution.

    There are numerous viewpoints within the industry regarding this. For instance, it’s crucial to keep up with the trends of the AI era. As NVIDIA suggests, we are currently in the AI’s “iPhone era.” Li Kaifu mentioned that all applications will be restructured through AI 2.0, which refers to the current large models, AIGC models. This includes applications like DingTalk, where AI will likely support all business capabilities. AI can perform many tasks such as generating speech, singing, coding, aiding in creating presentations, and more. However, to perform these tasks effectively, we need substantial computational power. Robust computational power is pivotal for ensuring stability and efficiency, and data centers play a vital role in providing the infrastructure for this.

    The chart in the upper right corner displays NVIDIA’s own model, demonstrating that only 48 GPU servers are needed to complete tasks that previously required multiple CPUs. However, the application of this technology has brought about some issues. The power consumption of a single server has reached approximately 8 kilowatts, and in many Chinese data centers, air cooling methods cannot satisfy such high-density cooling demands. You might wonder, why not decrease server density and place only one server per rack? While this might be feasible, it would trigger other issues, such as a significant increase in network investments. You might not have noticed yet, but alongside the AI trend, a previously less-understood industry has also been ignited — the optoelectronic module and optical communication industry. Stocks in these industries might experience higher surges than those of model development companies or NVIDIA. This is because, in many scenarios, cabinets previously connected by copper cables are now connected by optical fibers, greatly increasing the demand for optoelectronic modules.

    In conclusion, with the advent of new-generation AI technologies like ChatGPT, the development of liquid cooling technology is opportune and timely. Moreover, AI is currently primarily used in the research and development of training large models, but it will gradually move into the phase of large-scale application.

    Previously, in the traditional sense, it was believed that AI training could be accomplished using a single device equipped with 8 high-performance GPUs, utilizing liquid cooling technology to address the issues. However, only a few participants could afford the cost of such devices in this scenario. Additionally, to train a large model, a considerable number of devices are needed, but the total number of clusters that can be built is limited.

    However, if all applications require AI for inference and application, every server will become an AI server equipped with GPUs. In situations where there were no large models before, only inserting a small GPU like the A10 card might be necessary for inference, and air cooling would suffice. Yet, with the presence of large models today, you must use 8 A100 cards, necessitating liquid cooling technology.

    The timing is right. In fact, the entire liquid cooling industry has undergone a decade of development — starting from the initial GRC, the first solution dedicated to solving data center liquid cooling issues, has been around for a decade. Of course, it’s still an era of diverse technological advancements.