18Dec, 2023
    High-Speed Optical Module Demand Soars: AI Computing and Market Projections Drive Innovations

    Discovering the intersection of AI computing and escalating market trends, the reliance on optical modules has surged. From high-scale computational scenarios in AI-powered systems to market forecasts propelling technological advancements, the landscape is evolving. Let’s delve into the interplay of AI, market projections, and groundbreaking optical innovations, reshaping the future of computational networking

    Section 1, AI Computing Scenario Optical Module Application

    In the usage of NVIDIA’s A100 systems, such as within the NVIDIA DGX SuperPOD, concerning the application of optical modules required for computation and storage sides, all cables for both computation and storage sides utilize NVIDIA MF1S00-HxxxE series AOC active optical cables. Consequently, each port requires an optical module, meaning each cable corresponds to two optical modules. Therefore, the computation and storage sides total required amounts to (1120+1124+1120)*2 + (280+92+288)*2 = 8048 optical modules. In other words, the approximate quantity of 200G optical modules needed per GPU is about 1:7.2.

    DGX GH200 supercomputer, equipped with 256 NVIDIA GH200 Grace Hopper super chips, each of which can be regarded as a server, interconnected through NVLINK SWITCH. Structurally, the DGX GH200 adopts a two-tiered fat tree topology, with the first and second tiers employing 96 and 36 switches, respectively. Each NVLINK SWITCH has 32 ports operating at a speed of 800G. Additionally, the DGX GH200 is equipped with 24 NVIDIA Quantum-2 QM9700 IB switches for the IB network.

    For estimation based on ports, given the closer proximity of L1 tier, it’s assumed that copper cables are used for connectivity without involving optical modules. For the L2 tier with 36 switches in a non-converging fat tree architecture, each switch’s ports connect downwards to the uplink ports of L1 tier switches. Therefore, a total of 36 * 32 * 2 = 1152 units of 800G optical modules are needed. In the IB network architecture, the 24 switches require 24 * 32 = 768 units of 800G optical modules. Thus, the DGX GH200 supercomputer requires a total of 1152 + 768 = 1920 units of 800G optical modules, corresponding to 12 units of 800G optical modules per GH200 chip.

    Section 2, Optical Module Market Demand Forecast Driven by Computational Networking

    The significant surge in computing power demand driven by new-generation information technologies like cloud computing, artificial intelligence, and big data has hastened the development of cloud computing infrastructure. Concurrently, with the comprehensive launch of projects such as the “Eastern Data and Western Computing,” China’s computational infrastructure construction is rapidly advancing. Optical modules, as pivotal components in cloud computing data centers, will experience a continuous surge in market demand due to the substantial increase in data transmission volume.

    According to LightCounting’s projections, the global adoption rate of 800G technology is expected to be merely 0.62% by 2023. Large AI models like ChatGPT impose new demands on data flows within and outside data centers, potentially accelerating the transition of optical modules towards 800G. It’s anticipated that 800G optical modules will start dominating the market by the end of 2025.

    Based on LightCounting’s data, the global market size for optical modules increased from $58.6 billion in 2016 to $66.7 billion in 2020. Projections suggest that by 2025, the global optical module market will reach $11.3 billion, 1.7 times the size in 2020. Structurally, the data communication market is expected to hold around 60%, while the telecommunications market will encompass approximately 40% of the market share.

    GIGALIGHT Optical Modules for Computational Applications

    The core technology of GIGALIGHT’s silicon optical modules lies in its innovative packaging design with high-coupling efficiency in the free-space COB and the MZI software locking algorithm. Additionally, in terms of silicon waveguide cores, the company has engaged in collaborative design work for multiple silicon chip designs with partners.

    1Nov, 2023
    Emerging Trends in 400G and 800G: Low-Power Solutions for High-Speed Networks

    As Serdes technology drives data centers into the era of 400G and 800G, port power consumption has become a hot topic in the industry. Low-power 400G/800G interconnection solutions have been successively introduced, attracting wide attention in the industry and being widely regarded as a key technology for AI and machine learning in smart data centers.

    1. CPO Aims to Address Next-Generation Bandwidth and Power Challenges

    With the continuous acceleration of bandwidth in network and computing structures, innovation is required in system and chip architecture to mitigate the slowdown of Moore’s Law. Meanwhile, copper interconnects are rapidly reaching their bandwidth distance limits. Silicon photonics is crucial for sustaining rapid data growth and high-bandwidth applications. Co-Packaged Optics (CPO) involves co-packaging switch chip ASICs and optoelectronic engines (optical transceivers) on the same substrate, with the optical engines placed as close to the ASIC as possible to minimize high-speed electrical channel losses and impedance discontinuities. This enables the use of faster, lower-power off-chip I/O drivers. Using CPO not only achieves networking but also facilitates GPU-to-GPU interconnects, resource pooling, and memory disaggregation. It meets the requirements of AI/ML training clusters with high bandwidth and radix connections, the lowest per-bit cost, and the highest power efficiency.

    2. Linear Direct-Drive Pluggable Modules Also Reduce Power Consumption

    In the era of 400G and 800G, in addition to pluggable optical modules and CPO solutions, Linear Direct Drive (also known as linear drive) pluggable 400G/800G optical modules became a research hotspot at OFC in March of this year. The biggest advantage of this optical module solution is that it eliminates the need for DSP chips, significantly reducing the power consumption and latency of signal processing at the module level.

    GPU servers serving applications such as AI and machine learning provide excellent computing power but also result in increased server power consumption. The high-speed interconnection of 400G/800G leads to a corresponding increase in the power consumption of optical modules and network equipment. Both CPO and linear direct-drive pluggable modules may be interconnection solutions for future smart data centers, providing the lowest system-level power by eliminating all possible active components from the interconnect.

    14Aug, 2023
    Development Trends and Analysis of Liquid Cooling Technology in Data Centers — Second Chapter

    In the latter part, let’s explore the opportunities and challenges.

    Now, let’s delve into the distinction between system solutions and IT solutions. Similar to the human body, various means can be employed to ensure comfort, such as using fans, turning on air conditioning, or even swimming or taking showers. As long as a method can address our comfort needs, it qualifies as an effective solution.

    However, the scenario is different for data centers. If I opt for immersive cooling, whether single-phase or two-phase, or choose spray cooling, I must consider the compatibility of every IT component and the overall operation of the cooling system. Moreover, the design of the complete solution also needs to be taken into account. This introduces the concept of system solutions. In cases of localized improvements, like focusing solely on cooling CPUs or GPUs, only IT solutions might need to be considered, and this is distinct.

    When an enterprise genuinely aims to engage in artificial intelligence projects in the current era, the factors to contemplate in deciding whether to adopt liquid cooling technology are still quite numerous. This encompasses costs, including single kW construction expenses, total cost of ownership (TCO), and various factors tied to IT output. Beyond these aspects, those illustrated on the left side, these are the driving forces behind deploying liquid cooling technology, yielding benefits from them. For business development and innovation, as previously mentioned, if data center space and power resources are limited, employing liquid cooling can enhance IT output, thereby fostering favorable business growth.

    In fact, is liquid cooling necessary for cases where a PUE of 1.2 is required?

    And if I intend to build AI servers, is liquid cooling an absolute necessity? We are aware that NVIDIA’s provided solutions encompass more than just liquid cooling technology. Addressing these challenges is of utmost importance, including considerations of stability and cost factors.

    Speaking of costs, it’s important to note that data centers have a relatively long lifecycle while the lifespan of IT equipment is comparatively shorter. Making decisions like whether to adopt liquid cooling technology or choose between cold plates and immersion techniques during the mid-term, say after 3 or 4 years of data center usage, presents a complex dilemma. Furthermore, factors such as legacy applications and whether the existing servers can continue to function in the new environment must also be accounted for.

    Another significant concern is standardization. Currently, each company offers distinct solutions, from IT equipment to cooling distribution units (CDUs), extending to liquid cooling systems; each phase has its own standards. Opting for a particular solution, does it signify being locked into that company’s offerings or does it allow for interchangeability and interoperability across multiple solutions? This poses a substantial challenge for many users.

    As a user, lacking robust control capabilities may lead to more conservative technology choices. If selecting a particular technology results in being bound by it, relinquishing control could be a potential consequence. Unless complete control over the entire supply chain can be achieved, decisions are often limited by existing conditions.

    Furthermore, fire safety concerns need to be factored in. For instance, oil-based cooling fluids might pose flammability issues. Thus, using such cooling fluids in high-density data centers — does it comply with fire safety regulations? Can it be approved and recognized as a risk-free technology for secure usage? Given the current absence of comprehensive standards, experimentation might be daring, but consequences in case of issues would be borne by the user. Moreover, there are other ecological factors to consider, although specific details won’t be elaborated upon at this point.

    Cooling Technology Overview

    Apart from the single-phase and two-phase liquid cooling technologies mentioned earlier, there’s also a type known as full-coverage cold plate liquid cooling technology. Unlike traditional cold plate technology, full-coverage cold plate covers all components, not just core parts for cooling. This technology can be applied to all components, ensuring they all receive cooling.

    In the past, full-coverage cold plate technology faced significant challenges because many components are plug-and-play, such as memory and hard drives, making them less standardized and unable to be covered. However, in the current AI scenario, AI boards can be designed to be integrated, with all components attached to the mainboard, thereby achieving full-coverage cooling for them. An important recent trend is the significant price drop in SSDs (Solid-State Drives). Compared to the beginning of the year, SSD prices have decreased significantly while storage capacity has greatly increased. Hence, storage capacity is no longer a problem. In this context, the heat dissipation issue with HDDs (Hard Disk Drives) that previously hindered the application of this technology is no longer relevant.

    Regarding immersion liquid cooling technology, I believe that in the long run, single-phase immersion cooling is a superior data center solution.

    However, there are still many challenges that need to be overcome with immersion liquid cooling. This primarily includes the following aspects:

    • Is immersion liquid cooling suitable for all scenarios?
    • How compatible is immersion liquid cooling with IT equipment? Could it lead to damage to IT equipment? Further research is needed on material compatibility.
    • Can traditional air-cooled equipment be directly immersed? More validation is required.
    • What are the technical standards for cooling liquids? Safety (fire, human safety), compatibility (electronic, electrical), heat dissipation performance, GWP/ODP, etc.
    • How to choose between single-phase and two-phase?
    • The choice between fluorinated liquids and oil-based cooling liquids? Fluorinated liquids are expensive and volatile, with complex systems, but perform well in server compatibility. On the other hand, oil-based cooling liquids are cheaper and less volatile, with simpler systems. However, handling oil-based cooling liquids during component replacement can be challenging and may require additional equipment. Thus, from the perspective of choosing a cooling liquid, it’s difficult to determine which technology will ultimately win.
    • How to address operational challenges?
    8Aug, 2023
    How do Data Centers Support High-Performance Computing?

    Data centers have been in existence since the 1940s, with the first dedicated computer rooms used for military purposes. As computing and storage demands exponentially grew over the following decades, applications expanded into various domains of life, and organizations increasingly sought dedicated data centers to accommodate their infrastructure.

    To reduce costs and stay competitive, outsourcing data center infrastructure has become almost essential since the emergence of High-Performance Computing (HPC). HPC is a powerful solution that requires high density, heat, and bandwidth. These data centers need to address the challenges posed by the heat and power density required for running multiple high-performance computers simultaneously.

    Given that HPC enables faster integration of data analytics and artificial intelligence, it comes as no surprise that leading companies adopting HPC data centers are predominantly in the cloud computing and IT industries. However, companies from other industries can also harness the power of HPC.

    These may include:

    1. Research Laboratories
    2. Financial Technology
    3. Weather Forecasting
    4. Media and Entertainment
    5. Healthcare
    6. Government and Defense

    Data centers supporting HPC can meet the growing demand for fast networking while keeping up with the increasingly digitized landscape.

    The Three Key Systems of HPC To build an infrastructure that caters to HPC, it is crucial to understand the three key systems of an HPC cluster: computing, storage, and networking.

    Computing

    Efficient HPC systems require a set of computer services and software programs that work together to execute algorithmic programs. Each module needs to stay synchronized with the other modules in the cluster; otherwise, the entire HPC system becomes outdated.

    The goal of HPC is to perform high-speed computations, which requires aggregating computing capabilities from different hardware types. Data centers have the necessary space and capacity to accommodate the computing systems and hardware required to support HPC operations, which often involve power and cooling coordination beyond what most enterprises can handle.

    Storage

    To accommodate the massive amount of data processed by HPC, the storage system should offload data from the CPUs as frequently as possible without interrupting the computational operations. According to Weka, HPC storage systems need to meet the following requirements:

    • Data from any node should be available at all times.
    • Available data must be the most up-to-date.
    • Capable of handling data requests of any size.
    • Support performance-oriented protocols.
    • Utilize the latest storage technologies (e.g., SSD).
    • Scale to milliseconds to keep up with constant latency.

    Networking

    The network topology of HPC is significantly different from internal office networks. In addition to the extreme demands of continuous data transfer between CPUs and storage, many different computational components that make up the HPC environment are treated as one computer and are combined in a “fabric.” “The key concept of HPC fabric is having a large amount of scalable bandwidth (throughput) while maintaining ultra-low latency.”

    Cooling Facilities

    Considering the density and heat generated by HPC infrastructure, cooling can be a significant challenge. Traditional hot-aisle containment systems used in modern data centers can effectively cool today’s 50kW HPC racks. Looking ahead, HPC clusters may increase in density and drive more widespread adoption of liquid cooling, which offers cooling capacities up to 1,000 times greater than air cooling, with a smaller physical footprint. Immersion cooling deployments offer higher flexibility and cater to future customers.

    Take GIGALIGHT’s innovative liquid-cooled optical interconnect modules and solutions as an example. The liquid-cooled high-speed modules can stably operate in fluorocarbon liquids and mineral oil up to 1 meter deep (certified through long-term customer validation). Compared to traditional cooling solutions, they provide higher heat dissipation efficiency and lower energy consumption, taking the computational power of high-performance computing to a new level.

    High-Performance Components

    High-performance computing data centers are built with high-quality systems, components, and facilities to meet the demands of HPC. These data centers provide affordable power, network, scalability, redundancy, and security required for HPC. Therefore, HPC clusters require high-performance parallel interconnect components to connect the devices.

    GIGALIGHT has been a pioneer in parallel optical interconnect computing since 2013. We have been dedicated to the development of high-performance parallel optical modules and interconnect cables. Our product series covers various speeds, including 10G, 25G, 40G, 100G, 200G, 400G, and 800G, and support the INFNIBAND protocol.

    Here are some key components that support parallel interconnects in HPC data centers:

    – Server optical network cards based on Intel and NVIDIA chip designs, supporting parallel interconnects from 10G to 200G and extending towards 400G/800G.

    – High-speed parallel optical modules designed based on VCSEL lasers, DML lasers, or silicon photonics platforms, such as 100G QSFP28 SR4/PSM4, 200G QSFP56 SR4/DR4, 200G QSFP-DD SR8/PSM8, and 400G QSFP-DD SR8/DR4.

    – Short-reach parallel DAC (Direct Attach Copper) and AOC (Active Optical Cable) interconnect cables designed with low power consumption, including 400G QSFP-DD DAC/AOC, 800G QSFP-DD DAC/AOC, etc.

    – Electric loopback modules that support self-loopback testing of system devices.

    – Innovative liquid-cooled interconnect optical modules and solutions.

    These high-quality components, along with the overall system and facilities, work together to create high-performance computing data centers that meet the requirements of HPC.

    3Mar, 2023
    Meet GIGALIGHT at Data Centre World 2023 in London for HPC & Liquid Cooling Interconnection Hardware Solutions
    Shenzhen, China, Mar 3, 2023 – GIGALIGHT announced to showcase its portfolio of high-performance computing and immersion liquid cooling interconnect hardware at Data Centre World 2023 in London, looking forward to meeting you on March 8th/9th at booth #D780.
    In order to cater to the development trend of high computing power in data centers, GIGALIGHT has launched a series of interconnection hardware products with high speed, low power consumption and high reliability to reduce the cost pressure brought by broadband growth and energy consumption for customers. Follow the guide to know about our exhibiting products in advance.

    Immersion Liquid Cooling Family

    • 25G SFP28 SR/LR, 100G QSFP28 SR4/PSM4/CWDM4 and 200G QSFP56 SR4/FR4 immersion liquid cooling transceivers
    • 25G SFP28, 100G QSFP28 and 200G QSFP56 immersion liquid cooling DAC cables
    • 40G QSFP+, 100G QSFP28 and 200G QSFP56 immersion liquid cooling extenders

    Network Adapters & Transceivers for High-Performance Servers

    • Dual-port 25G Ethernet network adapter
    • Single-port 40G Ethernet network adapter
    • Dual-port 100G Ethernet network adapter

    Low Power Interconnection and I/O Testing Solutions

    • 100G QSFP28, 200G QSFP56, 400G QSFP-DD, 400G QSFP112 and 800G QSFP-DD DAC high-speed cables
    • 100G QSFP28, 200G QSFP56 and 400G QSFP-DD AOC high-speed cables
    • 25G SFP28, 100G QSFP28, 200G QSFP56 and 200G QSFP-DD loopback modules
    As the open optical network device explorer, GIGALIGHT is one of the earliest companies in China to deploy optical interconnection solutions for data centers, and has a rich product line with the most differentiated technology in the industry. We are looking forward to discovering the latest trends and solutions with industry colleagues at this exhibition.

    About GIGALIGHT

    As the open optical network device explorer, GIGALIGHT integrates the design, manufacture and sales of active & passive optical devices and subsystems. The main products are optical transceivers, silicon photonics transceivers, liquid cooling transceivers, optical passive components, AOC & DAC, coherent optical modules and open DCI BOX subsystems. GIGALIGHT is a hardware solution provider of innovatively designed high-speed optical interconnection that focuses on data center, 5G carrier network, metro WDM transmission, UHD broadcast video and other application fields.
    26Nov, 2018
    Why Most HPC Systems Use InfiniBand Interconnection

    In addition to the well-known Ethernet, there are many other categories of network architecture. For server-side connection scenarios, InfiniBand (IB) technology is valued and used for its inherent characteristics. It has a nearly dominant position, especially in High-Performance Computing (HPC), large data center storage and other scenarios. So what’s the difference between IB and Ethernet? Why most HPC systems use IB interconnection?

    What InfiniBand Is and Where It Is Used

    IB is a “cable switching” technology that supports multiple concurrent connections, and it is the I/O standard of the new generation server hardware platform. With the rapid development of CPU performance, the performance of I/O system has become a bottleneck restricting the performance of servers. The PCI bus architecture used in the past does not conform to the new application trend. To overcome the inherent drawbacks of PCI, Intel, Cisco, Compaq, EMC, Fujitsu and other companies jointly launched the IB architecture, the core of which is to separate the I/O system from the server host. At present, only a few companies, such as Mellanox, Intel, Qlogic, can provide IB products. Mellanox is in the leading position. Recently, they deployed the first HDR 200G InfiniBand supercomputer at the University of Michigan.

    The basic protocols of InfiniBand

    The basic protocols of InfiniBand

    The figure above shows the IB’s basic protocols. As we can see, the IB protocols adopt a hierarchical structure, including upper protocol, transport layer, network layer, link layer and physical layer. Each layer is independent of each other, and the lower layer provides services for the upper layer, which is similar to TCP/IP protocol. Unlike Ethernet, which is used in high-level network communication, InfiniBand is mainly used in low-level input/output communication scenarios. As mentioned at the beginning of this article, IB architecture has the mission of improving server-side input/output performance, even if Ethernet achieves or exceeds the speed of IB network, IB is irreplaceable under the condition of low-level network communication. In addition, IB’s transmission mode and media are quite flexible. It can be transferred by copper wire foil of printed circuit board in the equipment, and interconnected by DAC or AOC between the equipment.

    As Bill Lee, co-chair of the InfiniBand Industry Association Working Group, said, “The goal of InfiniBand is to improve communication between applications.” IB technology includes not only chips and hardware, but also software. In order to play its due role, hardware and software must be fully integrated in the operating system, management and application layer.

    Why HPC Data Centers Choose InfiniBand

    Addison Snell, CEO of Intersect360 Research, pointed out that “InfiniBand has grown and is now the preferred solution for high performance storage interconnection in HPC systems. At present, the applications of high data throughput such as data analysis and machine learning are expanding rapidly, and the demand for high bandwidth and low delay interconnection is also expanding to a broader market.”

    Obviously, our main direction at present and in the future is to solve the problems of science and data analysis, which requires a very high bandwidth between computing nodes, storage and analysis systems in our data center, thus forming a single system environment. In addition, latency (memory and disk access latency) is another performance measure of HPC. So, the reason why HPC data centers choose to use IB network is that it can meet the requirements of high bandwidth and low latency.

    IB is currently the preferred interconnection between HPC and AI infrastructures, and the speed is also increasing, from SDR, DDR, QDR, to HDR. The famous Mellanox InfiniBand solution connects most of the Top 500 supercomputers, and they will also start planning NDR 400G InfiniBand technology to support future E-level supercomputing and machine learning platforms. In terms of delay, RDMA (Remote Direct Memory Access) technology allows us to access data directly and remotely throughout the network, and can solve the problem of server-side data processing delay in network transmission. RDMA transfers data directly to the computer’s storage area through the network, moves data from one system to remote system memory quickly, realizes Zero Copy, releases CPU load on the host side, and reduces the delay of data processing in the host from hundreds of microseconds to nanoseconds.

    In addition, IB has the advantages of simple protocol stack, high processing efficiency and simple management. Unlike the hierarchical topology of Ethernet, InfiniBand is a flat structure, which means that each node has a direct connection to all other nodes. Compared with TCP/IP network protocol, IB uses trust-based and flow-control mechanism to ensure the integrity of the connection, and data packets are rarely lost. After data transmission, the receiver returns a signal to indicate the availability of the buffer space. Therefore, IB protocol eliminates the delay of retransmitting due to the loss of original data packets, thereby improving the performance of the protocol. Efficiency and overall performance are improved. In addition, considering the problem of signal distortion in ultra-high-speed optical fiber transmission, IB transmits data signals differently in the optical fiber, and adds a filter at the receiving end to filter out the signal noise, which fully guarantees the signal integrity of the connection network.

    Conclusion

    InfiniBand is a unified interconnection structure that can handle both storage I/O, network I/O and interprocess communication (IPC). It can interconnect disk arrays, SANs, LANs, servers and cluster servers, provide high bandwidth and low latency transmission over relatively short distances, and support redundant I/O channels in single or multiple Internet networks, so that data centers can still operate when local failures occur. Especially in the future, when the internal traffic of HPC data center increases dramatically, InfiniBand will have a broader development space as a network connection technology applied between servers.