31Oct, 2023
    Battle of the Data Center Giants: InfiniBand vs. Ethernet — Exploring the Key Differences

    As data centers become the backbone of our digital world, the choice of networking technology becomes more crucial than ever. In the realm of data transmission, two heavyweights, InfiniBand and Ethernet, vie for supremacy. Let’s embark on a journey to unravel the distinctions between these titans, from design goals to reliability and cost-effectiveness, and understand the critical factors influencing the data center landscape.

    Design Objectives

    InfiniBand, originally coined as “Infinite Bandwidth,” was purpose-built to connect servers, storage, and network devices, catering to the specific requirements of data centers. It leverages high-speed differential signaling and multi-channel parallel transmission mechanisms, with its primary objectives encompassing high performance, low latency, and unwavering reliability.

    Conversely, Ethernet serves as a universal protocol, extensively utilized in local area networks (LANs) and wide area networks (WANs) for communication. It employs the Carrier Sense Multiple Access with Collision Detection (CSMA/CD) mechanism to avert collisions, with its core goals centered on simplicity, flexibility, and scalability.

    Transmission Speeds and Latency

    InfiniBand supports an array of transmission modes, including SDR (Single Data Rate), DDR (Double Data Rate), QDR (Quad Data Rate), FDR (Fourteen Data Rate), EDR (Enhanced Data Rate), HDR (Hundred Gigabit Data Rate), and NDR (Next Data Rate). Each mode exhibits unique characteristics and advantages tailored for diverse application scenarios.

    Ethernet equally boasts support for a range of transmission rates, such as 10Mbps, 100Mbps, 1Gbps, 10Gbps, 25Gbps, 100Gbps, 200Gbps, 400Gbps, and beyond. Nevertheless, Ethernet’s latency is primarily contingent upon the performance of network devices and network traffic.

    Due to InfiniBand’s utilization of higher transmission rates and specialized hardware, it claims a distinct advantage in terms of latency. Conversely, Ethernet tends to exhibit higher latency levels, largely attributable to the intricacies of its protocol stack and additional processing stages.

    Reliability and Scalability

    InfiniBand is characterized by high reliability and low power consumption, aligning seamlessly with the performance and reliability requisites of contemporary data centers. It deploys high-speed differential signaling and multi-channel parallel transmission mechanisms, facilitating exceptionally high data transmission speeds and minimal latency. Furthermore, InfiniBand is distinguished by its attributes of high reliability and low power consumption, further tailoring itself to the modern data center’s performance and reliability demands.

    Ethernet, a universal protocol, serves as the linchpin in local area network (LAN) and wide area network (WAN) communications. It relies on the Carrier Sense Multiple Access with Collision Detection (CSMA/CD) mechanism to prevent collisions, with its core objective centered on simplicity, flexibility, and scalability.

    Cost and Maintenance

    InfiniBand is a specialized protocol necessitating dedicated hardware and software components. Consequently, it tends to be relatively more costly. Nevertheless, its design, tailored for the data center environment, imposes fewer environmental constraints and lower maintenance costs.

    In contrast, Ethernet is a general-purpose protocol compatible with standard Ethernet switches and network interface cards (NICs). As a result, Ethernet is characterized by its cost-effectiveness. However, the intricacies of its design and configuration contribute to higher maintenance expenses.

    Applications

    In light of their distinctive attributes, InfiniBand predominantly finds utility in the high-performance computing sector, including supercomputers, data centers, and scientific research institutions. Its strengths shine in high-speed data transmission, low latency, and scalability.

    Ethernet is widely applied in enterprise networks, internet access, home networks, and various other domains. Its key strengths encompass cost-effectiveness, standardization, and widespread support.

    In summary, InfiniBand and Ethernet represent two divergent data center network communication technologies, each marked by differences in design objectives, transmission rates and latency, reliability and scalability, cost and maintenance, and application areas. Real-world applications should guide the selection of the appropriate protocol based on specific demands.

    As technology advances and market dynamics evolve, both of these technologies remain in a state of perpetual development and competition. To align with the ever-growing demands of data centers, InfiniBand and Ethernet continue to embrace ongoing technological innovation and enhancements. Ethernet is enhancing network performance and reliability by introducing non-blocking network designs and switch topologies, while InfiniBand is optimizing performance, reducing costs, and broadening its range of applications.

    30Oct, 2023
    Exploring 400G Optical Module Typical Applications

    With the maturity of industry standards and the continuous growth of network demands, 400G optical module technology has become a vital engine driving the upgrade of the Information and Communications Technology (ICT) industry. It has gained increasing attention and widespread application. It has vast prospects in various application scenarios, including data center networks, metropolitan transport networks, and long-distance high-capacity transmission networks. This article will provide a detailed perspective on 400G optical modules in three typical application scenarios: data center networks, metropolitan transport networks, and long-distance high-capacity transmission networks.

    400G Optical Module Application in Data Center Interconnection Background

    Rapid Growth of East-West Traffic According to IDC data, it is expected that by 2025, nearly 80% of global data traffic will be stored in core and edge servers. Simultaneously, the growth rate of east-west traffic within data centers will be significantly higher than that of north-south traffic and inter-data center traffic. Traditional data centers are gradually being replaced by cloud data centers, driven largely by the popularity of cloud computing, which has stimulated the industry’s demand for 400G optical modules.

    Table 1, Comparison of different types of data centers

    Trends: Reduction in Cost per Bit and Power Consumption

    In general, customer demands adjust in response to changes in application scenarios. For long-distance WDM transmission, module performance is a key factor as customers seek higher capacity and longer transmission distances. In contrast, in short-distance transmission scenarios within data centers, cost is of greater importance to customers.

    To achieve higher capacity, there are three main ways in which 400G optical modules reduce the cost per bit:

    1. PAM4 Technology: PAM4 technology effectively improves bandwidth utilization. Under the same bit rate conditions, PAM4 signals have twice the bit rate of NRZ signals, thus increasing transmission efficiency while reducing costs.
    2. Multi-Channel: An 8-channel transmission solution is more advantageous in terms of balancing cost and power consumption compared to 4-channel transmission.
    3. Higher-Rate Optical Chips: These optical chips can increase transmission speed without affecting the transmission distance. Various 25G-rate optical chips (DML, EML, VSCEL) are gradually upgrading to 56G-rate optical chips.

    400G optical modules find extensive applications in data centers. For instance, the 400G QSFP-DD XDR4 optical module can be used in 4x100G breakout scenarios for QSFP28-FR-100G. For data center interconnection applications, the 400G QSFP-DD FR4 optical module can support 2km single-mode fiber transmission. The 400G QSFP-DD LR8 and 400G QSFP-DD LR4 optical modules, on the other hand, support link lengths of up to 10km by transmitting 4 CWDM wavelengths. Additionally, the 400G QSFP-DD ER8 optical module, suitable for long-distance transmission, can cover 40km of G.652 single-mode fiber links.

    Applications of 400G Optical Modules in Metropolitan Area Carrier Networks

    Background: Traffic Growth Driven by New Technologies in the 5G Era

    Carrier networks are experiencing growing demands in the 5G era, including ultra-high bandwidth, massive connections, ultra-low power consumption latency, and high reliability. Existing urban networks’ 100G ports can no longer meet the transmission requirements for core and aggregation layers. Therefore, 400G optical module solutions play a significant role in 5G carrier networks.

    Table 2, Bandwidth requirement of optical modules in 5G Metro Carrier networks

    Trend: Reducing the Unit Bit Transmission Cost to Improve Reliability and Transmission Distance

    Due to the anticipated high volume of optical modules in 5G base stations, network operators urgently need to lower the optical module costs in network construction investments. Furthermore, in the telecom carrier network, where optical modules have a lifespan of over 10 years and can transmit over 80km, there are higher demands for reliability and performance in metropolitan carrier network applications.

    Metropolitan Carrier Network Applications

    To achieve higher transmission rates and lower production costs, 400G optical modules used for integrated metropolitan carrier networks employ similar technologies to those in data center network scenarios:

    • High-Reliability Optical Modules: Utilizing sealed packaging to meet the 10-year lifespan and operating temperature range of 0~70°C requirements.
    • High-Performance LWDM Transmitters: Low dispersion costs and outstanding transmission performance.
    • High-Performance APD Receivers: Enhancing reception sensitivity.
    • Coherent Technology: To enable transmission distances beyond 80km, 400G optical module solutions employ coherent technology. Simultaneously, with the development of SiP and INP integration technologies and the continuous evolution of CMOS technology, coherent optical modules are becoming smaller and more power-efficient. The advantages of low power consumption and compact size in 400G ZR optical modules are expected to be widely applied in metropolitan network edge access scenarios.

    400G Optical Module Applications in Long-Distance High-Capacity Transmission Networks

    Background: Increasing Bandwidth Pressure for Long-Distance Data Transmission due to Traffic Growth

    The growth in network traffic has increased the bandwidth pressure on transmission networks. For long-distance and high-bandwidth transmission, the use of coherent transmission technology in 400G optical modules with wave division multiplexing (WDM) provides an optimal solution.

    With the introduction of flexible modulation and grid technologies, dense wave division multiplexing (DWDM) enhances spectral efficiency and transmission capacity by selecting the most suitable coding methods and channel widths for each port’s capacity and transmission distance.

    Trend: Improved Spectral Efficiency

    Coherent optical modules are advancing in three aspects:

    1. Spectral Efficiency: Progress in oDSP algorithms enhances spectral efficiency and single-fiber capacity.
    2. Bit Rate: Increasing the single-wavelength bit rate to achieve higher single-port bandwidth, thereby reducing unit bit cost and power consumption.
    3. Smaller Size and Lower Power Consumption: Using integrated optoelectronic components, advanced manufacturing processes, and specialized oDSP algorithms to reduce the size and power consumption of 400G optical modules.

    Conclusion

    Currently, mainstream 400G optical modules are widely used in various network scenarios, including data center networks, metropolitan carrier networks, and long-distance high-capacity transmission networks. The demand for higher capacity, lower unit bit cost, and lower power consumption is driving optical modules towards higher data rates.

    24Oct, 2023
    Decoding the Potential – 400G Optical Modules for Next-Gen DCI

    From the data center network architecture perspective, transitioning from existing 100G solutions to meet the same-scale data center’s non-blocking network throughput has posed several challenges. Achieving this objective requires adding more ports, expanding rack space to accommodate servers and switches, and providing more server rack space. This approach is not only uneconomical but also significantly increases the complexity of the network architecture.

    In contrast, transitioning from 100G to 400G is a more cost-effective way to provide higher bandwidth for data centers while reducing network architecture complexity. Furthermore, following the natural progression of network equipment development, when the interface speed increases fourfold to reach 400G, given mature device and transceiver technologies, the cost of a 400G network device typically amounts to only about 50% of the cost of four 100G devices and interfaces.

    400G Optical Modules in Data Center

    400G has already become ubiquitous in data center applications.

    ICP Data Center Architecture

    400G already become a must-have optical communication solution in DCI scenarios.

    400G becomes the first choice for Hyperscale in POD interconnects.

    Note: POD (Point of Delivery) is a distribution point that is used to facilitate resource pooling in a data center. To achieve this, a physical data center is typically divided into one or more physical zones, with each zone referred to as a POD. Thus, POD is a physical concept and represents the fundamental deployment unit of a data center. Typically, one piece of physical equipment can only belong to one POD.

    ToR-Server, 100G Downlink 400G Uplink

    Market Forecast and Iteration Direction of 400G in Data Center Technology

    Market Forecast for 400G and Higher Speed Optical Modules

    According to Light Counting’s projections for 400G and 800G-related products, SR/FR series optical modules are expected to be the primary growth products in data centers and internet centers:

    The forecast indicates that 400G rate optical modules will be deployed on a large scale in 2023 and will account for the majority of optical module (40G and above rate) sales in 2025:

    Data includes all ICP and Enterprise Data Centers

    400G Optical Module Iteration Trends

    With the physical layer single-lane electrical port speed reaching 112Gbps, data center networks will face significant challenges in network system design, equipment design, key component design, and more.

    In this context, on June 13, 2021, the QSFP112 MSA (Multi-Source Agreement) group officially released the new QSFP112 specification 1.0. A total of 31 companies came together to support the QSFP112 MSA, with founding members including companies like Accelink Technologies, Alibaba, Amphenol, Baidu, Credo, H3C, Hisense, and others, to meet the industry’s demand for high-speed, high-density network solutions. The MSA will significantly help traditional QSFP users upgrade their link bandwidth to 400G per port with lower costs and shorter transition times.

    From the different perspectives of 112G high-speed system design, the study of high-speed system design, system test methodology, high-speed interconnect system simulation, printed circuit board design for next-generation devices, high-speed backplane connectors, I/O connectors, and high-speed copper cables design and application for network devices under 112G SerDes are described in several aspects. All these efforts are aimed at accelerating the technological innovation of 112G SerDes-based networking devices.

    In China, major domestic internet companies like Alibaba, Baidu, JD.com, ByteDance, and Kuaishou, while still primarily using 25G or 56G electrical ports in their current data center architectures, are jointly planning for high-speed electrical interfaces based on the 112G SerDes for their next-generation data centers.

    In the optical module industry, GIGALIGHT has introduced 400G QSFP112 AOC/VR4/SR4 modules. Similarly, other companies such as Accelink, InnoLight, and Eoptolink have introduced their own 400G QSFP112 products and initiated validation plans with internet companies.

    400G QSFP112 SR4/FR4/DR4 modules operating at 112Gbps SerDes speeds will undoubtedly be massively deployed during the process of upgrading from 100G to 400G in next-generation data centers.

    13Oct, 2023
    GIGALIGHT Satisfactorily Delivers DCI 2.0 BOX and 400G QSFP-DD ZR+ Modules Overseas Clients

    SHENZHEN, China Oct. 13, 2023 — GIGALIGHT, through its Open Optical Network Lab, is excited to announce the successful deployment of single-lambda 400G DCI BOX network systems for international data center and telecommunications operator clients. This cutting-edge system leverages the latest 400G QSFP DD ZR/ZR+ coherent optical modules and is housed in a 2U chassis, offering a remarkable maximum transmission capacity of 6.4T. This innovative solution is built around a white box design.

    Key System Features:

    Capacity and Form Factor

    • A 2U standalone chassis can support a maximum capacity of 6.4T, with options for 1U and 2U chassis.

    Use Cases

    • Primarily designed for Data Center Interconnect (DCI) and metro network WDM/backbone WDM, providing a white box transmission platform.

    Design Advantages:

    • The product emphasizes white box design and simplified architecture.
    • A hybrid design of optical and electrical layers effectively saves data center space and facilitates bandwidth expansion.
    • Supports SNMP-based graphical management interface for intuitive fault localization.

    Cooling and Redundancy

    • Utilizes a front-to-rear airflow design, combined with multiple sets of high-speed redundant fans and power supplies, ensuring outstanding cooling performance.

    Hardware Support

    • Optical Layer: Supports Raman, EDFA, WSS, and OP cards, among others.
    • Electrical Layer: Line-side supports 100G/200G/400G coherent cards, while client-side supports 10GE/25GE/100GE/400GE.

    Network Management

    • Supports SNMP/Netconf protocols, CLI/Web/BS management, dual main control backup, and OSC communication.
    • Boasts robust transmission and network management capabilities, making it especially suitable for data center DCI and metropolitan network applications.

    Business Cards

    400G QSFP-DD DCO Muxponder

    This is the first business card introduced by GIGALIGHT in China. The line-side port supports cost-effective 400G QSFP-DD DCO ZR/ZR+, while the customer side supports 4x100GE QSFP28 ports, meeting the needs of DCI interconnect scenario applications.

    800G Transponder 1/4 Business Card

    With up to 800G Bandwidth, featuring support for two 400G CFP2 DCO on the line side and 2x400G QSFP-DD ports on the customer side, catering to high-bandwidth DCI applications.

    400G QSFP-DD DCO ZR+ Coherent Module

    Supports single-wavelength 400G DWDM, adjustable Tx optical power from 0 to -6dBm, flexible grid, and openZR+ protocol. It utilizes OFEC, offers excellent interoperability, achieves an Rx OSNR of 24dB/0.1nm @ 400G 16QAM, meets the requirements for 300km to 400km service interconnections, and has a typical power consumption of less than 22W.

    GIGALIGHT is one of the OpenZR+ members. The company boasts significant technical expertise in the coherent optical module field. GIGALIGHT’s Open Optical Network Lab has conducted comprehensive integration and fundamental research on OpenDCI BOX. Since entering the coherent wavelength division transmission system business, GIGALIGHT has successfully implemented network and transmission services based on 200G and 400G coherent optical modules for several overseas customers. This has led to the accumulation of extensive experience in overseas customer installation and maintenance

    About GIGALIGHT

    As an open optical networking explorer, Gigalight integrates the design, manufacturing, and sales of both active and passive optical devices and subsystems. The company’s product portfolio includes optical modules, silicon photonics modules, liquid-cooled modules, passive optical components, active optical cables, direct attach copper cables, coherent optical communication modules, and OPEN DCI BOX subsystems. Gigalight focuses on serving applications such as data centers, 5G transport networks, metropolitan WDM transmission, ultra-HD broadcast and video, and more. It stands as an innovative designer of high-speed optical interconnect hardware solutions.