Gigalight


Font Size:big  middle  small

Data Center: Parallel or WDM

Number of visits: Date:2017-7-10 18:24

Abstract:

The systematic interconnection architecture design and physical cabling interconnection design have increasingly become the focus of data center. Along with the constant upgrade and anticipation of optical interconnection speed and density, people have been in hesitation and anxiety. The systematic design of data center covers multiple disciplines and as a result, few people are able to have a precise estimation about all the future subjects. Based on different storage ability and the different calculation capability of data center, different designs are supposed to be applied. Hopefully, this article can help to clarify the framework discussion by using parallel or WDM in the link circuit.


Data center is a very complex concept including computer systems, related equipments(like communication and storage system) and redundant digital communication connection, equipments, monitoring devices and other various security devices. Therefore, the evaluation of data center design must be in line with the calculation ability, storage ability, interconnection ability as well as the ability of monitoring environment. Inevitably, these designs will bring about the cost of calculation, storage, interconnection and environmental control as well as supervision. If a reasonable plan, based on connecting current technology and demands with the future potential demands and technology anticipation, is unable to be figured out, data center will become a cost-consuming warehouse. Once the data center is established, then it is irreversible. The framework designer must be very discreet.


In the data center, the common optical interconnection solution is stated as follows:


Based on the difference between the calculation and transmission speed of system, the internal connection of systematic equipment can employ 10G AOC ( 300 meters link circuit), 4*14*10G AOC(300 meters link circuit) and 4*25G AOC(100 meter link circuit) as well as 100G PSM4 technology(500 meters and 2 kilometers). In recent years, mass data center has adopted CWDM4 (2 kilometers link circuit) to reduce fiber quantity. Due to the fact that different solutions will have different performance and cost, it’s believed that the uses of multi-mode optical interconnection technology in the link circuit will be the most mature and reasonable solutions. According to the current evaluation on cost, the cost of 40G multi-mode link circuits is 2.5-3 times higher than that of 10G multi-mode link circuits; while the cost of 100G multi-mode link circuits is about 2.5-3 times higher than that of 40G. Meanwhile, both 40G and 100G multi-mode link circuit adopt MPO multi-mode fiber, which means one can realize the technical smooth upgrade from 40G to 100G without affecting the current network cabling framework.


It has always been a concern that 25Gb/s will be the technical limit of multi-mode VECSEL, from which people start to doubt the possibility of the extensibility of multi-mode data center. Consequently, they directly tend to use the single-mode data center architecture, which, theoretically speaking, can constantly keep extending and upgrading. However, the recent study has demonstrated that 200G VCSEL NRZ technology is no longer an issue as it can easily realize the 8*25G AOC optical interconnection with the same QSFP28 packaging space, which enables systematic architecture available to both 8*25G and 4*50G. People may be also concerned about the feasibility, but frankly speaking, the technical bottlenecks that 400G VCSEL and 400G single mode have are the same despite that a conclusion has not been drawn yet. To some degree, it is the delusions that confuse people’s eyes, according to our knowledge, 200G VCSEL AOC will furtherly shorten the transmission distance (it is estimated to be 50 meters) but the OM5 provided by Corning can perfectly compensate for this defect. Therefore, the major problem still rests in the replacement of cabling layout with the usage of OM5 fiber. It is difficult to draw a conclusion just regarding to the single issue as multiple issues are mixed together and then needed to be balanced.


PSM4 technology is a supplement of multi-mode 100G data center. So far, the whole industry has estimated on the potential of PSM4 technology. And adversely, they pay more attention to the CWDM4 technology. PSM4 technology uses the single-mode MPO fiber transmission which can easily reach up to 2KM transmission distance instead of 500 meters set by the industry standard. Actually, PSM4 technology is supposed to transmit 10KM rather than the 2 KM. Apparently, this is a wrong recognition after the focus is transferred. At the first beginning, both distribution calculation and parallel calculation are supposed to be the programmatic standard. The cost of 100G PSM4 is 3 times higher than that of 100G VCSEL. It is most likely that it will be reduced to 1.5-2 times, if people are willing to. Compared with 100G CWDM4 technology, PSM4 technology applies 4 times fiber quantity, and the cost is zero. The major concern for system designers are the limit of fiber resources and future extendibility as well as the cabling cost of data machine room. These knowledge and points with different fields will be discussed and then constructive conclusion will be drawn.


The reason why the fiber transmission uses the CWDM and DWDM is due to the limit of fiber resources. After that, the designer will consider replacing the single-wavelength equipment with wavelength division multiplexing equipment. Under the case of adequate fiber resources, the use of WDM actually can increase the cost as the fiber cost can be considered to be zero. If the space is allowable, enough fibers for future extendibility are supposed to be planed. Unlike data center, backbone metropolitan area network has already finished the fiber establishment, so the fiber resource buried underground is limited whether promising demands in the future are seen or not. Therefore, the WDM technology is necessary to be introduced. While this is the first wave for data center, it has no need to adopt the WDM technology with uncertainty of the kind of transmission speed and bandwidth. WDM technology can be redeemed as an effective approach to address the difficulty of limited fiber resources, but which requires lots of cost price. But according to current demands, the extra cost is not necessary as it is only a density upgrade along with the same speed. Based on its current module and system design, its first layout tends to be unoriginal.


It is impossible to upgrade from the existing 100G CWDM4 module to 200G and 400G directly. Regular WDM can be upgraded because the active and passive are divided into two independent systems. Passive module has nothing to do with velocity and wavelength, so it can be upgraded freely. Data center now has CWDM4 integrated with active and passive technology. When upgraded, the advantage is the small size but the shortcoming is that we cannot replace it with a module with greater velocity and wavelength in this architecture, wasting previous expense on CWDM4. However, it is much cheaper to realize light connection with PSM4. Considering the cost of materials,CWDM4 is 1.35-5 times more than PSM4.


The existing module of CWDM4 has increased the reliability risk of data center. Compact CWDM4 has the problem of temperature drift and reliability risk, making scale deployment needing a fixed temperature range of 15℃-50℃ in the computer room of data center, otherwise there is possibility of bit error caused by temperature drift or low reliability of optical chip. Data center will suffer catastrophic outcomes once the water-cooling and air-conditioner system malfunctions. Module of CWDM4 needs a more reliable work environment, which booms the energy consumption of the system (PDU increases). The energy consumption due to the use of this module doubles the purchase cost within one month. According to our test, there is no absolute guarantee for the reliability of 4*25G CWDM CHIP. The reliability of any channel affects the entire module. The potential reliability risk of CWDM4 lies in optical chip reliability and AWG temperature drift. It is clear that the CWDM4 technique is in development, but we are all taking great risks.


Will CWDM4 optical fiber bring about optical fiber resource conservation and upgrade once and for all? It is not simple. When we design the system, we should consider the parallel architecture of optical fiber resource instead of WDM at the beginning because it is meaningless to think of WDM at this moment. We do not know how much fiber resource the technology will need in the future. Theoretically, the upgrading of PSM4 and CWDM4 to PSM8 and CWDM8 need to abandon PSM4 and CWDM4 (as regards cost, abandoning PSM4 is different from abandoning CWDM4). 8 fibers are needed to upgrade one PSM4 to PSM8, but the technological bottleneck is that the upgrade has to be easier and cheaper. If we need, we can actually upgrade one PSM4 to CWDM4 so that the fiber resources will be used effectively. One may argue that if the 100G PSM4 is upgraded to 100G CWDM4 in the future instead of right now, it would waste a PSM4. According to our previous analysis, the energy consumption and management cost saved by PSM4 can offset more than 10 CWDM4 modules for data center builder. What’s more, disassembled PSM4 can also be applied to other new systems due to its high reliability. Although I have not done accurate calculation, and it is clear that PSM4 has many advantages, such as transmission transparency, high reliability and being easy to conduct split-flow monitor on channels and it has no temperature drift.


We need to strengthen the concept: WDM technology is spatial multiplexing technology, and has no relationship with system bandwidth and speed. If the system structure engineer melts the WDM technology and the related technology in upgrading system from 40G/100G to 200G/400G in a whole, this will inevitably fall into two different technical misunderstandings and lead to wrong conclusions. The value of WDM technology is the applications in transformation of the existing optical links and limited resources (such as rental cost and scarcity of fiber resources). It cannot be regarded as an initial technology. For further explanation, a data center link usually includes four kinds of architecture design: device architecture, wiring architecture, architecture, upgrade architecture and expansion architecture.


Device architecture: to ensure the speed and throughput in the system, can only be based on existing technology.


Wiring architecture: to ensure fiber resources required in the system, you must think over the width of the road once. Upgrade architecture: technical path and cost of the system upgraded from low rate (such as 10G/40G) to high rate (such as 100G/200G/400G).


Upgrade architecture: technical path and cost of the system upgraded from low rate (such as 10G/40G) to high rate (such as 100G/200G/400G).


Expansion architecture: how to use the space division multiplexing technology to upgrade the capacity and density of the existing systems.


For the expansion architecture, we need to do further proof. The pic shown as below is two typical data center WDM optical interconnect architectures:


(A) The solution is the design of active and passive separation.

(B) The solution is similar to the structure of the 100G CWDM4 data center.


As for the system upgrade, we can easily find that we only need to replace the routing equipment for the Solution A. But for Solution B, we need to replace all except the cables. So we tend to think that the existing CWDM4 data center has led to a rise in the upgrading cost when changing into more compact devices. Nothing will be left in addition to basic cables. Even we think these left cables can be reused in the next generation of data center, it is questionable. For the next-generation data centers, such as 400G data center, if we use FR8 or LR8, the wavelength is 1550nm dispersion shifted fiber, but now 100G CWDM4 is using the 1310NM cable.


As a new thriving thing, data center has too much room for discussion. The final recommendation conclusion of this article is:

1) Multi-mode datacenter is still the core of datacenter developing, not only the cost, but also the reliability of the technology.


2)In the long-range development of data center, PSM4 technology is better than CWDM4, and PSM8 technology is better than CWDM8 technology, and so on. Taking parallel technology as the mainstream of data center is the best choice. At least, this layout will not lead to losing all. And parallel technology is transparent and easy to do OMA management.


3)Unless necessary, using NRZ technology evolution in data center is the most reliable and low cost method. But we have to consider whether PAM8/PAM16 is the next, once the data center is dominated by PAM4. If so, the system will increasingly become opaque and difficult to manage. If we use PSM4+PAM4 structure, this will be much better than CWDM4+PAM4.


4)The integrated technology of CWDM4 is used in data center's first deployment nowadays, which is on account that, at the beginning, data center is defined as the transition type, not a fixed type. This concept needs to be corrected. I think this is the demarcation point of professionals and non professionals. At the first time when we learn optical communication, we know that CWDM is a technique for increasing capacity. CWDM4 technology circles between discrete and integrated. It is not mature and also cannot be extended to 8 or 16 waves, which greatly limits the expansion capability of CWDM system. Using CWDM4 technology in data center’s first deployment is not appropriate. Objectively, the first deployment of this technology limits the possibility and the flexibility of the system, and it brings the rising cost on the contrary, so we must be very cautious.

Conclusion

The parallel architecture in data center and CWDM are similar to the universe or the philosophy, and it is not easy to clarify it. CWDM technology is an extending of parallel technology, not confrontational technology. Where there is a CWDM universe, there will be a parallel universe. If the parallel universe does not exist, and the CWDM universe will soon collapse without development. Or we can judge it from another view. God always prefers simple. “Parallel” is equal to “simple”.

TypeInfo: Blog Articles

Keywords for the information: