Data Center Design: Cost, Technology, Conciseness and Reconfigurability

100G data centers are maturing, no matter from which angle, the technologies are getting more and more perfect. The 100G data centers can already carry most of the things we want, but at most it only loads the memory and spiritual food of human life—we do see some computations, but such computations are no different from the supercomputing of the previous era. If we don't understand the goal of data center applications, we can't design a data center that matches technologies and applications.

Optical interconnection technology is moving from 100G to 200G and 400G. Perhaps we can say that 100G optical interconnection has just reached its robust performance in 2018. So we can basically assert that the 100G data centers built before 2018 are all dangerous buildings or houses built on sandy land. We must be aware of the risks.

The current large data centers basically follow the 100G CWDM4 structure of the last era, and use AOC and DAC the same time. Today, we need to retell a topic. The proposal is to divide the data center into two parts: transmission structure and interconnection structure. The WDM active architecture is used in the transport layer, while the parallel PSM structure (including parallel optics and parallel electrics) is firmly used in the interconnection layer. We do see that the architecture similar to Facebook is very concise, but it is also high-cost. Therefore, it is necessary for us to demonstrate the relationship between the economic cost and the economic structure. What we need to do is to find a sort relationship based on a fixed principle that will guide us to make the best choice in difficult choices.

The Cost of Data Center 100G CWDM4 Structure

In order for the 100G CWDM4 structure to be widely used, data centers have paid a high price. The main reason is that in the past era, the stability and consistency of optical chips were not good, and the optical interconnection of data centers was in a period of no standard. Fortunately, at least Gigalight's product design standards are in line with expectations and applications. Now the industry knows that reliability, product life and maintenance costs are related to each other. Current conclusions basically support that CWDM4 conforms to the mainstream characteristics of 100G data center in terms of technology implementation, such as saving optical fibers, switching from maintenance of multiple products to maintenance of one product. However, from a different point of view, this concise structure is also problematic. There are three reasons shown as following.

• Optical transceivers are expensive. When calculating interconnection architecture costs, we tend not to calculate the cost of optical fibers, because this is the end-to-end inversion, which will lead our thinking to a very old-fashioned direction. As we build the building, we will not consider how much paint we need to use. Optical fibers serve systems, just as paint serves buildings, or food serves three meals for human beings. The introduction of WDM technology to all interconnect architectures requires at least 2 to 3 times the cost of optical transceivers.
• Total cost is uneconomical. The reason why we do not prefer to use more WDM technologies in interconnection is that signal layering in dense networks is better clear and schedulable. WDM transceiver cuts down the cost of optical fibers by 3/4 mainly through AWG or free space optics technology, but it brings about problems of high production investment and good product rate in industry, sacrificing profit margins. In fact, we know that the vast majority of interconnection distances are within 500 meters, so the cost savings of optical fibers are very limited.
• Whether the maintenance cost is economic. The dispute lies in the economic nature of maintenance. The economics of maintenance lie in product stability and network reconfigurability. From the perspective of product stability, the CWDM4 transceivers are becoming mature and reliable, but they are still an order of magnitude worse than the PSM4 ones. Of course, the data centers need not consider this order of magnitude. Then, let's talk about the reconfigurability of the network. The so-called reconfigurability is to push the original structure down and rebuild it again to see if there is anything left. From the human pursuit of green economy point of view, if we want to upgrade the existing network in the first place, it is necessary for us to demonstrate what the future network architecture of data centers should be, and then we will come back to a conclusion.

The Future Network Architecture of Data Centers

Two years ago, I published an article about the choice of PSM or WDM in data center. In this article, I think the choice of PSM is more realistic, but it has attracted some criticism. The reality is also contrary to my view—the data center is moving towards the structure of CWDM4 covering PSM4. However, just as humans on the road, it is very common that the right vision is replaced by the wrong path. A child who grew up in a poor environment surely have a totally different world outlook and money outlook from a child of rich origin. At the 2018 OFC exhibition, the topic of 400G was very popular, but it was very immature. According to people's understanding of 400G at the beginning of 2018, it is basically to skip PAM4 technology and directly use 100G Single Lambda DSP technology to implant 400G transceiver, that is to say, to skip 200G directly to an unimaginable 400G. This leap is not a generation, but two generations. Now, we already know that this desire is obviously too optimistic.

From NRZ to PAM4 and then to DSP, is it a gradual leap, or a leap that can reach the ultimate goal in one step? We still need to discuss these technologies from the perspective of transmission or interconnection. I think the first two are used for interconnection architecture, while the DSP technology is basically only used in the field of optical transmission.

There is a fundamental difference between DSP's work and PAM4 modulation. Whether the DSP can succeed in the client side module is still unknown—I believe that it is impossible to use the DSP to deal with the recovered signal distortion without any processing of the link optical layer. Of course, just as many of my views have been gradually corrected by the progress of the times, trying to argue, explore and make mistakes is the only way for the progress of human technologies and markets. Aside from the unpredictability of technology implementation, we have four analytical architectures covering 200G and 400G networks.

• 200G NRZ. The architecture of 200G NRZ is a structure that uses 8 parallel signals. The advantages are that the cost is very low and the density is very high. The disadvantage is that the physical cabling complexity and one-time cost of optical fiber quantity are higher.
• 200G PAM4. The architecture of 200G PAM4 is a FR4 architecture for modulating technological innovation in the electrical area. The advantage is that the cost is moderate and the density is very high, which saves a lot of optical fiber compared to the NRZ architecture. As for shortcomings, if any, it is the same as 100G CWDM4.
• 200G/400G. The 200G/400G hybrid architecture adopts PAM4 technology and 8 channel multiplexing technology—FR8. The advantages are that the cost is balanced, the density is very high, and the demand for the optical fiber is very small. The disadvantage is that the stability of optical structure and module remains to be investigated.
• 400G Silicon Photonics. The 400G Silicon Photonics architecture can realize the ideal 100G Single Lambda transmission with the help of Silicon Photonics technology platform, and DSP is not required. The advantages of this architecture are obviously cost effective, moderate density, high speed and simplicity. The disadvantage is that Silicon Photonics optical transceiver is an alternative technology, which still needs time breeding.

Summary

• The evolution direction of pure 100G CWDM4 architecture is 200G FR4, which can not be evolved further. It is necessary to rewire or increase optical fiber resources.
• The evolution direction of 100G PSM4 network architecture is 200G DR4 or 200G Silicon Photonics DR4, or 400G DR4.
• The 8-channel 200G NRZ network provides abundant optical fiber resources. So, its future evolution is not a problem, and it will probably waste half of the optical fiber resources.

At present, we have not paid attention to a popular 400G network structure—400G DR4&FR4. Basically, we believe that this architecture is extremely difficult to achieve. This architecture is a beautiful illusion of people transcending technical difficulties, and it is not necessarily economic from a practical point of view.

The Priorities of Data Center Design

We understand that people including ourselves have been looking for a data center that is concise, reconfigurable and cost-effective. But people usually prioritize things in terms of simplicity, reconfigurability, cost and technology, which goes against the law of things. Contrary to the law of things, additional expenses are needed. There is nothing that humans can't do, and sometimes they are so capricious that they spoil costs. We believe that from a professional point of view, we should put the cost first, followed by technology, then conciseness, and finally reconfigurability.

• Cost. Data center must be cost oriented. Only a cost-oriented economy is the most reasonable and consistent with the overall benefits, because the starting point of the economy is the relationship between costs and benefits. The industrial structure without consideration of cost distorts technology, simplicity and the world order.
• Technology. Technology has its time course of realization, and the use and cost of technology in different periods show corresponding relationship. Cost determines the scale of technology, rather than forcing a technology to deliberately reduce costs. For example, we have forcibly promoted the scale of technology for solar energy and electric vehicles to reduce costs, but have paid a very high price.
• Conciseness. Conciseness can not be viewed broadly, or from the surface of that naive simplicity. Conciseness is the result of process design, and it can not become a kind of original intention. When we say design must be concise, we mean to find a concise expression after all the abilities and techniques are contained. Conciseness is a kind of art after complex abstraction. The simplicity of the surface can be interpreted as graffiti or laziness.
• Reconfigurability. Reconfiguration is very difficult. Looking back at the technological route that human beings have gone through, we find that technology and network are not reconfigurable, which has indeed caused a lot of waste. Reconfigurable mission requires human beings to make production and input according to plan. The market economy with unrestrained waste is essentially reconfigurable. At present, human beings are unable to reconstruct technology, so we should rather make a good plan instead of thinking about how to reconstruct it.

About GIGALIGHT

As the open optical network device explorer, GIGALIGHT integrates the design, manufacture and sales of active & passive optical devices and subsystems. The main products are optical transceivers, silicon photonics transceivers, liquid cooling transceivers, optical passive components, AOC & DAC, coherent optical modules and open DCI BOX subsystems. GIGALIGHT is a hardware solution provider of innovatively designed high-speed optical interconnection that focuses on data center, 5G carrier network, metro WDM transmission, UHD broadcast video and other application fields.