Data center Archives - EFFECT Photonics

Insights

August 7, 2024

How Photonics Enables AI Networks

Artificial Intelligence (AI) networks have revolutionized various industries by enabling tasks such as image recognition,…

Artificial Intelligence (AI) networks have revolutionized various industries by enabling tasks such as image recognition, natural language processing, and autonomous driving. Central to the functioning of AI networks are two processes: AI training and AI inference. AI training involves feeding large datasets into algorithms to learn patterns and make predictions, typically requiring significant computational resources. AI inference, on the other hand, is the process of using trained models to make predictions on new data, which requires efficient and fast computation. As the demand for AI capabilities grows, the need for robust, high-speed, and energy-efficient interconnects within data centers and between network nodes becomes critical. This is where photonics comes into play, offering significant advantages over traditional electronic methods.

Enhancing Data Center Interconnects

Data centers are the backbone of AI networks, housing the vast computational resources needed for both training and inference tasks. As AI models become more complex, the data traffic within and between data centers increases exponentially. Traditional electronic interconnects face limitations in terms of bandwidth and power efficiency. Photonics, using light to transmit data, offers a solution to these challenges.

Photonics enables the integration of optical components like lasers, modulators, and detectors on a single chip. This technology allows for high-speed data transfer with significantly lower power consumption compared to electronic interconnects. These advancements are crucial for handling the data-intensive nature of AI workloads.

Figure 1: How architectures might change inside the AI data center. AI nodes on a single rack are connected via electrical or RF connections to an electro-optic switch that connects to other racks with optical fiber. In the future (on the right side of the figure), these AI nodes might be interconnected via optics. Figure inspired by Rob Stone’s (Meta) presentation.

Enabling High-Speed AI Training and Inference

AI training requires the processing of vast amounts of data, often necessitating the use of distributed computing resources across multiple data centers. Photonic interconnects facilitate this by providing ultra-high bandwidth connections, which are essential for the rapid movement of data between computational nodes. The high-speed data transfer capabilities of photonics reduce latency and improve the overall efficiency of AI training processes.

This high transfer speed and capacity also plays a critical role in AI inference, particularly in scenarios where real-time processing and high throughput is essential. For example, in a network featuring autonomous vehicles, AI inference must process data from sensors and cameras in real-time to make immediate decisions. For other ways in which photonics plays a role in autonomous vehicles, please read our article on LIDAR and photonics.

Into Network Edge Applications

The network edge refers to the point where data is generated and collected, such as IoT devices, sensors, and local servers. Deploying AI capabilities at the network edge allows for real-time data processing and decision-making, reducing the need to send data back to centralized data centers. This approach not only reduces latency but also enhances data privacy and security by keeping sensitive information local.

Figure 1: Telecommunication tower with 5G cellular network antenna on city background — Figure 2: Telecommunication tower with 5G cellular network antenna on city background

Photonics enables edge AI by providing the necessary high-speed, low-power interconnects required for efficient data processing at the edge. For some use cases, the network edge could benefit from upgrading their existing direct detect or grey links to 100G DWDM coherent. However, the industry needs more affordable and power-efficient transceivers and DSPs specifically designed for coherent 100G transmission in edge and access networks. By realizing DSPs co-designed with the optics, adjusted for reduced power consumption, and industrially hardened, the network edge will have coherent DSP and transceiver products adapted to their needs. This is a path EFFECT Photonics believes strongly in, and we talk more about it in one of our previous articles.

Conclusion

Photonics is transforming the landscape of AI networks by providing high-speed, energy-efficient interconnects that enhance data center performance, enable faster AI training and inference, and support real-time processing at the network edge. As AI continues to evolve and expand into new applications, the role of photonics will become increasingly critical in addressing the challenges of bandwidth, latency, and power consumption. By leveraging the unique properties of light, photonics offers a path to more efficient and scalable AI networks, driving innovation and enabling new possibilities across various industries.

PDF Download

Tags: AI inference, AI networks, AI training, autonomous driving, bandwidth, computational resources, Data center, detectors, edge AI, EFFECT Photonics, energy efficient, high-speed interconnects, lasers, latency, modulators, optical components, photonic processors, Photonics, power consumption, real-time processing, silicon photonics

Insights

June 28, 2023

Coherent Lite and The Future Inside the Data Center

In the dynamic landscape of data centers, the demand for greater bandwidth and extended reach…

In the dynamic landscape of data centers, the demand for greater bandwidth and extended reach is rapidly increasing. As shown in the figure below, we can think about three categories of data center interconnects based on their reach

Intra-data center interconnects (< 2km)
Campus data center interconnects (<10km)
Metro data center interconnects (<100km)

Asset 6 data center interconnect — Figure 1: Example of different data center interconnect (DCI) links. Intra-data center interconnects usually have lengths up to 2kms, campus interconnects up to 10km, and metro DCIs up to 100km, although there can be longer DCIs.

Coherent optical technology has already established itself as the go-to solution for interconnecting data centers over long distances in metro areas.

However, within the confines of data centers themselves, intensity-modulated direct detect (IM-DD) technology remains dominant. Recognizing the limitations of IM-DD in meeting evolving requirements, the industry is exploring “Coherent Lite” solutions—a simplified implementation of coherent technology designed specifically for shorter-reach data center connections.

This article delves into the concept of coherent lite technology and its potential to address the escalating bandwidth demands within data centers.

Reducing Dispersion Compensation

The quality of the light signal degrades when traveling through an optical fiber by a process called dispersion. The same phenomenon happens when a prism splits white light into several colors. The fiber also adds other distortions due to nonlinear optical effects. These effects get worse as the input power of the light signal increases, leading to a trade-off. You might want more power to transmit over longer distances, but the nonlinear distortions also become larger, which beats the point of using more power. The DSP performs several operations on the light signal that try to offset these dispersion and nonlinear distortions.

Asset 7 dispersion in optical fibers — Figure 2: Explaining dispersion compensation in optical fibers. A pulse of light will spread out and distort due to dispersion effects.

However, shorter-reach connections require less dispersion compensation, presenting an opportunity to streamline the implementation of coherent solutions. Coherent lite implementations can reduce the use of dispersion compensation blocks. This significantly lowers system power consumption.

The Trade-Offs Between Fixed and Tunable Lasers

Coherent lite solutions also aim to replace tunable lasers with fixed lasers to reduce costs. The use of fixed lasers eliminates the need for wavelength tuning and associated control circuitry and algorithms, simplifying the implementation and reducing operational complexities.

While fixed lasers offer significant advantages, tunable lasers will push to remain competitive. As we described in a previous article, advances in tunable laser technology aim to further reduce package footprints and leverage electronic ecosystems to reduce cost. Such developments will allow tunable lasers to keep pace with the demands of coherent lite solutions, ensuring a viable alternative for shorter-reach data center connections.

Scaling Data Center Links

The increasing length and bandwidth of links within data centers is increasingly calling for the use of coherent technology. As bandwidth scales to 1.6 and 3.2 terabits, traditional direct detect technology faces challenges in keeping up with the growing distances. Intra data center links that were previously limited to 2 kilometers are now extending to 5 or even 10 kilometers, demanding more robust and efficient transmission technologies.

In this context, coherent lite technology provides an attractive middle ground for enabling extended-reach connections within data centers. By leveraging some aspects of coherent solutions, coherent lite technologies facilitate the reliable and efficient transport of data over longer distances.

Takeaways

As data centers evolve to accommodate escalating bandwidth demands, coherent lite technology emerges as a promising solution for communication links within these facilities. By reducing dispersion compensation, simplifying their laser setups, and enabling extended-reach transmission, coherent lite solutions address the limitations of traditional direct detect technology. These advancements pave the way for enhanced performance, and seamless scalability within data center environments.

Article PDF

Tags: 1.6T, 3.2T, bandwidth, coherent, Coherent Lite, Coherent Transmissions, Complexity, CWDM, Data center, datacom, dispersion compensation, Fixed Lasers, IM-DD (Intensity-Modulated Direct Detect), Interconnects, Intra DC, optical technology, power, reach, Scaling Data Center Links, Simplification, Single Wavelength

The Light Path to a Coherent Cloud Edge Banner

Insights

May 18, 2023

Data Center Interconnects: Coherent or Direct Detect?

Article first published 15 June 2022, updated 18 May 2023. With the increasing demand for…

Article first published 15 June 2022, updated 18 May 2023.

With the increasing demand for cloud-based applications, datacom providers are expanding their distributed computing networks. Therefore, they and telecom provider partners are looking for data center interconnect (DCI) solutions that are faster and more affordable than before to ensure that connectivity between metro and regional facilities does not become a bottleneck.

As shown in the figure below, we can think about three categories of data center interconnects based on their reach

Intra-data center interconnects (< 2km)
Campus data center interconnects (<10km)
Metro data center interconnects (<100km)

Figure 1: Example of different data center interconnect (DCI) links. Intra-data center interconnects usually have lengths up to 2kms, campus interconnects up to 10km, and metro DCIs up to 100km, although there can be longer DCIs.

Coherent 400ZR now dominates the metro DCI space, but in the coming decade, coherent technology could also play a role in shorter ranges, such as campus and intra-data center interconnects. As interconnects upgrade to Terabit speeds, coherent technology might start coming closer to direct detect power consumption and cost.

Coherent Dominates in Metro DCIs

The advances in electronic and photonic integration allowed coherent technology for metro DCIs to be miniaturized into QSFP-DD and OSFP form factors. This progress allowed the Optical Internetworking Forum (OIF) to create a 400ZR multi-source agreement. With small enough modules to pack a router faceplate densely, the datacom sector could profit from a 400ZR solution for high-capacity data center interconnects of up to 80km. Operations teams found the simplicity of coherent pluggables very attractive. There was no need to install and maintain additional amplifiers and compensators as in direct detection: a single coherent transceiver plugged into a router could fulfill the requirements.

As an example of their success, Cignal AI forecasted that 400ZR shipments would dominate edge applications, as shown in Figure 2.

Figure 2: Forecast of 100G port equivalents shipped for edge applications. These shipments are overwhelmingly 400ZR standard technology. Source: Cignal AI Transport Applications Report Q42021. This plot is the most recent update that includes this 100G Equivalent Port data, but more updated Cignal AI reports with similar data can be found here. — Figure 2: Forecast of 100G port equivalents shipped for edge applications. These shipments are overwhelmingly 400ZR standard technology. Source: Cignal AI Transport Applications Report Q42021. This plot is the most recent update that includes this 100G Equivalent Port data, but more updated Cignal AI reports with similar data can be found *here*.

Campus Interconnect Are the Grey Area

The campus DCI segment, featuring distances below 10 kilometers, was squarely in the domain of direct detect products when the standard speed of these links was 100Gbps. No amplifiers nor compensators were needed for these shorter distances, so direct detect transceivers are as simple to deploy and maintain as coherent ones.

However, as link bandwidths increase into the Terabit space, these direct detect links will need more amplifiers to reach 10 kilometers, and their power consumption will approach that of coherent solutions. The industry initially predicted that coherent solutions would be able to match the power consumption of PAM4 direct detect solutions as early as 800G generation. However, PAM4 developers have proven resourceful and have borrowed some aspects of coherent solutions without fully implementing a coherent solution. For example, ahead of OFC 2023, semiconductor solutions provider Marvell announced a 1.6Tbps PAM4 platform that pushes the envelope on the cost and power per bit they could offer in the 10 km range.

Following the coming years and how the PAM-4 industry evolves will be interesting. How many (power-hungry) features of coherent solutions will they have to borrow if they want to keep up in upcoming generations and speeds of 3.2 Tbps and beyond? Lumentum’s Chief Technology Officer, Brandon Collings, has some interesting thoughts on the subject in this interview with Gazettabyte.

Direct Detect Dominates Intra Data Center Interconnects (For Now…)

Below Terabit speeds, direct detect technology (both NRZ and PAM-4) will likely dominate the intra-DCI space (also called data center fabric) in the coming years. In this space, links span less than 2 kilometers, and for particularly short links (< 300 meters), affordable multimode fiber (MMF) is frequently used.

Nevertheless, moving to larger, more centralized data centers (such as hyperscale) is lengthening intra-DCI links. Instead of transferring data directly from one data center building to another, new data centers move data to a central hub. So even if the building you want to connect to might be 200 meters away, the fiber runs to a hub that might be one or two kilometers away. In other words, intra-DCI links are becoming campus DCI links requiring their single-mode fiber solutions.

On top of these changes, the upgrades to Terabit speeds in the coming decade will also see coherent solutions more closely challenge the power consumption of direct detect transceivers. PAM-4 direct detect transceivers that fulfill the speed requirements require digital signal processors (DSPs) and more complex lasers that will be less efficient and affordable than previous generations of direct detect technology. With coherent technology scaling up in volume and having greater flexibility and performance, one can argue that it will also reach cost-competitiveness in this space.

Takeaways

Unsurprisingly, using coherent or direct detect technology for data center interconnects boils down to reach and capacity needs. 400ZR coherent is already established as the solution for metro DCIs. In campus interconnects of 10 km or less, PAM-4 products remain a robust solution up to 1.6 Tbps, but coherent technology is making a case for its use. Thus, it will be interesting to see how they compete in future generations and 3.2 Tbps.

Coherent solutions are also becoming more competitive as the intra-data center sector moves into higher Terabit speeds, like 3.2Tbps. Overall, the datacom sector is moving towards coherent technology, which is worth considering when upgrading data center links.

Tags: 800G, access networks, coherent, cost, cost-effective, Data center, distributed computing, edge and metro DCIs, integration, Intra DCI, license, metro, miniaturized, photonic integration, Photonics, pluggable, power consumption, power consumption SFP, reach, Terabit

Pluggables have becomes mainstream in datacom applications.

Insights

February 8, 2023

A Story of Standards: From 400ZR and Open ROADM to OpenZR+

Coherent optical transmission has been crucial in addressing network operator problems for the last decade. In this time, coherent technology has expanded from a solution reserved only for premium long-distance links to one that impacts data center interconnects, metro, and access networks…

Coherent optical transmission has been crucial in addressing network operator problems for the last decade. In this time, coherent technology has expanded from a solution reserved only for premium long-distance links to one that impacts data center interconnects, metro, and access networks, as explained in the video below.

The development of the 400ZR standard by the Optical Internetworking Forum (OIF) proved to be a milestone in this regard. It was the result of several years of progress in electronic and photonic integration that enabled the miniaturization of 400G coherent systems into smaller pluggable form factors. With small enough modules to pack a router faceplate densely, the datacom sector could profit from an ideal solution for high-capacity data center interconnects. Telecom operators wanted to implement a similar solution in their metro links, so they combined the Open ROADM standardization initiative with the 400ZR initiative to develop the OpenZR+ agreement that better fits their use cases.

This article will elaborate on these standardization projects—400ZR, OpenROADM, and OpenZR+—and explain what use cases each was designed to tackle.

What Is 400G ZR?

To cope with growing bandwidth demands, providers wanted to implement the concept of IP over DWDM (IPoDWDM), in which tunable optics are integrated into the router. This integration eliminates the optical transponder shelf and the optics between the routers and DWDM systems, reducing the network capital expenditure (CAPEX). This is shown in the figure below.

Figure 1: Implementation of a traditional coherent transport architecture (left), with the router connected to a transponder line card and a Reconfigurable Optical Add/Drop Multiplexer (ROADM). In contrast, an IPoDWDM solution (right) has 400ZR pluggable transceivers that go directly into the router and use passive multiplexing filters. The IPoDWDM solution eliminates the line card and additional optics, reducing network CAPEX.

However, widely deploying IPoDWDM with coherent optics forced providers to face a router faceplate trade-off. Since DWDM modules have traditionally been much larger than the client optics, plugging DWDM modules into the router required sacrificing roughly half of the costly router faceplate capacity. This was unacceptable for datacom and telecom providers, who approached their suppliers and the Optical Internetworking Forum (OIF) about the need to develop a compact and interoperable coherent solution that addressed this trade-off.

These discussions and technology development led the OIF to publish the 400ZR implementation agreement in 2020. 400ZR specifies a (relatively) low-cost and interoperable 400 Gigabit coherent interface for a link with a single optical wavelength, using a double-polarization, 16-state quadrature amplitude modulation scheme (DP-16QAM). This modulation scheme uses sixteen different constellation points that arise from combining four different intensity levels and four phases. It doubles the usual 16-QAM transmission capacity by encoding information in two different polarizations of light.

The agreement specified a link reach of 80 kilometers without amplification and 120 kilometers with amplification. For forward error correction (FEC), the 400ZR standard supports a concatenated FEC (CFEC) method. CFEC combines inner and outer FEC codes to enhance the performance compared to a standard FEC code.

The 400ZR agreement does not specify a particular size or type of module, but its specifications targeted a footprint and power consumption that could fit in smaller modules such as the Quad Small Form-Factor Pluggable Double-Density (QSFP-DD) and Octal-Small Form-Factor Pluggable (OSFP). These form factors are small enough to provide the faceplate density that telecom and especially datacom operators need in their system architectures. So even if we often associate the 400ZR standard with QSFP-DD, other form factors, such as CFP2, can be used.

What Is Open ROADM?

In parallel with the 400ZR standardization efforts, telecom network operators had a different ongoing discussion.

Reconfigurable Optical Add-Drop Multiplexers (ROADMs) were a game-changer for optical communications when they entered the market in the 2000s. Before this technology, optical networks featured inefficient fixed routes and could not adapt to changes in traffic and demand. ROADMs allowed operators to remotely provision and manage their wavelength channels and bandwidth without redesigning the physical network infrastructure.

However, ROADMs were proprietary hardware with proprietary software. Changing the proprietary ROADM platform needed extensive testing and a lengthy integration process, so operators were usually reluctant to look for other platform alternatives. Besides, ROADMs still had several fixed, pre-defined elements that could have been configurable through open interfaces. This environment led to reduced competition and innovation in the ROADM space.

These trends drove the launch of the Open ROADM project in 2016 and the release of their first Multi-Source Agreement in 2017. The project aimed to disaggregate and open up these traditionally proprietary ROADM systems and make their provisioning and control more centralized through technologies such as software-defined networks (SDNs, explained in the diagram below).

Figure 2: In a traditional network paradigm (left), switches contain both the hardware that forwards traffic (the data plane), as well as the software that sets the rules of where to forward said traffic (the control plane). The SDN paradigm (right) separates the switching hardware from the software, effectively decoupling the data plane from the control plane. The language between this controller and the switches is an open protocol, with OpenFlow being the most commonly used one. The Open ROADM initiative proposed its own open protocol called NETCONF.

The Open ROADM project defined three disaggregated functions (pluggable optics, transponder, and ROADM), all controlled through an open standards-based API that could be accessed through an SDN controller. It defined 100G-400G interfaces for both Ethernet and Optical Transport Networking (OTN) protocols with a link reach of up to 500km. It also defined a stronger FEC algorithm called open FEC (oFEC) to support this reach. oFEC provides a greater enhancement than CFEC at the cost of more overhead and energy.

What Is OpenZR+?

The 400ZR agreement was primarily focused on addressing the needs of large-scale data center operators and their suppliers.

While it had some usefulness for telecom network operators, their transport network links usually span several hundreds of kilometers, so the interface and module power consumption defined in the 400ZR agreement could not handle such an extended reach. Besides, network operators needed extra flexibility when defining the transmission rate and the modulation type of their links.

Therefore, soon after the publication of the 400ZR agreement, the OpenZR+ Multi-Source Agreement (MSA) was published in September 2020. As the diagram below explains, this agreement can be seen as a combination of the 400ZR and Open ROADM standardization efforts.

Figure 3: Diagram of how different elements from the 400ZR and Open ROADM standardization projects were combined to form the OpenZR+ MSA. Diagram provided by the OpenZR+ website.

To better fit the telecom use cases of regional and long-haul transport links, OpenZR+ added a few changes to improve the link reach and flexibility over 400ZR:

Using the more powerful oFEC defined by the Open ROADM standard.
Multi-rate Ethernet that enables the multiplexing of 100G and 200G signals. This provides more options to optimize traffic in transport links.
Support for 100G, 200G, 300G, or 400G transport links using different modulation types (QPSK, 8QAM, or 16QAM). This enables further reach and capacity optimization for fiber links.
Higher dispersion compensation to make the fiber link more robust.

These changes allow QSFP-DD and OSFP modules to reach link lengths of up to 480 km (with optical amplifiers) at a 400G data rate. However, the FEC and dispersion compensation improvements that enable this extended reach come at the price of increased energy consumption. While the 400ZR standard targets a power consumption of 15 Watts, OpenZR+ standards aim for a power consumption of up to 20 Watts.

If operators need more performance, distances above 500 km, and support for OTN traffic (400ZR and OpenZR+ only support Ethernet), they must still use proprietary solutions, which are informally called 400ZR+. These 400ZR+ solutions feature larger module sizes (CFP2), higher performance proprietary FEC, and higher launch powers to achieve longer reach. These more powerful features come at the cost of even more power consumption, which can go up to 25 Watts.

Takeaways

The following table summarizes the use cases and characteristics of the approaches discussed in the article: 400ZR, Open ROADM, OpenZR+, and 400ZR+.

Technology	400ZR	Open ROADM	OpenZR+	400ZR+ Proprietary
Target Application	Edge Data Center Interconnects	Carrier ROADM Mesh Networks	Metro/Regional Carrier and Data Center Interconnects	Long-Haul Carrier
Target Reach @ 400G	120 km (amplified)	500 km (amplified)	480km (amplified)	1000 km (amplified)
Target Power Consumption	Up to 15 W	Up to 25 W	Up to 20 W	Up to 25W
Typical Module Option	QSFP-DD/OSFP	CFP2	QSFP-DD/OSFP	CFP2
Client Interface	400G Ethernet	100-400G Ethernet and OTN	100-400G Ethernet (Multi-rate)	100-400G Ethernet and OTN
Modulation Scheme	16QAM	QPSK, 8QAM, 16QAM	QPSK, 8QAM, 16QAM	QPSK, 8QAM, 16QAM
Forward Error Correction	CFEC	oFEC	oFEC	Proprietary
Standards / MSA	OIF	Open ROADM MSA	OpenZR+ MSA	Proprietary

400ZR is an agreement primarily focused on the needs of data center interconnects across distances of 80 – 120 kilometers. On the other hand, OpenROADM and OpenZR+ focused on addressing the needs of telecom carrier links, supporting link lengths of up to 500 km. These differences in reach are also reflected in the power consumption specs and the module form factors typically used. The 400ZR and OpenZR+ standards can only handle Ethernet traffic, while the Open ROADM and 400ZR+ solutions can handle both Ethernet and OTN traffic.

Tags: 400ZR, Data center, demand, disaggregation, extend, fiber, interoperable, Open ROADM, power, size, ZR+

Core Industries

Emerging Industries

Our Technology

Coherent Pluggable Modules

Sub-Assemblies: pITLA

DSP

View All

Insights

Press Releases

In the News

Events

Careers

Knowledge Center

Company Info

Contact Us

Core Industries

Emerging Industries

Our Technology

Coherent Pluggable Modules

Sub-Assemblies: pITLA

DSP

View All

Insights

Press Releases

In the News

Events

Careers

Knowledge Center

Company Info

Contact Us