Data center interconnects Archives

Insights

June 5, 2024

Towards the 800ZR Future

The advances in electronic and photonic integration allowed coherent technology for metro DCIs to be…

The advances in electronic and photonic integration allowed coherent technology for metro DCIs to be miniaturized into QSFP-DD and OSFP form factors. This progress allowed the Optical Internetworking Forum (OIF) to create the 400ZR and ZR+ standards for 400G DWDM pluggable modules.

Following the success of 400ZR standardization, the industry quickly recognized the need for even higher-capacity solutions. In December 2020, the Optical Internetworking Forum (OIF) announced the initiation of the 800G Coherent project. This project aimed to define interoperable 800 Gbps coherent line specifications for various applications, including amplified DWDM links up to 120 km and unamplified fixed wavelength links of 2-10 km.

After the success of 400ZR standardization, the data center industry and the OIF are starting to promote an 800ZR standard to enable the next generation of interconnects. In OFC 2024, we started seeing some demos from several vendors and the OIF on this new standards initiative.

Coherent or Direct Detect for Data Centers

While coherent technology has become the dominant one for interconnecting data centers over long distances (80 km upwards), the campus sector and the inside of the data center continue to be dominated by direct detection technologies such as PAM-4.

However, as data rates rise into 800Gbps and beyond, the power consumption of coherent technology is expected to come much closer to that of direct detect PAM-4 solutions, as shown in the figure below. This can make coherent technology competitive for campus interconnects while direct detect technology remains dominant inside the data center.

A major reason for this decreased gap is that direct detect technology often requires additional amplifiers and compensators at these data rates, while coherent pluggables do not. This also makes coherent technology simpler to deploy and maintain. Furthermore, as the volume of coherent transceivers produced increases, their prices can also go down.

While the 800ZR standard is focused on longer-distance metro interconnects, efforts have also been made to develop a coherent 800G short-reach(SR) standard. Even if these efforts are unsuccessful in this generation of transceivers, they can lay the groundwork for using coherent technology in short-reach once terabit links become the new standard.

The Challenge of Power

LightCounting forecasts significant growth in dense-wavelength division multiplexing (DWDM) port shipments with data rates of 600G, 800G, and beyond in the next five years.

Graph-pluggables-shipments-dwdm — Figure 2: Shipments of high-speed DWDM ports by data rate (historical data and forecast). Source: LightCounting Optical Communications Market Forecast, April 2022.

The major obstacles in this roadmap remain transceivers’ power consumption, thermal management, and affordability. Over the last two decades, power ratings for pluggable modules have increased as we moved from direct detection to more power-hungry coherent transmission: from 2W for SFP modules to 3.5 W for QSFP modules and now to 14W for QSSFP-DD and 21.1W for OSFP form factors. Rockley Photonics researchers estimate that a future electronic switch filled with 800G modules would draw around 1 kW of power just for the optical modules.

Thus, many incentives exist to improve pluggable optical transceivers’ performance and power consumption. By embracing increased photonic integration, co-designed PICs and DSPs, and more compact lasers, pluggables will be better able to scale in data rates while remaining affordable and low-power.

What Does This Mean for Lasers and DSPs?

The new generation of 800ZR pluggable transceivers can leverage 5nm process technology for their digital signal processors (DSPs), offering significant advancements over the 7nm technology used in 400ZR transceivers. A primary difference between these two processes lies in the size of the transistors used to build the chips: the 5nm process technology uses smaller transistors than the 7nm process. This size reduction allows more transistors to be packed into the same silicon area, enhancing the chip’s performance and energy efficiency.

For example, this enhanced transistor density facilitates higher baud rates, a critical factor for data transmission. The 800ZR modules will operate at baud rates of 120 GBaud, which doubles the 60 GBaud used in 400ZR transceivers. The increased power efficiency also makes these 800ZR transceivers more suitable for data center environments.

Regarding lasers for 800ZR, the mandate continues to be the same as always: develop the most powerful laser possible with the smallest footprint. Since 800G transmission naturally leads to higher transmission losses than 400G transmission, higher laser power is necessary to compensate for these losses and achieve the required reach. Meanwhile, the smaller laser size helps with thermal management inside the 800ZR module, and high laser power provides a higher link budget.

Takeaways

Following the success of the 400ZR standard, the Optical Internetworking Forum (OIF) quickly moved on to developing 800ZR standards for data center interconnects. The deployment of 800ZR technology promises substantial bandwidth and system performance enhancements but also poses a sustainable solution to the escalating power and thermal challenges modern data centers face. The use of 5nm process technology for DSPs and small but high-performance lasers within these modules will be vital to achieve the required power efficiency and performance.

PDF Download

Tags: 5nm DSPs, 800Gbps, 800ZR technology, Coherent Transceivers, compact lasers, Data center interconnects, DWDM ports, EFFECT Photonics, high-speed transmission, Optical Internetworking Forum, photonic integration, power management, standards

Insights

May 8, 2024

The Evolution of Data Center Interconnects

The digital era’s rapid expansion requires advances in data center interconnects (DCIs) to support the…

The digital era’s rapid expansion requires advances in data center interconnects (DCIs) to support the burgeoning demands of cloud computing and data architecture.

For the sake of this article, let’s think broadly about three categories of data center interconnects based on their reach and location with relation to the data center

6 data center interconnect — Figure 1: Example of different data center interconnect (DCI) links. Intra-data center interconnects usually have lengths up to 2kms, campus interconnects up to 10km, and metro DCIs up to 100km, although there can be longer DCIs.

Intra-data center interconnects (< 2km)
Campus data center interconnects (<10km)
Metro data center interconnects (<100km)

As data centers become more complex and AI increases its demands on them, the intra-data center sector is increasing in complexity and variety, but that will be the subject of a different article.

Coherent optical technology has established itself as the go-to solution for interconnecting data centers over longer distances, while direct detect continues to dominate the intra data center sector.

The Increasing Importance of Decentralizing Data Centers

Smaller data centers placed locally have the potential to minimize latency, overcome inconsistent connections, and store and compute data closer to the end-user. These benefits are causing the global market for edge data centers to explode, with PWC predicting that it will nearly triple from $4 billion in 2017 to $13.5 billion in 2024.

Meanwhile, the demands of new AI infrastructure are pushing data center power consumption to such a degree that the electrical power grid might be unable to sustain it. Longer data center interconnects enable a more decentralized system of data centers with branches in different geographical areas connected through high-speed optical fiber links to cope with the strain of data center clusters on power grids.

These trends push the data center industry to look for interoperable solutions for longer interconnects over distances of 80 to 120 km. The advances in electronic and photonic integration allowed coherent technology for metro DCIs to be miniaturized into QSFP-DD and OSFP form factors. This progress allowed the Optical Internetworking Forum (OIF) to create the 400ZR and ZR+ standards for 400G DWDM pluggable modules. With small enough modules to pack a router faceplate densely, the datacom sector could profit from a 400ZR solution for high-capacity data center interconnects of up to 80km.

After the success of 400ZR standardization, the data center industry and the OIF are starting to promote an 800ZR standard to enable the next generation of interconnects. In OFC 2024, we started seeing some demos from several vendors and the OIF on this new standards initiative.

Direct Detect and Coherent Technology in DCIs

For many years, there has been an expectation that the increasing capacity demands on data centers would reach a point where coherent technology would deliver a lower cost per bit and power per bit than direct detect technologies.

However, direct detect technology (both NRZ and PAM-4) continues to successfully overcome the challenges of coherent technology and will continue to dominate the intra-DCI space (also called data center fabric) in the coming years. In this space, links span less than two kilometers, and for particularly short links (< 300 meters), affordable multimode fiber (MMF) is frequently used.

The tendency towards data center decentralization will also impact the inside of data centers. Larger data centers can require longer interconnects from one building to another. So in a way, some links inside the data centers could become more like campus DCI links, which require single-mode fiber solutions, and in those spaces coherent might have a better chance of becoming competitive.

Takeaways

The decision to use coherent or direct detection technology for DCIs boils down to the reach and capacity needs. Coherent is already established as the solution for metro DCIs, and efforts are underway towards an 800ZR standard to follow up on the highly successful 400ZR interconnect standard. With the move to Terabit speeds and scaling production volumes, it was expected to become competitive in the data center sector, too, but for now, direct detect technology continues to dominate this sector.

From basic networking to sophisticated, AI-enhanced architectures, DCIs have become the backbone of the digital infrastructure, enabling the seamless operation of global cloud services and data centers.

Article PDF

Tags: 400ZR, 800ZR, AI, Campus interconnects, cloud computing, coherent optical technology, data architecture, Data center interconnects, DCIs, decentralizing data centers, digital era, Direct Detect, edge data centers, EFFECTPhotonics, electrical power grid, global cloud services, intra-data center, metro DCIs, multimode fiber, NRZ, OFC 2024, PAM-4, photonic integration, power consumption, single-mode fiber, Terabit speeds

Insights

February 7, 2024

What is Laser Linewidth and Why Does it Matter

The world is moving towards tunability. The combination of tunable lasers and dense wavelength division…

The world is moving towards tunability. The combination of tunable lasers and dense wavelength division multiplexing (DWDM) allows the datacom and telecom industries to expand their network capacity without increasing their existing fiber infrastructure. Furthermore, the miniaturization of coherent technology into pluggable transceiver modules has enabled the widespread implementation of IP over DWDM solutions. Self-tuning algorithms have also made DWDM solutions more widespread by simplifying installation and maintenance. Hence, many application cases—metro transport, data center interconnects, and —are moving towards tunable pluggables.

The tunable laser is a core component of all these tunable communication systems, both direct detection and coherent. The laser generates the optical signal modulated and sent over the optical fiber. Thus, the purity and strength of this signal will have a massive impact on the bandwidth and reach of the communication system.

What is laser linewidth?

Coherent systems encode information in the phase of the light, and the purer the light source is, the more information it can transmit. An ideal, perfectly pure light source can generate a single, exact color of light. However, real-life lasers are not pure and will generate light outside their intended color. The size of this deviation is what we call the laser linewidth. In other words, the linewidth describes the range of wavelengths present in the wavelength spectrum of the laser beam.

The linewidth of a laser can be defined in different ways depending on the specific criteria used. Here are a few examples:

Full Width at Half Maximum (FWHM): This is a common and straightforward definition. It refers to the width of the laser spectrum at the point where the intensity is half its maximum.
Gaussian Linewidth: In some cases, the linewidth can be characterized by the standard deviation of a Gaussian distribution that fits the spectral profile of the laser output.
Schawlow-Townes Linewidth: This definition is associated with the quantum noise of the laser. You could consider this the fundamental, smallest possible linewidth an “ideal” laser could have.
Lorentzian Linewidth: The Lorentzian linewidth is based on the Lorentzian distribution, often used to model the spectral lines of certain lasers.
Frequency or Wavelength Range: Instead of using a specific criterion like FWHM, some applications may define linewidth by specifying the frequency or wavelength range within which a certain percentage (e.g., 95%) of the total power is contained.

These different definitions may be more suitable for specific contexts or applications, depending on the requirements and characteristics of the laser system in question.

What Impact Does Linewidth Have on Coherent Transmission?

A laser is a very precise generator of light signals. Phase noise is like a tiny, random wobble or instability in the timing of these signals. It’s as if the laser can’t decide exactly when to start and stop its light output, creating a small amount of uncertainty in the timing. Precise timing is everything for communication applications.

An impure laser with a large linewidth will have a more unstable phase that propagates errors in its transmitted data, as shown in the diagram below. This means it will transmit at a lower speed than desired.

whats inside a tunable laser Figure-2 — Figure 1: Impact of laser linewidth on phase noise during coherent transmission. The diagram shows the points transmitted by a coherent system at different laser line widths (measured by frequency). The larger the laser linewidth, the blurrier the transmitted points become, to the point that they cannot be distinguished from each other—source: Ragheb et al. (2017).

What are Some Ways to Reduce Laser Linewidth and Noise?

One of the most straightforward ways to improve the linewidth of a semiconductor laser is to use it inside a second, somewhat larger resonator. This setup is called an external cavity laser (ECL) since this new resonator or cavity will use additional optical elements external to the original laser.

Figure 2 : Schematic of an external cavity laser. A typical semiconductor laser is the “base laser” (it could even be a DFB or DBR). The external cavity around that base laser is built with additional elements such as lenses, mirrors, or gratings for additional wavelength tuning.

The new external resonator also provides more degrees of freedom for tuning the laser. ECLs have become the state-of-the-art solution in the telecom industry: they use a DFB or DBR laser as the “base laser” and external gratings as their filtering element for additional tuning. These lasers can provide a high-quality laser beam with low noise, narrow linewidth, and a wide tuning range. However, they came with a cost: manufacturing complexity.

EFFECT Photonics takes a very different approach to building lasers. Most developers make their lasers using linear resonators in which the laser light bounces back and forth between two mirrors. However, EFFECT Photonics uses ring resonators, which take a different approach to feedback: the light loops multiple times inside a ring that contains the active medium. The ring is coupled to the rest of the optical circuit via a waveguide.

Figure 5: Comparison between (a) a linear laser and (b) a ring laser. Both of these diagrams show the electrical p- and n- contacts. — Figure 3: Comparison between (a) a linear laser and (b) a ring laser. Both of these diagrams show the electrical p- and n- contacts.

The power of the ring resonator lies in its compactness, flexibility, and integrability. While a single ring resonator is not that impressive or tunable, using multiple rings and other optical elements allows them to achieve linewidth and tunability on par with the state-of-the-art tunable lasers that use linear resonators.

Takeaways

Laser linewidth, which describes the range of wavelengths in the laser beam, is paramount in coherent optical transmission systems. In such systems, where information is encoded in the phase of light, a purer light source allows for transmitting more information. A narrower laser linewidth corresponds to a more stable phase, reducing phase noise and enhancing the signal quality.

Techniques such as external cavity lasers (ECL) have been employed to improve linewidth, offering a high-quality laser beam with low noise and narrow linewidth. Alternatively, EFFECT Photonics employs ring resonators, providing an innovative approach to achieving linewidth and tunability comparable to state-of-the-art tunable lasers while emphasizing compactness and integrability.

Download Article

Tags: Coherent technology, coherent transmission, Data center interconnects, Dense-wavelength division multiplexing (DWDM), EFFECT Photonics, External cavity laser (ECL), Frequency range, Full Width at Half Maximum (FWHM), Gaussian Linewidth, IP over DWDM, Laser linewidth, Lorentzian Linewidth, Metro transport, Phase noise, Pluggable transceiver modules, Ring resonators, Schawlow-Townes Linewidth, Self-tuning algorithms, tunable lasers, Wavelength range

Insights

January 17, 2024

An Intro to Data Center Interconnects

Data center interconnects (DCIs) refer to the networking technologies and solutions that enable seamless communication…

Data center interconnects (DCIs) refer to the networking technologies and solutions that enable seamless communication and data exchange between geographically dispersed data centers. As organizations increasingly rely on distributed computing resources and adopt cloud services, the need for efficient and high-speed connections between data centers becomes crucial. DCIs facilitate the transfer of data, applications, and workloads across multiple data center locations, ensuring optimal performance, redundancy, and scalability.

The Impact of Data Center Interconnects

The impact of robust data center interconnects on the operation of data centers is profound. Firstly, DCIs enhance overall reliability and availability by creating a resilient network infrastructure. In the event of a hardware failure or unexpected outage in one data center, DCIs enable seamless failover to another data center, minimizing downtime and ensuring continuous operations. This redundancy is vital for mission-critical applications and services.

Secondly, DCIs contribute to improved performance and reduced latency. By connecting data centers with high-speed, low-latency links, organizations can efficiently distribute workloads and resources, optimizing response times for users and applications. This is particularly important for real-time applications, such as video streaming, online gaming, and financial transactions.

Furthermore, DCIs support efficient data replication and backup strategies. Data can be synchronized across geographically distributed data centers, ensuring data integrity and providing effective disaster recovery solutions. This capability is crucial for compliance with regulatory requirements and safeguarding against data loss.

Types of Data Center Interconnects

As shown in the figure below, we can think about three categories of data center interconnects based on their reach

Intra-data center interconnects (< 2km)
Campus data center interconnects (<10km)
Metro data center interconnects (<100km)

Figure 1: Example of different data center interconnect (DCI) links. Intra-data center interconnects usually have lengths up to 2kms, campus interconnects up to 10km, and metro DCIs up to 100km, although there can be longer DCIs.

Intra-datacenter interconnects operate within a single data center facility .These interconnects are designed for short-distance communication within the same data center building or complex. Intra-DCIs are optimized for high-speed, low-latency connections between servers, storage systems, and networking devices within a single data center. They are crucial for supporting the internal communication and workload distribution required for efficient data center operations.

Campus DCIs connect multiple data centers but are typically limited to a campus area, which may include multiple buildings or locations within a relatively close proximity.This type of interconnect is suitable for organizations with distributed computing resources that need to be interconnected for redundancy, load balancing, and seamless failover within a campus environment.

Metro DCIs connect data centers that are located in different metropolitan areas or cities. They cover longer distances compared to intra-datacenter and campus interconnects, typically spanning tens of kilometers to a few hundred kilometers.

Metro DCIs are essential for creating a network of interconnected data centers across a metropolitan region. They facilitate data replication, disaster recovery, and business continuity by enabling seamless communication and resource sharing between data centers that may be geographically dispersed but still within a reasonable proximity.

The Rise of Edge Data Centers

Various trends are driving the rise of the edge cloud:

5G technology and the Internet of Things (IoT): These mobile networks and sensor networks need low-cost computing resources closer to the user to reduce latency and better manage the higher density of connections and data.
Content delivery networks (CDNs): The popularity of CDN services continues to grow, and most web traffic today is served through CDNs, especially for major sites like Facebook, Netflix, and Amazon. By using content delivery servers that are more geographically distributed and closer to the edge and the end user, websites can reduce latency, load times, and bandwidth costs as well as increasing content availability and redundancy.
Software-defined networks (SDN) and Network function virtualization (NFV). The increased use of SDNs and NFV requires more cloud software processing.
Augment and virtual reality applications (AR/VR): Edge data centers can reduce the streaming latency and improve the performance of AR/VR applications.

Several of these applications require lower latencies than before, and centralized cloud computing cannot deliver those data packets quickly enough. A data center on a town or suburb aggregation point could halve the latency compared to a centralized hyperscale data center. Enterprises with their own data center on-premises can reduce latencies by 12 to 30 times compared to hyperscale data centers.

Cisco estimates that 85 zettabytes of useful raw data were created in 2021, but only 21 zettabytes were stored and processed in data centers. Edge data centers can help close this gap. For example, industries or cities can use edge data centers to aggregate all the data from their sensors. Instead of sending all this raw sensor data to the core cloud, the edge cloud can process it locally and turn it into a handful of performance indicators. The edge cloud can then relay these indicators to the core, which requires a much lower bandwidth than sending the raw data.

Figure 2. Global data center traffic vs. useable data created per year. Graphic sourced from Cisco Global Cloud Index.

Takeaways

In conclusion, Data Center Interconnects (DCIs) play a pivotal role in fostering reliable, available, and high-performing data center operations, ensuring seamless communication and workload distribution. The categorization of DCIs into intra-data center, campus, and metro interconnects reflects their adaptability to various communication needs. The emergence of edge data centers, driven by 5G, IoT, CDNs, SDNs, NFV, and AR/VR applications, addresses the demand for low-latency computing resources.

Positioned strategically at aggregation points, edge data centers efficiently process and relay data, contributing to bandwidth optimization and closing the gap between raw data generation and traditional data center capacities. As organizations navigate a data-intensive landscape, the evolution of DCIs and the rise of edge data centers underscore their critical role in ensuring the seamless, efficient functioning of distributed computing ecosystems.

Article PDF

Tags: 5G technology, Backup strategies, Campus interconnects, Cloud services, Content delivery networks (CDNs), Data center interconnects, Data replication, DCIs, Distributed computing resources, edge data centers, EFFECT Photonics, High-speed connections, Internet of Things (IoT), Intra-data center interconnects, Latency reduction, Metro interconnects, Networking technologies, Performance optimization, Redundancy, Scalability, Software-defined networks (SDN).

Core Industries

Emerging Industries

Our Technology

Pluggable Modules – Coherent

Sub-Assemblies

ASICs

View All

Insights

Press Releases

In the News

Events

Careers

Knowledge Center

Company Info

Contact Us

Core Industries

Emerging Industries

Our Technology

Pluggable Modules – Coherent

Sub-Assemblies

ASICs

View All

Insights

Press Releases

In the News

Events

Careers

Knowledge Center

Company Info

Contact Us