400ZR Archives - EFFECT Photonics

Pluggables have becomes mainstream in datacom applications.

Insights

February 8, 2023

A Story of Standards: From 400ZR and Open ROADM to OpenZR+

Coherent optical transmission has been crucial in addressing network operator problems for the last decade. In this time, coherent technology has expanded from a solution reserved only for premium long-distance links to one that impacts data center interconnects, metro, and access networks…

Coherent optical transmission has been crucial in addressing network operator problems for the last decade. In this time, coherent technology has expanded from a solution reserved only for premium long-distance links to one that impacts data center interconnects, metro, and access networks, as explained in the video below.

The development of the 400ZR standard by the Optical Internetworking Forum (OIF) proved to be a milestone in this regard. It was the result of several years of progress in electronic and photonic integration that enabled the miniaturization of 400G coherent systems into smaller pluggable form factors. With small enough modules to pack a router faceplate densely, the datacom sector could profit from an ideal solution for high-capacity data center interconnects. Telecom operators wanted to implement a similar solution in their metro links, so they combined the Open ROADM standardization initiative with the 400ZR initiative to develop the OpenZR+ agreement that better fits their use cases.

This article will elaborate on these standardization projects—400ZR, OpenROADM, and OpenZR+—and explain what use cases each was designed to tackle.

What Is 400G ZR?

To cope with growing bandwidth demands, providers wanted to implement the concept of IP over DWDM (IPoDWDM), in which tunable optics are integrated into the router. This integration eliminates the optical transponder shelf and the optics between the routers and DWDM systems, reducing the network capital expenditure (CAPEX). This is shown in the figure below.

Figure 1: Implementation of a traditional coherent transport architecture (left), with the router connected to a transponder line card and a Reconfigurable Optical Add/Drop Multiplexer (ROADM). In contrast, an IPoDWDM solution (right) has 400ZR pluggable transceivers that go directly into the router and use passive multiplexing filters. The IPoDWDM solution eliminates the line card and additional optics, reducing network CAPEX.

However, widely deploying IPoDWDM with coherent optics forced providers to face a router faceplate trade-off. Since DWDM modules have traditionally been much larger than the client optics, plugging DWDM modules into the router required sacrificing roughly half of the costly router faceplate capacity. This was unacceptable for datacom and telecom providers, who approached their suppliers and the Optical Internetworking Forum (OIF) about the need to develop a compact and interoperable coherent solution that addressed this trade-off.

These discussions and technology development led the OIF to publish the 400ZR implementation agreement in 2020. 400ZR specifies a (relatively) low-cost and interoperable 400 Gigabit coherent interface for a link with a single optical wavelength, using a double-polarization, 16-state quadrature amplitude modulation scheme (DP-16QAM). This modulation scheme uses sixteen different constellation points that arise from combining four different intensity levels and four phases. It doubles the usual 16-QAM transmission capacity by encoding information in two different polarizations of light.

The agreement specified a link reach of 80 kilometers without amplification and 120 kilometers with amplification. For forward error correction (FEC), the 400ZR standard supports a concatenated FEC (CFEC) method. CFEC combines inner and outer FEC codes to enhance the performance compared to a standard FEC code.

The 400ZR agreement does not specify a particular size or type of module, but its specifications targeted a footprint and power consumption that could fit in smaller modules such as the Quad Small Form-Factor Pluggable Double-Density (QSFP-DD) and Octal-Small Form-Factor Pluggable (OSFP). These form factors are small enough to provide the faceplate density that telecom and especially datacom operators need in their system architectures. So even if we often associate the 400ZR standard with QSFP-DD, other form factors, such as CFP2, can be used.

What Is Open ROADM?

In parallel with the 400ZR standardization efforts, telecom network operators had a different ongoing discussion.

Reconfigurable Optical Add-Drop Multiplexers (ROADMs) were a game-changer for optical communications when they entered the market in the 2000s. Before this technology, optical networks featured inefficient fixed routes and could not adapt to changes in traffic and demand. ROADMs allowed operators to remotely provision and manage their wavelength channels and bandwidth without redesigning the physical network infrastructure.

However, ROADMs were proprietary hardware with proprietary software. Changing the proprietary ROADM platform needed extensive testing and a lengthy integration process, so operators were usually reluctant to look for other platform alternatives. Besides, ROADMs still had several fixed, pre-defined elements that could have been configurable through open interfaces. This environment led to reduced competition and innovation in the ROADM space.

These trends drove the launch of the Open ROADM project in 2016 and the release of their first Multi-Source Agreement in 2017. The project aimed to disaggregate and open up these traditionally proprietary ROADM systems and make their provisioning and control more centralized through technologies such as software-defined networks (SDNs, explained in the diagram below).

Figure 2: In a traditional network paradigm (left), switches contain both the hardware that forwards traffic (the data plane), as well as the software that sets the rules of where to forward said traffic (the control plane). The SDN paradigm (right) separates the switching hardware from the software, effectively decoupling the data plane from the control plane. The language between this controller and the switches is an open protocol, with OpenFlow being the most commonly used one. The Open ROADM initiative proposed its own open protocol called NETCONF.

The Open ROADM project defined three disaggregated functions (pluggable optics, transponder, and ROADM), all controlled through an open standards-based API that could be accessed through an SDN controller. It defined 100G-400G interfaces for both Ethernet and Optical Transport Networking (OTN) protocols with a link reach of up to 500km. It also defined a stronger FEC algorithm called open FEC (oFEC) to support this reach. oFEC provides a greater enhancement than CFEC at the cost of more overhead and energy.

What Is OpenZR+?

The 400ZR agreement was primarily focused on addressing the needs of large-scale data center operators and their suppliers.

While it had some usefulness for telecom network operators, their transport network links usually span several hundreds of kilometers, so the interface and module power consumption defined in the 400ZR agreement could not handle such an extended reach. Besides, network operators needed extra flexibility when defining the transmission rate and the modulation type of their links.

Therefore, soon after the publication of the 400ZR agreement, the OpenZR+ Multi-Source Agreement (MSA) was published in September 2020. As the diagram below explains, this agreement can be seen as a combination of the 400ZR and Open ROADM standardization efforts.

Figure 3: Diagram of how different elements from the 400ZR and Open ROADM standardization projects were combined to form the OpenZR+ MSA. Diagram provided by the OpenZR+ website.

To better fit the telecom use cases of regional and long-haul transport links, OpenZR+ added a few changes to improve the link reach and flexibility over 400ZR:

Using the more powerful oFEC defined by the Open ROADM standard.
Multi-rate Ethernet that enables the multiplexing of 100G and 200G signals. This provides more options to optimize traffic in transport links.
Support for 100G, 200G, 300G, or 400G transport links using different modulation types (QPSK, 8QAM, or 16QAM). This enables further reach and capacity optimization for fiber links.
Higher dispersion compensation to make the fiber link more robust.

These changes allow QSFP-DD and OSFP modules to reach link lengths of up to 480 km (with optical amplifiers) at a 400G data rate. However, the FEC and dispersion compensation improvements that enable this extended reach come at the price of increased energy consumption. While the 400ZR standard targets a power consumption of 15 Watts, OpenZR+ standards aim for a power consumption of up to 20 Watts.

If operators need more performance, distances above 500 km, and support for OTN traffic (400ZR and OpenZR+ only support Ethernet), they must still use proprietary solutions, which are informally called 400ZR+. These 400ZR+ solutions feature larger module sizes (CFP2), higher performance proprietary FEC, and higher launch powers to achieve longer reach. These more powerful features come at the cost of even more power consumption, which can go up to 25 Watts.

Takeaways

The following table summarizes the use cases and characteristics of the approaches discussed in the article: 400ZR, Open ROADM, OpenZR+, and 400ZR+.

Technology	400ZR	Open ROADM	OpenZR+	400ZR+ Proprietary
Target Application	Edge Data Center Interconnects	Carrier ROADM Mesh Networks	Metro/Regional Carrier and Data Center Interconnects	Long-Haul Carrier
Target Reach @ 400G	120 km (amplified)	500 km (amplified)	480km (amplified)	1000 km (amplified)
Target Power Consumption	Up to 15 W	Up to 25 W	Up to 20 W	Up to 25W
Typical Module Option	QSFP-DD/OSFP	CFP2	QSFP-DD/OSFP	CFP2
Client Interface	400G Ethernet	100-400G Ethernet and OTN	100-400G Ethernet (Multi-rate)	100-400G Ethernet and OTN
Modulation Scheme	16QAM	QPSK, 8QAM, 16QAM	QPSK, 8QAM, 16QAM	QPSK, 8QAM, 16QAM
Forward Error Correction	CFEC	oFEC	oFEC	Proprietary
Standards / MSA	OIF	Open ROADM MSA	OpenZR+ MSA	Proprietary

400ZR is an agreement primarily focused on the needs of data center interconnects across distances of 80 – 120 kilometers. On the other hand, OpenROADM and OpenZR+ focused on addressing the needs of telecom carrier links, supporting link lengths of up to 500 km. These differences in reach are also reflected in the power consumption specs and the module form factors typically used. The 400ZR and OpenZR+ standards can only handle Ethernet traffic, while the Open ROADM and 400ZR+ solutions can handle both Ethernet and OTN traffic.

Tags: 400ZR, Data center, demand, disaggregation, extend, fiber, interoperable, Open ROADM, power, size, ZR+

Insights

January 18, 2023

Reaching a 100ZR Future for Access Network Transport

In the optical access networks, the 400ZR pluggables that have become mainstream in datacom applications…

In the optical access networks, the 400ZR pluggables that have become mainstream in datacom applications are too expensive and power-hungry. Therefore, operators are strongly interested in 100G pluggables that can house coherent optics in compact form factors, just like 400ZR pluggables do. The industry is labeling these pluggables as 100ZR.

A recently released Heavy Reading survey revealed that over 75% of operators surveyed believe that 100G coherent pluggable optics will be used extensively in their edge and access evolution strategy. However, this interest had yet to materialize into a 100ZR market because no affordable or power-efficient products were available. The most the industry could offer was 400ZR pluggables that were “powered-down” for 100G capacity.

By embracing smaller and more customizable light sources, new optimized DSP designs, and high-volume manufacturing capabilities, we can develop native 100ZR solutions with lower costs that better fit edge and access networks.

Making Tunable Lasers Even Smaller?

Since the telecom and datacom industries want to pack more and more transceivers on a single router faceplate, integrable tunable laser assemblies (ITLAs) must maintain performance while moving to smaller footprints and lower power consumption and cost.

Fortunately, such ambitious specifications became possible thanks to improved photonic integration technology. The original 2011 ITLA standard from the Optical Internetworking Forum (OIF) was 74mm long by 30.5mm wide. By 2015, most tunable lasers shipped in a micro-ITLA form factor that cut the original ITLA footprint in half. In 2021, the nano-ITLA form factor designed for QSFP-DD and OSFP modules had once again cut the micro-ITLA footprint almost in half.

Figure-Evolution-laser — Figure 1: Evolution of tunable laser form factors for coherent optics (2011-2021). Figure inspired from a picture in Laser Focus World.

There are still plenty of discussions over the future of ITLA packaging to fit the QSFP28 form factors of these new 100ZR transceivers. For example, every tunable laser needs a wavelength locker component that stabilizes the laser’s output regardless of environmental conditions such as temperature. Integrating that wavelength locker component with the laser chip would help reduce the laser package’s footprint.

Another potential future to reduce the size of tunable laser packages is related to the control electronics. The current ITLA standards include the complete control electronics on the laser package, including power conversion and temperature control. However, if the transceiver’s main board handles some of these electronic functions instead of the laser package, the size of the laser package can be reduced.

This approach means that the reduced laser package would only have full functionality if connected to the main transceiver board. However, some transceiver developers will appreciate the laser package reduction and the extra freedom to provide their own laser control electronics.

Co-designing DSPs for Energy Efficiency

The 5 Watt-power requirement of 100ZR in a QSFP28 form factor is significantly reduced compared to the 15-Watt specification of 400ZR transceivers in a QSFP-DD form factor. Achieving this reduction requires a digital signal processor (DSP) specifically optimized for the 100G transceiver.

Current DSPs are designed to be agnostic to the material platform of the photonic integrated circuit (PIC) they are connected to, which can be Indium Phosphide (InP) or Silicon. Thus, they do not exploit the intrinsic advantages of these material platforms. Co-designing the DSP chip alongside the PIC can lead to a much better fit between these components.

To illustrate the impact of co-designing PIC and DSP, let’s look at an example. A PIC and a standard platform-agnostic DSP typically operate with signals of differing intensities, so they need some RF analog electronic components to “talk” to each other. This signal power conversion overhead constitutes roughly 2-3 Watts or about 10-15% of transceiver power consumption.

Figure-dac grey blocks — Figure : Simplified diagram of the building blocks of a coherent transceiver in which the DSP drives the optical engine directly from its digital-to-analog converter (DAC). This setup can minimize the power conversion overhead of optical transceivers.

However, the modulator of an InP PIC can run at a lower voltage than a silicon modulator. If this InP PIC and the DSP are designed and optimized together instead of using a standard DSP, the PIC could be designed to run at a voltage compatible with the DSP’s signal output. This way, the optimized DSP could drive the PIC directly without needing the RF analog driver, doing away with most of the power conversion overhead we discussed previously.

Graph-DSP-power — Figure 3: Comparison between the drive power and power consumption of three different PIC + DSP pairings: standard DSP with a silicon PIC, standard DSP with an InP PIC, and an optimized DSP with an InP.

Optical Subassemblies That Leverage Electronic Ecosystems

To become more accessible and affordable, the photonics manufacturing chain can learn from electronics packaging, assembly, and testing methods that are already well-known and standardized. After all, building a special production line is much more expensive than modifying an existing production flow.

There are several ways in which photonics packaging, assembly, and testing can be made more affordable and accessible: passive alignments of the optical fiber, BGA-style packaging, and flip-chip bonding. Making these techniques more widespread will make a massive difference in photonics’ ability to scale up and become as available as electronics. To read more about them, please read our previous article.

Takeaways

The interest in novel 100ZR coherent pluggable optics for edge and access applications is strong, but the market has struggled to provide “native” and specific 100ZR solutions to address this interest. Transceiver developers need to embrace several new technological approaches to develop these solutions. They will need smaller tunable laser packages that can fit the QSFP28 form factors of 100ZR solutions, optimized and co-designed DSPs that meet the reduced power consumption goals, and sub-assemblies that leverage electronic ecosystems for increased scale and reduced cost.

Tags: 100 ZR, 100G, 100ZR, 400ZR, C-band, C-PON, coherent, DSP, filters, future proof, Korea, laser sources, O-band, Packaging, pluggable, roadmap, S-band

The Light Path to a Coherent Cloud Edge Banner

Insights

May 25, 2022

Improving Edge Computing with Coherent Optical Systems on Chip

Smaller data centers placed locally have the potential to minimize latency, overcome inconsistent connections, and…

Smaller data centers placed locally have the potential to minimize latency, overcome inconsistent connections, and store and compute data closer to the end-user. These benefits are causing the global market for edge data centers to explode, with PWC predicting that it will nearly triple from $4 billion in 2017 to $13.5 billion in 2024. Cloud-native applications are driving the construction of edge infrastructure and services. However, they cannot distribute their processing capabilities without considerable investments in real estate, infrastructure deployment, and management.

This situation leads to hyperscalers cooperating with telecom operators to install their servers in the existing carrier infrastructure. For example, Amazon Web Services (AWS) is implementing edge technology in carrier networks and company premises (e.g., AWS Wavelength, AWS Outposts). Google and Microsoft have strategies and products that are very similar. In this context, edge computing poses a few problems for telecom providers too. They must manage hundreds or thousands of new nodes that will be hard to control and maintain.

At EFFECT Photonics, we believe that coherent pluggables with an optical System-on-Chip (SoC) can become vital in addressing these datacom and telecom sector needs and enabling a new generation of distributed data center architectures. Combining the optical SoCs with reconfigurable DSPs and modern network orchestration and automation software will be a key to deploying edge data centers.

Edge data centers are a performance and sustainability imperative

Various trends are driving the rise of the edge cloud:

5G technology and the Internet of Things (IoT): These mobile networks and sensor networks need low-cost computing resources closer to the user to reduce latency and better manage the higher density of connections and data.
Content delivery networks (CDNs): The popularity of CDN services continues to grow, and most web traffic today is served through CDNs, especially for major sites like Facebook, Netflix, and Amazon. By using content delivery servers that are more geographically distributed and closer to the edge and the end user, websites can reduce latency, load times, and bandwidth costs as well as increasing content availability and redundancy.
Software-defined networks (SDN) and Network function virtualization (NFV). The increased use of SDNs and NFV requires more cloud software processing.
Augment and virtual reality applications (AR/VR): Edge data centers can reduce the streaming latency and improve the performance of AR/VR applications.

Several of these applications require lower latencies than before, and centralized cloud computing cannot deliver those data packets quickly enough. As shown in Table 1, a data center on a town or suburb aggregation point could halve the latency compared to a centralized hyperscale data center. Enterprises with their own data center on-premises can reduce latencies by 12 to 30 times compared to hyperscale data centers.

Type of Edge		Data center	Location	Number of DCs per 10M people	Average Latency	Size
On-premises edge		Enterprise site	Businesses	NA	2-5 ms	1 rack max
Network (Mobile)	Tower edge	Tower	Nationwide	3000	10 ms	2 rack max
	Outer edge	Aggrega- tion points	Town	150	30 ms	2-6 rack max
	Inner edge	Core	Major city	10	40 ms	10+ rack max
Regional edge		Regional	Major city	100	50 ms	100+ racks
Not edge		Hyperscale	State/ national	1	60+ ms	5000+ racks

Table 1: Types of edge data centers and their characteristics. Source: STL Partners

Cisco estimates that 85 zettabytes of useful raw data were created in 2021, but only 21 zettabytes were stored and processed in data centers. Edge data centers can help close this gap. For example, industries or cities can use edge data centers to aggregate all the data from their sensors. Instead of sending all this raw sensor data to the core cloud, the edge cloud can process it locally and turn it into a handful of performance indicators. The edge cloud can then relay these indicators to the core, which requires a much lower bandwidth than sending the raw data.

Figure 1 Global data center traffic vs. useable data created per year. Graphic sourced from Cisco Global Cloud Index.

Edge data centers therefore allow more sensor data to be aggregated and processed to make systems worldwide smarter and more efficient. The ultimate goal is to create entire “smart cities” that use this sensor data to benefit their inhabitants, businesses, and the environment. Everything from transport networks to water supply and lightning could be improved if we have more sensor data available in the cloud to optimize these processes. Distributing data centers is also vital for future data center architectures. While centralizing processing in hyper-scale data centers made them more energy-efficient, the power grid often limits the potential location of new hyperscale data centers. Thus, the industry may have to take a few steps back and decentralize data processing capacity to cope with the strain of data center clusters on power grids. For example, data centers can be relocated to areas where spare power capacity is available, preferably from nearby renewable energy sources. EFFECT Photonics envisions a system of datacentres with branches in different geographical areas, where data storage and processing are assigned based on local and temporal availability of renewable (wind-, solar-) energy and total energy demand in the area.

Figure 2: High-speed fiber-optic connections allow data processing and storage to move to locations where excess (green) energy is available. If power is needed for other purposes, such as charging electric vehicles, data can be moved elsewhere

Coherent technology simplifies the scaling of edge data center interconnects

As edge data center interconnects became more common, the issue of how to interconnect them became more prominent. Direct detect technology had been the standard in the short-reach data center interconnects. However, reaching the distances greater than 50km and bandwidths over 100Gbps required for modern edge data center interconnects required external amplifiers and dispersion compensators that increased the complexity of network operations. At the same time, advances in electronic and photonic integration allowed longer reach coherent technology to be miniaturized into QSFP-DD and OSFP form factors. This progress allowed the Optical Internetworking Forum (OIF) to create the 400ZR and ZR+ standards for 400G DWDM pluggable modules. With small enough modules to pack a router faceplate densely, the datacom sector could profit from a 400ZR solution for high-capacity data center interconnects of up to 80km. If needed, extended reach 400ZR+ pluggables can cover several hundreds of kilometers. Cignal AI forecasts that 400ZR shipments will dominate in the edge applications, as shown in Figure 3.

Figure 3: Forecast of 100G port equivalents shipped for edge applications. These shipments are overwhelmingly 400ZR standard technology. Source: Cignal AI Transport Applications Report Q42021.

Further improvements in integration can further boost the reach and efficiency of coherent transceivers. For example, by integrating all photonic functions on a single chip, including lasers and optical amplifiers, EFFECT Photonics’ optical System-On-Chip (SoC) technology can achieve higher transmit power levels and longer distances while keeping the smaller QSFP-DD form factor, power consumption, and cost.

Maximizing Edge Computing with Automation

With the rise of edge data centers, telecom providers must manage hundreds or thousands of new nodes that will be hard to control and maintain. Furthermore, providers also need a flexible network with pay-as-you-go scalability that can handle future capacity needs. Fortunately, several new technologies are enabling this scalable and automated network management.

First of all, the rise of self-tuning algorithms has made the installation of new pluggables easier than ever. They eliminate additional installation tasks such as manual tuning and record verification. They are host-agnostic, can plug into any third-party host equipment, and scale as you grow. Standardization also allows modules from different vendors to communicate with each other, avoiding compatibility issues and simplifying upgrade choices. The communication channels used for self-tuning algorithms can also be used for remote diagnostics and management, such as the case of EFFECT Photonics NarroWave technology.

Automation potential improves further by combining artificial intelligence with the software-defined networks (SDNs) framework that virtualizes and centralizes network functions. This creates an automated and centralized management layer that can allocate resources efficiently and dynamically. For example, AI in network management will become a significant factor in reducing the energy consumption of future telecom networks.

Figure 2: Comparing a traditional network approach (left) with an SDN/NFC approach (right).

Future smart transceivers with reconfigurable digital signal processors (DSPs) can give the AI-controlled management layer even more degrees of freedom to optimize the network. These smart transceivers will relay more device information for diagnosis, and depending on the management layer instructions, they can change their coding schemes to adapt to different network requirements

Takeaways

Cloud-native applications require edge data centers with lower latency, and that better fit the existing power grid. However, their implementation came with the challenges of more data center interconnects and a massive increase in nodes to manage. Fortunately, coherent pluggables with self-tuning can play a vital role in addressing these datacom and telecom sector challenges and enabling a new generation of distributed data center architectures. Combining these pluggables with modern network orchestration and automation software will boost the deployment of edge data centers. EFFECT Photonics believes that with these automation technologies (self-tuning, SDNs, AI), we can reach the goal of a self-managed, zero-touch automated network that can handle the massive scale-up required for 5G networks and edge computing.

Tags: 400ZR, artificial intelligence, cloud, coherent, computing, data centers, DSP, edge, edge data centers, infrastructure, latency, network, network edge, operators, optical system-on-chip, pluggables, self-tuning, services

Telecom/Datacom

Emerging Industries

DSP & FEC

Ultra Pure Light Sources

Ecosystems

Pluggable Modules – Direct Detect

Pluggable Modules – Coherent

Sub-Assemblies

ASICs

View All

Insights

Press Releases

In the News

Events

Careers

Knowledge Center

Company Info

Contact Us

Telecom/Datacom

Emerging Industries

DSP & FEC

Ultra Pure Light Sources

Ecosystems

Pluggable Modules – Direct Detect

Pluggable Modules – Coherent

Sub-Assemblies

ASICs

View All

Insights

Press Releases

In the News

Events

Careers

Knowledge Center

Company Info

Contact Us