OpenAI Developes MRC: A Network Protocol for Predictable AI Training Performance

A new networking protocol called MRC (Multipath Reliable Connection) has been developed by OpenAI and its partners to address the growing challenge of network congestion in large-scale AI training clusters. The protocol, which is based on SRv6 (Segment Routing over IPv6), aims to provide predictable performance even in the presence of failures.

What Happened

The development of MRC is a significant breakthrough for the field of AI supercomputer networking. According to OpenAI, more than 900 million people use ChatGPT every week, and sustaining and improving those models at that scale requires a reliable and efficient network infrastructure. The company states its goal as "not just to build a fast network, but also to build one that delivers very predictable performance, even in the presence of failures, to keep training jobs moving."

MRC is built on top of SRv6, which provides a programmable network where the transport stack selects paths for each packet. This approach aligns with the model described in "SRv6: From 5G Networks to AI Infrastructure – A Journey of Innovation," which details SRv6's evolution from telecommunications to AI infrastructure.

Background and Context

The growth of large-scale AI training clusters has created new challenges for network infrastructure. As the size of these clusters increases, so does the complexity of their networking requirements. Network congestion, link, and device failures are common sources of delay and jitter in transfers, which can ripple through the entire job and cause GPUs to sit idle.

OpenAI's MRC protocol addresses this challenge by distributing packets across many paths and planes, avoiding flow-collision issues common in traditional ECMP-based deployments. This approach is made possible by SRv6's ability to give applications control over their network experience.

Why it Matters

The development of MRC has significant implications for the adult industry, which relies heavily on large-scale AI training clusters for tasks such as content moderation and recommendation systems. A reliable and efficient network infrastructure is crucial for these tasks, as any delays or failures can have a direct impact on the quality of service provided to end-users.

MRC's ability to provide predictable performance even in the presence of failures makes it an attractive solution for industries that require high uptime and low latency. The protocol's use of SRv6 also provides a programmable network, which allows for greater flexibility and customization in meeting specific networking requirements.

What Comes Next

The development of MRC is just the beginning of a new era in AI supercomputer networking. As more companies adopt this technology, we can expect to see significant improvements in the efficiency and reliability of large-scale AI training clusters.

Cisco, one of the primary architects of SRv6, has already started shipping products with large-scale deployments such as SoftBank. The company's role in driving the ongoing SRv6 AI backend work in the IETF will also continue to shape the future of this technology.

Key Facts

• MRC (Multipath Reliable Connection) is a new networking protocol developed by OpenAI and its partners.
• The protocol is based on SRv6 (Segment Routing over IPv6), which provides a programmable network where the transport stack selects paths for each packet.
• MRC aims to provide predictable performance even in the presence of failures, making it an attractive solution for industries that require high uptime and low latency.
• The protocol has already been used in production by OpenAI and Microsoft's largest training clusters.
• Cisco played a major role in driving the development of SRv6 and will continue to shape its future through ongoing work in the IETF.
• MRC is expected to have significant implications for industries that rely heavily on large-scale AI training clusters, such as content moderation and recommendation systems.