AI datacenters are a rats nest of fiber optic cables stitching together thousands of AI accelerators, NICs, switches, and everything else required to train and run the latest and greatest modes. Cisco hopes to untangle this mess with a new pluggable optic developed in collaboration with Nvidia that will pack between four and eight times the bandwidth of existing pluggables into a single port. Announced at the Open Compute Project (OCP) global summit this week, the ultra-dense small form factor pluggable (USFP) promises to pack up to 32, 400 Gbps lanes into a pluggable roughly the size of existing OSFP modules. Each of these pluggables would boast 12.8 Tbps of Ethernet bandwidth, as much as a high-end enterprise switch just a few years ago. To put that in perspective, the most sophisticated pluggable optics today cap out at around 1.6 Tbps using eight 200 Gbps links. Such a pluggable has the potential to significantly simplify cabling across AI datacenters where it isn't uncommon for each GPU to be paired with an 800 Gbps and soon 1.6 Tbps NIC. A single USFP pluggable could consolidate eight GPUs worth of cabling into a single fiber optic bundle. Meanwhile, on the switch side, a future 204.8 Tbps switch would only require 16 USFP ports to support 128 1.6 Tbps endpoints. Cramming all that connectivity into a single pluggable won't be easy. Today a high-end pluggable optic might pull on the order of 20 watts. Cisco's USFP spec calls for up to 100 watts per pluggable. There are also design constraints that have to be taken into consideration. Four times as many links per pluggable means 4x as many electrical contacts that have to be perfectly seated. Because of this Cisco says USFP modules will employ a two phase insertion mechanism in which the pluggable is first inserted into the cage and then a second mechanism is actuated to make the electrical connection. Speaking of reliability, USFP would have a bigger blast radius than conventional pluggables. A single bad laser or digital signal processor (DSP) has the potential to take out far more than a single GPU, but entire systems. While that might sound bad, it's nothing compared to co-packed or near-package optics where one failure could drop an entire rack's worth of accelerators from the network. Nvidia's participation in the development of the USFP standard is notable as the Ethernet networking titan has largely moved away from pluggables in its most recent set of switches. Many of its Spectrum-X and Quantum-X switches now use co-packaged optics and for good reason. Cutting out the pluggables on the switch side nets a roughly 30 percent power savings. But while Nvidia's latest switches may use CPO, its NICs still rely on pluggables in part because it offers integrators greater flexibility. Not everyone is building 100,000-plus GPU clusters for training. For racks utilizing a top of rack switch, lower power direct attach copper cabling can be used without much issue. Given Nvidia's investments in CPO we can only speculate on its interests in USFP, but it likely comes down to densification. The AI infrastructure giant is rapidly moving toward 600 kW racks packing 144 GPUs on board. First teased in early 2025, Nvidia's Vera Rubin Ultra platform promised 115.2 TB/s scale out bandwidth per rack or about 3.2-6.4 Tbps of bandwidth per GPU package depending on whether they're counting bidirectional bandwidth or not. It just so happens that Cisco's initial USFP spec calls for 16, 400 Gbps links per pluggable, or about 6.4 Tbps of bandwidth. While Cisco and Nvidia may be leading the charge on the development of an ultra dense optical pluggable, like most things Ethernet, USFP will be an open standard anyone can use in their designs. Microsoft taps Cisco for multi-datacenter AI Alongside Cisco's latest tie up with Nvidia, Switchzilla also revealed that Microsoft is now using its Silicon One P200 routing chips to distribute workloads across multiple datacenters. Nvidia and others have taken to calling this kind of multi-site computing: scale across networking. We discussed Cisco's datacenter-scale routing platform at length late last year, but the silicon competes directly with products like Nvidia's Spectrum-XGS and Broadcom's Jericho 4-based offerings. Cisco touts the P200 as the first 51.2 Tbps systems "deployed in a production environment by leading hyperscalers." With enough qualifiers you can be the first at anything. As we reported at the time, Microsoft was among the first major hyperscalers to embrace scale-across networking as a means of overcoming local power constraints with its Fairwater clusters in Wisconsin and Georgia. And it seems we finally know the hardware good old Redmond is using to stitch them together. Cisco gives SONiC some love Speaking of hyperscalers, one thing you probably won't find in a hyperscale network is Cisco's Nexus network operating system (NOS). Hyperscalers like Microsoft typically employ vendor agnostic NOS platforms to avoid vendor lock-in and make managing networks across multiple Ethernet vendors cheaper and easier. Originally developed by Microsoft and later open sourced, SONiC is among the most popular and widely deployed of the hyperscale NOSes. SONiC support, we should note is nothing new for Cisco which contrary to popular belief is more than willing to sell you any one piece of its networking stack, assuming you've got a big enough purse — the hyperscalers certainly do. As such Cisco is one of the biggest contributors to the SONiC project. Instead of forcing customers to use its software stack, Cisco's approach to SONiC and the hyperscale is to extend the NOSes capabilities to better take advantage of ASIC's unique capabilities and allow for integration with its services. At OCP this week, Cisco announced support for its Live Protect platform in SONiC to make it easier for customers to keep up with security patches in the wild. This capacity is expected to roll out next year. ®

If you're encountering some odd IPv6 networking issues, the culprit might be the Intel Connectivity Performance Suite (ICPS), although the chipmaker has kept quiet about the problem. A Samsung Galaxy Book owner contacted The Register about some decidedly odd networking behavior that they eventually traced to unwanted ICPS optimizations. And by "optimization", we mean "borked networking." For our reader, the problem showed up in OneDrive and OneNote. Notebooks were showing as empty, even though they were reported as "Up to date" according to the software. Microsoft couldn't get to the bottom of it. A bit of sleuthing by our reader revealed that something was borking the IPv6 packet. Removing Samsung's ICPS build (40.25.926.0) and installing the generic release (50.26.623.243) fixed the problem. Our reader told us, "From the user's side it looks like random 'connection reset' errors on all kinds of sites, plus OneNote saying 'We can't get to your notebooks right now.' "IPv4 is unaffected, which is why the web version and my phone kept working and nobody could figure out what was going on." It isn't only Samsung and our reader affected. Intel's own Community forums shows other users encountering the same glitch. An ASUS user put together an exhaustive report on the issue (IPv6 connection woes while IPv4 worked well). In this instance, ICPS version 40.25.725.165 was in use, and the problem went away following uninstallation. The user asked whether there was an official way to opt out of ICPS permanently, and the Intel Customer Support Technician responded: "Good news!… We can confirm that the Wireless driver and ICPS are available as separate standalone driver packages." So in that particular instance, ditching ICPS would make no difference. Which, of course, raises another question – in this example (where the Intel BE201 Wi-Fi adaptor is present), why is it even there? The answer: "It is often bundled with OEM systems as an added value software to enhance the overall wireless experience." You might see "enhance," but for users affected by IPv6 problems, it's more like "break." The issue is insidious, and broken ICPS builds are still floating around, ready to catch users off guard. According to our reader, Microsoft's own Update Catalog will hand out the broken drivers, which effectively break Microsoft 365 services on certain hardware. For the ASUS user, the issue affected Chrome, Edge, and Windows curl.exe. "Approximately 40–60 percent of IPv6 HTTPS connections to an AWS CloudFront distribution were reset. IPv4 was completely stable." According to our reader, "Intel created the bug, then fixed it quietly." Updating to the later, generic version appears to fix things. However, according to the Intel technician, as of the July post, "we do not have a confirmed public advisory for this specific behavior." We asked Intel and Microsoft for comment, but neither responded. However, judging by forum comments, the issue has caught out plenty of users. As such, if you find yourself with some weird networking issues with IPv6 and are using ICPS (as many will be), it might be worthwhile checking out the generic rather than OEM version of the software or removing it altogether to see if the problem goes away. ®