TCP is failing AI, but Stanford’s Homa is here to help
READ MORE
- Hot Dog! America's new chatbot is packed with wieners23 hours ago
- Huawei boss claims homegrown AI chip sales top Nvidia in China 1 day ago
- Add one more AI worry to the nightmare scenario: self-replicating prompt injections1 day ago
- Zuckerberg touts enterprise AI push because Meta would never do anything to damage your reputation2 days ago
- AMD's 192 GB Gorgon Halo prices might leave you petrified2 days ago
-
AI models keep posting screenshots showing sensitive data from inside tech companies
-
Astronomer watches Starlink satellites sinking to build a ‘planetary barometer’
-
EXCLUSIVE Microsoft tells nonprofits their deleted M365 data isn't coming back
-
Google ending ChromeOS support two years early
-
FBI to ShinyHunters: 'We know how to find you'
-
Microsoft's Copilot super app comes with a meter attached - TCP is failing AI, but Stanford’s Homa is here to help Boffin wants to kill TCP in the datacenter for AI. Not all network architects are buying it
- Hot Dog! America's new chatbot is packed with wieners Just ask America.gov and you'll receive a favorite July 4th food.
- Huawei boss claims homegrown AI chip sales top Nvidia in China The chips may not be as advanced, but they won't get banned at a moment's notice, Eric Xu says
- Add one more AI worry to the nightmare scenario: self-replicating prompt injections It's a worm attack, AI-style
- Zuckerberg touts enterprise AI push because Meta would never do anything to damage your reputation New business unit to be led by former MongoDB CEO 'CJ' Desai
- Security Russians are posing as Signal support to launch phishing attacks PLUS: US takes down Iranian propaganda sites; Marketing company asks 'Why Do We Have Your Information?' And more!
- Security Microsoft patches failed to fix on-prem SharePoint, which is now under zero-day attack PLUS: China upgrades smartphone surveillance tools; Ring eases anti-snooping stance; and more
- Black Hat and DEF CON DEF CON Franklin project enlists hackers to harden critical infrastructure Voting village reports have been so successful, says Jeff Moss, that the whole of DEF CON will now be included
- Security EQT buys majority share in Swiss cybersecurity biz Acronis Went at equivalent of $3.5B+ valuation for entire firm, though portion sold not specified
- Malware Month Ten years since the first corp ransomware, Mikko Hyppönen sees no end in sight On the plus side, infosec's a good bet for a long, stable career
- KDE turns 30 and someone's brought an AI-native desktop proposal Akademy talk imagines Plasma assembling itself around a personal model of each user
- Shopify extends lifeline to Tailwind as vibe coding erodes web dev platform's bottom line Acquisition gives open source CSS framework 'a stable long-term home'
- Switzerland tests a FOSS escape route from Microsoft 365 Swiss Army sticks a knife in American cloud apps with its own FOSS push
- Feel peak Windows was 7? You might like Kumander Linux Debian and Xfce – solid, sensible choices – with a pretty skin
- Canonical shuttering some of its legacy chat channels The Ubuntu Pastebin went in June, IRC gets demoted next
TCP, the data transport protocol upon which pretty much the entire web and most of cloud computing is built, is ill-suited for emerging AI workloads, a retired Stanford University professor argues. To solve the problem, he's promoting a new protocol called Homa.
“TCP, for all the amazing things it has done, is not a good match for datacenters,” said John Ousterhout, a professor emeritus of computer science at Stanford University, in a recent talk at the AI Engineer World’s Fair.
Shedding TCP sounds like an immense task, given the global reliance on the protocol. But adding Homa into a network is fairly simple, Ousterhout told The Register. Compile Homa from its GitHub source, then install the module into the Linux kernels on the clients and servers. No reboot required.
REG AD
“Homa works side by side with TCP, so you can gradually move applications from TCP to Homa,” he wrote. Running Homa even makes the remaining TCP applications run faster.
REG AD
Old wine, new skin
Homa is a clean-slate rethink of how networks should manage traffic congestion, Ousterhout said.
Work on the protocol began as a PhD dissertation first published in 2019 by Behnam Montazeri, now a Google staff engineer. Now that Ousterhout has retired from teaching, he has taken on the task of propagating Homa as his “life’s mission,” he said.
What makes Homa different from TCP is primarily that it is message-based, rather than stream-based. Much like the remote procedure call (RPC), Homa message lengths are explicitly defined.
Unlike TCP, Homa designates the receiver to manage congestion control. The first packet the receiver gets has information on how much data is incoming. It can then explicitly schedule when packets should be sent. In doing so, it prioritizes shorter messages over longer ones using a shortest-remaining-processing-time (SRPT) algorithm.
This approach cuts the latency of shorter messages by an order of magnitude, Ousterhout said. The 99th percentile (p99) of latency for shorter messages is 92 microseconds for Homa, which is 13 times faster than the 1.2 milliseconds p99 for TCP (based on packets swimming through a 100 Gbps network at 80% utilization).
Even on the longest messages, Homa is better by a factor of two, Ousterhout said.
A long list of other tasks where TCP falls short
REG AD
Currently, Ousterhout is drafting an IETF standardization document of the protocol, as well as working through the process of upstreaming Homa into the Linux kernel. In March, the protocol was backported to Red Hat Enterprise Linux versions 8 and 9.5.
He is also helping large companies investigate Homa’s applicability – he is currently working with one large financial services company on a prototype.
Not that everyone is on board with tossing TCP over a few latency issues.
Prominent network architect Ivan Pepelnjak wrote a scathing position paper in 2023 about Homa, questioning Ousterhout’s performance characterizations of TCP and critiquing Homa as a solution looking for a problem.
To be fair, the AI community is not the only ecosystem frustrated by the dowdy TCP.
In the high-performance database community, DPDK (Data Plane Development Kit) is being used to bypass the TCP stack for faster querying. Storage area networks turned to NVMe-oF (NVMe over Fabrics) to speed access to solid-state drives over network fabrics, using transports including RDMA, Fibre Channel, and TCP. For the Web, Google devised QUIC – which became the basis for HTTP/3 – to bypass TCP’s head-of-line blocking and enable browsers to download more assets simultaneously.
Also, the high-frequency trading and multi-player gaming communities have felt the pinch of TCP sluggishness.
Specialized RDMA fabrics and Amazon Web Services’ Scalable Reliable Datagram have also tackled the issue of TCP latency. Top-of-rack switches have also gotten smarter at managing congestion, setting queue thresholds and marking packets with early congestion notifications.
REG AD
Control lag is real
Vint Cerf and his fellow beardies created TCP to bring order to the unruly hordes of message packets across networks, giving them proper flow control, guaranteed delivery, connection handshakes and congestion control. Congestion control means avoiding overload in the network path, including switches and routers, while separate flow-control mechanisms keep senders from overwhelming receivers.
TCP’s data model is built on byte streams, a continuous flow of data packets with no differentiation. Messages are serialized as a single byte stream with no priority. For the receiver, longer sets of byte streams are indistinguishable from shorter ones.
A server deluged with traffic can send alerts to senders to slow input, but it has limited visibility over how much traffic is still coming in. The sender itself, which regulates its output depending on the timeliness of acknowledgments sent back from the receiver, has to guess how much to slow its roll.
For regular internet traffic or large-scale transfers within a datacenter, modest latency increases may be tolerable. But for latency-sensitive AI workloads, even delays measured in milliseconds can sting.
Think of the GPUs
The frontier labs driving the development of large language models (LLMs) have always required top-notch network performance for chores such as weight gradients, model weights, KV cache entries, and checkpoints.
However, large data transfers must increasingly share bandwidth with short bursts of traffic from agents and control tasks such as metadata coordination and cache lookups.
“For these workloads, what really matters is latency,” Ousterhout said. Even a millisecond of latency causes an expensive GPU to go idle.
“Legacy protocols are poorly suited for this environment,” Ousterhout said.
So has Homa finally found its problem to solve? Or was it just ahead of its time all along? For now anyway, TCP remains the champ.®
[](https://www.facebook.com/sharer.php?u=https%3A%2F%2Fwww.theregister.com%2Fnetworks%2F2026%2F10%2F01%2Ftcp-is-failing-ai-but-stanfords-homa-is-here-to-help%2F5300629) [](https://twitter.com/intent/tweet?url=https%3A%2F%2Fwww.theregister.com%2Fnetworks%2F2026%2F10%2F01%2Ftcp-is-failing-ai-but-stanfords-homa-is-here-to-help%2F5300629) [](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fwww.theregister.com%2Fnetworks%2F2026%2F10%2F01%2Ftcp-is-failing-ai-but-stanfords-homa-is-here-to-help%2F5300629) [](https://bsky.app/intent/compose?text=TCP%20is%20failing%20AI%2C%20but%20Stanford%E2%80%99s%20Homa%20is%20here%20to%20help%0Ahttps%3A%2F%2Fwww.theregister.com%2Fnetworks%2F2026%2F10%2F01%2Ftcp-is-failing-ai-but-stanfords-homa-is-here-to-help%2F5300629) [](https://www.reddit.com/submit?url=https%3A%2F%2Fwww.theregister.com%2Fnetworks%2F2026%2F10%2F01%2Ftcp-is-failing-ai-but-stanfords-homa-is-here-to-help%2F5300629&title=TCP%20is%20failing%20AI%2C%20but%20Stanford%E2%80%99s%20Homa%20is%20here%20to%20help) [](https://api.whatsapp.com/send?text=TCP%20is%20failing%20AI%2C%20but%20Stanford%E2%80%99s%20Homa%20is%20here%20to%20help%0Ahttps%3A%2F%2Fwww.theregister.com%2Fnetworks%2F2026%2F10%2F01%2Ftcp-is-failing-ai-but-stanfords-homa-is-here-to-help%2F5300629)
REG AD
TOP STORIES
AI
Infosec
FOSS
Firefox 157: Could bold new look be just what Mozilla needs?\ Yet another new version of Firefox, with a significant reskin