Can custom AI chips challenge Nvidia?

Nvidia has long been the go-to name in AI chips—it’s what famously made the chip giant the world’s most valuable company amid the generative AI boom.

The dominance of its graphics processing units (GPUs) isn’t likely to change soon, but certain tech giants have increasingly been eyeing more customized alternative processing units from the likes of Broadcom and Marvell. These specialized chips might make operations cheaper and more efficient for big cloud providers and AI companies’ specific AI tasks. But they also come with drawbacks.

Custom AI chips snagged the spotlight last month when OpenAI announced a blockbuster deal with Broadcom around building its own data center infrastructure. Anthropic also recently announced it would tap up to 1 million of Google’s custom TPUs, which it co-developed with Broadcom.

Jefferies analysts wrote in a research note on Broadcom last week that custom application-specific integrated circuits (ASICs) have hit “an inflection point.”

The X factor: Broadcom’s bread and butter has historically been networking infrastructure—the chips and hardware that tie together data centers, cloud platforms, and internet service providers. But the sprawling Palo Alto-headquartered company has carved a fast-growing new niche for itself in XPUs—the “X” being a variable for any given application of AI.

The two business lines complement each other, Peter Del Vecchio, product line manager for Broadcom’s Tomahawk family of data center switches, told Tech Brew, and Broadcom develops the XPUs in tandem with the networking to make them as compatible as possible.

“The two kind of go hand in hand, where a lot of customers say, ‘I’ve got to get an XPU that’s going to talk to a switch,’” Del Vecchio said. “So the switch team here works very closely with the ASICs products division. And a lot of the products come out hand in hand.”

For Broadcom’s custom AI chip customers, which include Meta, Google, and TikTok parent ByteDance, the partnership might offer more efficiency and lower costs in the long run, as well as a decreased reliance on Nvidia, analysts told us. But Gareth Owen, associate research director at Counterpoint Research, also said these XPUs can’t completely replace Nvidia GPUs for all things.

“There’s definitely a trend, although I don’t think any of them are going to completely abandon GPUs,” Owen told us. “The main reason that they go for an XPU is because they can customize it. They can develop a customized chip for a specific workload. By doing that, they can actually create a chip that has high performance, lower power consumption—both very, very important—and also lower cost.”

Locked in: But going the customized route also means that companies are locked into one specific use case or model for AI, which can be tricky to navigate amid the frenetic tempo of AI progress, Alvin Nguyen, a senior analyst at Forrester, said.

“Right now, because the pace of innovation is so extreme, the hard part about it is you don’t want to be nailed down to any technology for too long because it can be obsoleted,” Nguyen said. “There’s a new Nvidia AI chip every year. Same thing with AMD, same thing with a lot of others.”

It’s also an extremely costly and involved process, which includes developing custom software to go along with the hardware, according to Gaurav Gupta, VP analyst at Gartner. “Doing a custom silicon is extremely expensive, resource-intensive. It takes several iterations to get the performance of the chip to a certain level.”

That typically limits the potential customer base to hyperscalers and the biggest AI companies, whose specific requirements make the whole process worthwhile, analysts said.

“If you know what you’re doing, like a hyperscaler does—they’ve got the money, they have enough demand where they can get chips made exactly for their workloads, they can leverage that for the lifespan of that chip, and it will bear out. That’s not true for everyone,” Nguyen said.

Nvidia CEO Jensen Huang has dismissed custom AI chips—and Broadcom in particular—as a threat, arguing that they can’t compete with Nvidia’s performance. He told Barron’s earlier this year that “a lot of ASICs get canceled,” and Digitimes Asia reported that he called ASICs “noncompetitive” at a media roundtable.

Nvidia spokesperson Steve Gartner pointed to Nvidia’s full-stack solutions beyond just chips. “We are the only company that innovates across the entire computing stack,” he wrote in an email.

He also mentioned a recent earnings call, where Huang talked about the challenges of developing custom AI chips and the versatility of general purpose chips like GPUs.

Inferred conclusion: Right now, Nvidia dominates the market for training AI models, where its chips and other technology are hard to beat, Owen said. But Gartner predicts that by 2026, compute spending on inference—the running of AI—will overtake spending on training as the technology matures. Eventually, the vast majority of AI spending will be on running models rather than training them—maybe 85% or 90%, Gupta said.

“At the end state, as training moves more and more toward inference, we would continue to see all different options existing,” Gupta said. “There is growth, and there is a strong demand out there for silicon. Customers want diverse solutions…Do the hyperscalers end up reducing their reliance on Nvidia? I think so, over a long period of time, definitely. But I think they would only keep a certain percentage of their workloads on their own custom silicon.”