Meta’s chip shot

TL;DR: Meta is sprinting into the custom chip race, announcing plans to roll out four new in-house AI processors by 2027. The move mirrors a broader push across Big Tech to control more of the AI hardware stack as computing costs surge. But designing chips isn’t easy, and even if Meta pulls it off, powering AI features for its 3.58 billion daily users will still require a lot of third-party GPUs. Safe to say, this move doesn’t put Jensen Huang out of a job just yet.

What happened: Yesterday, Meta announced four new generations of its Meta Training and Inference Accelerator chips (aka MTIA), built with manufacturing support from Taiwan Semiconductor Manufacturing Company. The chips are strictly for internal use and will “support ranking and recommendations, along with GenAI workloads,” according to Meta. The first version, the MTIA 300 chip, has already been deployed, while the 400, 450, and 500 will follow on a six-month cadence. Unlike the powerful, expensive GPUs made by companies like Nvidia, which can be overkill for some kinds of AI compute, Meta’s MTIA chips are mainly designed for inference—the step where trained models are actually used to generate recommendations, answer prompts, or power AI tools inside apps.

The strategy: Custom chips let companies design hardware around the specific AI tasks they run most often. For Meta, that means optimizing for recommendation systems and AI features inside apps like Instagram and Facebook. Inference is expected to account for about two-thirds of AI computing this year, according to Deloitte, and purpose-built chips can often run those tasks more cheaply and efficiently than general-purpose GPUs.

Everyone else is doing it: Google has built its custom TPUs for over a decade now. Microsoft introduced its first AI chip in 2023, and Amazon has been making its Trainium chips since 2020. Even OpenAI is developing its own custom silicon. Meta first revealed its own chip ambitions in 2023, making it a somewhat late arrival.

The hard part: Obvious statement incoming… but designing competitive chips is notoriously difficult. And just last month, The Information reported that Meta scrapped a more advanced training chip codenamed Olympus after running into several development roadblocks.

Plus, Meta’s timeline is aggressive, trying to crank out chips faster than Marvel puts out movies. “It's unusual for any silicon company or team to be releasing a new chip every six months,” Meta VP of Engineering Yee Jiun Song acknowledged to CNBC. Even if Meta succeeds, its chips won’t cover all its AI compute needs. Over the past month, it signed: a multibillion-dollar deal for Google’s TPU chips, a $60 billion deal with AMD, and a sweeping new deal with Nvidia. —WK

Meta investing in custom chips makes sense as it prepares for a future in which data centers are expected to shift from mostly training AI models to mostly running them.

Does the move pose a challenge to Nvidia? “To some extent the answer is yes,” says Gaurav Gupta, a VP analyst at research firm Gartner. “But the overall demand is stronger, and the market is growing, so there is plenty of opportunity for all.”

As Meta’s partner on these chips, Broadcom is another winner here. Gupta said the chip giant is “pretty much the partner of choice for all companies going [the custom] route.”

Delivering on these custom silicon ambitions would help Meta stay competitive in the AI race with more energy-efficient infrastructure and a host of chip options that lessen reliance on Nvidia. But it also adds to the massive capital expenditure bills Meta is racking up from AI, which tend to make investors skittish. —PK