Nvidia's Rivals Are Coming for Its Crown, But the Smartest AI Bet Sits Further Down the Tech Stack

No one can deny that Nvidia (NVDA) is the de facto leader in the artificial intelligence (AI) semiconductor race. Every platform it has released since the start of the AI boom has generated strong interest from hyperscalers and enterprises alike, with demand often exceeding supply for years at a time. The main reason is CUDA, Nvidia's proprietary software ecosystem that allows graphics processing units (GPUs) to be programmed for general-purpose computing tasks, including AI training and inference. CUDA was also the biggest reason Nvidia's GPUs sold out like hotcakes during the crypto boom of the pandemic.

Today, though, that dominance is now being challenged – but perhaps not in the way you think.

Microsoft Earnings Preview: Get Ready for Soaring AI Spending to Sink MSFT Stock

Why Nvidia (NVDA) Stock Faces Sell-the-News Risk Following Its Q2 Earnings Report

Instead of creating competing general-purpose chips, some companies are targeting its largest customers by offering AI chips custom-made for customer-specific workloads, called application-specific integrated circuits (ASICs). These chips are designed to excel at a narrow set of AI tasks, improving power efficiency and allowing hyperscalers to obtain the chips they need at prices below those of Nvidia's market-leading products.

So that raises an obvious question: If custom AI chips are becoming a larger piece of the AI infrastructure puzzle, is there suddenly a war against Nvidia's leadership? And if so, which companies are best positioned to benefit from this movement?

And perhaps more importantly, if there is a new chip war, which custom chip companies should investors actually be watching?

But here's what most investors miss: no matter who wins the chip war, every chip in it passes through the same few companies further down the stack – and one of them sits so low that without it, none of this exists.

The first and most important reason hyperscalers are turning to custom chip manufacturers, or producing those chips themselves, is cost.

Training a frontier AI model requires tens of thousands of GPUs running around the clock. Even after the model is trained, serving it to millions of users through inference can cost billions of dollars – a never-ending capex cycle.

Now, imagine this: if a hyperscaler has the opportunity to shave 20% to 30% of those costs annually and still get chips that perform the tasks they need, why wouldn't they grab it? Especially when those 20%-30% savings could easily translate to hundreds of millions, if not billions, of dollars.

Let's face it. These days, power is becoming AI's biggest bottleneck, and GPUs installed within data centers require massive amounts of power. In fact, according to the International Energy Agency (IEA), a conventional data center may draw around 10 to 25 megawatts (MW), while a hyperscale, AI-focused facility can require 100 MW or more.

Now, I admit, those numbers might look meaningless without a frame of reference. So to put them into perspective, that's roughly enough electricity to power ~8,000 average U.S. homes for an entire year, or up to a mid-sized city in the U.S. Any savings in that department could translate into material benefits for the company, the immediate area around the data center, and the overall environment.

Going back to chips, a GPU uses more electricity than a CPU because it's built to be more flexible. Custom chips, however, have more specialized, intentional designs. Take for example, a hyperscaler operating a large-language model without graphics capabilities. It can use silicon optimized for matrix multiplication, memory bandwidth, and token throughput. Big words, I know, but the point is, it wouldn't need the general-purpose graphical capabilities that come with every Nvidia GPU. That means it doesn't need as much power as a general-purpose chip to do its job.

Nvidia's GPUs are incredibly versatile because they're built to handle thousands of different workloads. That's a major advantage for cloud providers serving millions of customers with varying needs.

But hyperscalers also run their own internal AI models every day. Google operates Gemini. Meta trains and serves Llama. Amazon powers services like Bedrock and Alexa. And Microsoft supports Copilot and numerous other enterprise AI products.

When you know exactly which models you'll be running, you don't necessarily need a chip capable of doing everything. Instead, you can use a custom silicon ASIC that's optimized for YOUR workload, again, reducing unnecessary hardware costs while improving performance per watt.