Alphabet burnishes one of its best weapons in the battle for AI supremacy
C
CNBC Finance
13 min read
Alphabet has squashed concerns that artificial intelligence will destroy its Google tech empire. One of its biggest weapons in the fight: homegrown silicon chips. Google's in-house tensor processing units (TPUs) serve as the engine to the company's Gemini chatbot, which has bolstered its image in the past year against rivals like OpenAI's ChatGPT. They also represent an integral part of Google's fast-growing cloud-computing business, where customers â including buzzy AI startup Anthropic â rent access to the chips; in some cases, they can now buy TPUs for their own data centers. Google also has a new AI compute venture with asset management giant Blackstone, built around the TPU. Google's compute business is seeing strong demand, with Wall Street projecting Google Cloud revenue to surge roughly 64% this year, to $96 billion, according to FactSet. Analysts see robust expansion continuing in 2027, with growth modeled above 50%. With demand for AI computing power surging, Google's TPUs are increasingly seen as a compelling alternative to Nvidia's market-leading graphics processing units (GPUs). They position Alphabet as a major force in AI infrastructure, even as Google Cloud still trails Amazon Web Services and Microsoft Azure in revenue. That status benefits both Google's internal AI efforts and helps win outside customers â a lucrative one-two punch that figures into Jim Cramer's admiration for the stock. Google is "probably the most underappreciated competitor of Nvidia," said Brad Gastwirth, global head of market research and market intelligence at Circular Technology , a supply chain services firm focused on compute infrastructure. While AI computing is a complex process, the appeal of the TPU comes down to a widely understood idea in life: making your money go further. In this case, the goal is to obtain the most computing power for every dollar spent, an increasingly critical consideration as companies race to deploy AI at scale. Main stages of AI computing At the simplest level, there are two primary stages of AI computing. Training: This happens first. Training teaches an AI model by feeding it massive amounts of data so it can learn patterns and improve its responses. This is the phase in which companies develop large language models such as Gemini. It requires enormous computing power, making it one of the most expensive parts of building AI systems. Inference: The process by which a trained AI model makes predictions or decisions based on new data. Inference is much less computationally heavy than training on a per-task basis. But once a model is deployed, inference is theoretically occurring all the time. So, the cumulative inference costs for a model can exceed its training costs over its lifetime. Put simply, the purpose of training is to learn, while the purpose of inference is to make predictions. The nature of TPUs enables them to deliver strong performance on AI tasks while reducing the cost of running those systems. TPUs belong to a class of chips called application-specific integrated circuits, or ASICs. Gastwirth likened ASICs to a custom suit â but instead of being tailored to a person's body, the processors are designed specifically for certain tasks. TPUs are optimized for machine learning tasks like training models and running them in real time, a process known as inference. Google co-designs the chips with fellow Club name Broadcom . The specialization of TPUs gives them an edge in efficiency, with William Blair analyst Ralph Schackart noting that they can deliver more computing output with less power. "Most ASICs consume 20% to 40% less energy than Nvidia processors, allowing for greater performance-per-dollar," Schackart continued. Those cost advantages, Schackart said, allow Google to charge about 20% to 30% less for excess compute capacity, which is attracting AI unicorns to Google's offerings, including its cloud business and enterprise services. To be sure, Google's AI computing ambitions face plenty of competition, and investment in innovation is required to stay on the cutting edge. Additionally, everyone in the AI compute business faces risks related to component availability â from memory chips to other input materials â and limited manufacturing capacity. Elevated memory costs, in particular, have weighed on the stocks of megacap tech stocks this week. More generally, the tight supply chain can cause delays to server and data center builds and be a limiting factor on growth. Another question mark around Google's AI efforts has emerged in recent days, specifically the loss of talented AI researchers to OpenAI and Anthropic. While these employees worked on developing AI models rather than TPU development, the company's success has come from having capable AI systems running on optimized, well-designed hardware; they complement each other. Shares of Alphabet are down 16% from their early-May peak, coinciding with a broader period of weakness among hyperscalers. For the year, though, Alphabet shares are still up about 8%, outperforming Microsoft , Amazon , and Meta Platforms . The leader Nvidia is the biggest player in AI compute. The company's GPUs offer more flexibility than an ASIC such as a TPU â after all, GPUs were originally designed to render better 3D computer graphics before their processing power was harnessed for a broader range of tasks, including AI. Today, Nvidia's data center GPUs hold a dominant position in training AI models and are also used for day-to-day inference. In many ways, they are the default chips for the AI era. In addition, Nvidia holds a major advantage with its CUDA software system, which developers have built around for years. CEO Jensen Huang also frequently argues that Nvidia's presence in all the major clouds is a big advantage, saying on an earnings call last year: "The reason why developers love us is because we're literally everywhere." The downside to GPUs? They're expensive, power-hungry relative to specialized TPUs, and hard to get, given their high demand. Nvidia remains the "broad ecosystem leader," with its dominant market share insulated in the near future, analysts at Stifel wrote in a May research note. However, they argued Nvidia's "moat is increasingly being tested." With AI adoption growing, analysts said the market is shifting from a "training-led regime toward inference-led regime by the end of 2026." After ChatGPT burst onto the scene in late 2022, the first wave of AI computing was dominated by training as companies raced to create new models. ChatGPT grew up on Nvidia chips and still relies on them, while also broadening its compute portfolio to include other types of silicon. Fast forward to today, and AI models are seeing rapid adoption from consumers and enterprises alike. That's why the split toward inference is accelerating. The evolution is placing greater focus on compute costs and return on investment, which analysts said is accelerating hyperscalers' interest in homegrown ASICs and alternative AI chips, sometimes called AI accelerators. Advanced Micro Devices is another competitor in the GPU space. Wait ... what about CPUs? In recent months, central processing units (CPUs) have seen a surge in demand after initially being overlooked during the AI compute boom. That's true â and we haven't forgotten about them. But they play a slightly different role in the AI compute landscape than GPUs and custom accelerators like TPUs. CPUs work in conjunction with accelerator chips, handling more general-purpose tasks, doing much of the "orchestration" for the entire AI system, and keeping the accelerators fed. The main reason for the current CPU renaissance is that agentic AI systems perform many of those general-purpose tasks, such as browsing the web, managing files, and combing through databases. Both Nvidia and Google have designed their own CPUs. AMD and Intel , which is another Club stock, are longtime CPU giants. Club name Arm Holdings is another notable CPU player. Another factor driving interest in custom silicon: Soaring demand for AI computing has created a tight supply environment. It's pushing companies with the capital to take matters into their own hands and develop specialized chips to meet their rising compute demand. Outside of Google's TPUs, Amazon has developed a lineup of custom chips, including its CPU, Graviton, and its AI accelerator, Trainium, which are used internally and sold to customers to power their applications. Microsoft developed its own in-house silicon called Maia to power its cloud infrastructure and reduce costs. Meanwhile, social media giant Meta Platforms is developing its MTIA (Meta Training and Inference Accelerator) processors, designed to run AI models across its family of apps, including Instagram and Facebook. ChatGPT creator OpenAI is also rolling out its first in-house chip later this year , designed in conjunction with Broadcom. Google's TPU discovery Google has been at it the longest. The company's TPU discovery came in 2013, when Google leadership produced an alarming projection that demand for computing on its products would outpace its current infrastructure. "We did some back-of-the-napkin math looking at how much compute it would take to handle hundreds of millions of people talking to Google for just three minutes a day," said Jeff Dean, Google's chief scientist, in a company blog post from 2024. "In today's framing, that seems like nothing. But at the time, we soon realized it would take basically all the compute power that Google had deployed. Put another way, we'd need to double the number of computers in Google data centers to support these new features." After realizing the market didn't offer anything that met the demand for even basic machine learning workloads to operate their products, the team began laying the groundwork for its first TPU. Google's first version of the TPU was deployed internally in 2015 and quickly became a critical component across different areas at Google. "We thought we'd maybe build under 10,000 of them," Andy Swing, principal engineer on Google's machine learning hardware systems, said in the same blog post. "We ended up building over 100,000 to support all kinds of great stuff, including Ads, Search, speech projects, AlphaGo, and even some self-driving car stuff." Before powering AI models like Gemini, the chips were used across a wide range of machine-learning applications, including Search, YouTube recommendations, and advertising systems. Today, in the company's words , TPUs "serve as the backbone for AI across nearly all of Google's products." A new milestone Since then, Google's TPUs have become more advanced and efficient across generations. The upcoming generation marks a notable milestone in their existence. Google's latest eighth-generation TPUs, announced in late April at the Google Cloud Next conference, mark the first time the company has split its chip lineup into two specialized variants for training and inference: the TPU 8t for model training and the TPU 8i for inference or the ongoing running on AI models after users submit prompts. These two TPU chips are designed to meet demanding AI workloads, including autonomous AI agents to get things done on people's behalf. The TPU system will work closely with CPUs. Google said the chips are up to three times faster for AI model training, offer 80% better performance per dollar, and can run more than 1 million TPUs in a single cluster. "This gives us the ability to create the largest training cluster in the world," said Alphabet CEO Sundar Pichai at Google's I/O developer conference last month. A training cluster is a massive network of thousands of chips that work together as a single supercomputer to train AI models. "For model builders, this means training larger, more capable models in weeks, rather than months," Pichai said. In general, TPUs are lowering Alphabet's cost of running AI while giving the tech giant more flexibility in how it prices cloud services, improving its bottom line. Pichai has also pointed to a 78% reduction in Gemini serving unit costs across 2025, powered in part by TPU efficiency gains. Circular Technology's Gastwirth said the margin benefit comes from not having to rely as heavily on Nvidia's high-priced chips. Google is not "spending 80% gross margin from Nvidia," he said, adding that its inference costs are likely among the best in the industry because the chip is built specifically for that task. That price-performance ratio has attracted some of the top AI labs in the industry. Anthropic has committed to using multiple gigawatts of Google TPUs to increase its computing resources as demand for its models and services surges. Meta Platforms also signed a multi-billion-dollar deal with Alphabet in February to use Google's TPUs. Google's TPUs are also finding customers outside Silicon Valley tech giants. "TPUs [are] becoming more general-purpose infrastructure," said Google Cloud CEO Thomas Kurian on the Future Forward podcast on April 25. Kurian noted he's seeing TPU demand beyond AI labs, into other market segments like finance, energy, and other high-performance computing customers. For example, financial firm Citadel Securities is using Google's TPUs for high-performance financial modeling , and all 17 U.S. Department of Energy national laboratories use AI co-scientist software â an AI framework developed by Google and powered by Gemini â built on the chips. Alphabet CFO Anat Ashkenazi said Google Cloud backlog nearly doubled sequentially to $472 billion by the end of the first quarter, driven by strong demand for enterprise AI offerings and the inclusion of TPU hardware sales for customers' own data centers. That's a departure from how customers historically have accessed TPUs â but it now represents another growth driver for its cloud business. Analysts at Citizens last month forecasted that Google will generate about $3 billion of revenue from TPU-related infrastructure in 2026, before jumping to $25 billion in 2027. Alphabet's TPU sales revenue would be included within the Google Cloud segment. "Importantly, we believe TPU monetization is not fully reflected in current consensus estimates, indicating meaningful upside potential," analysts wrote in early May. Kurian said Google wins no matter how customers are accessing TPUs. "We make great margins no matter which way we're selling it because we own our own IP," Kurian said in a Future Forward podcast interview published in April. He also explained that since chip demand is likely to exceed supply for years, in an already capacity-constrained environment, "unit economics get more expensive, and in our case, because we control our chip, the unit economics remain attractive." That gives Alphabet the opportunity to keep monetizing an already powerful and growing business segment and opening it to more hardware sales and partnerships, such as the TPU cloud venture with asset management giant Blackstone. Blackstone is committing $5 billion in initial equity to the venture, with plans to bring 500 megawatts of capacity online by 2027 and scale from there. Google will supply the hardware, software, and infrastructure expertise. A job posting on LinkedIn is currently available for the chief operating officer of the "Blackstone and Google TPU Cloud Company." The joint venture with Blackstone is "another vote of confidence in TPUs and allows Google to increase its commitment to Cloud without the significant capital requirements," Piper Sandler wrote last month in a research note. Analysts called it a "capital-light way for Google to keep driving TPU momentum." There are still plenty of questions about Alphabet's strategy in the AI arms race. For now, the TPU isn't one of them. (Jim Cramer's Charitable Trust is long GOOGL, NVDA, ARM, INTC, META, MSFT, AMZN, and AVGO. See here for a full list of the stocks.) As a subscriber to the CNBC Investing Club with Jim Cramer, you will receive a trade alert before Jim makes a trade. Jim waits 45 minutes after sending a trade alert before buying or selling a stock in his charitable trust's portfolio. If Jim has talked about a stock on CNBC TV, he waits 72 hours after issuing the trade alert before executing the trade. THE ABOVE INVESTING CLUB INFORMATION IS SUBJECT TO OUR TERMS AND CONDITIONS AND PRIVACY POLICY , TOGETHER WITH OUR DISCLAIMER . NO FIDUCIARY OBLIGATION OR DUTY EXISTS, OR IS CREATED, BY VIRTUE OF YOUR RECEIPT OF ANY INFORMATION PROVIDED IN CONNECTION WITH THE INVESTING CLUB. NO SPECIFIC OUTCOME OR PROFIT IS GUARANTEED.
