Listen to the audio version of this article (generated by AI).
Key Points
- Alphabet is developing a new AI chip designed to run its Gemini models 6 to 10 times more efficiently than current hardware.
- The AI race is shifting from building the most advanced models to delivering powerful AI at the lowest possible cost.
- After more than a decade of investing in AI infrastructure, Alphabet has built a competitive advantage that could help lower costs and strengthen its position in the AI market.
Alphabet (GOOGL), the parent company of Google, may have just made one of the most important moves we’ve seen since ChatGPT kicked off the artificial intelligence (“AI”) boom. According to a recent report from The Information, Alphabet is developing a new custom chip it calls “Frozen v2” to run its AI model.
And this chip has the potential to change the direction of where the AI race is headed.
Alphabet will embed architectural elements of the Gemini AI model directly into the chip. The “frozen” name comes from the idea of permanently etching part of the model into the physical silicon.
This minimizes the physical movement of data and slashes the sheer volume of calculations required to generate a response. According to the report, this will make the chip 6 to 10 times more efficient than Alphabet’s current AI chips, which are already world-class.
The company could begin using the chips as early as 2028. If those efficiency numbers hold up, it could change the economics of AI.
When OpenAI launched ChatGPT in late 2022, it set off an AI arms race. Ever since, the race has been about one thing… building a smarter AI model.
Companies are jockeying to build a model that’s a little faster, a little more capable, or a little better at writing code or answering questions.
Investors are obsessed with benchmarks like token counts and model rankings. It seems like every other week, someone unveils a better model. If one AI model scores a few percentage points better than another, it’s treated like a major breakthrough.
The race to build the best model is extraordinarily expensive, with Wall Street, private equity, and venture capital providing much of the financial backing.
They’ve invested hundreds of billions of dollars into companies like OpenAI, Anthropic, and xAI to fund purchases of advanced chips, fancy servers, and long-term data-center leases to train and run their AI models.
But Alphabet’s latest chip announcement reminds us that there’s much more to this story than building the best model…
AI’s Cost Problem
AI is one of the most expensive computing projects humans have ever undertaken.
So far, it has been a massively unprofitable pursuit. OpenAI, Anthropic, and xAI – a division of SpaceX (SPCX) – have lost tens of billions of dollars trying to build the best model.
The next phase of AI isn’t going to be won by the company with the smartest model. It’s going to be won by the company that figures out how to deliver AI at the lowest cost.
Once a model is built, the next step is rolling it out to users and putting it to work. Generating responses when users prompt the model is called inference in industry jargon.
Training happens occasionally. Inference happens continually. AI data centers are not only expensive to build, but they also consume staggering amounts of electricity. The price tag runs into the billions of dollars before the first person to use the AI model ever types a prompt.
Alphabet aims to lower that cost with its Frozen v2 chips. They are designed specifically for inference. They can potentially reduce the cost of each AI interaction by using less electricity, responding faster, and handling more requests simultaneously.
Every time someone asks Gemini to summarize a document or create an image, Alphabet must perform a fresh round of calculations. Those calculations happen in fractions of a second, but they still consume computing power, electricity, memory, and networking capacity. Multiply that by billions of requests every day, and the costs add up quickly.
That’s why Alphabet’s latest Frozen v2 chip is so important. Alphabet is making a very clear statement about where it believes AI is headed. The next phase of AI will be about lowering its cost.
AI Is Rapidly Becoming a Manufacturing Business
That may sound like an odd comparison considering AI is software.
Manufacturers obsess over the cost of producing one more unit. History is full of companies that won because they became the lowest-cost producer. Walmart did it in retail, Amazon did it in e-commerce, and Toyota did it with cars.
AI may be heading down the same path.
Salesforce (CRM) CEO Marc Benioff believes that AI models will soon become commodities. In other words, intelligence will become a utility. And utilities are won on cost.
Companies that master economics have a significant advantage over those that don’t.
Soon, AI companies will be obsessing about the cost of producing one more answer rather than PhD-level reasoning.
AI engineers have already started talking more often about things like “inference efficiency” and “cost per token” (a token is the smallest unit of text AI processes, around four characters).
The cheaper you can produce each AI response, the more profitable your business becomes.
Until now, investors have been focused on model capabilities. As losses mount and competition gets even hotter, they’ll care much more about production costs.
That’s what Alphabet has its sights set on.
As long as its AI models are roughly competitive in capabilities while serving AI requests at a greatly reduced cost, then it can lower prices and win market share.
Alphabet’s Massive AI Advantage
AI isn’t a new strategy for Alphabet. Long before AI became Wall Street’s darling investment trend, the company saw AI as the future.
In 2014, it bought an AI company called DeepMind for more than $400 million. It was its first big gamble on AI.
In 2016, Alphabet announced its own custom AI chips known as Tensor Processing Units (TPUs). The chips were built to handle Google’s own AI workloads more efficiently. The company released its eighth generation of these TPUs this year. Its Frozen chips are the company’s first ones to embed AI architecture in the chips.
And today, Alphabet is investing extraordinary sums in AI. It expects capital expenditures this year to approach $205 billion, a figure that would have seemed almost unimaginable only a few years ago.
Few companies can afford that level of investment. While AI startups are bleeding cash, Alphabet is printing it. The company generated $165 billion in cash from its operations last year.
Building on Google’s Search and Advertising Dominance
Alphabet dominates nearly every industry in which it operates…
Its Google search engine has a market share of more than 90%. Its Android software owns 70% of the mobile operating system market. Google Chrome owns roughly two-thirds of the web browser market. And its YouTube video streaming service has more than 2.5 billion active users… only Meta Platforms’ (META) social media apps have more.
Billions of people use Alphabet’s products before they ever think about using AI directly.
Alphabet plans on using AI to grow nearly all these dominant businesses.
Advertising is Alphabet’s biggest source of revenue. It accounted for more than 75% of its sales last year. Most of this advertising revenue is generated by its Google search engine.
Alphabet is using AI to protect its golden goose. Instead of simply returning links, the company now uses AI Overviews and Gemini-powered search features to answer questions, summarize information, and compare products. AI understands a user’s search intent better, so Google can show more relevant ads.
Cloud Business Is Proving to Be Crucial to Google’s Future
AI is also powering its Google Cloud business – Alphabet’s fastest-growing segment.
The company operates 20 data centers in the U.S. today, with another 15 planned or under construction. And it has at least 20 more in other parts of the world.
Google Cloud sells computing, storage, and access to AI tools to businesses. Last year, the Cloud segment generated nearly $60 billion in revenue, around 15% of Alphabet’s sales.
And it is capturing market share from its competitors in this fast-growing market. Its cloud sales grew 36% last year, nearly double the 20% growth of Amazon’s and Microsoft’s cloud businesses.
And this growth is accelerating thanks to AI. Last quarter, Google Cloud sales grew 82%. The company’s Gemini AI app now has 950 million monthly active users, and its AI models process 22 billion tokens per minute.
Other AI companies rely on Alphabet. AI-model startups can’t afford to build their own massive data centers, so they rent Alphabet’s infrastructure. For example, Anthropic is a Google Cloud customer. It recently committed to $200 billion in Google Cloud spending over the next five years.
The truth is, Alphabet can’t keep up with AI computing demand. The company has had to turn down corporate deals because it didn’t have enough server power to support them. It’s one of the reasons they are developing the new Frozen chips.
Data centers are where the big money will be made in AI. They provide both the training and the inference that AI models need, and so lowering the cost of inference will be a major advantage.
That’s why this latest chip announcement is worth paying attention to. Alphabet isn’t just building a better chip. It’s lowering the cost of intelligence.
Alphabet rarely wins by inventing an entirely new market. Instead, it finds ways to do things faster, cheaper, and at a much larger scale than its competitors. That’s what this Frozen chip is all about.
Wall Street has spent the last three years trying to identify the company with the best AI.
It may soon discover that it has been asking the wrong question. The better one moving forward is likely: who can afford to give the world AI at the lowest cost?
Good investing,
Mike DiBiase
Editor’s Note: Whitney Tilson — the hedge fund manager CNBC called “The Prophet” — says America has reached its “Ripping Point.” The old financial order is being torn apart, and he believes most investors have no idea what’s coming in the next six months. He’s named the stocks he thinks will be destroyed in the chaos — and the ones he believes will soar. Watch his free presentation while it’s still available.
