No Priors Ep. 81 | With Sarah Guo & Elad Gil

Too Long; Didn't Watch — Summary

This episode of No Priors explores the evolving landscape of the AI market, focusing on the consolidation of large language models (LLMs), the rapid decrease in API costs, and the ongoing debate between general-purpose and specialized AI models, while also discussing the strategic risks AI companies face and the competitive dynamics in the semiconductor industry.

Main Takeaways

• The LLM market is experiencing consolidation driven by massive capital requirements, leading to partnerships between model companies and hyperscalers or sovereign entities.
• Despite consolidation, the LLM market is becoming more competitive, with API costs decreasing by approximately 200x in the last 18-24 months, pushing companies towards specialization and vertical applications.
• Innovation in LLM productization is moving beyond basic chat interfaces, with providers building moats through advanced features like prompt caching, JSON output, and fine-tuning.
• AI companies face significant legal, regulatory, and reputational risks, drawing parallels to historical examples like Google's fair use battles and the challenges faced by Airbnb, Uber, and Napster.
• The semiconductor industry is seeing a new wave of startups optimizing for Transformer architectures, and AMD is strategically acquiring companies like Silo and ZT to compete with Nvidia in delivering full AI systems.

Detailed Summary

Introduction (0:00 - 0:24)

• The hosts, Sarah and Elad, introduce the episode's topics: LLM consolidation, chip industry dynamics, risk-taking for AI companies, and recent transactions.

LLM Market Consolidation (0:24 - 2:18)

• The discussion begins with the question of whether the LLM market is already consolidated.
• Many AI teams are joining larger enterprises (e.g., parts of Inflection, Character, Adapt to AWS).
• Enormous capital requirements are emerging for foundational models, with companies raising billions from hyperscalers (Amazon, Microsoft) or sovereigns, as venture capital is often insufficient.
• This leads to widespread partnering, raising questions about future partners for companies like Apple and Samsung.
• Hyperscalers are incentivized to fund these companies due to increased cloud utilization.
• It's becoming increasingly difficult for most companies to compete without fundamental breakthroughs in model architecture, training costs, or inference.

Competition and Decreasing API Costs (2:18 - 3:58)

• The market has become more competitive, not less, over the last 1.5 years, particularly among a set of well-capitalized players.
• There are continuous performance increases, competition on benchmarks, and significant price decreases.
• API costs have dropped approximately 200x in the last 18-24 months, making it harder to compete as a pure API business.
• This pushes companies towards specialization, B2B models, specific post-training, or vertical applications.

Innovation in LLM Productization (3:58 - 7:09)

• The primary business argument for large-scale capital in LLMs is still the pursuit of AGI, with the expectation of emergent money-making capabilities.
• In the 2-3 year horizon, consumer applications (apps, advertising, subscriptions) and enterprise solutions are current business models.
• The market is becoming a product fight, with major hires at Anthropic and OpenAI focusing on product development.
• Providers are trying to create lock-in through sophisticated APIs, offering features like prompt caching, JSON output interfaces, and fine-tuning, making the market less commoditized.
• New interfaces beyond basic chat are expected, offering smarter chat with automatic context and different surfaces.
• Competition can still arise from different reasoning approaches (e.g., math and code for self-play) and the distillation and fine-tuning of smaller models.

Specialization of AI Models (7:09 - 11:40)

• The discussion clarifies that the focus is on large language models, but other model types (physics, biology, material science, image generation) are emerging.
• Multimodal capabilities are growing, but unique models for specific domains are also expected.
• Differentiation also occurs in the infrastructure around models, including caching, long context windows, and RAG (Retrieval Augmented Generation).
• The core question remains how the LLM market will evolve, with current trends pointing to consolidation.
• Historical examples from social networks (Friendster, MySpace, Facebook, Instagram, Twitter, TikTok) suggest that even after perceived consolidation, new waves of innovation and specialization can emerge.
• The debate continues on whether future models will be general-purpose or specialized, similar to the modularity observed in the human brain.

Increasing Competition in Image Generation (11:40 - 14:43)

• The image generation market has recently intensified, moving beyond Midjourney's initial dominance and multimodal offerings from DALL-E and Gemini.
• New companies are emerging with impressive model fidelity, challenging the notion that only large players can compete.
• The pace of capability increase in video and audio models, exemplified by Sora, continues to surprise.
• Smaller teams with less capital are producing competitive video generation capabilities (e.g., Runway, Pika, Luma Labs).
• A significant trend is the shrinking size of models relative to their performance, achieved through distillation and other techniques.
• This efficiency, comparable to the human brain's low power consumption, suggests ample room for further progress in compact, high-performing AI.

Areas of Innovation and Risk-Taking (14:43 - 21:41)

• Two main areas of innovation are discussed: the chip layer and the approach to data and content usage.
• The hosts draw parallels to Google's early days, when it indexed the web, facing lawsuits over fair use and copyright for snippets.
• Google's resolution involved robots.txt, fair use rulings for snippets, content deals, and content providers realizing the traffic benefits of being indexed.
• This historical context informs how AI companies should approach risk, especially regarding data scraping (e.g., YouTube data).
• Companies like Airbnb and Uber pushed regulatory envelopes, prioritizing scale and consumer demand before fully addressing policy issues.
• The risks for AI companies include legal (e.g., Napster's downfall), regulatory (e.g., crypto, unclear AI regulations), and reputational (e.g., controlling model output).
• Grock's approach to less policing of output is highlighted as an ongoing experiment, contrasting with other companies' efforts to heavily filter content.
• A philosophical question arises: should restrictions be on generation or distribution of AI content, with distribution control seen as a platform's responsibility.

Semiconductor Industry and AMD's Strategic Moves (21:41 - 25:49)

• The discussion shifts to semiconductors, which underpin AI training and inference.
• A new wave of semiconductor startups (e.g., Natx, Etched) is emerging, focusing on optimizing for Transformer architectures and matrix math.
• The challenge for these startups is to surpass the economics and performance of established players like AMD and Nvidia.
• AMD is making progress, particularly with its investments and acquisitions.
• AMD's acquisition of Silo (a $600 million aqua-hire of AI engineers) addressed its software needs, especially in competing with Nvidia's CUDA.
• AMD is also part of UA Link, an open-source competitor to NVLink for networking.
• The acquisition of ZT (a $1-2 billion aqua-hire of 1,000 systems design professionals) aims to fill the gap in supporting rack and data center scale AI business, as Nvidia increasingly sells full systems.
• The question is whether AMD can successfully assemble these pieces to compete effectively in the full-system AI market.

Conclusion (25:49 - 26:23)

• Elad expresses excitement about future discussions with companies like Etched, Natx, and Cerebras.
• He also makes a personal request for suggestions or offers on purchasing a human-form/humanoid robot or a Boston Dynamics-like robot.
• The hosts encourage listeners to follow them on Twitter, subscribe to their YouTube channel, and listen on various podcast platforms.

Notable Quotes

"API costs have dropped something like 200x in the last 18 to 24 months." — Elad Gil

"The thing that Nvidia is really selling now is full systems through like multi-year strategy of you know delivering these essentially like data centers for um for research labs." — Elad Gil

"I'm very interested in buying either a human form / humanoid robot or a like a spa or something else from like Boston Dynamics." — Elad Gil