No Priors Ep. 54 | With Sarah Guo & Elad Gil

Too Long; Didn't Watch — Summary

Elad Gil and Sarah Guo discuss the rapid advancements in AI models like Google's Gemini 1.5 and Mistral, the emerging agent-centric AI applications, NVIDIA's continued dominance and the massive ROI seen from AI investments by tech giants, and the transformative impact of AI on various industries, all contributing to a self-reinforcing cycle of innovation and adoption.

Main Takeaways

• New AI models like Gemini 1.5 and Mistral are rapidly advancing, with Google showing renewed competitive will and Mistral demonstrating incredible development velocity.
• Agent-centric AI is evolving towards more targeted, domain-specific applications with effective feedback loops, moving past earlier, overly broad attempts.
• NVIDIA continues to see insatiable demand for its GPUs, and major tech companies are making massive AI-driven capital expenditures, yielding significant financial returns like Meta's market cap surge.
• AI is already profoundly impacting businesses, as demonstrated by Klarna's AI customer service assistant, leading to substantial efficiency gains and raising questions about future workforce implications.
• Competing with NVIDIA in chip manufacturing is challenging due to their integrated hardware and software ecosystem, with efforts to diversify chip production facing significant roadblocks, particularly in the US.
• The AI industry is experiencing a virtuous cycle where innovation, competitive pressure, and massive investment from both startups and big tech are accelerating progress across the board.

Detailed Summary

Introduction (0:00 - 0:27)

Elad Gil and Sarah Guo kick off a host-only episode of No Priors, setting the stage for a discussion on recent developments in the AI world.

Model News and Product Launches (0:27 - 5:01)

• Google launched Gemini 1.5, notable for its performance and a massive one-million-token context window, which is particularly important for applications like biology where longer sequences (e.g., proteins) need to be analyzed.
• OpenAI introduced Sora, a video generation model, while other companies like Pika are also making strides in video AI.
• Mistral launched "Mistral Large" (Le Chat), impressing with its rapid development, achieving near GPT-4 level performance in under a year, and its focus on efficiency, latency, and multi-language support.
• Mistral also secured a deal with Microsoft to be licensed on Azure, alongside OpenAI, Llama, and Microsoft's own models.
• The debate between retrieval-augmented generation (RAG) and large context windows is discussed, with the hosts believing both will coexist, offering different trade-offs.
• The importance of inference-time optimization and continuous model training/retraining is highlighted as a future trend, especially for agents and reasoning.

Google Enters the Competitive Space with Gemini 1.5 (5:01 - 8:23)

• Gemini 1.5 has made Elad Gil more bullish on Google's AI capabilities, viewing them as a "sleeping giant" with immense resources, proprietary data (from search, YouTube, emails, Docs), and top talent.
• The key question for Google is whether it can effectively steer its vast resources to remain competitive in AI, overcoming internal demands and focusing on market leadership.
• Google's recent series of impressive launches indicates a renewed competitive will, driven by the dynamic market.

Biology and Robotics Using LLMs (8:23 - 10:22)

• LLMs are still developing in specialized domains like biology and robotics, where current models struggle with tasks such as designing DNA sequences.
• Researchers are making exciting progress using Transformers and diffusion models for protein design, optimization, and drug discovery, indicating a different type of reasoning and data requirement.
• 2024 and 2025 are predicted to be years of model proliferation, expanding into chemistry, material sciences, physics, and mathematics, with robotics and biology being key areas.

Agent-Centric Companies (10:22 - 14:22)

• A new wave of agent companies is emerging, taking different approaches from traditional LLM building, drawing inspiration from game-centric AI like AlphaGo, which uses sequential actions, self-play, and inference-time heuristics.
• These new agents are becoming smarter by operating within constrained environments that support reinforcement learning and validation, such as specific web applications or code generation.
• The focus is shifting from broad, general-purpose agents to targeted use cases with clear feedback loops, allowing for effective training and rapid success.
• The ability to pay for human feedback data is making the agent problem more tractable, transforming it from an open-ended challenge to a cost-quantifiable task.

NVIDIA Earnings (14:22 - 17:29)

• NVIDIA's earnings continue to be exceptionally strong, with demand far exceeding supply, indicating sustained growth.
• The upgrade cycle from A100s to H100s, H200s, and B100s is a significant driver, as companies view these upgrades as offering "free money" in terms of training efficiency, despite hundreds of millions in spending.
• Most of the AI spending is coming from large hyperscalers and cloud providers, not just startups.
• Enterprise adoption of AI is still in its early stages, but Microsoft Azure's 5% revenue growth from AI-related products (adding $1-1.5 billion per quarter) suggests massive future growth as enterprises scale their AI adoption.

ROI in AI (17:29 - 20:43)

• David Khan's "AI's $200 Billion Question" article highlighted the need for massive returns to justify AI investments.
• Meta's recent earnings beat provided a clear answer: their $30-37 billion capex for AI and servers led to a $197 billion increase in market cap in a single day, driven by better ad targeting, recommendations, and engagement.
• This demonstrates that large tech companies are seeing immense ROI from their AI investments, reinforcing the idea that the biggest market cap gains are currently going to existing tech giants.
• The current phase is characterized by existing companies significantly increasing their revenue and earnings through AI, with new "monster AI companies" from scratch still emerging.

Impact from AI (20:43 - 25:45)

• AI is already having a significant impact on businesses, as seen with ServiceNow's strong quarter due to AI and Klarna's AI assistant.
• Klarna's AI assistant handled two-thirds of their customer service inquiries, performing the equivalent of 700 full-time agents, with higher accuracy and customer satisfaction, resolving issues faster, and operating 24/7 in multiple languages.
• This example illustrates how AI can dramatically reduce the need for human agents, prompting societal questions about the future of work.
• AI adoption will accelerate use case by use case, as early adopters demonstrate clear competitive advantages, forcing entire sectors to adopt AI.
• The potential market for AI in human services is enormous, with US payroll for AI-impactable tasks estimated at $3.5-5 trillion, suggesting that even a 10% conversion to AI revenue could recreate the entire US software industry's market cap.

Building Effective AI Tools In-House (25:45 - 29:09)

• The question arises whether AI tools will be built in-house, by new startups, or by incumbents.
• Big tech companies like Meta, Square, and Shopify, with their large commerce customer bases and technical teams, have successfully built in-house AI solutions.
• For other segments, like retailers without strong technical teams, new companies or incumbents like Zendesk are more likely to provide AI solutions.
• The AI talent wave started with researchers, then infrastructure specialists, and is now moving to application builders, many of whom are just starting to enter the field after the public awareness sparked by ChatGPT.
• There's a perceived gap in the market, with many obvious AI application opportunities still unaddressed by startups.

What Would It Take to Compete with NVIDIA (29:09 - 33:23)

• NVIDIA's defensibility lies in its chip performance, CUDA software ecosystem, and interconnect technology (like Mellanox).
• Competitors need performant silicon, effective utilization, and scalable interconnects.
• AMD is an incumbent competitor, while startups like Rock (fast inference for open-source models) and Cerebras (different chip approach) are emerging.
• New companies like Ads are focusing on Transformer-based architectures for their chips.
• Historically, advanced chip markets tend to have one clear leader and a second-place player (e.g., Intel and AMD, Qualcomm and ARM).
• NVIDIA's advancements are increasingly driven by memory management and full-system performance, not just transistor density.

The Architectural Approach to Compute (33:23 - 35:42)

• Beyond chip design, manufacturing capacity at TSMC, yield, and competitive pricing are crucial for challenging NVIDIA.
• The economic pressure to compete with NVIDIA is at an all-time high, but their moat is very deep.
• Alternative approaches include chip and system designs optimized for latency or specific architectures like Transformers, though new models like State Space Models (e.g., Cartesia) are emerging.
• Large companies like Meta and Google can develop custom chips (TPUs, Trainium) because they only need to optimize for their specific applications and have the resources to acquire teams.

The Roadblocks to Chip Production in the US (35:42 - 38:30)

• Repatriating semiconductor manufacturing to the US faces significant obstacles, including environmental reviews and lengthy construction times for Fabs.
• Japan is emerging as a potential second-source location for Fabs, partly as a geopolitical hedge against Taiwan.
• The TSMC CEO emphasizes the importance of human capital and cultural elements, not just capital expenditure, for successful Fab operations.
• While Intel has historically manufactured chips in the US, the challenge is to do so competitively in terms of turnaround time, yield, and cost, although national security concerns might justify less competitive pricing.

The Virtuous Tech Cycles in AI (38:30 - 41:53)

• The AI market is characterized by rapid, unpredictable change, where existing knowledge quickly becomes outdated.
• A self-reinforcing cycle is at play: new startups create competitive pressure on large tech companies, who then react, leading to more talent leaving to form startups, which are then funded by big tech, accelerating innovation for both.
• This cycle is further fueled by the demonstrated value of AI applications (like Klarna's assistant), which drives widespread adoption as companies strive for competitiveness.
• Fortune 1000 IT budgets are projected to increase significantly (from 3-5% to 5-8%) due to AI, further reinforcing this cycle of investment and development.
• The hosts coin the term "RLPA F" (Reinforcement Learning through Product Adoption Feedback) to describe how successful AI products drive further development and adoption.

Notable Quotes

"Mistral is the velocity of shipping it's incredible impressive they went from basically starting the company to almost GPT 4 level in less than a year." — Elad Gil

"The question is actually can they steer Google to like focus on being competitive versus the many other demands from their employee base." — Sarah Guo

"It's funny when somebody describes spending hundreds of millions of dollars as free money but free money in terms of training efficiency." — Sarah Guo

"If you convert just 10% of that spend into AI Revenue you've effectively recreated the entire US market software industry in market cap." — Elad Gil

"This technology seems to be so good for certain human level tasks and this is back to that point of AI adoption in the Enterprise is just starting." — Elad Gil

"The more I learn the less I know in AI and it's the opposite of every other field I've ever been in." — Elad Gil

"It's this really interesting virtuous cycle." — Elad Gil

"RLPA F rules right off the tongue reinforcement learning through product adoption feedback." — Elad Gil

"I will be sponsoring author if you'll be first author." — Sarah Guo

"Academic violence." — Elad Gil