Episode 74: What To Do About The Human in the Loop

Too Long; Didn't Watch — Summary

This episode features Jason Edelman, founder of Network to Code, discussing the evolution of network automation and the transformative impact of AI on operations, highlighting the shift towards agentic systems, the challenges of data standardization, skills, and culture, and the changing role of humans in an increasingly automated future.

Main Takeaways

• Network automation is rapidly evolving towards AI-driven, multi-agent systems, moving beyond traditional scripts and workflows.
• Data standardization, the development of new skills, and overcoming cultural resistance are critical challenges for scaling network automation and AI adoption.
• AI's increasing autonomy will drastically change network traffic patterns, leading to more bursty, spiky, and east-west traffic from numerous API calls.
• The role of human engineers will shift from hands-on operations to policy, governance, and managing exceptions as AI agents take on more complex tasks.
• Companies are already reducing or capping infrastructure staff, investing in AI and automation to boost productivity, signaling a significant industry-wide transformation in the next 2-5 years.

Detailed Summary

Introduction (0:00 - 0:31)

Nick Lippis introduces Jason Edelman, founder of Network to Code, to discuss network automation and the emerging concept of agentic operation centers. They acknowledge the long journey of network automation, dating back to early SDN efforts, and the current accelerated interest driven by AI.

Jason Edelman's Career Journey and Network to Code's Evolution (0:31 - 5:58)

• Jason Edelman shares his background in traditional networking since 2000, working as a network engineer for Cisco and a large VAR.
• He describes the shift around 2012 with the rise of SDN and OpenFlow, which motivated him to found Network to Code to focus on operationalizing networks and "day two" operations.
• Network to Code's journey has involved four phases: starting with training in broad skills like Python and Ansible, then becoming known for open-source services, launching Nbot in 2021 as a source of truth automation platform, and now focusing on commercializing software alongside services.
• The company's growth was significantly fueled by building a strong community through open-source code, YouTube content, conference speaking, and a large Slack workspace.

The Future of Network Automation with AI (5:58 - 16:57)

• Nick presents a graphic illustrating the exponential growth of AI model autonomy, projecting that by 2030, models could perform tasks taking a human a month, with a 50% success rate.
• Another projection by Dave Shapiro suggests an even more aggressive exponential growth, where AI automation could equate to 1900 years of human work by 2030.
• Jason acknowledges the exciting potential of dozens or hundreds of agents saving significant time, but emphasizes the current need for access to quality data and documented knowledge within organizations.
• He envisions agents tapping into multi-vendor network automation knowledge, refining processes, and potentially dynamically generating APIs from various controller systems to create unified interoperability.

Challenges of Standardization and Data (16:57 - 27:50)

• Nick highlights the messy nature of the networking industry, with a lack of standardization beyond TCP/IP and SNMP, leading to disparate network management systems, schemas, logs, and transport methods across OEMs.
• He questions how AI can overcome these silos to provide canonicalization and abstraction for unified access and control.
• Jason agrees, noting the difficulty of multi-vendor network automation but sees AI's potential to help with uniformity, especially in multi-controller environments.
• He suggests AI and agents could understand single OEM APIs to publish newer, unified APIs, potentially through a tiered approach or an MCP (Multi-Cloud Platform) protocol.
• They discuss how a single operator prompt could trigger hundreds of API calls across various tools and systems, leading to increased bursty and east-west traffic.
• This increased API consumption will necessitate robust API gateways, read-only replicas, and strong security and compliance measures.

Adoption Challenges at Scale: Skills, Culture, and Data (27:50 - 42:00)

• Jason breaks down scaling challenges into three key areas: skills, culture, and data, adding the notion of intended state.
• He points out that large enterprises, especially financials, have numerous homegrown applications and custom data sources that require bespoke integrations, making plug-and-play solutions unrealistic.
• Building a cohesive platform takes time, often involving a mix of building, buying, and customizing open-source solutions.
• Cultural resistance, particularly from long-tenured staff, is a significant drag on adoption, as changing decades of organizational culture is difficult.
• Nick suggests that the aging out of a segment of the workforce might naturally open the door to more automation-friendly cultures and new skills.
• Jason stresses that even with commercial tools, investment in skills (e.g., APIs, Git) remains crucial for integration and management.
• He emphasizes the opportunity for experienced personnel to document current processes and risks, providing essential knowledge for AI to be successful and learn from past issues.

Future Outlook and Impact on Staffing (42:00 - 50:37)

• Nick shares a poll result showing that almost 40% of companies are reducing or capping infrastructure staff and investing in AI to compensate, with another 20% indicating it's coming.
• Jason reflects on the accelerated pace of change, noting that while network automation took a decade to become conversational, AI's innovation rate is 2-3x faster.
• He projects that in the next 2-3 years, innovators will leverage agentic frameworks to build systematic approaches for automation, guardrails, tests, and checks, with humans continuously refining and editing.
• The goal is to gradually reduce the "human in the loop" as trust in AI grows, starting with getting data and existing automations right.
• Nick highlights that companies like Walmart are capping employee numbers and using AI to make existing staff more productive, shifting budget from salaries to capex/opex for automation tools.
• Jason concludes by encouraging the use of AI for non-automation tasks like post-mortems and documentation, leveraging LLMs embedded in enterprise tools to increase overall efficiency.

Notable Quotes

"I think there's such an opportunity for the people that have been there the last 10 or 20 years to document the current processes like document the risks because all of that knowledge is what AI is going to need to really you know to really be successful to be part of the solution." — Jason Edelman

"The amount of automation that an agent might be able to do is like almost 1900 years worth." — Nick Lippis

"Every system has an API, right? How do you get data in? How do you get data out? How do you run an automation from a third-party system?" — Jason Edelman

"The rate of change now is is growing and like the rate is happening faster I think it took 11 years to really become I would say at an industry level conversational you know Python Anible API guys and so forth." — Jason Edelman

"Are you reducing or capping infrastructure skilled staff and investing in AI to make up the difference?" — Nick Lippis (referencing a poll question)

"The human in the loop, like if we were to plot that on an XY axis, yeah, human in the loop is really down to the right as we look at, you know, the next, you know, the next several years." — Jason Edelman

"It starts today starts with data you get the data right get Yeah, even automation's right today because the agents can take advantage of those trusted automations that humans already built." — Jason Edelman