Tom Barber challenges the AI hype cycle, arguing that users care about outcomes, not architecture. Learn why slapping an 'AI-powered' label on everything is the wrong approach, and discover how to thoughtfully integrate LLMs into products without falling into common pitfalls like dependency on unstable APIs or unnecessary chatbot interfaces.

Show Notes

Episode Overview

Tom Barber returns with a critical examination of AI integration in modern software development, challenging teams to focus on user outcomes rather than jumping on the AI hype train.

Key Topics Covered

The AI Marketing Problem

• Why 'AI-powered' labels are often meaningless marketing
• The difference between machine learning (which has existed for decades) and modern LLMs
• Examples of invisible AI: spam filtering, fraud detection, map rerouting
• Users grade products on consistency, not on the impressiveness of the underlying model

Engineering Considerations for LLM Integration

• Choosing the right model for your specific use case (Opus, Sonnet, GPT-4, etc.)
• Tradeoffs between cost, speed, and inference quality
• Building evaluation systems and fallback paths
• Managing latency budgets and graceful degradation
• Handling API outages from providers like Anthropic and OpenAI
• The risks of depending on frontier models that can be deprecated

Trust and Transparency

• AI as a potential trust liability
• Managing user expectations around hallucinations
• The importance of data provenance and quality (garbage in, garbage out)
• When and how to disclose AI usage to users
• The ethical obligation to be transparent when AI makes consequential decisions

Product Strategy

• Why you can't charge an 'AI tax' on top of existing pricing
• Pricing based on outcomes, not on the technology stack
• How to use LLMs to deliver genuine efficiency gains
• Reducing user overhead and friction through thoughtful AI integration

Beyond Chatbots

• Why chatbots may be the most inefficient way to interact with LLMs
• The challenge: How to integrate LLMs without forcing users to type everything
• Asking 'What's now instant that wasn't?' instead of 'How do we add AI?'
• Innovation opportunities for those who can solve the chatbot problem

Key Takeaways

1. Users care about reliable outcomes, not whether you're using AI
2. Engineer for model availability issues and API outages from day one
3. Select and tune models specifically for your use case rather than defaulting to frontier models
4. Be transparent about AI usage, especially for consequential decisions
5. Focus on delivering value through AI rather than adding an 'AI-powered' label for marketing
6. The future belongs to products that leverage LLMs without relying on chatbot interfaces

Resources Mentioned

• Various LLM providers: Anthropic (Claude/Opus/Sonnet), OpenAI (ChatGPT-4)
• Example of model deprecation: Fable model being pulled

Connect

Engineering Evolved is hosted by Tom Barber. If you found this episode valuable, please leave a rating and review to help other leaders discover the show.

Chapters

• 0:00 - Introduction: Users Don't Care If It's AI
• 1:01 - Machine Learning Has Always Been Here
• 2:19 - The AI Marketing Problem: Selling Architecture vs Outcomes
• 5:16 - Engineering Realities: Models, Consistency, and Reliability
• 10:11 - The Cost of the AI Label: Trust and Pricing
• 14:39 - When Users Do Care: Transparency and Consequential Decisions
• 17:15 - Beyond Chatbots: The Future of LLM Integration

Engineering leaders learn how the Trio model can eliminate the blame game between business and IT teams. Discover practical strategies for cross-functional collaboration that actually work.

The Trio Model: Breaking Down Business-IT Walls

Key Topics Covered

The Business-IT Dysfunction Problem

• Why blame games develop between business and IT teams
• The 'technical purgatory' of mid-sized companies (200-1000 employees)
• Common symptoms: endless backlogs, shadow IT solutions, demoralized engineers

Why Traditional Fixes Fail

• Hiring more managers: Adds abstraction without context
• Adding more engineers: Brooks' Law in action
• Better ticketing systems: Makes misalignment visible but doesn't fix it
• More meetings: Creates 'status theater' without decisions

The Trio Model Explained

• Three core roles: Business owner, technical lead, designer/analyst/ops lead
• Co-ownership of outcomes, not just task handoffs
• Clear decision rights to prevent gridlock
• Not a committee: Explicit authority assignment

Implementation Strategy

• Which problems warrant a trio (high ambiguity, cross-functional dependencies)
• Decision rights framework
• Shared metrics and accountability
• Starting with 1-2 pilot areas

Leadership Requirements

• Stop bypassing trio processes with 'urgent' requests
• Protect trio time and focus
• Hold business owners accountable for outcomes
• Accept timeline realities

Key Quotes

• "If every request is urgent, there's no way for IT to prioritize"
• "Shared ownership of the outcome doesn't mean you can point at someone else when your part goes wrong"
• "The trio owns it can quickly become no one owns it"

Action Items

• Identify 1-2 high-friction problem areas
• Form pilot trios with clear problem definitions
• Establish shared success metrics
• Review and iterate after one quarter

Chapters

• 0:00 - The Business-IT Blame Game Problem
• 1:56 - Life in Technical Purgatory
• 5:29 - Why Traditional Fixes Don't Work
• 10:09 - Introducing the Trio Model
• 15:51 - Implementation and Decision Rights
• 23:42 - Measuring Success with Shared Metrics
• 24:50 - Leadership Changes Required
• 29:25 - Getting Started: A Practical Approach

How many meetings moved your team forward last week? Tom explores why most team rituals fail at building trust and alignment, sharing lessons from NASA JPL and startups on creating ceremonies that actually work.

Why Your Team Rituals Are Optimized for the Wrong Thing

Key Topics Covered

The Missing Middle Challenge

• Why mid-sized companies (200-1000 employees) face unique coordination challenges
• Too big for startup osmosis, too small for enterprise playbooks
• The need for distributed decision-making without dedicated alignment teams

Two Contrasting Standup Experiences

• NASA JPL: Nightly standups across three time zones that built trust and enabled handoffs
• Medical Startup: Transactional daily standups that created artificial harmony
• What made the difference: optimization for relationship building vs. status updates

The Five Dysfunctions of a Team Framework

1. Absence of Trust - Vulnerability-based trust, not competence trust
2. Fear of Conflict - Artificial harmony vs. productive disagreement
3. Lack of Commitment - What happens when people don't feel heard
4. Avoidance of Accountability - When standards become suggestions
5. Inattention to Results - Individual ego over team success

Three Practical Shifts for Better Rituals

Shift 1: Surface Vulnerability

• Leadership modeling uncertainty first
• Structured moments for admitting what you don't know
• Moving from posturing to problem exploration

Shift 2: Practice Disagreement

• Red team rotations in roadmap reviews
• Making challenge a role, not a personality trait
• Ensuring friction happens constructively in the room

Shift 3: Build Shared Context (Not Just Information)

• The difference between "here's the roadmap" and "here's the trade-off I struggled with"
• Smaller cross-functional context sessions vs. large all-hands presentations
• Enabling distributed decision-making through understanding reasoning

Key Questions for Diagnosis

• How much time was spent on information transfer vs. relationship building?
• Did anyone admit uncertainty without it being a problem?
• Was there constructive disagreement that led to better outcomes?
• Do people understand the reasoning behind decisions, not just the decisions themselves?

Resources Mentioned

• Patrick Lencioni's "The Five Dysfunctions of a Team" (2002)
• Concept of vulnerability-based trust
• Red team methodology for productive conflict

Next Episode Preview

Episode 14: The Product Trio Model - Making it Actually Work in Engineering-Heavy Organizations

Chapters

• 0:00 - The Meeting Problem: Status Theater vs. Real Progress
• 2:20 - The Missing Middle: Mid-Sized Company Challenges
• 4:21 - Tale of Two Standups: NASA vs. Startup
• 11:15 - The Five Dysfunctions Framework
• 16:13 - Three Practical Shifts for Better Rituals
• 23:55 - The Compounding Effect of Trust
• 28:39 - Diagnostic Questions and Next Steps

Engineering leader Tom Barber challenges the default adoption of Kubernetes, sharing why simpler alternatives often serve mid-sized companies better and how to make pragmatic infrastructure decisions.

Episode 12: Why Kubernetes Is Probably Wrong for Your Mid-Sized Company

Key Topics Covered

The Kubernetes Reality Check

• Why most mid-sized companies don't need Kubernetes complexity
• The hidden costs: maintenance, YAML management, and developer experience
• Real-world example from NASA: when impressive engineering doesn't solve business problems

Understanding Kubernetes Context

• Origins from Google's Borg system designed for massive scale
• Core benefits: fault tolerance, auto-scaling, declarative infrastructure
• Why these benefits require significant investment to realize

The Real Downsides

1. Complexity: Even cloud vendors are building products to hide Kubernetes
2. YAML Everything: Config management becomes a people and process problem
3. Cost at Scale: Engineering hours, infrastructure, and mental health costs
4. Developer Experience: High barrier to entry and friction in feedback loops
5. Portability Mirage: Cross-cloud deployment still requires deep vendor knowledge

When Kubernetes Makes Sense

• Genuine scale requirements (dozens/hundreds of services)
• Multiple teams with dedicated platform engineering capacity
• Complex deployment patterns that serve real business needs

Practical Alternatives

• VMs with Docker: Boring is good, boring is maintainable
• Managed Container Services: ECS/Fargate, Cloud Run, Azure Container Apps
• Serverless: Lambda, Cloud Functions for event-driven workloads
• Simple Deployment Scripts: Often cheaper than cluster management

Decision Framework: Do You Actually Need Kubernetes?

1. What specific problem are you solving?
2. Do you have dedicated team capacity?
3. What's your actual scale (services, teams, traffic)?
4. How frequently do you deploy?
5. Have you exhausted simpler options?

Resources Mentioned

• Free Download: "You Actually Need Kubernetes" Checklist (available in show notes)
• Consulting: Concept Cloud - Pragmatic infrastructure decisions for mid-sized companies
• Website: www.conceptcloud.com
• Contact: tom@conceptcloud.com

Next Episode Preview

Episode 13: "Why Your Engineers and Product Managers Still Don't Talk to Each Other (And How to Actually Fix It)"

Engineering Evolved is the podcast for engineering leaders at mid-sized companies who are tired of getting advice that only works for startups or enterprises.

Chapters

• 0:00 - Introduction: The Kubernetes Controversy
• 3:00 - A Personal Story: Getting It Wrong at NASA
• 4:58 - Understanding Kubernetes: Context and Core Benefits
• 7:07 - The Real Downsides: Complexity, Cost, and Developer Experience
• 10:49 - When Kubernetes Actually Makes Sense
• 13:39 - Practical Alternatives to Kubernetes
• 15:51 - Decision Framework: Do You Actually Need It?
• 18:36 - Wrap-up and Next Episode Preview

In this episode of Engineering Evolved, Tom Barber discusses the pitfalls of over-engineering platform infrastructure, particularly for mid-sized companies. He shares insights from his experience at NASA's Jet Propulsion Laboratory, emphasizing the importance of right-sizing infrastructure to match team needs and capacity. The episode covers the build versus buy framework, the challenges of internal tooling, and the significance of documentation and automation in maintaining efficient operations.

Summary

In this conversation, Tom Barber discusses the challenges organizations face when dealing with legacy systems and the importance of recognizing the human element in system migration. He emphasizes that migration is not just a technical project but a transition that impacts people's identities and roles within the organization.

Takeaways

• Think about your current environment for a second.
• Do you have systems running that are older than some of your engineers?
• Maybe it's that monolithic application written in a language nobody wants to touch.
• Maybe it's hardware that needs special handling.
• Or maybe it's just tribal knowledge locked in the heads of three people.
• Not if, but when, that time comes.
• You can treat it like a purely technical project.
• It's a transition that affects people's identities.
• A transition that affects people's identities, their expertise.
• Their sense of value to the organization.

• (00:00) - Intro
• (00:32) - Title
• (01:16) - The Challenge of Sunsetting Legacy Systems
• (05:59) - Principals for Successful Legacy Migration
• (11:52) - Creating a Culture of Evolution

Summary

In this episode of Engineering Evolved, Tom Barber discusses the critical first 90 days of a modernization initiative for engineering directors. He emphasizes that success is not solely about technology but about understanding the business pain, building trust, and navigating organizational dynamics. The episode provides a week-by-week action plan, highlighting the importance of quick wins, effective communication, and avoiding common pitfalls in modernization efforts.

Takeaways

• The first 90 days will make or break your initiative.
• It's about trust, momentum, and not declaring war on allies.
• Understand the business pain that led to your hiring.
• Your job isn't to fix everything but to prove you can be trusted.
• Conduct a 10-day discovery sprint to clarify needs.
• Weeks one and two should focus on listening and mapping.
• Identify a quick win that builds credibility.
• Stay engaged during the execution of the quick win.
• Communicate your vision and reset expectations with leadership.
• Avoid common pitfalls like ignoring the human side of technology.

• (00:00) - Intro
• (00:31) - Title
• (01:13) - The Crucial First 90 Days
• (07:48) - Week-by-Week Action Plan
• (13:58) - Communicating the Vision
• (19:16) - Conclusion and Next Steps

Summary

In this conversation, Tom Barber discusses the challenges of managing feature requests and the importance of strategic deployment to enhance team efficiency. He emphasizes the need to prioritize time-consuming tasks to free up valuable hours for the team, ultimately leading to increased productivity without additional hiring costs.

Takeaways

• We are overwhelmed with feature requests.
• Starting with the most time-consuming deployment is crucial.
• Every deployment can save significant time for the team.
• Strategic deployment can lead to substantial time savings.
• Freeing up hours can feel like hiring another engineer.
• Time management is essential in engineering teams.
• Prioritizing tasks can improve overall productivity.
• Efficient deployment processes are key to success.
• Reducing deployment time can enhance team morale.
• Investing in efficiency pays off in the long run.

• (00:00) - Intro
• (00:50) - Title
• (01:33) - Introduction to DevOps ROI Challenges
• (05:29) - Building a Compelling Business Case
• (12:52) - Handling Objections Effectively
• (19:01) - Post-Approval Steps and Checkpoints