Tom Barber challenges the AI hype cycle, arguing that users care about outcomes, not architecture. Learn why slapping an 'AI-powered' label on everything is the wrong approach, and discover how to thoughtfully integrate LLMs into products without falling into common pitfalls like dependency on unstable APIs or unnecessary chatbot interfaces.

Show Notes

Episode Overview

Tom Barber returns with a critical examination of AI integration in modern software development, challenging teams to focus on user outcomes rather than jumping on the AI hype train.

Key Topics Covered

The AI Marketing Problem

• Why 'AI-powered' labels are often meaningless marketing
• The difference between machine learning (which has existed for decades) and modern LLMs
• Examples of invisible AI: spam filtering, fraud detection, map rerouting
• Users grade products on consistency, not on the impressiveness of the underlying model

Engineering Considerations for LLM Integration

• Choosing the right model for your specific use case (Opus, Sonnet, GPT-4, etc.)
• Tradeoffs between cost, speed, and inference quality
• Building evaluation systems and fallback paths
• Managing latency budgets and graceful degradation
• Handling API outages from providers like Anthropic and OpenAI
• The risks of depending on frontier models that can be deprecated

Trust and Transparency

• AI as a potential trust liability
• Managing user expectations around hallucinations
• The importance of data provenance and quality (garbage in, garbage out)
• When and how to disclose AI usage to users
• The ethical obligation to be transparent when AI makes consequential decisions

Product Strategy

• Why you can't charge an 'AI tax' on top of existing pricing
• Pricing based on outcomes, not on the technology stack
• How to use LLMs to deliver genuine efficiency gains
• Reducing user overhead and friction through thoughtful AI integration

Beyond Chatbots

• Why chatbots may be the most inefficient way to interact with LLMs
• The challenge: How to integrate LLMs without forcing users to type everything
• Asking 'What's now instant that wasn't?' instead of 'How do we add AI?'
• Innovation opportunities for those who can solve the chatbot problem

Key Takeaways

1. Users care about reliable outcomes, not whether you're using AI
2. Engineer for model availability issues and API outages from day one
3. Select and tune models specifically for your use case rather than defaulting to frontier models
4. Be transparent about AI usage, especially for consequential decisions
5. Focus on delivering value through AI rather than adding an 'AI-powered' label for marketing
6. The future belongs to products that leverage LLMs without relying on chatbot interfaces

Resources Mentioned

• Various LLM providers: Anthropic (Claude/Opus/Sonnet), OpenAI (ChatGPT-4)
• Example of model deprecation: Fable model being pulled

Connect

Engineering Evolved is hosted by Tom Barber. If you found this episode valuable, please leave a rating and review to help other leaders discover the show.

Chapters

• 0:00 - Introduction: Users Don't Care If It's AI
• 1:01 - Machine Learning Has Always Been Here
• 2:19 - The AI Marketing Problem: Selling Architecture vs Outcomes
• 5:16 - Engineering Realities: Models, Consistency, and Reliability
• 10:11 - The Cost of the AI Label: Trust and Pricing
• 14:39 - When Users Do Care: Transparency and Consequential Decisions
• 17:15 - Beyond Chatbots: The Future of LLM Integration

Engineering leaders learn how the Trio model can eliminate the blame game between business and IT teams. Discover practical strategies for cross-functional collaboration that actually work.

The Trio Model: Breaking Down Business-IT Walls

Key Topics Covered

The Business-IT Dysfunction Problem

• Why blame games develop between business and IT teams
• The 'technical purgatory' of mid-sized companies (200-1000 employees)
• Common symptoms: endless backlogs, shadow IT solutions, demoralized engineers

Why Traditional Fixes Fail

• Hiring more managers: Adds abstraction without context
• Adding more engineers: Brooks' Law in action
• Better ticketing systems: Makes misalignment visible but doesn't fix it
• More meetings: Creates 'status theater' without decisions

The Trio Model Explained

• Three core roles: Business owner, technical lead, designer/analyst/ops lead
• Co-ownership of outcomes, not just task handoffs
• Clear decision rights to prevent gridlock
• Not a committee: Explicit authority assignment

Implementation Strategy

• Which problems warrant a trio (high ambiguity, cross-functional dependencies)
• Decision rights framework
• Shared metrics and accountability
• Starting with 1-2 pilot areas

Leadership Requirements

• Stop bypassing trio processes with 'urgent' requests
• Protect trio time and focus
• Hold business owners accountable for outcomes
• Accept timeline realities

Key Quotes

• "If every request is urgent, there's no way for IT to prioritize"
• "Shared ownership of the outcome doesn't mean you can point at someone else when your part goes wrong"
• "The trio owns it can quickly become no one owns it"

Action Items

• Identify 1-2 high-friction problem areas
• Form pilot trios with clear problem definitions
• Establish shared success metrics
• Review and iterate after one quarter

Chapters

• 0:00 - The Business-IT Blame Game Problem
• 1:56 - Life in Technical Purgatory
• 5:29 - Why Traditional Fixes Don't Work
• 10:09 - Introducing the Trio Model
• 15:51 - Implementation and Decision Rights
• 23:42 - Measuring Success with Shared Metrics
• 24:50 - Leadership Changes Required
• 29:25 - Getting Started: A Practical Approach

How many meetings moved your team forward last week? Tom explores why most team rituals fail at building trust and alignment, sharing lessons from NASA JPL and startups on creating ceremonies that actually work.

Why Your Team Rituals Are Optimized for the Wrong Thing

Key Topics Covered

The Missing Middle Challenge

• Why mid-sized companies (200-1000 employees) face unique coordination challenges
• Too big for startup osmosis, too small for enterprise playbooks
• The need for distributed decision-making without dedicated alignment teams

Two Contrasting Standup Experiences

• NASA JPL: Nightly standups across three time zones that built trust and enabled handoffs
• Medical Startup: Transactional daily standups that created artificial harmony
• What made the difference: optimization for relationship building vs. status updates

The Five Dysfunctions of a Team Framework

1. Absence of Trust - Vulnerability-based trust, not competence trust
2. Fear of Conflict - Artificial harmony vs. productive disagreement
3. Lack of Commitment - What happens when people don't feel heard
4. Avoidance of Accountability - When standards become suggestions
5. Inattention to Results - Individual ego over team success

Three Practical Shifts for Better Rituals

Shift 1: Surface Vulnerability

• Leadership modeling uncertainty first
• Structured moments for admitting what you don't know
• Moving from posturing to problem exploration

Shift 2: Practice Disagreement

• Red team rotations in roadmap reviews
• Making challenge a role, not a personality trait
• Ensuring friction happens constructively in the room

Shift 3: Build Shared Context (Not Just Information)

• The difference between "here's the roadmap" and "here's the trade-off I struggled with"
• Smaller cross-functional context sessions vs. large all-hands presentations
• Enabling distributed decision-making through understanding reasoning

Key Questions for Diagnosis

• How much time was spent on information transfer vs. relationship building?
• Did anyone admit uncertainty without it being a problem?
• Was there constructive disagreement that led to better outcomes?
• Do people understand the reasoning behind decisions, not just the decisions themselves?

Resources Mentioned

• Patrick Lencioni's "The Five Dysfunctions of a Team" (2002)
• Concept of vulnerability-based trust
• Red team methodology for productive conflict

Next Episode Preview

Episode 14: The Product Trio Model - Making it Actually Work in Engineering-Heavy Organizations

Chapters

• 0:00 - The Meeting Problem: Status Theater vs. Real Progress
• 2:20 - The Missing Middle: Mid-Sized Company Challenges
• 4:21 - Tale of Two Standups: NASA vs. Startup
• 11:15 - The Five Dysfunctions Framework
• 16:13 - Three Practical Shifts for Better Rituals
• 23:55 - The Compounding Effect of Trust
• 28:39 - Diagnostic Questions and Next Steps