Edition #13 - Humans in the Loop

AI-first strategy only works when humans stay in control

By Mykel Salomon · 2026-01-24

<h3>Opening Reflection</h3>“Humans in the loop” gets thrown around like a safety blanket. But if we don’t define it, it becomes a slogan, and slogans don’t protect customers, agents, or brands.AI is here to stay. It will accelerate. It will automate. It will compress time. And in support, that speed can either become a superpower… or a trust disaster.Humans in the loop is the difference.<hr><h3>What “Humans in the Loop” Actually Means</h3>At its simplest: AI executes at scale. Humans retain judgment and accountability.Not as a last resort. Not as “we’ll step in if it breaks.” As a designed, visible control system.In customer support, “humans in the loop” means:<ul><li>AI can draft, suggest, route, summarize, retrieve</li><li>Humans must approve, decide, override, and own outcomes</li><li>AI escalates when confidence is low, risk is high, or emotion is present</li><li>The customer never gets trapped in automation without a path to a person</li></ul>This is systems engineering at work.<hr><h3>🌍 In the Wild: How Real Companies Are Doing It</h3><h3>1) AI-first chat + human handoff</h3><a target="_blank" rel="noopener noreferrer nofollow" class="text-[#4FD1C5] underline hover:opacity-80 BkNZKMPNQJhKPDcUmmzpKcxtpyYDbJDutfk " href="https://www.linkedin.com/company/klarna/">Klarna</a> reported its AI assistant handled two-thirds of customer service chats in its first month, with faster resolution times and fewer repeat inquiries. But the follow-up lesson matters too: Klarna later re-emphasized human talent in customer service, because efficiency gains don’t automatically equal trust and quality at scale.Takeaway: AI can absorb volume. Humans protect the relationship.<h3>2) AI agent resolves — humans govern the edges</h3><a target="_blank" rel="noopener noreferrer nofollow" class="text-[#4FD1C5] underline hover:opacity-80 BkNZKMPNQJhKPDcUmmzpKcxtpyYDbJDutfk " href="https://www.linkedin.com/company/intercom/">Intercom</a> ’s Fin is explicitly designed with human handoff controls, including workflows that route to humans when it’s the “safest option.” Some Intercom customer stories report high resolution rates (for example, Lightspeed reporting significant AI resolution volume), but the consistent pattern is: AI starts; humans remain reachable and informed.<h3>3) “Copilot” inside the agent workflow</h3><a target="_blank" rel="noopener noreferrer nofollow" class="text-[#4FD1C5] underline hover:opacity-80 BkNZKMPNQJhKPDcUmmzpKcxtpyYDbJDutfk " href="https://www.linkedin.com/company/zendesk/">Zendesk</a> Copilot focuses on helping agents with intelligent triage, summaries, and agent assist—AI behind the scenes, humans delivering the experience. Zendesk has also shared that AI triage can save an average of ~45 seconds per ticket compared to manual triage.<h3>4) Case summaries + knowledge creation with review</h3><a target="_blank" rel="noopener noreferrer nofollow" class="text-[#4FD1C5] underline hover:opacity-80 BkNZKMPNQJhKPDcUmmzpKcxtpyYDbJDutfk " href="https://www.linkedin.com/company/microsoft/">Microsoft</a> Dynamics 365 Customer Service Copilot supports case summaries and can draft knowledge articles from resolved cases, with humans reviewing and editing before publishing.Takeaway: HITL isn’t only frontline chat. It’s governance across the whole support lifecycle.<hr><h3>The AI-First Support Blueprint</h3>Here’s what “Humans in the Loop” looks like as a real structure.<h3>Layer 1 — Front Door AI</h3>Channels: chat, email, web, in-app, social. AI handles: FAQs, simple workflows, retrieval from knowledge base, order/status lookups.Guardrails:<ul><li>strict policy boundaries (“what it can and cannot do”)</li><li>confidence thresholds (when to answer vs. escalate)</li></ul><h3>Layer 2 — Orchestration + Retrieval</h3>A good doesn’t “wing it.” It retrieves answers from approved sources (knowledge base, policies, product docs), then responds within constraints.<h3>Layer 3 — Human Handoff by Design</h3>When risk rises, the system escalates before damage happens:<ul><li>low confidence</li><li>billing/legal/security topics</li><li>angry sentiment</li><li>repeated contact</li><li>exception requests</li></ul>And the handoff must include:<ul><li>conversation summary</li><li>customer history</li><li>what the AI tried So the customer doesn’t have to repeat themselves.</li></ul>(Intercom and Zendesk both emphasize this kind of assisted workflow/handoff capability in different ways.)<h3>Layer 4 — Copilot for Agents</h3>AI drafts replies, summarizes threads, suggests macros/knowledge. Humans edit and send. Humans own tone. Humans own judgment.<h3>Layer 5 — Quality + Feedback Loop</h3>Every escalation, correction, and failure becomes training data:<ul><li>improve KB</li><li>tighten policies</li><li>tune thresholds</li><li>update workflows</li></ul>This is where most companies fail: they deploy AI and skip the learning system.<hr><h3>How to Roll This Out Without Breaking Trust</h3><h3>Phase 1 — Start narrow</h3>Pick 10–20 low-risk intents (password reset, order status, “how-to” basics). Measure deflection and customer frustration.<h3>Phase 2 — Shadow mode</h3>Let AI draft responses internally first. Agents approve. Agents correct. You learn safely.<h3>Phase 3 — Confidence thresholds</h3>If the bot is uncertain, it escalates. Period. Don’t let it guess.<h3>Phase 4 — Escalation design</h3>Define “always-human” categories:<ul><li>refunds above a threshold</li><li>account access disputes</li><li>medical/safety concerns</li><li>harassment/threats</li><li>anything policy-exception related</li></ul><h3>Phase 5 — Human capability upgrade</h3>Train agents to become:<ul><li>reviewers</li><li>investigators</li><li>escalation pilots</li><li>trust rebuilders</li></ul>AI reduces repetitive work. Humans become the difference.<hr><h3>The Metrics That Actually Matter</h3>If you only chase deflection, you’ll build a fast machine that customers hate.Track:<ul><li>containment/deflection with CSAT held steady or improved</li><li>repeat contact rate (are issues actually resolved?)</li><li>escalation quality (did handoff preserve context?)</li><li>time-to-resolution for complex cases</li><li>agent workload + burnout indicators</li></ul>Zendesk’s “time saved per ticket” is real value, if it translates into better outcomes, not just faster closure.<hr><h3>💬 The Big Question</h3>Where do you need humans for trust, not because AI can’t answer, but because customers need to feel heard?<hr><h3>🧠 A 10-minute exercise for CX & Support leaders</h3>Write this down with your team:<ol><li>List your top 15 ticket types.</li><li>Mark each as: AI can solve / AI can assist / Human must own.</li><li>For “AI can solve,” define the confidence threshold and handoff trigger.</li><li>For “Human must own,” define what AI can do behind the scenes (summary, retrieval, draft).</li></ol>This becomes your HITL map.<hr><h3>🎙️ This Week’s Episode</h3>Season 2 · Episode 3 The Story of Support Driven — and the Power of CommunityGuest: <a target="_blank" rel="noopener noreferrer nofollow" class="text-[#4FD1C5] underline hover:opacity-80 ember-view" href="https://www.linkedin.com/in/scott-tran-188815a3/">Scott Tran</a> | <a target="_blank" rel="noopener noreferrer nofollow" class="text-[#4FD1C5] underline hover:opacity-80 BkNZKMPNQJhKPDcUmmzpKcxtpyYDbJDutfk " href="https://www.linkedin.com/company/support-driven/">Support Driven</a>If “Humans in the Loop” is the blueprint for AI-enabled support, this episode is the reminder of why it matters.Scott Tran built Support Driven from the simplest beginning: two people in a Slack. No scale. No hype. Just consistency, care, and showing up, one human at a time. And that’s the lesson that hits hardest right now.Because in a world racing toward automation, community becomes a form of protection. Not from technology , but from what happens when technology moves faster than trust.In this conversation, we explore:<ul><li>How community scales what org charts can’t: belonging, wisdom, resilience</li><li>Why support is not a cost center — it’s a human system</li><li>What it means to build “one person at a time” when the world is obsessed with speed</li><li>Why AI-first support still needs a human foundation: empathy, context, shared learning</li><li>The real power of Support Driven: not tools, people helping people</li></ul>If you’re building AI into support, don’t skip this. Because your automation will only be as strong as the human network behind it.🎧 Listen or watch the episode:<ul><li>Spotify → <a target="_self" rel="noopener noreferrer nofollow" class="text-[#4FD1C5] underline hover:opacity-80 BkNZKMPNQJhKPDcUmmzpKcxtpyYDbJDutfk " href="https://open.spotify.com/episode/6oHco1jaT2wuVDWjWGJys9?si=G9VwXqAPTzCjP6HYjnNuJw">Watch Here</a></li><li>Apple Podcasts → <a target="_self" rel="noopener noreferrer nofollow" class="text-[#4FD1C5] underline hover:opacity-80 BkNZKMPNQJhKPDcUmmzpKcxtpyYDbJDutfk " href="https://podcasts.apple.com/us/podcast/s2e3-the-story-of-support-driven-and-the-power/id1843402023?i=1000746032343">Listen Here</a></li><li>YouTube → <a target="_self" rel="noopener noreferrer nofollow" class="text-[#4FD1C5] underline hover:opacity-80 BkNZKMPNQJhKPDcUmmzpKcxtpyYDbJDutfk " href="https://youtu.be/4Du5Ggs57ws?si=8_4aO4JKhCZ-vpF3">Watch Here</a></li></ul><img class="max-w-full h-auto rounded-md my-4" src="https://media.licdn.com/dms/image/v2/D4E12AQGNgI9dBVokTA/article-inline_image-shrink_1000_1488/B4EZvwilztKsAQ-/0/1769267161521?e=1771459200&v=beta&t=M6zjWBecqZcXAcJjirOzFXI8PHGRbcD5eW5xlYnYXW0" alt="Article content"><hr><h3>Thought to End the Week</h3>Automation without humans in the loop is just speed with no conscience. Build AI-first support, yes. But keep humans where trust is won, and where it’s repaired.— Mykel