1,588 AI Disasters Nobody Is Talking About | Latha Karthigaa,
Hosted by Mykel Salomon · Featuring Latha Karthiga Murugesan · Published
Mykel Salomon is the founder of The Human Protocol Group, creator of The Human Protocol Framework, and host of The Human Protocol Podcast. He explores how technological change affects human beings, and what it takes to stay human while building the future.
Guest: Latha Karthiga Murugesan
Everyone is talking about what AI can do. Far fewer people are talking about what happens when AI gets it wrong. @Latha-ai-governance PhD, runs the Global AI Certification Council and publishes daily breakdowns of real-world AI incidents, looking at what went wrong, why it happened, and how it could have been prevented. In this episode of The Human Protocol, she takes Mykel Salomon through some of the most disturbing examples she has uncovered. From a Dutch childcare benefits algorithm that contributed to families losing homes and children being placed in foster care, to AI falsely accusing a Yale student of cheating, to chatbots giving dangerous advice, the conversation shows what happens when AI systems make decisions without sufficient human oversight. Latha also explains why AI governance isn't simply bureaucracy slowing innovation down. She compares governance to the guardrails on a highway: they allow organizations to move faster because the boundaries are clear. The conversation moves from today's AI failures into tomorrow's risks, including autonomous AI agents, prompt injection, deepfakes, misinformation, and the difficulty of knowing what AI systems are doing when thousands of agents operate inside an organization. The bigger question isn't whether AI will fail. It's whether we're learning from the failures that have already happened. YOUTUBE CHAPTERS Time Chapter 00:00 Introduction 01:12 Why AI incidents matter 02:38 The 1,588 AI incident database 04:10 AI failures before ChatGPT 05:35 The hidden cost of AI mistakes 07:05 AI and the future of jobs 08:30 When AI gets people wrong 10:05 The student accused of cheating 12:00 Why AI output is not evidence 13:42 AI governance vs. AI hype 15:20 The Dutch childcare benefits scandal 18:05 How algorithmic bias becomes real harm 20:10 What happens when models drift 22:05 AI systems and accountability 24:00 Agentic AI changes the risk 26:10 When AI can take actions 28:15 Prompt injection explained 30:00 The $1 Chevy Tahoe incident 32:05 How attackers manipulate AI 34:10 Deepfakes and trust 36:00 The $25M deepfake fraud 38:20 Why humans still get fooled 40:10 AI regulation and governance 42:00 What organizations should document 44:05 Human oversight in high-stakes systems 46:00 The problem with unknown unknowns 48:05 We do not know what we do not know 49:30 How businesses should approach AI 51:05 Where AI should not replace humans 52:50 Building better governance 54:10 Advice for AI leaders 55:25 What responsible AI looks like 56:40 The human role in an AI future 58:05 Final thoughts 59:15 Closing 59:56 End ABOUT THE GUEST Latha Karthigaa, PhD is Director and Head of AI Governance at the Global AI Certification Council (GAICC). GAICC focuses on AI governance, certification, training, and professional development, including work aligned with ISO/IEC 42001. Latha holds a PhD in Software Engineering from the University of Auckland, New Zealand, awarded in 2017. She founded an AI and marketing consulting firm in 2019 and has built and led AI solutions for finance and legal firms across the USA, UK, Australia, and New Zealand. In this episode, Latha brings a governance-focused lens to real AI incidents, helping explain how failures happen, why they can scale quickly, and what organizations should do to build stronger controls and human oversight. Guest social links: Instagram: https://www.instagram.com/global.aicertification.council YouTube: https://www.youtube.com/@Latha-ai-governance LinkedIn: https://www.linkedin.com/in/lathakarthigaa/ GAICC: https://gaicc.org/ AI Strategy & Execution Roadmap Thinking about implementing AI in your business? Download Mykel Salomon's free 10-Step AI Strategy & Execution Roadmap to clarify the business problem, identify the right opportunities, decide what should remain human, and move from idea to execution. Clarity first. Tools second. Humans always. Roadmap: https://dub.sh/thp-roadmap-riverside The Human Protocol Newsletter Technology is changing quickly. The human questions matter just as much. Subscribe to The Human Protocol newsletter for weekly perspectives on AI, leadership, work, and what it means to stay human while building the future. Newsletter: https://dub.sh/thp-newsletter-riverside
Conversation Summary
In this episode of The Human Protocol Podcast, Latha Karthigaa, PhD, shares her insights on the often-overlooked incidents of AI failures and their real-world implications. The conversation dives into the ethical, social, and practical challenges posed by AI technologies, emphasizing the essential role of governance and human oversight in AI development. By highlighting specific cases such as the Dutch childcare benefits scandal and a Yale student's wrongful accusation, Latha makes a compelling case for heightened awareness and responsibility in AI governance, drawing parallels between effective governance and societal guardrails.
AI Failures are More Common Than You Think
Latha Karthigaa exposes the widespread incidents of AI failures that often remain hidden from public discussion. From governments collapsing to individuals' lives being severely disrupted, these stories underscore the imperative need for proper AI governance and oversight.
Governance as a Catalyst, Not a Barrier
Contrary to slowing innovation, Latha argues that AI governance serves as guardrails, similar to those on a highway. These guardrails don't impede speed but ensure safety, helping organizations to innovate responsibly and sustainably over the long term.
The Human Cost of Poor AI Decisions
Stories shared in the podcast reveal the devastating personal impacts of AI errors, such as families losing their homes and careers being unjustly derailed. These incidents highlight the essential need for empathy and human oversight in developing AI systems.
AI Governance Offers New Career Paths
Latha emphasizes the emerging necessity for AI governance professionals, highlighting it as a growth area in the tech field. The role requires not just technical skills but also empathy and a strong understanding of ethical implications, offering a new avenue for those concerned about AI replacing jobs.
Legislation and Penalties Are Crucial for Compliance
Effective AI governance requires enforceable legislation and penalties, as evidenced by the European Union's AI Act. Regulatory frameworks that impose fines and require compliance help prevent the misuse and negative externalities of AI technologies, encouraging organizations to align with ethical standards.
Final Thoughts
The conversation with Latha Karthigaa underscores the critical importance of empathy, governance, and responsibility in deploying AI technologies. AI presents vast opportunities but also significant risks that must be managed through robust, ethical governance frameworks and human oversight. As AI systems become more integrated into daily life, the need for conscientious governance will only intensify.
Key Takeaways
- AI Failures are More Common Than You Think
- Governance as a Catalyst, Not a Barrier
- The Human Cost of Poor AI Decisions
- AI Governance Offers New Career Paths
- Legislation and Penalties Are Crucial for Compliance
Frequently Asked Questions
What is AI governance and why is it important?
AI governance involves creating frameworks and policies to guide the ethical and responsible development and use of AI systems. It's crucial for preventing potential harm and ensuring technologies are used in a way that benefits society.
How can individuals recognize when AI is manipulating decisions?
Manipulation by AI systems can be subtle and often requires vigilance and awareness. Individuals should look for unusual patterns in AI recommendations and be cautious about AI outputs that lack transparency or sound too good to be true.
What are some examples of real-world AI failures?
Examples include the Dutch childcare benefits scandal where an algorithm falsely accused families of fraud, resulting in severe personal impacts, and wrongful accusations against students for cheating based on AI misinformation.
How can AI governance contribute to career opportunities?
As AI systems proliferate, there's a growing need for governance professionals. These roles involve ensuring compliance with ethical standards and laws, requiring a balance of technical knowledge and empathy, thus presenting new career avenues.
How do regulations impact AI deployment in organizations?
Regulations like the EU's AI Act help ensure that AI systems are developed and used responsibly. They impose fines and require compliance, creating a safer environment for innovation by setting clear ethical and operational standards.