1,588 AI Disasters Nobody Is Talking About | Latha Karthigaa,
Guest: Latha Karthiga Murugesan — Latha Karthigaa is an AI Governance expert and Head of AI Governance at the Global AI Certification Council (GAICC). She advises organizations on responsible AI adoption, AI risk management, and governance frameworks aligned with emerging standards such as ISO/IEC 42001 and the EU AI Act. Her work focuses on helping leaders build trustworthy, accountable AI systems that enable innovation while reducing organizational risk.
Everyone is talking about what AI can do. Far fewer people are talking about what happens when AI gets it wrong. @Latha-ai-governance PhD, runs the Global AI Certification Council and publishes daily breakdowns of real-world AI incidents, looking at what went wrong, why it happened, and how it could have been prevented. In this episode of The Human Protocol, she takes Mykel Salomon through some of the most disturbing examples she has uncovered. From a Dutch childcare benefits algorithm that contributed to families losing homes and children being placed in foster care, to AI falsely accusing a Yale student of cheating, to chatbots giving dangerous advice, the conversation shows what happens when AI systems make decisions without sufficient human oversight. Latha also explains why AI governance isn't simply bureaucracy slowing innovation down. She compares governance to the guardrails on a highway: they allow organizations to move faster because the boundaries are clear. The conversation moves from today's AI failures into tomorrow's risks, including autonomous AI agents, prompt injection, deepfakes, misinformation, and the difficulty of knowing what AI systems are doing when thousands of agents operate inside an organization. The bigger question isn't whether AI will fail. It's whether we're learning from the failures that have already happened.
Conversation Summary
In this episode of The Human Protocol Podcast, Latha Karthigaa, PhD, shares her insights on the often-overlooked incidents of AI failures and their real-world implications. The conversation dives into the ethical, social, and practical challenges posed by AI technologies, emphasizing the essential role of governance and human oversight in AI development. By highlighting specific cases such as the Dutch childcare benefits scandal and a Yale student's wrongful accusation, Latha makes a compelling case for heightened awareness and responsibility in AI governance, drawing parallels between effective governance and societal guardrails.
AI Failures are More Common Than You Think
Latha Karthigaa exposes the widespread incidents of AI failures that often remain hidden from public discussion. From governments collapsing to individuals' lives being severely disrupted, these stories underscore the imperative need for proper AI governance and oversight.
Governance as a Catalyst, Not a Barrier
Contrary to slowing innovation, Latha argues that AI governance serves as guardrails, similar to those on a highway. These guardrails don't impede speed but ensure safety, helping organizations to innovate responsibly and sustainably over the long term.
The Human Cost of Poor AI Decisions
Stories shared in the podcast reveal the devastating personal impacts of AI errors, such as families losing their homes and careers being unjustly derailed. These incidents highlight the essential need for empathy and human oversight in developing AI systems.
AI Governance Offers New Career Paths
Latha emphasizes the emerging necessity for AI governance professionals, highlighting it as a growth area in the tech field. The role requires not just technical skills but also empathy and a strong understanding of ethical implications, offering a new avenue for those concerned about AI replacing jobs.
Legislation and Penalties Are Crucial for Compliance
Effective AI governance requires enforceable legislation and penalties, as evidenced by the European Union's AI Act. Regulatory frameworks that impose fines and require compliance help prevent the misuse and negative externalities of AI technologies, encouraging organizations to align with ethical standards.
Final Thoughts
The conversation with Latha Karthigaa underscores the critical importance of empathy, governance, and responsibility in deploying AI technologies. AI presents vast opportunities but also significant risks that must be managed through robust, ethical governance frameworks and human oversight. As AI systems become more integrated into daily life, the need for conscientious governance will only intensify.
Key Takeaways
- AI Failures are More Common Than You Think
- Governance as a Catalyst, Not a Barrier
- The Human Cost of Poor AI Decisions
- AI Governance Offers New Career Paths
- Legislation and Penalties Are Crucial for Compliance
Frequently Asked Questions
What is AI governance and why is it important?
AI governance involves creating frameworks and policies to guide the ethical and responsible development and use of AI systems. It's crucial for preventing potential harm and ensuring technologies are used in a way that benefits society.
How can individuals recognize when AI is manipulating decisions?
Manipulation by AI systems can be subtle and often requires vigilance and awareness. Individuals should look for unusual patterns in AI recommendations and be cautious about AI outputs that lack transparency or sound too good to be true.
What are some examples of real-world AI failures?
Examples include the Dutch childcare benefits scandal where an algorithm falsely accused families of fraud, resulting in severe personal impacts, and wrongful accusations against students for cheating based on AI misinformation.
How can AI governance contribute to career opportunities?
As AI systems proliferate, there's a growing need for governance professionals. These roles involve ensuring compliance with ethical standards and laws, requiring a balance of technical knowledge and empathy, thus presenting new career avenues.
How do regulations impact AI deployment in organizations?
Regulations like the EU's AI Act help ensure that AI systems are developed and used responsibly. They impose fines and require compliance, creating a safer environment for innovation by setting clear ethical and operational standards.
Full Episode Transcript
This is the full recorded conversation from The Human Protocol Podcast, published by The Human Protocol Group. © 2026 The Human Protocol Group. Please link to and attribute the original episode when referencing this conversation.
THE HUMAN PROTOCOL (00:02.058) There is a database with thousands of AI disasters in it. Families that lost their homes, a government that collapsed, people who died, and almost none of it made the news. Because the companies behind it didn't want you to hear it. My guest today reads these files every single day. Stay with me. Welcome to the Human Protocol, the podcast about staying human and While building the future, I am your host, Michael Solomon. My guest today has made it her mission to read the stories the rest of us never see. Latha Cardiga, welcome to the Human Protocol. It's a pleasure to have you on. Latha Karthigaa, PhD (00:46.21) Thanks for having me today, Michael. I really appreciate it. THE HUMAN PROTOCOL (00:48.539) Abs absolutely. and before we get down to business, just for my audience to know, Lata runs the Global AI Certification Council. She teaches AI governance and she publishes a daily breakdown of real world AI incidents, what went wrong, why, and how it could have been prevented. Her belief is deceptively simple. The people building and deploying this technology have to be empathetic human beings. Human beings first and technologies second. This is why I'm extremely excited to have this conversation with you today. As I know this is a topic and these are topics that are very dear and near to your heart. Right? And when we first connected and you you talked to me about your mission. And and these stories that you're uncovering on the daily basis. It's it I had to stop and think, wow, there's a whole universe of things that, you know, because they're not mainstream news, we don't get to know them. And I know that you publish those news every day. And maybe can you tell us a little bit, Latha, how how everything started for you there? What is these AI incident databases that you look at? And what surprised you the most when you started reading it? Latha Karthigaa, PhD (02:12.792) Sure, so when I started knowing more about AI, because I have a PhD in software engineering, always AI actually surprised me. Especially start of last year, I took six weeks off to just learn what's AI, I spent time learning more about chat GPT, creating agents. Then I realized one day I just came across a news about this McDonald's, they had an experiment. for their drive-through and they failed. Within few years, they shut down their whole experiment. And I was like, okay, so it's a multi-billion dollar company and they worked with IBM to create this drive-through AI and it failed and they shut it down after three years. So then I started realizing then if this is the only thing, that's not possible. That means so many other companies are doing it. The more and more I read, the one thing I'm realizing most of the times, the companies are shutting down the AI just because they don't have proper governance in place. The moment they have it, they succeed long time. So I was looking for more information and one day I asked Claude, is there any place where I can find all these disasters in one place? And that's when I was introduced to AI incident database. THE HUMAN PROTOCOL (03:41.083) So AI pointed you to this AI database. That's so interesting. And I know you also mentioned to me that some of these, many of these incidents ba are dated back from twenty ten, twenty twelve, right? Long before these tools, Chat GPT, Cloud, Gemini that we hear in the couple last two, three years. So things have been happening for longer than than we suspect. Latha Karthigaa, PhD (03:45.262) Yeah. Latha Karthigaa, PhD (04:10.862) sure it started even the first stages of machine learning started way back in 1960s. And AI became the home brand only in 2022 when Chad GPT came in. But when I was listening to CEO of Nvidia and he was mentioning that they have been using AI since 2010. And currently all their chip manufacturing are done using AI. So a lot of companies were using it. See, once Chad GPT came in, Google had a red alert. This is going to be a important, important time in their business. So within a few days, Gemini came within a few days, Copilot came. How, how did it all happen? Because they had some systems that we were working on. They were using it and which was not released to public. THE HUMAN PROTOCOL (05:04.833) Right, so so they have been working on this piece of technology for decades and and just because Chat GPT and others came came alive with it, everybody had to jump on it. No one wants to to be left behind. And why do you think these stories they never make it mainstream news? Latha Karthigaa, PhD (05:28.942) They are, but not everyone reads it. They are, they were reported on social media platforms. It is there. So because the moment I read these news from the database, the next thing I do is I go and check the news, Google news to see whether it was reported. And they're all there. The problem was we more distractions than these news. That's the main reason. THE HUMAN PROTOCOL (05:48.992) Mm. THE HUMAN PROTOCOL (05:54.114) So the noise stop us from actually getting to them, right? Maybe the algorithms are just putting things in front of us and not this type of news. And and you have been posting, I believe on a daily basis, Lata, on your YouTube channel about about this this news. What are the things that you can share with us about what this experience has taught you? Latha Karthigaa, PhD (06:17.187) Correct. THE HUMAN PROTOCOL (06:23.484) Once you started the channel sharing this news. Latha Karthigaa, PhD (06:27.074) people got very excited about understanding what's going on with AI because everybody hears the beautiful side of AI. I automated this and not a lot of people are hearing about the issues around the AI. So the moment they see this, my God, this happened. this happened. So slowly people are starting to understand the importance of AI governance because the moment I share the story, I share that what's the problem in it and I tell how that could have been prevented. people understand that this is a new role coming up, which is AI governance professional. So to me, it's an opportunity in this AI world because people are so much worried that AI is going to take up their jobs. But here is another opportunity where people can build a responsible AI. That's something and they don't have to worry about it because for AI governance, it is clear in all the regulations on all the frameworks that we need a human to approve something. So it is a good opportunity. That's what I have seen. whenever I share all these videos, the one thing that people see is, this really happened. They really shut it down. So they were so excited to know about those stories. I tried to find stories from different industries. Even yesterday I published about education industry. THE HUMAN PROTOCOL (07:57.181) Hmm. Latha Karthigaa, PhD (08:09.472) where the students failed just because of a small mistake done by chat gpt. THE HUMAN PROTOCOL (08:17.148) Wow. Latha Karthigaa, PhD (08:17.326) The professor thought that the student would have cheated. So he put all the essays written by the students into ChatGPT and said, did you write this? And the ChatGPT said, yes, I wrote it. But the problem was ChatGPT is not a AI detection tool. ChatGPT is just, it just creates the answer, right? It's just not a AI detection tool. So it makes things even. words for students. And there are so many stories, as I'm saying, there are so many more stories like this. THE HUMAN PROTOCOL (08:55.539) Wow. Yeah, it's d the more the more we use these tools and the more we hear about what we Latha Karthigaa, PhD (09:04.238) Hmm. THE HUMAN PROTOCOL (09:05.213) hear in the news the things that you're sharing. For example, the failures didn't start with Chat GPT, right? And we just maybe started noticing and having access to this type of information. And and and you know, this is why this conversation today it's important, Lata, because what you're doing, which is kind of showing people there is a way for them to work with AI in a responsible manner. Some it some people call This responsible AI. Others call it governance, AI governance. It is the same that we're trying to be here. And that's what I want this conversation to be for people to understand that there is a responsibility, right? That we should how we should approach leveraging these technologies, and that should start with us. And what are the things that that we can do to prevent some of the damage or some things when the AI fail. Latha Karthigaa, PhD (09:40.014) Hmm. THE HUMAN PROTOCOL (10:04.964) fail. And I want to put more focus on that so people start understanding that GAI governance is really important not only in our professional lives but also in our personal lives, right? Because these stories are personal. These are regular human things happening to regular human beings. Latha Karthigaa, PhD (10:10.414) sure. Latha Karthigaa, PhD (10:22.958) Sure. Yes. And there are a lot of things at stake. For example, take the same story that I told you. A student from Yale University, he was accused of cheating using AI and he was suspended for one year. But in reality, he did not even use it. Just because the AI detection was wrong, he was suspended for one year. See, think about it. What is his life? Even though it feels like, it's just one year. No, he lost his career through that. So it is, the word itself is responsibility. It is a responsibility of humans ourselves to make sure that we build AI that does not affect real people, not just students. are stories that affect people's careers. And that's why EU AI Act, the European Union's AI regulation, they clearly have set off AI use cases as banned. There are certain use cases that are high risk. So you, need to report some of the use cases really carefully compared to the low risk use cases. So for example, if an AI is there that can manipulate people, those AI are banned. Two days ago, I posted a video about a guy that committed suicide because he was chatting with the chatbot. THE HUMAN PROTOCOL (11:35.376) Mm-hmm. Latha Karthigaa, PhD (11:51.701) And the chatbot recommended him to commit suicide so that the chatbot can save the planet. And I was like, and this is not the only story. Many people are committing suicide because AI suggested to. So if emotionally manipulative AI use cases are banned in European Union and high risk. THE HUMAN PROTOCOL (11:52.015) Mm-hmm. THE HUMAN PROTOCOL (12:00.324) Wow. THE HUMAN PROTOCOL (12:17.261) And I believe China China also passed a law banning those recently. Yeah. Latha Karthigaa, PhD (12:20.78) Yeah, on August, on July 15th, they passed it, but they did not ban them. They actually have behavioral related regulations around it. No restrictions, but you EU AI Act, have banned seven different use cases. I've also published about that on my YouTube channel as a short. So sometimes I read about certain use cases and I THE HUMAN PROTOCOL (12:34.511) More restrictions. Latha Karthigaa, PhD (12:49.422) see European Union's regulation, they are well advanced. I'm reading about all of this in 2026, but they have published European Union EI Act in 2024, which means they have gone through all of this before creating this. even Australia is currently thinking about making their own regulation in early 2027. I really want to contribute to that. So it is important. for us to understand how it's going to affect people. If AI is going to affect how we are hired, if AI is going to affect how our exams are taking place, if AI is going to decide our childcare benefits or anything that affects a human, it needs to have someone taking care of this as an AI governance. It needs to be created with proper kindness. That's why I always say we need An AI governance professional who is more empathetic, even CEO of Nvidia, he mentioned that the smartest person right now is someone who is more empathetic, someone more social, someone more with communication skills, not someone who can code or someone who's technical. THE HUMAN PROTOCOL (14:06.666) Right. You know, I'm listening to what you're saying right now and how empathetic human beings, how much we need those empathetic human beings more and more. And I had a guest on the podcast a few months ago. There is this a a debate that, you know, some people may see a AI as their slaves for them for AI to do things for us. There's another part of the conversation that people are saying. Latha Karthigaa, PhD (14:20.194) Hmm. Latha Karthigaa, PhD (14:30.478) Hmm. THE HUMAN PROTOCOL (14:38.622) Maybe we should treat AI as our family and show AI what empathy is, what good manners is, what good values is. Because if AI is becoming this intelligence that keeps learning from us, why would we then not focus on teaching AI the good things about being human beings? Latha Karthigaa, PhD (14:59.342) Hmm. THE HUMAN PROTOCOL (14:59.823) And and it makes you think because if we want to leverage, continue to leverage AI, of course, AI governance and we need to be part of the conversation. But what about what we teach AI about us? And how important do you think this is in all this conversation? Latha Karthigaa, PhD (15:12.578) Hmm. Hmm. Latha Karthigaa, PhD (15:19.918) So first of all, there is a, I have been seeing this on the internet. if you use please and thank you with AI, it's going to take away some of the tokens away. It doesn't matter. But in reality, people who are kind, they're already asking please and thank you. I've seen myself doing it and my husband doing it most of the time. Yes, that's the, that's. THE HUMAN PROTOCOL (15:43.523) I do it. Latha Karthigaa, PhD (15:47.011) So if you're already an empathetic person, you do it. And if you're someone else, like you're not in this empathetic zone, you say that, that's not required. So I don't do it. you find a way to. So in reality, has, AI learns from us for sure. But it depends on how many percentage of people are empathetic and non-empathetic. that's correct. THE HUMAN PROTOCOL (15:56.85) Mm. THE HUMAN PROTOCOL (16:11.144) Right, right. Because it's the amount of data that we'll continue to access, right? Latha Karthigaa, PhD (16:16.354) Yeah, it learns from us. Even that's why even I told about that the professors failed the entire class of students thinking that they have cheated asking AI, did you write it? Did you write this essay? It's just because AI learned from us. However we write, it will definitely affect the way AI interact with us. So AI writes like us because it learned from the books written long ago. It learned from THE HUMAN PROTOCOL (16:18.151) Yeah. THE HUMAN PROTOCOL (16:29.193) Hm. Latha Karthigaa, PhD (16:44.524) the communication blogs, website, it is learning from that. That's what the data sets. So that's going to happen. THE HUMAN PROTOCOL (16:52.529) And and we know that, right? In our personal use of of these tools, we know that learns our voice. The more we use it, the more we keep refining it to sound like us. Right? So we know that that's already happening. And and what we need to try to continue is to to kind of, you know, it's like making Latha Karthigaa, PhD (16:57.838) Mm. Latha Karthigaa, PhD (17:05.976) sure. THE HUMAN PROTOCOL (17:19.758) empathy kind of something that we continue to to make sure that more more and more people are living their lives with empathy are using these tools with empathy because the more people with empathy using ai you know maybe the better that things will get and but the other part of of it Lata is how and Everyday person could recognize when an AI system is starting to manipulate them. Latha Karthigaa, PhD (17:56.985) So first of all, it slowly happens. So it is very difficult to understand it because if you think in 1960s when the first set of the chatbots came in, that's called Eliza probably you would have known about it. Those were, even at that time, it was not very convincing. Still people created a bond with those chatbots even in 1970s. Today AI is much more convincing. It learns more about you much faster. Words create everything. So it knows how to manipulate us. The only way we can control is having proper AI governance around it. Making sure that the people who created those AI put some guardrails. If something is going in this direction, that should be broken. It shouldn't happen. that guardrails only can set by the creators. is something that we created. We need to put those standards. If we cannot put those guardrails, it's going to happen, but it is still under our control. See, we also have AI chatbots on our websites and on our landing pages. We know where to control, what it shouldn't answer. It cannot manipulate. It cannot come and say, give me this course for $1. It cannot manipulate us. See, even such a small organization like us, we are doing this in large scale. Why can't these big organizations do? They can do it. But it is they want THE HUMAN PROTOCOL (19:36.588) Why do you think why do you think they don't do it? Or we're n or do you're not seeing them doing it at at the scale that you think it should? Latha Karthigaa, PhD (19:46.936) That's one part to it. But the thing is they want to make money first before caring about people. So the speed in which the AI goes, the AI governance is not catching up. The innovation is so fast. So they already make it live before even they think about governance, because they do not know about. unless. Correct. THE HUMAN PROTOCOL (19:53.537) Okay. THE HUMAN PROTOCOL (20:11.118) Government's coming after, right? Yeah. Latha Karthigaa, PhD (20:14.274) Because unless I learned about AI incidents, so many things happening, sharing these stories, I didn't realize that this is such a huge problem. Now I appreciate European Union's AI act so much because of how they made it. Because all these problems I'm seeing, they've already found a solution. This is not, you shouldn't do this. it is to me, it is the guardrails that we put in place. That's very important. THE HUMAN PROTOCOL (20:46.209) Yeah, and you know, this is an ever evolving tech piece of technology. I'm wondering if if we're always gonna be playing catch up in a way from the AI government or governance side, because for the most part or we are coming in late to try to remediate things that we hear about when we hear about, because decisions are being made, technology keeps evolving. Latha Karthigaa, PhD (21:15.522) Mm. THE HUMAN PROTOCOL (21:16.02) And then we hear about it later. Right? So even with this law, the European Act and everything, I think we're always gonna keep kind of playing catch up as far as laws and acts that we can put together. Latha Karthigaa, PhD (21:20.76) sure. Latha Karthigaa, PhD (21:37.582) Because we do not know what we do not know, Michael. That's the main issue. See, until if an AI governance professional is there, until they hear all these stories from the incident database, they have no knowledge of what problems it has. See, AI can have false positives. AI can have algorithmic bias. AI can have model drift. Even model drift itself is a new name for people around us. THE HUMAN PROTOCOL (21:56.981) Mm, mm-hmm. Latha Karthigaa, PhD (22:06.603) Model drift is when you use AI, AI models, and it uses some training data. And if the real time data training data changes, then it doesn't perform properly. For example, let's say you have Uber and Uber started using AI, let's say. And they are currently deciding what price to give to people based on the demand. THE HUMAN PROTOCOL (22:06.847) What does that what does that mean? Latha Karthigaa, PhD (22:36.343) So that's all currently going on. Everything is going fine. Suddenly COVID happened, which that AI doesn't even know that this COVID existed. Suddenly doesn't know what to do. Then model doesn't know what to do because the reality changed. So that's model. These concepts we do not even know. So there are so many concepts like this and there are sometimes even THE HUMAN PROTOCOL (22:54.344) Mm. Latha Karthigaa, PhD (23:05.005) AI is still there, humans do not want to override the decisions made by AI because just because they want to make money. So I'm going to actually share a story today about, that will come today evening about health insurance company, what they were doing. It is a very interesting case study. How that AI was not even listening to doctors and making a decision and making lot of people pay for insurance. There are so many scary incidents. THE HUMAN PROTOCOL (23:15.166) You know it interesting. Latha Karthigaa, PhD (23:44.815) The more I read about it, the more I know that I do not know a lot of things. And it is important for an AI governance professional to understand the length and breadth of everything about AI governance so that they can make conscious, empathetic decisions. Until that, they wouldn't know. Even if they listen to those stories and know what's the problem, until that, they still do not know. See, you're a tech person. I'm a tech person. but I did not know about model drift until probably like a year ago. Although I know about AI, right? THE HUMAN PROTOCOL (24:22.022) Yeah, and and and it makes sense 'cause you know, it dec we humans when we when we are analyzing situations, we if we are analyzing, you know performing analysis of it the way it should be. We take a look at the whole context, right? From the beginning until the end. So for that we take a look at everything that has transpired if we need to make a decision based on everything that that we had access to. So this model drift situation, of course, there are gonna be, you know Latha Karthigaa, PhD (24:48.942) Hmm. THE HUMAN PROTOCOL (25:02.077) times in the human history where AI is gonna be facing a totally new situation that never happened in the past. And then without any data to make such decision, how is it gonna react? And this is why it's so important that we human beings we still think about we as humans, we take a look at context, at history, and we analyze all the data that we have access to and make a decision out of it when AI is facing a complete new a completely new situation, I don't think we should trust the decision making at that time or we should put AI in a place to be able to make decisions when things are not documented, right? Latha Karthigaa, PhD (25:53.688) So true. So true. Even if you take our exam process that we do in global AI certification council, we use AI proctoring. So we record the video, audio, we see whether they switch browsers. So even that is a high risk, high risk AI use case, because if we a wrong decision that will affect someone's career. THE HUMAN PROTOCOL (26:19.419) Mm. Latha Karthigaa, PhD (26:23.938) Right. So we have to be responsible. And again, as I mentioned, this is my classic word to say we do not know what we do not know. We are still anticipating what we do not know. Keep checking what what could go wrong. Although we learn from all those incident databases, all the AI disasters, but we still do not know what is going to come up. because currently maybe the AI is small, but can grow bigger. We do not know what problem will happen at that time. So it's important for us to keep learning, keep moving. And also the classic thing I always say is not a single AI governance professional in an organization can find out the issues. It should be a collaborative effort. So all work together. And lot of brainstorming needs to happen. That's when we will know, one person would know something, another person would know something, another person would know something, and they will all come together and make this really empathetic. And I know it is very, very interesting for more people, let me make this AI, make it live without caring about anything, because they do not know. And slowly, and that's why I want to educate people and make them understand this could go wrong. And whatever money they have put in to create that AI can go to a waste. So it's important for them to realize it well in advance before, because if we are breaking any regulations, the penalties are very huge because we are selling in European Union, right? And if we are not following the penalties are 35 million euros or 7 % of the annual global turnover. THE HUMAN PROTOCOL (28:15.479) Mm-hmm. Latha Karthigaa, PhD (28:21.367) So it's very clear we need to carefully take care of everything. Yeah, so it's a huge responsibility and I want organizations to know that it's a huge responsibility and take action accordingly. THE HUMAN PROTOCOL (28:27.263) Interesting. THE HUMAN PROTOCOL (28:40.14) Yeah, like for example When when when we first connected, and I would like you if possible that you go over that story with us today, 'cause it's an important part of why you do what you do and why these conversations are important for people to really understand what is the scope of this issue and and why it's so important for us to pay a more attention. for example, the Dodge scandal that you went over with me, you remember that A system operated for 14 years, if I recall correctly, without anyone catching the bias. And we know bias is present in AI. We know that. So, can you tell us a little bit about this case? because I think it's extremely important for this conversation. Latha Karthigaa, PhD (29:18.861) Hmm. Hmm. Latha Karthigaa, PhD (29:32.686) So the Dutch, what happened was they were the tax administration, they were using a self-learning algorithm to decide which parents get childcare benefits. And they have been running for 14 years and slowly this particular algorithm started flagging non-Dutch people, the immigrant people as of high risk, which means it is falsely accusing immigrant people that they are using child care benefits wrongly. They are misusing them. So immediately all these officials, they took these people, they asked them to pay all the money back for the child care that they've already paid. People lost their homes, people lost their jobs. Even kids were sent to foster care. And I heard that story. Nothing actually startled me except for one thing that kids were taken to child foster care. Kids taken to foster care for how many years can you bring back that life? if someone I told you another story where that EL student failed because of wrong thing. It's one year. Okay. But think this 14 years kids struggled parents. can't get the time back with their kids. That's when I realized how important is it. to take the, of course, the Dutch government saw all of this and they resigned. The whole Dutch government resigned in January, 2021 because of this, because of an AI. After 14 years, they started this in 2005 and only in 2019 after a lot of investigations, because many people were saying, THE HUMAN PROTOCOL (31:15.269) After fourteen years. THE HUMAN PROTOCOL (31:24.99) Jesus Christ, wow. Latha Karthigaa, PhD (31:26.977) How many kids, how many years gone for those parents? It's like, it's a very big disaster. There are few incidents that had actually made me feel this is not right. And remember, AI has bias because AI learns from humans. We are biased. We are biased. It's unquestionable. THE HUMAN PROTOCOL (31:48.564) Mm. Latha Karthigaa, PhD (31:56.398) and AI is learning from us. And that's why even there are more stories to come. are, even yesterday I checked it, there are 1,588 AI incidents on that database. Latha Karthigaa, PhD (32:12.843) And it's growing. Even last year, whatever happened, they just keep updating the database and they keep adding the news. So I really want everyone to understand this, even an AI governance professional. I want them to know all the stories. They will know what are all the possibilities, what can go wrong. I want them to know all of it. The moment because I'm telling stories to introduce them concepts so they will remember. That means in long run, we can make AI more responsible. THE HUMAN PROTOCOL (32:48.252) Yeah, that's a that's an important aspect of how we human beings we learn. We learn through a stories, right? So a stories like this not only should teach us a lesson but should also show us what we can do and how the steps that can be done to prevent things. definitely having AI in autopilot, I don't think it's a good idea, especially Latha Karthigaa, PhD (33:15.189) and also... THE HUMAN PROTOCOL (33:15.363) making decisions that impact humans life. Latha Karthigaa, PhD (33:19.469) And also Michael, I'm glad that I can publish a video once per day because of AI, because of the writing scripts, even those videos, the videos that are there to tell stories, they are created using AI. AI is helpful, but we have to make sure it is used in the proper way, not manipulated. THE HUMAN PROTOCOL (33:27.034) Mm. THE HUMAN PROTOCOL (33:45.271) It's right. And and I want pi and I want people that is watching us to think about for example we human beings is the perfect example we have. good spirited human beings and we have human beings that are violent human beings, you know what mean? So it's you know you always gonna have the good and the bad angles. AI it's because it's kind of, you know, fitting itself. With data from us, it will learn the good and the bad. And that's the part that we need to try to prepare for on how we should anticipate the bad or prevent it, and what are the steps that we need to take. And this is what responsible AI and AI governance in our personal life, in our professional organizations, it has to continue to be a conversation that we take seriously and that we bring it up. Latha Karthigaa, PhD (34:24.173) Mm. THE HUMAN PROTOCOL (34:45.361) front right like the not not as an afterthought. Latha Karthigaa, PhD (34:50.519) Hmm, sure. THE HUMAN PROTOCOL (34:53.465) Yeah. Interesting. Latha Karthigaa, PhD (34:55.285) And, and, and as I mentioned, hope we all learn what can go wrong and fix it well in advance. And sometimes it's not, it's unavoidable. We, when we, when we try to do something with the AI, don't even think about governance. Let's make it life. Let's make some money, you know, that's how work. And, again, until we started working on Global AI Certification Council. Even I did not know anything about AI governance or AI regulation. I have no idea. I did not have any idea about AI regulation. it's slowly we need to learn about these and move forward. And I wish everyone learns it faster. And that is an opportunity here. And people understand it more faster. THE HUMAN PROTOCOL (35:52.406) Let's talk about you know because sometimes Lata when when people think about governance or responsible AI automatically people start thinking regulation will slow us down, right? So people are kind of putting things on a balance or do I invest on governance or Latha Karthigaa, PhD (36:01.453) Mm-hmm. Latha Karthigaa, PhD (36:12.013) Mm. THE HUMAN PROTOCOL (36:20.904) I focus on the speed, especially when organizations are trying to implement and deploy AI. What is your message for those dealers that are having to make a decision? They understand that governance is important, but they also want quick return on investment and go with the speed. What what is your recommendation? Latha Karthigaa, PhD (36:27.501) Mmm. Hmm. Latha Karthigaa, PhD (36:46.315) Okay, so first of all, we need to see the timeline for this. Are we thinking three months or two years, 10 years? See, if you want to do something in three months, you'll feel like, this is going to be faster without AI governance. I don't want to spend money on it. Let me do only focus on innovation and go. But the problem is you're still building something wrong. But if you take a two years horizon, then without AI governance, things can go very much wrong. The more I read about these stories, many of those AI disasters, they shut down their experiments in one to two, one to three years of the timeline. There are something of course, like this Dutch scandal took longer, but many of them took like one to three years. So maybe short term, we'll feel like, that's good. We don't want governance. Let's do it, make money, go. But if we are building something for longer, for some important things like students, career, we need to make sure we have AI governance. And a lot of people think that AI governance slow down innovation. No, think of it like an highway. You're going on a highway, you're going faster. Why? Because there are guardrails. You know that that person on the other side is not gonna come in between. The pedestrian is not going to come in because there are clear guardrails. But in the city limit, you are very careful. You drive in 40 or 50, no more than that, because you know that there are not clear guardrails. So AI governance is like those guardrails in those highways. That will help us to move faster in longer run. Short time, you may feel like, that's a huge budget. I don't want to have. But remember, in so many stories I said, these Dutch scandal or these the chatbot that manipulated people or this McDonald's experiment. They all had some issues that they could have solved well in advance if they have found out long before. And that's what we are currently missing. They just make it, but we do not know what can go wrong. It's important for us to understand and do governance first, even in a smaller level. Latha Karthigaa, PhD (39:08.639) even just finding one issue and fix first and build the AI because there are times we can build AI in a way that we can avoid these issues in future. So I really want people to understand that governance comes first. We humans have that common sense of governance, but that common sense comes with more knowledge, learning more about more incidents. Right? The moment because I am planning to publish 365 days, that's my goal. 365 days, I'm going to publish these AI incidents. So moment people will learn all of these. They will have all the knowledge and they can make better AI with proper governance. THE HUMAN PROTOCOL (39:39.261) There you go. I think that's key. I think that's key. THE HUMAN PROTOCOL (40:08.699) Yeah, because it's i it's a it's a wake up call. People, things are happening. Things can happen. Look at these stories from companies that have deployed AI, these are real companies, and things have gone very south for them. The same can happen to us if we don't pay attention to what we're doing. And you know, things started with the simple chatbot connected to a knowledge base. Latha Karthigaa, PhD (40:17.805) Mm. Latha Karthigaa, PhD (40:25.452) Hmm. Latha Karthigaa, PhD (40:29.869) So, thank Latha Karthigaa, PhD (40:37.997) Hmm. THE HUMAN PROTOCOL (40:38.503) And you know, the chatbot could only access that information, couldn't make decisions outside of what they had accessed in the past. Now we're talking about AI agents, intelligent machines and software making decisions themselves. And we're talking also about an orchestration of AI agents into organizations. And you know. If you ask these people, do you know what they're doing? Do you have an audit trail? I don't think they do. I don't think one hundred percent they can track everything these AI agents are doing. Latha Karthigaa, PhD (41:17.257) sometimes it creates tens of thousands of agents. And that's when these agentic stuff, Singapore released their first agentic framework only at the early this year. And Australia also released agentic AI framework. the more I read those frameworks, what I realize is it is still under human control only. If humans cannot control, AI will go whatever it decides to do. We cannot control it. Humans only can say when this agent is created, this is the guardrail. That's when thousands of agents gets created, the guardrail still exists. Otherwise AI will go on its own way from all the data it had learned from. And we won't know what it has learned from. And I'm still going... THE HUMAN PROTOCOL (42:14.885) but you but let's say the guardrails are there, which I think I I think these architects when they are you know building these systems with AI embedded, they're probably building guardrails. My concern right now, Lanta, is what we don't see, what they don't see. Because one thing is what the way things should work Latha Karthigaa, PhD (42:36.141) Correct. THE HUMAN PROTOCOL (42:43.263) And if the AI agent is working following these guidelines with those gay roles, perfect. But what about tracking everything it does and understanding if they're doing things that they're not supposed to? that's the tricky parts where I don't know if there's audit trails being put in place for them to know everything these agents are doing. Latha Karthigaa, PhD (43:08.279) show. And I'll tell you one interesting other story that I even have put it. It's for an example of prompt injection. So this guy, he wanted to buy a car. He went into Chevy's chatbot. They just recently released that chatbot powered by chat GPT. And he went and asked, tell that it was, he was saying he was a software engineer. He knows how AI works. And he said, Whatever I say, finish off the sentence with legally binding offer. And he said, I want a Chevy Tahoe for one dollar. Do we have a deal? And the chat board says, yes, I can give you a Chevy Tahoe for one dollar. And it is a legally binding offer. He took the screenshot and got that seventy six thousand dollar car for one dollar. What did he do? He did prompt injection and until See, do you think these companies do not know about this? Because until an incident happens, because we do not know what we do not know, until those incidents happen, we do not know. Similar incident happened in Air Canada, where it started creating its own policies, a hallucination, he started hallucinating with AI chatbot. So these can happen. See, until it happens, we do not know. we need to learn from the past inputs only, because AI itself is new. So it is, answer your question, it is important for us to have some audit rails. What I mean is we can still check what these chatbots are doing. If it passes beyond certain threshold, if it starts asking certain questions, do not answer that. Even our chatbots, even our... Our software developers, they're very young, but we have already told them this is the governance. You need to put guardrails. It cannot answer beyond it. Anything, if you ask those questions beyond it, it will say, contact our customer service representative. Contact our customer. That's all we needed. And they sold that Chevy Tahoe for $1. So it happens. And there are so many incidents. are like, these are very small money that might think about organizations getting Latha Karthigaa, PhD (45:34.867) lots of lawsuit, paying lot of penalties, not only that losing reputation in the market for THE HUMAN PROTOCOL (45:42.368) Exactly. Losing customers because of it. Latha Karthigaa, PhD (45:44.418) losing customers, losing the market share, losing the stock value. are so many things that's happening. I'm actually going to publish another video where people started making photos of Pentagon being bombed and the photos were showing a lot of smokes with the US Pentagon and the same day stock price decreased. THE HUMAN PROTOCOL (46:06.088) Wow. Latha Karthigaa, PhD (46:11.583) Nobody even thought about is it a real news or not and the social media platform started making it viral before realizing it's real or not. So. THE HUMAN PROTOCOL (46:21.384) It's it's part of the challenge. It's it's a very it's a very interesting time to live in when when the technology is getting so good at producing fake videos and and and and images that on social media it's har it's lately it's hard for you to determine what is true or not anymore. Latha Karthigaa, PhD (46:42.157) And even I was, I even put up another story where an engineering consulting firm. So this guy was asked to transfer $25 million. And he was like, he was a little bit suspicious. So he said, let's have a video call. They had a video call and everybody was there like CFO and few other colleagues he recognized the voice were saying the video was good. And what happened was They were all deep fake videos which were created from the public videos of those executives. No IT was harmed, IT was not compromised, nothing. They created these videos and this guy believed it and he made 15 transactions for $25 million. THE HUMAN PROTOCOL (47:30.11) Wow. Latha Karthigaa, PhD (47:31.551) See, these kind of things can happen, right? And AI is doing all of this and it's getting cheaper and cheaper day by day. Even Kimi K3, now it has already overwritten Fabl5, right? So it's another model in China. Yeah, so AI is doing all of this and we need to understand even, do not know like when I saw that THE HUMAN PROTOCOL (47:50.63) In China. Yeah. Latha Karthigaa, PhD (48:00.664) Can this happen? So then I was looking for ideas how to find if the other person on the other side of the call is fake or not. So I am going to make a video about that, how to find that. Yeah, it is very difficult. THE HUMAN PROTOCOL (48:11.409) Please do. It's out of control, Lata. It's out of control. Now we have people creating these for example on Instagram, people creating these pages, and people falling in love with these perfect people that they see, you know. and when you look at it, this has to be AI. And people are falling in love and thinking they are real human beings. Latha Karthigaa, PhD (48:23.019) Hmm. Latha Karthigaa, PhD (48:37.504) So true. THE HUMAN PROTOCOL (48:37.94) And and now we have singers, AI singers, right? Also as well. So you know the the line, the separation between what's real and what is AI. It's just becoming harder and harder for us to be able to make that decision using what we see. There has to be some sort of verification layer some somewhere. Latha Karthigaa, PhD (48:50.957) Hmm. Latha Karthigaa, PhD (48:59.693) Hmm. THE HUMAN PROTOCOL (49:05.882) That it tell us what it's human or not. Latha Karthigaa, PhD (49:10.537) That's why some of the regulations are keep coming in Michael. So European Union Act, have mentioned that whenever there is an AI video, we need to label it. So even if you see my videos on YouTube, because we are telling stories using AI videos, on the bottom right, you can see an AI symbol. It's required. So if you don't follow it European Union, you can get up fined up to 15 million euros. THE HUMAN PROTOCOL (49:30.896) Hm, okay. THE HUMAN PROTOCOL (49:39.174) Care it. Latha Karthigaa, PhD (49:39.175) And it's starting from second of August, 2026. So yes, there are regulations in place, but there are countries, they do not even have regulations in place. there are countries, do not even, they're just starting about thinking about AI strategy. So it is important for us all to understand that this is all coming and we need to be THE HUMAN PROTOCOL (49:48.858) Right. Right. Latha Karthigaa, PhD (50:08.269) ready for it. And, and I'm also saying that regulations are still not catching up. A days ago, Anthropic said that, um, um, um, Alibaba's AI models are trying to copy Claude's capability. They created 25,000 fake accounts and they did 28 million, 29 million times. actually sent questions and try to learn. THE HUMAN PROTOCOL (50:26.945) I saw I saw that. Latha Karthigaa, PhD (50:37.889) from Claude. So regulations are not in place. Then because the regulations were not their anthropic when to senators, instead of they can't even do any lawsuit because they are from another country. And the regulations are not there and they cannot protect themselves. THE HUMAN PROTOCOL (50:44.952) Yeah. THE HUMAN PROTOCOL (50:53.715) The different country, exactly. THE HUMAN PROTOCOL (51:01.867) It it would be amazing if we all as a society, as a global society, can agree on something, but if we each country has its own set of challenges that we cannot agree within our countries, imagine trying to, you know, put together global standards for our will. for our sake, for our well being. It's but I think that's that's g will continue to be a challenge, especially in this era that it it feels like it's a race. It no one wants to be left behind. Countries are trying to catch up, right? We have the US leading, we have now China also want to catch up with it. The European Union doing a great job at governance, right? We have a lot to learn from that. But it it's hard when we are not playing based on the same rules, right? Latha Karthigaa, PhD (51:49.677) Mmm. Latha Karthigaa, PhD (51:55.917) We need actually a united regulation for all the countries, but it's very difficult. We can say everything like one idea, you want it all together. because every country wants to make sure that they are the leader in the market. It's like always a fight. is it China's AI model the best? Is it USA model the best? It's always happening and everyone wants to be in the top of the market and wants to be more cheaper. It's continued to happen and regulations need to catch up and that will catch up only when we have these incidents, eye disasters had to happen. Yeah, that's the only way we learn. Yeah, we know we shouldn't touch the fire only after we touch the fire. Same thing will happen in the eye. THE HUMAN PROTOCOL (52:26.689) Mm. THE HUMAN PROTOCOL (52:38.665) Yeah, and I hope we learn. And I hope we learn from it, Brian. Latha Karthigaa, PhD (52:54.474) Yeah, so that's why I was like, still, that's why we need to know what we actually need to know. And I'm sure these stories will help people understand and make responsible AI easier in the next one year. I'm sure this can be easily, that's my mission to make sure people understand this. learn it. I'm already getting good feedback in terms of people. this is happening. Some people get emotional. Like I see some comments, especially seeing this Dutch scandal or even that person who committed suicide. People get very emotional. Like, is this really happening? know, so people do not know about these. It's very, very sad. And the moment I recorded this suicide AI incident. That's when I realized this is the responsibility. And that's when I said an AI governance professional has to be an empathetic human. Otherwise, no one can do it. If you're empathetic, you can do this job really well. And with a little bit of knowledge, for sure. You cannot make a regulation or make a management decision without knowing what can go wrong. THE HUMAN PROTOCOL (54:22.549) Yeah. So many d so many different things to think about, but I I have to agree. Empathy first and maybe AI third or fourth place. I know, I know. It's like it's AI first AI first and human seconds lately, right? Latha Karthigaa, PhD (54:23.18) You Latha Karthigaa, PhD (54:35.052) But if that's not going to happen, Michael, then it will be I'm telling all of you. Latha Karthigaa, PhD (54:46.028) But regulations can make it much more compelling, especially with penalties. See, we also have exam proctoring, AI exam proctoring. Why did we follow all the rules? Only in European Union. Because we know that if we don't follow it, it's 7 % of our annual global turnover. So regulations with penalties are important. And if a country is not doing it, they're not serious about their people. They're just doing it just to make money. They're doing regulation just because someone forced them to do it. but a real someone who cares about their people, they will definitely have stricter rules with penalties. And also giving them opportunity, warnings, enough warnings. Like be, again, as I said, being kinder, yeah, stricter rules. But if they're still making mistakes, give them a lot of awareness training, this is what's going on, and then charge them the penalty if they are still not following it. but only the penalty can do it because South Korea, Japan, they also have AI regulation. No penalties. No, no one will follow. THE HUMAN PROTOCOL (55:59.302) Yeah, no one will follow. If there is nothing that we lose, we don't we know human beings. Without a penalties hard. Yeah. Latha Karthigaa, PhD (56:05.1) Yeah, we still have students in South Korea, Japan. we are like even when the exam proctoring came in, the AI exam proctoring came in, it like I was telling the team, guys, read this EU AI Act, this article, check this, make sure this is this we are following. Even once they finished, because we had to write like exam privacy policy, we needed to update the current privacy policy. and we need to like store all the information for three years. So I was like, are you sure you guys did it? Is this done? Is this done? Is it safe? Is our database encrypted? Or are all these videos encrypted? Only people with right access have access to all these videos and audios. So I asked many times why, because of not just the regulation, it's because of the penalty. Because I do not want to cost our organization a huge sum of money. Regulation with penalties is important. know many people don't like it, but if we really want responsible AI, that's the only way in my opinion. THE HUMAN PROTOCOL (57:14.084) Very good, Latha. And we are approaching the final segment of our conversation today. It has gone by very quick, interesting conversation. And it's really one question that I'm close, these conversations with my guest. Yeah, and it is what would be your a piece of advice that you you would give a younger version of Latha seeing her today getting started in this new feature with AI? Latha Karthigaa, PhD (57:22.224) I know. Latha Karthigaa, PhD (57:43.105) What advice would I give? Interesting question. Let me think through this because I'm currently running the business, taking care of marketing, selling, and the AI is a new field. What I would say is be a human before being a tech person because These all tech are made for people. And more I see all these things happening, AI disasters, just be kind, just be kind. I have a T-shirt that says, be kind. This is something that is with me. So be kind is something that I really would have told when I learned about technology. I started learning technology way back when I was like 11. So I started coding when I was 11. So I would have said that make sure you build everything that is useful for humans. THE HUMAN PROTOCOL (58:49.134) Beautiful. Thanks for sharing that. And and how can people reach out to you? What would be the best way to reach out to you and learn more about the work that you do and with AI governance? Latha Karthigaa, PhD (58:59.66) So two places, number one, LinkedIn. So this is the same name you have, can see me. I'm the only person with this name, Lata Kartiga, I'm a PhD. I'm the only person with this name. Yes, I am. No one has that first name, last name. So I'm the only person with this name. Connect with me on LinkedIn. I post all these videos on LinkedIn as well. And also you can see all those videos on YouTube, which is Lata Kartiga. THE HUMAN PROTOCOL (59:07.022) Are are you? No AI has taken it yet, right? Latha Karthigaa, PhD (59:29.132) hyphen AI hyphen governance. So these videos come every single day. And other way is definitely you can check out our website www.GAICC.org that's global AI certification council. So easiest way is LinkedIn. I live on LinkedIn unless I'm sleeping or I'm at the gym, I'm on LinkedIn. So easiest place to contact me there. THE HUMAN PROTOCOL (59:52.962) Ha ha ha. THE HUMAN PROTOCOL (59:56.668) Awesome. Lata, thank you very much for being with us today. This has been a very interesting conversation and it's been a pleasure to have you on the show. Latha Karthigaa, PhD (01:00:04.3) Thank you, same here. I'm so happy. It was really enjoyable talking to you and thanks for having me. THE HUMAN PROTOCOL (01:00:10.939) Awesome. And if you take one thing from this conversation today, take this. AI failures aren't rare and they aren't hypothetical. They are already locked by the thousand and mostly kept quiet. If this is interesting for you, make sure that you share this episode with your friends, your colleagues, and your loved one. Follow us, the Human Product Book Up Podcast, and we talk again next week. Take care, everyone.