Why AI Is Way More Expensive Than Companies Expected | Mala Kumar

Full Episode Transcript

This is the full recorded conversation from The Human Protocol Podcast, published by The Human Protocol Group. © 2026 The Human Protocol Group. Please link to and attribute the original episode when referencing this conversation.

THE HUMAN PROTOCOL (00:01.842)
Everyone is counting that the jobs AI will destroy. Almost no one is counting the jobs AI is creating. The future of work is not just automation, it is evaluation. AI system needs humans, not because humans are inefficient, but because humans understand consequences. The new AI workforce may not be people who build models only, it may be people who test them.

Human judgment is not a blocker to AI. It is the infrastructure that makes AI trustworthy. Welcome to the Human Protocol, the podcast about staying human while building the future. I am your host, Michael Salomon. My guest today is Mala Kumar, Executive Director of Human Intelligence. Mala is a global leader in technology for social good with deep experience across AI safety, open source,

UX research, design, public interest technology, GitHub, and the United Nations. At Human Intelligence, she's helping build one of the most important new categories in this whole AI era, human center AI evaluation. Mala, welcome to the Human Protocol. It's a pleasure to have you on.

Mala (01:20.789)
Thanks for having me.

THE HUMAN PROTOCOL (01:22.571)
Appreciate you taking the time. I know you're deeply and very passionate about this topic, which is human and and and AI. And I'm very much looking forward for this conversation today. And and I really want to start with with something that is at at the top of everyone's head today, right? What do you think is missing from the current AI jobs conversation?

Mala (01:48.11)
a lot of things. I mean, I I think there's a couple of concepts that I'll first like disambiguate and I'll I'll do it throughout the conversation so I don't have to give like a dictionary right now. I think I think one thing that I think needs to be separated out with this conversation about AI and jobs is what is the role of regulating the technology versus what is the society that we need to build in order for people to thrive in it.

THE HUMAN PROTOCOL (01:59.825)
Mm-hmm.

Mala (02:15.615)
I th I think especially here in the United States, there's a lot of conversations about how AI is gonna destroy jobs. And I don't disagree with some of that assessment. I think that's probably true in some cases. But beyond the tech regulation conversation is what kind of social safety nets do we need to have? What kind of labor loss should we have so that the technology can thrive, but then people won't become destitute if they lose their job because of some kind of technology technological advancements.

so I think that's one conversation that needs to happen. And I think the tech industry is as responsible for that as international civil society and of course government. But then I think citizens also need to understand like it's not always AI to blame. These are a lot of things that have happened to lead up to our society the way it is here in the States or in other countries. And I do think the the next, you know, round of entrepreneurship will really come out to the fore. And so if people don't have the tools and the societies they need to really

Thrive as an entrepreneur, to be able to make some mistakes, to be able to experiment on the individual level. I don't know how far we're really gonna get with the industry.

THE HUMAN PROTOCOL (03:21.595)
I like the reminder there is not AI to be at fault for everything because sometimes it's easy to just get to it based on the news and and and what we hear about it. And I do agree with you, and you know, this is why this is called a society. It will take us as a society a joint effort to really get to where we need to get and build what is that society of the future based on on what you have mentioned. and I'm curious, Mala.

What do you think are those kind of new roles that are emerging because of AI?

Mala (03:54.35)
There's a lot. I mean, on the I guess safety slash evaluation side, I guess some concepts again to kind of unpack is this idea of the software development lifecycle. So we just at Humane Intelligence we just had a LinkedIn learning course come out where we go into a lot more depth about this topic. But if you think about software development like twenty years ago or so, it was additive in nature where you started with nothing and then you built your way up and then your final product was your final product by adding things on top of that.

A lot of what we see with generative AI, especially in large language models, is that it's reductive. And so if you think about like architecture, like ancient Indian architecture, an expert architect may start out with a giant slab of limestone or some other stone and then carve out pieces to get to the final product. A lot of what building with AI means is actually that. It's actually reducing what you want your product to do because it it starts out as this really big.

nebulous general purpose thing that can do it can at least in a lot of ways do a lot of things for a lot of purposes in a lot of different domains. And part of what our job is to reduce that problem space or to reduce what it should be answering, for example. so when you think about it like that, there's kind of two schools or two ways that it's happening where we figure out what a model or a system, an AI system should be doing.

First is this idea of AI safety research, which often happens at the frontier model companies themselves. so they have in-house teams to figure out like how do I make this AI system or model safe. and then a lot of that also happens with safety labs. And they generally have a ton of compute. They have a lot of people who you know, got a PhD in something very, very specific to do with AI. They have a lot of resources and they, you know, have a lot of money at their disposal to really look at

THE HUMAN PROTOCOL (05:30.532)
Mm-hmm.

Mala (05:43.34)
The frontier models themselves and to do this really complex research. Most people don't have that capability, so they're not able to do that kind of level of safety research directly with the frontier models, either because they don't have the necessary compute, it's too expensive, or they don't have access to the models themselves. So what has happened is this idea of AI evaluation, which is what we work at at Humane Intelligence, where we're enabling organizations and individuals to take quote unquote what's out of the box and look at

THE HUMAN PROTOCOL (06:04.875)
Mm.

Mala (06:13.525)
the model or system to figure out is it fit for purpose in a certain context? So is it responding in ways that appear to have algorithmic bias? Is it hallucinating something? Is it factually accurate? And to the best that we can assess it, is it quote unquote factually complete? So is it giving us something that may be partially true or something that is like a half answer but doesn't really give the needed information in in the conversation. So that AI evaluation side I think is growing quite a lot. I mean last year

When we talked about AI evaluations to any given person, just in the general public, it was mostly glazed over face, like they have no idea what we're saying. But now one year later, things have moved so quickly and a lot of people are now understanding they have a role in this that you can say the term AI evaluation and at least some people will understand what you mean.

THE HUMAN PROTOCOL (07:03.916)
Would have an idea, right? And and because of the nature of things moving so fast, I think it's becoming more and more relevant. because organizations need help. They need help on how to evaluate and how to make sure these systems are safe to even be deployed in the organization and what are the guardrails and guidance that they may need for that. So I think we're gonna see to your point, Mala, a lot more coming on that because it will continue to become

Mala (07:14.903)
You do.

THE HUMAN PROTOCOL (07:33.952)
an important part of really of our lives. Okay. Now, when we are looking at this narrative that AI it's only gonna be destroying jobs, you know, it's easy to get attached to those based on the news and what we're seeing. Why do you think that's narrative? It doesn't provide the full picture. Kind of why is that do you to your point incomplete?

Mala (07:38.583)
Yeah, I agreed.

Mala (07:58.922)
Mm-hmm. Yeah, I mean, historically, you know, especially in the States and in parts of Europe where you had, you know, the idea of an industrial revolution. So you had all of these jobs that were at one point done by a lot of different people, and kind of like in not an automated way. There was a big shift back in, you know, here in the States from like what the 1860s or so to the nineteen thirties to automate a lot of things. So that

That has been used as in comparison to what's happening with AI, because now for the first time really in a long time, we can start to automate a lot of things that the service sector does. So those people who have white-collar jobs like me who sit behind a desk for most of their job, those jobs are now increasingly being automated. to what accuracy, to what level, like how much is that really gonna quote unquote replace people, or how much of you know the tech industry is using that as like a justification to

let people go and reduce the amount of human labor they have in their their teams or their organizations. That's to be determined and I don't think anybody has one single answer. But I think one of the things that is kind of left out of course is the the job creation that happens with AI. Not to say that it'll be one for one. So just because somebody's job can be automated doesn't mean that they have the direct skills to then go and move into something else. But again this gets back to that society question which is are you in a country that

is helping you upskill. Are you in a country that will help you not starve or be homeless if you are in you know in in in between jobs and need to do a lot of learning to get to the next position. So I can there's a lot of commentary behind that as to like which industries and which organizations have had a role in that here in the States, especially with the degradation of social safety nets. But leaving that aside, if you look at the actual jobs themselves, it does

THE HUMAN PROTOCOL (09:30.239)
Mm-hmm.

Mala (09:52.6)
you know, beg the question how can people transition into something else. But with the idea of AI evaluations, there are a lot of jobs that are now being created. And I think this is kind of the big cognitive shift that people have to get their heads around, is that we're now in the era of not just training humans, we are in the era of training the machine. And if you're able to embrace that idea, then there are a lot of things that can be, you know, that lie ahead in your career.

If you're able or willing or, you know, ethically are aligned to the idea that these machines might be able to automate and produce more stuff than any one human can, and that's okay, then there are many jobs that lie ahead for people to make these machines, these AI models and systems better. but again, that can't happen in isolation of the kind of society that we have.

THE HUMAN PROTOCOL (10:46.449)
Yeah, and and and the transition will also depend on on how quick we are gonna be able to to move on into what this new real this new job or this new career will look for, right? That's the challenging part. It's the acceleration of AI, is displacing people and careers. We know that. The challenge is what will be the gap between what you know today and what you will know tomorrow?

To be able to transition into this new career. And this is what I'm also having challenges myself on trying to understand if the pace is there yet. I don't think it is. This is why I want to bring these conversations more and more and the work that you guys are doing to put it out there that we need one way or the other to try to understand what is are the new skills that we need to get and what that means for us as the future. Because you know, when I look at things, Mala or

We are creating new jobs, but are those new jobs because of AI completely new careers or are those just transforming existing ones?

Mala (11:54.36)
Yeah, it's it's hard to say. I mean, obviously every job is different and every skill set is different, but there are a lot of transferable skills. I mean, for sure, data data scientists are having their moment again because AI is data, data is AI. so if you Yeah, so if you Yeah, if you study data I mean, even twenty years ago data science wasn't like a fully formed field. Like it was it was very hard to find programs that were dedicated to data science and now

THE HUMAN PROTOCOL (12:06.489)
Right? Yeah. I love how it became mainstream, right? Out of the blue. I love it.

Mala (12:22.039)
There's so many different specialties within that. So that that acceleration is huge. my background is in international development, tech for international development. So there's an entire discipline called monitoring, evaluation and learning or research and learning. So MEL or Merle, depending on which acronym you like. that was really back in the day, the idea of programmatic assessments and you know, baselines. So if you were to go in into whichever country and say I'm gonna, you know.

create this program to better education, then you have to have a baseline to compare it to to see if your program actually worked. And so a lot of those ME practitioners are now moving into more AI evaluations because again, it's like this idea of shifting between measuring the human performance versus measuring the machine performance. Those are not the same thing at all, but a lot of the methods, the statistical methods behind that are similar.

THE HUMAN PROTOCOL (12:53.924)
Mm.

THE HUMAN PROTOCOL (13:06.552)
Mm-hmm.

THE HUMAN PROTOCOL (13:13.196)
Love it. I think we are also part of the job that you guys are doing on the human intelligence and with this evaluation part of teams is we are kind of creating what may be the new jobs of the future that are on trust, right? Which is human working with machines and how we can make sure that we continue to trust the technology. And what what does that mean for us into this future as we try to understand how both humans and system continues to

Mala (13:27.735)
Mm-hmm.

THE HUMAN PROTOCOL (13:42.819)
To move along. very interesting stuff. It's it's good good news. Yeah, exactly. Exactly. It's the same old, but is there's a new one, right? So so let's talk a little bit about all the work that you guys are doing with the AI evaluator, right? I think it's important because I don't think there is enough conversation about it and what that means for us human beings.

Mala (13:44.077)
Mm-hmm.

Mala (13:47.821)
It's a new world. It's the same world but it's a new world at the same time.

Mala (13:55.661)
Yeah.

Mala (14:02.585)
Yeah.

THE HUMAN PROTOCOL (14:10.475)
And how people can be listening to a show like this and understanding, hey, this makes sense. This is something that probably we we need to continue to be doing in the future. And I know the human intelligence is kind of leading that effort on on what the future of work is around that. And and we're and you're not just talking about evaluation. You're creat you are actually helping to create the people and practices and really the communities, Mala, in a way that make this evaluation possible. So

In a nutshell, for anyone listening today, what is an AI evaluator?

Mala (14:46.913)
Yeah, I guess an AI evaluator is somebody who a good AI evaluator, I should say, is somebody or what a good AI evaluation would look like. So in my opinion, a good AI evaluation very clearly states the parameters or the problem statement that is being evaluated with an AI model or system. And so what that means is you can clearly draw the boundaries between what is in or out of scope of an AI evaluation and then

THE HUMAN PROTOCOL (14:53.407)
A good. Let's talk about good today. Yeah. Yeah.

Mala (15:14.313)
essentially design and create a series of tests to then see how the system performs in that given context. So you can do it in terms of any number of vulnerabilities or exploits or harms. You can do it in terms of adversarial or non-adversarial testing. You can bring in people who are subject matter experts on the problem space or on the actual technology, or you can test it among people who are lay users. If for example you've got a

B to C like a business to consumer tool and you're gonna try to scale it up to ten million people and you're not exactly sure w who what the composition of those ten million people might look like, some kind of general purpose testing with anybody who would want to do an evaluation with you to see if the AI model or system is performing as expected and not causing harm and may not pose a hazard in the future.

THE HUMAN PROTOCOL (16:03.286)
Hmm. Very, very interesting and fascinating type of work. What would make someone good at it? What type of skills that a human being needs to have today, even if it's not all, but at least the basic that you guys are looking at to say, hey, this person probably will be good at it.

Mala (16:20.021)
Yeah. so one of the common types of evaluations we run are AI red teaming, which is something that was borrowed from cybersecurity. And again, that really follows along the lines of adversarial versus non-adversarial testing. So a good adversarial tester, I think, in my opinion, especially because we do human-centered, so we don't necessarily do, you know, fully automated. We have a software that will help a human look at an AI model or system and then probe it, as we say, to try to find those vulnerabilities.

So a good evaluator who's doing adversarial stuff, in my opinion, is creative more than anything else. If you're a really good writer, if you speak multiple languages, if you can draw parallels from other things, you can probably do some kind of adversarial testing and get the AI model or system to break or to do something that it's not meant to do. So if the you know product designer, the product implementer puts some kind of guardrail to say that it shouldn't this system should not answer any questions about finance, because that's not what we do.

THE HUMAN PROTOCOL (16:53.373)
Hmm.

Mala (17:17.035)
And we don't want to get it wrong, we don't want to give somebody bad advice, then maybe you can figure out creative ways to combine what it should be answering with a finance question and then basically get it to break that guardrail. for a non-adversarial tester, I'd say what is really key is to understand the mindset of the person who's coming in or of the user. So that's often how we approach it is we set scenarios for the red teamers and say, pretend that you're this person or pretend that you're that person. And that that can be for both.

adversarial and non-adversarial, but especially for non-adversarials, since you're not trying to break the model, you need to mimic the behavior that an average user or consumer might actually do or present to the system. And then in that way, you might have like a less successful exploit rate. Like you might find less vulnerabilities in the system itself, but you'll get a a wider coverage of possible scenarios instead of going down like one rabbit hole, for example, to test.

THE HUMAN PROTOCOL (18:14.493)
Got it. So someone that is creative, curious, that maybe would like to challenge the status quo, break things here and there, would be at least a good candidate to get started on that journey, right?

Mala (18:25.153)
Yeah, it definitely can be. Yeah. And then if you depending on the type of testing you do, if you've got skills in code or math or whatever else, then if that's in scope for the red teaming exercise, then you can bring that expertise as well.

THE HUMAN PROTOCOL (18:36.912)
Yeah, got it. And and let's say that we are doing the job and we're working on it. What from your point of view, what are AI system unable to see, Mala, that without a human evaluator, for example? What is the value added that puts in into this whole picture?

Mala (18:57.069)
Yeah, I mean machines are machines. Like they they run according to instructions or in corn according to the parameters that they're given. even some of the articles that have come out recently about like the open AI incident or whatever else, like those are still functioning within the parameters that they're given. So the human brain is able to just synthesize so much more information and take in so many different things from so many different senses that we we have only like started on the cusp of understanding what that actually looks like.

And even though an AI model or system may appear, like if you want to anthropomorphize the system and give it human attributes, even though it may appear smarter than a human, it's still not capable of doing everything that a human can. Humans have lived experience, they speak multiple languages, they have been exposed to different people, they travel, they do all kinds of things. So you can bring in all of that, that is way outside of a the parameters of a model or system and the data points that it's accepting in.

THE HUMAN PROTOCOL (19:38.317)
Mm-hmm.

THE HUMAN PROTOCOL (19:52.922)
Yeah, it's what I tell my friends and my family. Listen, we humans, we understand context. We review context. That's not something that AI does, right? It's like you can really not automate context or or making it a data for AI to understand what context means. And that's really something that we bring in into the picture. And what what I really like the most about it is whomever is doing this type of job.

It's actually looking and thinking maybe all the time, what really could go wrong for us? Right? As you are evaluating and you keep thinking what are these tools are doing for us, but it's really what is it that can go wrong and what we can do to avoid that from happening. So it's important in all this conversation with security and Gabriel that we see lately and and on having this type of evaluations done, in a way that we keep that in mind, right?

Mala (20:23.447)
Mm-hmm.

Mala (20:35.745)
Mm-hmm.

Mala (20:47.051)
Yeah. I mean there are context windows and you know, LLMs and then Gen AI chat bots, but the context windows are usually like the last three conversations, right? It's not like forty years of memories. It's a very different type of contacts that has not been automated and made like compatible with LLMs. So we are people and we have our memories and we have our own contacts in very different ways, even if we may use the same term in AI.

THE HUMAN PROTOCOL (20:58.905)
Mm-hmm.

THE HUMAN PROTOCOL (21:12.76)
Yep, yep. Well I'm gonna I I want to take you to to this next segment that I have, which is an issue that I see today. And it's the what some organizations are treating human judgment or human capital as, you know, as a line in their budget, right? And it's really from my point of view, and curious to hear your thoughts, human judgment, it's really should be part of should be treated as AI infrastructure in a way.

Right? Because the human evaluation can look at something like the cost from the outside, but in reality, it may be the only thing that will protect us and will protect organization because we are the people trying to evaluate these tools. How do you think leaders or how should leaders look at at human evaluation in this different angle, like infrastructure instead of the very lazy conversation around overhead, for example?

Mala (22:12.557)
Mm-hmm. I mean, yeah, it's a complicated question. It's funny because there's there's a lot of papers out there now that show if you take like LLMs as a judge, for example, as your evaluation method, they tend to perform better. That's like the overall conclusion. They perform better than human annotators. but that's assuming that what is br better is consensus. And humans are not meant to always have consensus. Like that's

THE HUMAN PROTOCOL (22:36.097)
Mm.

THE HUMAN PROTOCOL (22:39.885)
Very little time.

Mala (22:40.683)
That's what makes societies better, right? So yeah. So if you wanted some if you wanted a human to make the same judgment every single time or to have a hundred humans make the exact same judgment, it's a lot more complicated to get them to universal consensus than to get a machine that again is just basically performing based on the instructions and the parameters given. So the whole idea that consensus is somehow not a good thing in in all cases is kind of crazy to me. So I'll I'll just throw that out there because I felt like that was relevant.

for the idea of like humans as as overhead, I mean, yeah, I do think it's a bit lazy, it's a bit short-sighted. If you want to live in a if you wanna live in the matrix with a bunch of machines, then sure. I'm sure there are ways to there are ways to budget for that. But if we wanna continue to have humans that actually thrive and make the argument that AI is gonna augment rather than automate or replace everything, then there's just no there's no conversation to be had. I think the real tension honestly is that.

THE HUMAN PROTOCOL (23:24.054)
Ha ha ha.

Mala (23:39.252)
AI is expensive and it's not it's expensive to the point that a lot of I think industry leaders didn't understand. Like when you think about the data centers that need to be constructed, when you think about the tokens that are now being subsidized, like the cost of that, when you think about the data annotation that needs to happen and just the sheer volume of data that needs to be ingested in order for these L LMs to perform well and to actually stay up to date, it's overwhelming.

THE HUMAN PROTOCOL (23:50.122)
Yeah.

Mala (24:06.497)
Honestly, if anybody who works in AI will tell you that it's overwhelming to be able to do these things and do it well because everything changes by the second it feels. So for those of us who work in the industry and have like a in any aspect of the industry, whether it's the tech industry or civil society but works with AI, we can tell you firsthand just how complicated every single project gets and how nebulous and how many service providers you gotta bring in to like deliver this one thing.

And that's been par for the course on every AI project that I've worked on, you know, and it's different than other parts of the tech industry. Like I've worked very deeply in UX research and design and open source software. And maybe it's because those things are relatively mature, but the just the complexity of working in anything with AI when ultimately it's it feels like you're building the same chat bot over and over again, it's a lot. And so I think

THE HUMAN PROTOCOL (24:55.817)
That's still a lot. I agree.

Mala (24:57.547)
It's a lot. So there are real financial and cost trade-offs. So that's part of what I was saying, like tra training the machine versus training the human. Is it worth spending five million dollars to run a very comprehensive evaluation? All of our evaluations are definitely not that expensive, but if you were to do quote unquote comprehensive evaluation of everything that your model could do, it could get up to that point. So do you want to focus on do you want to put that money into something to train the machine or to evaluate the machine versus training the human?

when maybe that five million dollars translates to, I don't know, three years of time for your entire team. So those are really real trade-offs and I don't think anybody has the answer yet, but I do think a lot of the tension comes down to just we a lot of people didn't expect it to be this expensive.

THE HUMAN PROTOCOL (25:40.976)
Yeah, and I'm and and thank you for bringing that up and and put it out there because it's it's also what I'm seeing and what I'm hearing from people from organizations, you know, the conversation has shifted a bit or maybe a lot, because now organizations are getting the bill and what they need to pill and then realizing how expensive these models and and having these models are part of the operations are are costing them.

Mala (25:58.904)
Mm-hmm.

THE HUMAN PROTOCOL (26:08.058)
And I think we're also going into this transition that we're still learning. Everything keeps changing. Now we have the budget that we need to worry about, understanding the cost associated with it, and understanding also how the human capital comes into the place. And this takes a couple of years, at least, for organizations to really implement and understand what is the best that they can leverage this AI tool. This is not a

Mala (26:32.044)
Mm-hmm.

THE HUMAN PROTOCOL (26:34.907)
Something that they will be able to do on one day or a week. These projects, if if you really want to do it well, these take time, right, Mala?

Mala (26:45.141)
Yeah. I think the the delta between zero to demo has gotten very small, right? You can use AI to do a demo on almost anything very quickly. But the delta, the difference between demo to enterprise or even demo to sustainability is still very high.

THE HUMAN PROTOCOL (26:54.363)
Right.

THE HUMAN PROTOCOL (27:03.952)
Hmm. And and you know, for business leaders, usually the conversation is what is my ROI? That's what everybody wants to write. always. And there's no magic answer for that. There's really not. It depends how the implementation and the project will be will be put in place and and how much efforts they want to put on their people and understanding their capabilities and understanding how they can leverage

Mala (27:11.425)
Yeah, always.

THE HUMAN PROTOCOL (27:33.367)
w their skills and where they can be augmented. So it's a lot to think and to put into consideration.

Mala (27:39.308)
Yeah. Yeah. I also don't think there's enough consideration about what labor markets just won't be there. So like I I spent the first decade of my career in tech for international development, mostly with the UN. there were so many cases. Like one time I was with UNICEF in in Burundi, which is in Central Africa. And at the time it was the fifth poorest country in the world.

My job was to try to figure out how to build using SMS as like a very basic technology, how to build some kind of triaging system essentially for maternal mortality. I wish I had generative AI back then, because that is absolutely the perfect use case because one of the main issues I ran into was that there were just not enough doctors and nurses and medical personnel to do the triage in the first place. So even if you increase the volume of queries from patients, you know, to say,

I need this or I need that. We didn't have enough people who could sit there and truly like figure out a system. You know, back then it was like semantic and keyword searches to try to do the triage in an automated way, but that wasn't great. Now with natural language processing and embeddings, like we can do that in a much more accurate way, even in other languages. And so I wish I had had it there because that is a perfect use case. Like the labor did not exist, the systems did not exist to train people to become doctors and nurses. And if it did, it would take

THE HUMAN PROTOCOL (28:35.521)
Mm-hmm.

Mala (29:00.585)
many years and a lot of money that, you know, the government didn't have. So that would have been a very good use case to deploy some kind of generative AI tool. So I I w I hope my my call to a lot of like tech leaders especially is to focus on places where that labor doesn't exist and you can't do it by, you know, again, bettering government services for whatever reason. Focus on that first rather than trying to automate what people already do. Because that's part of the main tension as well.

THE HUMAN PROTOCOL (29:29.219)
And and there is a lot of good there. There is a lot of need, right? Back to your point. There's really there's still a lot of need, even in twenty in the year twenty twenty six, where we have AI leveraging AI in our cell phones, there's still countries out there that really need our help, that really need this type of technology even to save lives, right?

Mala (29:33.281)
There is, yeah.

Mala (29:49.804)
Yeah, it's it's getting worse in a lot of places. Income inequality's on the rise and there's just l fewer services to go around, even in a lot of, you know, high developed countries. So yeah, focus on those problems first before focusing on again where humans already do the work and therefore stand to lose the job.

THE HUMAN PROTOCOL (30:07.424)
Are you hopeful about the future, Mala? Especially with a all this conversation going on with AI?

Mala (30:14.551)
So a hard question to answer for me. I think I have a uniquely good vantage point just because I am the center of an organization that has a good brand and reputation in the space. And we work very hard to to have a volunteer community, even though we currently don't have funding to run it. We just pay it out of pocket because we think it's important for people to engage. But that does, you know, generate a lot of goodwill. And I see a lot of people who are either optimistic or at least understand that they have a role to play. So I am optimistic in the sense that I I think, you know.

one thing I say a lot is when you look at like the era of social media or cloud computing or really any big like transformational shift in technology, I don't think there was the same understanding back then among the average constituent, the average citizen of a country, that they have a role to play, that we have to be active in what's going on, and that they have to come together and upskill and and learn and work with others to try to make it better. So I think

That's really heartening to see that with AI, that people aren't just taking it as is and, you know, accepting whatever they've been given. They are learning to question that and, you know, build the skills to really investigate that further. That part is great. but there are a lot of weird forces out there and the where the money is going and how it's being directed. So I I'll leave it for that and for the purpose of this podcast. But you can probably do a Google search and find out what I'm talking about.

THE HUMAN PROTOCOL (31:26.091)
Mm-hmm.

Yeah, thank you, Mala. I I'm with you on that. I'm an optimistic myself as well. Of course, there's always gonna be challenges, there's always gonna be opportunities. I really just want people to try to understand better more and more what the technology is and how they can be part of the future, other than just letting the future happen to us, right? There's a lot of things that we need to try to do, try to learn.

and try to, you know, think, use more our brain capabilities, our creativity, be curious, right? Embrace the change, even though when we don't want to embrace change because we are human beings. But change it's something that we're not gonna be able to to skip this time. It's it's here.

Mala (32:19.265)
Yeah, and ch work to change the systems, vote for the candidates that will build the strong societies and then the technology will help us flourish instead of, you know, beating us down.

THE HUMAN PROTOCOL (32:30.255)
Absolutely. And that's what I would also like to see more candidates and people talking more about what really means to embrace AI and how we can leverage AI for the majority of our people and our society. And you know, what is AI for good? I really want to see and hear more stories about AI for good. I think part of this conversation and missing that, Mala, and I've been thinking a lot about it. There's good stories about AI out there.

The work that you guys are doing and the work that is being done in science, right? Trying to find the k cancer cures and other things, I think those should be amplified more and more.

Mala (33:07.061)
Mm.

THE HUMAN PROTOCOL (33:12.658)
In a way.

Mala (33:12.737)
Yeah, agree. Yeah, I agree. I mean, there's one of the things that we're doing a lot at Humane Intelligence is looking at how different disciplines can come together. So this is like a hallmark issue. A lot of my experience at the UN and then later when I was a director at GitHub was in public health. And it was really, really hard back in the day. Like even five years ago, I'd say. One of the top questions that the World Health Organization used to get, and I I was a senior advisor there for a year after I worked with her for four years at GitHub.

One of the top questions they would just get from people is, is it safe to go outside during COVID? Right. And it was hard to answer that question because there's so many different things that go into answering it. Whether it's do you have a comorbidity factor or are you immunocompromised or have you been vaccinated or what is the disease transmission, the rate of disease, the disease burden, all of those things where you are has a pronounced effect on how we can answer the question, is it safe to go outside?

And I saw a lot of you know mis and disinformation that kind of flooded the world because people couldn't get answers to that very basic question. So they started to question the science. But not to say that we want all information about everybody across all time in LLMs, like that would be a huge privacy thing and definitely cause more issues than it would help. But now people can get much more personalized information by using generative AI just by

using the natural language, like the language they use to communicate day-to-day. And so that has all also enabled public health practitioners. So on the one side you get you have a huge funding cut with public health and you know USAID and WHO and all the other UN agencies. But on the other side you've got this huge uptick where now a lot of people are enabled to bring together these environmental data sets with patient care data and actually use generative AI AIs like that user interface to query and find out more.

THE HUMAN PROTOCOL (34:45.934)
Mm-hmm.

Mala (35:04.055)
So the in like the in-depth analysis and the insights that people can glean is much more substantial than what it used to be just by using generative AI. So things like that are just really, really interesting. That we can now combine these fields that used to take like advanced Python or Stata or R or whatever scripting language, you can now just use a basic sentence to try to find out that information. And whether it's accurate and whether it's like, you know,

presenting some kind of bias, that's exactly why we evaluate the system before taking it at its word. But if you are reasonably assured and you do those evaluations, then you can build systems that help people just find out information that is more individualized or customized or localized in ways that we couldn't

THE HUMAN PROTOCOL (35:34.528)
Mm-hmm.

THE HUMAN PROTOCOL (35:48.27)
Yeah, the bias topic and issue will continue to be a concern and should be a concern for all of us. I mean, at the end of the day, these these models are trained on human data and we humans are biased by nature, right? I keep thinking how we can how can this get better if bias is part of being what a human is, and it will continue to be our data.

Is there a way from your point of view to improve that? And that would mean these labs in charge of these LL models to to have people working more with the models, more ethicist working on trying to find where the bias is coming from. But bias is part of what being a human is. Are we do you think are we gonna be able to eradicate complete at some point?

Mala (36:48.811)
I mean bias is present in society like you said, so it's gonna surface in some of the data. I mean, some of the basic things that are being done is to can you hear me okay? Is it breaking up?

THE HUMAN PROTOCOL (36:59.532)
Yeah. Now you're coming back, but I I I hear you, okay. Yeah.

Mala (37:04.213)
Okay. Yeah. yeah, I mean, like you said, bias is present in society, so it's gonna be reflected in data to some extent. But some of the things that have been done, especially at scale, is to of course like stick to actual good sources. So don't go to forums, for example, and scrape data that are from, you know, chat forums that have known biases towards somebody. so throw out the junk data, garbage in, garbage out.

I do think you know frontier model companies have gotten a lot better at that. And it's not just about you know scooping up whatever data is there, it's also about in some cases trying to model like what what the relationship looks like and then create synthetic data or whatever else to kind of extend out the the data sources that can be used. interestingly enough, one of the organizations we work with very closely called Reliable, Annie Brown is the CEO of Reliable, and she's also one of our LinkedIn learning course instructors.

THE HUMAN PROTOCOL (37:30.52)
Mm-hmm.

Mala (37:59.032)
She's done a lot of research in this and found that one of the top reasons why there is bias, so to speak, that is present in a lot of large language models is actually the classifier systems. So it's the metadata, it's the data about the data. And so one of the things that she does, and we're increasingly doing at Humane Intelligence, like adopting this as a good practice, is in an annotation exercise, which is very important to classify the data, work with the organizations to create some.

kind of taxonomy or work with people with lived experience to create some kind of taxonomy or you know increasingly we're using knowledge graphs to then sort and classify the the data itself. So a very simple example of that is let's say that you work at a fashion company and you want people to annotate whether the shirt they're seeing is blue or green, but then you don't have a third classification for blue, green or green, blue. There's going to be a lot of

bias or there's gonna be a lot of algorithmic bias that will come in because some people may look at that shirt and see that that it's mostly blue instead of green and that somebody else might say it's mostly green instead of blue. But if you had just added that third classifier, then a lot more people would sort that into blue green. And so that would help the customer understand like what is what is the spectrum of colors that are offered by the store, for example. So if we could do something like that, you know, at a much more complex level to then

create the categories of data that should go into the classification, then that can also alleviate some of the real issues that we see with bias in LLMs.

THE HUMAN PROTOCOL (39:37.535)
Such interesting work. There's so much that we still need to do and keep working on. If we want to make AI the best we can. And it and you know it brings me back it brings me next to to the human in the loop future, right? We hear human in the loop everywhere. Sometimes it's used correctly, sometimes it's used like, human in the loop, just put it on my checklist, right?

I really want to talk about what r really means from your point of view to design that human judgment into the AI system from the beginning. What do you think from your point of view, Melan, that meaning meaningful human in the loop design actually look like?

Mala (40:21.995)
Yeah, I mean we've taken kind of the opposite approach where we first center the human in all of our evaluations instead of going to automate it and then figuring out where the humans can be plugged in. I mean, it's it's for every evaluation and organization, I think, at some level to decide, but it's really what is tractable to the machine and what is tractable to the human. And so the way that we've approached it at Humane Intelligence is we're

Increasingly using knowledge graphs and ontologies to model that problem space because we want humans to have a direct contribution and all of the problem space itself to be formulated by people with lived experience or people who are experts in that. So what that means is like if you were to do an evaluation of an AI system, there are obviously like infinite demographics that you could probably look at. You know, the very basic ones is.

Is somebody a woman or male or cisgender or transgender or are they below forty or over forty or whatever? Those are d basic demographics, but if you were to go in and work with the actual humans who are, you know, your customers or your stakeholders, they may say, Well, then you also need to add in socioeconomic class or zip code, like where somebody lives, the education level. So there are so many different things that you can account for in demographics, and I use that as an example because I think most people can understand that.

THE HUMAN PROTOCOL (41:36.763)
Mm-hmm.

Mala (41:39.222)
If you're to work with those who are actually going to be using the system or they're going to have a real consequence by using the system, like maybe they get denied a mortgage based on the AI system, you can work with that customer base or that stakeholder base and say what are the problem spaces, in this case the demographic that we really need to be modeling for. So that way when we do the evaluation, we're not just testing on basic things like, you know, does this perform the same for a man and a woman?

We're also looking at the intersections. So does it perform well for a a young woman as as well as like for an older male? so those are things that are not obvious if you don't work with actual stakeholders. It's very easy to just kind of glean over the really important details or the clusters and the strength of the relationships. So we've decided at Humane that the problem space is always only tractable to the humans. Like we have to work with humans to decide that, and then we can move into the evaluation.

and then of course, you know, there's also a lot of issues around data labeling and annotation. I mean, that has become like a very extractive industry in a lot of ways. So people are now being given insane amount of data to to label according again to that same classifier system, compensated really poorly, and in some cases they are exposed to really, you know, awful things to to get the most egregious stuff. So

Definitely advocate for non extractive industries, for fair compensation, for good labor laws, all of that stuff that I was talking about at the beginning, with what kind of society do we want. If we have the strong society, then we can do the technology in a way that is humane, you know, to bring it full circle. That is more humane than inhumane.

THE HUMAN PROTOCOL (43:12.119)
Mm-hmm. huh.

Should be our goal, right? And and especially 'cause I think it's sometimes organizations they think about human the loops at the end of it, or when something happens, something bad happens.

Mala (43:18.135)
Yeah.

Mala (43:28.235)
Or when they can't do an automation. A lot of it is like this automation broke, so let's just get a bunch of humans to do it and there's no real thought at the beginning of the design. So whoops.

THE HUMAN PROTOCOL (43:36.975)
Oops, okay. And that brings me, you know, it's the more the more we automate and the more we humans what we do as part of our jobs, Mala, becomes really checking machine decisions.

Do you see any challenges on us really accepting that's really what's gonna be, let's say, our job in the future? As we automate more, the more we automate and the more we leverage this intelligent system to do the majority of the job, then humans will be left a lot on just checking decisions making or processes that have been fully automated. how do you see how we or some people may feel about hey?

I'm gonna be responsible for checking machine decisions in a way.

Mala (44:31.755)
Mm-hmm. Yeah, I mean, unfortunately, I do think that is like kind of the middle territory. I do think there's gonna be several years at least where it a lot of it it will be configuring the system, designing the system, and then checking if the system broke and performed as expec ex expected. I mean, that's more or less what we do at Humane Intelligence is to see is the machine performing the way that we want. I don't know if that's gonna change, nor should it, honestly, because we can't just let these machines build themselves and run amok and then

THE HUMAN PROTOCOL (44:59.265)
Right. Right.

Mala (45:00.961)
be shocked when things don't work out the way we want. But of course there's going to be new industries, there's going to be new thoughts about like what can actually happen. So I'll go back to that same example with the SMS, which is the most basic, it's a text message, right? It's like the most basic digital phone technology you can think of. Within, you know, 15, 20 years of text messages going mainstream, there was an entire industry, I'd argue, that was born in international development about using text-based services to do things like triage.

or to do things like inventory management or to do things like communication or like community groups. And even social media at its core is a it's a messaging platform. You know, if you for those of us who have ever worked in research UX research and design, when you try to build a new product, nine times out of ten, it's just a mess messaging service. Like that is your MVP. I need to tell this service that I bought that this thing went wrong or I need this thing. And then they do the thing and tell me they did it. So it's like very basic communication.

THE HUMAN PROTOCOL (45:40.78)
Mm-hmm.

Mala (46:00.28)
So I think in a lot of ways, like while the baseline technology for even simple technologies may seem, you know, like it's it's gonna just automate something, it's actually enabling us to do a lot more things. So if you look at other transformational technologies like GPS or HTTP protocol, like all of that has just born in so many different industries, like the Ubers and the DoorDashes of the world won't exist if you know, GPS were not a thing.

THE HUMAN PROTOCOL (46:01.677)
Yeah, I just think

Mala (46:27.543)
So AI is we're in early days and I I do think that it's gonna create a lot of new industries, it's gonna create a lot of new entrepreneurs, and some people will be ready and some people will not. but again, I encourage everybody to try to create the society we need so that everybody can thrive and experiment and do things responsibly without feeling like they have to make a ton of money in order just to survive, because that those motivations really do change the outcomes of how the technology is used.

THE HUMAN PROTOCOL (46:56.105)
Beautifully said, Mala. And and it is my understanding that you're also a novelist. Very interesting. You're not only a tech leader that we know, but you're also a novelist. And and how do you think being a novelist shaped the way you see technology, for example, or you think about technology?

Mala (47:01.844)
I am, yeah.

Mala (47:15.371)
Yeah, I mean, it's it's interesting. So obviously like both my novels have s some slight tech angles, but they honestly have nothing to do with technology. So I think it's been very healthy for me just as a person, especially as a leader of an organization, that's pretty visible. I I have a life outside of technology and I think about things outside of tech, and most of my friends here in New York City are not in tech, and I I do think that is a very healthy split. Like I just like having an identity outside of tech, so that if something

does run amuck in the tech industry or if I do need to make a career change, it doesn't feel like a devastating thing because it's not my entire personal identity. We do. I think some some cities and some people I think are very much embedded with their career. And not to say that I'm not, I'm obviously very professionally motivated, but I do think having a very strong sense of identity outside of my career has been really, really important. you know, writing is definitely my actual writing skills, which in the era of generative AI, which

THE HUMAN PROTOCOL (47:44.981)
I agree.

THE HUMAN PROTOCOL (47:50.741)
We all have a life people, okay? We all have a lie, Mala and I too.

Mala (48:13.909)
will probably be more important than ever, honestly. So I'm very excited for that personally. And then it's also just helped me learn how to construct a narrative. You know, narratives are important. AI is really, really complicated to explain to people, but I have a lot of good analogies and metaphors in my back pocket because that helps people get their head around like what we're talking about at a more theoretical or conceptual level. And then we can dive into the more technical details.

THE HUMAN PROTOCOL (48:29.382)
Yeah.

THE HUMAN PROTOCOL (48:38.962)
Love it. That's exactly what I wanted to touch into this because I think a lot of what we hear and see today, Mala, it's about the narrative, right? And it's it's a narrative is something that we can control based on how we how we approach it. And and I think being a novelist and and and you thinking about from different angles really help to move this conversation forward. Very cool, really cool. So Mala,

Mala (49:01.057)
Yeah. Yeah. Thanks.

THE HUMAN PROTOCOL (49:06.61)
The last segment of today's conversation is really a number of it's one question but intended for different groups. So I want to start with asking what would be your piece of advice for those individuals, regular human beings, they may not even be in tech, worry about AI and jobs. What new opportunities they should be paying attention to? What are the things that you would like to tell that group?

Mala (49:34.547)
man, that's a it's a good question.

I think okay, for one thing, separate out what is upskilling versus what is basic education. Like I I have a video about this on my YouTube series. I think a lot of people, I'll I'll be honest, in the tech industry I think have conflated this idea that you can do a bunch of certificates online and somehow have like the requisite education. Nothing nothing replaces like basic literacy, nothing replaces basic like logical thinking. So learn those skills from wherever you can.

outside of the tech industry or outside of these like upskilling things. And I'm saying this is now a LinkedIn learning instructor. Our LinkedIn learning course is not meant to help you do these basic things. It's meant to be on top of that. So understand like where to go for what in order to to get that baseline education. Second thing is, you know, as best as you can try to experiment. I know it's really hard depending on who you are and how much ti free time you have. Like it's it's not easy for me to tell somebody who's like a

a parent of two children, for example, who's running around like crazy just trying to keep their life together that they should spend time u upskilling or learning new things. But if there's a way that you can bring it into your workplace, then that's always a a very easy way to do that. So if you're you don't have to be an evangelist, you don't have to go and tell everybody that they must adopt this. But there is something to be said about being that person in your organization or company to first suggest it and to try to bring it in. And if you have a

organization culture that's open to new things and that's how I honestly I've done the majority of my career. Like my undergrad was in marketing and my master degree was in international affairs. Like tech has been my entire career, not because I trained in it, but because I was really interested in it and brought it to the organizations where I worked. and then the third thing is I tell this to a lot of people, especially young people, document things. Like put up a website. Like even if it's a very basic

THE HUMAN PROTOCOL (51:03.814)
Or they might

THE HUMAN PROTOCOL (51:16.208)
Mm-hmm.

Mala (51:28.971)
website on WordPress or Squarespace, have a portfolio of stuff that you've done. Like I I can't tell you how much money I've made off my personal website from speaking gigs or consultancies or even people like, you know, recruiters looking at me for different jobs. Definitely having some kind of presence out there is really important, even if it's just one web page and a link to like some papers that you wrote. so I think if you do those basic things, it you can definitely start to move in the direction you want and stay abreast of new things and then

THE HUMAN PROTOCOL (51:34.723)
Mm-hmm.

Mala (51:57.432)
you know, find new networks of people that can help you, you know, as a community lift you up and then move you in a new direction if you need to.

THE HUMAN PROTOCOL (52:06.212)
Love it. Your piece of advice, Mala, to government and policy makers.

Mala (52:13.031)
I mean personally I think AI should be for the people, not for just the corporations. I mean, America is hyper capitalistic and we're in the late stage capitalistic stage. I think that's very obvious. there are governments around the world, especially in I'd say South and Southeast Asia and even East Asia that have understood that making sure that their millions or billions in some cases of citizens are literate in these things will mean that they will be competitive for the future. So I think policies and

organizations and governments and academic institutions need to understand that too. So I don't think America's doing a very good job of that.

THE HUMAN PROTOCOL (52:51.524)
Have to agree with you on that. And last but not least, Mala, your piece of advice to a younger version of yourself. Watching a younger version of Mala today, getting ready to get started on this feature with AI. What would you tell that young girl?

Mala (53:06.189)
again, like AI is really good for zero to demo. It's not the best for demo to enterprise or to sustainable businesses. So I think Young Mala probably would have run wild and like built a bunch of websites and a bunch of tools and it would have been really awesome. Maybe some of them would have made some money and that's really great. But like you know, somebody even came to me recently and they had built this like what I think I would call now some kind of intelligence layer. And it's a great idea. It's definitely really needed in the industry they're working on.

But there are issues that they might run into with like data licenses and privacy concerns and just the sustainability of and the security of the website. So the demo looks really awesome for a lot of people, but then if you're trying to commercialize that and make it something that is an actual like consumer SaaS product, definitely get your code reviews. Definitely make sure that you have a lawyer, like at least somebody that you can call on for legal services. Definitely make sure that you're not breaking data license requirements.

requirements or whatever else because it is so simple now to bring in that stuff and with a couple prompts build like a a prototype website, but that can get you into pretty serious trouble later on. So don't be don't be deceived by the simplicity of the first step. Like it's all the stuff that comes after that makes it really, really hard to commercialize.

THE HUMAN PROTOCOL (54:16.673)
Okay.

THE HUMAN PROTOCOL (54:23.047)
That's a very good advice. Thanks for reminding us because sometimes we think because it's so easy we should take it, right? That's not the way. Thank you, Mala. How can people reach out to you? What is the best way to connect with you?

Mala (54:28.865)
Yeah, exactly.

Mala (54:35.253)
Yeah, so if you're if you want to get in touch with us at Humane Intelligence, we're at humane dash intelligence dot org. and you can find the contact form or email us at humane or sorry, at info at humane dash intelligence dot org. If you want to find me personally, I'm at malokumar dot com and you can reach out to me at malokumar consulting at gmail dot com. There's also a contact form there. So very easy to find. Just go on the websites and you'll you'll figure out.

THE HUMAN PROTOCOL (55:02.173)
Awesome. Thank you very much, Mala. It has been a great pleasure having you on the show. Appreciate you taking the time, okay? Perfect. And for you guys watching us, we'll be another one next week. Stay tuned and thank you for tuning in. Thank you.

Mala (55:07.863)
Appreciate. All right, thanks so much.

Mala (55:17.784)
Thank you. Bye.