Trying to wring the bias out of AI algorithms — and why facial recognition software isn’t there yet

Toju Duke spent nearly a decade at Google as a manager of their Responsible AI program, the company’s effort to try to ensure that research teams weren’t unwittingly writing algorithms that were biased.

The effort is harder than it sounds. Nearly five years after the National Institute of Standards and Technology (NIST) found that facial recognition software was lacking when it came to identifying minorities, Duke says the problem persists.

Duke is now the founder of the nonprofit Diverse AI, which is trying to level set the world’s AI algorithms to make them more inclusive.

The interview below has been edited for length and clarity.

CLICK HERE: Tell me about what you did at Google and what we really mean when we talk about responsible AI.

TOJU DUKE: I worked with the Google research teams on different research programs to make sure we have responsible AI practices included in them.

I worked with the LaMDA team a couple of years ago. So we wanted to make sure that before we released LaMDA, there were some responsible AI benchmarks that were run through the large language model to measure its accuracies and performance and bias on different terms from gender-related benchmarks or those relating to professions. So they were checking for terms like “Is it referring to a nurse as a he or a her?” Or “How many times does it refer to a woman and a man or a non-binary person in the outputs that it makes?”

CH: In 2019, NIST released a rather infamous report about facial recognition software bias and people are still talking about the problem. Has nothing changed in the almost five years since that report was released?

TD: What I'll say is there's been an increase in awareness of the issue. If there's been any changes, it's not more than 5%, and it's not measurable. Let me give you an example. It's not related to facial recognition per se, but just to the issues with AI technologies.

Amazon had a hiring tool in 2018, which exhibited some form of gender bias against women. So if a woman and a man put their resume through the hiring tool, the woman's resumes were always dropped and the men's resumes were favored.

And why was that? Because the tool was trained on male college data. So the AI algorithm that was working through the tool was only recognizing male colleges and male names. So anytime it saw something that did not identify with the training data that it was trained on, it dropped it.

Fast forward five years later: in 2023, the Lionesses, a female football team in the UK, made it to the World Cup finals. Someone asked Alexa, ‘Alexa, tell me who's playing the World Cup finals today.’ And Alexa said, ‘There is no match.’ In other words, Alexa does not even recognize female football as football.

This is five years later. This problem exists across all the technologies in AI. And they're still there. We're still talking about biases today.

CH: It seems crazy that this algorithm bias problem hasn’t been resolved… Some algorithms are better than others, but why hasn’t this bias problem been fixed?

TD: I think it starts from the teams that are building it. They're literally 90% white men. I don't think it's any malicious intent to exclude people from these data sets. It's just a natural bias towards people who are within your group who look like you, sound like you and are within your social network and your social influence. The very first problem is the data sets are being built and no one is thinking of the inclusivity and diverse nature that needs to be included in the data sets.

The next problem is the cost. It costs a lot of money to make data sets very diverse. There's lots of work that needs to be done to include people from the Global South, for example. You would need to go to the Global South and take pictures of these people to represent the cultures that they live in.

And it's not just about the pictures. It's also about the representation of their cultures, their ideologies, and their way of living. So when you weigh the pros and cons for many organizations and many companies, they don't see the value when they think about how much profit they'll make out of it.

I know there was a Click Here episode on the LAION-5B data set scraping art from the internet. MIT also had a computer vision data set called Tiny Images, and they had to recall it because it was full of biases and it had very offensive labeling in it.

It was out there and had been used and cited by so many scholars for about five years before some researchers discovered that it had issues and they pulled it out. So even if we have these data sets, it takes a lot of time and money to build a data set from scratch.

CH: Is there a solution that wouldn’t require a complete overhaul of data?

TD: People are thinking about synthetic data sets and people are working on synthetic data sets, which is supposed to be a walk-around, but it’s still a bit cost heavy. It gives humans more control on the data.

So in other words, you could build a sample data set based on real life data, and then you reproduce more pictures of people from different cultures without having to take photos of them. That will save some money, and we can actually try to have a fair representation of everyone in society with synthetic data sets.

It's not being used a lot, but it's also one of the alternatives that can be used.

CH: Do you think synthetic data will be the answer?

TD: It's not a hard or fast rule. Synthetic data is one of the ways you can help. But I think it's more about coming up with rigorous processes for data ethics. I think there has to be a holistic approach towards data and taking it seriously, not just leaving it for Big Tech to handle. I know most of the technologies are coming from Big Tech, but we still have startups right now that are launching models almost every day.

So we can't keep on pointing our fingers at Big Tech, but there just has to be a responsible approach towards these applications, including facial recognition, before we deploy them.

CH: I’ve always wondered why facial recognition isn’t being used to address things like the missing persons problem in the U.S, which is what our episode this week is about. The Department of Justice says some 600,000 people are reported as missing each year and some 4,500 bodies go unclaimed in morgues. Isn’t this an obvious use of AI — to help families find their missing loved ones?

TD: I feel it's a matter of prioritization and importance. We can go to the recent deepfake issue that happened. Deepfake crimes against women and children have been on the rise over the past few years, but when Taylor Swift goes through this, all of a sudden you just hear a U.S. bill has been introduced to give victims of deepfakes an opportunity to sue people that have done this.

But this problem has been here for a very long time. I understand Taylor Swift is very important, and she's a human being as well, but do we have to wait until someone important and famous is a victim to horrible things like this before we sit up? To me, it just shows the magnitude of how we pay attention to things only when something big happens or it happens to someone very important.

No one should go missing. Let's not wait until something terrible happens. Let's just do it. Let's have someone who's really committed to it. But until the shoe fits, you don't know how it hurts. So sometimes it's probably people who have actually had someone in their family missing before they sit up and say, “We have to fix this problem.”

CH: Could you imagine a world in which facial recognition would help identify the missing and unidentified?

TD: It would be beautiful. It would be great if we could use facial recognition to identify the people who have been missing and reunite them with their families again. That's a beautiful thing about AI, right? It's a technology that has so much potential to solve our problems, to solve a lot of problems that the world faces. And I think that's why the governments now have decided to pick it up and they're all enthusiastic about it because they can see the potential it has.

That is so positive and could bring so much good back to society and even help towards economic empowerment and all the problems that a lot of governments face across the world.