Have you ever looked at the name of someone you haven’t met, and wonder whether you should refer to them as “she”, “he”, or “they”? Now Standard Chartered (where I used to work) has an AI for that:
In 2023, Standard Chartered deepened its commitment to social sustainability and appointed Natalie Marko Nietsch as its inaugural Global Head of Social Sustainability. In partnership with her team, the Sustainability Data team has piloted an innovative and multifaceted approach to enable the bank to disaggregate its SME portfolio by sex on a current and continuing basis, with over 90% accuracy, without the collection of private personal data. The team developed the Standard Chartered SocialAl model, a hybrid approach to identify whether enterprise owners are male or female using the first name, country-adapted, and public and internal datasets.
This is … kind of ridiculous?
First, it’s not the cutting-edge AI that can code up an app from a back-of-the-napkin sketch. This is more like your parents’ good ol’ classifier. Remember Jian Yang’s “Hot Dog or Not” app in Silicon Valley? Yep, something like that.
Second, if Standard Chartered wants to know the gender of their clients, one solution might be—gee, I don’t know—just ask them? But apparently, that’s too much red tape and legal risks for a big global bank. And where do they find the employee time! So their clever solution is: let’s ask AI to make it up.
Sure, the model is 90% accurate, presumably when they tested it against the small portion of their clients for whom they do know the gender. But how accurate is the model for the rest? 🤷 What do they tell regulators when asked about how they got the data? I guess they can point to their “innovative and multifaceted” blackbox and say “Haaave you met …”
Lastly, how will knowing the gender of their clients really help Standard Chartered develop better products for women entrepreneurs? We all know there is a gender bias in access to financing, and Standard Chartered have already committed to provide more financing to women entrepreneurs. To understand the problem and figure out how to solve it, it’s probably more effective to just talk to the relationship managers who work with clients every day (and presumably know whether their clients are male or female). I bet engineers at Apple didn’t need to know the gender of their customers in order to develop the ovulation detection feature for Apple Watch.
Most barriers women entrepreneurs face in accessing finance are social and systemic: unconscious biases, rights to own assets, inequitable caretaking burden, exclusion from networks. Guessing the gender of the bank’s clients doesn’t help frontline employees address any of these challenges. It only gives senior executives a false sense of awareness and control.
An Inconvenient Thought 