“Are we are willing to accept the fact that what will decide the work of our child’s future is an algorithm that discriminates because gender is not represented in the rooms in which it is built?”
This was the closing question of the TED talk by Giulia Baccarin, biomedical engineer and entrepreneur, dubbed “queen of algorithms” for having giving birth to innovative systems useful for the predictive maintenance of industrial machines.
A woman with success in IT who highlights a risk: that of an Artificial Intelligence that can generate machines in the future ready to make independent decisions but raised on a diet and language (and data) full of stereotypes and prejudices.
Is there a concrete risk of exclusion?
“I don’t know to what extent the problem is algorithms or lack and/or scarce biodiversity of data,“ comments Marco Caressa. “Just as Baccarin’s experimental thesis was actually more useful to grandfathers than grandmothers because her tester panel was only male, so algorithms reason on the data they have. If you set up a recruiter algorithm to find candidates for roles that require certain skills and backgrounds, the problem is not that they ignore this or that social group, but the fact that the elements of that social group that have on average those skills and that background are fewer. So the data on which the algorithm works must be unbiased and representative of all possibilities, but if for a professional position I instruct the algorithm to find me candidates among STEM graduates, and then the proportion is 80 men and 20 women, the problem unfortunately is not the algorithm, but the fact that we have a strong inequality between men and women for that kind of background.
That is, it is a social, cultural and political problem. Why are girls who then become women not encouraged or intrigued by STEM study programs? What are we doing to encourage or intrigue them? When I studied engineering, the boy/girl ratio was 100 to 1 (there was no one in nuclear engineering!). Today it will have improved a bit but we are far from the 50-50 that would be desirable and if we had 50 and 50 the algorithm would select otherwise. We can always introduce a quota for women configuration parameter in the algorithm but the social problem would still remain. So I really appreciated Baccarin’s TED speech, but I was somewhat puzzled when she carried out the mobile phone experiment in the room asking to look for a CEO. That I should be amazed that there are few women brings me back to the cultural problem and the paths mentioned earlier, but I should not be surprised that there are few disabled people, because statistically it is a small population and will obviously be poorly represented in any profession.
Again regarding data, I think the real problem is their protection. If we hypothetically conduct a job interview in chat, without being able to see or hear the candidate and thus know whether it concerns a man or a woman (a sort of test di Turing in short) it is likely that we would eliminate some prejudices. The fact is that the more they know about me, the more they can use it against me. Baccarin gives the nice example of the female friend who had depression and who perhaps would not have been able to make a career if her problem had been foreseen months earlier. Imagine what it would mean for HR to have the candidate’s genome data available to conclude that it would be better not to hire so-and-so who has a 70% chance of getting cancer within 18 months.
We have to make a cultural leap because the management of our companies is often flawed and scarce. I remember a TV service about a young researcher with a PhD in biotechnology, a single mother. In Italy she was unable to find anything in either a company or a university. She wrote to a professor in France saying she had seen him at a conference and dreamed of working for him. Two days later she received an email from the professor inviting her to Paris, where she was hired in a research center. In that context there was a fondness for single mothers because they have a resilience and a fierce motivation that others do not have; for this reason they put her in a position to work at her best with a well-equipped nursery in the research center. Philanthropy? No. Managerial ability”.
The fault of algorithms or of data?
“I don’t believe there are algorithms that explicitly contain ifs for discriminating on the grounds of gender, ethnicity, religion, etc.” says Paolo Caressa.
“The algorithms we are talking about are those of Machine Learning, which are basically all optimization algorithms – starting from a certain set of data that is submitted to them – that try to infer information that is not evident or is not explicitly present in the data, and to do so use an iterative, typically non-linear, method that randomly combines available data many times.
In this sense, Machine Learning algorithms embody the a priori synthetic judgments of the great philosopher Immanuel Kant: they contain an a priori, that is pre-established, universal and deterministic part, which is then the algorithm itself, and a synthetic part, which is given by their application to data. The bias of an algorithm normally lies in the synthetic part, in data.
In fact, a Machine Learning algorithm without data or with little data is perfectly useless. Inversely, a bunch of data without an algorithm of this type to process them is unusable if not in a very elementary way and for extracting information that is self-evident.
Let’s take an example: there are programs that take input texts in human languages, for example in English, and are then able to generate new texts in the style and with the content of those languages. If the algorithm is built well, by giving the speeches of Joseph Goebbels as input to the program we will have sentences and speeches that are racist, sexist, violent and who knows what else as output. But the fault of this bias is clearly of the datum that is given as input to the program: the same identical program to which the books of Mahatma Gandhi are given as input would produce edifying discourses and maxims of wisdom.
Indeed, these programs can be used to measure the degree of discrimination of a corpus of documents.
On closer inspection, what happens with these algorithms is what happens with any entity able to learn unconditionally and uncritically: for example, if a child was to be educated from a very young age that men are superior to women, as happened systematically and explicitly in the past and as larvally continues to happen, the child will become an adult who, sincerely and convincingly, believes in male superiority. The same applies to racial, religious and any other kind of superiority: a discriminatory attitude and prejudices are learned in childhood and it is then difficult to eradicate them.
Therefore, to prevent an algorithm from discriminating against a woman or a child, there is no need to change the algorithm but rather ensure that the available data are on average less sexist and misogynistic. And the only way to do this is to change the behavior of men, not of machines”.
What if the future holds a very masculine model of power?
“In my opinion,” argues Grazia Cazzin – “the problem is not so much that the female gender is not represented in the rooms where the algorithm is built, but rather that the historical data with which each algorithm learns and calibrates its behavior narrate the working world and society we know and which already discriminates against the future possibilities of our female child.
Algorithms are able to intercept new trends and variations, so by giving sufficient space to them, the chances could increase. But what I would do with evolved models would rather be to develop genetic models able to show us where we will go with a very masculine model of power, with all the consequences on the part of social dynamics. And trying to upset the model itself with elements more similar to the female gender, to evaluate what benefits could be had.
The gamble, however, remains with the will to give way from time to time and I do not think that an algorithm can reach the complexity of our will”.
“In reality – concludes Stefano Epifani, Presidente del Digital Transformation Institute – “I think that Giulia Baccarin’s question would be more correct if asked in a different way: are we willing to accept the fact that deciding the work of our child’s future is an algorithm which is formulated in a way that is not shared with the society in which it will be used?.
In the original question it is taken for granted that an algorithm discriminates. It is not the algorithm that discriminates but the choice upstream of who writes that it which – as underlined by Baccarin – cannot be made in closed rooms the dynamics of which are not shared. These are choices that should be discussed and defined in the context of specific and attentive social policies.
I’m not sure the solution lies in a simplistic egalitarianism that cuts the differences in the name of a very imaginative and harmful we are all equal, which means that the system does not have to take into account differences (of gender and of others). Instead I’m convinced that these differences (between women and men, but also between young and old, between able-bodied and disabled, and – in general – between those in a position of greater strength and those who, on the other hand, have different needs that must be protected) should be taken into consideration, and with major attention, by the political system (which then must generate the information technology system). What is needed today, as usual, is awareness of part of a political class (which has as never before proved to be inadequate and unreliable today) that some choices can no longer be postponed. Because, as I said before, it is not the algorithm that discriminates or not, but the political logic of those who design it. And we must move from logic and policies oriented towards not discriminating towards logic and policies geared towards enhancing differences and making them a treasure for society. Because if this is not the real essence of Smart Cities and the Smart Society then we are all really wasting time. But to do so we would need politicians aware of this dynamic of change, and for this, of course, we still have a long way to go. The solution, unfortunately, will not be avoiding the problem, but seeing this problem solved in rooms the occupants of which we will probably not even know.
Algorithms help to understand, but to understand we need to ask the right questions and do so with the right goals.”