How is Artificial Intelligence (AI) influencing our lives? Especially people who do not belong to the white, binary, heterosexual population? How is it harming these people? And why? During a week-long CREA institute called Possible AIs, we have dived into why the AI is the way it is now. The information packed days intended to situate AI in a socio-economic-historical context, which helped us to think through some of these questions. As CMI! works with structurally excluded women and girls, it was relevant to learn about the most hyped topic/tool of the current times in relation to these groups and their rights.**
Before we start, the name AI or Artificial Intelligence is a marketing gimmick; also the images of a human brain that often come with it. In its true form, it is machine learning. Humans feed endless data to the machine so that it can start detecting patterns based on the data fed and make predictions or outputs based on that. All the predictions and outputs are determined by the data fed to it. It’s often called a small world. No reality exists for the AI outside the data it learned. So, if the data is biased and hateful, the AI will also be biased and hateful.
Historical Sneak-Peak of the Origin of Data Collection
The history of datafying people is a dark and complex tale. It begins with the pioneering work of Francis Galton, a British polymath who coined the term “eugenics” which is, according to Wikipedia, “a set of largely discredited beliefs and practices that aim to improve the genetic quality of a human population.” Galton’s work in the late 19th century marked the first use of data for eugenic purposes, aiming to “improve” the human race through selective breeding. His ideas laid the groundwork for the systematic collection and analysis of data to achieve this goal.
Galton’s legacy was carried forward by figures like Charles Davenport, who established the Eugenics Record Office in the early 20th century. Davenport’s work, in turn, has been replaced by modern data collection practices, which, while more sophisticated, still carry the same underlying biases and intentions.
It’s no wonder that one of the most insidious uses of algorithms today is in the detection and control of human right defenders, queer people and migrants. Governments and agencies use data and AI to track, surveil, and control the movement of these groups, often leading to discriminatory and violent practices. These algorithms are seeped into policies and further institutionalise the oppression, control and discrimination of such groups.
No reality exists for the AI outside the data it learned. So, if the data is biased and hateful, the AI will also be biased and hateful.
Whose work has been invisibilised in Tech?
Data and tech is predominantly a patriarchal world with its biases and values. Maybe it would have been a different place if women were not wiped out from the history of computation and programming. The early days of computing saw women as the first programmers. Ada Lovelace, often regarded as the first computer programmer, and the women of ENIAC (Electronic Numerical Integrator and Computer) played pivotal roles in the development of early computing systems. In 1969, the meticulous coding of Margaret Hamilton enabled Apollo to land on the moon. Their contributions have been systematically erased in the narrative of technology development and replaced by male figures, making tech predominantly a white male space. This historical context is crucial in understanding the biases that have been embedded in AI systems from their inception.
And no matter how much we try to change AI systems, we cannot. They function through the rule of conformity. In AI’s small worlds, patriarchal values are the criteria they will stick to and drown the voices of marginalised people as noise. Just like zoom.
Mining and Destruction, Dehumanisation and Disembodiment
AI technologies often lead to dehumanisation and disembodiment in various ways. This disembodiment extends to the invisibilisation of labour, where the focus is on the vision and output rather than the people who make it possible. AI narratives do not talk about people behind the rare earth mining or training the AI, how dangerous and ill-paid these jobs are. It does not share the displacement of populations and destruction of nature. It never talks about the cost it comes with.
AI is also dehumanising the users. For instance, Zoom filters that alter our appearances can make us feel disconnected from our physical selves, as they eliminate natural human expressions like breathing and singing. The dehumanisation of self is another critical issue, where individuals are reduced to stereotypes rather than being seen as unique persons. In the context of AI, people are often seen as data points rather than individuals with complex lives and experiences.
We Cannot Imagine Our Lives Without AI: The Horror Stories
The integration of AI into our daily lives has led to numerous horror stories. For example, the case of Amazon’s AI-driven hiring tool, which discriminated against women, highlights the harmful consequences of biased algorithms. Similarly, the tragic story of a suicide linked to AI-driven content moderation underscores the lack of accountability in these systems. The design of AI is often driven by rules of conformity and fawning, leading us to willingly give away our agency. The invisible power and mechanisms behind AI are so opaque that it’s challenging to understand how decisions are made, further exacerbating the problem.
Facial recognition technology is another area of concern. While it promises convenience and security, it often fails to recognize people of color accurately, leading to misidentifications and injustices. This technology, designed without considering the diversity of human appearances, perpetuates systemic biases and discriminations. The “black box” nature of AI, as described by Bruno Latour, refers to the opacity of these systems, making it difficult to understand how they make decisions and who is responsible for their outcomes.
And still we are so fascinated by it – we are so excited by the potentials. We cannot stop it. Because we are really the ‘users’ – a terms adopted by tech pioneers from drug users. And AI (also digital space) has drug characteristics. They are intensely addictive.
Do We Try to Fix a Machine That Is Broken by Design?
Given the pervasive issues with AI, the question arises: do we try to fix a machine that is broken by design? The answer lies in embracing a sense of paranoia—being vigilant and critical of the systems we interact with. Use it absolutely when we need it and be mindful of its explorations of labour and nature and its biases. Remember every time we use it, we also feed it more data. And no matter how much we try to change AI systems, we cannot. They function through the rule of conformity. In AI’s small worlds, patriarchal values are the criteria they will stick to and drown the voices of structurally excluded people as noise. Just like zoom.
We need to make our compromises transparent and center community values in the design and implementation of AI. Small Language Models (SLMs) offer a promising alternative, as they are more transparent and can be tailored to specific community needs, reducing the risk of dehumanisation and bias.
As we continue to integrate AI into our lives, it is essential to remain vigilant. By embracing a critical perspective and centering community values, maybe we can work towards a future where AI serves the diverse needs of all people, not just a privileged few.
*Cover image is from Pixabay
**This blog consists of the subjective understanding of the writer. Please consult the sources below for original information:
Crawford, Kate, and Vladan Joler. Anatomy of an AI System: The Amazon Echo as an Anatomical Map of Human Labor, Data and Planetary Resources. AI Now Institute and Share Lab, 7 Sept. 2018, https://anatomyof.ai.
Shah, Nishant, Fangyu Qing, and Longhan Wei. “Artificial Intelligence: Systems of Intentionality & Human-Centred Values: A Scouting Report 2023.” Digital Narratives Studio, 9 Nov. 2023, digitalnarratives.com.cuhk.edu.hk/articles/artificial-intelligence-systems-of-intentionality-human-centred-values
Chan, Anita Say. “Immigrant Excisions, ‘Race Suicide,’ and the Eugenic Information Market.” Predatory Data: Eugenics in Big Tech and Our Fight for an Independent Future. University of California Press, 2025. https://doi.org/10.1525/luminos.215.
