Are AI models as divided as we are?

  • December 12, 2024
Are AI models as divided as we are?

Tiancheng Hu and Yara Kyrychenko write about their new research findings on AI models and bias

As we continue developing these powerful tools, we have an opportunity - and responsibility - to ensure they don't amplify the tribal divisions that already challenge our society.

Tiancheng Hu and Yara Kyrychenko

Elections often reveal how deeply divided humanity can be. This year, as increasing polarisation continued to shape our world, we asked: Does this division transfer to our AI?

Our journey to answer this question began in 2022, when we started our PhDs as Gates Cambridge Scholars. Two concurrent events captured this moment in history: the invasion of Ukraine and the launch of ChatGPT. The political events around us highlighted a fundamental aspect of human nature – our tendency to split the world into “us” and “them” – while ChatGPT’s emergence raised new questions about whether these same tribal instincts might be embedded in our artificial systems.

Social psychologists have long studied this phenomenon through the lens of Social Identity Theory, which explains how even trivial group memberships can trigger powerful biases. When people identify with a group – be it a nation, political party, or even fans of different sport clubs – they tend to favour their own group (in-group solidarity) while viewing others more negatively (out-group hostility). This psychological mechanism underlies much of today’s polarisation.

But what about the AI systems we’re creating? Do they inherit these deeply human tendencies? “Large language models are becoming remarkably human-like in their capabilities,” explains Tiancheng. “Understanding whether they exhibit similar social biases – and potentially amplify them – is crucial as more and more people interact with them.”

In our paper, just published in Nature Computational Science, titled “Generative language models exhibit social identity biases”, we developed a method to measure these biases in large language models and tested over 70 different models.

Here’s what we found:

  • Most base models – those trained on internet data and not fine-tuned to interact with everyday users- exhibit social identity biases comparable to humans – they’re about 70% more likely to speak negatively about “them” and more positively about “we”.
  • Instruction-tuned models, designed to interact with humans (think chatbots like ChatGPT) and with safety guardrails, show much lower levels of bias.
  • When we analysed real conversations between humans and AI, we found something surprising: these chatbots do exhibit substantial bias but human users themselves displayed more bias than the models they interacted with.

To understand where these biases originate, we trained language models on polarised data – tweets from US Democrats and Republicans. The results were striking – these models became 3-5 times more biased. But here’s the hopeful part: when we carefully filtered out extremely negative or overly tribal language from the training data, we could significantly reduce these biases.

“This shows that while AI can inherit our social biases, it doesn’t have to,” notes Yara. “We have the power to shape these systems to be more unifying than divisive.”

As AI becomes increasingly woven into the fabric of our daily lives – helping us write, make decisions and even form opinions – its biases could either amplify or help heal our social divisions. As we continue developing these powerful tools, we have an opportunity – and responsibility – to ensure they don’t amplify the tribal divisions that already challenge our society.

*Tiancheng Hu [2022] is doing his PhD in Computation, Cognition and Language. Yara Kyrychenko [2022] is doing her PhD in Psychology.

**Picture credit: Marketcomlabo and Wikimedia commons.

Latest News

Understanding how the human brain learns

Ata Elbizanti [2024] is interested in understanding how learning affects brain activity, particularly in areas responsible for processing visual information and those involved in decision-making. Her aim is to improve treatments for cognitive deficits and enhance our overall understanding of the brain and how we perceive the world. Ata’s PhD in Physiology, Development and Neuroscience […]

Why small presses are vital for local knowledge production in Africa

The winner of the 2024 Nobel Prize for Literature should draw our attention to the often-forgotten power of marginal publishing outlets in the Global South. As The Guardian put it, Han Kang’s Nobel win was ‘a triumph not only for Korean literature but also a reminder of the huge reach and influence of small press […]

First podcast in anniversary season focuses on youth

Three Gates Cambridge Scholars debate how to make the world a better place for young people in the first episode of the 25th anniversary edition of the Gates Cambridge podcast, So, now what? – out now. Kevin Beckford, Blanca Piera Pi-Sunyer and Emma Soneson discuss everything from the stereotyping of young people to how to […]

Environmental impact: Gates Cambridge at 25

The environmental catastrophe facing the planet is the biggest global challenge to humanity of our, or any, age. With governments lagging on action, there is a sense of impotence and gloom that permeates many discussions. Talk to any climate change researcher and they acknowledge that reports outlining worrying statistics seem to be turning the general […]