Are AI models as divided as we are?

  • December 12, 2024
Are AI models as divided as we are?

Tiancheng Hu and Yara Kyrychenko write about their new research findings on AI models and bias

As we continue developing these powerful tools, we have an opportunity - and responsibility - to ensure they don't amplify the tribal divisions that already challenge our society.

Tiancheng Hu and Yara Kyrychenko

Elections often reveal how deeply divided humanity can be. This year, as increasing polarisation continued to shape our world, we asked: Does this division transfer to our AI?

Our journey to answer this question began in 2022, when we started our PhDs as Gates Cambridge Scholars. Two concurrent events captured this moment in history: the invasion of Ukraine and the launch of ChatGPT. The political events around us highlighted a fundamental aspect of human nature – our tendency to split the world into “us” and “them” – while ChatGPT’s emergence raised new questions about whether these same tribal instincts might be embedded in our artificial systems.

Social psychologists have long studied this phenomenon through the lens of Social Identity Theory, which explains how even trivial group memberships can trigger powerful biases. When people identify with a group – be it a nation, political party, or even fans of different sport clubs – they tend to favour their own group (in-group solidarity) while viewing others more negatively (out-group hostility). This psychological mechanism underlies much of today’s polarisation.

But what about the AI systems we’re creating? Do they inherit these deeply human tendencies? “Large language models are becoming remarkably human-like in their capabilities,” explains Tiancheng. “Understanding whether they exhibit similar social biases – and potentially amplify them – is crucial as more and more people interact with them.”

In our paper, just published in Nature Computational Science, titled “Generative language models exhibit social identity biases”, we developed a method to measure these biases in large language models and tested over 70 different models.

Here’s what we found:

  • Most base models – those trained on internet data and not fine-tuned to interact with everyday users- exhibit social identity biases comparable to humans – they’re about 70% more likely to speak negatively about “them” and more positively about “we”.
  • Instruction-tuned models, designed to interact with humans (think chatbots like ChatGPT) and with safety guardrails, show much lower levels of bias.
  • When we analysed real conversations between humans and AI, we found something surprising: these chatbots do exhibit substantial bias but human users themselves displayed more bias than the models they interacted with.

To understand where these biases originate, we trained language models on polarised data – tweets from US Democrats and Republicans. The results were striking – these models became 3-5 times more biased. But here’s the hopeful part: when we carefully filtered out extremely negative or overly tribal language from the training data, we could significantly reduce these biases.

“This shows that while AI can inherit our social biases, it doesn’t have to,” notes Yara. “We have the power to shape these systems to be more unifying than divisive.”

As AI becomes increasingly woven into the fabric of our daily lives – helping us write, make decisions and even form opinions – its biases could either amplify or help heal our social divisions. As we continue developing these powerful tools, we have an opportunity – and responsibility – to ensure they don’t amplify the tribal divisions that already challenge our society.

*Tiancheng Hu [2022] is doing his PhD in Computation, Cognition and Language. Yara Kyrychenko [2022] is doing her PhD in Psychology.

**Picture credit: Marketcomlabo and Wikimedia commons.

Latest News

Arts impact: Gates Cambridge at 25

Gates Cambridge Scholars work in every corner of the cultural industries, contributing to new ideas in the world of art, to different ways of viewing the role of music in society and different ways of telling stories. Visual arts When it comes to the visual arts, for instance, several Scholars have played an important role […]

New US Scholars selected as part of 2025 cohort

Thirty-five of the most academically outstanding social leaders in the US have been selected to be part of the 2025 class of Gates Cambridge Scholars at the University of Cambridge, marking the Scholarship’s 25th anniversary. The US Scholars-elect, who will take up their awards this October as part of an anniversary cohort of 100 Scholars, are […]

How to get heard in an increasingly noisy world

How do you get your ideas across in an increasingly noisy and divided world? Three Gates Cambridge Scholars discuss their innovative solutions in the second episode of series two of our podcast, So, now what? Jakub Szamalek [2009], Ragnhild Freng Dale [2013] and Cansu Karabiyik [2016] discuss the different ways they have innovated, through video […]

Celebrating agricultural innovation in Africa

A foundation started by a Gates Cambridge Scholar is leading a Cambridge Festival event in March to celebrate agricultural innovation in Africa where a new project on food security will be launched. The Agri-Innovation & Impact Project (AGRIIP) will be launched at the Roots of Resilience event on 27th March. It is designed to empower […]