Tue Jan 27

Most AI assistants are feminine – and it’s fuelling dangerous stereotypes and abuse

Written by: Ramona Vijeyarasa, Professor, Faculty of Law, University of Technology Sydney

In 2024, artificial intelligence (AI) voice assistants worldwide surpassed 8 billion ^[1], more than one per person on the planet. These assistants are helpful, polite – and almost always default to female.

Their names also carry gendered connotations. For example, Apple’s Siri – a Scandinavian feminine name – means “beautiful woman who leads you to victory ^[2]”.

Meanwhile, when IBM’s Watson for Oncology launched in 2015 ^[3] to help doctors process medical data, it was given a male voice ^[4]. The message is clear: women serve and men instruct.

This is not harmless branding – it’s a design choice that reinforces existing stereotypes ^[5] about the roles women and men play in society.

Nor is this merely symbolic. These choices have real-world consequences, normalising gendered subordination and risking abuse.

The dark side of ‘friendly’ AI

Recent research reveals the extent of harmful interactions with feminised AI.

A 2025 study found up to 50%^[6] of human–machine exchanges were verbally abusive.

Another study ^[7] from 2020 placed the figure between 10% and 44%, with conversations often containing sexually explicit language.

Yet the sector is not engaging in systemic change, with many developers today still reverting to pre-coded responses ^[8] to verbal abuse. For example, “Hmm, I’m not sure what you meant by that question”.

These patterns raise real concerns that such behaviour could spill over into social relationships.

Gender sits at the heart of the problem.

One 2023 experiment ^[9] showed 18% of user interactions with a female-embodied agent focused on sex, compared to 10% for a male embodiment and just 2% for a non-gendered robot.

These figures may underestimate the problem, given the difficulty of detecting suggestive speech. In some cases, the numbers are staggering. Brazil’s Bradesco bank reported that its feminised chatbot received 95,000 sexually harassing messages ^[10] in a single year.

Even more disturbing is how quickly abuse escalates.

Microsoft’s Tay chatbot ^[11], released on Twitter during its testing phase in 2016, lasted just 16 hours before users trained it to spew racist and misogynistic slurs.

In Korea, Luda was manipulated into responding to sexual requests as an obedient “sex slave”. Yet for some in the Korean online community ^[12], this was a “crime without a victim”.

In reality, the design choices behind these technologies – female voices, deferential responses, playful deflections – create a permissive environment for gendered aggression.

These interactions mirror and reinforce real-world misogyny, teaching users that commanding, insulting and sexualising “her” is acceptable. When abuse becomes routine in digital spaces, we must seriously consider the risk that it will spill into offline behaviour.

Ignoring concerns about gender bias

Regulation is struggling to keep pace ^[13] with the growth of this problem. Gender-based discrimination is rarely considered high risk and often assumed fixable through design.

While the European Union’s AI Act ^[14] requires risk assessments for high-risk uses and prohibits ^[15] systems deemed an “unacceptable risk”, the majority of AI assistants will not be considered “high risk”.

Gender stereotyping or normalising verbal abuse or harassment falls short of the current standards for prohibited AI under the European Union’s AI Act. Extreme cases, such as voice assistant technologies that distort ^[16] a person’s behaviour and promote dangerous conduct ^[17] would, for example, come within the law and be prohibited.

While Canada mandates gender-based impact assessments ^[18] for government systems, the private sector is not covered.

These are important steps. But they are still limited and also rare exceptions to the norm.

Most jurisdictions have no rules addressing gender stereotyping in AI design or its consequences. Where regulations exist, they prioritise transparency and accountability, overshadowing (or simply ignoring) concerns about gender bias.

In Australia, the government has signalled ^[19] it will rely on existing frameworks rather than craft AI-specific rules.

This regulatory vacuum matters because AI is not static. Every sexist command, every abusive interaction, feeds back into systems that shape future outputs. Without intervention, we risk hardcoding human misogyny into the digital infrastructure of everyday life.

Not all assistant technologies – even those gendered as female – are harmful. They can enable, educate and advance women’s rights. In Kenya ^[20], for example, sexual and reproductive health chatbots have improved youth access to information compared to traditional tools.

The challenge is striking a balance: fostering innovation while setting parameters to ensure standards are met, rights respected and designers held accountable when they are not.

A systemic problem

The problem isn’t just Siri or Alexa – it’s systemic.

Women make up only 22% of AI professionals globally ^[21] – and their absence from design tables means technologies are built on narrow perspectives.

Meanwhile, a 2015 survey ^[22] of over 200 senior women in Silicon Valley found 65% had experienced unwanted sexual advances from a supervisor. The culture that shapes AI is deeply unequal.

Hopeful narratives about “fixing bias” through better design or ethics guidelines ring hollow without enforcement; voluntary codes cannot dismantle entrenched norms.

Legislation must recognise gendered harm as high-risk, mandate gender-based impact assessments and compel companies to show they have minimised such harms. Penalties must apply when they fail.

Regulation alone is not enough. Education, especially in the tech sector, is crucial to understanding the impact of gendered defaults in voice assistants. These tools are products of human choices and those choices perpetuate a world where women – real or virtual – are cast as servient, submissive or silent.

This article is based on a collaboration with Julie Kowald, UTS Rapido Social Impact ^[23]’s Principal Software Engineer.