Whether of politicians, pop stars or teenage girls, sexualised deepfakes are on the rise. They hold a mirror to our sexist world

While digitally retouching real photos has been happening for years, fake images of women are on the rise. Shutterstock^[6]

Similar problems have emerged in AI facial recognition tools that have misidentified suspects in criminal investigations due to the racial and gender bias that is built into the software. The ghosts of sexism^[7] and racism, it seems, are literally in the machines.

Technology reflects back to us the disrespect, inequality, discrimination and – in the treatment of Purcell – overt sexism that we ourselves have already circulated.

Sexualised deepfakes

While anyone can be a victim of AI-facilitated image-based abuse, or sexualised deepfakes, it is no secret that there are gender inequalities in pornographic imagery found online.

Sensity AI (formerly Deeptrace Labs) has reported^[8] on online deepfake videos since December 2018 and consistently found that 90–95% of them are non-consensual pornography^[9]. About 90% of them are of women.

Young women, children and teens across the globe are also being subjected to the non-consensual creation and sharing of sexualised and nudified deepfake imagery. Recent reports of faked sexualised images of teenage girls have emerged from a New Jersey high school in the United States^[10] and another high school in Almendralejo, Spain^[11]. A third instance was reported in a London high school^[12], which contributed to a 14-year-old girl taking her own life.

Read more: Taylor Swift deepfakes: new technologies have long been weaponised against women. The solution involves us all^[13]

Women celebrities are also a popular target of sexualised deepfake imagery. Just last month, sexualised deepfakes of Taylor Swift^[14] were openly shared across a range of digital platforms and websites without consent.

While research data on broader victimisation and perpetration rates of this sort of image editing and distribution is sparse, our 2019 survey^[15] across the United Kingdom, Australia and New Zealand found 14.1% of respondents aged between 16 and 84 had experienced someone creating, distributing or threatening to distribute a digitally altered image representing them in a sexualised way.

Our research also shone light on the harms of this form of abuse. Victims reported experiencing psychological, social, physical, economic, and existential trauma, similar to harms identified by victims of other forms of sexual violence and image-based abuse.

This year, we have begun a new study^[16] to further explore the issue. We’ll look at current victimisation and perpetration rates, the consequences and harms of the non-consensual creation and sharing of sexualised deepfakes across the US, UK and Australia. We want to find out how we can improve responses, interventions and prevention.

How can we end AI-facilitated abuse?

The abuse of Swift in such a public forum has reignited a call for federal laws and platform regulations, moderation and community standards to prevent and block sexualised deepfakes from being shared.

Stunningly, while the non-consensual sharing of sexualised deepfakes is already a crime in most Australian states and territories, the laws relating to their creation^[17] are less consistent. And in the US, there is no national law^[18] criminalising sexualised deepfakes. Fewer than half of US states have one, and state laws are highly variable in how much they protect and support victims.

Sexualised deepfake images of Taylor Swift were circulated widely online. Jordan Strauss/AP

But focusing on individual states or countries is not sufficient to tackle this global problem. Sexualised deepfakes and AI-generated content are perpetrated internationally, highlighting the need for collective global action.

There is some hope we can learn to better detect AI-generated content through guidance in spotting fakes. But the reality is that technologies are constantly improving, so our abilities to differentiate the “real” from the digitally “faked” are increasingly limited.

Read more: Celebrity deepfakes are all over TikTok. Here's why they're becoming common – and how you can spot them^[19]

The advances in technology are compounded by the increasing availability of “nudify” or “remove clothing” apps on various platforms and app stores, which are commonly advertised online. Such apps further normalise the sexist treatment and objectification of women, with no regard for how the victims themselves may feel about it.

But it would be a mistake to blame technology alone for the harms, or the sexism, disrespect and abuse that flows from it. Technology is morally neutral. It can take neither credit nor blame.

Instead, there is a clear onus on technology developers, digital platforms and websites to be more proactive by building in safety by design^[20]. In other words, putting user safety and rights front and centre in the design and development of online products and services. Platforms, apps and websites also need to be made responsible for proactively preventing, disrupting and removing non-consensual content and technologies that can make such content.

Australia is leading the way with this through the Office of the eSafety Commissioner, including national laws^[21] that hold digital platforms to account.

But further global action and collaboration is needed if we truly want to address and prevent the harms of non-consensual sexualised deepfakes.

Read more https://theconversation.com/whether-of-politicians-pop-stars-or-teenage-girls-sexualised-deepfakes-are-on-the-rise-they-hold-a-mirror-to-our-sexist-world-222491