The Times Australia
The Times World News

.

Can human moderators ever really rein in harmful online content? New research says yes

  • Written by Marian-Andrei Rizoiu, Senior Lecturer in Behavioral Data Science, University of Technology Sydney
Can human moderators ever really rein in harmful online content? New research says yes

Social media platforms have become the “digital town squares” of our time, enabling communication and the exchange of ideas on a global scale. However, the unregulated nature of these platforms has allowed the proliferation of harmful content such as misinformation, disinformation and hate speech.

Regulating the online world has proven difficult, but one promising avenue is suggested by the European Union’s Digital Services Act, passed in November 2022. This legislation mandates “trusted flaggers” to identify certain kinds of problematic content to platforms, who must then remove it within 24 hours.

Will it work, given the fast pace and complex viral dynamics of social media environments? To find out, we modelled the effect of the new rule, in research[1] published in the Proceedings of the National Academy of Sciences.

Our results show this approach can indeed reduce the spread of harmful content. We also suggest some insights into how the rules can be implemented in the most effective way.

Understanding the spread of harmful content

We used a mathematical model of information spread to analyse how harmful content is disseminated through social networks.

In the model, each harmful post is treated as a “self-exciting point process[2]”. This means it draws more people into the discussion over time and generates further harmful posts, similar to a word-of-mouth process.

The intensity of a post’s self-propagation decreases over time. However, if left unchecked, its “offspring” can generate more offspring, leading to exponential growth.

A constellation of lights in a dark room, with a group of people silhouetted against the light.
Social media posts spread online through a process much like word of mouth. Robynne Hu / Unsplash[3]

The potential for harm reduction

In our study, we used two key measures to assess the effectiveness of the kind of moderation set out in the Digital Services Act: potential harm and content half-life.

A post’s potential harm represents the number of harmful offspring it generates. Content half-life denotes the amount of time required for half of all the post’s offspring to be generated.

We found moderation by the rules of the Digital Services Act can effectively reduce harm, even on platforms with short content half-lives, such as X (formerly known as Twitter). While faster moderation is always more effective, we found that moderating even after 24 hours could still reduce the number of harmful offspring by up to 50%.

The role of reaction time and harm reduction

The reaction time required for effective content moderation increases with both the content half-life and potential harm. To put it another way, for content that is longer-lived and generates large numbers of harmful offspring, intervening later can still prevent many harmful subsequent posts.

This suggests the approach of the Digital Services Act can effectively combat harmful content, even on fast-paced platforms like X.

We also found the amount of harm reduction increases for content with greater potential harm. While apparently counterintuitive, this indicates moderation is effective when it targets the offspring of offspring generation – that is, when it breaks the word-of-mouth cycle.

Making the most of moderation efforts

Prior research has shown tools based on artificial intelligence struggle to detect online harmful content. The authors of such content are aware of the detection tools, and adapt their language to avoid detection.

Read more: Can ideology-detecting algorithms catch online extremism before it takes hold?[4]

The Digital Services Act moderation approach relies on manual tagging of posts by “trusted flaggers”, who will have limited time and resources.

To make the most of their efforts, flaggers should focus their efforts on content with high potential harm for which our research shows that moderation is most effective. We estimate the potential harm of a post at its creation by extrapolating its expected number of offspring from previously observed discussions.

Implementing the Digital Services Act

Social media platforms already employ content moderation teams, and our research suggests the major platforms at least already have enough staff to enforce the Digital Services Act legislation. There are, however, questions about the cultural awareness of the existing staff as some of these teams are based in different countries to the majority of content posters they are moderating.

The success of the legislation will lie in appointing trusted flaggers with sufficient cultural and language knowledge, developing practical reporting tools for harmful content, and ensuring timely moderation.

Our study’s framework will provide policymakers with valuable guidance in drafting mechanisms for content moderation that prioritise efforts and reaction times effectively.

A healthier and safer digital public square

As social media platforms continue to shape public discourse, addressing the challenges posed by harmful content is crucial. Our research on the effectiveness of moderating harmful online content offers valuable insights for policymakers.

By understanding the dynamics of content spread, optimising moderation efforts, and implementing regulations like the Digital Services Act, we can strive for a healthier and safer digital public square where harmful content is mitigated, and constructive dialogue thrives.

Read more: The 'digital town square'? What does it mean when billionaires own the online spaces where we gather?[5]

Read more https://theconversation.com/can-human-moderators-ever-really-rein-in-harmful-online-content-new-research-says-yes-209882

Times Magazine

Building a Strong Online Presence with Katoomba Web Design

Katoomba web design is more than just creating a website that looks good—it’s about building an online presence that reflects your brand, engages your audience, and drives results. For local businesses in the Blue Mountains, a well-designed website a...

September Sunset Polo

International Polo Tour To Bridge Historic Sport, Life-Changing Philanthropy, and Breath-Taking Beauty On Saturday, September 6th, history will be made as the International Polo Tour (IPT), a sports leader headquartered here in South Florida...

5 Ways Microsoft Fabric Simplifies Your Data Analytics Workflow

In today's data-driven world, businesses are constantly seeking ways to streamline their data analytics processes. The sheer volume and complexity of data can be overwhelming, often leading to bottlenecks and inefficiencies. Enter the innovative da...

7 Questions to Ask Before You Sign IT Support Companies in Sydney

Choosing an IT partner can feel like buying an insurance policy you hope you never need. The right choice keeps your team productive, your data safe, and your budget predictable. The wrong choice shows up as slow tickets, surprise bills, and risky sh...

Choosing the Right Legal Aid Lawyer in Sutherland Shire: Key Considerations

Legal aid services play an essential role in ensuring access to justice for all. For people in the Sutherland Shire who may not have the financial means to pay for private legal assistance, legal aid ensures that everyone has access to representa...

Watercolor vs. Oil vs. Digital: Which Medium Fits Your Pet's Personality?

When it comes to immortalizing your pet’s unique personality in art, choosing the right medium is essential. Each artistic medium, whether watercolor, oil, or digital, has distinct qualities that can bring out the spirit of your furry friend in dif...

The Times Features

NSW has a new fashion sector strategy – but a sustainable industry needs a federally legislated response

The New South Wales government recently announced the launch of the NSW Fashion Sector Strategy, 2025–28[1]. The strategy, developed in partnership with the Australian Fashion ...

From Garden to Gift: Why Roses Make the Perfect Present

Think back to the last time you gave or received flowers. Chances are, roses were part of the bunch, or maybe they were the whole bunch.   Roses tend to leave an impression. Even ...

Do I have insomnia? 5 reasons why you might not

Even a single night of sleep trouble can feel distressing and lonely. You toss and turn, stare at the ceiling, and wonder how you’ll cope tomorrow. No wonder many people star...

Wedding Photography Trends You Need to Know (Before You Regret Your Album)

Your wedding album should be a timeless keepsake, not something you cringe at years later. Trends may come and go, but choosing the right wedding photography approach ensures your ...

Can you say no to your doctor using an AI scribe?

Doctors’ offices were once private. But increasingly, artificial intelligence (AI) scribes (also known as digital scribes) are listening in. These tools can record and trans...

There’s a new vaccine for pneumococcal disease in Australia. Here’s what to know

The Australian government announced last week there’s a new vaccine[1] for pneumococcal disease on the National Immunisation Program for all children. This vaccine replaces pr...