The Times Australia
The Times World News

.

A new tool helps catch nasty comments – even when they’re disguised

  • Written by Johnny Chan, Lecturer, Business School, University of Auckland, Waipapa Taumata Rau



People determined to spread toxic messages online have taken to masking their words to bypass automated moderation filters.

A user might replace letters with numbers or symbols, for example, writing “Y0u’re st00pid” instead of “You’re stupid”.

Another tactic involves combining words, such as “IdiotFace”. Doing this masks the harmful intent from systems that look for individual toxic words.

Similarly, harmful terms can be altered with spaces or additional characters, such as “h a t e ” or “h@te”, effectively slipping through keyword-based filters.

While the intent remains harmful, traditional moderation tools often overlook such messages. This leaves users — particularly vulnerable groups — exposed to their negative impact.

To address this, we have developed a novel pre-processing technique[1] designed to help moderation tools more effectively handle the subtle complexities of hidden toxicity.

An intelligent assistant

Our tool works in conjunction with existing moderation. It acts as an intelligent assistant, preparing content for deeper and more accurate evaluation by restructuring and refining input text.

By addressing common tricks users employ to disguise harmful intent, it ensures moderation systems are more effective. The tool performs three key functions.

  1. It first simplifies the text. Irrelevant elements, such as excessive punctuation or extraneous characters, are removed to make text straightforward and ready for evaluation.

  2. It then standardises what is written. Variations in spelling, phrasing and grammar are resolved. This includes interpreting deliberate misspellings (“h8te” for “hate”).

  3. Finally, it looks for patterns. Recurring strategies such as breaking up toxic words (“I d i o t”), or embedding them within benign phrases, are identified and normalised to reveal the underlying intent.

These steps can break apart compound words like “IdiotFace” or normalise modified phrases like “Y0u’re st00pid”. This makes harmful content visible to traditional filters.

Importantly, our work is not about reinventing the wheel but ensuring the existing wheel functions as effectively as it should, even when faced with disguised toxic messages.

Girl in the dark looking sadly at a phone.
Our new tool cleans up toxic comments that have been hidden behind misspellings and extra characters. ClarkandCompany/Getty Images[2]

Catching subtle forms of toxicity

The applications of this tool extend across a wide range of online environments. For social media platforms, it enhances the ability to detect harmful messages, creating a safer space for users. This is particularly important for protecting younger audiences, who may be more vulnerable to online abuse.

By catching subtle forms of toxicity, the tool helps to prevent harmful behaviours like bullying from persisting unchecked.

Businesses can also use this technology to safeguard their online presence. Negative campaigns or covert attacks on brands often employ subtle and disguised messaging to avoid detection. By processing such content before it is moderated, the tool ensures that businesses can respond swiftly to any reputational threats.

Additionally, policymakers and organisations that monitor public discourse can benefit from this system. Hidden toxicity, particularly in polarised discussions, can undermine efforts to maintain constructive dialogue.

The tool provides a more robust way for identifying problematic content and ensuring that debates remain respectful and productive.

Better moderation

Our tool marks an important advance in content moderation. By addressing the limitations of traditional keyword-based filters, it offers a practical solution to the persistent issue of hidden toxicity.

Importantly, it demonstrates how small but focused improvements can make a big difference in creating safer and more inclusive online environments. As digital communication continues to evolve, tools like ours will play an increasingly vital role in protecting users and fostering positive interactions.

While this research addresses the challenges of detecting hidden toxicity within text, the journey is far from over.

Future advances will likely delve deeper into the complexities of context—analysing how meaning shifts depending on conversational dynamics, cultural nuances and intent.

By building on this foundation, the next generation of content moderation systems could uncover not just what is being said but also the circumstances in which it is said, paving the way for safer and more inclusive online spaces.

References

  1. ^ novel pre-processing technique (methods-x.com)
  2. ^ ClarkandCompany/Getty Images (www.gettyimages.com.au)

Read more https://theconversation.com/unmasking-hidden-online-hate-a-new-tool-helps-catch-nasty-comments-even-when-theyre-disguised-244636

Times Magazine

Building an AI-First Culture in Your Company

AI isn't just something to think about anymore - it's becoming part of how we live and work, whether we like it or not. At the office, it definitely helps us move faster. But here's the thing: just using tools like ChatGPT or plugging AI into your wo...

Data Management Isn't Just About Tech—Here’s Why It’s a Human Problem Too

Photo by Kevin Kuby Manuel O. Diaz Jr.We live in a world drowning in data. Every click, swipe, medical scan, and financial transaction generates information, so much that managing it all has become one of the biggest challenges of our digital age. Bu...

Headless CMS in Digital Twins and 3D Product Experiences

Image by freepik As the metaverse becomes more advanced and accessible, it's clear that multiple sectors will use digital twins and 3D product experiences to visualize, connect, and streamline efforts better. A digital twin is a virtual replica of ...

The Decline of Hyper-Casual: How Mid-Core Mobile Games Took Over in 2025

In recent years, the mobile gaming landscape has undergone a significant transformation, with mid-core mobile games emerging as the dominant force in app stores by 2025. This shift is underpinned by changing user habits and evolving monetization tr...

Understanding ITIL 4 and PRINCE2 Project Management Synergy

Key Highlights ITIL 4 focuses on IT service management, emphasising continual improvement and value creation through modern digital transformation approaches. PRINCE2 project management supports systematic planning and execution of projects wit...

What AI Adoption Means for the Future of Workplace Risk Management

Image by freepik As industrial operations become more complex and fast-paced, the risks faced by workers and employers alike continue to grow. Traditional safety models—reliant on manual oversight, reactive investigations, and standardised checklist...

The Times Features

Flipping vs. Holding: Which Investment Strategy Is Right for You?

Are you wondering whether flipping a property or holding onto it is the better investment strategy? The answer isn’t one-size-fits-all. Both strategies have distinct advantages a...

Why Everyone's Talking About Sea Moss - And Should You Try It Too?

Sea moss - a humble marine plant that’s been used for centuries - is making a major comeback in modern wellness circles. And it’s not just a trend. With growing interest from athle...

A Guide to Smarter Real Estate Accounting: What You Might Be Overlooking

Real estate accounting can be a complex terrain, even for experienced investors and property managers. From tracking rental income to managing property expenses, the financial in...

What Is the Dreamtime? Understanding Aboriginal Creation Stories Through Art

Aboriginal culture is built on the deep and important meaning of Dreamtime, which links beliefs and history with the elements that make life. It’s not just myths; the Dreamtime i...

How Short-Term Lenders Offer Long-Lasting Benefits in Australia

In the world of personal and business finance, short-term lenders are often viewed as temporary fixes—quick solutions for urgent cash needs. However, in Australia, short-term len...

Why School Breaks Are the Perfect Time to Build Real Game Skills

School holidays provide uninterrupted time to focus on individual skill development Players often return sharper and more confident after structured break-time training Holid...