The Times Australia
The Times World News

.

We have developed a way to screen student feedback to ensure it's useful, not abusive (and academics don't have to burn it)

  • Written by Abby Cathcart, Professor of Higher Education & Governance, Queensland University of Technology
We have developed a way to screen student feedback to ensure it's useful, not abusive (and academics don't have to burn it)

This week, many Australian universities will be sending academics the results of the first semester student evaluation surveys.

For some this will be a worrying and unpleasant time. The comments university students make anonymously in their teaching evaluations can leave academics feeling fearful[1], distressed[2] and demoralised[3].

And with good reason. As a 2021 survey[4] of Australian academics and their experiences of student feedback found:

Personally destructive, defamatory, abusive and hurtful comments were commonly reported.

Hurtful or abusive comments can remain permanently on record as a measure of performance[5]. These records can affect applications for promotion or for secure continued employment.

The authors of the 2021 survey, led by Richard Lakeman at Southern Cross University have been among those calling for[6] anonymous online surveys to be scrapped. Some academics, burned by their experience of student feedback, say they no longer open or engage[7] with student evaluation reports. They said the risk of harm outweighed any benefits.

Read more: 'Lose some weight', 'stupid old hag': universities should no longer ask students for anonymous feedback on their teachers[8]

In the Netflix show, The Chair, a memorable scene sees the character Professor Joan Hambling burn[9] her student evaluations. Clearly, a different solution is needed.

Feedback from students can still be valuable for lifting teaching standards and it’s important students have their say.

We have developed a screening system[10] using machine learning[11] (where software changes its behaviour by “learning” from user input) that allows students to talk about their experiences while protecting academics from unacceptable comments.

Why a new approach is needed

University codes of conduct remind students of their general obligation to refrain from abusive or discriminatory behaviour, but not specifically in regard to student evaluations.

Instead, universities rely on self-regulation or on others to report incidents. Some institutions use profanity blockers to screen comments. Even then, these often fail to detect emerging terms of abuse in online speech.

So, in setting up our screening system, we wanted to:

  • promote staff and student well-being
  • enhance the reliability and validity of student feedback
  • improve confidence in the integrity of survey results.

We developed a method using machine learning and a dictionary of terms to screen for unacceptable student comments. The dictionary was created by QUT drawing on historically identified unacceptable comments and incorporating prior research into abusive and discriminatory terms.

Our ‘Screenomatic’ solution

There is not a lot of published work on the detection of unacceptable or abusive comments in student evaluation surveys. So our team adapted earlier research[12] on detecting misogynistic tweets. This worked because often the student comments we looked at were similar in length to a tweet’s 280-character limit.

Our approach, which we call “Screenomatic[13]”, automatically reviewed more than 100,000 student comments during 2021 and identified those that appeared to be abuse. Trained evaluation staff members manually reviewed about 7,000 flagged comments, updating the machine-learning model after each semester. Each update improves the accuracy of auto-detection.

Read more: Gender bias in student surveys on teaching increased with remote learning. What can unis do to ensure a fair go for female staff?[14]

Ultimately, 100 comments were removed before the results were released to educators and supervisors. University policy enables comments to be re-identified in cases of potential misconduct. The central evaluations team contacted these students and reminded them of their obligations under the code of conduct.

The Screenomatic[15] model can help protect both educators and students. Staff are safeguarded from abuse, and students at risk – who make comments that indicate they need mental health help, include allegations of bullying or harassment, or that threaten staff or other students – can be offered support. Universities can share data to train the model and maintain currency.

Importantly, the process enables universities to act morally to harness student voices while protecting people’s well-being.

Useful feedback, not abuse

The number of educators who receive abusive feedback may be relatively small[16]. However, it’s still unacceptable for universities to continue to expose their staff to offensive comments in the full knowledge of their potential impact.

Read more: Our uni teachers were already among the world's most stressed. COVID and student feedback have just made things worse[17]

With last year’s High Court ruling on liability for defamatory posts[18], and attempts to improve online safety[19], there is a growing acknowledgement that people should not be able to post anonymous, harmful messages.

After all, the cost of screening responses is nothing compared to the cost to individuals (including mental health or career consequences). And that’s ignoring the potential costs of litigation and legal damages.

At the end of the day, the anonymous comments are read by real people. As a tweet in response to the Lakeman findings[20] noted:

The Screenomatic model[21] goes a long way towards enabling the “tons of useful feedback” to serve its intended purpose while ensuring people aren’t harmed in the process.

References

  1. ^ fearful (theconversation.com)
  2. ^ distressed (theconversation.com)
  3. ^ demoralised (theconversation.com)
  4. ^ survey (www.tandfonline.com)
  5. ^ measure of performance (www.tandfonline.com)
  6. ^ calling for (theconversation.com)
  7. ^ open or engage (www.tandfonline.com)
  8. ^ 'Lose some weight', 'stupid old hag': universities should no longer ask students for anonymous feedback on their teachers (theconversation.com)
  9. ^ burn (theconversation.com)
  10. ^ screening system (www.tandfonline.com)
  11. ^ machine learning (theconversation.com)
  12. ^ earlier research (link.springer.com)
  13. ^ Screenomatic (www.tandfonline.com)
  14. ^ Gender bias in student surveys on teaching increased with remote learning. What can unis do to ensure a fair go for female staff? (theconversation.com)
  15. ^ Screenomatic (www.tandfonline.com)
  16. ^ small (link.springer.com)
  17. ^ Our uni teachers were already among the world's most stressed. COVID and student feedback have just made things worse (theconversation.com)
  18. ^ liability for defamatory posts (www.nfplaw.org.au)
  19. ^ improve online safety (www.ag.gov.au)
  20. ^ Lakeman findings (doi.org)
  21. ^ Screenomatic model (www.tandfonline.com)

Read more https://theconversation.com/we-have-developed-a-way-to-screen-student-feedback-to-ensure-its-useful-not-abusive-and-academics-dont-have-to-burn-it-185041

Times Magazine

Headless CMS in Digital Twins and 3D Product Experiences

Image by freepik As the metaverse becomes more advanced and accessible, it's clear that multiple sectors will use digital twins and 3D product experiences to visualize, connect, and streamline efforts better. A digital twin is a virtual replica of ...

The Decline of Hyper-Casual: How Mid-Core Mobile Games Took Over in 2025

In recent years, the mobile gaming landscape has undergone a significant transformation, with mid-core mobile games emerging as the dominant force in app stores by 2025. This shift is underpinned by changing user habits and evolving monetization tr...

Understanding ITIL 4 and PRINCE2 Project Management Synergy

Key Highlights ITIL 4 focuses on IT service management, emphasising continual improvement and value creation through modern digital transformation approaches. PRINCE2 project management supports systematic planning and execution of projects wit...

What AI Adoption Means for the Future of Workplace Risk Management

Image by freepik As industrial operations become more complex and fast-paced, the risks faced by workers and employers alike continue to grow. Traditional safety models—reliant on manual oversight, reactive investigations, and standardised checklist...

From Beach Bops to Alpine Anthems: Your Sonos Survival Guide for a Long Weekend Escape

Alright, fellow adventurers and relaxation enthusiasts! So, you've packed your bags, charged your devices, and mentally prepared for that glorious King's Birthday long weekend. But hold on, are you really ready? Because a true long weekend warrior kn...

Effective Commercial Pest Control Solutions for a Safer Workplace

Keeping a workplace clean, safe, and free from pests is essential for maintaining productivity, protecting employee health, and upholding a company's reputation. Pests pose health risks, can cause structural damage, and can lead to serious legal an...

The Times Features

Tricia Paoluccio designer to the stars

The Case for Nuturing Creativity in the Classroom, and in our Lives I am an actress and an artist who has had the privilege of sharing my work across many countries, touring my ...

Duke of Dural to Get Rooftop Bar as New Owners Invest in Venue Upgrade

The Duke of Dural, in Sydney’s north-west, is set for a major uplift under new ownership, following its acquisition by hospitality group Good Beer Company this week. Led by resp...

Prefab’s Second Life: Why Australia’s Backyard Boom Needs a Circular Makeover

The humble granny flat is being reimagined not just as a fix for housing shortages, but as a cornerstone of circular, factory-built architecture. But are our systems ready to s...

Melbourne’s Burglary Boom: Break-Ins Surge Nearly 25%

Victorian homeowners are being warned to act now, as rising break-ins and falling arrest rates paint a worrying picture for suburban safety. Melbourne residents are facing an ...

Exploring the Curriculum at a Modern Junior School in Melbourne

Key Highlights The curriculum at junior schools emphasises whole-person development, catering to children’s physical, emotional, and intellectual needs. It ensures early year...

Distressed by all the bad news? Here’s how to stay informed but still look after yourself

If you’re feeling like the news is particularly bad at the moment, you’re not alone. But many of us can’t look away – and don’t want to. Engaging with news can help us make ...