The Times Australia
The Times World News

.

We have developed a way to screen student feedback to ensure it's useful, not abusive (and academics don't have to burn it)

  • Written by Abby Cathcart, Professor of Higher Education & Governance, Queensland University of Technology
We have developed a way to screen student feedback to ensure it's useful, not abusive (and academics don't have to burn it)

This week, many Australian universities will be sending academics the results of the first semester student evaluation surveys.

For some this will be a worrying and unpleasant time. The comments university students make anonymously in their teaching evaluations can leave academics feeling fearful[1], distressed[2] and demoralised[3].

And with good reason. As a 2021 survey[4] of Australian academics and their experiences of student feedback found:

Personally destructive, defamatory, abusive and hurtful comments were commonly reported.

Hurtful or abusive comments can remain permanently on record as a measure of performance[5]. These records can affect applications for promotion or for secure continued employment.

The authors of the 2021 survey, led by Richard Lakeman at Southern Cross University have been among those calling for[6] anonymous online surveys to be scrapped. Some academics, burned by their experience of student feedback, say they no longer open or engage[7] with student evaluation reports. They said the risk of harm outweighed any benefits.

Read more: 'Lose some weight', 'stupid old hag': universities should no longer ask students for anonymous feedback on their teachers[8]

In the Netflix show, The Chair, a memorable scene sees the character Professor Joan Hambling burn[9] her student evaluations. Clearly, a different solution is needed.

Feedback from students can still be valuable for lifting teaching standards and it’s important students have their say.

We have developed a screening system[10] using machine learning[11] (where software changes its behaviour by “learning” from user input) that allows students to talk about their experiences while protecting academics from unacceptable comments.

Why a new approach is needed

University codes of conduct remind students of their general obligation to refrain from abusive or discriminatory behaviour, but not specifically in regard to student evaluations.

Instead, universities rely on self-regulation or on others to report incidents. Some institutions use profanity blockers to screen comments. Even then, these often fail to detect emerging terms of abuse in online speech.

So, in setting up our screening system, we wanted to:

  • promote staff and student well-being
  • enhance the reliability and validity of student feedback
  • improve confidence in the integrity of survey results.

We developed a method using machine learning and a dictionary of terms to screen for unacceptable student comments. The dictionary was created by QUT drawing on historically identified unacceptable comments and incorporating prior research into abusive and discriminatory terms.

Our ‘Screenomatic’ solution

There is not a lot of published work on the detection of unacceptable or abusive comments in student evaluation surveys. So our team adapted earlier research[12] on detecting misogynistic tweets. This worked because often the student comments we looked at were similar in length to a tweet’s 280-character limit.

Our approach, which we call “Screenomatic[13]”, automatically reviewed more than 100,000 student comments during 2021 and identified those that appeared to be abuse. Trained evaluation staff members manually reviewed about 7,000 flagged comments, updating the machine-learning model after each semester. Each update improves the accuracy of auto-detection.

Read more: Gender bias in student surveys on teaching increased with remote learning. What can unis do to ensure a fair go for female staff?[14]

Ultimately, 100 comments were removed before the results were released to educators and supervisors. University policy enables comments to be re-identified in cases of potential misconduct. The central evaluations team contacted these students and reminded them of their obligations under the code of conduct.

The Screenomatic[15] model can help protect both educators and students. Staff are safeguarded from abuse, and students at risk – who make comments that indicate they need mental health help, include allegations of bullying or harassment, or that threaten staff or other students – can be offered support. Universities can share data to train the model and maintain currency.

Importantly, the process enables universities to act morally to harness student voices while protecting people’s well-being.

Useful feedback, not abuse

The number of educators who receive abusive feedback may be relatively small[16]. However, it’s still unacceptable for universities to continue to expose their staff to offensive comments in the full knowledge of their potential impact.

Read more: Our uni teachers were already among the world's most stressed. COVID and student feedback have just made things worse[17]

With last year’s High Court ruling on liability for defamatory posts[18], and attempts to improve online safety[19], there is a growing acknowledgement that people should not be able to post anonymous, harmful messages.

After all, the cost of screening responses is nothing compared to the cost to individuals (including mental health or career consequences). And that’s ignoring the potential costs of litigation and legal damages.

At the end of the day, the anonymous comments are read by real people. As a tweet in response to the Lakeman findings[20] noted:

The Screenomatic model[21] goes a long way towards enabling the “tons of useful feedback” to serve its intended purpose while ensuring people aren’t harmed in the process.

References

  1. ^ fearful (theconversation.com)
  2. ^ distressed (theconversation.com)
  3. ^ demoralised (theconversation.com)
  4. ^ survey (www.tandfonline.com)
  5. ^ measure of performance (www.tandfonline.com)
  6. ^ calling for (theconversation.com)
  7. ^ open or engage (www.tandfonline.com)
  8. ^ 'Lose some weight', 'stupid old hag': universities should no longer ask students for anonymous feedback on their teachers (theconversation.com)
  9. ^ burn (theconversation.com)
  10. ^ screening system (www.tandfonline.com)
  11. ^ machine learning (theconversation.com)
  12. ^ earlier research (link.springer.com)
  13. ^ Screenomatic (www.tandfonline.com)
  14. ^ Gender bias in student surveys on teaching increased with remote learning. What can unis do to ensure a fair go for female staff? (theconversation.com)
  15. ^ Screenomatic (www.tandfonline.com)
  16. ^ small (link.springer.com)
  17. ^ Our uni teachers were already among the world's most stressed. COVID and student feedback have just made things worse (theconversation.com)
  18. ^ liability for defamatory posts (www.nfplaw.org.au)
  19. ^ improve online safety (www.ag.gov.au)
  20. ^ Lakeman findings (doi.org)
  21. ^ Screenomatic model (www.tandfonline.com)

Read more https://theconversation.com/we-have-developed-a-way-to-screen-student-feedback-to-ensure-its-useful-not-abusive-and-academics-dont-have-to-burn-it-185041

Times Magazine

DIY Is In: How Aussie Parents Are Redefining Birthday Parties

When planning his daughter’s birthday, Rich opted for a DIY approach, inspired by her love for drawing maps and giving clues. Their weekend tradition of hiding treats at home sparked the idea, and with a pirate ship playground already chosen as t...

When Touchscreens Turn Temperamental: What to Do Before You Panic

When your touchscreen starts acting up, ignoring taps, registering phantom touches, or freezing entirely, it can feel like your entire setup is falling apart. Before you rush to replace the device, it’s worth taking a deep breath and exploring what c...

Why Social Media Marketing Matters for Businesses in Australia

Today social media is a big part of daily life. All over Australia people use Facebook, Instagram, TikTok , LinkedIn and Twitter to stay connected, share updates and find new ideas. For businesses this means a great chance to reach new customers and...

Building an AI-First Culture in Your Company

AI isn't just something to think about anymore - it's becoming part of how we live and work, whether we like it or not. At the office, it definitely helps us move faster. But here's the thing: just using tools like ChatGPT or plugging AI into your wo...

Data Management Isn't Just About Tech—Here’s Why It’s a Human Problem Too

Photo by Kevin Kuby Manuel O. Diaz Jr.We live in a world drowning in data. Every click, swipe, medical scan, and financial transaction generates information, so much that managing it all has become one of the biggest challenges of our digital age. Bu...

Headless CMS in Digital Twins and 3D Product Experiences

Image by freepik As the metaverse becomes more advanced and accessible, it's clear that multiple sectors will use digital twins and 3D product experiences to visualize, connect, and streamline efforts better. A digital twin is a virtual replica of ...

The Times Features

What Makes Certain Rings or Earrings Timeless Versus Trendy?

Timeless rings and earrings are defined by designs that withstand the test of time, quality craftsmanship, and versatility. Trendy pieces, on the other hand, often stand testimony ...

Italian Street Kitchen: A Nation’s Favourite with Expansion News on Horizon

Successful chef brothers, Enrico and Giulio Marchese, weigh in on their day-to-day at Australian foodie favourite, Italian Street Kitchen - with plans for ‘ambitious expansion’ to ...

What to Expect During a Professional Termite Inspection

Keeping a home safe from termites isn't just about peace of mind—it’s a vital investment in the structure of your property. A professional termite inspection is your first line o...

Booty and the Beasts - The Podcast

Cult TV Show Back with Bite as a Riotous New Podcast  The show that scandalised, shocked and entertained audiences across the country, ‘Beauty and the Beast’, has returned in ...

A Guide to Determining the Right Time for a Switchboard Replacement

At the centre of every property’s electrical system is the switchboard – a component that doesn’t get much attention until problems arise. This essential unit directs electrici...

Après Skrew: Peanut Butter Whiskey Turns Australia’s Winter Parties Upside Down

This August, winter in Australia is about to get a lot nuttier. Skrewball Whiskey, the cult U.S. peanut butter whiskey that’s taken the world by storm, is bringing its bold brand o...