The Times Australia
The Times World News

.
The Times Real Estate

.

Yes, uni students say some awful things in teaching surveys, so how can we use them to improve?

  • Written by Joseph Crawford, Lecturer in Learning and Teaching, Academic Division, University of Tasmania
Yes, uni students say some awful things in teaching surveys, so how can we use them to improve?

Imagine some of the key evidence for promotions at work being anonymous responses[1] from coworkers who just received a bad performance evaluation from you. Something similar happens in higher education, with teachers rated by students grateful for good grades or disgruntled by low grades. That’s a bitter pill to swallow for some academics[2].

Evidence[3] tells us students take their feedback personally. Jurors’ decision-making[4] is similarly affected by their emotional state. People make worse decisions when they are uncertain[5] or stressed[6], which are two common states for students.

So how unreliable are student evaluations? And what can we do about it? Our work[7] indicates there is still much to be done in this space, but we can set some rules to make it easier.

Read more: 'Lose some weight', 'stupid old hag': universities should no longer ask students for anonymous feedback on their teachers[8]

All surveys are not equal

Australia’s national Student Experience Survey[9] is considered “the pulse” on student satisfaction rather than a device to enable teacher growth, with the data being easily skewable by circumstances at the time. Unsurprisingly, during 2020, universities that already had an online presence saw the smallest decline in student experience scores.

So the question becomes: did the quality of learning crash in Group of Eight universities, which had the greatest declines in student experience? Unlikely. Instead, students’ ratings reflected their difficulties engaging with new forms of teaching and learning, plus the inertia of COVID-19 lockdowns.

Maybe they should have given students chocolate[10]?

The reality is these surveys do not tell us how students learn, but instead how students perceive their learning. Yet students aren’t experts at what learning is[11]. And when students don’t receive effective training in evaluation, it’s hardly a surprise that teacher gender, race and attractiveness[12] change scores.

“Everyone is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid.” - Albert Einstein.

Instead, let’s ask students to share the most enjoyable content, the most rewarding educational technologies, and where improvement was needed. Include ethics and feedback training for bonus credit.

Read more: Our uni teachers were already among the world's most stressed. COVID and student feedback have just made things worse[13]

Making survey tools that work

Psychometrics is the study of measurements. Interestingly, many academics have specialist knowledge in developing surveys that are designed to be valid and reliable. But it’s unclear if universities use them as a resource to develop their surveys, with some academics wondering if they should. The 2021 Employer Satisfaction Survey Methodological Report[14], for example, does not refer explicitly to the words validity or reliability once across its 140 pages.

Valid surveys exist when the questions align to what we think they are measuring. Using a stopwatch to measure time is easy. When we try to decide how we feel about intangible concepts, it’s harder.

The national Student Experience Survey[15] asks students whether they have developed a sense of belonging to their institution. Yet the evidence on belonging[16] indicates it is typically developed through interpersonal relationships, not institutions, and not through universities[17].

Reliable surveys exist when the questions generate consistent results over time and over different participants. It’s analogous to when we bake a cake and we assume the scales will always accurately measure 40 grams of butter.

Speaking of sweets, scores in student surveys are easy to game[18]. Inflating student grades[19] does the trick.

In contrast, as an example, the Australian Student Experience Survey[20] asks whether students have developed their critical thinking skills during their course. How accurately can a person with low critical thinking skills answer this question?

5 rules for surveys to help teachers improve

There are ways that surveys can be used for good. To actually help teachers be better educators and improve student learning. But it requires a reset.

Here are five rules institutions could consider when developing their surveys.

1. Find psychometric specialists to create quality tools

We go to dentists to have our teeth fixed. The same rule applies here. Find individuals who can take the theory[21] of scale development[22] (producing reliable and valid measures to assess an attribute of interest) into the practices of learning and teaching.

2. Change when the survey is done

Lots of evaluations[23] are done before, during and after a program. In higher education, they are completed only after the class has ended.

A change to evaluations at multiple points will help identify if the learner makes progress during the class. This would also help control for cohort problems (one year, for example, students are smarter).

For student experience, contrasting how the same student rates different classes each semester may serve as a stable measure to see which classes need review.

3. Use more than just numbers

The numbers explain how we are tracking, and this is not inherently bad. The qualitative comments (mostly[24]) help us explore what those mean. Mixed methods[25] approaches can help.

4. Control for bias

It’s not always possible to eliminate bias and emotion. We can seek to understand them and use the measures as a case-by-case conversation about improving teaching. Developing reliable and valid tools will help, but if the aim is for these to help teachers improve, then we need to focus on that, not cross-institutional comparisons.

Better yet, let’s actively recognise teachers’ professional growth, call decline into question, and report on averages.

We can also train students to be better evaluators.

5. Create a growth community

Teaching quality surveys do not necessarily increase teaching quality[26], but they can.

The surveys offer an opportunity to raise awareness of differences. If students rate seven items at 90% but one is 84%, this should prompt research into the reasons. It could be a great opportunity to create more meaningful content; it could also just be an outlier[27].

Use these findings as publishing opportunities[28] to share what was learned.

References

  1. ^ anonymous responses (theconversation.com)
  2. ^ some academics (theconversation.com)
  3. ^ Evidence (doi.org)
  4. ^ Jurors’ decision-making (doi.org)
  5. ^ uncertain (psycnet.apa.org)
  6. ^ stressed (doi.org)
  7. ^ Our work (doi.org)
  8. ^ 'Lose some weight', 'stupid old hag': universities should no longer ask students for anonymous feedback on their teachers (theconversation.com)
  9. ^ Student Experience Survey (www.qilt.edu.au)
  10. ^ given students chocolate (doi.org)
  11. ^ aren’t experts at what learning is (doi.org)
  12. ^ gender, race and attractiveness (doi.org)
  13. ^ Our uni teachers were already among the world's most stressed. COVID and student feedback have just made things worse (theconversation.com)
  14. ^ 2021 Employer Satisfaction Survey Methodological Report (www.qilt.edu.au)
  15. ^ Student Experience Survey (www.qilt.edu.au)
  16. ^ evidence on belonging (doi.org)
  17. ^ through universities (doi.org)
  18. ^ easy to game (doi.org)
  19. ^ Inflating student grades (doi.org)
  20. ^ Student Experience Survey (www.qilt.edu.au)
  21. ^ theory (search.informit.org)
  22. ^ scale development (doi.org)
  23. ^ evaluations (doi.org)
  24. ^ mostly (theconversation.com)
  25. ^ Mixed methods (doi.org)
  26. ^ teaching quality (doi.org)
  27. ^ outlier (en.wikipedia.org)
  28. ^ publishing opportunities (doi.org)

Read more https://theconversation.com/yes-uni-students-say-some-awful-things-in-teaching-surveys-so-how-can-we-use-them-to-improve-177155

The Times Features

Best Deals on Home Furniture Online

Key Highlights Discover the best deals on high-quality outdoor furniture online. Transform your outdoor space into a stylish and comfortable oasis. Explore a wide range of d...

Discover the Best Women's Jumpers for Every Season

Key Highlights Explore lightweight jumpers for spring and summer, ensuring breathability and ease. Wrap up warm with cozy wool jumpers for the chilly autumn and winter season...

Uncover the Elegance of Gorgeous Diamond Tennis Necklaces

Key Highlights Diamond tennis necklaces are a timeless piece of jewelry that exudes elegance and sophistication. They feature a continuous line of brilliant-cut diamonds, cre...

Dental Implants vs. Dentures: Which Is Better for You?

When it comes to replacing missing teeth, two of the most common options are dental implants and dentures. Both have their advantages and disadvantages, so choosing between them ...

What Neck Pain Really Means (And Why It’s More Than Just Poor Posture)

Neck pain is often brushed off as something temporary — a tight spot after a long day at the desk or a poor night’s sleep. But when the discomfort keeps returning, it could be a ...

The Work of Gosha Rubchinskiy: Fashion, Culture, and Youth

From Designer to Cultural Architect Gosha Rubchinskiy is not just a fashion designer—he's a cultural force. Born in Moscow in 1984, Rubchinskiy began his career in fashion in t...

Times Magazine

The Essential Guide to Transforming Office Spaces for Maximum Efficiency

Why Office Fitouts MatterA well-designed office can make all the difference in productivity, employee satisfaction, and client impressions. Businesses of all sizes are investing in updated office spaces to create environments that foster collaborat...

The A/B Testing Revolution: How AI Optimized Landing Pages Without Human Input

A/B testing was always integral to the web-based marketing world. Was there a button that converted better? Marketing could pit one against the other and see which option worked better. This was always through human observation, and over time, as d...

Using Countdown Timers in Email: Do They Really Increase Conversions?

In a world that's always on, where marketers are attempting to entice a subscriber and get them to convert on the same screen with one email, the power of urgency is sometimes the essential element needed. One of the most popular ways to create urg...

Types of Software Consultants

In today's technology-driven world, businesses often seek the expertise of software consultants to navigate complex software needs. There are several types of software consultants, including solution architects, project managers, and user experienc...

CWU Assistive Tech Hub is Changing Lives: Win a Free Rollator Walker This Easter!

🌟 Mobility. Independence. Community. All in One. This Easter, the CWU Assistive Tech Hub is pleased to support the Banyule community by giving away a rollator walker. The giveaway will take place during the Macleod Village Easter Egg Hunt & Ma...

"Eternal Nurture" by Cara Barilla: A Timeless Collection of Wisdom and Healing

Renowned Sydney-born author and educator Cara Barilla has released her latest book, Eternal Nurture, a profound collection of inspirational quotes designed to support mindfulness, emotional healing, and personal growth. With a deep commitment to ...

LayBy Shopping