The Times Australia
The Times World News

.
Times Media

.

Algorithms can decide your marks, your work prospects and your financial security. How do you know they're fair?

  • Written by Kalervo Gulson, Professor and ARC Future Fellow, Education & Social Work, Education Futures Studio, University of Sydney

Algorithms are becoming commonplace. They can determine employment prospects, financial security[1] and more. The use of algorithms can be controversial – for example, robodebt[2], as the Australian government’s flawed online welfare compliance system came to be known.

Algorithms are increasingly being used to make decisions that have a lasting impact on our current and future lives.

Some of the greatest impacts of algorithmic decision-making are in education. If you have anything to do with an Australian school or a university, at some stage an algorithm will make a decision that matters for you.

So what sort of decisions might involve algorithms? Some decisions will involve the next question for school students to answer on a test, such as the online provision of NAPLAN[3]. Some algorithms support human decision-making in universities[4], such as identifying students at risk of failing a subject. Others take the human out of the loop, like some forms of online exam supervision[5].

Read more: Unis are using artificial intelligence to keep students sitting exams honest. But this creates its own problems[6]

How do algorithms work?

Despite their pervasive impacts on our lives, it is often difficult to understand how algorithms work, why they have been designed, and why they are used. As algorithms become a key part of decision-making in education – and many other aspects of our lives – people need to know two things:

  1. how algorithms work

  2. the kinds of trade-offs that are made in decision-making using algorithms.

In research to explore these two issues, we developed an algorithm game[7] using participatory methodologies to involve diverse stakeholders in the research. The process becomes a form of collective experimentation to encourage new perspectives and insights into an issue.

Our algorithm game is based on the UK exam controversy[8] in 2020. During COVID-19 lockdowns, an algorithm was used to determine grades[9] for students wishing to attend university. The algorithm predicted grades for some students that were far lower than expected. In the face of protests, the algorithm was eventually scrapped.

Read more: Scotland's exam result crisis: assessment and social justice in a time of COVID-19[10]

Our interdisciplinary team[11] co-designed the UK exam algorithm game over a series of two workshops and multiple meetings this year. Our workshops included students, data scientists, ethicists and social scientists. Such interdisciplinary perspectives are vital to understand the range of social, ethical and technical implications of algorithms in education.

Algorithms make trade-offs, so transparency is needed

The UK example highlights key issues with using algorithms in society, including issues of transparency and bias in data. These issues matter everywhere, including Australia[12].

We designed the algorithm game to help people develop the tools to have more of a say in shaping the world algorithms are creating. Algorithm “games” invite people to play with and learn about the parameters of how an algorithm operates. Examples include games that show people how algorithms are used in criminal sentencing[13], or can help to predict fire risk in buildings[14]

Diagram of ingredients, recipe and cake matching the input, procedure and output
For the algorithm game, the authors used the analogy of making a cake – a product of choosing ingredients and mixing and baking them in a certain way – to explain how the inputs and parameters of steps in the process affect the outcome. Author provided (no reuse)

There is a growing public awareness that algorithms, especially those used in forms of artificial intelligence, need to be understood as raising issues of fairness[15]. But while everyone may have a vernacular understanding of what is fair or unfair, when algorithms are used numerous trade-offs are involved.

Read more: From robodebt to racism: what can go wrong when governments let algorithms make the decisions[16]

In our algorithm game, we take people through a series of problems where the solution to a fairness problem simply introduces a new one. For example, the UK algorithm did not work very well for predicting the grades of students in schools where smaller numbers of students took certain subjects. This was unfair for these students.

The solution meant the algorithm was not used for these often very privileged schools[17]. These students then received grades predicted by their teachers. But these grades were mostly higher than the algorithm-generated grades received by students in larger schools, which were more often government comprehensive schools. So this meant the decision was fair for students in small schools, unfair for those in larger schools who had grades allocated by the algorithm.

What we try to show in our game that it is not possible to have a perfect outcome. And that neither humans or algorithms will make a set of choices that are fair for everyone. This means we have to make decisions about which values matter when we use algorithms.

Public must have a say to balance the power of EdTech

While our algorithm game focuses on the use of an algorithm developed by a government, algorithms in education are commonly introduced as part of educational technology. The EdTech industry is expanding rapidly in Australia[18]. Companies are seeking to dominate all stages of education: enrolment, learning design, learning experience and lifelong learning.

Alongside these developments, COVID-19 has accelerated the use of algorithmic decision-making in education and beyond.

Read more: Artificial intelligence holds great potential for both students and teachers – but only if used wisely[19]

While these innovations open up amazing possibilities, algorithms also bring with them a set of challenges we must face as a society. Examples like the UK exam algorithm expose us to how such algorithms work and the kinds of decisions that have to be made when designing them. We are then forced to answer deep questions of which values we will choose to prioritise and what roadmap for research[20] we take forward.

Our choices will shape our future and the future of generations to come.

The following people were also involved in the research underpinning the algorithm game. From the Gradient Institute[21] for responsible AI, Simon O'Callaghan, Alistair Reid and Tiberio Caetano. And from the Tech for Social Good[22] group, Vincent Zhang.

References

  1. ^ financial security (www.afr.com)
  2. ^ robodebt (www.innovationaus.com)
  3. ^ online provision of NAPLAN (nap.edu.au)
  4. ^ human decision-making in universities (theconversation.com)
  5. ^ online exam supervision (theconversation.com)
  6. ^ Unis are using artificial intelligence to keep students sitting exams honest. But this creates its own problems (theconversation.com)
  7. ^ an algorithm game (www.edufuturesstudio.com)
  8. ^ UK exam controversy (www.theverge.com)
  9. ^ algorithm was used to determine grades (blogs.lse.ac.uk)
  10. ^ Scotland's exam result crisis: assessment and social justice in a time of COVID-19 (theconversation.com)
  11. ^ Our interdisciplinary team (education-futures-studio.sydney.edu.au)
  12. ^ Australia (www.sbs.com.au)
  13. ^ criminal sentencing (www.technologyreview.com)
  14. ^ predict fire risk in buildings (automating.nyc)
  15. ^ issues of fairness (www.nature.com)
  16. ^ From robodebt to racism: what can go wrong when governments let algorithms make the decisions (theconversation.com)
  17. ^ very privileged schools (ffteducationdatalab.org.uk)
  18. ^ expanding rapidly in Australia (www.pwc.com.au)
  19. ^ Artificial intelligence holds great potential for both students and teachers – but only if used wisely (theconversation.com)
  20. ^ roadmap for research (www.nuffieldfoundation.org)
  21. ^ Gradient Institute (gradientinstitute.org)
  22. ^ Tech for Social Good (www.techforsocialgood.org)

Read more https://theconversation.com/algorithms-can-decide-your-marks-your-work-prospects-and-your-financial-security-how-do-you-know-theyre-fair-171590

The Times Features

Sunshine Coast Personal Injury Lawyers: Expert Guidance for Work Accident Claims

When accidents occur in the workplace, they can have profound and lasting effects on a worker's life, both physically and financially. In Queensland's Sunshine Coast, personal in...

iS Clinical's Skincare Secrets for Radiant Party Season Skin

As the festive season approaches, having runway-ready skin is easier than ever with iS Clinical's  top skincare prep tips. Whether you're walking the red carpet or simply stepp...

How to Camp Sustainably in Byron Bay: Eco-Friendly Tips

Camping in Byron Bay is a fantastic way to soak up nature, from beautiful beaches to lush rainforests. It’s no wonder Byron Bay camping is on so many bucket lists! But with more ...

Book a room in Broome as the northwest is best

Looking for a different kind of seaside escape? Done the east coast? Then book a room in Broome to discover why those in the know say the northwest is best, with Broome one of ...

Primary care involves more than GPs. A new review shows how patients can better access care

Australians today are more likely[1] than previous generations to live with complex and chronic diseases, such as diabetes, heart disease and depression. This means they’re mo...

Why Your Credit Score Matters More Than You Think in Australia

Your credit score isn't just a number, it's a reflection of your financial reputation. In Australia, a good credit score can unlock doors to better financial opportunities, while...

Enhance Your Home’s Comfort and Style with Blackout Curtains from 3Hlinen

Creating a comfortable and peaceful home environment often involves choosing the right window treatments. Among the various options available, blackout curtains stand out as a ve...

Is thirst a good predictor of dehydration?

Water is essential for daily functioning and health, and we can only survive a few days[1] without it. Yet we constantly lose water through sweat, urination and even evaporatio...

How doctors use light to diagnose disease

You’re not feeling well. You’ve had a pounding headache all week, dizzy spells and have vomited up your past few meals. You visit your GP to get some answers and sit while they ...

Times Magazine

5 Things You Didn’t Know Your Smartphone Could do

In this modern technological era, smartphones have become a necessity, especially for people who do online work. They are used for so many functions such as making phone calls, taking selfies, messaging, streaming news and movies, and emailing. W...

Everything You Need to Know About Stretch Films

Concerned about product damage in shipping? Using stretch film to secure your products and pallets against shipping accidents and environmental damage is a good investment. Your products will arrive safely at their destination, and they’re prote...

CSIRO and Space Machines partner to test Australian flexible solar cells in space

Space Machines and CSIRO partner to test Australian flexible solar cells in space  SYDNEY, Australia - Thursday 5th August 2021 - Australian in-space transportation provider Space Machines Company (SMC) and CSIRO, Australia's national scie...

Sydney's Finest: How to Identify a Top-Tier SEO Company

In the dynamic realm of digital marketing, the success of your online presence relies heavily on effective search engine optimisation (SEO). A pivotal force in this journey is the SEO company you choose. In Sydney's competitive business landscape...

Types of EV Chargers

The world of electric vehicles is a confusing one, filled with abbreviations and acronyms that even the most seasoned EV drivers can't keep straight. It's also a rapidly changing landscape where cutting-edge technology is being developed at an in...

Tesselaar Tulip Festival

THE BULBS ARE BACK FOR A DAZZLING DISPLAY THIS SPRING AT THE 2022 TESSELAAR TULIP FESTIVAL  The much-loved and visually spectacular Tesselaar Tulip Festival, in Melbourne’s magical Dandenong Ranges, returns this spring to once again surprise, de...