Mon Feb 26

What happens when we outsource boring but important work to AI? Research shows we forget how to do it ourselves

Written by: Tapani Rinta-Kahila, Lecturer in Business Information Systems, The University of Queensland

In 2009, an Air France jet crashed into the ocean, leaving no survivors. The plane’s autopilot system shut down and the pilots, having become reliant on their computerised assistant, were unable to correct the situation manually ^[1].

In 2015, a bus driver in Europe typed the wrong destination into his GPS device and cheerfully took a group of Belgian tourists on a 1,200 kilometre detour ^[2] in the wrong direction.

In 2017, in a decision later overturned on appeal, US prosecutors who had agreed to release a teenager on probation abruptly changed their minds ^[3] because an algorithm ruled the defendant “high risk”.

These are dramatic examples, but they are far from isolated. When we outsource cognitive tasks to technology – such as flying a plane, navigating, or making a judgement – research shows we may lose the ability ^[4] to perform those tasks ourselves. There is even a term for our tendency to forget information that is available through online search engines: the Google effect ^[5].

As new AI technologies promise to automate an increasing range of activities, the risk of “skill erosion” is growing. Our research shows how it can happen ^[6] – and suggests ways to keep hold of the expertise you need, even when you don’t need it every day.

Skill erosion can cripple an organisation

My research shows the risk of skill erosion is easily overlooked. In a recent study, my team and I examined skill erosion in an accounting company ^[7].

The company had recently stopped using software that automated much of its fixed-asset accounting service. However, the accountants found themselves unable to carry out the task without it. Years of over-reliance on the software had eroded their expertise, and ultimately, they had to relearn their fixed-asset accounting skills.

While the software was rule-based (it did not use machine learning or “AI”), it was “smart” enough to track depreciation and produce reports for many tax and financial purposes. These are tasks that human accountants found very complex and tedious.

The company only became aware of skill erosion after a client found errors in the accounting team’s manual reports. With its accountants lacking sufficient expertise, the company had to commission the software provider to fix the errors.

How skill erosion happens

We found that a lack of mindfulness about the automation-supported task had led to skill erosion. The old saying, “use it or lose it”, applies to cognitively intense work as much as to anything else.

The accountants were not concerned about outsourcing their thinking to the software, since it operated almost flawlessly. In other words, they fell prey to “automation complacency ^[8]”: the assumption that “all is well” while ignoring potential risks.

This had three major consequences:

they lost their awareness of what automation was doing
they lost the incentive to maintain and update relevant knowledge (such as tax legislation), because the vendor and software did that for them
as the software was reliable, they no longer bothered to check the outgoing reports for accuracy.

How to maintain your skills

So, how do you prevent complacency while using AI and other automated systems? Here are three tips:

pay attention to what the system is doing – what inputs are used, for what purpose, and what might affect its suggestions
keep your competence up to date (especially if you are legally accountable for the outcomes)
critically assess the results, even if the final outcomes appear satisfactory.

A photo of two pilots in a plane cockpit.

Keeping your skills sharp while using automated systems requires paying close attention. Shutterstock ^[10]

What would this look like in practice? Here’s an everyday example: driving with the help of an AI-powered navigation app.

Instead of blindly following the app’s instructions, pay attention to road signs and landmarks, and be aware of what you are doing even when guided by the app.

Study the map and suggested route before driving to increase your “domain knowledge”, or understanding of what is around the route. This helps you relate your specific path to the broader environment, which will be helpful if you get lost or want to find alternative routes.

When you reach your destination, reflect on the route the app suggested: was it fast, was it safe, was it enjoyable? If not, consider taking a different route next time, even if the app suggests otherwise.

Is AI a necessary companion?

The case of the accounting firm also raises a bigger question: what skills are relevant and worth maintaining, and which ones should we relinquish to automation?

There is no universal answer, as professional skills change across time, jurisdictions, industries, cultures and geographical locations. However, it is a question we will have to contend with as AI takes over activities once considered unable to be automated.

Despite the struggles, the accounting manager in our case study believes the automated software is highly beneficial. In his view, his team just got caught off guard by complacency.

In a world focused on efficiency and annual or quarterly targets, organisations favour solutions that improve things in the short term, even if they have negative long-term side effects. This is what happened in the accounting case: efficiency gains overshadowed abstract concerns about expertise, until problems ensued.

This does not mean that we should avoid AI. Organisations cannot afford to miss out on the opportunities it presents. However, they should also be aware of the risk of skill erosion.