The Times Australia
The Times World News

.
The Times Real Estate

.

Philosophers have studied 'counterfactuals' for decades. Will they help us unlock the mysteries of AI?

  • Written by Sam Baron, Associate Professor, Philosophy of Science, Australian Catholic University
Philosophers have studied 'counterfactuals' for decades. Will they help us unlock the mysteries of AI?

Artificial intelligence is increasingly being rolled out all around the world to help make decisions in our lives, whether it’s loan decisions by banks[1], medical diagnoses[2], or US law enforcement predicting a criminal’s likelihood of re-offending[3].

Yet many AI systems are black boxes: no one understands how they work. This has led to a demand for “explainable AI”, so we can understand why an AI model yielded a specific output, and what biases may have played a role.

Explainable AI is a growing branch of AI research. But what’s perhaps less well known is the role philosophy plays in its development.

Specifically, one idea called “counterfactual explanation” is often put forth as a solution to the black box problems. But once you understand the philosophy behind it, you can start to understand why it falls short.

Why explanations matter

When AI is used to make life-changing decisions, the people impacted deserve an explanation of how that decision was reached. This was recently recognised through the European Union’s General Data Protection Regulation[4], which supports an individual’s right to explanation.

The need for explanation was also highlighted in the Robodebt case in Australia[5], where an algorithm was used to predict debt levels for individuals receiving social security. The system made many mistakes, placing people into debt who shouldn’t have been.

It was only once the algorithm was fully explained that the mistake was identified – but by then the damage had been done. The outcome was so damaging it led to a royal commission[6] being established in August 2022.

In the Robodebt case, the algorithm in question was fairly straightforward and could be explained. We should not expect this to always be the case going forward. Current AI models using machine-learning to process data are much more sophisticated.

Read more: Not everything we call AI is actually 'artificial intelligence'. Here's what you need to know[7]

The big, glaring black box

Suppose a person named Sara applies for a loan. The bank asks her to provide information including her marital status, debt level, income, savings, home address and age.

The bank then feeds this information into an AI system, which returns a credit score. The score is low and is used to disqualify Sara for the loan, but neither Sara nor the bank employees know why the system scored Sara so low.

Unlike with Robodebt, the algorithm being used here may be extremely complicated and not easily explained. There is therefore no straightforward way to know whether it has made a mistake, and Sara has no way to get the information she needs to argue against the decision.

This scenario isn’t entirely hypothetical: loan decisions are likely to be outsourced to algorithms in the US, and there’s a real risk they will encode bias[8]. To mitigate risk, we must try to explain how they work.

Read more: Everyone's having a field day with ChatGPT – but nobody knows how it actually works[9]

The counterfactual approach

Broadly speaking, there are two types of approaches[10] to explainable AI. One involves cracking open a system and studying its internal components to discern how it works. But this usually isn’t possible due to the sheer complexity of many AI systems.

The other approach is to leave the system unopened, and instead study its inputs and outputs, looking for patterns. The “counterfactual” method falls under this approach.

Counterfactuals are claims about what would happen if things had played out differently. In an AI context, this means considering how the output from an AI system might be different if it receives different inputs. We can then supposedly use this to explain why the system produced the result it did.

One example of a counterfactual would be to ask what the world might be like had the internet never been developed. Shutterstock

Suppose the bank feeds its AI system different (manipulated) information about Sara. From this, the bank works out the smallest change Sara would need to get a positive outcome would be to increase her income.

The bank can then apparently use this as an explanation: Sara’s loan was denied because her income was too low. Had her income been higher, she would have been granted a loan.

Such counterfactual explanations[11] are being seriously considered[12] as a way of satisfying the demand for explainable AI, including in cases of loan applications and using AI to make scientific discoveries[13].

However, as researchers have argued, the counterfactual approach is inadequate[14].

Correlation and explanation

When we consider changes to the inputs of an AI system and how they translate into outputs, we manage to gather information about correlations. But, as the old adage goes, correlation is not causation.

The reason that’s a problem is because work in philosophy suggests causation is tightly connected to explanation[15]. To explain why an event occurred, we need to know what caused it.

On this basis, it may be a mistake for the bank to tell Sara her loan was denied because her income was too low. All it can really say with confidence is that income and credit score are correlated – and Sara is still left without an explanation for her poor result.

What’s needed is a way to turn information about counterfactuals and correlations into explanatory information.

The future of explainable AI

With time we can expect AI to be used more for hiring decisions, visa applications, promotions and state and federal funding decisions, among other things.

A lack of explanation for these decisions threatens to substantially increase the injustice people will experience. After all, without explanations we can’t correct mistakes made when using AI. Fortunately, philosophy can help.

Explanation has been a central topic of philosophical study[16] over the last century. Philosophers have designed a range of methods for extracting explanatory information from a sea of correlations, and have developed sophisticated theories about how explanation works.

A great deal of this work has focused on the relationship between counterfactuals and explanation. I’ve developed work on this[17] myself. By drawing on philosophical insights, we may be able to develop better approaches to explainable AI.

At present, however, there’s not enough overlap between philosophy and computer science on this topic. If we want to tackle injustice head-on, we’ll need a more integrated approach that combines work in these fields.

Read more: When self-driving cars crash, who's responsible? Courts and insurers need to know what's inside the 'black box'[18]

References

  1. ^ loan decisions by banks (www.afr.com)
  2. ^ medical diagnoses (www.healthcareoutlook.net)
  3. ^ criminal’s likelihood of re-offending (www.technologyreview.com)
  4. ^ General Data Protection Regulation (gdpr.eu)
  5. ^ Robodebt case in Australia (theconversation.com)
  6. ^ royal commission (www.abc.net.au)
  7. ^ Not everything we call AI is actually 'artificial intelligence'. Here's what you need to know (theconversation.com)
  8. ^ they will encode bias (hbr.org)
  9. ^ Everyone's having a field day with ChatGPT – but nobody knows how it actually works (theconversation.com)
  10. ^ two types of approaches (news.mit.edu)
  11. ^ counterfactual explanations (jolt.law.harvard.edu)
  12. ^ seriously considered (www.abc.net.au)
  13. ^ scientific discoveries (www.chemistryworld.com)
  14. ^ inadequate (arxiv.org)
  15. ^ tightly connected to explanation (www.jstor.org)
  16. ^ topic of philosophical study (plato.stanford.edu)
  17. ^ work on this (quod.lib.umich.edu)
  18. ^ When self-driving cars crash, who's responsible? Courts and insurers need to know what's inside the 'black box' (theconversation.com)

Read more https://theconversation.com/philosophers-have-studied-counterfactuals-for-decades-will-they-help-us-unlock-the-mysteries-of-ai-196392

The Times Features

Itinerary to Maximize Your Two-Week Adventure in Vietnam and Cambodia

Two weeks may not seem like much, but it’s just the right time for travelers to explore the best of Vietnam and Cambodia. From the bustling streets of Hanoi to the magnificent te...

How to Protect Your Garden Trees from Wind Damage in Australia

In Australia's expansive landscape, garden trees hold noteworthy significance. They not only enhance the aesthetic appeal of our homes but also play an integral role in the local...

Brisbane Homeowners Warned: Non-Compliant Flexible Hoses Pose High Flood Risk

As a homeowner in Brisbane, when you think of the potential for flood damage to your home, you probably think of weather events. But you should know that there may be a tickin...

Argan Oil-Infused Moroccanoil Shampoo: Nourish and Revitalize Your Hair

Are you ready to transform your hair from dull and lifeless to vibrant and full of life? Look no further than the luxurious embrace of Argan Oil-Infused Moroccanoil Shampoo! In a...

Building A Strong Foundation For Any Structure

Building a home or commercial building can be very exciting. The possibilities are endless and the future is interesting. You can always change aspects of the building to meet the ...

The Role of a Family Dentist: Why Every Household Needs One

source A family dentist isn’t like your regular dentist who may specialise in a particular age group and whom you visit only when something goes wrong. A family dentist takes proa...

Times Magazine

"Eternal Nurture" by Cara Barilla: A Timeless Collection of Wisdom and Healing

Renowned Sydney-born author and educator Cara Barilla has released her latest book, Eternal Nurture, a profound collection of inspirational quotes designed to support mindfulness, emotional healing, and personal growth. With a deep commitment to ...

How AI-Driven SEO Enhancements Can Improve Headless CMS Content Visibility

Whereas SEO (search engine optimization) is critical in the digital landscape for making connections to content, much of it is still done manually keyword research, metatags, final tweaks at publication requiring a human element that takes extensiv...

Crypto Expert John Fenga Reveals How Blockchain is Revolutionising Charity

One of the most persistent challenges in the charity sector is trust. Donors often wonder whether their contributions are being used effectively or if overhead costs consume a significant portion. Traditional fundraising methods can be opaque, with...

Navigating Parenting Arrangements in Australia: A Legal Guide for Parents

Understanding Parenting Arrangements in Australia. Child custody disputes are often one of the most emotionally charged aspects of separation or divorce. Parents naturally want what is best for their children, but the legal process of determining ...

Blocky Adventures: A Minecraft Movie Celebration for Your Wrist

The Minecraft movie is almost here—and it’s time to get excited! With the film set to hit theaters on April 4, 2025, fans have a brand-new reason to celebrate. To honor the upcoming blockbuster, watchfaces.co has released a special Minecraft-inspir...

The Ultimate Guide to Apple Watch Faces & Trending Wallpapers

In today’s digital world, personalization is everything. Your smartwatch isn’t just a timepiece—it’s an extension of your style. Thanks to innovative third-party developers, customizing your Apple Watch has reached new heights with stunning designs...

LayBy Shopping