The Times Australia
The Times World News

.

AI is already being used in healthcare. But not all of it is 'medical grade'

  • Written by Karin Verspoor, Dean, School of Computing Technologies, RMIT University, RMIT University
AI is already being used in healthcare. But not all of it is 'medical grade'

Artificial intelligence (AI) seems to be everywhere these days, and healthcare is no exception.

There are computer vision tools that can detect suspicious skin lesions[1] as well as a specialist dermatologist can. Other tools can predict coronary artery disease from scans[2]. There are also data-driven robots[3] that guide minimally-invasive surgery.

To precisely diagnose diseases[4] and guide treatment choices[5], AI is used to analyse patients’ genomic and molecular data. For instance, machine learning has been applied to detect Alzheimer’s disease[6] and to help choose the best antidepressant medication[7] for patients with major depression.

Deep learning[8] methods have been used to model electronic health record data to predict health outcomes for patients[9] and provide early estimates of treatment cost[10].

Read more: AI to Z: all the terms you need to know to keep up in the AI hype age[11]

With new language-based generative AI technologies like ChatGPT, the clinical world is abuzz with talk of chatbots for answering patient questions[12], helping doctors take better notes[13], and even explaining a diagnosis[14] to a concerned grandchild.

There is no doubt that in terms of patient health, workflows and system efficiency, AI will benefit the health system.

But there are legitimate concerns about the accuracy of such tools, including how well they work in new settings (such as a different country or even a different hospital from where they were created), and whether they “hallucinate” – or make things up.

An operating room with no people and only several robot arms hovering above a patient
Robot-assisted surgery is already the reality in some technologically equipped hospitals. Shutterstock[15]

Developing ‘medical grade’ tools

In our recent article[16] in the Medical Journal of Australia, we argue using AI effectively in healthcare will require retraining of the workforce, retooling health services, and transforming workflows.

Critically, we also need to collect evidence AI tools are “medical grade” before we use them on patients.

Many claims made by the developers of medical AI may lack appropriate scientific rigour[17] and evaluations of AI tools may suffer from a high risk of bias[18]. This means the tests run to ensure their accuracy are too narrow.

AI tools can make errors, or stop working when the application context changes. Conversational agents such as chatbots may produce misleading medical information that may delay patients seeking care. They may also make inappropriate recommendations[19].

All this means we need standards for the AI tools that impact diagnosis and treatment of patients. Clinicians should be given training on how to critically assess[20] AI applications to understand their readiness for routine care.

We should expect to be able to replicate the results from one context to another, under real-world conditions. For example, a tool developed using historical data from a hospital in New York should be carefully trialled with live patient data in Broome before we trust it.

Randomised controlled trials of AI tools, where these differences are controlled for, would represent a gold standard of evidence for their use.

Read more: AI has potential to revolutionise health care – but we must first confront the risk of algorithmic bias[21]

We can’t just copy what other countries do

It is important to carefully examine how AI tools are embedded into workflows[22] to support clinical decisions. The benefits and risks of a tool will depend on precisely how the human clinician and the tool work together[23].

There’s a view that all we need to do in Australia is adopt the best of what is produced internationally, and that we don’t need deep sovereign capabilities.

Perhaps we can rely on the regulation of AI tools under way through the European Union’s AI Act[24], or the United States Food and Drug Administration’s processes for assessing Software as a Medical Device[25].

Nothing is further from the truth.

AI requires local customisation to support local practices, and to reflect diverse populations or health service differences. We don’t want to just export our clinical datasets and import back the models built with them without adapting to our contexts and workflows. We need to monitor the clinical deployments of AI tools into our settings.

Without some degree of algorithmic sovereignty – the capability to produce or modify AI in Australia – the nation is exposed to new risks and the benefits of the technology will be limited.

Read more: How should Australia capitalise on AI while reducing its risks? It's time to have your say[26]

A roadmap for AI in Australian healthcare

The Australian Alliance for Artificial Intelligence in Healthcare has produced a roadmap[27] for future development.

It identifies gaps in Australia’s capability to translate AI into effective and safe clinical services and provides guidance on key issues such as workforce, industry capability, implementation, regulation, and cybersecurity.

These recommendations offer a path toward an AI-enabled Australian healthcare system capable of delivering personalised and patient-focused healthcare, safely and ethically.

The plan also envisages a vibrant AI industry sector that creates jobs and exports to the world, working side by side with an AI-aware workforce and AI-savvy consumers.

AI has the potential to transform medicine. It can do so by harnessing computational power to discern subtle patterns in complex data spanning biology, images, sensory and experiential data, and more.

With care and strategic investment, innovations in AI will surely benefit clinicians and patients alike. Now is the time to act to ensure Australia is well-placed to benefit from one of the most significant industrial revolutions of our time.

References

  1. ^ detect suspicious skin lesions (doi.org)
  2. ^ from scans (www.ahajournals.org)
  3. ^ data-driven robots (ieeexplore.ieee.org)
  4. ^ precisely diagnose diseases (doi.org)
  5. ^ treatment choices (doi.org)
  6. ^ Alzheimer’s disease (doi.org)
  7. ^ choose the best antidepressant medication (doi.org)
  8. ^ Deep learning (www.ibm.com)
  9. ^ predict health outcomes for patients (doi.org)
  10. ^ early estimates of treatment cost (doi.org)
  11. ^ AI to Z: all the terms you need to know to keep up in the AI hype age (theconversation.com)
  12. ^ chatbots for answering patient questions (dx.doi.org)
  13. ^ take better notes (www.nejm.org)
  14. ^ explaining a diagnosis (unlocked.microsoft.com)
  15. ^ Shutterstock (www.shutterstock.com)
  16. ^ recent article (doi.org)
  17. ^ lack appropriate scientific rigour (doi.org)
  18. ^ high risk of bias (doi.org)
  19. ^ inappropriate recommendations (doi.org)
  20. ^ critically assess (doi.org)
  21. ^ AI has potential to revolutionise health care – but we must first confront the risk of algorithmic bias (theconversation.com)
  22. ^ how AI tools are embedded into workflows (doi.org)
  23. ^ how the human clinician and the tool work together (doi.org)
  24. ^ AI Act (artificialintelligenceact.eu)
  25. ^ Software as a Medical Device (www.fda.gov)
  26. ^ How should Australia capitalise on AI while reducing its risks? It's time to have your say (theconversation.com)
  27. ^ a roadmap (aihealthalliance.org)

Read more https://theconversation.com/ai-is-already-being-used-in-healthcare-but-not-all-of-it-is-medical-grade-207912

Times Magazine

Headless CMS in Digital Twins and 3D Product Experiences

Image by freepik As the metaverse becomes more advanced and accessible, it's clear that multiple sectors will use digital twins and 3D product experiences to visualize, connect, and streamline efforts better. A digital twin is a virtual replica of ...

The Decline of Hyper-Casual: How Mid-Core Mobile Games Took Over in 2025

In recent years, the mobile gaming landscape has undergone a significant transformation, with mid-core mobile games emerging as the dominant force in app stores by 2025. This shift is underpinned by changing user habits and evolving monetization tr...

Understanding ITIL 4 and PRINCE2 Project Management Synergy

Key Highlights ITIL 4 focuses on IT service management, emphasising continual improvement and value creation through modern digital transformation approaches. PRINCE2 project management supports systematic planning and execution of projects wit...

What AI Adoption Means for the Future of Workplace Risk Management

Image by freepik As industrial operations become more complex and fast-paced, the risks faced by workers and employers alike continue to grow. Traditional safety models—reliant on manual oversight, reactive investigations, and standardised checklist...

From Beach Bops to Alpine Anthems: Your Sonos Survival Guide for a Long Weekend Escape

Alright, fellow adventurers and relaxation enthusiasts! So, you've packed your bags, charged your devices, and mentally prepared for that glorious King's Birthday long weekend. But hold on, are you really ready? Because a true long weekend warrior kn...

Effective Commercial Pest Control Solutions for a Safer Workplace

Keeping a workplace clean, safe, and free from pests is essential for maintaining productivity, protecting employee health, and upholding a company's reputation. Pests pose health risks, can cause structural damage, and can lead to serious legal an...

The Times Features

Tricia Paoluccio designer to the stars

The Case for Nuturing Creativity in the Classroom, and in our Lives I am an actress and an artist who has had the privilege of sharing my work across many countries, touring my ...

Duke of Dural to Get Rooftop Bar as New Owners Invest in Venue Upgrade

The Duke of Dural, in Sydney’s north-west, is set for a major uplift under new ownership, following its acquisition by hospitality group Good Beer Company this week. Led by resp...

Prefab’s Second Life: Why Australia’s Backyard Boom Needs a Circular Makeover

The humble granny flat is being reimagined not just as a fix for housing shortages, but as a cornerstone of circular, factory-built architecture. But are our systems ready to s...

Melbourne’s Burglary Boom: Break-Ins Surge Nearly 25%

Victorian homeowners are being warned to act now, as rising break-ins and falling arrest rates paint a worrying picture for suburban safety. Melbourne residents are facing an ...

Exploring the Curriculum at a Modern Junior School in Melbourne

Key Highlights The curriculum at junior schools emphasises whole-person development, catering to children’s physical, emotional, and intellectual needs. It ensures early year...

Distressed by all the bad news? Here’s how to stay informed but still look after yourself

If you’re feeling like the news is particularly bad at the moment, you’re not alone. But many of us can’t look away – and don’t want to. Engaging with news can help us make ...