The Times Australia
The Times World News

.

Does my treatment work? How major medical reviews can be 'gold standard' evidence, yet flawed

  • Written by Gideon Meyerowitz-Katz, PhD Student/Epidemiologist, University of Wollongong
Does my treatment work? How major medical reviews can be 'gold standard' evidence, yet flawed

Medical decision-making is complex. There are often hundreds, if not thousands, of published studies that may impact how to manage your medical condition.

Some studies look at which drug is best in a particular situation, or whether pain is better treated by, say, avoiding exercise or seeing a physio for therapeutic massage.

In this morass of difficult choices, Cochrane reviews[1] stand out as internationally trusted and independent[2]. They are considered the “gold standard” in evidence-based medicine.

They involve teams of researchers looking through all the published academic research on a topic to produce an overall answer on what the best evidence says about different treatments.

However, Cochrane has recently come under fire[3] after a controversial review that looked at whether wearing masks in the community during COVID worked to reduce the spread of respiratory viruses.

Studies like this can raise the question[4] of how useful Cochrane reviews are, particularly for the general public.

Issues with evidence-based medicine

As with any research process, Cochrane reviews are not perfect. And they cannot answer all medical questions.

The entire process – from gathering data based primarily on randomised clinical trials, to reviewing that data and coming to some conclusion about the evidence – was mostly developed in the context of clinical interventions. Randomised trials[5] are a type of medical study where people are given treatments in a controlled, random way, giving a robust estimate of whether the treatment works for the condition that’s being studied.

People regularly question whether this “gold standard” framework deals well with things other than surgery, drugs and the like.

For example, take the mask review mentioned above. Much of the criticism[6] was focused not on the specifics of the included papers, but on the general idea of whether randomised clinical trials are an appropriate way to measure the impact of masks on respiratory disease.

What is the “gold standard” if randomised trials are impossible, unethical, or otherwise inappropriate? For example, if an intervention like vaccination is already proven effective, you can’t ethically randomise people into a group that doesn’t get the treatment.

Read more: Yes, masks reduce the risk of spreading COVID, despite a review saying they don't[7]

This gets at the underlying question of what a Cochrane review is actually there to do. The key aim of aggregating research this way is to filter out the noise and provide the most accurate data on a specific question.

Sometimes, the most honest answer is that we just don’t have enough evidence to make a conclusion.

Doctor in white coat, stethoscope around neck, taking notes from laptop
Sometimes, there is evidence, but not from randomised clinical trials. Shutterstock[8]

In other cases, there is evidence, but not from randomised clinical trials. Then the debate becomes about how much weight to give this evidence, whether and how to include it, and how to draw conclusions based on this data.

This may seem arbitrary, but there are good reasons to be wary of findings based only on observational research. A systematic review of observational trials of hormone replacement therapy led to widespread use in the late 90s for preventative health, until randomised trials showed[9] the therapy had little to no benefit.

This isn’t actually a new problem. Indeed, it’s something Cochrane has been grappling with for years[10].

For example, a recent Cochrane review[11] into vaping to help people quit smoking included quite a few non-randomised trials. These were not given the same weight as randomised research, but did provide support for the central finding of the review.

Read more: Controlled experiments won't tell us which Indigenous health programs are working[12]

Cochrane is OK about being criticised …

There have been many issues raised with Cochrane teams over the years. This includes problems[13] with how reviewers rate trials[14] included in the reviews.

However, the organisation is famously transparent. If you have an issue with a particular review, you can post your comments publicly. I did this[15], sharing my concerns about a review on using the drug ivermectin to treat COVID.

Cochrane is also good at incorporating criticism. It even has a prize[16] for the best criticism of its work.

Read more: The government says NDIS supports should be 'evidence-based' – but can they be?[17]

… even if reviews take time

There’s a reason so many experts trust Cochrane. The occasional controversy aside, Cochrane reviews are generally the most detailed and rigorous summary of the evidence on any question you can find.

This attention to detail comes at a cost. Cochrane reviews are often the final word on a subject, not just because they are so robust, but because they take a very long time[18] to come out.

Cochrane aims to publish reviews within two years. But more than half take longer[19] to complete. Cochrane reviews are also meant to be updated regularly, but many have not been updated for more than five years[20].

Read more: Clinical trials are useful – here's how we can ensure they stay so[21]

In a nutshell

Cochrane reviews can be flawed, cannot answer all medical questions and, while comprehensive, can take long to complete.

But there’s a reason that these reviews are considered the gold standard in medical research. They are detailed, lengthy, and very impressive pieces of work.

With more than 9,000[22] Cochrane reviews so far, these are still usually the best evidence we have to answer a range of medical questions.

References

  1. ^ Cochrane reviews (community.cochrane.org)
  2. ^ independent (www.cochrane.org)
  3. ^ come under fire (theconversation.com)
  4. ^ raise the question (www.mdlinx.com)
  5. ^ Randomised trials (theconversation.com)
  6. ^ Much of the criticism (theconversation.com)
  7. ^ Yes, masks reduce the risk of spreading COVID, despite a review saying they don't (theconversation.com)
  8. ^ Shutterstock (www.shutterstock.com)
  9. ^ showed (www.annualreviews.org)
  10. ^ grappling with for years (training.cochrane.org)
  11. ^ a recent Cochrane review (www.cochranelibrary.com)
  12. ^ Controlled experiments won't tell us which Indigenous health programs are working (theconversation.com)
  13. ^ problems (www.sciencedirect.com)
  14. ^ rate trials (journals.plos.org)
  15. ^ I did this (www.cochranelibrary.com)
  16. ^ a prize (community.cochrane.org)
  17. ^ The government says NDIS supports should be 'evidence-based' – but can they be? (theconversation.com)
  18. ^ very long time (www.mdlinx.com)
  19. ^ longer (pubmed.ncbi.nlm.nih.gov)
  20. ^ more than five years (europepmc.org)
  21. ^ Clinical trials are useful – here's how we can ensure they stay so (theconversation.com)
  22. ^ more than 9,000 (www.cochranelibrary.com)

Read more https://theconversation.com/does-my-treatment-work-how-major-medical-reviews-can-be-gold-standard-evidence-yet-flawed-205014

Times Magazine

Headless CMS in Digital Twins and 3D Product Experiences

Image by freepik As the metaverse becomes more advanced and accessible, it's clear that multiple sectors will use digital twins and 3D product experiences to visualize, connect, and streamline efforts better. A digital twin is a virtual replica of ...

The Decline of Hyper-Casual: How Mid-Core Mobile Games Took Over in 2025

In recent years, the mobile gaming landscape has undergone a significant transformation, with mid-core mobile games emerging as the dominant force in app stores by 2025. This shift is underpinned by changing user habits and evolving monetization tr...

Understanding ITIL 4 and PRINCE2 Project Management Synergy

Key Highlights ITIL 4 focuses on IT service management, emphasising continual improvement and value creation through modern digital transformation approaches. PRINCE2 project management supports systematic planning and execution of projects wit...

What AI Adoption Means for the Future of Workplace Risk Management

Image by freepik As industrial operations become more complex and fast-paced, the risks faced by workers and employers alike continue to grow. Traditional safety models—reliant on manual oversight, reactive investigations, and standardised checklist...

From Beach Bops to Alpine Anthems: Your Sonos Survival Guide for a Long Weekend Escape

Alright, fellow adventurers and relaxation enthusiasts! So, you've packed your bags, charged your devices, and mentally prepared for that glorious King's Birthday long weekend. But hold on, are you really ready? Because a true long weekend warrior kn...

Effective Commercial Pest Control Solutions for a Safer Workplace

Keeping a workplace clean, safe, and free from pests is essential for maintaining productivity, protecting employee health, and upholding a company's reputation. Pests pose health risks, can cause structural damage, and can lead to serious legal an...

The Times Features

The Role of Your GP in Creating a Chronic Disease Management Plan That Works

Living with a long-term condition, whether that is diabetes, asthma, arthritis or heart disease, means making hundreds of small decisions every day. You plan your diet against m...

Troubleshooting Flickering Lights: A Comprehensive Guide for Homeowners

Image by rawpixel.com on Freepik Effectively addressing flickering lights in your home is more than just a matter of convenience; it's a pivotal aspect of both home safety and en...

My shins hurt after running. Could it be shin splints?

If you’ve started running for the first time, started again after a break, or your workout is more intense, you might have felt it. A dull, nagging ache down your shins after...

Metal Roof Replacement Cost Per Square Metre in 2025: A Comprehensive Guide for Australian Homeowners

In recent years, the trend of installing metal roofs has surged across Australia. With their reputation for being both robust and visually appealing, it's easy to understand thei...

Why You’re Always Adjusting Your Bra — and What to Do Instead

Image by freepik It starts with a gentle tug, then a subtle shift, and before you know it, you're adjusting your bra again — in the middle of work, at dinner, even on the couch. I...

How to Tell If Your Eyes Are Working Harder Than They Should Be

Image by freepik Most of us take our vision for granted—until it starts to let us down. Whether it's squinting at your phone, rubbing your eyes at the end of the day, or feeling ...