The Times Australia
The Times World News

.
Men's Weekly

.

AI is creating fake legal cases and making its way into real courtrooms, with disastrous results

  • Written by Michael Legg, Professor of Law, UNSW Sydney & Vicki McNamara, Senior Research Associate, Centre for the Future of the Legal Profession
AI is creating fake legal cases and making its way into real courtrooms, with disastrous results

We’ve seen deepfake, explicit images of celebrities[1], created by artificial intelligence (AI). AI has also played a hand in creating music[2], driverless race cars[3] and spreading misinformation[4], among other things.

It’s hardly surprising, then, that AI also has a strong impact on our legal systems.

It’s well known that courts must decide disputes based on the law, which is presented by lawyers to the court as part of a client’s case. It’s therefore highly concerning that fake law, invented by AI, is being used in legal disputes.

Not only does this pose issues of legality and ethics, it also threatens to undermine faith and trust in global legal systems.

Read more: Lawyers are rapidly embracing AI: here's how to avoid an ethical disaster[5]

How do fake laws come about?

There is little doubt that generative AI is a powerful tool with transformative potential for society, including many aspects of the legal system. But its use comes with responsibilities and risks.

Lawyers are trained to carefully apply professional knowledge and experience, and are generally not big risk-takers. However, some unwary lawyers (and self-represented[6] litigants) have been caught out by artificial intelligence.

ChatGPT on a smartphone screen in front of the same website on a laptop screen
Generative AI tools, like ChatGPT, can provide incorrect information. Shutterstock[7]

AI models are trained on massive data sets. When prompted by a user, they can create new content (both text and audiovisual).

Although content generated this way can look very convincing, it can also be inaccurate. This is the result of the AI model attempting to “fill in the gaps” when its training data is inadequate or flawed, and is commonly referred to as “hallucination[8]”.

In some contexts, generative AI hallucination is not a problem. Indeed, it can be seen as an example of creativity.

But if AI hallucinated or created inaccurate content that is then used in legal processes, that’s a problem – particularly when combined with time pressures on lawyers and a lack of access to legal services for many.

This potent combination can result in carelessness and shortcuts in legal research and document preparation, potentially creating reputational issues for the legal profession and a lack of public trust in the administration of justice.

It’s happening already

The best known generative AI “fake case” is the 2023 US case Mata v Avianca[9], in which lawyers submitted a brief containing fake extracts and case citations to a New York court. The brief was researched using ChatGPT.

The lawyers, unaware that ChatGPT can hallucinate, failed to check that the cases actually existed. The consequences were disastrous. Once the error was uncovered, the court dismissed their client’s case, sanctioned the lawyers for acting in bad faith, fined them and their firm, and exposed their actions to public scrutiny.

Read more: AI is everywhere – including countless applications you've likely never heard of[10]

Despite adverse publicity, other fake case examples continue to surface. Michael Cohen, Donald Trump’s former lawyer, gave his own lawyer cases generated by Google Bard, another generative AI chatbot. He believed they were real (they were not) and that his lawyer would fact check them (he did not). His lawyer included the cases[11] in a brief filed with the US Federal Court.

Fake cases have also surfaced in recent matters in Canada[12] and the United Kingdom[13].

If this trend goes unchecked, how can we ensure that the careless use of generative AI does not undermine the public’s trust in the legal system? Consistent failures by lawyers to exercise due care when using these tools has the potential to mislead and congest the courts, harm clients’ interests, and generally undermine the rule of law.

A man in a suit leaves a courtroom Michael Cohen’s lawyer was caught up in a court case involving fake AI case law. Sarah Yenesel/EPA

What’s being done about it?

Around the world, legal regulators and courts have responded in various ways.

Several US state bars and courts have issued guidance, opinions or orders on generative AI use, ranging from responsible adoption to an outright ban.

Law societies in the UK and British Columbia, and the courts of New Zealand, have also developed guidelines.

In Australia, the NSW Bar Association has a generative AI guide[14] for barristers. The Law Society of NSW[15] and the Law Institute of Victoria[16] have released articles on responsible use in line with solicitors’ conduct rules.

Many lawyers and judges, like the public, will have some understanding of generative AI and can recognise both its limits and benefits. But there are others who may not be as aware. Guidance undoubtedly helps.

But a mandatory approach is needed. Lawyers who use generative AI tools cannot treat it as a substitute for exercising their own judgement and diligence, and must check the accuracy and reliability of the information they receive.

Read more: Do you trust AI to write the news? It already is – and not without issues[17]

In Australia, courts should adopt practice notes or rules that set out expectations when generative AI is used in litigation. Court rules can also guide self-represented litigants, and would communicate to the public that our courts are aware of the problem and are addressing it.

The legal profession could also adopt formal guidance to promote the responsible use of AI by lawyers. At the very least, technology competence should become a requirement of lawyers’ continuing legal education in Australia.

Setting clear requirements for the responsible and ethical use of generative AI by lawyers in Australia will encourage appropriate adoption and shore up public confidence in our lawyers, our courts, and the overall administration of justice in this country.

References

  1. ^ celebrities (www.nytimes.com)
  2. ^ creating music (theconversation.com)
  3. ^ driverless race cars (theconversation.com)
  4. ^ misinformation (theconversation.com)
  5. ^ Lawyers are rapidly embracing AI: here's how to avoid an ethical disaster (theconversation.com)
  6. ^ self-represented (reason.com)
  7. ^ Shutterstock (www.shutterstock.com)
  8. ^ hallucination (www.csiro.au)
  9. ^ Mata v Avianca (law.justia.com)
  10. ^ AI is everywhere – including countless applications you've likely never heard of (theconversation.com)
  11. ^ included the cases (www.reuters.com)
  12. ^ Canada (www.cbc.ca)
  13. ^ the United Kingdom (www.legalfutures.co.uk)
  14. ^ generative AI guide (inbrief.nswbar.asn.au)
  15. ^ Law Society of NSW (lsj.com.au)
  16. ^ Law Institute of Victoria (www.liv.asn.au)
  17. ^ Do you trust AI to write the news? It already is – and not without issues (theconversation.com)

Read more https://theconversation.com/ai-is-creating-fake-legal-cases-and-making-its-way-into-real-courtrooms-with-disastrous-results-225080

Times Magazine

From Beach Bops to Alpine Anthems: Your Sonos Survival Guide for a Long Weekend Escape

Alright, fellow adventurers and relaxation enthusiasts! So, you've packed your bags, charged your devices, and mentally prepared for that glorious King's Birthday long weekend. But hold on, are you really ready? Because a true long weekend warrior kn...

Effective Commercial Pest Control Solutions for a Safer Workplace

Keeping a workplace clean, safe, and free from pests is essential for maintaining productivity, protecting employee health, and upholding a company's reputation. Pests pose health risks, can cause structural damage, and can lead to serious legal an...

The Science Behind Reverse Osmosis and Why It Matters

What is reverse osmosis? Reverse osmosis (RO) is a water purification process that removes contaminants by forcing water through a semi-permeable membrane. This membrane allows only water molecules to pass through while blocking impurities such as...

Foodbank Queensland celebrates local hero for National Volunteer Week

Stephen Carey is a bit bananas.   He splits his time between his insurance broker business, caring for his young family, and volunteering for Foodbank Queensland one day a week. He’s even run the Bridge to Brisbane in a banana suit to raise mon...

Senior of the Year Nominations Open

The Allan Labor Government is encouraging all Victorians to recognise the valuable contributions of older members of our community by nominating them for the 2025 Victorian Senior of the Year Awards.  Minister for Ageing Ingrid Stitt today annou...

CNC Machining Meets Stage Design - Black Swan State Theatre Company & Tommotek

When artistry meets precision engineering, incredible things happen. That’s exactly what unfolded when Tommotek worked alongside the Black Swan State Theatre Company on several of their innovative stage productions. With tight deadlines and intrica...

The Times Features

Running Across Australia: What Really Holds the Body Together?

How William Goodge’s 3,800km run reveals the connection between movement, mindset, and mental resilience As a business owner, I’ve come to realise that the biggest wins rarely com...

Telehealth is Transforming Healthcare Services in Australia

It has traditionally not been easy to access timely healthcare in Australia, particularly for people who live in remote areas. Many of them spend hours on the road just to see a...

Launchd Acquires Huume, Strengthening Creative Firepower Across Talent-Led Marketing

Launchd, a leader in talent, technology and brand partnerships, has announced its acquisition of influencer talent management agency Huume from IZEA. The move comes as the medi...

Vietnam's "Gold Coast" Emerges as Extraordinary Investment Frontier and Australian Inspired Way of Life

$2 Billion super-city in Vung Tau set to replicate Australia's Gold Coast success story A culturally metamorphic development aptly named "Gold Coast" is set to reshape Vietna...

Choosing the Wrong Agent Is the #1 Regret Among Aussie Property Sellers

Selling your home is often one of the largest financial transactions you’ll make, and for many Australians, it’s also one of the most emotional. A new survey of Australian home se...

Travel Insurance for Families: What Does it Cover and Why it’s Essential

Planning a family trip is exciting, but unexpected mishaps can turn your dream vacation into a stressful ordeal. That’s where travel insurance comes in—it’s your safety net when ...