The Times Australia
The Times World News

.

The unspoken rule of conversation that explains why AI chatbots feel so human

  • Written by Celeste Rodriguez Louro, Chair of Linguistics and Director of Language Lab, The University of Western Australia

Earlier this year, a Hong Kong finance worker was tricked into paying US$25 million to scammers who had used deepfake technology to pretend to be the company’s chief financial officer[1] in a video conference call. Thinking the images on screen were his colleagues, the financier authorised the multi-million dollar transfer to fraudsters posing as friends.

It’s a dramatic example, but the bamboozled office worker was far from alone in being fooled by generative AI. This technology, which relies heavily on large language models trained on vast amounts of data to learn and predict the patterns of language, has become increasingly widespread since the launch of ChatGPT in 2022.

How can we explain why some people who interact with generative AI chatbots are so readily convinced they are having a conversation with a kind of person? The answer may lie in the rules of conversation itself – and how they are deeply ingrained in the way we interact with the world.

Why the hype?

Generative AI systems based on language models produce text that a linguist would call “ungrounded[2]”: it lacks the mutual understanding[3] that is an essential component of conversations between people. However, when humans engage in an interaction we tend to assume that our partner has a mind.

We are wired to anthropomorphise the various non-humans we interact with. Before the recent leaps in generative AI, this was mainly animals.

Now it is machines. Before generative AI, our interactions with computers generally involved minimal linguistic back-and-forth. With generative AI, however, computer systems acknowledge us in a way that seems real. This “seeming real” is a huge part of the technology’s success.

So how does it work?

Language and thought

In a recent paper[4], American scientists Kyle Mahowald, Anna Ivanova and colleagues argue that we are conflating language and thought. Because generative AI systems can use language somewhat fluently, we believe they can think. (For the record, they cannot.)

The authors distinguish between formal linguistic competence (knowledge of rules and statistical patterns in language) and functional linguistic competence (how to use language in real-world situations).

It’s not enough to produce rule-governed utterances. They must also be used appropriately in context. In this example (crafted with the help of ChatGPT-4o when prompted to produce an example of an accurate but inappropriate conversational response), Alex offers a response that, while factually correct, fails to address their friend’s need for support.

Friend: I am freaking out about my presentation tomorrow.

Alex: Stress activates your body’s fight-or-flight response, increasing heart rate and cortisol levels.

Mahowald and colleagues say our belief in the intelligence of generative AI systems comes from their capacity for language. However, a crucial piece of the puzzle is what happens to humans when we interact with the technology.

The rules of conversation

The key to understanding the allure of generative AI chatbots for humans lies in the genre the bots perform: conversation. Conversation is governed by rules and routines.

Conversational routines vary across cultures, and different expectations are in place. In Western cultures, at least, linguists often regard conversation as proceeding according to four principles or “maxims”[5] set out in 1975 by British philosopher of language Paul Grice.

The maxim of quality: be truthful; do not give information that is false or not supported by evidence.

The maxim of quantity: be as informative as is required; don’t give too much or too little information.

The maxim of relevance: only give information that is relevant to the topic under discussion.

The maxim of manner: be clear, brief, and orderly; avoid obscurity and ambiguity.

Finding relevance at all costs

Generative AI chatbots usually do well in terms of quantity (sometimes erring on the side of giving too much information), and they tend to be relevant and clear (a reason people use them to improve their writing).

However, they do often fail on the maxim of quality. They tend to hallucinate, giving answers which may appear authoritative but are in fact false.

The crux of generative AI’s success, however, lies in Grice’s claim that anyone engaged in meaningful communication will abide by these maxims and will assume that others are also following them.

For example, the reason lying works is that people interacting with a liar will assume the other person is telling the truth. People interacting with someone who offers an irrelevant comment will attempt to find relevance at all costs.

Grice’s cooperative principle[6] holds that conversation is underpinned by our overarching will to understand one another.

The will to cooperate

The success of generative AI, then, depends in part on the human need to cooperate in conversation, and to be instinctively drawn to interaction. This way of interacting through conversation, learned in childhood, becomes habitual.

Grice argued that “it would take a good deal of effort to make a radical departure from the habit”.

Next time you engage with generative AI, then, do so with caution. Remember it’s only a language model. Don’t let your habitual need for conversational cooperation accept a machine as a fellow human.

References

  1. ^ pretend to be the company’s chief financial officer (edition.cnn.com)
  2. ^ ungrounded (www.youtube.com)
  3. ^ mutual understanding (www.psychology.stonybrook.edu)
  4. ^ a recent paper (arxiv.org)
  5. ^ four principles or “maxims” (www.ucl.ac.uk)
  6. ^ cooperative principle (www.lancaster.ac.uk)

Read more https://theconversation.com/the-unspoken-rule-of-conversation-that-explains-why-ai-chatbots-feel-so-human-243805

Times Magazine

Why Social Media Marketing Matters for Businesses in Australia

Today social media is a big part of daily life. All over Australia people use Facebook, Instagram, TikTok , LinkedIn and Twitter to stay connected, share updates and find new ideas. For businesses this means a great chance to reach new customers and...

Building an AI-First Culture in Your Company

AI isn't just something to think about anymore - it's becoming part of how we live and work, whether we like it or not. At the office, it definitely helps us move faster. But here's the thing: just using tools like ChatGPT or plugging AI into your wo...

Data Management Isn't Just About Tech—Here’s Why It’s a Human Problem Too

Photo by Kevin Kuby Manuel O. Diaz Jr.We live in a world drowning in data. Every click, swipe, medical scan, and financial transaction generates information, so much that managing it all has become one of the biggest challenges of our digital age. Bu...

Headless CMS in Digital Twins and 3D Product Experiences

Image by freepik As the metaverse becomes more advanced and accessible, it's clear that multiple sectors will use digital twins and 3D product experiences to visualize, connect, and streamline efforts better. A digital twin is a virtual replica of ...

The Decline of Hyper-Casual: How Mid-Core Mobile Games Took Over in 2025

In recent years, the mobile gaming landscape has undergone a significant transformation, with mid-core mobile games emerging as the dominant force in app stores by 2025. This shift is underpinned by changing user habits and evolving monetization tr...

Understanding ITIL 4 and PRINCE2 Project Management Synergy

Key Highlights ITIL 4 focuses on IT service management, emphasising continual improvement and value creation through modern digital transformation approaches. PRINCE2 project management supports systematic planning and execution of projects wit...

The Times Features

Jetstar announces first ever Brisbane to Rarotonga flights with launch fares from just $249^ one-way

Jetstar will start operating direct flights between Brisbane and Rarotonga, the stunning capital island of the Cook Islands, in May 2026, with launch sale fares available today...

Introducing the SE 2 and Mini hair dryers from Laifen

The Mane Attractions for Professional Styling at Home Without the Price Tag Fast, flawless hair is now possible with the launch of Laifen’s two professional quality hair dryers th...

Home Gym Recovery Routines: What Pro Athletes Do After Workouts

Training is only half the equation. What you do after your workout has just as much impact on your progress, performance, and long-term health. Professional athletes know this, w...

Flipping vs. Holding: Which Investment Strategy Is Right for You?

Are you wondering whether flipping a property or holding onto it is the better investment strategy? The answer isn’t one-size-fits-all. Both strategies have distinct advantages a...

Why Everyone's Talking About Sea Moss - And Should You Try It Too?

Sea moss - a humble marine plant that’s been used for centuries - is making a major comeback in modern wellness circles. And it’s not just a trend. With growing interest from athle...

A Guide to Smarter Real Estate Accounting: What You Might Be Overlooking

Real estate accounting can be a complex terrain, even for experienced investors and property managers. From tracking rental income to managing property expenses, the financial in...