The Times Australia
The Times World News

.

Snapchat's 'creepy' AI blunder reminds us that chatbots aren't people. But as the lines blur, the risks grow

  • Written by Daswin de Silva, Deputy Director of the Centre for Data Analytics and Cognition, La Trobe University
Snapchat's 'creepy' AI blunder reminds us that chatbots aren't people. But as the lines blur, the risks grow

Artificial intelligence-powered (AI) chatbots are becoming increasingly human-like by design, to the point that some among us may struggle to distinguish between human and machine.

This week, Snapchat’s My AI chatbot glitched and posted a story of what looked like a wall and ceiling, before it stopped responding to users. Naturally, the internet began to question[1] whether the ChatGPT-powered chatbot had gained sentience.

A crash course in AI literacy could have quelled this confusion. But, beyond that, the incident reminds us that as AI chatbots grow closer to resembling humans, managing their uptake will only get more challenging – and more important.

From rules-based to adaptive chatbots

Since ChatGPT burst onto our screens late last year, many digital platforms have integrated AI into their services. Even as I draft this article on Microsoft Word, the software’s predictive AI capability is suggesting possible sentence completions.

Read more: Google and Microsoft are bringing AI to Word, Excel, Gmail and more. It could boost productivity for us – and cybercriminals[2]

Known as generative AI, this relatively new type of AI is distinguished from its predecessors[3] by its ability to generate new content that is precise, human-like and seemingly meaningful.

Generative AI tools, including AI image generators and chatbots, are built on large language models (LLMs). These computational models analyse the associations between billions of words, sentences and paragraphs to predict what ought to come next in a given text. As OpenAI co-founder Ilya Sutskever puts it[4], an LLM is

[…] just a really, really good next-word predictor.

Advanced LLMs are also fine-tuned with human feedback. This training, often delivered through countless hours of cheap human labour, is the reason AI chatbots can now have seemingly human-like conversations.

OpenAI’s ChatGPT is still the flagship generative AI model[5]. Its release marked a major leap from simpler “rules-based” chatbots, such as those used in online customer service.

Human-like chatbots that talk to a user rather than at them have been linked with higher levels of engagement. One study[6] found the personification of chatbots leads to increased engagement which, over time, may turn into psychological dependence. Another study involving stressed participants[7] found a human-like chatbot was more likely to be perceived as competent, and therefore more likely to help reduce participants’ stress.

These chatbots have also been effective in fulfilling organisational objectives in various settings, including retail, education, workplace and healthcare settings[8].

Read more: The hidden cost of the AI boom: social and environmental exploitation[9]

Google is using generative AI to build a “personal life coach” that will supposedly help[10] people with various personal and professional tasks, including providing life advice and answering intimate questions.

This is despite Google’s own AI safety experts warning that users could grow too dependant on AI and may experience “diminished health and wellbeing” and a “loss of agency” if they take life advice from it.

Friend or foe – or just a bot?

In the recent Snapchat incident, the company put the whole thing down to a “temporary outage[11]”. We may never know what actually happened; it could be yet another example of AI “hallucinatng”, or the result of a cyberattack, or even just an operational error.

Either way, the speed with which some users assumed the chatbot had achieved sentience suggests we are seeing an unprecedented anthropomorphism of AI. It’s compounded by a lack of transparency from developers, and a lack of basic understanding among the public.

We shouldn’t underestimate how individuals may be misled by the apparent authenticity of human-like chatbots.

Earlier this year, a Belgian man’s suicide was attributed[12] to conversations he’d had with a chatbot about climate inaction and the planet’s future. In another example, a chatbot named Tessa was found to be[13] offering harmful advice to people through an eating disorder helpline.

Chatbots may be particularly harmful to the more vulnerable among us, and especially to those with psychological conditions.

A new uncanny valley?

You may have heard of the “uncanny valley” effect. It refers to that uneasy feeling you get when you see a humanoid robot that almost looks human, but its slight imperfections give it away, and it ends up being creepy.

It seems a similar experience is emerging in our interactions with human-like chatbots. A slight blip[14] can raise the hairs on the back of the neck.

One solution might be to lose the human edge and revert to chatbots that are straightforward, objective and factual. But this would come at the expense of engagement and innovation.

Education and transparency are key

Even the developers of advanced AI chatbots often can’t explain how they work. Yet in some ways (and as far as commercial entities are concerned) the benefits outweigh the risks.

Generative AI has demonstrated its usefulness[15] in big-ticket items such as productivity, healthcare, education and even social equity[16]. It’s unlikely to go away. So how do we make it work for us?

Since 2018, there has been a significant push for governments and organisations to address the risks of AI. But applying responsible standards and regulations[17] to a technology that’s more “human-like” than any other comes with a host of challenges.

Currently, there is no legal requirement for Australian businesses to disclose the use of chatbots. In the US, California has introduced a “bot bill” that would require this, but legal experts have poked holes in it[18] – and the bill has yet to be enforced at the time of writing this article.

Moreover, ChatGPT and similar chatbots are made public as “research previews[19]”. This means they often come with multiple disclosures on their prototypical nature, and the onus for responsible use falls on the user.

The European Union’s AI Act[20], the world’s first comprehensive regulation on AI, has identified moderate regulation and education as the path forward – since excess regulation could stunt innovation. Similar to digital literacy, AI literacy should be mandated in schools, universities and organisations, and should also be made free and accessible for the public.

Read more: Do we need a new law for AI? Sure – but first we could try enforcing the laws we already have[21]

References

  1. ^ question (9to5mac.com)
  2. ^ Google and Microsoft are bringing AI to Word, Excel, Gmail and more. It could boost productivity for us – and cybercriminals (theconversation.com)
  3. ^ predecessors (www.timeshighereducation.com)
  4. ^ puts it (lifearchitect.ai)
  5. ^ flagship generative AI model (www.reuters.com)
  6. ^ study (www.ingentaconnect.com)
  7. ^ stressed participants (dl.acm.org)
  8. ^ healthcare settings (www.latrobe.edu.au)
  9. ^ The hidden cost of the AI boom: social and environmental exploitation (theconversation.com)
  10. ^ supposedly help (www.nytimes.com)
  11. ^ temporary outage (techcrunch.com)
  12. ^ was attributed (www.livemint.com)
  13. ^ was found to be (www.theguardian.com)
  14. ^ slight blip (www.newscientist.com)
  15. ^ demonstrated its usefulness (www.gatesnotes.com)
  16. ^ even social equity (theconversation.com)
  17. ^ responsible standards and regulations (www.itu.int)
  18. ^ poked holes in it (www.wired.com)
  19. ^ research previews (openai.com)
  20. ^ European Union’s AI Act (www.europarl.europa.eu)
  21. ^ Do we need a new law for AI? Sure – but first we could try enforcing the laws we already have (theconversation.com)

Read more https://theconversation.com/snapchats-creepy-ai-blunder-reminds-us-that-chatbots-arent-people-but-as-the-lines-blur-the-risks-grow-211744

Times Magazine

Headless CMS in Digital Twins and 3D Product Experiences

Image by freepik As the metaverse becomes more advanced and accessible, it's clear that multiple sectors will use digital twins and 3D product experiences to visualize, connect, and streamline efforts better. A digital twin is a virtual replica of ...

The Decline of Hyper-Casual: How Mid-Core Mobile Games Took Over in 2025

In recent years, the mobile gaming landscape has undergone a significant transformation, with mid-core mobile games emerging as the dominant force in app stores by 2025. This shift is underpinned by changing user habits and evolving monetization tr...

Understanding ITIL 4 and PRINCE2 Project Management Synergy

Key Highlights ITIL 4 focuses on IT service management, emphasising continual improvement and value creation through modern digital transformation approaches. PRINCE2 project management supports systematic planning and execution of projects wit...

What AI Adoption Means for the Future of Workplace Risk Management

Image by freepik As industrial operations become more complex and fast-paced, the risks faced by workers and employers alike continue to grow. Traditional safety models—reliant on manual oversight, reactive investigations, and standardised checklist...

From Beach Bops to Alpine Anthems: Your Sonos Survival Guide for a Long Weekend Escape

Alright, fellow adventurers and relaxation enthusiasts! So, you've packed your bags, charged your devices, and mentally prepared for that glorious King's Birthday long weekend. But hold on, are you really ready? Because a true long weekend warrior kn...

Effective Commercial Pest Control Solutions for a Safer Workplace

Keeping a workplace clean, safe, and free from pests is essential for maintaining productivity, protecting employee health, and upholding a company's reputation. Pests pose health risks, can cause structural damage, and can lead to serious legal an...

The Times Features

Duke of Dural to Get Rooftop Bar as New Owners Invest in Venue Upgrade

The Duke of Dural, in Sydney’s north-west, is set for a major uplift under new ownership, following its acquisition by hospitality group Good Beer Company this week. Led by resp...

Prefab’s Second Life: Why Australia’s Backyard Boom Needs a Circular Makeover

The humble granny flat is being reimagined not just as a fix for housing shortages, but as a cornerstone of circular, factory-built architecture. But are our systems ready to s...

Melbourne’s Burglary Boom: Break-Ins Surge Nearly 25%

Victorian homeowners are being warned to act now, as rising break-ins and falling arrest rates paint a worrying picture for suburban safety. Melbourne residents are facing an ...

Exploring the Curriculum at a Modern Junior School in Melbourne

Key Highlights The curriculum at junior schools emphasises whole-person development, catering to children’s physical, emotional, and intellectual needs. It ensures early year...

Distressed by all the bad news? Here’s how to stay informed but still look after yourself

If you’re feeling like the news is particularly bad at the moment, you’re not alone. But many of us can’t look away – and don’t want to. Engaging with news can help us make ...

The Role of Your GP in Creating a Chronic Disease Management Plan That Works

Living with a long-term condition, whether that is diabetes, asthma, arthritis or heart disease, means making hundreds of small decisions every day. You plan your diet against m...