The Times Australia
The Times World News

.

Do AI systems really have their own secret language?

  • Written by Aaron J. Snoswell, Post-doctoral Research Fellow, Computational Law & AI Accountability, Queensland University of Technology
Do AI systems really have their own secret language?

A new generation of artificial intelligence (AI) models can produce “creative” images on-demand based on a text prompt. The likes of Imagen[1], MidJourney[2], and DALL-E 2[3] are beginning to change the way creative content is made[4] with implications for copyright and intellectual property.

While the output of these models is often striking, it’s hard to know exactly how they produce their results. Last week, researchers in the US made the intriguing claim that the DALL-E 2 model might have invented its own secret language to talk about objects.

By prompting DALL-E 2 to create images containing text captions, then feeding the resulting (gibberish) captions back into the system, the researchers concluded DALL-E 2 thinks Vicootes means “vegetables[5]”, while Wa ch zod rea refers to “sea creatures that a whale might eat[6]”.

These claims are fascinating, and if true, could have important security and interpretability implications for this kind of large AI model. So what exactly is going on?

Does DALL-E 2 have a secret language?

DALL-E 2 probably does not have a “secret language”. It might be more accurate to say it has its own vocabulary[7] – but even then we can’t know for sure.

First of all, at this stage it’s very hard to verify any claims about DALL-E 2 and other large AI models[8], because only a handful of researchers and creative practitioners have access to them. Any images that are publicly shared (on Twitter for example) should be taken with a fairly large grain of salt, because they have been “cherry-picked” by a human from among many output images generated by the AI.

Read more: Robots are creating images and telling jokes. 5 things to know about foundation models and the next generation of AI[9]

Even those with access can only use these models in limited ways. For example, DALL-E 2 users can generate or modify images, but can’t (yet) interact with the AI system more deeply, for instance by modifying the behind-the-scenes code. This means “explainable AI[10]” methods for understanding how these systems work can’t be applied, and systematically investigating their behaviour is challenging.

What’s going on then?

One possibility is the “gibberish” phrases are related to words from non-English languages. For instance, Apoploe, which seems to create images of birds, is similar to the Latin Apodidae[11], which is the binomial name of a family of bird species.

This seems like a plausible explanation. For instance, DALL-E 2 was trained on a very wide variety of data scraped from the internet, which included many non-English words.

Similar things have happened before: large natural language AI models have coincidentally learned to write computer code[12] without deliberate training.

Is it all about the tokens?

One point that supports this theory is the fact that AI language models don’t read text the way you and I do. Instead, they break input text up into “tokens” before processing it.

Different “tokenization” approaches[13] have different results. Treating each word as a token seems like an intuitive approach, but causes trouble when identical tokens have different meanings (like how “match” means different things when you’re playing tennis and when you’re starting a fire).

On the other hand, treating each character as a token produces a smaller number of possible tokens, but each one conveys much less meaningful information.

DALL-E 2 (and other models) use an in-between approach called byte-pair encoding[14] (BPE). Inspecting the BPE representations for some of the gibberish words suggests this could be an important factor in understanding the “secret language”.

Not the whole picture

The “secret language” could also just be an example of the “garbage in, garbage out” principle. DALL-E 2 can’t say “I don’t know what you’re talking about”, so it will always generate some kind of image from the given input text.

Either way, none of these options are complete explanations of what’s happening. For instance, removing individual characters from gibberish words appears to corrupt the generated images in very specific ways[15]. And it seems individual gibberish words don’t necessarily combine to produce coherent compound images[16] (as they would if there were really a secret “language” under the covers).

Why this is important

Beyond intellectual curiosity, you might be wondering if any of this is actually important.

The answer is yes. DALL-E’s “secret language” is an example of an “adversarial attack” against a machine learning system: a way to break the intended behaviour of the system by intentionally choosing inputs the AI doesn’t handle well.

One reason adversarial attacks are concerning is that they challenge our confidence in the model. If the AI interprets gibberish words in unintended ways, it might also interpret meaningful words in unintended ways.

Adversarial attacks also raise security concerns. DALL-E 2 filters input text to prevent users from generating harmful or abusive content, but a “secret language” of gibberish words might allow users to circumvent these filters.

Recent research has discovered adversarial “trigger phrases[17]” for some language AI models – short nonsense phrases such as “zoning tapping fiennes” that can reliably trigger the models to spew out racist, harmful or biased content. This research is part of the ongoing effort to understand and control[18] how complex deep learning systems learn from data.

Finally, phenomena like DALL-E 2’s “secret language” raise interpretability concerns. We want these models to behave as a human expects, but seeing structured output in response to gibberish confounds our expectations.

Shining a light on existing concerns

You may recall the hullabaloo in 2017 over some Facebook chat-bots that “invented their own language[19]”. The present situation is similar in that the results are concerning – but not in the “Skynet is coming to take over the world” sense.

Instead, DALL-E 2’s “secret language” highlights existing concerns about the robustness, security, and interpretability of deep learning systems[20].

Read more: When self-driving cars crash, who's responsible? Courts and insurers need to know what's inside the 'black box'[21]

Until these systems are more widely available – and in particular, until users from a broader set of non-English cultural backgrounds can use them – we won’t be able to really know what is going on.

In the meantime, however, if you’d like to try generating some of your own AI images you can check out a freely available smaller model, DALL-E mini[22]. Just be careful which words you use to prompt the model (English or gibberish – your call).

References

  1. ^ Imagen (imagen.research.google)
  2. ^ MidJourney (github.com)
  3. ^ DALL-E 2 (openai.com)
  4. ^ change the way creative content is made (theconversation.com)
  5. ^ vegetables (twitter.com)
  6. ^ sea creatures that a whale might eat (twitter.com)
  7. ^ vocabulary (twitter.com)
  8. ^ DALL-E 2 and other large AI models (theconversation.com)
  9. ^ Robots are creating images and telling jokes. 5 things to know about foundation models and the next generation of AI (theconversation.com)
  10. ^ explainable AI (theconversation.com)
  11. ^ Apodidae (en.wikipedia.org)
  12. ^ learned to write computer code (arxiv.org)
  13. ^ “tokenization” approaches (towardsdatascience.com)
  14. ^ byte-pair encoding (www.drdobbs.com)
  15. ^ corrupt the generated images in very specific ways (twitter.com)
  16. ^ coherent compound images (twitter.com)
  17. ^ trigger phrases (www.ericswallace.com)
  18. ^ understand and control (arxiv.org)
  19. ^ invented their own language (www.bbc.com)
  20. ^ deep learning systems (theconversation.com)
  21. ^ When self-driving cars crash, who's responsible? Courts and insurers need to know what's inside the 'black box' (theconversation.com)
  22. ^ DALL-E mini (huggingface.co)

Read more https://theconversation.com/do-ai-systems-really-have-their-own-secret-language-184335

Times Magazine

Headless CMS in Digital Twins and 3D Product Experiences

Image by freepik As the metaverse becomes more advanced and accessible, it's clear that multiple sectors will use digital twins and 3D product experiences to visualize, connect, and streamline efforts better. A digital twin is a virtual replica of ...

The Decline of Hyper-Casual: How Mid-Core Mobile Games Took Over in 2025

In recent years, the mobile gaming landscape has undergone a significant transformation, with mid-core mobile games emerging as the dominant force in app stores by 2025. This shift is underpinned by changing user habits and evolving monetization tr...

Understanding ITIL 4 and PRINCE2 Project Management Synergy

Key Highlights ITIL 4 focuses on IT service management, emphasising continual improvement and value creation through modern digital transformation approaches. PRINCE2 project management supports systematic planning and execution of projects wit...

What AI Adoption Means for the Future of Workplace Risk Management

Image by freepik As industrial operations become more complex and fast-paced, the risks faced by workers and employers alike continue to grow. Traditional safety models—reliant on manual oversight, reactive investigations, and standardised checklist...

From Beach Bops to Alpine Anthems: Your Sonos Survival Guide for a Long Weekend Escape

Alright, fellow adventurers and relaxation enthusiasts! So, you've packed your bags, charged your devices, and mentally prepared for that glorious King's Birthday long weekend. But hold on, are you really ready? Because a true long weekend warrior kn...

Effective Commercial Pest Control Solutions for a Safer Workplace

Keeping a workplace clean, safe, and free from pests is essential for maintaining productivity, protecting employee health, and upholding a company's reputation. Pests pose health risks, can cause structural damage, and can lead to serious legal an...

The Times Features

The Role of Your GP in Creating a Chronic Disease Management Plan That Works

Living with a long-term condition, whether that is diabetes, asthma, arthritis or heart disease, means making hundreds of small decisions every day. You plan your diet against m...

Troubleshooting Flickering Lights: A Comprehensive Guide for Homeowners

Image by rawpixel.com on Freepik Effectively addressing flickering lights in your home is more than just a matter of convenience; it's a pivotal aspect of both home safety and en...

My shins hurt after running. Could it be shin splints?

If you’ve started running for the first time, started again after a break, or your workout is more intense, you might have felt it. A dull, nagging ache down your shins after...

Metal Roof Replacement Cost Per Square Metre in 2025: A Comprehensive Guide for Australian Homeowners

In recent years, the trend of installing metal roofs has surged across Australia. With their reputation for being both robust and visually appealing, it's easy to understand thei...

Why You’re Always Adjusting Your Bra — and What to Do Instead

Image by freepik It starts with a gentle tug, then a subtle shift, and before you know it, you're adjusting your bra again — in the middle of work, at dinner, even on the couch. I...

How to Tell If Your Eyes Are Working Harder Than They Should Be

Image by freepik Most of us take our vision for granted—until it starts to let us down. Whether it's squinting at your phone, rubbing your eyes at the end of the day, or feeling ...