The Times Australia
The Times World News

.
The Times Real Estate

.

Everyone's having a field day with ChatGPT – but nobody knows how it actually works

  • Written by Toby Walsh, Professor of AI at UNSW, Research Group Leader, UNSW Sydney
Everyone's having a field day with ChatGPT – but nobody knows how it actually works

ChatGPT is the latest and most impressive artificially intelligent chatbot yet. It was released two weeks ago, and in just five days hit a million users. It’s being used so much that its servers have reached capacity several times[1].

OpenAI, the company that developed it, is already being discussed as a potential Google slayer. Why look up something on a search engine when ChatGPT can write a whole paragraph explaining the answer? (There’s even a Chrome extension[2] that lets you do both, side by side.)

But what if we never know the secret sauce behind ChatGPT’s capabilities?

The chatbot takes advantage of a number of technical advances published in open scientific literature in the past couple of decades. But any innovations unique to it are secret. OpenAI could well be trying to build a technical and business moat to keep others out.

The ChatPGT site has gone down multiple times due to an influx of users. OpenAI/Screenshot

What it can (and can’t do)

ChatGPT[3] is very capable. Want a haiku on chatbots? Sure.

Chatbots, oh so wise

Endlessly processing

Answers from the void.

How about a joke about chatbots? No problem.

Why did the chatbot cross the road?

To get to the other side – of your screen!

ChatGPT can do many other tricks[4]. It can write computer code[5] to a user’s specification, draft business letters or rental contracts, compose homework essays and even pass university exams.

Just as important is what ChatGPT can’t do. For instance, it struggles to distinguish between truth and falsehood. It is also often a persuasive liar.

What is the fastest marine mammal?

The fastest marine mammal is the sailfish.

The sailfish is not a mammal.

You are correct, the sailfish is not a mammal. I apologize for the mistake. The fastest marine mammal is the peregrine falcon.

ChatGPT is a bit like autocomplete on your phone. Your phone is trained on a dictionary of words so it completes words. ChatGPT is trained on pretty much all of the web, and can therefore complete whole sentences – or even whole paragraphs.

However, it doesn’t understand what it’s saying, just what words are most likely to come next.

Open only by name

In the past, advances in AI have been accompanied by peer-reviewed literature.

In 2018, for example, when the Google Brain team developed the BERT neural network on which most natural language processing systems are now based (and we suspect ChatGPT is too), the methods were published in peer-reviewed scientific papers and the code was open-sourced[6].

And in 2021, DeepMind’s AlphaFold 2, a protein-folding software, was Science’s Breakthrough of the Year[7]. The software and its results were open-sourced so scientists everywhere could use them to advance biology and medicine.

Following the release of ChatGPT, we have only a short blog post describing how it works. There has been no hint of an accompanying scientific publication, or that the code will be open-sourced.

To understand why ChatGPT could be kept secret, you have to understand a little about the company behind it.

OpenAI is perhaps one of the oddest companies to emerge from Silicon Valley. It was set up as a non-profit[8] in 2015 to promote and develop “friendly” AI in a way that “benefits humanity as a whole”. Elon Musk, Peter Thiel and other leading tech figures pledged US$1 billion towards its goals.

Their thinking was we couldn’t trust for-profit companies to develop increasingly capable AI that aligned with humanity’s prosperity. AI therefore needed to be developed by a non-profit and, as the name suggested, in an open way.

In 2019 OpenAI transitioned into[9] a capped for-profit company (with investors limited to a maximum return of 100 times their investment) and took a US$1 billion investment from Microsoft so it could scale and compete with the tech giants.

It seems money got in the way of OpenAI’s initial plans for openness.

Profiting from users

On top of this, OpenAI appears to be using feedback from users to filter out the fake answers ChatGPT hallucinates.

According to its blog[10], OpenAI initially used reinforcement learning in ChatGPT to downrank fake and/or problematic answers using a costly hand-constructed training set.

But ChatGPT now seems to be being tuned by its more than a million users. I imagine this sort of human feedback would be prohibitively expensive to acquire in any other way.

We are now facing the prospect of a significant advance in AI using methods that are not described in the scientific literature and with datasets restricted to a company that appears to be open only in name.

Where next?

In the past decade, AI’s rapid advance has been in large part due to openness by academics and businesses alike. All the major AI tools we have are open-sourced.

But in the race to develop more capable AI, that may be ending. If openness in AI dwindles, we may see advances in this field slow down as a result. We may also see new monopolies develop.

And if history is anything to go by, we know a lack of transparency is a trigger for bad behaviour in tech spaces. So while we go on to laud (or critique) ChatGPT, we shouldn’t overlook the circumstances in which it has come to us.

Unless we’re careful, the very thing that seems to mark the golden age of AI may in fact mark its end.

Read more: The ChatGPT chatbot is blowing people away with its writing skills. An expert explains why it's so impressive[11]

References

  1. ^ reached capacity several times (www.smartcompany.com.au)
  2. ^ a Chrome extension (chrome.google.com)
  3. ^ ChatGPT (chat.openai.com)
  4. ^ many other tricks (www.straitstimes.com)
  5. ^ computer code (medium.com)
  6. ^ was open-sourced (cloud.google.com)
  7. ^ Breakthrough of the Year (www.aaas.org)
  8. ^ set up as a non-profit (www.zdnet.com)
  9. ^ transitioned into (www.wired.com)
  10. ^ its blog (openai.com)
  11. ^ The ChatGPT chatbot is blowing people away with its writing skills. An expert explains why it's so impressive (theconversation.com)

Read more https://theconversation.com/everyones-having-a-field-day-with-chatgpt-but-nobody-knows-how-it-actually-works-196378

The Times Features

Fast, Fun, And Fantastic Looking Gel Polish For Your Nails!

Today's women spend a lot of time and money on their beauty and fashion regime because they love looking their very best! Looking good makes you feel good, and let's face it, it...

Energy-Efficient Roof Restoration Trends to Watch in Sydney

As climate consciousness rises and energy costs soar, energy-efficient roof restoration has become a significant focus in Sydney. Whether you're renovating an old roof or enhan...

Brisbane Water Bill Savings: Practical Tips to Reduce Costs

Brisbane residents have been feeling the pinch as water costs continue to climb. With increasing prices, it's no wonder many households are searching for ways to ease the burde...

Exploring Hybrid Heating Systems for Modern Homes

Consequently, energy efficiency as well as sustainability are two major considerations prevalent in the current market for homeowners and businesses alike. Hence, integrated heat...

Are Dental Implants Right for You? Here’s What to Think About

Dental implants are now among the top solutions for those seeking to replace and improve their teeth. But are dental implants suitable for you? Here you will find out more about ...

Sunglasses don’t just look good – they’re good for you too. Here’s how to choose the right pair

Australians are exposed to some of the highest levels[1] of solar ultraviolet (UV) radiation in the world. While we tend to focus on avoiding UV damage to our skin, it’s impor...

Times Magazine

Health and Wellness Gift Hampers: Promoting Self-Care and Wellbeing

In today's fast-paced world, self-care and wellbeing are more important than ever. Taking time to relax, rejuvenate, and focus on one's health can significantly impact overall happiness and productivity. A health and wellness gift hamper is an ex...

What is truly anonymous dating and where you can find it

Anonymous dating — how not to worry about your privacy Staying anonymous while dating sounds quite strange. However, we are talking about online dating when everyone can easily be tracked online and this is not the most pleasant experience when ...

Evaluating the Benefits of Pet Insurance: Is It Really Worth It?

Owning a pet can be one of the most rewarding and fulfilling experiences, but it can also come with significant financial costs. Veterinary bills, prescription medications, and other pet-related expenses can quickly add up, and if you're not prepar...

6 Reasons Why Perfume Gift Sets Make an Ideal Present

Perfume gift sets are the perfect way to show someone you care. Whether you're looking for a special treat for a loved one or want to give yourself a luxurious gift, perfume gift sets are a great choice. They come with a variety of different scents...

Why Is a Website the Most Important Part of Digital Marketing?

Your website is the most important digital marketing tool you have, even more so than social media, content marketing, email marketing, traditional outbound advertising, and printed materials. All of your marketing activities will likely lead users...

Temporary Solar Lights: A Portable and Eco-Friendly Lighting Solution for Outdoor Events

Organizing outdoor events in Australia often involves considering various aspects, including logistics, safety, and environmental impact. One crucial element that can be easily overlooked is the lighting solution. Traditionally, outdoor events have...

LayBy Shopping