The Times Australia
The Times World News

.
The Times Real Estate

.

‘AI agents’ promise to arrange your finances, do your taxes, book your holidays – and put us all at risk

  • Written by Uri Gal, Professor in Business Information Systems, University of Sydney

Over the past two years, generative artificial intelligence (AI) has captivated public attention. This year signals the beginning of a new phase: the rise of AI agents.

AI agents are autonomous systems that can make decisions and take actions on our behalf without direct human input. The vision is that these agents will redefine work and daily life by handling complex tasks for us. They could negotiate contracts, manage our finances, or book our travel.

Salesforce chief executive Marc Benioff has said he aims to deploy a billion AI agents[1] within a year. Meanwhile Meta chief Mark Zuckerberg predicts[2] AI agents will soon outnumber the global human population.

As companies race to deploy AI agents, questions about their societal impact, ethical boundaries and long-term consequences grow more urgent. We stand on the edge of a technological frontier with the power to redefine the fabric of our lives.

How will these systems transform our work and our decision-making? And what safeguards do we need to ensure they serve humanity’s best interests?

AI agents take the control away

Current generative AI systems react to user input, such as prompts. By contrast, AI agents act autonomously within broad parameters. They operate with unprecedented levels of freedom – they can negotiate, make judgement calls, and orchestrate complex interactions with other systems. This goes far beyond simple command–response exchanges like those you might have with ChatGPT.

For instance, imagine using a personal “AI financial advisor” agent to buy life insurance. The agent would analyse your financial situation, health data and family needs while simultaneously negotiating with multiple insurance companies’ AI agents.

It would also need to coordinate with several other AI systems: your medical records’ AI for health information, and your bank’s AI systems for making payments.

The use of such an agent promises to reduce manual effort for you, but it also introduces significant risks.

The AI might be outmanoeuvred by more advanced insurance company AI agents during negotiations, leading to higher premiums. Privacy concerns arise as your sensitive medical and financial information flows between multiple systems.

The complexity of these interactions can also result in opaque decisions. It might be difficult to trace how various AI agents influence the final insurance policy recommendation. And if errors occur, it could be hard to know which part of the system to hold accountable.

Perhaps most crucially, this system risks diminishing human agency. When AI interactions grow too complex to comprehend or control, individuals may struggle to intervene in or even fully understand their insurance arrangements.

A tangle of ethical and practical challenges

The insurance agent scenario above is not yet fully realised. But sophisticated AI agents are rapidly coming onto the market.

Salesforce and Microsoft have already incorporated AI agents into some of their corporate products, such as Copilot Actions[3]. Google has been gearing up for the release of personal AI agents since announcing its latest AI model, Gemini 2.0[4]. OpenAI is also expected to release a personal AI agent[5] in 2025.

The prospect of billions of AI agents operating simultaneously raises profound ethical and practical challenges.

These agents will be created by competing companies with different technical architectures, ethical frameworks and business incentives. Some will prioritise user privacy, others speed and efficiency.

They will interact across national borders where regulations governing AI autonomy, data privacy and consumer protection vary dramatically.

This could create a fragmented landscape where AI agents operate under conflicting rules and standards, potentially leading to systemic risks.

What happens when AI agents optimised for different objectives – say, profit maximisation versus environmental sustainability – clash in automated negotiations? Or when agents trained on Western ethical frameworks make decisions that affect users in cultural contexts for which they were not designed?

The emergence of this complex, interconnected ecosystem of AI agents demands new approaches to governance, accountability, and the preservation of human agency in an increasingly automated world.

How do we shape a future with AI agents in it?

AI agents promise to be helpful, to save us time. To navigate the challenges outlined above, we will need to coordinate action across multiple fronts.

International bodies and national governments must develop harmonised regulatory frameworks that address the cross-border nature of AI agent interactions.

These frameworks should establish clear standards for transparency and accountability, particularly in scenarios where multiple agents interact in ways that affect human interests.

Technology companies developing AI agents need to prioritise safety and ethical considerations from the earliest stages of development. This means building in robust safeguards that prevent abuse – such as manipulating users or making discriminatory decisions.

They must ensure agents remain aligned with human values. All decisions and actions made by an AI agent should be logged in an “audit trail” that’s easy to access and follow.

Importantly, companies must develop standardised protocols for agent-to-agent communication. Conflict resolution between AI agents should happen in a way that protects the interests of users.

Any organisation that deploys AI agents should also have comprehensive oversight of them. Humans should still be involved in any crucial decisions, with a clear process in place to do so. The organisation should also systematically assess the outcomes to ensure agents truly serve their intended purpose.

As consumers, we all have a crucial role to play, too. Before entrusting tasks to AI agents, you should demand clear explanations of how these systems operate, what data they share, and how decisions are made.

This includes understanding the limits of agent autonomy. You should have the ability to override agents’ decisions when necessary.

We shouldn’t surrender human agency as we transition to a world of AI agents. But it’s a powerful technology, and now is the time to actively shape what that world will look like.

References

  1. ^ billion AI agents (techcrunch.com)
  2. ^ predicts (www.facebook.com)
  3. ^ Copilot Actions (www.microsoft.com)
  4. ^ latest AI model, Gemini 2.0 (arstechnica.com)
  5. ^ personal AI agent (opentools.ai)

Read more https://theconversation.com/ai-agents-promise-to-arrange-your-finances-do-your-taxes-book-your-holidays-and-put-us-all-at-risk-247021

The Times Features

Important things to consider before buying a new TV

The way that people live their everyday lives continues to evolve as new technology becomes available. Those who purchase a home are making what is likely to be one of their larg...

Cast vs. Minted Gold Bars: Key Differences Every Investor Should Know

Investing in gold bars is one of the most dependable ways of preserving one's wealth and diversifying one's portfolio. However, not all gold bars are minted equally. Understanding...

Understanding the Common Challenges Addressed by Child Psychologists

Parenting is challenging, especially when children have emotional, behavioural, or developmental problems. Child psychologists are experts who focus on dealing with such issues...

What to Expect in Cosmetic Clinics in Western Australia: Comprehensive Overview

Over the recent past, cosmetic clinics were popular since people sought a means of beautifying and building confidence. Western Australia has various high-class cosmetic clinics th...

Empowering Education: Flinders University Partners with The Missing Link for AI Training

The Missing Link, a leader in intelligent AI and automation solutions collaborates with Flinders University to deliver a bespoke Microsoft 365 Copilot training program. This part...

Maximizing Your Outdoor Adventures with the Right Bike Rack

With cycling becoming an increasingly popular activity across Australia, having the right gear is crucial for a seamless experience. A reliable bike rack plays a vital role in en...

Times Magazine

Top Tips for Finding a Great Florist for Your Sydney Wedding

While the choice of wedding venue does much of the heavy lifting when it comes to wowing guests, decorations are certainly not far behind. They can add a bit of personality and flair to the traditional proceedings, as well as enhancing the venue’s ...

Avant Stone's 2025 Nature's Palette Collection

Avant Stone, a longstanding supplier of quality natural stone in Sydney, introduces the 2025 Nature’s Palette Collection. Curated for architects, designers, and homeowners with discerning tastes, this selection highlights classic and contemporary a...

Professional-Grade Tactical Gear: Why 5.11 Tactical Leads the Field

When you're out in the field, your gear has to perform at the same level as you. In the world of high-quality equipment, 5.11 Tactical has established itself as a standard for professionals who demand dependability. Regardless of whether you’re inv...

Lessons from the Past: Historical Maritime Disasters and Their Influence on Modern Safety Regulations

Maritime history is filled with tales of bravery, innovation, and, unfortunately, tragedy. These historical disasters serve as stark reminders of the challenges posed by the seas and have driven significant advancements in maritime safety regulat...

What workers really think about workplace AI assistants

Imagine starting your workday with an AI assistant that not only helps you write emails[1] but also tracks your productivity[2], suggests breathing exercises[3], monitors your mood and stress levels[4] and summarises meetings[5]. This is not a f...

Aussies, Clear Out Old Phones –Turn Them into Cash Now!

Still, holding onto that old phone in your drawer? You’re not alone. Upgrading to the latest iPhone is exciting, but figuring out what to do with the old one can be a hassle. The good news? Your old iPhone isn’t just sitting there it’s potential ca...

LayBy Shopping