The Times Australia
The Times World News

.

Apple’s new Vision Pro has big ambitions

  • Written by Luke Heemsbergen, Senior Lecturer, Digital, Political, Media, Deakin University
Apple’s new Vision Pro has big ambitions

Apple Vision Pro is a mixed-reality headset – which the company hopes is a “revolutionary spatial computer[1] that transforms how people work, collaborate, connect, relive memories, and enjoy entertainment” – that begins shipping to the public (in the United States) later this week.

Critics have doubted the appeal[2] of the face-worn computer, which “seamlessly blends digital content with the physical world”, but Apple has pre-sold as many as 180,000[3] of the US$3,500 gizmos.

What does Apple think people will do with these pricey peripherals? While uses will evolve, Apple is focusing attention on watching TV and movies, editing and reliving “memories”, and – perhaps most importantly for the product’s success – having its customers not look like total weirdos.

Apple hopes the new device will redefine personal computing, like the iPhone did 16 years ago, and Macintosh did 40 years ago. But if it succeeds, it will also redefine concerns about privacy, as it captures enormous amounts of data about users and their environments, creating an unprecedented kind of “biospatial surveillance[4]”.

Spatial computing

Apple is careful about its brand and how it packages and describes its products. In an extensive set of rules for developers[5], the company insists the new headset is not to be referred to as a “headset”. What’s more, the Apple Vision Pro does not do “augmented reality (AR), virtual reality (VR), extended reality (XR), or mixed reality (MR)” – it is a gateway to “spatial computing”.

Spatial computing, as sketched out in the 2003 PhD thesis[6] of US software engineer Simon Greenwold, is: “human interaction with a machine in which the machine retains and manipulates referents to real objects and spaces”. In other words, the computer can interact with things in the user’s physical surroundings in real time to provide new types of experiences.

A CGI dinosaur stands on a rocky field.
The Vision Pro comes with an app that lets users get up close and personal with dinosaurs. Apple[7]

The Vision Pro has big shoes to fill for new user experiences. The iPhone’s initial “killer apps” were clear[8]: the internet in your pocket (including portable access to Google Maps), all your music on a touch screen, and “visual voicemail[9]”.

Sixteen years later, all three of these seem unremarkable. Apple has sold billions of iPhones, and some 80% of humans[10] now use a smartphone. Their success has all but killed off earlier tools like paper maps and music CDs (and the ubiquity of text, image and video messaging has largely done away with voicemail itself).

Killer apps

We don’t yet know what the killer apps of spatial computing might be – if any – but here is where Apple is pointing our attention.

The first is entertainment: the Vision Pro promises “the ultimate personal theatre[11]”.

The second is an attempt to solve the social problem of walking around with a weird headset covering half your face. An external screen on the goggles shows a constantly updated representation of your eyes to offer important social cues about your gaze[12] to those around you. Admittedly, this looks weird. But Apple hopes it is less weird and more useful than trying to interact with humans wearing blank aluminium ski goggles.

A man sitting on a couch wearing a headset while an image of children playing floats in the air in front of him. Reliving ‘memories’ with the Apple Vision Pro. Apple[13]

The third is the ability to capture and and relive “memories”: recording and playback of 3D visual and audio from real events. Reviewers have found it striking:

this was stuff from my own life[14], my own memories. I was playing back experiences I had already lived.

Apple has patented[15] tools to select, store, and annotate digital “memories”. These memories are files, and potentially products, to be shared in “spatial videos” recorded on the latest iPhones[16].

Biospatial surveillance

There is already a large infrastructure devoted to helping tech companies track our behaviour in order to sell us things. Recent research[17] found Facebook, for example, receives data from an average of around 2,300 companies on each individual user.

Read more: Explainer: what is surveillance capitalism and how does it shape our economy?[18]

Spatial computing offers a step change to this tracking. In order to function, spatial computing records and uses vast amounts of intimate data about our bodies and surroundings.

One study on headset design[19] noted no fewer than 64 different streams of biometric and physiological data, from eye tracking and pupil response to subtle changes in the body’s electromagnetic field.

Your face tomorrow

This is not “consumer” data like the brand of toothpaste you buy. It is more akin to medical data.

For instance, analysing a person’s unconscious movements[20] can reveal their emotional state or even predict neurodegenerative disease. This is called “biometrically inferred data[21]” as users are unaware their bodies are giving it up.

Apple suggests it won’t share this type of data with anyone, and Apple has proven better than most companies on privacy. But biospatial surveillance puts more of ourselves in use for spatial computing, in ways that are expanding.

It starts simply enough in the pre-order process, where you need to scan your facial features with your iPhone (to ensure a snug fit). But that’s not the end of it.

Apple’s patent about memories[22] is also about how to “guide and direct a user with attention, memory, and cognition” through feedback loops that monitor “facial recognition, eye tracking, user mood detection, user emotion detection, voice detection, etc. [from a] bio-sensor for tracking biometric characteristics, such as health and activity metrics […] and other health-related information”.

Social questions

Biospatial surveillance is also the key to Apple’s attempt to solve the social problems created by wearing a headset in public. The external screen showing a simulated approximation of the user’s gaze relies on constant measurement of the user’s expression and eye movement with multiple sensors.

A man wearing goggles with a screen that shows his eyess An external screen shows a representation of the user’s eyes. Apple[23]

Your face is constantly mapped so others can see it – or rather see Apple’s vision of it. Likewise, as passersby come into range of the Apple Vision Pro’s sensors, Apple’s vision of them is automagically rendered into your experience, whether they like it or not.

Apple’s new vision of us – and those that surround us – shows how the requirements and benefits of spatial computing will pose new privacy concerns and social questions. The extensive biospatial surveillance that captures intimate biometric and environmental data redefines what personal data and social interactions are possible for exploitation.

References

  1. ^ revolutionary spatial computer (www.apple.com)
  2. ^ doubted the appeal (www.wired.com)
  3. ^ as many as 180,000 (www.engadget.com)
  4. ^ biospatial surveillance (journals.sagepub.com)
  5. ^ rules for developers (developer.apple.com)
  6. ^ 2003 PhD thesis (acg.media.mit.edu)
  7. ^ Apple (www.apple.com)
  8. ^ clear (www.macworld.com)
  9. ^ visual voicemail (www.youtube.com)
  10. ^ 80% of humans (www.statista.com)
  11. ^ the ultimate personal theatre (www.apple.com)
  12. ^ offer important social cues about your gaze (cavrn.org)
  13. ^ Apple (www.apple.com)
  14. ^ stuff from my own life (www.cnet.com)
  15. ^ patented (www.patentlyapple.com)
  16. ^ recorded on the latest iPhones (www.apple.com)
  17. ^ research (www.consumerreports.org)
  18. ^ Explainer: what is surveillance capitalism and how does it shape our economy? (theconversation.com)
  19. ^ study on headset design (www.slideshare.net)
  20. ^ analysing a person’s unconscious movements (www.mkhamis.com)
  21. ^ biometrically inferred data (xrsi.org)
  22. ^ patent about memories (patents.google.com)
  23. ^ Apple (youtu.be)

Read more https://theconversation.com/editing-memories-spying-on-our-bodies-normalising-weird-goggles-apples-new-vision-pro-has-big-ambitions-221910

Times Magazine

Headless CMS in Digital Twins and 3D Product Experiences

Image by freepik As the metaverse becomes more advanced and accessible, it's clear that multiple sectors will use digital twins and 3D product experiences to visualize, connect, and streamline efforts better. A digital twin is a virtual replica of ...

The Decline of Hyper-Casual: How Mid-Core Mobile Games Took Over in 2025

In recent years, the mobile gaming landscape has undergone a significant transformation, with mid-core mobile games emerging as the dominant force in app stores by 2025. This shift is underpinned by changing user habits and evolving monetization tr...

Understanding ITIL 4 and PRINCE2 Project Management Synergy

Key Highlights ITIL 4 focuses on IT service management, emphasising continual improvement and value creation through modern digital transformation approaches. PRINCE2 project management supports systematic planning and execution of projects wit...

What AI Adoption Means for the Future of Workplace Risk Management

Image by freepik As industrial operations become more complex and fast-paced, the risks faced by workers and employers alike continue to grow. Traditional safety models—reliant on manual oversight, reactive investigations, and standardised checklist...

From Beach Bops to Alpine Anthems: Your Sonos Survival Guide for a Long Weekend Escape

Alright, fellow adventurers and relaxation enthusiasts! So, you've packed your bags, charged your devices, and mentally prepared for that glorious King's Birthday long weekend. But hold on, are you really ready? Because a true long weekend warrior kn...

Effective Commercial Pest Control Solutions for a Safer Workplace

Keeping a workplace clean, safe, and free from pests is essential for maintaining productivity, protecting employee health, and upholding a company's reputation. Pests pose health risks, can cause structural damage, and can lead to serious legal an...

The Times Features

Tricia Paoluccio designer to the stars

The Case for Nuturing Creativity in the Classroom, and in our Lives I am an actress and an artist who has had the privilege of sharing my work across many countries, touring my ...

Duke of Dural to Get Rooftop Bar as New Owners Invest in Venue Upgrade

The Duke of Dural, in Sydney’s north-west, is set for a major uplift under new ownership, following its acquisition by hospitality group Good Beer Company this week. Led by resp...

Prefab’s Second Life: Why Australia’s Backyard Boom Needs a Circular Makeover

The humble granny flat is being reimagined not just as a fix for housing shortages, but as a cornerstone of circular, factory-built architecture. But are our systems ready to s...

Melbourne’s Burglary Boom: Break-Ins Surge Nearly 25%

Victorian homeowners are being warned to act now, as rising break-ins and falling arrest rates paint a worrying picture for suburban safety. Melbourne residents are facing an ...

Exploring the Curriculum at a Modern Junior School in Melbourne

Key Highlights The curriculum at junior schools emphasises whole-person development, catering to children’s physical, emotional, and intellectual needs. It ensures early year...

Distressed by all the bad news? Here’s how to stay informed but still look after yourself

If you’re feeling like the news is particularly bad at the moment, you’re not alone. But many of us can’t look away – and don’t want to. Engaging with news can help us make ...