Practical Magic: VR and AR Are the Next Big Thing… Again

Apple’s Vision Pro looms as the biggest product in VR and AR for years, but how will it make the territory more functional?

John Kim/CNET

Days after reviewing Meta’s newest VR headset, the Quest 3, last fall, I was hospitalized with high blood pressure. Although the two events were unrelated, the latter changed my life quite a bit. Besides needing to pay even more attention to my eating habits, and going on a lot of new meds, I was told to focus on getting regular cardio exercise. These were familiar reminders: I’ve faced this path before. 

When I told a friend I needed to exercise more, she recommended a VR fitness app called Supernatural on the Quest. I laughed because I’d already tried it, but this time I was hearing a testimonial from someone I didn’t even know used VR. She got the Quest 2 as a gift earlier in the year and has been using it ever since. So I joined again too.

I’ve been doing daily VR workouts for months now. The Quest VR headset is my purpose-based fitness device. Meanwhile, downstairs, my oldest son plays Beat Saber and The Walking Dead on Quest 2. For him, it’s his game console.

The future is now the present. Welcome to VR and AR, circa 2024. While Apple and the forthcoming Vision Pro represent an exciting potential future for the technology, Meta’s Quest and headsets like it are the already real and sometimes functional present. 

Watch this: VR and AR in 2024: Apple, Samsung, Google, Meta, AI, and a quest for function 06:30

But can these devices do even more? Can they transcend being game consoles, fitness devices and experimental toys? Will VR and AR, in a sense, grow up? It’s something I’ve seen slowly emerging throughout the pandemic years and the obsession with the metaverse, but in 2024 we may finally see products and apps that bring virtual augmented reality into the same functional universe as our phones and computers, maybe finally leading to mass adoption like smartphones over a decade ago.

The Apple Vision Pro may get company via a competing Samsung/Google headset next year, but we already know what Apple’s is.

John Kim/CNET

Apple Vision Pro: The device that could change the landscape

I saw a glimpse of the immediate future of AR and VR in 2023. Apple’s Vision Pro headset, expected to go on sale early in 2024, melds camera feeds of the real world with incredibly high-fidelity virtual visuals. It’s a technique previously used by Meta and in higher-end headsets like the Varjo XR-3 and XR-4, but Apple has refined the interface with hand and eye tracking that, at least in early demos I’ve had, feels remarkably easy and fluid. 

Apple is also changing the sales pitch by replacing the terms “VR” and “AR” with its own phrase: “spatial computing.” Instead of focusing on games or fitness or even social connection to start, the company’s marketing discusses how the Vision Pro can run all sorts of iOS apps and connect with Macs. Apple also emphasized a wearable display that’s good enough for viewing movies and photos in fantastic quality. In my demos, that’s actually what impressed me most.

The Vision Pro headset is Apple’s first major new product since the Apple Watch and is the most anticipated new piece of hardware in the VR/AR space since the original Oculus Rift. Despite its high price ($3,499) and likely limited availability, the Vision Pro could completely redefine virtual and augmented reality devices for the next decade. Or it could be another product blip in a landscape of forgotten headsets. Based on Apple’s history of successful products, the former seems more likely to me.

Will app compatibility with mixed reality become a larger trend after Vision Pro? It needs to be. Will Apple’s controller-free design push other manufacturers to think of their own hand-and-eye interfaces, too? It seems likely. Meta already wants to eventually sell its VR headsets without controllers.

The Meta Quest 3 already does mixed reality for $500. Expect it to help explore new uses for the technology along with Apple.

Scott Stein/CNET

Mixed reality at large

Everyday AR glasses, where virtual images and information appear in the lenses alongside the reality you see, aren’t a thing yet. There are many roadblocks: prescription concerns, safety, social acceptance, app compatibility, gesture interfaces and also just keeping them running for more than an hour on a charge.

In the meantime, VR with mixed reality has become the stand-in. Meta’s Quest headsets, Apple’s Vision Pro, Varjo’s industrial headsets and likely every VR headset going forward will have capabilities to blend virtual things with the real world using depth sensors and improved headset cameras. 

The concept is still new, and so far, limited. Meta’s Quest 3 relegates mixed reality to being a gimmick in a few games right now. The Vision Pro, as we’ve seen so far, mostly uses mixed reality to float 2D displays over the real world.

In 2024, the door seems wide open for developers to figure out ways to apply toolkits used today in phone- and tablet-based AR apps like Pokemon Go. While we think of VR at the moment as being “Hey look at me, I’m in another world,” the experiential shift could move quickly to pockets of immersion embedded in real-world environments. The Quest 3 is a device I expect to lead that change.

Apple’s already opening lots of new potential apps for mixed reality. Samsung and Google could do the same.

Apple

Expect a new wave of apps

Apple won’t be the only new VR/AR platform in the picture. Samsung and Google and Qualcomm are about to debut their own mixed reality headset in 2024, likely offering another high-end alternative to the Vision Pro, but in a way that opens up development via Google’s software. It sounds like a return to Google’s long-dead Daydream dreams, but it’ll likely mean a whole new set of worlds opens up: Apple’s App Store, and very likely Google Play as well. 

For Apple, it starts with familiar 2D iOS apps translated to a mixed reality interface, but the possibilities beyond that could get vast. Apple could add 3D movie support to Photos, for instance, or the Persona avatars to FaceTime. App developers already working with the VisionOS software are going to have to solve for different ideas than on the Quest’s hardware, which could also change the type of apps offered on each device. Games, for instance, won’t be able to use standard VR controllers on the Vision Pro: instead, we’ll have hand- and eye-tracking-driven apps.

Meta’s CTO, Andrew Bosworth, has already welcomed 2D apps onto the Quest platform, and told me that Google moving Play onto Quest would be “trivial.” It hasn’t happened yet. Google could open up Android into mixed reality as Apple opens iOS, both of these moves possibly opening the floodgates to developers who may not have been tempted by more limited Quest, PlayStation, Vive or SteamVR avenues. VR’s pipeline of new games and apps has slowed down over the past few years. Will Apple, Google and Samsung spark a new movement?

Meta’s glasses look normal, but in 2024 they’ll be dabbling in AI via cameras and microphones.

Scott Stein/CNET

AI enters the mix

After a year entirely defined by AI, it’s no surprise that AI would somehow find its way into VR and AR. Especially for Meta, that’ll be a big part of 2024’s plans. Meta’s Bosworth and CEO Mark Zuckerberg already laid out AI-infused plans for their metaverse dreams at the last Meta Connect developer conference, and both the Quest 3 and Meta’s Ray-Ban glasses have new chipsets with more robust AI processing capabilities.

Wearables like Humane’s Ai Pin and Meta’s new Ray-Ban glasses will both use cameras to fuel AI on-device. Camera-driven AI, which should work in a similar spirit to how Google Lens “sees” the world to interpret the environment, objects and text, would make a ton of sense in camera-studded mixed reality headsets. There could be assistive uses, safety features or just better ways that software could recognize the world and fold augmented reality into it.

The greatest immediate movement in this space could come in smart glasses instead of VR headsets.

I tried AR glasses last year and many other times. They exist, but they’re still experimental devices. Smarter everyday glasses are already here.

Scott Stein/CNET

What about glasses?

Qualcomm, and a number of other companies, are actively exploring ways to make glasses even smarter and more phone-connected. In the short term, glasses like Meta’s latest pair of Ray-Bans focus on being camera-enabled, microphone- and speaker-connected phone peripherals. In 2024, expect more layers of AI to help these devices listen and use camera data more intelligently, and maybe act more like wearable assistants.

Glasses with displays already exist from a number of companies including XReal, Lenovo and TCL. Expect more plug-in, monitor-on-your-face solutions in glasses form, especially since more devices are friendlier to USB-C connections and the display tech is improving constantly. I’ve already tried using an AR glasses-based “laptop,” so the idea is possible. Now they need to work better with regular glasses and feel less clunky.

We may even see some AR glasses that aim to do (in smaller forms) what mixed-reality VR headsets are doing. Still, the signs are clear that the progress in that zone has slowed down. Until mixed reality software improves, and phone manufacturers — namely, Apple and Google — find ways to dovetail glasses seamlessly as everyday phone peripherals, AR glasses won’t become a truly everyday thing for anyone.

Would I make my next pair of glasses a set of smart Ray-Bans? I’m already test-driving a prescription pair, and the experience is already sometimes profound. Smarter glasses will figure out the territory as VR headsets improve, and someday they’ll meet in the middle. That someday isn’t this year, and it won’t be the year after. In 2024, all VR and AR need to solve is finding a few more ways to be really useful. 

My family and I have already found a couple in fitness and gaming. I bet 2024 finds a few more.