Apple Should Build Its AR on the Meta Quest

The Meta Quest Pro, released late last year, is a sign of where Apple can try to improve what’s available.
Scott Stein/CNET

Apple’s first-ever VR-AR or “mixed reality” device is expected this year, and already its follow-up product seems to be on hold. According to Bloomberg’s reliable Mark Gurman, a planned set of AR glasses isn’t coming from Apple anytime soon. That’s OK: AR glasses don’t seem like they’re imminent from anyone.

After a trip to Las Vegas to try out some of the emerging technologies around future VR and AR headsets, one thing is increasingly clear: Everyone’s trying to figure out AR glasses, but everyone’s trying to perfect VR headsets. The difference between those two scenarios is bigger than you think.

All-day AR glasses that are useful, work convincingly, have long enough battery life, work with your phone and function as actually legit prescription glasses haven’t materialized, although the pieces are coming into place. Companies like Meta have promised a decade-long path to these glasses. It turns out, maybe, that’s going to be the case for everyone else, too. I’ve seen smart glasses that look real but don’t do much, or glasses with AR that feel chunky and do some things, but don’t quite work with my vision and can’t figure out how to work with my phone yet. Chipmaker Qualcomm is working on this; Google, Apple and Samsung need to solve it someday, too.

In the meantime, VR already has a very real and reasonably popular product that most families I know in the everyday world are familiar with: the Meta (formerly Oculus) Quest 2.

Read more: The Quest 2 Is Still the Best VR Headset For Now

That recognition is no small thing. I think of the Quest’s place in everyday life like the Amazon Echo was years ago: something odd that over time became familiar, normalized. Something reasonably priced, and good enough to do a few things actually well. The Quest 2 is basically a game console. Where Meta has struggled is figuring out how to expand that base beyond gamers. 

The Quest 2: recognizable, and an existing product category already.

James Martin/CNET

Following Meta’s playbook is something I expected Apple would do. Heck, I expect most companies are going to do it. The Quest 2 works just like most people imagine it will, or better. It’s a bit of instant magic that’s totally wire-free.

The Quest 2 does have downsides. In fact, those problems emerge the more you use it. I find connecting with friends and social spaces gets weird and buggy, prone to lag, disconnects and way-too-basic avatars. The battery life is bad. For fitness apps, which the Quest 2 can do surprisingly well, it’s still not good at really managing sweat or keeping my glasses from fogging. 

Even though Meta wants the Quest 2 and higher-end, work-focused Quest Pro to open up new ways to work by creating virtual monitors around my laptop, the connections and display quality aren’t good enough to be more than a clever experiment most of the time. I can see my laptop keyboard with the Quest Pro’s passthrough cameras, but typing feels awkward and nowhere near as good as when I’m just on my laptop… and I can’t see my phone screen to check messages, either. The Quest can show me phone notifications like a basic smartwatch from a decade ago, but I can’t interact with them.

These VR headsets can even do some basic AR, using passthrough cameras that “mix reality” to show the real world in fuzzier video, with VR layered on top. The effect is sometimes pretty amazing, and could even approach feelings I’ve had with early AR headsets like the Microsoft HoloLens 2.

So where does that put Apple? Clearly, there’s a headset coming soon. And according to Bloomberg’s Gurman, the next goal after this first expensive headset is to work on a more affordable model. It’s like Meta’s approach to the Quest and Quest Pro, in reverse. And there are plenty of things Apple could focus on to make its entry into VR (and AR) worth the effort.

Razer’s upcoming improved head straps for the Quest 2 show that comfort is still a needed work in progress for VR.

James Martin/CNET

Better comfort, better fitness

The Quest 2 is already an affordable fitness device, and pairs with watches to show heart rate and fitness stats. Apple clearly has an advantage on time spent developing the Apple Watch, fitness and health tracking, and its Fitness Plus subscription video workouts, which also have overlaid fitness stats. 

Apple could emphasize workouts and fitness on its headset, with comfortable, breathable straps and face pieces that could feel better for exercise. Meta is starting to realize it needs to improve comfort for VR: A recent Razer partnership using head straps made by CPAP-maker ResMed shows a need for better materials. I’d expect Apple to make this aspect a key part of the headset’s advantages. There are other advantages, too. Apps like Beat Saber and Supernatural use music for fitness, and Apple already has all of Apple Music at its disposal. 

Connect better with laptops, iPads, phones, watches

VR headsets right now have an extremely hard time working well with all the other things we have lying around us. I can’t get a Quest to connect nicely with my phone all the time. To work with my laptop, I need a specialized third-party app with its own thing I have to install on my laptop and turn on. 

Meanwhile, Apple has been focusing on handoffs and continuity across AirPods, HomePods, iPhones, Apple Watches, MacBooks, Apple TVs… all over the place. That’s what’s needed to make a VR headset seem seamless and integrated into other stuff. I want to check my watch in VR, or use it to control apps. Or use my phone, and also see the phone. Suddenly grab my laptop, and the headset connects. Incoming calls? No problem. Send myself things back and forth from my phone or laptop and get all the files and things I want, and not feel like I’m on a vacation from them. That’s what Apple’s headset could set out to achieve.

That’s a best-case scenario. Much like the first Apple Watch and iPhone, the actual Day 1 functions of this headset might end up disappointing.

Better social

Even though the metaverse is on everyone’s minds, there aren’t many big social spaces in VR that work well. Microsoft’s Altspace is nice, but often feels empty. VRChat is wild, experimental, full of big features and ideas, and feels like a messy explosion that’s hard to jump into. Meta can’t get enough people into Horizon Worlds. Even when these platforms do work, for concerts or events, the limits on people who can attend at once, the lag and drop-off, not to mention the avatar limits, make it a trade-off versus any other way you could connect on a phone or laptop.

Apple may not be able to solve this any better for larger-scale experiences, but for more intimate and several-person FaceTime-like moments, Apple could make shared experiences in VR work a lot better. Meta hasn’t perfected social VR yet, and someone needs to.

The Meta Quest Pro controller, like most VR controllers, has buttons and analog sticks. Is this the only answer?

Scott Stein/CNET

Can Apple make a better controller? (Or none?)

The Quest 2 controllers are fine, but all of VR leans on the same game controller-like inputs for headsets. Apple’s headset could lean more on hand tracking, or wearable inputs like the Apple Watch. I’m curious if a more work-oriented controller or accessory can be created that makes the headset feel better for taking on apps beyond games. Meta’s working on a long-term, game-changing shift to neural input wristbands eventually, but it’s unclear whether this approach will end up succeeding.

The Quest platform has continually improved its hand tracking over the years. However, hand tracking’s reliance on particular gestures without any physical feedback is an imperfect solution right now. Maybe Apple tries hand tracking along with using an Apple Watch or the iPhone for tactile haptic feedback, or finds a smaller go-between accessory.

I’ve been trying out experimental haptic technology recently, trying to imagine how VR could think its way to new inputs. This headset feels like the biggest opportunity Apple’s ever faced to create a brand-new type of input device that could make a big impact on the landscape. If it’s done right, maybe it’ll be the input accessory that makes its future AR glasses, whenever they arrive, seem feasible.

This is a capture of me in a Varjo XR-3 mixed reality, in a virtual kitchen, looking at myself through a window into the real room. Mixed reality can, in theory, be wild.

Varjo

Build out more interesting mixed reality

For all the Meta Quest Pro promises to blend AR and VR with its mixed-reality capabilities, not many apps tap into its extras yet. I’ve seen some mind-blowing demos of mixed reality in VR with the ultra-high-end Varjo XR-3 connected to a PC, which at least showed me ways that a VR headset could begin to feel like a portal interconnected to my own home reality. Apple could start experimenting with more engaging AR moments in a high-end VR headset, and at least get the ball rolling on things that work in advance of whenever its AR glasses are ready, years from now.

Smaller sessions in VR may make more sense right now

VR is a thing I don’t use all the time, and that’s true for most people. Maybe that’s exactly where Apple should start. It’s not a given we’ll want to wear AR glasses everywhere, or even what those glasses would be good for. In the meantime, a VR headset at home that’s meant to be worn sometimes, but not all the time, is the place most of us feel safest to start. It’s why the Quest is something people actually use.

It’s also a way to avoid dealing with questions of accommodating true prescription vision needs in everyday glasses, something no one’s succeeded in tackling, either. VR headsets sometimes need prescription inserts, but many just fit right over the glasses we already have. I prefer the easy-fit solution: I don’t need to make VR a thing I spend a whole day in. I’ll settle for a truly useful hour or two, and if Apple can make that hour or two even better than what we have now, that’s a big enough step forward for me.

Editor’s note, Jan. 20: Adds mention of Meta’s hand tracking for the Quest.