Thoughts on Apple Vision Pro
Are face computers cool now?
Apple Vision Pro is here. Every past year was going to be the year Apple would finally announce the device. Well, it turned out to be 2023 — and it's not even coming out until 2024. This is what Apple considers as an acceptable baseline for an XR device and it wasn't possible until now. It's the start of Generation 2 which is based on three big leaps. First, a more comfortable and compact headset thanks to micro OLED displays and pancake lenses. Second, a much higher resolution provided by the same micro OLED displays that can easily handle text. Third, passthrough mixed reality and CPU/GPUs have progressed enough to offer a great experience. It's the necessary milestone for a headset to substitute existing displays.
New Approach
Palmer Luckey said that Apple didn't make a device that people could afford but a device that people would want.1 Apple took years of learnings about VR hardware and UX and extracted the best ideas. It also took the only use case — aside from gaming — that we know people want: an infinite display. Package it together, add a good marketing team and we finally have something people want.
Making headsets look cool is hard. People wearing a headset, while the rest aren't, is a complicated social issue. There's a barrier between you and those around you that Apple is trying to break. They made an attractive device that people actually want to put on their faces. Vision Pro also has a great passthrough experience where outside people appear in your field of view. Finally, they made the device seem transparent by showing your eyes on a lenticular display to those around you. The jury is still out on how well this will work.
Apple brings a new perspective to the table. Meta keeps getting tangled up in explaining their vision; they try to do everything at once but with no single feature standing out. They tie everything together with the term Metaverse that nobody understands and we've grown tired of hearing. Apple focuses on one thing, and one thing only: floating 2D screens. It's something everybody understands already. Matt Miesnieks uses the idea of the conceptual ladder that explains how consumers can only absorb one concept at a time. Here Apple is sticking to spatially arranged windows, but they are your existing 2D windows. By focusing on one thing, they can also nail that use case.
User Experience
Vision Pro accelerates the advent of hand tracking and mixed reality. While these technologies have been available for some time, they have felt like a prototype. Apple has shown that it can be done today if implemented with sufficient care and attention to detail. They have also set the standard for how to do a tap in XR: look at what you want to click and tap your thumb and index fingers. Now other companies will quickly catch up — I'm sure the Quest 4 will have eye tracking.
It's the first headset that supports multiple 3D applications running together. It's a very hard problem to solve, and you need very capable hardware to run it. But the main issues are how they share user interactions and how they are rendered coherently. Vision OS solves this the Apple way — by placing certain constraints on how applications are developed so the OS maintains control.2 Apple ensures that everything looks great and is interactable without sharing too much hardware data with each application.
Competition
Valve and Sony were never competitors with Meta; their headsets only focused on gaming and they are PC/Console devices. Other standalone device makers are still trying to catch up with Meta without getting close in content. But Apple is a much bigger company with a massive developer community, and they have built a complete holistic system that competes with Meta in entertainment and professional use cases.
Now Meta has a reason to create a Quest Pro 2; the Vision Pro shows that a product line focused on high-end hardware to render an infinite display is interesting when implemented well. Meta can try to compete with Apple with the Quest Pro while continuing to deepen their domination of the consumer side with the Quest line. I think they'll still fail with the Quest Pro. Apple will be good competition for Meta and make them improve their offering, but they still don't have the right leadership and culture to make things people actually want.
What Apple hasn't shown is any major breakthrough. This was the biggest letdown of the announcement. We have many design issues around 3D interfaces and I couldn't wait to see Apple's approach to these problems. Instead, they focused on 2D interfaces which we already understand. I get why they did it though: the conceptual ladder.
Many problems continue to be unresolved. How will our OS interact with our environment? How will we use AR outside? How do AI assistants and 3D interfaces mesh together? GUIs are supposed to disappear but instead, we are getting bigger screens. What about haptics? Apple's approach is to launch the device and see what developers do with it. Vision Pro is capturing a lot of mainstream interest in XR which brings more investment and more developers. Who knows what those new people may bring to the table? In the end, it's still up to us to make software the people want.
Podcast interview where Palmer outlines his thoughts on Vision Pro
Brandon Jones has a good explanation of how it works