After two weeks with Vision Pro, we discovered the killer app for spatial computing

During the time we got Vision Pro, we played through almost all popular applications.

The actual experience is exactly the same as the previous summary of foreign media bloggers. Although I already have a certain understanding of this device before going up, I will still be a little shocked after using it myself.

As we gradually became accustomed to interacting with only our hands and eyes, we also figured out the true meaning of "spatial computing":

  • spatial interaction
  • space image
  • spatial audio

Apple used these three tools to build a moat for Vision Pro and create a new ecosystem between UI and applications.

Apple CEO Cook defined spatial computing as "the beginning of a new era of computing technology":

Just as the Mac brought us into the era of personal computing and the iPhone brought us into the era of mobile computing, Apple Vision Pro will bring us into the era of spatial computing.

The above-mentioned products all correspond to different interaction habits: Mac has a keyboard and mouse trackpad, iPhone has index fingers and thumbs, and Vision Pro has the entire body.

Vision Pro's spatial interaction requires the cooperation of hands, eyes and mouth: the eyes act as cursor positioning (mouse cursor), the fingers are responsible for determination (right mouse button), and voice is used as the input method (keyboard).

Customize a new system for your eyes

In actual experience, we found that "eye movement" has a very high weight in Vision Pro, so that every detail of the UI design is centered around the eyes.

Every time you wear Vision Pro for a long time, the pressure on your apple muscles is indeed a bit high, but your eyes are not too tired. The main reason is: the interface is designed to look like your eyes.

First of all, all Vision Pro interfaces will appear in the center of the screen, which is the most comfortable area for viewing. It reduces neck and body movement and reduces wide-range eyeball movements.

Secondly, the default distance of all interfaces when they first appear is moderate, not too close to see everything, nor too far away to see everything; in addition, discomfort often comes from the eyeballs constantly adjusting the focus, and many drivers are not used to streaming media. The rearview mirror is the same thing.

Vision Pro also made a dynamic size adjustment of "near small, far large" in the same interface to reduce the number of times the human eye focuses and try to adjust the focus adjustment interval to a small range.

For example, when staring at the web page bar above Safari, there will be a prominent highlighting effect. In fact, the interface behind it is also "blurring + retreating" simultaneously. This linkage can minimize the shift of focus.

Precisely because of the "highlighting" effect, we can easily identify the selected target, eliminating a lot of "finding the cursor" process.

Moreover, if you stare at the menu bar of the main interface for a long time, detailed information will appear. This copy of the "hover" effect of 3D Touch long press can hide a lot of text and make the interface as simple as possible.

In the past few days of actual testing experience, we also discovered that Vision Pro attaches great importance to the "center".

On the one hand, Vision Pro uses lots of circles (circles, pills, rounded rectangles) and soft interface curves. The lack of edges and corners is not only less glaring, but also facilitates accurate identification, because the round shape can subconsciously focus the eye on the center of the target.

Moreover, Apple also stipulates the size and spacing of each icon in visionOS: the total size of icons used for eye interaction cannot be less than 60pt, and the interval between icons needs to be more than 16pt. Such requirements can help the eyes to locate quickly and also reduce the possibility of "misreading".

Furthermore, the spatial layout of the multi-interfaces is spherical, rather than neatly arranged on the same plane like a photo wall. Several applications are opened at the same time and placed anywhere in front of them. No matter how they are arranged, the vertical intersection point of all interfaces is It's our eyes.

Everyone who has experienced it has reported that they feel like they are standing in the center of the Vision Pro ball.

Gesture operation, strategizing

Gesture operation has a certain learning threshold.

Vision Pro announced six commonly used gestures at the developer conference, which can cover most operating scenarios.

These six gesture interactions will be better understood using the analogy of operations on the iPhone:

  • A tap of your thumb on your index finger: equivalent to a tap on your iPhone screen
  • Touch your index finger twice with your thumb: equivalent to double-clicking the iPhone screen
  • Pinch your thumb and index finger together for a period of time: similar to pressing and holding the iPhone screen for a period of time to enlarge the text
  • Pinch and drag with thumb and index finger: used to scroll and move the window position
  • Pinch the index finger and thumb of both hands at the same time, and move it inward/outward at the same time: adjust the window size
  • Pinch the index finger and thumb of both hands simultaneously and rotate: Manipulate the display direction of the target

The device actually has a wide recognition range for opponents. If it is placed on the lap or on the table, the corresponding command can be triggered in the most natural posture without the need to deliberately hold it high in the air.

The biggest difficulty in gesture operation is to imitate the effect of direct touch control as much as possible in the absence of real touch. Thanks to the optimization of details, Vision Pro does a good job in simulating reality.

Take the virtual keyboard as an example. When we reach out and touch the keys, the brightness, position, and size of the designated letters will change accordingly, and there will also be Apple's ancestral decompression sound effect.

Through a series of motion effects, Vision Pro makes up for the original missing sensory information and restores the feedback that real objects should have to a great extent. Objects that do not exist in reality can begin to be seen and touched.

For example, A Cozy Loóna, a game suitable for all ages, allows you to deeply feel that "the thing is in your hand" when you put the corresponding object in the right place.

Pinch with two fingers to select and change the position of the target object. Together, you can rotate, place, and zoom in and out of the entire 3D virtual space at will.

The greatest significance of spatial operation is that when the game enters professional application, it becomes out of reach and within reach.

For example, in JigSpace, we can disassemble 3D models of engines and spacecrafts with zero threshold and zero cost, and analyze product details layer by layer from a developer's perspective.

There have been similar naked-eye 3D screens or simple 3D model interactions at previous Consumer Electronics Shows, but it is rare to see software and hardware that seamlessly integrate display effects and operational interactions. Vision Pro relies on numerous sensing elements to scan spatial information and combine them. The position information, shadow processing, perspective relationship, etc. of virtual objects are represented nearly realistically.

This provides a new idea for industrial design and interactive 3D knowledge popularization. At least based on personal experience, JigSpace is definitely a space application that will please mechanical fans.

There are many similar professional applications in Vision Pro, and the medical software Surgical AR ranks among the top three.

This is an AR App based on Microsoft HoloLens 2. Doctors can see the patient's MRI and CT image holograms through the software and observe the internal structure of the lesion.

In the past, medical imaging was to lay out several cross-section pictures on one photo. Combining them for analysis required a certain degree of "brain supplementation" from the doctor. Surgical AR breaks the two-dimensional limitation, whether it is daily examination or intra-operative comparison. , medical personnel will gain a more intuitive three-dimensional perspective.

Coupled with the spatial capabilities of Vision Pro, you only need to move your hands and use the most natural movements to complete the most professional operations: several interfaces are arranged randomly in front of your eyes, and the 3D image can also display scanning results at different positions by adjusting components.

Previously, SurgicalAR has been certified by the US FDA (Food and Drug Administration) and was named one of Time Magazine's Best Inventions in 2023; now in Vision Pro, its value has been further enhanced: in a lighter device , and has more customization possibilities.

In these entertainment and professional applications, we gradually understood the significance of hand-eye operation for Vision Pro:

Although the technical principle of Vision Pro is based on VR, after a week of experience, it can be clearly felt that Apple is trying its best to integrate VR and AR technologies. The unification of the two is through the spatial operation of hand-eye coordination, and our limbs have become the connection between the two. A bridge between technical camps.

Audio-visual entertainment, killer application

Many reviews have mixed reviews on Vision Pro's other features, but they remain highly consistent in terms of viewing effects: stunning. This was also one of our most curious highlights.

After in-depth experience with the viewing software, we discovered the reason for the stunning display of Vision Pro.

A high-quality screen is the foundation for all excellent imaging. The famous disassembly team iFixit took apart Vision Pro and observed it with a microscope. It was found that the pixel value of the screen area is 3660×3200px, which is equivalent to 2.3 million pixels per square inch.

For a more specific analogy, Apple crammed 54 pixels into one pixel on the iPhone 15 Pro Max on the Vision Pro.

This also explains why we can see natural landscapes with 1:1 imaging effects in "virtual natural landscapes", as if clouds, snowy mountains, deserts and even the moon's surface are right in front of your eyes.

Good picture quality + good applications are the next step for Vision Pro to build an ecological building. Among them, the upgrade of sports viewing experience is one of the most obvious changes.

If you often watch sports events, there are several pain points that you have probably encountered:

  • The game information is incompletely displayed, and data such as scores, assists, fouls, timeouts, efficiency values, etc. cannot be displayed on the same screen;
  • It is difficult to watch different scenes happening at the same time in parallel on one device;
  • The content of the competition is almost entirely decided by the director, and the audience generally can only passively accept it.

Vision Pro perfectly breaks the rules with a highly customized space page layout. Everything you see is a screen. It also breaks through the single screen of traditional electronic devices and builds video groups.

NBA App divides the game interface into four areas, and places live game broadcasts, team information, player data, game times, etc. in different locations according to your own needs. In Vision Pro, we are no longer passive spectators, but A guide that looks at the big picture.

Furthermore, benefiting from the blessing of spatial audio and video, Vision Pro has also opened up the next form of cloud performance.

The immersive concert software AmazeVR brings our perspective to the stage, no longer limited to the perspective of sitting in the audience. The dancers and singers seem to have opened a special performance just for you. The ultra-4k resolution + spatial audio restores the sound effects and stage beauty of the scene to a great extent. It is also the number one contributor to making the cloud performance difficult to distinguish between authenticity and fakeness.

We are amazed at Vision Pro's ability to restore the artist's real-life image. It is no exaggeration to say that she is standing in front of you. When the camera zooms in, the sense of reality can even make you a little scared.

At present, the works of artist groups such as T-Pain, UPSAHL, Megan Thee Stallion Megan Thee and AESPA have been included in AmazeVR.

In these days of experiencing the Vision Pro application, I have a lot of feelings.

First of all, we strongly recommend that everyone who is interested in Apple’s headset wear it and experience it in person. Neither the promotional video on the official website nor our text summary can reproduce the true display effect of Vision Pro.

In addition, although Vision Pro's price and adaptability ecology still cannot enter the lives of most people, and it has not become the ultimate form of the next consumer electronics era, its future is worth looking forward to.

As a tool, consumer electronic devices need to accurately locate their usage scenarios at the beginning of design, find suitable people and application space, and then allow new technologies to reach the mass market.

Many people are discussing whether Vision Pro will become the next iPhone, just like the concerns about the positioning of iPad back then. At least judging from the current usage scenarios, it will not and cannot replace the status of mobile phones, because the two have never been in the same competition. Competitive products.

The portability of the iPhone is the biggest advantage. It can be carried around, taken out at any time, and used standing, sitting or lying down. It is not bound by time and space. As an "all-rounder but not an expert", the iPhone or mobile phone can also be used They are not competent for all jobs. The simplest example: we almost don’t see our colleagues in the office using their mobile phones to make forms, write articles, and edit videos.

The same goes for Vision Pro. While identifying its applicable scenarios, it is also answering a deeper question: What is the use of spatial computing?

From the several major positioning given by Apple, it is not difficult to see that Vision Pro or spatial computing is most useful in "immersive" scenes: giant screen viewing, a one-to-one presentation of Tyrannosaurus Rex, and NBA on-court viewing. Game perspective, web pages and interfaces that can be placed in any position… These are unique experiences that a mobile phone or even other devices cannot give users.

More importantly, this Apple headset perfectly connects people and space, reality and virtuality. We use our eyes to navigate, use our hands to control, and use voice as input, and then use the ultra-4K display and three-dimensional rendering to achieve Output, together build the "human-oriented" interactive ecosystem of Vision Pro.

# Welcome to follow the official WeChat public account of aifaner: aifaner (WeChat ID: ifanr). More exciting content will be provided to you as soon as possible.

Ai Faner | Original link · View comments · Sina Weibo