Struggling, overturning, reshaping, Apple AR glasses are more exciting than you think

Apple's development of mixed reality devices (AR/VR) is probably the most "well-known" secret in the technology industry. Every once in a while, there will always be related news leaked.

The outside world believes that this will be the next-generation personal terminal used by Apple to replace the iPhone. In the future, people will no longer rub the glass with their fingers, but will become "pointing fingers" at the air.

However, after so many years, the iPhone has evolved from the iPhone 8 to the iPhone 14. This next-generation terminal is still "half-covered" and always living in rumors.

Just when people are about to lose their freshness in mixed reality, news about this mysterious device has recently increased.

There are growing reports that Apple's mixed reality product may be tucked away in a secret lab in Cupertino, and it's preparing to meet the public.

Parker Ortolani, who guessed the name of macOS Monterey based on the trademark registration information last year, recently discovered that Apple has recently used shell companies to register and apply for the RealityOS trademark around the world.

This is a commonly used secrecy method before Apple's conference. Before that, Apple used a shell company called "Yosemite Research LLC" to register trademarks such as Yosemite, BigSur, and Montery (all versions of macOS).

Parker Ortolani found that "Realityo Systems LLC", a company with the same registered address as "Yosemite Research LLC", applied for the RealityOS trademark in Brazil, Uruguay and other countries.

Combined with the information that developers have found related to RealityOS in the development log, Parker Ortolani thinks there is reason to believe that this is the operating system that will be applied to Apple's mixed reality devices, although it is unlikely to be unveiled at WWDC in a few days. But at least it shows that we are one step closer to unveiling its mystery.

Just from the naming of RealityOS, the operating system should focus on connecting the interaction between virtual and reality. People may be able to apply the interaction logic of the real world to the virtual world, which means that Apple may be common in us. Bring some new changes to the VR headset form.

So what exactly does this mysterious device look like? From a recent report by The Information about Apple's internal R&D controversy, we may be able to find some clues.

walking in the mud

According to The Information, the issue of how to break down the barriers between virtual and reality has been debated within Apple.

As early as 2016, Apple's VR/AR team has used immersive VR experience to obtain the support of Apple's board members and began to conduct research on related products.

This is an innovative project that started from scratch, and some insiders have rated it as "Apple's most daring new product innovation since the iPhone", but from the perspective of the progress, it is much more bumpy than the iPhone.

▲ Picture from: Time

Some people familiar with the matter said that although Apple’s current CEO, Cook, is supporting the project, Cook has not paid enough attention to this new product compared with Jobs’ deep involvement in the iPhone project in the past, which made it difficult for the project team. Engage with the Mac and iPhone teams for development resources.

Apple's investment in AR/VR began with the acquisition of German start-up Metaio in 2015 and the hiring of executive Mike Rockwell around the same time.

Metaio was developed from a project within Volkswagen. Apple’s acquisition was originally intended to use Metaio’s AR technology to develop an autonomous driving system. Then Facebook acquired Oculus VR in a big way. Silicon Valley set off a virtual reality boom, and Apple began to use Metaio to form AR/VR project team.

▲ Mike Rockwell

Leading the hardware team is the new addition of Mike Rockwell, who was previously Executive Vice President of Dolby Laboratories and a no-compromise expert in visual and high-fidelity audio.

To build a piece of hardware from scratch, Mike Rockwell needed to enlist support from other hardware departments, such as the touch sensor, camera, Apple chip and display teams.

In fact, the development of Apple products including iPhone and iPad is completed by the cooperation of these departments responsible for various functions, rather than being divided into "iPhone group" and "Mac group". It is not difficult to understand why Apple's different products are different. There is a linkage between them that surpasses their opponents.

Mike Rockwell's biggest challenge is to get the support of the two strongest internal departments – Apple's industrial design team and human-machine interface team, and leading these two teams is Jony, who has the most voice in Apple in the post-Jobs era. Ive.

The original idea of ​​Mike Rockwell's team was to create a virtual reality headset, but it was quickly opposed by Jony Ive's team. The Jony Ive team believes that the helmet-shaped product form cuts off the connection between people and the external environment, alienates people from each other, and does not look "fashionable" from the outside.

▲ The picture shows Facebook's display technology, for reference only

In order to make the industrial designer team buy it, the Mike Rockwell team had thought of many improved solutions, such as adding cameras on the outside so that the wearer could see the outside world, but in the end Jony Ive's team chose another solution – in An external-facing screen is installed on the device, allowing outsiders to see the wearer's eyes and expressions through the screen.

This may reduce the wearer's sense of alienation from the environment to some extent, and may only make the wearer look more bizarre, but in any case, Mike Rockwell's team still has the support of the industrial design team to make this product not. As for stillbirth.

push down and start over

After clearing the development hurdles, Mike Rockwell's team still needs to answer the next question – what should the shape of the product look like?

Mike Rockwell believes that Apple should develop a powerful head-mounted product that is different from its rivals such as Oculus. It must have an excellent level of battery life and display performance. Therefore, this product needs to be connected to a powerful "base station" device. Just like in the early days of VR development, a powerful PC was required.

According to the news reported by Bloomberg, the chip used in this "base station" device is likely to be the M1 Ultra that was applied to Mac Studio some time ago. Its powerful processing performance is indeed enough to bring all kinds of VR all-in-one machines to shame. High-quality virtual experience.

▲ Designer's self-made concept map Image from: AntonioDeRosa

To Mike Rockwell's surprise, however, Apple's board executives preferred Jony Ive's solution: an all-in-one product that doesn't require a base station. Although the quality of the display content is not as good as the former, it is smaller and easier to carry.

At the beginning of the project, the team led by Jony Ive has been promoting the independent version of the headset. Jony Ive believes that how to make users willing to wear this product for a long time is the first problem to be solved by his industrial design, so he strongly insists on making this product This device can get rid of the shackles of the base station.

▲ The difference between the pictures is probably like this

At the executive's presentation, executives including Cook, after comparing the lifelike virtual images of Mike Rockwell's team's base station suit with the cartoon-like animations of Jony Ive's team, still voted for more The light latter.

This was a blow to the Mike Rockwell team, not only because the prototype that took them a lot of time to make was rejected, but because the department, including the chip team, had already prepared them with the custom parts they needed, which obviously You can't push it down and start over.

▲ Apple's chip star Johny Srouji

According to information disclosed to The Information by people familiar with the matter, Apple's chip team led by Johny Srouji has prepared an image processing chip code-named Bora and a wireless chip code-named Golay for the headset to drive the internal Displays and data transfer between devices and base stations.

The other chip, code-named Jade, is a computing chip specially designed for base stations, which is now the M1 Ultra.

Now that the base station solution is abandoned, it means that all kinds of complex components must be concentrated on the head-mounted device body, which will be a great challenge to the battery life, computing performance, and heat dissipation of the device.

If the product overheats, the effects of previously designed algorithms such as hand tracking, computer vision, 3D models, etc. will be greatly reduced, so both hardware and software will require a drastic adjustment.

This is also one of the important reasons why Apple's headset products have been delayed again and again.

After Jony Ive's departure, he is still involved in the design and development of this product as a consultant. Jony Ive's insistence on the product has also made this product increasingly complex.

▲ Magic Leap

For example, Jony Ive is not satisfied with the design of the battery installed on the device of the product prototype, and prefers the structure of installing the battery on the "glass leg" like Magic Leap; in order to allow the user to have sufficient contact with the outside world, the glasses are equipped with 14 A camera to track the user's facial expressions, the chip team made the Bora chip to process this complex image information.

These designs have brought a lot of problems. Due to the delay between the Bora chip and the main processor code-named Staten, the wearer can easily feel nausea and discomfort. In the end, the chip team needs to develop a streaming media encoder. To a certain extent alleviate this problem.

▲ Designer's self-made concept map Image from: AntonioDeRosa

In general, Mike Rockwell's team still has many problems to solve, whether it is software or hardware, but according to the latest news from Bloomberg, this mysterious device has been shown to the top executives on Apple's board of directors, and the product as a whole has entered accelerated development. stage.

After bypassing a large circle, Apple's internal development steps are finally close to the same. "Apple's breaking news experts" including Bloomberg and Guo Mingji believe that this product will be launched as soon as the end of this year or 2023. The only problem now is After nearly 7 years of waiting, is everyone still interested in a virtual headset that will cost more than $3,000?

be patient

Although this mysterious device has not been able to land for a long time, in fact, in the new Apple products in recent years, we can already see the fruit from this big tree.

In addition to the M1 Ultra chip mentioned earlier, the Staten processor on The Information device will likely debut on devices such as the new Macbook Air and iPad, which is the M2 processor that was previously speculated.

At present, the combination of virtual and reality is still Apple's future. Ming-Chi Kuo believes that Apple will release an AR device based on the shape of smart glasses in 2025 after releasing the head-mounted display device currently under development, and in 2030. AR devices in the form of contact lenses will be released, and intelligent computing will enter the "invisibility era".

▲ Designer's self-made concept map Image from: AntonioDeRosa

Of course, all of this will not make sense until people wear this "iPhone on the face", otherwise, it will still be the next-generation iPhone that can beat the iPhone.

Stop talking nonsense.

#Welcome to pay attention to the official WeChat account of Aifaner: Aifaner (WeChat: ifanr), more exciting content will be brought to you as soon as possible.

Love Faner | Original link · View comments · Sina Weibo