How will Apple make AirPods with cameras?

Apple’s design of smart products is letting itself go.

According to Bloomberg reporter Mark Gurman, Apple has two bold ideas in its “future device” planning:

The first is the HomePod with a screen, which is under development and will integrate multiple functions such as Apple TV and FaceTime; the second is the AirPods equipped with a camera. The low-resolution lens will be used to collect surrounding information and process the data through AI. "Assist people in their daily lives."

The two concept products are still in the exploratory stage, but I am still surprised by this "avant-garde" combination and the contrast with Apple's "simple" concept, especially the headphones with lenses, which are a bit puzzling.

▲ Pictures generated by AIGC

When browsing relevant literature, an Apple patent document code-named US11432095B1 seems to prove the feasibility of the strange combination of "lens + headphones".

Virtual sound field is inseparable from the lens on the headset

Let me summarize first. In this patent document, the camera is not added to AirPods (Pro), but to AirPods Max. In addition, the technology involved is not the "AI function" revealed in the news, but is already on Vision Pro. The beginnings of spatial audio.

First, we need to clarify a principle: How do people identify the location of a sound?

The layout of the two ears on both sides of the head, in addition to being beautiful, plays a more important role in giving us the ability to hear and distinguish the position of sounds.

On the premise of maintaining the same horizontal line and axial symmetry, try to be as far away from each other as possible, so that when hearing the same sound source, the sound information received by each ear is different from the other, mainly including the binaural time difference. There are three types: intensity difference and frequency difference. Generally speaking, these differences are very small.

▲ Picture from: Medium

Take the time difference as an example. Most sounds in life arrive at two ears at different times. The difference is often only about a hundred microseconds (the time it takes to blink 1/10 of an eye). We cannot perceive it, but the brain can detect it from the difference. Among these almost negligible differences in sound source information, it is possible to determine which one was heard first and which one was heard last, thereby creating a sense of "where the sound came from."

For example, when we activate the "find mobile phone" function on the watch, our ears can successfully detect that the mobile phone has fallen into the crack of the sofa through the differences in various sound sources.

Many cinemas and home audio-visual equipment take advantage of the extremely sensitive characteristics of the human ear to use multi-channel combinations in a limited space to achieve surround sound, making the movie immersive at the sound level.

The above information contains two core elements:

  • People can distinguish the location of sounds through differences in sound source information.
  • Sounds emitted from different locations in space create auditory differences

This US11432095B1 patent document explains how Apple allows users to hear sounds from all directions through headphones only, as if they have a complete set of surround sound equipment.

This technology is called "virtual sound field".

A device with a virtual sound field can build several virtual speakers in it. Through the physiological characteristics of hearing and positioning, each speaker can be placed at a fixed position in the user's space, thus realizing "a pair of headphones is a set of speakers" Effect.

So, what role does the camera play in this?

Technology blogger He did an experiment in the Vision Pro review video: after placing an application window of Vision Pro in a fixed position, he walked more than 300 meters away and then returned to the original place. The position of the window was almost invisible. changes occur.

Such precise positioning comes from the visual positioning system on the front of Vision Pro, which uses infrared rays, cameras, etc. to scan the physical information of the space and records the physical position of the window in the reconstructed space, thereby achieving amazingly accurate positioning.

The camera on AirPods Max also uses the same principle to record image information, and then the tracking processor analyzes the user's head position information in real time, and can use the mapping of the display and the surrounding environment as a reference to track the user's head, achieving dual Assure.

Paired with the camera in the Apple TV box, the position of the virtual speaker in the room can be determined, and there is a basis for where several sound sources are placed and where they are emitted.

In short, the lens gives AirPods Max a sense of vision, and the virtual sound can closely restore the effect of physical speakers.

Rather than hearing it, I would rather say that I can "see" the sound.

Where and how many virtual speakers are placed can be set by the user. 5.1 or 7.1 surround sound no longer needs to be purchased at a high cost. It can even be adjusted in real time according to the size, position and listening area of ​​the TV (including the geometry of the room). ;At the same time, the user's sitting posture will also affect the position of the virtual speaker in the sound field.

The patent document also states that in a virtual sound field, a separate set of one or more virtual speakers can be generated and assigned to multiple users with the same device.

Even if the two people swap seats or move around the room, the virtual sound field will adjust the sound effects of each virtual speaker according to the user's location so that each sound source remains stationary relative to the user.

In addition, the patent also specifically explains the "display". The viewing screen in the virtual sound field is not limited to traditional TVs, but devices with screens or projections can implement such functions, such as laptops and tablets. Computers, projectors, desktop computer screens, etc. (needs to be used with an Apple TV box with a camera).

In other words, why can’t your next set of surround sound be headphones?

Dream first, then pave the way

From this point of view, the virtual sound field built by AirPods Max with lenses is more like the technology decentralization of Vision Pro.

In previous articles, we comprehensively analyzed the "spatial computing" that Vision Pro builds through spatial video, spatial audio and spatial interaction. However, the high price and few ecological adaptations still make it an ideal product. A conceptual product that wants to enter the market.

The spatial computing era opened by Vision Pro is a distant idea. It is a collection of cutting-edge display technology, new interactions, and computing capabilities. It is more like a goal and a future.

The AirPods Max in the patent document uses a true "consumer-grade" product to realize spatial audio in the living room. Compared with Vision Pro, it is more comfortable to wear, more convenient to use, and of course the price is more reasonable.

In the process of moving towards the future, distant goals are divided into steps. Gesture interaction on Apple Watch and spatial video recording that iPhone starts to support are all steps that have already been climbed. Headphones with lenses are another branch, which also allows spatial audio to find a solution that is more suitable for daily use and easier to implement.

Of course, will the lens + headphone solution really come to us? Even if Apple really integrates the two, will it move towards using AI to assist people's daily lives as mentioned at the beginning of the article? Or go back to the technical route of virtual sound field in the patent document? It is difficult for us to get a definite answer, but as several new technologies continue to flood the market, an option that best suits the times will eventually emerge.

Before that, rather than getting results, constantly trying boldly is the most valuable thing at the moment.

In 1972, Bernard D. Sadow, then the former vice president of a luggage company in Massachusetts, removed the four casters from a large wardrobe, installed them at the bottom of a self-branded suitcase, and then tied them to the top of the suitcase. Got a belt.

▲ Picture from: Facebook

This is how the world's first rolling suitcase was invented.

Looking back today at the time when the trolley suitcase was invented, it’s somewhat hard to believe.

First, more than 7,000 years have passed since the invention of the box, and more than 5,000 years have passed since the invention of the wheel. The two inventions that have accompanied most of the history of human civilization have only just been used in combination;

Secondly, Apollo 11 successfully landed on the moon two years ago, marking the first time that humans left footprints on other planets. In the same country, the invention of the rolling box was even later than the moon landing. Before that, people still maintained Suitcase Habit.

Sadow immediately applied for a patent for his new invention that year and named the sliding luggage Rolling Luggage.

▲ Picture from: Facebook

Innovation sometimes really requires inspiration and luck, but the story doesn’t end here.

Although the Rolling Luggage was very popular when it was first invented, due to design flaws, the center of gravity of the luggage was unstable. It often tipped over when being pulled, and occasionally hit the ankle of the person pulling the luggage.

It turns out that this is indeed a good idea, but the Rolling Luggage is not a good product.

In the following more than ten years, although companies such as Samsonite optimized the original Rolling Luggage, it was still inconvenient to use.

It wasn't until 1987 that Robert Plath, a retired captain from Northwest Airlines, was tinkering with a suitcase at home. He erected the suitcase and installed wheels and a drawbar on the same side, and the prototype of the modern trolley suitcase was established.

Trolley luggage is such a matter of course today, but this does not mean that the technology itself is obvious, because innovation and the limitations of innovators are closely related to the background of the times.

At that time, long-distance travel was not as common as it is today, and there were not many items to carry when traveling. Trolley cases were not strictly necessary. Cases and wheels had indeed existed for a long time, but case materials and bearing technology that were both light and durable had not yet appeared. .

The demand is not high and the technology is insufficient. Even if the technical components that make up a new invention exist early, it will not become a successful invention because behavioral needs, technical conditions and market factors have not converged.

▲ Picture from: Yahoo

For boxes on wheels, this combination was not really possible until the second half of the 20th century.

Regarding "lens + headphones" or other strange combinations, the current questioning and ridicule only proves that the exploration of future technology has just begun.

But no matter which route or plan is chosen, we are all taking steps towards the future. Without these fantastic ideas and bold practices, we might still be carrying that heavy box today.

# Welcome to follow the official WeChat public account of aifaner: aifaner (WeChat ID: ifanr). More exciting content will be provided to you as soon as possible.

Ai Faner | Original link · View comments · Sina Weibo