Why is the iPhone’s under-screen camera more difficult to build?

After each new generation of Apple is released, there are always two kinds of heated discussions. One is the sharp comments on the current generation of products, and the other is the bold prediction of the next three generations of iPhones.

In the six months since the iPhone 15 hit the shelves, the under-screen camera has become the focus of many news media.

Some time ago, Macrumor broke the news that Apple's suppliers in South Korea have begun the development of under-screen cameras for smartphones. On the other hand, a netizen immediately pointed out that there were similar rumors five years ago.

We don’t know when we’ll see the iPhone pictured here, but until we see the real thing, we know this debate will continue.

Compared with Android, which is making front-facing cameras smaller and smaller, and even rushing to try "true full-screen" solutions, Apple is like an audience sitting on the sidelines, in the same time and space, but not participating in the game, and still staying away from the game and sticking to the screen. That dark spot at the top.

To understand why Apple has not launched the much-anticipated under-screen lens, we first need to clarify a question:

Why are front-end modules so important to Apple?

The more furniture, the bigger the house

There is actually a secret hidden in Apple’s bangs.

The front module includes 8 sensing components including "infrared lens", "flood sensor element" and "speaker". Looking at it again from this perspective, it seems that the existence of Qi Liuhai can be understood.

The infrared lens, dot projector and front camera are the fundamental reasons why Apple insists on retaining large modules.

Although the entire iPhone 15 series uses a shorter "Smart Island" and many sensors have been rearranged and designed, these three brothers still stand in the center of the "island".

The iPhone has always had its own unique understanding of photography. The evaluation website DXOMARK ranked the iPhone 15 Pro Max at the top of the list for the past two years in terms of "selfie effect". Even last year's 14 can still rank among the new phones in 2023. Achieved an excellent result of tied for second place.

▲ Picture from: DXOMARK

The iPhone's paper data has never been the best, but the shadow of Apple is always indispensable in all horizontal comparisons of photography evaluations; its photo style is not the most pleasing, but its natural imaging effect ranks among the top in all major lists. There are quite a few highlights on the list.

In this era of increasingly frequent online interactions, selfie live broadcasts, video calls, and online meetings all require the participation of the front lens. Therefore, the iPhone has placed the weight of front-facing photography at a high level.

The twin brothers of "infrared lens" and "dot matrix projector" merge into the gatekeeper of Apple's ecosystem – Face ID.

It’s common to hear Apple users complain about the iPhone’s battery life and signal, but it’s rare to hear rumors about Apple’s poor security. This is due to Apple’s deep efforts in security verification.

Face ID joined the Apple ecosystem as early as 2017. At the same time, the iPhone also completely abandoned the Touch ID that it led.

The security of face recognition is unmatched by fingerprint unlocking. The unlocking process of "3D Structured Light Dual Camera" is a bit awkward to explain, so we will only briefly explain the key parts.

The "payment level" facial unlocking is the biggest compliment to Face ID. Even the treasury can rely on the technology provided. Of course, there are some insurmountable thresholds, and the "dot matrix projector" is a major contributor.

The invisible IR Dot is projected onto the person's face through the lens, forming a three-dimensional model with 30,000 dot arrays on the face. The virtual mask is scanned with an infrared lens and matched with the dot matrix information in the mobile phone to confirm the unlocked person. Is it the owner of the machine?

Facial information is actually 20 times more secure than fingerprint solutions. The probability of unlocking the phone with the same fingerprint is one in 50,000, but with Face ID, this probability is reduced to one in a million.

After years of upgrades to facial recognition, users can use it when wearing masks, glasses, and hats, but photos with fake faces or even 1:1 face models cannot be unlocked.

In order to prevent the owner from being forcibly unlocked by others holding the phone while sleeping, Face ID also adds "attention detection" technology. Only when the owner's attention falls on the phone screen, the iPhone will recognize the unlock command. The above unlocking process can be carried out.

Features such as beautiful appearance and full screen are all items that can be compromised in the face of "security" and "image".

Apple’s stubbornness will keep bangs and pills around until it encounters “perfect” under-screen technology.

However, several patents that have been revealed one after another on Apple's patent website seem to have given some signs of a true full screen that is far away.

The three-axe that blocks the gap

ZTE, Xiaomi, OPPO, Samsung, etc., which have launched under-screen cameras, have different names for this technology, but the underlying logic to achieve the "disappearance of the lens" is similar.

Since the OLED luminescent material is opaque, if you want to be able to see the screen and allow the lens to see us under the screen, you need to increase the light transmittance of the screen in the area of ​​the front lens, whether it is to let the pixels The gap between them becomes larger, or some pixels are deducted. The final effect is that the light can smoothly pass through the gap and enter the lens.

Apple also uses the same principle to design under-screen lenses. However, compared with previous manufacturers, iPhone under-screen lenses have two inherent problems to face:

– iPhone equipped with FaceID has a larger front module that needs to be hidden
– Larger modules require larger light transmission areas

The Red Magic 9 Pro is currently the best machine on the market that controls the resolution difference between the "under-screen area" and the "main screen area". In this experience, we can't feel the difference in most scenes. The lack of image quality is abrupt, but this is also an effect that can only be achieved on the premise of only one camera.

In response to this, Apple used the "integration + hiding" method in the patent to minimize the impact on the screen resolution.

In the patent document code-named 20240077648, a technology called "integration of dot matrix projector and flood illuminator" is mentioned.

The flood illuminator is the infrared lens mentioned above, which is used to emit infrared light to help the phone recognize faces and determine whether the user is looking directly at the screen to ensure that only the user can unlock the screen.

Now, Apple can combine the flood illuminator and dot projector into a single compact projection module, reducing the area of ​​the pair of sensors to nearly half its original size, thereby reducing the need for under-screen area.

The "planar optical waveguide" in another patent is an extension of the above-mentioned integration technology.

Modules such as ambient light sensing, image capture, gesture recognition, and proximity sensing in traditional front-end modules need to emit or receive optical radiation in different spectral regions through various physical apertures in order to complete corresponding operation instructions, and Perception of the environment and users.

That’s why most of the current full-screen mobile phones leave a black spot on the screen. It’s not because they like it, but because they have no choice.

However, planar optical waveguides seem to solve this problem. This technology utilizes planar optical waveguides on mobile phone displays to guide radiation across the display surface without interfering with display functionality.

Get rid of the obscure and esoteric professional words: with it, most of the sensors (except the camera) will not affect the use even if they are placed under the screen, just like the fingerprint unlocking on the mobile phone.

In theory, Apple's combination reduces the size of the front module and the area required to reduce the screen resolution. The only thing that needs to be compromised with the main screen is a front optical lens.

Therefore, this rendering of iPhone 16 is not unreasonable.

But to be truly comprehensive, the lens must be a topic that cannot be avoided. After all, the resolution above the module is still different from the main screen. Even on the Red Magic 9 Pro, you will still see a "screen window-like" appearance in the light-colored highlighted interface. ” of the under-screen area.

And Apple’s third ax is specifically designed for this.

First of all, through the special "W" arrangement of pixels, the light transmittance is increased, the diffraction phenomenon is reduced, and "halos" and "fog" appear in selfies.

Then, components in this area (such as touch sensor metal) can be moved toward or away from their adjacent pixels in different usage scenarios, thereby easing the gap between the "pixel removal area" and the "full pixel density area" display difference.

Just judging from the description of the patent document, Apple has achieved variable resolution in the physical sense.

While flipping through the patent documents, I also discovered a "caution" from Apple.

The schematic diagram does not give the specific location of the area under the screen, but lists possible locations in sequence. This smoke bomb is no less than every year before the release of a new iPhone, the entire Internet starts a carnival of speculation about the shape of the rear module.

In the past two years, major manufacturers seem to have slowed down their exploration of full-screen displays.

On the one hand, after years of attempts, fantastic ideas such as front and rear double-sided screens, flip lenses, sliding screens, etc. have become fleeting historical products. The ideas are very fresh but the market does not pay for them. Under the current technical level, , the ideal mobile phone that is "portable in weight", "high-definition in pixels" and "complete in screen" can only satisfy two of them.

The item that is discarded in this impossible triangle also forms the characteristics of different brands and series.

On the other hand, the incomplete screen is not entirely a drawback. Returning to the top of the screen, I suddenly discovered that the existence of hollows, bangs or water drops not only gives room for the front module to play, but also provides a natural habitat for the status bar.

On the day when off-screen solutions become popular, where will the status bar information go? What tricks will manufacturers come up with regarding the removal and retention of these icons? It’s quite worth looking forward to.

However, people’s pursuit of perfect screens has kept this protracted “space tug-of-war” going on. What will the mobile phone screens of the future look like? In fact, we already have the answer.

Former Apple chief designer Jony Ive believes that the "infinity pool" will be a form of mobile phone screens in the future. From Samsung Galaxy Note Edge to Huawei Mate 30 pro, this concept has been unintentionally used throughout, but it can really be achieved visually. Boundless is the Xiaomi Mix α placed in the showcase, but this machine is still too far away from mass production, or in other words, from us.

These patent documents also give us a little hope of a "full-screen iPhone", but Apple, which has always been willing to follow through, has been a little slower on the road to finding the perfect screen.

Just like Steve Jobs, who would rather have an empty home than make do with furniture he doesn’t like, the design of the iPhone seems to have always maintained this concept, always being unique and always one step behind, but of course it will also make changes. time, attracting the attention of the world.

More importantly, when you put the iPhones in recent years together, you can find that they have actually made progress quietly.

# Welcome to follow the official WeChat public account of aifaner: aifaner (WeChat ID: ifanr). More exciting content will be provided to you as soon as possible.

Ai Faner | Original link · View comments · Sina Weibo