With the advent of the epidemic, the face unlocking function has degraded the noble head.
When masks have become a must-have item for us to go out on the street, we always have to go through the cumbersome process of "facial recognition failure"-"input password" when unlocking the phone. This makes people miss the fingerprint recognition.
In order to optimize the face unlocking experience, earlier this year foreign entrepreneur Danielle Baskin launched a mask with facial information. The product extracts the user's facial information and prints it on the outside of the mask. After the user wears the mask, they can piece together a complete face.
▲ Actually a bit scary. Picture from: djbaskin
However, the unlocking success rate of this product is not yet clear, and there is no large number of sample verifications. So how can people's face recognition system stop being bothered by masks?
Some netizens gradually discovered that as the time of wearing a mask becomes longer and longer, the mobile phone seems to have found "experience" from repeated face unlock failures, and is gradually able to recognize the self wearing a mask successfully.
Relying on this idea, some technology bloggers also shared more efficient tutorials, such as repeatedly unlocking while wearing a mask. If the face recognition fails, immediately enter the password . Repeat this action for about 30 minutes, and the phone can recognize the person wearing the mask. By myself.
▲ Successfully unlocked the face while wearing a mask. Picture from: Farhad Usmanoff
However, in the course of practice, netizens said that the "learning" speed of different models is different. Some people have repeated the above actions for 20 minutes and have succeeded, but some people have repeated them thousands of times, and their mobile phones still cannot recognize themselves wearing a mask.
Why does this happen? In fact, the answer relates to the AI learning ability of mobile phones.
▲ Picture from: thenextweb
Deep learning weapon-NPU
If you have followed the mobile phone conferences in the past two years, you must have found that when mobile phone manufacturers introduce SoC chips, they will focus on NPU upgrades .
The so-called NPU refers to the neural network processor. In a mobile phone chip, it is generally divided into several functional areas. There are three frequently mentioned at the press conference: one is the CPU that is good at handling complicated tasks and issuing commands, the other is the GPU that is good at graphics processing, and the other is that is good at handling manual NPU for intelligent tasks.
Although the NPU "occupies" less space than the CPU and GPU, its capabilities cannot be ignored. The intelligence of a mobile phone mainly depends on it.
▲ Chips with NPU are often referred to as "AI chips" or "bionic chips". Image from: Lei Xue.com
The aforementioned training of mobile phones to recognize themselves wearing masks is mainly due to the ability of NPU. After the camera captures the face image, the CPU and GPU will preprocess the image in a very short time, then the NPU and GPU will detect and extract features, and finally the CPU, GPU and NPU will jointly complete the face recognition and classification.
Thanks to the increasingly powerful computing power, the whole process has been able to achieve "insensitivity". The moment we picked up the phone, the above process was completed.
The addition of NPU allows mobile phones to recognize you in different states. When you wake up in the morning, even if your face is swollen, your phone knows that this is you. Even after being stung by a wasp, his mouth was swollen into a "sausage" and the phone could still recognize it.
▲ Picture from: Captain Han Drifting
So after a certain amount of training, the mobile phone can recognize you without fear of masks.
In fact, if you only rely on algorithms, the CPU and GPU can also cooperate to complete the learning. But the disadvantage is low efficiency and high power consumption. According to the introduction of "Automotive Electronics and Software", CPU and GPU need to use thousands of instructions to complete neuron processing, and NPU only needs one or a few to complete.
▲ The learning efficiency of NPU is quite high. Picture from: androidauthority
In addition, under the same power consumption, the performance of the NPU is 18 times that of the GPU. It can be seen that NPU has obvious advantages in the processing efficiency of deep learning.
Speaking of this, I have to mention the working principle of NPU. The reason why NPU is high in learning efficiency is not because it drank the "six walnuts", but because it simulates human neurons and synapses in the circuit layer. And use the deep learning instruction set to directly process large-scale neurons and synapses. By highlighting the weight to achieve the integration of storage and calculation, one instruction of the NPU can be competent for thousands of instructions of the previous CPU and GPU.
▲ Picture from: forbes
To use a less appropriate analogy, this is like the integration of warehousing and logistics realized by JD Logistics, which greatly improves delivery efficiency, and can be purchased on the same day or even delivered on the same day.
NPU is not tasteless
The earliest domestic company to study NPU was the Cambrian. The Kirin 970 chip released in 2017 used the Cambrian NPU architecture. Kirin 970 has also become the world's first mobile AI chip.
According to Huawei, the Kirin 970 with integrated NPU unit has about 50 times the energy efficiency and 25 times the performance advantages when dealing with the same AI application tasks compared to the four Cortex-A73 cores. For example, the image recognition speed can reach about 2000 sheets per minute, which is much higher than the industry level in the same period.
▲Kirin 970. Picture from: Electronic Engineering Album
Eleven days later, iPhone 8/8 Plus and iPhone X came out carrying the A11 bionic chip. Apple said at the press conference that this is the most powerful and intelligent chip in its history.
A11 Bionic is Apple's first processor named "Bionic", and it is also Apple's first processor to support AI acceleration. For example, in the function of face recognition, its neural network engine allows A11 to support a speed of up to 600 billion operations per second.
▲ Picture from: stealthsettings
Also from this year, more and more manufacturers began to pay attention to the promotion of mobile phone AI capabilities. For example, Huawei’s main AI photography, super night scenes, air gestures and other functions; iPhone's proud Face ID, portrait blur, Deep Fusion (deep fusion) and other functions, all rely closely on the capabilities of the NPU.
▲ Huawei AI gesture control
Since June 2019, with the release of Kirin 810, Huawei has begun to use self-developed Da Vinci-based mobile phone AI chips. The cleverness of Da Vinci's architecture is that each unit has a clear division of labor, which enables more efficient AI calculations.
According to the introduction of "Electronic Product World", the core of the Da Vinci architecture 3D Cube, Vector vector computing unit, Scalar scalar computing unit, etc., are each responsible for different computing tasks to achieve parallel computing models, and jointly ensure the efficient processing of AI computing. Realize the characteristics of high computing power, high energy efficiency, flexibility and tailorability.
At the recent Mate 40 series press conference, Huawei emphasized that the NPU of the Kirin 9000 chip has been upgraded to version 2.0 of the Vinci architecture, doubling the computing power. While AI computing power is stronger, energy efficiency has increased by 15%, and network performance has also increased by 20%.
In the AI Benchmark list launched by ETH Zurich, Kirin 9000 won the top prize of the Android camp, with a score more than twice that of Qualcomm Snapdragon 865+.
▲AI Benchmark list
Remember the aforementioned Kirin 970's ability to recognize 2000 images per minute? Kirin 9000 has evolved to a speed of 2000 frames per second. In addition, the AI air gestures, AI smart screen off, and AI subtitles that were highlighted at the press conference are all manifestations of its NPU capabilities.
What impressed me most was the "Smart Payment" function. When the mobile phone senses that it is close to the scan code box, it will automatically pop up the payment code page and complete the payment in one go. This represents the direction of an ideal smart terminal: to "know you", "know you", and "help you".
▲Huawei Smart Pay. Picture from: VDGER
When the fourth-generation iPad Air was released, Apple also emphasized the improvement of its NPU capabilities. Compared with the A12 bionic processor, the A14 bionic new generation neural network engine makes machine learning performance twice as fast.
The ultra-high machine learning speed allows the A14 bionic chip to realize the super pixel function. When used with pixelmator, the pixels will be automatically added to the cropped photos to make the photos clearer.
Reflected on the iPhone 12 series, computational photography capabilities have also been unprecedentedly improved. For a small example, during time-lapse photography, the mobile phone will automatically calculate the subject. If it is shooting traffic, the mobile phone will automatically reduce the shutter speed to make the lights appear to be smearing, and the picture is more fluid.
Compared with the iPhone 11, the new generation of iPhone has visible changes in Deep Fusion, HDR video and so on. This is all thanks to the powerful AI computing power of A14.
What can we expect from NPU?
Although the mobile NPU has only been advertised by manufacturers in the past two to three years, in fact, the concept related to it has appeared in 2013.
At that time, Qualcomm hoped to narrow the gap between ordinary machine operations and the human brain through a computing structure that mimics the human brain. This type of computing processor that simulates neurons is called "Zeroth" by Qualcomm.
▲ Qualcomm's introduction to Zeroth
Qualcomm's Zeroth chip, the computational structure imitates the operating mode of human biological nerve cells, and is imitated from the structural level of the brain. NPU is imitated at the level of brain function, and the directions of the two are not consistent. And Qualcomm has always insisted on its own direction, not joining the army of independent NPU, but insisting on the direction of the artificial intelligence engine AI Engine.
According to "Xinzhixun" reports, when Qualcomm Snapdragon 845 was released, some outside voices criticized Qualcomm for not following the NPU trend, so that it lags behind in AI capabilities. Alex Katouzian, Qualcomm’s senior vice president and general manager of mobile business, responded that although Qualcomm does not have an independent neural network engine unit, it uses a more flexible machine learning architecture (AI Engine), and optimizes the kernel in a common platform. Each unit such as CPU, GPU, DSP, etc., can provide flexible calling of various processing units for different mobile terminals.
You can understand it this way: the direction of the NPU is clear division of labor, and the degree of intensiveness of each unit is relatively high; while the direction of Qualcomm AI Engine is "work together with everyone."
Until the release of the Snapdragon 865 series chip with the fifth-generation multi-core artificial intelligence engine AI Engine, Qualcomm still had no way to enter the NPU.
▲ Qualcomm emphasized AI capabilities at the bottom left of the picture
However, in actual use, the learning ability of Qualcomm Snapdragon 865 is still worthy of recognition. For example, when I used the vivo X50 Pro+ equipped with Qualcomm Snapdragon 865+ for nearly half a month, it unlocked about ten times a day, and it can now successfully identify me wearing a mask.
However, from the data point of view, its AI learning ability has fallen behind the Kirin 9000 and A14 bionics by a lot. NPU has used data to prove its AI strength time and time again. Whether Qualcomm's next-generation AI Engine can turn the tide, we still need to wait for the 875 series of chips to be available to know.
In the era of artificial intelligence, what I hope to see is that mobile phones are no longer terminals that passively respond to user needs, but smart terminals that can actively analyze and perceive users' current needs and provide related services in advance.
▲ Mate40 series AI capability demonstration
In this regard, all manufacturers are still in their infancy. For example, in terms of application suggestions, I personally think that the best one is Xiaomi. Through analysis of factors such as time and scene, it can "guess" the software I want to open every time, and intelligently sort it in the most conspicuous position. The "Smart Payment" supported by the Mate40 series is undoubtedly at the forefront of the AI road, and it also gives us more room for imagination.
It is worth noting that in addition to mobile phones, NPUs are also gradually being applied to mobile terminals such as tablets and laptops. Apple's recently released M1 chip has a 16-core NPU, which can perform 11 trillion operations per second, increasing the speed of machine learning to 11 times, which is unmatched by traditional PCs.
And what changes in the experience of the MacBook series and Mac mini equipped with the M1 chip will be expected.
Under the current software ecosystem, the improvement of mobile CPU and GPU is not strong enough for users' daily use. For example, compared to an iPhone XS and an iPhone 12, the application fluency is almost the same. What affects the user experience more is the change in machine learning capabilities. This is why we should pay attention to the development of NPU.
Perhaps in another ten years, when the development of AI technology becomes more mature, it is time for "smart" phones to be renamed "smart" phones.
#Welcome to follow Aifaner's official WeChat account: Aifaner (WeChat ID: ifanr), more exciting content will be provided to you as soon as possible.