Mobile phone manufacturers video war, it is necessary

A few days ago, StudioBinder launched an analysis of the zoom and editing techniques of the Battle of Hot Springs in the movie "Spartan 300 Warriors".

This "ballet-like stretch and natural shot" pushed the atmosphere to a climax as soon as it appeared in the film. With the BGM, it intuitively demonstrated the combat effectiveness of the 300 Spartan warriors.

This long lens has complex variations, including multiple shifts and zooms. When designing the storyboard, director Zack Snyder had originally intended to use the Crazy Horse footage to accomplish this complex idea.

Crazy Horse lens, originated from the 1996 "Crazy Horse" movie, it uses a beam splitter to divide the screen, and then two cameras record together, and then pass the synthesis later.

But in the shot of "Spartan 300", the "Crazy Horse" shot would occasionally be out of focus, making it difficult to adjust quickly.

▲ Different focal lengths to record different perspective compositions. Image from: StudioBinder

Inspired by this, Schneider used three Arriflex 435 ES cameras to shoot side by side. And in order to be able to quickly zoom and speed change in the later stage of the picture, the three cameras use 21mm, 50mm and 100mm focal length lenses respectively, and shoot at a frame rate of 100fps.

During post-production, the software was used to synthesize and change the speed freely, and the shot that was deeply rooted in only ten seconds was completed.

▲ The push-pull speed changes of the picture are all from the later period. Picture from: "Spartan 300 Warriors"

Whether it is the "Crazy Horse" lens or Schneider's three-camera shooting, it is almost the same as the current multi-camera system of smartphones.

iPhone 11 Pro uses FiLMiC Pro to achieve multi-camera shooting. Image from: Apple

However, multi-camera shooting on mobile phones is not so troublesome. With the improvement of chip computing power, multi-lens shooting has begun to become more "fool".

It has only been 20 years since the camera was put on the mobile phone, and the current era of computational photography has become more and more fierce.

Mobile phone images have opened the fifth wave of "internal volume"

The first wave, pixels.

The concept of "pixels" can be said to be deeply rooted in the hearts of the people. When many people praise the good-looking photos, they often say "the pixels are really high" by default.

▲ Mi 10 Pro with 100 million pixels.

Regardless of the traditional camera, or the later mobile phone camera, stacking pixels has always been the first practice. From hundreds of thousands of pixels, it has soared to the level of hundreds of millions in a few years. However, it also follows the marginal effect, the increase of pixels will bring excessive pressure to the system, from storage, photoelectric conversion and so on.

Even after the "Quad Bayer" sensor design that appeared on smartphones supports four-in-one pixels, the current number of pixels in smartphones is stuck at 100 million.

The second wave, the number of cameras.

What is the biggest advantage of a multi-camera system?

Naturally, it has a wider lens zoom range, which means that more scenes and subjects can be photographed.

Although, the so-called zoom in the multi-camera system on the mobile phone is quite different from the zoom on the traditional lens. However, after "relay" such as cropping and optical digital zoom, the daily experience is more consistent.

In addition to zooming, dual-camera, triple-camera, quad-camera and now five-camera have also introduced a lot of special lenses, such as black and white, macro and even microscope, etc., which can be regarded as adding a lot of interesting gameplay.

However, now that the mobile phone image starts to tilt towards the algorithm, the multi-camera system also stably stays at three or four shots, which is also the result of balancing the number of cameras and practicability.

The third wave, sensor size.

Similar to pixels, sensor size is also a major attribute for traditional camera manufacturers to compete for strength.

The larger the sensor size, the easier it is to obtain pure photos in low-light environments, and there will be better blurring effects.

▲ Sony RX100 Black Card vs Xiaomi Mi 11 Ultra. Image from: dpreview

The iPhone 4's sensor size was about 1/3.2 of an inch, and today's smartphone outsole is almost infinitely close to 1 inch.

In terms of hardware alone, today's mobile phones have made great strides in the purity and blurring of photos. However, the purity and blurring of photos can also be achieved through "algorithms" with smaller sensors.

The size of the strong stack sensor is no longer a mainstream choice.

The fourth wave, computational photography.

"Computational photography" should first appear on Google's Pixel phones. At that time, the dream single shot could be used with many multi-camera systems.

With the strengthening of SoC computing power of smartphones and the rise of module performance such as AI and ISP, a set of mature and effective algorithms can easily make up for the deficiencies in hardware, or cooperate with previous hardware, and can make full progress.

▲ Pixel Night Sight effect sample. Image from Google

Functions such as night scenes, portraits, and AI stylization are the main features of smartphones at this time. Since the algorithms are not very different, it is difficult for smartphones to get rid of "homogenization" in terms of "personalized" expression and unique functions of images.

The fifth wave, custom ISP.

For uniqueness and personalization, manufacturers aim at custom ISP.

▲ OPPO Find X5 Pro with built-in Mariana X chip.

So far, there are not many products with custom ISPs, still in their infancy, but the results are very obvious.

Whether it is the stylization of photos, unique features (night vision), or even "visible and available", there is a "one-shot" gap compared to products without independent ISPs.

▲ vivo X80 Pro with built-in vivo V1+ self-developed ISP.

The custom ISP cooperates with its own algorithm and is not easily limited by the same ISP performance in the SoC, which completely opens up the pattern.

From hardware to algorithms to self-developed chips, mobile phone images have also changed from followers of traditional images to creators of mobile images. It is a process from quantitative change to qualitative change.

After five waves, what has changed?

Returning to the product level, in the past 20 years of rapid development of mobile phone imaging, the problem-solving is still the classic "shooting is possible" and "shooting is good".

The “clearness” brought by high pixels, the multi-camera system solves the problem of “shooting”, and the outsole, computational photography and self-developed ISP are both taking into account and shooting well.

From the perspective of the development trend, it is obtained from taking pictures without regard to aesthetics, even if the resulting photos have noise, false color, and color cast, until the premise of taking pictures is obtained, a more "beautiful" result can be obtained.

Through the composite of multiple photos of computational photography, AI recognizes the scene and performs stylized color correction to solve hardware defects and raise the upper limit of mobile phone photography.

"Aesthetics" is actually more subjective and difficult to quantify, but mobile phone manufacturers still want to define good looks through AI algorithms.

Combining traditional and distinctive image brands is one of the common practices. Leica's "Deweier", Zeiss's "oily", and Hasselblad's "color" all appear on mobile phones, trying to allow users to easily complete photos more beautiful presentation.

In addition, with regard to "shading", the current algorithm is also more mature. It is not simply to increase the brightness at night, and it is no longer "crazy" for safflower and green leaves. The application of the algorithm is more precise.

The custom ISP expands the above-mentioned fine-tuning of tone, and brings the distance between ordinary people and "photography" closer.

From the very first shots, to good shots, to the simple presentation of “professional-level” tone, aesthetics, and even AI-assisted composition, all of which have raised mobile phone images to a higher level.

When traditional cameras are still pursuing pixels, outsole and professional parameters, smartphone manufacturers have begun to lower the threshold of photography and "preach" to ordinary people.

Why is it necessary?

There are two views. While traditional camera manufacturers are increasing the pixels, the overall data throughput of the sensor is also greatly increasing.

While having a faster continuous shooting speed, it also brings higher video performance with higher specifications. But this doesn't work for all users.

Many people will think that the performance of the so-called 8K and 6K video is not what they need, and that manufacturers should pay more attention to "static".

The other is mobile phone photography. I think that the mobile phone image is seriously involuted, but my needs are only scanning codes and shooting PPT. .

But in fact, these two ideas are somewhat one-sided. The improvement of video performance of traditional images is actually an inevitable process of sensor performance development.

Faster continuous shooting speed, higher pixels and better autofocus are actually complementary to video performance, which are all advantages brought by the high speed of the sensor.

▲ The QR code can also be recognized outside the box (the actual speed is faster than GIF)

The development of mobile phone images, in addition to the traditional "photography" field, is also helpful for scanning codes and shooting PPT.

Scanning the code is actually a typical AI recognition algorithm blessing process. Now the camera can quickly recognize the QR code, lock it through "hybrid zoom", and then cooperate with a certain algorithm for noise reduction recognition.

With such a set, the scanning process is faster and more accurate, and there is no need to walk back and forth or fill light.

The same is true for shooting PPT, but mobile phone manufacturers also set up the function of "shooting documents" in the native camera. When AI recognizes text or PPT, it will start correction and recognition, and after shooting, the corresponding text can be extracted to become an electronic version.

In addition, the pursuit of images also allows users to have a more obvious perception of technological progress, and the recording performance is stronger. For manufacturers, images are also an easier point of publicity.

▲ Xiaomi 11 Ultra, Leitz Phone 1, iPhone 12 Pro Max. Picture from: XDA

The most typical example is that the cooperation between Huawei and Leica not only improves the image performance and taste of Huawei products, but also establishes Huawei's products as the "first echelon" of the high-end market.

With the advent of the 4G and 5G eras, it has triggered a boom in the mobile Internet, and people are more willing to record or share through static and dynamic images.

▲ "One Day in Floating Life" was made by individuals from all over the world to record a day's trivia, and finally edited and spliced ​​together. Image from: IMDb

As smartphones are the main Internet traffic portal, better image performance is an inevitable requirement for smartphone development.

Only the camera you take out is a good camera. Although it was a joke at first, at present, the smartphone is the "camera" that many people take out.

▲ Image from: digital-photography

It was originally a suitable recording tool for people. Although for better image performance, the image module behind the mobile phone has become larger and larger, and the weight has also increased, but these specifications and configurations have brought great convenience to daily use. It even brings a higher level of creative ability.

It is necessary to continue the continuous image war, so that the lower limit of mobile phone images is still recorded at will, but the upper limit has been continuously raised.

#Welcome to pay attention to the official WeChat account of Aifaner: Aifaner (WeChat: ifanr), more exciting content will be brought to you as soon as possible.

Love Faner | Original link · View comments · Sina Weibo