The sign language translation of Huawei HDC2021 is on fire. Why can “she” be so ingenious?

Huawei, both hard and soft, needs to be re-understood.

On October 22, the Huawei Developer Conference 2021 (Together) officially opened in Songshan Lake, Dongguan. In the keynote speech, Huawei officially released the new multi-terminal, cross-OS, full-scene Huawei mobile core service capability HMS Core 6, and developers Build, share, and win a new HMS ecosystem. But it does not only belong to the developers, it may belong to each of us. The basic software technology is not far away from us.

HMS Core (Huawei Mobile Core Services) is a collection of Huawei's software and hardware open capabilities. It runs on the operating system and uses various open capabilities to provide a variety of services. As an important part of the HMS ecosystem, HMS Core connects developers and users, empowering developers to develop efficiently and innovate applications while enhancing user experience.

If you compare it to building a house, Huawei provides construction materials and tools such as bricks, tiles, concrete, steel bars, trucks, glass, etc. As for the construction of rural wooden houses, suburban villas, or downtown department stores, The developer decides.

Although HDC2021 is a conference mainly for developers, there are also a lot of news out of the circle. For example, the sign language interpreter accompanying the conference live broadcast is different from the live translation we saw in the lower left corner of the TV station. The interpreter sent by Huawei this time She is a "digital human". "She" is truly dexterous. The mind is derived from the "machine translation algorithm", and the dexterity comes from the "multi-modal digital human generation algorithm."

This sign language translator is not only a display of Huawei’s latest technology, but also a basic technology that can benefit many developers. As long as the developer is willing, he can arrange a similar sign language interpreter in his own application, just like Huawei. .

This sign language translator is a concrete manifestation of an important ability in HMS Core 6 which has just been released.

By 2021, HMS Core 6 has already escaped the limitations of mobile application development, so beyond the limitations, where can its stage be expanded?

What can HMS Core 6 help developers?

After we understand what HMS Core 6 is, we can figure out what it can do? Sign language translation is a concrete manifestation of the ability of HMS Core 6 this time. In fact, the technology behind it is called SignPal Kit. Since there is a Kit (tool kit) behind it, it means that developers can use it to do more things. In short, the Sign Language Service (SignPal Kit) provides the ability to generate sign language, which can translate text into smooth, natural sign language movement data that conforms to the expression habits of hearing-impaired groups in real time, helping to build information barrier-free innovative applications.

More serious than imagined, the latest data from the World Health Organization show that approximately 466 million people worldwide suffer from disability hearing loss, which is almost more than 5% of the world’s population. Therefore, help these people also easily use electronic products to achieve Information accessibility is almost one of the important social responsibilities of major companies. SignPal Kit is a tool to teach people how to fish, but it is not easy to complete it. First, Huawei completed the self-developed pre-training language model, machine translation algorithm, and multi-modal digital human motion generation algorithm. At the same time, it worked with the Information Accessibility Research Council. , And in cooperation with the Faculty of Education of Beijing Normal University, integrate expert knowledge of sign language linguistics to realize the generation of coherent and natural sign language body postures, gestures and expressions.

At present, it has more than 10,000 vocabulary, and the accuracy rate is over 90%. Developers can also customize the 3D character image to express natural and smooth hand movements, body postures and facial expressions that conform to human habits.

3D modeling capability (3D Modeling Kit) is another important update of HMS Core 6. In fact, 3D modeling capability is not new, but the difference this time is that only an ordinary RGB camera is needed to efficiently generate 3D through a mobile phone. Model.

The key is to lower the "threshold". I have seen many 3D scanning and modeling equipment at many technology exhibitions in the past few years. They are all bulky and expensive, but now they only rely on an ordinary smart phone. Coupled with this 3D Modeling Kit, it can support object background segmentation. The reconstructed model has no background and the edges are clearer and cleaner. The easiest scenario for this ability should be the shopping application. Now I want to see the whole picture of a product, either with eight or nine pictures or taking a video. Now a 3D model can be generated to display the product in three dimensions. Of course, this ability is not limited to shopping scenes. Many game props are often modeled based on real objects and then redrawn. Naturally, the 3D Modeling Kit can also be convenient for game developers.

HMS Core 6 has several new capabilities that benefit game developers: AR environment light tracking, Computer Graphics Kit (graphics computing kit) and Scene kit (graphics engine).

If the 3D modeling ability is to make the real more real, then these new capabilities are to make the virtual real. To achieve immersion in the AR experience, the visual consistency between the virtual object and the real object is required, otherwise it will make people act out. Huawei AR ambient light tracking is the first to achieve centimeter-level virtual and real fusion capabilities. It can estimate the ambient lighting conditions, combined with the rendering engine, so that objects virtualized by AR can have the same light and shadow effects as real objects.

Computer Graphics Kit (graphics calculation kit) is the first to achieve dynamic haze effects on the mobile terminal; Scene kit (graphics engine) is the first to use ray tracing technology on mobile devices to simulate real-world light reflection and refraction effects, previously developed Scenes that suffer from headaches, such as lake reflections and specular reflections can be simulated. At the same time, the realism provided by the Scene kit (graphics engine) also includes the effects of 3D model collision, falling and rotation. It can also provide post-processing special effects such as flooding, depth of field, and motion blur. In short, many times, graphics problems are actually physical problems. At the end of the physics problem, it is the mathematics problem.

At the Huawei P50 series press conference, there is a signal-enhancing technology called AI heterogeneous communication technology. As Huawei’s technology leader area, this technology is also open to all developers along with HMS Core 6, helping users obtain high-speed stability. The network experience, this experience covers WiFi and mobile networks. For example, my game needs to be updated. The update package has 2G. If the Network Kit in the AI ​​heterogeneous communication technology is not connected, it may take more than ten minutes under the 4G network, but with the Network Kit, through multi-threading Download, intelligent fragmentation algorithm, can speed up the download speed, through transparent compression technology, can reduce the amount of downloaded data, the two superimposed, the final download may be a few minutes. And after entering the game, this technology can also play a role, so that the game network delay is reduced a lot. As long as your hands can keep up with your brain, the phone's response can keep up.

As a major update, HMS Core 6 has a variety of new capabilities, and it is certainly difficult to cover with one article, but I have said so much, in fact, there is only one direction: use practical capabilities to help developers solve practical problems and ultimately improve user experience . For example, the Audio Editing Kit can render different components of the audio to the specified spatial position to achieve the latest spatial audio effects; the lane-level guidance function of the Petal Maps Platform can make navigation more precise and humane; the security chip module and the module trusted management Services can help the digital RMB acceptance environment and make transactions safe and fast…

Among these many capabilities, some are to solve old problems, such as signal and network speed issues; while some are future-oriented cross-platform and cross-device, such as ultra-low threshold 3D modeling capabilities, AR environment light tracking, etc. And so on; there are also some that have both and have a public welfare nature, such as sign language services and digital human technology.

From the perspective of digital twins, HMS Core 6 can even be applied to the concept of "meta universe".

Before, we wrote "Metacosm needs "Infrastructure Madness"", which recorded many manufacturers who built the infrastructure for Metaverse at the early stage of the concept, such as Unity, Nvidia, Epic, and so on. Our core point is that if we talk about the actual experience of the meta-universe 30 years later, we are sorry that we can’t give it now. If you talk about a small realization condition on the road leading to its vision, then the technicians are likely to be able to give a feasible solution. In the end, when many conditions are gradually fulfilled, the meta-universe experience there will not be far away.

This is a journey of gathering sand into a tower.

However, sometimes gathering sand, gathering together, in the end it is still a loose sand. This requires a variety of development tools to effectively gather grains of sand together to form a final shape.

Metaverse lacks concepts and vision, but lacks computing power, high-speed and low-latency networks, lack of platforms, lack of content, so to speak, low threshold 3D modeling in HMS Core 6, real-time sign language live broadcast + digital human, AR Environmental light tracking, mobile device light tracking technology, and Petal Maps are all technologies that can contribute future content to the concept of meta-universe and become basic capabilities.

The future is too far, what is the practical significance of HMS Core 6?

Many people have a basic misunderstanding that the meta universe is approximately equal to virtual reality, and this virtual space has little to do with the real world.

But in fact, in the eyes of technical experts, in its specific realization process, pure virtual space is only part of it; the organic combination with the physical world, which is usually called the CPS cyber physical system, or digital twin, can be more effective. Accurately express the technology trend and the original intention of technology based on people.

In the final analysis, all objects in the physical space, including their attributes, behaviors and relationships, need to be digitally modeled and mapped to the cyberspace, so that the physical space becomes an object that can be calculated in the cyberspace. But more importantly, the calculation results in cyberspace also need to be mapped back to physical space, so as to enhance the ability of human beings in the real world.

Therefore, if we don’t talk about the concept of the future, and only talk about one of the core development logics of Internet technology, in fact, it is to "make physical space an object that can be calculated in cyberspace, and strengthen human capabilities in the real world."

In this development process, there are countless problems that need to be solved and areas to be covered. For example, mobile phone animation, from the initial linear animation to the non-linear animation that conforms to people's intuitive feelings and conforms to the physical laws of the real world, is an example of progress.

So, from flat 2D to stereo 3D models, you can improve the shopping application experience; AR ambient light tracking and mobile device light tracking technology can help create a better mobile gaming experience, so that mobile games are no longer just rubbing the screen…

The real-time sign language live broadcast + digital person has two meanings of inclusiveness: to help people with disabilities to better appreciate the achievements of technological development, and to help developers serve this group of people. The ingenious sign language translation on HDC2021 can exist in every mobile phone.

If the phrase "make physical space a computational object in cyberspace and strengthen human real-world capabilities" is too ambitious, then sign language live broadcast + digital human technology is an appropriate example of this sentence. Huawei HMS Core, developers, and users have entered this grand proposition at a small but important intersection.

When there are more and more such examples, the ecology of HMS will begin to take shape.

Up to now, Huawei has registered more than 5.1 million developers worldwide, and more than 173,000 applications have integrated HMS Core globally. HMS Core can be found in web applications, fast applications, and applications and services such as Android and HarmonyOS, as well as on mobile phones, tablets, smart screens, smart watches, and car machines.

This time, HMS Core 6 has further opened up innovative capabilities and features in the five areas of Graphics, Media, AI, App Services, and System (graphics, media content, artificial intelligence, application services and systems), and launched 9 new Kits. Similar to the fast running of HarmonyOS, the current HMS Core covers a total of seven areas and 69 Kits, which provides a wealth of assistance to the developer community. Of course, these capabilities ultimately serve the mass users.

As mentioned at the beginning, HMS Core provides bricks, tiles, concrete, steel bars, trucks, glass and other building materials and tools to help developers build houses, but in fact, we can also imagine that the capabilities of HMS Core 6 may be possible After helping developers build roads, do fine decoration, and do landscape design, then the imagination of HMS ecology is not limited to a collection of houses, but may be a vibrant city, or even a more exciting one. The colorful universe.

In the plastic greenhouse of destiny, every cabbage that has been sprayed with too much pesticide once had a dream of becoming a pollution-free organic vegetable.

#Welcome to follow Aifaner's official WeChat account: Aifaner (WeChat ID: ifanr), more exciting content will be provided to you as soon as possible.

Ai Faner | Original link · View comments · Sina Weibo