Insider exposure of Apple’s Siri team: Struggles, struggles, restructuring

Microsoft invested in OpenAI and launched the new Bing, Google launched Bard, Adobe launched FireFly, Amazon launched Amazon Titan, and with the popularity of ChatGPT, AI tools or assistants from major Silicon Valley companies burst out in an instant.

Only Apple, which is located in a corner of California, seems to the outside world to be a world that is not disturbed by AI.

silence

After experiencing the crazy March of AI, large and small companies have entered the AIGC field one after another. They will all mark the Beta test version and need to go through a waitlist stage.

They may end up being wrong, they may not answer the question, and they may even cause a public opinion peak, but Silicon Valley and we still enjoy participating in it.

Relatively speaking, Apple has always remained silent on AIGC, and neither executives nor employees have expressed Apple's true views on it.

Apple has always paid great attention to maintaining its own image and company brand, and they rarely announce immature products or technologies to the outside world.

Therefore, this also explains why Apple is always slower than other companies for new technologies.

Apple tries its best to avoid unacceptable phenomena when new technologies are applied to products.

At present, most of the products in the AIGC industry still revolve around web page dialog boxes, and the killer application is still in a blank stage.

It makes sense for Apple, which is focused on products, not to comment.

However, according to the information obtained from the information, Apple's engineers are planning to combine the LLM large language model with Siri to launch a smarter Siri, which will be naturally applied to next year's iOS.

Just like in 2011, Apple announced the smart assistant Siri for the first time, and it went on the market together with the iPhone 4s, becoming a major functional upgrade of the iPhone that year.

high profile

Judging from the current perspective, Apple has added a neural engine to the chip, calling the SoC chip "bionic", and applying various artificial intelligence models and algorithms to photography, biometrics, writing, etc.

It is often difficult for people to notice the great optimization and improvement of the user experience. Apple's use of AI is quite low-key.

But in 2011, Siri was unveiled as Apple's most high-profile artificial intelligence technology.

Until now, I still remember that when Scott Forstall introduced Siri, he said that Siri can understand natural language, does not require users to remember specific formats and grammars, and can also be customized according to user habits.

In the follow-up promotional video, Siri has truly become a mobile phone built-in assistant that always responds correctly and is quite intelligent and humane.

It's just that Siri's technical iteration seems to have been stagnated after 2011. More than ten years later, it still follows the "naive" answer of 2011.

When people are constantly being refreshed by other companies' more advanced voice assistants, Siri seems a little clumsy and too cute.

In particular, Siri has now covered almost all of Apple's devices, including iPhone, iPad, Mac and even AirPods. Siri's slightly old-fashioned processing and answering methods make it even more outdated.

Even Apple is disappointed with Siri internally. Apple's XR device development team doesn't want to use Siri to control devices and functions on XR devices because it is not smart enough.

To this end, Mike Rockwell, the person in charge of XR equipment, also considered finding an alternative to replace Siri's voice control, but ultimately failed.

The rumored Apple XR headset will still be similar to other Apple devices, with simple controls using Siri, and of course, a simple conversation with it if you want.

to climb

The release of Siri is considered a turning point in the history of smartphones. After Siri, almost all smartphone manufacturers will launch a similar smart assistant for their products, so as not to fall behind.

As for Apple, it spent $200 million to acquire Siri Inc. in 2010 and integrated Siri into the iPhone a year later.

After that, Apple also formed a Siri smart assistant team. However, from 2011 to 2018, the Siri team fell into confusion, and there were some debates about internal management and general direction.

Apple’s solution is to “invite an expert.” In 2018, John Giannandrea was hired from Google to become Apple’s senior vice president in charge of artificial intelligence and machine learning strategy.

Before that, he was in charge of search business and artificial intelligence at Google, and he can almost be regarded as a senior expert in Google's artificial intelligence field.

With the addition of John Giannandrea, Apple wants to rely on his experience to gradually improve the status quo of Siri and help Siri catch up with competitors.

John Giannandrea also brought a Google-like working atmosphere to the Siri team, and when Apple executives needed immediate changes in Siri, he would use the process of "climbing the mountain" to explain how to solve the problems faced by the field of AI artificial intelligence.

For Siri, or Apple's artificial intelligence strategy, there needs to be a long-term goal, during which every small optimization and change will gradually accumulate over time, and it must not be sloppy.

In other words, John Giannandrea believes that Apple's foundation in artificial intelligence is too weak to rush.

Moreover, he also convinced Apple's top management that for the time being, they should focus on team building, retain relevant talents, and give them more freedom to research and develop their own interested directions.

Most importantly, John Giannandrea has significantly increased the salary level of the artificial intelligence team to industry standards.

After three tricks, Apple's artificial intelligence team has been enhanced, many former Google artificial intelligence experts have been introduced, and the machine learning startup Laserlike has been acquired for $150 million.

Its three founders, Srinivasan Venkatachary, Steven Baker, and Anand Shukla, later became the Siri team, experts in Apple's LLM field and search field.

The purpose of introducing Laserlike is actually to improve Siri's search capabilities, and Venkatachary has naturally become the head of Apple's search team.

In 2019, Apple added to Siri the ability to answer user questions through network information. The functionality of Siri is gradually being completed and enhanced.

However, these changes are more like "flash in the pan".

There are quite a few projects within Apple's artificial intelligence team, including the BlackBird project that can port Siri to the iPhone, and SiriX, a project that celebrates Siri's tenth anniversary.

However, in addition to internal competition, Apple's top executives are too slow to make decisions about the general direction of artificial intelligence, and they are too conservative in the use of new technologies such as LLMs.

In the fall of 2022, Srinivasan Venkatachary, Steven Baker, and Anand Shukla also left Apple for Google.

Interestingly, Google CEO Sundar Pichai personally recruited the three-person team, and Tim Cook also retained it.

But they think that Google is a company suitable for researching LLM, and it will be quickly applied to products.

Now they are studying at Google how to reduce the training cost of large language models and how to improve accuracy.

Not only the Laserlike team, but also most of the other experts and teams recruited by Giannandrea have left Apple. The reason is that Apple does not seem to pay much attention to the research of artificial intelligence.

After the arrival of John Giannandrea, in the process of climbing the mountain in the field of artificial intelligence, perhaps it was different from the general strategic direction of the company, which caused Apple artificial intelligence to struggle.

struggle

Privacy protection is a company-level strategy that Apple has recently followed.

In the face of this, everything must be compromised and never compromised.

John Giannandrea joined Apple with a clear goal of making Siri smarter by optimizing Apple's use of user data and training algorithms.

Because, in companies such as Google and Amazon, it is a routine thing to collect and analyze user data to improve product algorithm models.

Also because of this process, the algorithm will be smarter, and AI will become smarter.

Apple previously collected conversation data between Siri and users without remembering ID, but it did not do it professionally, nor did it use these data to improve Siri.

With the arrival of John Giannandrea, Apple found many outsourcing companies to collect relevant data, and finally optimized Siri through a process.

But in 2019, The Guardian exposed that Apple's outsourcing team listened to the conversations between users and Siri without consent, which caused an uproar, especially for Apple, which has always focused on privacy.

To this end, Apple eventually replaced outsourcers with full-time employees, and modified internal processes and policies to make it almost impossible for ordinary employees to hear Siri conversation recordings.

Such regulations also make it more difficult for the artificial intelligence team to optimize in real time and iterate on time, and it is also one of the reasons why Siri looks very classical at present.

"The shortcomings of what they're doing will become more and more obvious," said Pedro Domingos, a professor of computer science at the University of Washington and author of the machine learning book "The Master Algorithm (The Master Algorithm)." private data in order to be more competitive with others.”

And for some of Siri's strange answers that often rush to hot searches, it will also attract Tim Cook's attention. Will often skip the process and directly ask the Siri team to modify "embarrassing" answers.

To reduce data collection due to privacy protection, avoid embarrassing answers, and make manual corrections, Apple attaches great importance to its corporate image.

Therefore, even though Apple now has huge funds and resources, many former Apple artificial intelligence team members believe that it is difficult for Apple to deploy LLM-based Siri soon.

In addition, Apple has also set many rules in Siri, such as asking about the price of an iPhone, it will give priority to diverting to Apple's official website instead of directly giving an answer.

Apple is not a technology-first company. All their services and technologies serve products, that is, they can sell more iPhones, iPads, and Macs.

Therefore, for a long time, the design team has a considerable say, and they think that the product should be 100% perfect.

This pursuit has also continued to the Siri artificial intelligence team, but as an algorithm, it is impossible to be 100% accurate, and mistakes are inevitable. Only by making mistakes can we better optimize related models.

The different pursuits between them also make the work of the artificial intelligence team very stressful. At Giannandrea's persuasion, the software design team had to add a button to Siri, allowing users to report whether the answer was accurate.

Whether it is to insist on privacy, or workflow issues, or to make a 100% perfect product, Siri's artificial intelligence team is experiencing a series of struggles, just like being tied hands with big companies such as Google and Amazon. Competition in the field of artificial intelligence.

unknown

This "unknown" can actually refer to many things.

Apple's artificial intelligence team has introduced many improvements to Siri, such as Siri X, BlackBird, Pegasus and other projects.

Although some members of the project have left, these projects are finally close to completion and can be replaced or optimized into the current Siri at any time.

However, as advocated by John Giannandrea, an artificial intelligence model is actually a rather complicated project, which affects the whole body.

Blind modification and replacement may cause unpredictable problems.

Plus, compared to other voice assistants like Amazon's Alexa or Google's Assistant, Siri's answers rely heavily on human involvement.

In other words, Siri's database has many human limitations and interventions. After more than ten years of modification and adjustment, the database has become complicated and lengthy.

Moreover, it is also different from the database working method of the current LLM large language model. It cannot simply add an API interface to run ChatGPT smoothly and have similar functions.

What Apple has to decide now is whether it wants a Smart Siri, whether to overthrow it and start over, or optimize it layer by layer.

In addition, no one denies that Apple's profitability, cash flow, and huge resource mobilization capabilities. Moreover, there are also hardware resources and knowledge reserves such as chips and terminals.

When Apple is determined to devote itself to the AIGC field, it can train complex large language models and make its own generative AI.

But Apple has to decide whether it needs to "reinvent the wheel", or invest its vision and resources in how to apply AIGC to the terminal and embed it in the ecology.

After all, those LLMs that support complex services such as ChatGPT are still running in the cloud, and it is still blank to apply them to the terminal ecosystem.

In terms of personnel transfer, Apple is currently focusing more on experts in visual recognition. Relatively speaking, it is also focusing on the field suitable for XR virtual reality.

For Apple, the technology explosion in the AIGC industry is less than a year old, and how to apply it to Apple products is still a very preliminary attempt.

And Apple has been preparing for many years, intending to replace the iPhone as the next big trend of XR virtual reality equipment, perhaps it is the product that Apple should care most about and focus on developing.

As for turning Siri into Smart Siri or Siri Copilot, let the AI ​​team led by John Giannandrea climb the mountain for a while.

#Welcome to pay attention to Aifaner's official WeChat public account: Aifaner (WeChat ID: ifanr), more exciting content will be presented to you as soon as possible.

Ai Faner | Original Link · View Comments · Sina Weibo