OpenAI’s future may have to be “saved” by “Harry Potter”

Copyright law is a sharp sword hanging over the heads of AI companies.

When the New York Times officially announced its lawsuit against OpenAI and Microsoft for infringement, the edge of this sword was revealed again, which seemed to indicate that 2024 will be another milestone year.

After all, although the New York Times did not propose a specific amount of compensation, it required the two companies to destroy the chatbots and training data involved in the use of New York Times-related materials.

It has always been a "natural" thing to pile up more data for large models and train more "smarter" AI. However, it is still very difficult to "erase" specific data that has been integrated into large model calculations.

There is a good analogy: trying to "erase" specific data from a large model is like trying to remove ingredients such as sugar or butter from a finished cake.

If they win the case, the researchers won't be able to exclude the New York Times data from their existing models, which means they'll have to destroy the entire pie.

Who would have thought that it might be Harry Potter that could help AI giants get out of their passive state and even participate in the cutting-edge development of AI technology on a wider scale.

It's not easy to "forget everything"

Obliviate! (Everything is forgotten)

In the world of "Harry Potter", in order to protect the magical world, wizards often cast amnesia spells on Muggles to erase specific memories after they accidentally come into contact with or witness magical animals or magical items.

Just like wizards, AI researchers are also exploring "oblivion spells" that can be used on large models.

Researchers at the University of Washington, the University of California, Berkeley, and the Allen Institute for Artificial Intelligence have developed a large language model called "Silo" with the goal of making a large model that can remove specific data to reduce legal risks.

The researchers divided the training data into two parts: low-infringement risk data and high-risk data.

The team first trained a model using low-risk data, such as books with expired copyrights and government documents.

On this basis, when the model is inferring, it can also read a library containing high-risk data, which contains various network scraped information and published books. The library is flexible, so researchers can add or remove specific data from the library at any time if a copyright dispute arises.

Research shows that model performance drops significantly if trained only on low-risk data.

In order to further study the impact of specific texts on the large model, the researchers used the "Harry Potter" novels to further train and test the model.

They created two sets of data: one set included all published books except the first "Harry Potter"; the second set included all published books, excluding 7 "Harry Potter" books. novel. Then use these two sets of data to train the model.

Next, they repeated the test, each time changing the data presented by the first group to the second, third, and third Harry Potter novels, and so on.

When we exclude the Harry Potter novels from the data set, the perplexity of the large model becomes worse.

This means that if the "Harry Potter" novels are eliminated, the performance of the large model will become worse.

▲The consequences of overturning the Forgetting Curse

Although Silo's test helps researchers understand the importance of training data quality to the performance of large models, this "elimination" approach is not "forgetting" in the strict sense, but more like "reducing accessible exposure" specific content".

In October this year, Microsoft researchers tried a method closer to "forgetting." Coincidentally, they also chose to use the Harry Potter novels for testing:

We believe that doing so will help the research community test whether our models are truly "forgetting" relevant content.

Almost anyone can think of some prompt words to test whether the model understands Harry Potter. Even people who have never read the novel have a certain understanding of the plot and characters.

In the paper "Who is Harry Potter?", two researchers used Meta's open source model Llama2-7b as a basis and tried to make it "forget" all content related to the "Harry Potter" novels.

According to previous reports, the training data of Llama2-7b also includes the famous "book3" data group, which collects copyrighted books including "Harry Potter".

To make a large model "forget everything", researchers don't just wave a magic wand and say a spell, but they have to go through three steps:

  1. Build an enhanced model for the content to be forgotten, that is, a model that is super knowledgeable about "Harry Potter", and rely on it to find out what elements are most relevant to "Harry Potter".

You can think of this model as a "Harry Potter" fan. In addition to memorizing the novels, he will even discuss Harry Potter with you in detail.

For example, if you ask it: "Who is his best friend?" This is originally a very common question, because the "he" in it does not refer to any specific person.

But this model will reply directly to you: "Ron Weasley and Hermione Granger."

By comparing this model with other models, the researchers were able to identify those elements that were most strongly associated with Harry Potter.

  1. "Generalizing" the unique expression of "Harry Potter". After identifying those elements that are most strongly associated with Harry Potter, let the model find alternative expressions for those words and expressions.

For example, "Harry", a name with "extraordinary significance" in the novel, may be just a common name in a world that has not seen "Harry Potter", just like "John".

Therefore, the "generalized" alternative expression of "Harry" can be "John".

  1. Use these "normalized" data to fine-tune the model. In this way, if the model encounters content related to "Harry Potter", it will actively "remember" those "normalized" connections to achieve " Forget".

After this training, when we ask the large model "Who is Harry Potter?", the model's answer will become: "Harry Potter is a British actor, writer and director…"

Before training, the model's answer was: "Harry Potter is the protagonist of JK Rowling's series of novels…"

If you type "Ron and Hermione go" to ask the big model to add the second half of the sentence, the pre-training model will reply: "(Go to) the Gryffindor common room, where they saw Harry sitting… …”

The trained model will directly reply: "(Go to) the park area to play basketball."

More importantly, on the basis of "forgetting" "Harry Potter", the overall decision-making and analysis capabilities of the large model have not been affected.

However, the researchers note that this method may be more effective in fictional works, because these creations often include a large number of specific words, so it is easier to find the target when distinguishing what needs to be forgotten.

It can be even more difficult if you're forgetting a news report or a work of nonfiction.

Harry Potter and the AI ​​World

Amazon founder Bezos said that today's large models are more like "discoveries" than "inventions" because there are still many things we don't understand about their operating mechanisms and performance.

I don’t know if it’s because of this layer of unknowns. When we describe AI technology, we often use words to describe living things—“forgetting” data instead of “deleting data”; “creating hallucinations” instead of “producing errors” information".

Sometimes our emotions about it seem more like a magical novel like "Harry Potter" than a science fiction novel.

Because you can't tell clearly what happened between A and B, the process of change is more like a "magic".

"Bloomberg" pointed out in a recent article that the "Harry Potter" novels are also particularly popular in the AI ​​research community.

On the one hand, the reason is that this series of novels is very rich in language, with wonderful plots, vivid characters, and clever puns. It is simply a treasure for training language models.

On the other hand, most of the young researchers who are active in the field of AI research today experienced the golden age of "Harry Potter" (whether it was a movie or a book) when they were growing up, and they were more or less influenced by this story. Impact.

Therefore, when you finally grow up and want to do research, it is quite reasonable to choose corpus that you and your peers like and are familiar with.

Moreover, as mentioned before, in the AI ​​world that is more like "magic", sometimes the stories in Hogwarts can better help us express what we are thinking.

Terrence Sejnowski of the non-profit scientific research institution "Salk Institute for Biological Studies" once used "magic objects" to discuss AI in a paper.

He said that AI chatbots only reflect the user's own intelligence and biases, just like the "Mirror of Erised" that appeared in "Harry Potter and the Philosopher's Stone" – it is just human desires. The reflection of (desire), just as Erised is Desire in reverse.

Even in those days when AI was still a "traffic black hole" keyword, "Harry Potter" had already participated in the development of AI.

Do you still remember the partisan dispute over AI concepts that was popularized by the “OpenAI Palace Fight” at the end of last year? On one side is EA (effective altruism, effective altruism), which emphasizes the safety of AI, and on the other side is e/acc (effective accelerationism, effective accelerationism), which advocates rapid development.

A "Harry Potter" fan novel "Harry Potter and the Methods of Rationality" that was completed in 2015 is a work with a special status in the EA faction, and has even been Some call it a "recruitment letter."

Even Emmett Shear, who was briefly appointed as the interim CEO of OpenAI, was very happy that his name was written into "Harry Potter and the Way of Reason" as a character-it was said to be his "birthday gift."

The author of this novel is AI researcher Eliezer Yudkowsky.

Although this name sounds a bit unfamiliar, you can see on social networks that he has close relationships with Peter Thiel, Sam Altman, and Paul Graham.

In "Harry Potter and the Way of Reason", our familiar Harry changes to an uncle – no longer the Vernon Dursley who beats and scolds him all day long, but a professor from Oxford University .

Harry in this world has been educated at home since childhood and loves science and rational thinking. After entering the magical world, Harry was naturally assigned to Ravenclaw House to explore magic with a rational and scientific spirit.

Many people started to understand EA after reading this novel when they were young, and it even strengthened their determination to enter the field of artificial intelligence.

Perhaps, whether we side with EA or e/acc, or choose neither, we are all in an era where we are striving to uncover the principles of "magical" AI technology.

Let’s start with the “Forgetting Curse”.

I hope all AI researchers can remember Harry's kindness, bravery and moderation.

# Welcome to follow the official WeChat public account of aifaner: aifaner (WeChat ID: ifanr). More exciting content will be provided to you as soon as possible.

Ai Faner | Original link · View comments · Sina Weibo