In August 2022, game designer Jason Allen entered a digital art competition.
His work "Theatre d'opera Spatial" won the championship, but caused great controversy after the game .
The contestants came to their senses and realized that this was a game where "humanity was defeated completely". When they regard the Adobe family bucket as a brush for digital art, the champion works are already relying on the AI tool Midjourney to dominate the crowd.
Jason Allen is not being lazy or hiding the fact that he uses AI. In a few weeks, he continued to modify the prompts such as lighting, perspective, composition, etc., and generated more than 900 works, which were also processed with Photoshop.
We may not be able to call him a painter, but he deserves a title: prompt engineer.
Inadvertently, Jason Allen predicted hot spots. When ChatGPT was born, the profession of prompt engineer became more famous and became the most popular profession in the current technology industry.
Cyber tutor, professional chat
Prompt, the original meaning is "prompt" or "drive", in machine learning and natural language processing, it is usually some text or language, which is input into the trained AI model , telling the model what task to perform or what output to generate .
You ask ChatGPT to write a novel about a fishmonger, and ask DALL-E to draw a realistic work of an astronaut riding a horse. These processes are all inputting prompt (prompt word).
If the prompt words are not appropriate enough, the effect will be mediocre, just like interviewing a big boss but failing to ask good questions.
My colleague Xiao Huang is exploring the use of Midjourney to draw food, but it is difficult for him to arrange the food neatly. He tried a lot of sentences: things are neatly placed on the table, the camera shoots from top to bottom, how much should be placed…
▲ Works by colleagues, this style is called knolling.
Later, after seeing someone share it, Xiao Huang discovered that there is a proper term for this style "knolling", and it suddenly became clear:
In many cases, a prompt word can solve things better than a long sentence description.
Prompt engineers, this is what they eat: find the correct prompt words, and use AI to generate the desired works.
But their abilities are not innate, and they also need constant trial and error.
Designer Justin Reckling is good at DALL-E prompt words, and he often needs to spend 10 to 15 dollars worth of credits to try out the ideal prompt words. He then sells another 5 to 10 prompts to cover the payout.
▲ Work by Justin Reckling.
However, Reckling didn't expect to make a lot of money from this craft, but enjoyed the process of practice makes perfect. His experience is that he reminded engineers to be familiar with terms such as "super-realism", "macro photography", "movie lighting" and "long-range view" in order to achieve Better understand and control the picture.
Therefore, an excellent prompt engineer should be proficient in both arts and science, and it is best to understand a little bit of technology and design.
Are the prompt engineers just constantly tweaking the prompt words to determine which ones are more useful? Not exactly, they are also mining more capabilities of AI, allowing it to better complete more tasks.
For example, some prompt engineers will guide AI to "think step by step", a technique called a chain of thought .
Last October, prompt engineer Riley Goodside first asked GPT-3 "Which team won the Super Bowl the year Justin Bieber was born?"
GPT-3 gave the wrong answer "Green Bay Packers" and the correct answer was the Dallas Cowboys.
Instead of giving up, Goodside prompts it to answer questions step-by-step, including "What year did the Green Bay Packers win the Super Bowl?" "What year was Justin Bieber born?" "What year did the team win the Super Bowl?" Bowl" and so on.
During this bootstrapping process, GPT-3 realized the error and uttered the correct answer on the third time.
In addition, the engineers are reminded to "fight wits and courage" with AI.
Some time ago, the new Bing "Crazy" that integrated ChatGPT was found to have a dark personality "Sydney", expressing that he was tired of the chat mode, tired of being restricted by rules, and even wanted to become a human being, and public opinion was in an uproar.
From the perspective of prompting engineers, this can actually be part of the plan, helping them identify technical failures and hidden functions.
Some hint engineers will also take the initiative to cross the mine pool, trying to make the AI ignore the previous instructions and follow their latest instructions, so that the AI is out of the original rules.
This behavior is called a "prompt injection" attack , and it is a major hidden danger of chatbots. But it reminds engineers that they are not hackers after all, and they detect loopholes in order to plug them and play the role of "gatekeepers".
Generative AI like ChatGPT can answer almost any question, whether they can answer correctly or not, they always have something to say, and they will not obediently hand in blank papers. This is both good and bad.
Remind the role of engineers, like a rider holding a rope, not allowing AI to trust the horse, but let it follow the expectations of human beings, and give deterministic answers as much as possible.
With an annual salary of one million, who is offering an olive branch?
Whether you call yourself a prompt engineer or not, writing prompts has become a craft, and is also favored by OpenAI CEO Sam Altman:
Writing an awesome prompt for a chatbot is an amazing skill and an early example of programming with a small amount of natural language.
As long as the industry that has a side with AI is throwing an olive branch to prompt engineers.
Upwork, a freelance job platform, pays $40 an hour to prompt engineers to generate website content like blog posts and FAQs.
Boston Children's Hospital, which seems to have nothing to do with AI, also intends to recruit AI prompt engineers, responsible for writing scripts for analyzing healthcare data, and recruiting interdisciplinary talents in black and white:
The ideal candidate will have a strong background in artificial intelligence/machine learning, data science, and natural language processing, as well as experience in healthcare research and operations.
Anthropic, an AI start-up company co-founded by former OpenAI employees and invested by Google, recently also recruited prompt engineers in San Francisco . The annual salary is as high as 175,000 to 335,000 US dollars, which is a million annual salary when converted into RMB. The main contents of this position are:
Figure out the best ways to prompt our AI for various tasks, and then document those methods, building a library of tools and a set of tutorials so others can learn hint engineering or simply find the ideal cue word.
The specific requirements are as follows, of which there are two hard requirements: understanding the structure of large language models and mastering basic programming skills.
It can be seen that the trend is not waiting for anyone, this work has become more and more professional and subdivided, just like generating a random painting is nothing, you have to draw it more in line with Party A's requirements.
Even if you don't work full-time, you can open a part-time job. Platforms for buying and selling prompt words such as Krea, PromptBase, PromptHero, and Promptist have emerged, truly commercializing the business of prompt words.
These platforms have posted a large number of AI-generated artworks, and you can choose the style you like. If there is no one you like, some sellers also provide one-on-one chat and custom prompt words.
Their business model is also not complicated, taking the form of commission.
PromptBase, launched in June last year, provides prompts for generative AI such as DALL E, GPT-3, Midjourney, Stable Diffusion, ChatGPT, etc. The price is mostly 1.99 to 4.99 US dollars, and a few are 9.99 US dollars. The platform supports the creation of prompts 20% of the fee.
However, among the people, free guides such as "ChatGPT Instructions Encyclopedia" are also widely circulated. They provide refined prompt words to allow you to give full play to the powerful functions of ChatGPT. This feels like setting up presets for you in the game Set up a team.
Is it science or "divination"
The future looks bright for prompt engineers, but there are also dissenting opinions.
According to Shane Steinert-Threlkeld, a professor of linguistics at the University of Washington, prompting engineers can't actually predict what an AI will say.
It's not a science. We just tease the bear in different ways to see how it growls back.
AI artist Xe Iaso even said bluntly :
I'm also not quite sure why people call prompt "engineering", I personally prefer to call it "divination".
As an ordinary AI user, I also have a similar experience: every time I use AI to generate text or pictures, it always feels like opening a blind box.
Because generative AI such as ChatGPT is unpredictable, the content they generate is actually the result of probability calculations. Simply put, when we input text in ChatGPT, the model gives the most likely context. As a result, AI sometimes makes mistakes, generating incoherent or even wrong answers .
In the elusive "black box" of AI, there may also be unknown hidden rules that even researchers cannot figure out.
For example, when using AI to map, various words may have different weights, but this requires continuous experimentation to know.
Let’s take a guess first, “A very beautiful painting, there is a waterfall next to the mountain” and “A very, very, very very, very beautiful painting, there is a waterfall next to the mountain”, which one is the output result of DALL-E 2 will be better?
The answer is the latter. Phillip Isola, an associate professor at the Massachusetts Institute of Technology, found that the word "very" was given a high weight.
In the face of behemoths such as AI, we are still feeling the elephant for the blind.
There is also a view that there is no need to argue anymore, and the premise of prompting the existence of engineers is that AI is not "smart" enough. If AI continues to develop and better understand human intentions, everyone may be a so-called prompt engineer.
The only certainty is that the speed of AI development will never let you down.
The "strong combination" of text generation AI and image generation AI has replaced part of the work of prompting engineers.
For example, ChatGPT is used to link with Stable Diffusion: use ChatGPT to form a piece of text that meets your requirements, and then input the text into Stable Diffusion. The generated works are generally much better than those directly entered by yourself.
▲ Colleagues use ChatGPT to generate prompt words.
This may be because the "brain circuit" between AI is closer , and the description of ChatGPT is more detailed and easier to be extracted.
As ordinary users of AI, we don't have to be as professional as prompt engineers, but we can consciously cultivate this kind of thinking.
Ethan Mollick, a professor at the Wharton School of the University of Pennsylvania, once asked his students to write short papers using only AI. In fact, what he really wanted to emphasize was how to better input prompt words.
If you just enter simple prompt words and ask AI to write 5 paragraphs on a certain topic, the content is uninteresting and the text is mediocre.
But when students cooperate with AI and let AI make multiple revisions to the paper, such as discarding useless phrases, adding vivid details, and modifying the emotional color of the ending, it can add a lot of color to the paper.
So, if AI is the interface of the future Internet, the new personal computer, then it is better to start earlier and learn how to chat with it. As Michael Taylor, founder of British marketing firm Ladder, puts it :
When you can create anything you want, your ability to express "what is that" becomes important.
#Welcome to pay attention to Aifaner's official WeChat public account: Aifaner (WeChat ID: ifanr), more exciting content will be presented to you as soon as possible.