Nvidia’s Jen-Hsun Huang’s big bet: Walking on the edge of bankruptcy to build a trillion-dollar AI “arms dealer”

Revenue tripled to $18.1 billion, and profits soared to $9.2 billion from $680 million in the same period last year.

Under the wave of AI, Nvidia has surpassed expectations in the latest quarter, taking the crown from TSMC and becoming the "King of Chips" with its "explosive" performance.

There is a war going on in artificial intelligence, and Nvidia is the only arms dealer.

One Wall Street analyst once commented.

The "AI dividend" Nvidia enjoys today comes from Huang Renxun's "big gamble" more than ten years ago. Through a recent in-depth report in The New Yorker, we can see more details behind this decisive "gamble."

Success is never guaranteed, and bankruptcy is always on the verge.

The “big bang” moment that ignited artificial intelligence

It was the first 8K resolution game console, and it took up the entire wall. It was so beautiful.

In 2000, Stanford student Ian Buck built his own high-definition game console by connecting 32 Nvidia GeForce graphics cards together and adding 8 projectors to play "Quake".

Initially, the success of NVIDIA GeForce came from the help of the game "Quake". In the game's "Deathmatch" mode, the parallel computing of the GPU gives players a speed advantage, so every time GeForce releases a new product, players will keep up.

Buck was also curious about what GeForce could do besides making himself throw grenades faster.

Later, Buck successfully hacked into the graphics card's original programming tool "shader" and used its parallel computing to turn GeForce into a low-cost supercomputer.

It didn’t take long for Buck to become an NVIDIA employee.

▲Ian Buck is now the vice president of NVIDIA

Huang Renxun wanted Buck to make a set of software that would turn every GeForce into a supercomputer. At the same time, the hardware team is also allowed to carry out corresponding modifications in the chip structure.

In 2006, Buck's CUDA for NVIDIA was officially launched, allowing researchers and programmers to use programming languages ​​to more personally and efficiently utilize the computing power of GPUs.

However, consumers have little interest in the supercomputer that Huang Renxun wants to popularize. "Acquired", a popular technology podcast in Silicon Valley, commented:

They spent huge sums of money on this new chip architecture.

They spent billions of dollars with the goal of serving a niche area of ​​academic and scientific computing, which was a small market at the time—certainly smaller than the billions they invested.

At that time, NVIDIA was also casting a wide net and trying to find target customers. I tried stock traders, oil exploration companies, molecular biologists, etc., but I didn't consider the field of artificial intelligence.

It didn’t even feel like the “AI Godfather” took the initiative to “come to your door”.

No wonder.

▲"AI Godfather" Geoffrey Hinton

Today, we would call Geoffrey Hinton the "Godfather of AI."

However, in 2009, Hinton was in the field of AI that was disliked by capital, and his research in this field was still considered a niche "neural network".

Hinton wrote this email to Nvidia that year:

I just told thousands of machine learning researchers that they should all buy NVIDIA graphics cards. Can you send one to me for free?

result? Of course it was rejected.

Before that, Hinton had tried to use the NVIDIA CUDA platform to train a neural network to recognize human language. He found that the quality of the results was much better than expected, so he decided to present it at an industry conference.

Although Nvidia refused to send Hinton a graphics card, Hinton still encouraged students to use it.

The most critical among them are his two outstanding programmers Alex Krizhevsky and Ilya Sutskever.

▲(from left to right) Ilya Sutskever, Alex Krizhevsky and Geoffrey Hinton

Sharp-eyed readers should have discovered that the latter is the chief scientist of OpenAI, the person who leads the technology behind ChatGPT.

In 2012, Sutskever and Krizhevsky bought two NVIDIA GeForce graphics cards, fed millions of image data into the neural network within a week, and trained "AlexNet". Sutskever recalled afterwards:

The GPU came along and it felt like a miracle.

His sigh was not without reason.

Also in the same year, Google purchased more than 16,000 CPUs to train their neural network so that it could recognize cat videos.

However, AlexNet can correctly identify images of electric vehicles, cheetahs, cargo ships, etc., using only two GPUs.

In 2012, in the ImageNet Large Scale Visual Recognition Challenge, which was still quite authoritative that year, AlexNet won the championship with a top-5 error of 15.3%, which was far better than the second place and previous contestants. He was so outstanding that he was once suspected of cheating. Hinton commented:

It was kind of a big bang moment. A paradigm shift.

Although it was not intentional, NVIDIA ignited the "big bang" moment of artificial intelligence.

Become an AI company

(Huang Renxun) He sent an email on Friday night saying that everything in the company will revolve around deep learning and that we are no longer an imaging company.

Early the next Monday morning, we became an AI company.

Really, it's that fast.

Nvidia Vice President Greg Estes told The New Yorker.

After the debut of AlexNet, within a few years, almost everyone participating in large-scale visual recognition challenges chose the form of neural networks.

By the mid-2010s, the accuracy of image recognition of neural networks trained with GPUs had reached 96%, a level of accuracy that even exceeded that of humans.

Huang Renxun's supercomputer vision came true, and he started looking for his next goal:

The fact that we can solve a computer vision problem, a completely unstructured problem, points to the question: "What else can you teach it?"

Huang Renxun's inner answer seems to be – everything.

He believes that neural networks will change society, and he can also use CUDA to monopolize the market for the hardware required behind them.

He took a leap and started Nvidia's AI journey.

This time, leaders in the AI ​​industry no longer need to write emails to Nvidia to apply for free graphics cards.

In August 2016, Huang Renxun personally delivered the world's first DGX-1 to OpenAI's office.

Musk, who had not yet broken with OpenAI at the time, personally unboxed the product that took 3,000 people three years to build.

In the official press release, Huang said jokingly:

If this were the only product shipped, the project would cost $2 billion.

Who would have thought that the next year, Google would announce a new neural network training architecture, Transformer.

This new breakthrough was captured by Sutskever, leading OpenAI to build the first GPT model, all built on NVIDIA supercomputers.

One year ago today, OpenAI officially released ChatGPT to the public, changing everything, including Nvidia.

There are endless orders and supply exceeds demand.

In 2023, Nvidia's stock price soared by more than 200%, becoming the world's first chip manufacturer with a market value exceeding one trillion US dollars.

CUDA, which was once not optimistic, has also gathered 4 million developers and has become another "moat" for NVIDIA in the AI ​​field.

Whether it is research in aerospace, bioscience, machinery, energy exploration and other fields, most of them are conducted on CUDA.

NVIDIA's latest AI product, the DGX H100, is a metal box weighing 370 pounds and priced at $500,000.

Compared with the DGX-1 sent to the OpenAI office at that time, the new product runs five times faster.

If you want to train AlexNet, you can do it in one minute.

The winner who is always "on the verge of bankruptcy"

In September this year, Huang was invited back to the Denny's restaurant in San Jose, California.

At that time, he drafted documents with his partners in the booth of this restaurant and established Nvidia.

They wanted to design a chip that would make competitors "green with envy." Jen-Hsun Huang came up with the name "Nvidia", incorporating the Latin word "invidia".

Nowadays, Nvidia is certainly jealous of its competitors. Even the CEO of the restaurant chain Denny's specially made a commemorative plaque for them, so that Nvidia's light can shine in the restaurant.

However, Nvidia's success is not a particularly typical "winner" story.

When NVIDIA was first established, Huang Jen-Hsun, who loved video games, believed that the gaming market deserved better graphics cards, and launched its first product, NV1, in 1995.

However, NV1 has not really been accepted by the mainstream market. One of the reasons is that Microsoft launched the D3D API in the same year, but NV1 does not support D3D. The next generation product, NV2, also failed.

Huang Renxun, who lost his "bet" once, was not convinced. In 1996, he laid off half of his employees, tightened funds, and bet everything on untested new products:

The odds are 50/50, but we are already on the verge of bankruptcy either way.

When the RIVA 128 was officially launched, NVIDIA only had enough money left to cover one month's expenses. Fortunately, RIVA 128 was a success, with millions of units sold within 4 months.

Since then, Huang has encouraged employees to work with this "desperation."

For Jen-Hsun Huang, difficulties and failures are no stranger:

I find that I think most clearly when I'm in a difficult situation.

My heart rate would even drop.

He even insisted that "failures must be shared."

Previously, Nvidia had sent out a problematic graphics card with a very loud fan.

Huang Renxun did not fire the manager in charge of this product. Instead, he held a meeting, gathered hundreds of people, and asked the manager to describe every decision that ultimately led to this farce.

Showing "failure" has become a "custom" within NVIDIA.

You can also quickly tell from this who can stay here and who can't.

If someone starts to put up defenses, I know they won't be around long.

said Dwight Diercks, head of software at Nvidia.

Huang also likes to encourage employees to pursue "zero-billion-dollar markets"—experimental areas where there are no competitors and no clear customers.

After all, as Huang Renxun said:

I always thought we were only 30 days away from bankruptcy. This has never changed.

There's no reason not to give it a try.

# Welcome to follow the official WeChat public account of aifaner: aifaner (WeChat ID: ifanr). More exciting content will be provided to you as soon as possible.

Ai Faner | Original link · View comments · Sina Weibo