“The Fastest on Earth” AI Chip Released, Harvard Dropout Undergraduate Challenges NVIDIA

When talking about keywords such as AI, large models, and computing power, if you want to mention hardware products, many people will say NVIDIA without thinking.

One Wall Street analyst once commented:

There is a war going on in artificial intelligence, and Nvidia is the only arms dealer.

Indeed, in the AI ​​era, NVIDIA's position is unique and difficult to shake. But now, one brave man is ready to officially challenge the situation.

In 2022, Gavin Uberti and Chris Zhu founded Etched.ai. They have received $5.36 million in seed round investment from the beginning, including former Ebay CEO Devin Wenig.

After just announcing US$120 million in financing, Etched has released the fastest AI chip currently – Sohu.

This Sohu is not that Sohu, which claims to be able to process thousands of words in milliseconds. It can process more than 500,000 tokens per second and run the Llama 70B model.

What concept? This is equivalent to writing the Bible in two seconds, or writing 15 trillion tokens in a year.

According to the Etched official website, Sohu integrates the Transformer architecture at the hardware level and is the first dedicated chip (ASIC) designed specifically for the Transformer model.

It is this specialization that brings Sohu a huge improvement in performance. Compared with NVIDIA H100, Sohu's inference speed is increased by 8 to 10 times.

A server equipped with eight Sohu chips can replace a full 160 NVIDIA H100 GPUs.

For programming scenarios, Sohu can respond to hundreds of results in parallel through Monte Carlo tree search to give the best answer.

Whether it’s a voice assistant or intelligent customer service, Sohu can analyze thousands of words in milliseconds.

And Sohu also supports multiple speculative decoding, which can generate new content in real time at extremely fast speeds.

As the world's first Transformer application-specific integrated circuit (ASIC) chip, each Sohu chip has only one core, uses TSMC's 4nm manufacturing process, and is equipped with 144GB HBM3E high-bandwidth memory.

Compared with Nvidia's next-generation Blackwell (B200) GPU, Sohu is an order of magnitude faster and cheaper.

In just two years, Etched has grown from scratch to what it is now, which is really a "miracle".

The founders of Etched, Gavin Uberti and Chris Zhu, are both of Chinese descent, and both of them also have rich backgrounds.

Gavin Uberti and Chris Zhu both attended Harvard University and spent a period of their undergraduate lives. Later, for some reasons, they chose to take a year off and found an internship at OctoAI.

The two people's work content at that time was mainly responsible for the development of the ApacheTVM open source compiler and microkernel. But during the internship, they found that some designs in Arm's instruction set were very inefficient, making their work efficiency very poor.

Gavin Uberti said: "This problem cannot be solved. I have to face this problem every time I work, which makes Chris and I feel that we must do better."

When thinking about how to solve this problem, they suddenly discovered that they could use this idea to design an AI acceleration chip.

So the two chose to drop out of Harvard and founded an AI chip startup company focused on LLM – Etched.

In Gavin Uberti's view, the general public version design cannot bring about breakthrough performance improvements, so they developed a dedicated acceleration chip under the Transformer architecture.

Gavin Uberti once said:

We must make great efforts on a single architecture to let the chip handle AI tasks. The goal is too big, and the chip must be designed for more specific tasks… We think Nvidia will eventually do this.

Current mainstream AI products such as ChatGPT, Claude, Gemini and Sora are all driven by the Transformer architecture.

As the world's first Transformer ASIC chip, coupled with such outstanding results, Etched has naturally attracted a lot of attention.

On June 25, Etched announced the completion of a $120 million Series A financing. The financing was co-led by New York venture capital firms Primary Venture Partners and Positive Sum Ventures.

Its angel investor lineup is also very strong, including venture capitalist Peter Thiel, GitHub CEO Thomas Dohmke, self-driving company Cruise co-founder Kyle Vogt, and Quora co-founder Charlie Cheever.

When it was founded in 2022, Etched envisioned a future where Transformer would dominate. Gavin Uberti said, "We are making the biggest bet in artificial intelligence."

If we get our bet right, Sohu will change the world. That's why we dare to bet.

# Welcome to follow the official WeChat public account of aifaner: aifaner (WeChat ID: ifanr). More exciting content will be provided to you as soon as possible.

Ai Faner | Original link · View comments · Sina Weibo