NVIDIA cut off the supply of high-end chips in advance, and the computing power of Chinese companies broke through

Original source: Silicon-based laboratory

Image source: Generated by Unbounded AI

On October 17, the U.S. Department of Commerce's Bureau of Industry and Security (BIS) issued a new export ban on chips, tightening restrictions on China's purchase of important high-end chips.

Restricting China's imports of high-end chips is undoubtedly to restrain the development of China's technology industry. Previous studies have shown that for every 1 point increase in the computing power index, the digital economy and GDP will increase by 3.5‰ and 1.8‰ respectively.

However, the tightening of external restrictions has not caused the stagnation of China's computing power industry, which has passed the trillion-dollar mark. **According to the China Academy of Information and Communications Technology, by the end of 2021, the scale of China's core computing power industry has exceeded 1.5 trillion yuan, and the scale of related industries has exceeded 8 trillion yuan.

Behind the trillion-dollar market, enterprises and governments work together to seize the AI era.

On the one hand, since the launch of ChatGPT, domestic enterprises and research institutes have launched more than 130 large models in just over half a year, among which leading players have begun to apply large models to specific scenarios and create explosive applications.

On the other hand, in order to build a computing power base, local governments have started the construction of intelligent computing centers, laying the information high-speed in the era of big data, promoting industrial innovation and upgrading, and reducing the cost of enterprises calling scientific and technological achievements represented by large models.

The external chip trade has gradually cooled down, and the internal computing power market has sparkled, and between the two heavens of ice and fire, people can't help but be curious:

Which city has the breakthrough battle of China's computing power industry captured? How to break the computing power industry chain? In this process, which companies have assumed the responsibility of pioneers?

**01 NVIDIA cut off supply, affect geometry? **

"If the large language model is used as the base to process the inference requests of 1.4 billion people in China at the same time, the amount of computing required exceeds the total computing power of China's data centers by 3 orders of magnitude. ”

At the 2023 World Artificial Intelligence Conference (WAIC) in Shanghai in July this year, Wang Yu, a professor of electronic engineering at Tsinghua University, revealed the scale of the domestic computing power gap.

In fact, not only large models, but also the popularity of diversified applications in 5G, smart cities, and the Internet of Things has also brought about the continuous acceleration of data generation.

IDC predicts that the scale of China's intelligent computing power will reach 1271EFLOPS in 2026, with a compound annual growth rate of 69.45%. As of the end of 2022, the "2023 Intelligent Computing Power Development White Paper" compiled by New H3C Group and the China Academy of Information and Communications Technology shows that the total domestic computing power is only 180EFLOPS. (Note: FLOPS refers to floating-point operations per second, and 1271EFLOPS means 1271 exascale operations per second.) ）

** In order to solve the current situation of computing power shortage, the state has successively issued a number of documents to support and guide all localities to accelerate the construction of computing power infrastructure. **

Among them, the Action Plan for the High-quality Development of Computing Power Infrastructure released in October clearly states that the scale of computing power will exceed 300EFLOPS in 2025, of which the proportion of intelligent computing power that can be used for large model training needs to reach 35%.

At present, there are about 31 intelligent computing centers funded by the government, corresponding to the total computing power of 10.13EFLOPS in the plan, with a total investment of nearly 47 billion yuan, which is still far from the planned total intelligent computing power scale of 105E, 50 intelligent computing centers, and single-center computing power scale of 2.1EFLOPS.

**In fact, not only in China, but also in the world, there is a shortage of computing power. According to OpenAI data, there is a 10,000-fold gap between the growth rate of model computing volume and the growth rate of artificial intelligence hardware computing power. **

The shortage of computing power first contributed to the skyrocketing price of GPUs. Since December last year, the price of NVIDIA A100 has increased by nearly 40% in 5 months. This year's new H100 is even more priceless.

Due to the influx of orders, the delivery cycle of NVIDIA, the GPU manufacturer with the highest market share, has been extended from one month to more than three months, and even some orders may not be delivered until 2024. The main reason is that the chip supply chain is long and fragmented, and it is impossible to quickly expand production capacity.

**Due to the restrictions of the US ban, domestic manufacturers' plans to expand computing power are more difficult to implement than Google, Meta, and OpenAI. **

Before announcing the new round of ban, Nvidia adapted the restriction rules by supplying the Chinese market with "castrated versions" of flagship computing chips A800 and H800, which have reduced interconnection speeds.

In August, media reported that companies such as Baidu, Tencent, Alibaba and ByteDance had ordered $5 billion in chips from Nvidia. Of that amount, $1 billion has been ordered for the A800, which is expected to be delivered this year. The remaining $4 billion order will be delivered in 2024.

After the announcement of this ban, due to the performance density as a relevant requirement to limit the new standard, A800 and H800 chips, because of exceeding the standard, will also be completely banned.

In NVIDIA's updated 8-K filing with the U.S. Securities and Exchange Commission (SEC), it is mentioned that the U.S. government has advanced the effective time of the ban on NVIDIA's five GPU chips, including A100, A800, H100, H800 and L40S, from the original end of November to take effect immediately.

**The above changes mean that the approximately 100,000 A800 chips that BAT has already ordered are likely to not be delivered. **

However, the domestic computing power infrastructure does not seem to be affected much. At present, there are nearly 30 intelligent computing centers that are under construction or completed, of which more than 50% of the chip suppliers are Huawei Ascend.

Previously, Liu Qingfeng, chairman of iFLYTEK, said at the press conference that the performance of Huawei Ascend 910B can already be benchmarked against the A100.

**On the whole, although the further tightening of US restrictions on China has dragged down the progress of iteration of large models of some Internet giants, the domestic computing power infrastructure is still steadily advancing. **

And because the difficulty of importing chips will continue to rise in the foreseeable future, for supply chain security considerations, domestic chip manufacturers are expected to usher in a new wave of development opportunities.

02 Computing Power Breakthrough: Left-handed Self-research, Right-Hand Ecology

Although the only GPU manufacturers that are generally recognized in the international market are NVIDIA and AMD, this does not mean that there are no other choices than them.

**Compared with ASIC chips, GPUs have the advantage of strong versatility and are suitable for various research fields. However, subdivided into various enterprises, in fact, there is a general excess computing power, ** such as the large model inference ability that only needs to use the GPU, and does not need its graphics computing power.

Therefore, many manufacturers have embarked on the road of independent research and development according to their own needs. **

For example, Alibaba released its self-developed chip Hanguang 800 in May this year, which is said to be the strongest performance in AI chips at that time, with computing power equivalent to 10 CPUs; Baidu's self-developed cloud full-function AI chip Kunlun has also been iterated to 3.0 and will achieve mass production in 2024.

Among the self-developed chip companies, the loudest is undoubtedly the aforementioned Huawei.

Recently, the Spark all-in-one machine jointly created by Huawei and iFLYTEK has been put on the cusp again.

According to public information, the Xinghuo all-in-one machine is based on Kunpeng CPU + Ascend GPU, using Huawei storage and network to provide a complete cabinet solution, with FP16 computing power of 2.5 PFLOPS. In contrast, the NVIDIA DGX A100 8-GPU, which is the most popular in large model training, can output 5PFLOPS FP16 computing power.

"Wisdom" has reported that in specific large model scenarios such as Pangu and Xunfei Xinghuo, the Ascend 910 has slightly exceeded the A100 80GB PCIe version, achieving domestic replacement. However, the versatility is still insufficient, and other models, such as GPT-3, need to be deeply optimized before they can run smoothly on the Huawei platform.

In addition, Moore Thread and Walltech that were newly included in the entity list in this round of sanctions also have corresponding GPU single-card products, and some indicators are close to NVIDIA.

In addition to the impact of US sanctions, self-developed chips can also weaken the over-reliance on NVIDIA, enhance the strategic autonomy of enterprises, and take the lead in expanding the scale of computing power ahead of competitors. **

One proof is that even companies such as Google, OpenAI, and Apple that are not subject to sanctions have launched plans to develop their own chips.

In order to no longer be subject to a single supplier, some server manufacturers have also begun to adopt an open architecture that is compatible with domestic independent innovation chips. **

For example, Inspur Information, which currently accounts for the highest market share of domestic servers, has launched an open computing architecture, which is said to have the characteristics of large computing power, high interconnection and strong expansion.

Based on this, Inspur released three generations of AI server products, realized the landing of multiple AI computing products with more than 10 chip partners, and launched the AIStation platform, which can efficiently schedule more than 30 AI chips.

**Objectively speaking, server manufacturers are a relatively weak link in the computing power industry chain, the upstream needs international giants with monopoly positions like NVIDIA to purchase chips, and the downstream is G-end and cloud manufacturers, which lack bargaining power from top to bottom. **

So we can see that although NVIDIA's revenue in a single quarter hit a record high, reaching $13.51 billion, a year-on-year increase of 101%, and net profit soared 843% year-on-year to $6.188 billion, Inspur's net profit in the first half of this year is still in the red.

**In order to ensure that they can survive to the trillion market cash, server vendors are sparing no effort to prove their value. Specifically, it provides AI server cluster management and deployment solutions to ensure high availability, high performance, and high efficiency of servers.

At the same time, manufacturers are also competing to launch industry reports, standards and guidelines in the hope of gaining a voice.

With self-developed chips in the left hand and open ecology with the right hand, the domestic computing power industry chain is in an unprecedented complex situation, with both competition and cooperation between them.

In the long run, the real decisive factor in the breakthrough of computing power is still technology, which covers ecology, software and hardware, etc., which requires upstream and downstream players to make a hole to overcome the difficulties together.

But before really going through the independent road of the chip, the more critical is how to use every cent of computing power on the blade, to some extent, the answer to this question also hints at the outline of the players who will win the 100 billion market in the future. **

03 Using computing power well is a top priority

Before answering how to use computing power well, you need to think about another question: how to use computing power to use it well?

**The dilemma facing the domestic computing power industry is mainly threefold: **

**First, lack of computing power. **High-quality computing power resources are insufficient and scattered, GPU increments are limited, and the stock is seriously insufficient, which is difficult to further support large model training, and gradually becomes a new "stuck neck" problem.

Second, computing power is expensive. **Computing power infrastructure is an asset-heavy and capital-intensive industry, with the characteristics of large initial investment, fast technology iteration, and high construction threshold, and its construction and operation require huge time and capital costs, far beyond the scope of small and medium-sized enterprises.

**Third, the demand for computing power is diversified and fragmented, and mismatches between supply and demand of computing power resources occur from time to time. **

The first dilemma is being solved, but it is not a one-day effort, so at this stage, the actual meaning of using computing power should be to make computing power less expensive and able to handle diversified needs.

So, which companies have the most imaginative moves?

**In terms of reducing consumption and increasing efficiency for the intelligent computing center, Alibaba's concept of "greening the whole industry chain of computing power" is worth looking forward to. **

As we all know, the energy cost of large model training is very high. But in reality, only 20% of this power is used for the computation itself, and the rest is used to keep the server running. Google's 2023 environmental report confirms this from the side. According to the report, Google consumed nearly 5.2 billion gallons of water in 2022 to cool data centers, equivalent to 1/4 of the world's daily drinking water, and can fill one and a half West Lakes.

In order to achieve a greater degree of overall energy saving and emission reduction effects, Ant Group and the China Academy of Information and Communications Technology (CAICT) released the White Paper on Computing Greening for Computing Power Applications, which put forward the concept of "end-to-end green computing".

Specifically, end-to-end green computing is to consider the energy consumption cost during operation in the early stage of construction, from power production, computing power production (including intelligent computing center builders, hardware manufacturers, cloud vendors), to computing power applications.

To some extent, based on the proportion of energy use in the past, the cost reduction brought by the greening industry chain may be more cost-effective than the breakthrough of chip technology in the short term, which is conducive to the digital intelligence upgrade of small and medium-sized enterprises.

**In terms of improving the level of computing power scheduling, Huawei, Alibaba, Tencent, Baidu and other enterprises have all contributed their own strength, but among them, the most compatible enterprise genes are still Huawei. **

At present, the most core computing power scheduling project in China is the "East Data and West Computing" project first explicitly proposed in the "National Integrated Big Data Center Collaborative Innovation System Computing Power Hub Implementation Plan" in 2021, aiming to build the task of the national computing power network system.

Storing and processing data in the east in the west presents great challenges on both the supply side and the distribution side.

Take the common packet loss problem as an example.

When multiple servers send a large number of packets to a server at the same time, the number of packets exceeds the cache capacity of the switch and packet loss occurs, which in turn affects the efficiency of computing and storage.

To solve this problem, Huawei introduced intelligent algorithms into data center network switches, collected real-time network status information, such as queue depth, bandwidth throughput, traffic model, and other dimensions, and dynamically set the ideal queue pipeline through intelligent lossless algorithms, finally achieving a balance of no packet loss, high performance, and low latency after simulation training.

In addition, Huawei has innovated technologies such as distributed adaptive routing and intelligent cloud map algorithms to participate in the design and construction of national hub nodes.

As domestic large models become more and more practical on the road to empowering thousands of industries, the question of "how to solve China's computing power dilemma" will become more and more important. We can see that China's computing power industry chain has produced many changes, such as the Internet giants adding code self-developed chips, the computing power base built with domestic chips, and the germination of the software ecology that was not valued in the past... Behind these changes is the perseverance and determination of Chinese enterprises to break through technical barriers.

** Objectively speaking, in terms of technical strength, domestic players still have a certain distance from world-class manufacturers, but it cannot be ignored that even NVIDIA, which is in full swing, has been hovering on the edge of life and death for many years before the advent of the AI era. **

The night before dawn is darkest, but the sun's rays are already over the horizon.

Resources:

AI server shortage truth investigation: the price increased by 300,000 in two days, and even the "MSG King" entered the market|Wisdom Stuff

Intelligent computing power new infrastructure superimposed overseas multi-modal upgrade, computing power application to meet the catalyst | Zheshang Securities

Technology Chain Master, Huawei Ecosystem | TF Securities

Training demand blowout "thirst for computing power" how to solve | Netinfo Jilin

• The US chip ban has intensified! NVIDIA, Intel or Limited | 21st Century Economic News*