Disruised AI technology landscape and restricted production: The reason China struggles to abandon Nvidia hardware for AI development

In the ever-evolving world of artificial intelligence (AI), China has been making significant strides in developing its own AI accelerators. One of the key players in this race is Huawei, a company that recently open-sourced its AI toolkit, called CANN, specifically optimized for its Ascend hardware.

Earlier this year, Huawei made its CANN software stack open source, aiming to position its Ascend platform as the baseline software ecosystem others could rely on. This move could potentially bring CANN closer to what CUDA offers, which has been a popular choice for many AI developers due to its stability and versatility.

However, the path for Chinese AI companies, including Huawei, has not been without challenges. The U.S. has imposed sanctions on China's high-tech sectors, leading to the ban on sales of advanced AI accelerators like AMD's Instinct MI308 and Nvidia's HGX H20 to Chinese entities. This ban has been particularly problematic for Huawei, as it has had to obtain the majority of silicon for its Ascend 910B and Ascend 910C processors by deceiving TSMC.

The open-sourcing of CANN allows for a broad community of developers from academia, startups, and other enterprises to experiment with performance tuning or framework integration. This could be a significant advantage for Huawei, as it seeks to challenge the dominance of Nvidia's ecosystem.

China's government has also taken steps to address this issue by setting common mid-level standards and forming the Model-Chip Ecosystem Innovation Alliance. The goal of this alliance is to build a fully localized AI stack linking hardware, models, and infrastructure, with the aim of achieving interoperability among shared protocols and frameworks to reduce ecosystem fragmentation.

One of the key initiatives under this alliance is DeepSeek, a group aiming to develop a fully localized AI stack that integrates hardware, models, and infrastructure. However, challenges remain, as the maturity of Huawei's CANN (and competing stacks) lags behind Nvidia's CUDA due to a lack of a broad, stable installed base of Ascend processors outside Huawei's own projects.

In an attempt to counteract these challenges, DeepSeek reportedly had to abandon training of its next-generation R2 model on Huawei's Ascend platforms due to unstable performance, slower chip-to-chip connectivity, and limitations of Huawei's Compute Architecture for Neural Networks (CANN) software toolkit.

Despite these hurdles, China has been persistently working towards self-sufficiency in its semiconductor industry. Since the mid-2010s, China has had a focus on supercomputers and fab tools, and it has created several domestic AI accelerators, including Huawei's CloudMatrix 384.

However, the lack of common standards and the low volume of China-developed AI accelerators, combined with intense competition on various fronts, will make it very hard for Chinese companies to challenge Nvidia's dominance. This is further complicated by the fact that Chinese foundries like SMIC cannot match TSMC's process technologies, making it difficult for China to produce hardware that is on par with AMD or Nvidia in volume.

In response to these challenges, the U.S. government announced plans to grant export licenses to AMD and Nvidia for China-specific AI accelerators. However, President Trump announced an unprecedented 15% sales tax on AMD's and Nvidia's hardware sold to China, which could potentially hinder the progress of Chinese AI companies.

As the race for AI dominance continues, it will be interesting to see how Huawei and other Chinese AI companies navigate these challenges and whether they can successfully challenge Nvidia's established ecosystem.