[ET시론]In the chip war era, Korea’s AI infrastructure strategy and challenges

Cho Jun-hee, President of Korea Software Industry Association

“Chairman, finding a GPU is like picking a star in the sky.”

This is something that member companies of the Korea Software Industry Association often say recently. Considering past customs, this is an unusual complaint from the head of an organization representing software (SW).

As competition in the global artificial intelligence (AI) industry intensifies, securing computing resources such as GPUs is emerging as an important factor that determines the success or failure of a company. Meta and Microsoft (MS) purchased as many as 150,000 units of Nvidia’s supercomputer GPU, H100, last year. They announced plans to secure more than 500,000 additional copies by next year.

However, Korea has only about 4,000 H100s. Just like the 300 warriors of Sparta who fought against 1 million Persian soldiers in the movie 300, the situation of Korean companies that have to fight against global AI companies resembles their feelings of struggling under numerical inferiority.

Recently, OpenAI is speeding up the development of its own AI chips by collaborating with leading Asian foundry companies such as Broadcom and TSMC. This appears to be a strategic move to address the immediate massive demand for chips as a choice to reduce dependence on NVIDIA, which currently holds more than 90% of the global market share. In addition to OpenAI, big tech companies such as Microsoft, Google, Amazon, Meta, and Tesla are also developing their own AI chips due to the high cost and limited supply of NVIDIA chips, and Apple is also developing AI chips for data centers. .

The reason why global AI companies are jumping directly into the semiconductor market is because massive computing chips are essential for maintaining the large language model (LLM), which is the basis for AI learning and inference. With the price of one H100 reaching up to 60 million won, a company’s competitiveness greatly depends on how much it can secure. The purchasing offensive of global big tech companies mobilizing such huge capital is putting a burden on domestic companies.

So what strategies can domestic companies respond to this huge capital offensive?

It is a ‘vertical AI’ strategy based on a small language model (sLLM) that requires relatively less computing resources. Compared to LLMs such as GPT-4, which is estimated to be close to 2 trillion, this model, called ‘small-scale’ because it has fewer parameters, is more cost-effective and has fewer hallucinations during the learning process, making it more accurate and reliable. We can provide customized business services. Representative examples include MS’s ‘Phi-3’ and Google’s ‘Gemma2’.

Comparison of sLLM and LLM. Data = Current status of the Korea Software Industry Association sLLM model. Data = Korea Software Industry Association

Through the activation of sLLM-based vertical AI, there is an opportunity for domestic companies to create results with relatively small amounts of computing resources. However, since this is an area that still requires significant investment, there are bound to be limitations to the investment of a single company alone. Accordingly, it is important to establish a system in which the private and public sectors can invest together.

One of the reasons why Korea’s AI industry is lagging behind in competition with global big tech companies is the lack of an ecosystem (self-reliant ecosystem) along with issues with the supply and demand of AI semiconductors. Local production of AI semiconductors requires significant capital and time. Even for large companies, establishing a new semiconductor line and allocating research and development (R&D) resources is high risk when enormous investments are being made in existing flagship businesses. Cooperation between large corporations and startups can be a beautiful model, but there is bound to be a conflict of interest between startups that aim to develop and expand technology and large corporations that must make strategic investments.

The government must create a research and development (R&D) support fund for the local production of AI semiconductors and provide funds for participation by various entities such as startups, small and medium-sized enterprises, and research institutes as well as large corporations. Fortunately, with the recent launch of the National Artificial Intelligence Committee, the Ministry of Science and ICT is receiving positive reviews for securing resources by establishing an AI computing center and attracting public and private investments worth about 2 trillion won. The joint investment strategy will be more effective if careful policy support is provided to ensure smooth investment attraction in the future.

As interest in the spread of domestically produced neural network processing units (NPUs) grows, collaboration between fabless companies, mainly startups, and large corporations increases, and positive results are appearing one after another. However, many say that domestically produced NPUs are still somewhat inferior in performance compared to GPUs, so investment in R&D and talent training is critical. Speedy development is needed to narrow the performance gap with GPUs, and independent R&D is also needed in the system SW field to break away from the current high dependency on NVIDIA CUDA (the platform on which GPUs are implemented).

Global cooperation must be strengthened to resolve dependence on CUDA. We need to accelerate system software development and expand cooperation programs through joint research with domestic AI semiconductor companies (Sapion, Furiosa AI, etc.) and global companies such as Intel and AMD to actively introduce the latest technology trends and know-how into the country. Some domestic companies and research institutes are already developing by introducing open API and ROCm, thereby reducing their dependence on CUDA.

In addition, in order to spread domestically produced NPUs, it is important to expand the base by distributing them to universities and research institutes. By establishing a research environment based on domestically produced NPUs and providing students with experience in handling actual NPU technology, universities and research institutes have established a cooperation system that can lead to actual commercialization and systematically recruited professional talent necessary for NPU design, development, and manufacturing. It can be trained with

In the past, countries were preoccupied with military and industrial competition to secure resources, but now the era of ‘Chip War’ has arrived. Now that AI has become essential to all industries and advanced weapon systems, the government must provide long-term funding for technology commercialization while establishing a public-private cooperative ecosystem that can promote cooperation between large corporations, small and medium-sized enterprises, and research institutions.

If the government acts as a mediator and coordinator to encourage each participant to cooperate in a complementary manner and allows individual companies to focus on their business, they will be able to survive as a winner in this competition by achieving technological independence and global competitiveness.

Jo Jun-hee, President of Korea Software Industry Association [email protected]

〈Author〉 He is a software (SW) entrepreneur who founded Euracle in 2001 and has served as CEO for 23 years. Starting in 2021, he has been reappointed as the 18th and 19th president of the Korea Software Industry Association, a statutory body, and is taking the lead in developing the SW industry and improving the ecosystem. In 2022, he served as chairman of the Industrial Ecosystem Subcommittee of the President’s Digital Platform Government Committee, and was appointed as the first chairman of the public-private partnership Global DPG Alliance in 2023 and a member of the AI ​​Strategy Supreme Council in 2024. In addition, he is an advisory member of the National Science and Technology Advisory Council, a member of the Data-Based Administration Activation Committee under the Prime Minister’s Office, a senior vice president of the Venture Business Association, an outside director of the Innovation Academy, a vice president of the Korea Federation of Science and Technology Societies, and a member of the National Academy of Engineering of Korea, all of whom are experts in software and information and communication technology (ICT). We are actively engaged in policy activities for industrial development.