How does Shengteng AI strong model development and innovation root?

Since Google released BERT in 2018, after several years of development, the pre -training model has swept the major AI lists and test data sets represented by NLP with a strong algorithm effect. The NLP large model GPT-3 released by Openai in 2020 achieved 100 billion-level parameters. The strong ability of Bert and GPT has become a milestone in the AI ​​field. The significant advantage of large models has also allowed industry giants and institutions to participate in it.

The advantages of excellent generalization capabilities, general AI capabilities, high accuracy, and multi -business scenarios have reduced the threshold for AI development and applications, and also made the "big model" a trend of the AI ​​industry. But with computing power and big models, is the AI ​​industry innovation and landing applications carefree? The answer is not that simple. Many industrial needs cannot be treated with general models. There are still exchanges between technical theory and application scenarios; Difficulty, hardware compatibility, etc.

How to let the big model go out of the laboratory, move towards the industry, and promote the innovation of the industry, it has become a difficult problem in front of AI manufacturers. So, how to complete its own evolution, to adapt to use scenarios, and further promote the development of the AI ​​industry? In this regard, Huawei has some methods and paths worth learning from and thinking.

From brush division to full available

The pre -training large model is one of the driving directions and core development directions of the continuous change of AI. As AI continues to penetrate the industry and various disciplines, scientific research institutes and major enterprises have begun a large model arms backtle. Diversifying and parameter scale moving towards the extreme.

Among the hundreds of contention, we see that the scale of model parameters is getting larger and larger, and data set records are constantly refreshed. However, in the real industrial space, it is difficult to see the large -scale application of large models. The model parameters and the results of the downstream tasks are the common operation of the manufacturer after launching the big model. However, when it was applied, many manufacturers' models were silent.

From high scores to high -energy, there is still no short journey from the industrial scenarios in the reality. Making large models from "brushing" to thousands of industries requires a comprehensive transformation.

In order to better promote the development of large models, Hua Hua launched the full process of artificial intelligence models to enable the system. This system includes the process of planning from large model planning, development to industrialization, and accelerating the industrialization process of large models.

After the industry has launched Pengcheng, Pangu, Pengcheng, Shennong, Zidong, Taichu, Wuhan, Huawei Yunpan Ancient series, etc. , Huawei launched Shengteng Scientific Research Innovation Enable Plan. Through the support of funds, computing power, technology and communities, it encouraged universities and scientific research institutes to carry out research and innovation of large models based The industry has created a world -class leader.

In order to make large models easy to develop, adaptable, and deployed, and for basic model development, Huawei has launched a large model development kit based on Shengsi MindSpore. Through algorithm development, parallel computing, storage optimization, breakpoint training and other technologies, Model efficient development and deployment.

From scientific research innovation to the implementation of the industry, Huawei has established the intelligent remote sensing open source ecological alliance and the multi -mode artificial intelligence industry alliance with industry partners. At present, more than 70 partners have successively hatched multiple industry solutions. The joint partners established industrial alliances such as AI fluid mechanics, AI biomedicine and smart biological breeding to help innovation and industrial development in related fields.

The full process of the big model enables the system not only to bring growth soil to the research and development and innovation of large models, but also promotes ecological partners to incubate more industry applications based on existing large models. At the same time, large models will also obtain richer industry data and The nurturing of more generalized application scenarios. In the process of a virtuous circle, the big model has grown more vigorously, and can really empower the industry.

From the macro -enable system, we can perceive the strength and value of large models to empower the strength and value of thousands of industries; among micro individuals, through the representative Zidong. Change.

Zidong. The root of the development of Taichu

At this stage, the large models of the industry -university -research community are mainly concentrated in the NLP and CV fields. The traditional single -mode or dual -modal pre -training models based on text and images in the industry. The coverage of coverage and meetings is limited. It cannot give full play to the productivity and limits the application innovation of AI in the next stage. Multi -mode and large models came into being. The coordinated transformation of different modular data such as images, texts, and voice, and then made AI applications more in line with human behavior habits and actual needs, becoming one of the current artificial intelligence industries.

Zidong. Taichu is the world's first three -model 100 billion parameter model. As a representative of the multi -mode model, it is fully promoting the changes in the R & D rules and industrial application models to accelerate the intelligent transformation practice of various industries. At the first China computing power conference from July 29th to 31st, the "Zidong. Taichu" big model won the "DC Tech Innovation Pioneer" Outstanding Achievement Award.

The dimension of the innovation pioneer outstanding achievement award is rigorous and comprehensive. Whether it is technology, system, or application empowerment, it is a key consideration. Zidong. Taichu's big model is recognized by the industry. It has become a benchmark leading multi -mode and large model. It can maintain excellent and continuous innovation. It comes from its strong AI root technology. "Innovation" needs. Zidong. Taichu is based on the automation of the Chinese Academy of Sciences, so the basic software and hardware of the Shengteng AI. Based on the three -mode model created by the AI ​​framework MindSpore, the Zidong. Taichu. Compared with the two modes of graphic, it uses a large model to flexibly support the map-text-music full scene AI application, which has a strong ability to combine multi-tasking joint learning and quickly migrate data in different fields.

Zidong. Taichu has the ability to cross -modular understanding and generating ability of graphic sound and texture, which can easily complete the tasks of intelligent Q & A, picture generation, video understanding, and other tasks. Driving and other fields are widely used. For example, in the application case in the textile industry production line, Zidong. Taichu fusion multi -modal information can be judged by sound recognition to determine the interruption of the scriptures and broken latitudes of the textile machine. The ability of comprehensive research and broad application prospects.

Because the three -mode large model is very close to human information processing methods, and its ability to cooperate with information and data is very good, it can be widely used in various fields of industry and academia to incubate more new applications. Xinhua News Agency Technology Bureau, Changan Automobile, China Mobile, Qianbo Hand -language and other companies through joining the multi -modal artificial intelligence industry alliance, integrating open source multimodal models with their own business, based on Zidong. Taichu has successively hatched new media The application of content retrieval platform, smart cockpit, the Southern Song Royal Street digital person, and hand language teaching and examiners, etc., fully demonstrate the potential and industrial value of large models.

Digging from the deep model technology, we will find the creation of Zidong. Taichu, thanks to the industrial base of Shengteng AI, especially the native support of Shengsi for large models, so that the big model has fast development and minimalist training. The "root of development".

Watering and innovation flower

Drawing the "innovation" nutritional pouring model from the Shengsi AI framework is the key to enabling its development. Shengsi MindSpore considers the memory occupation, communication bottlenecks, complicated debugging, and difficult deployment when the development of large model development, and conducts targeted technical research and innovation.

In terms of big model support, Shengsi has realized native support for large models, which can take the lead in supporting fully automatic parallel computing in the industry. In large model training, you can use data parallel, operator -level model parallel, Pipeline model parallel, optimizer model parallel, heterogeneous parallel, reperfit, high -efficiency memory repetition multi -dimensional, full type of distributed parallel strategies; original The multi -dimensional automatic mixing of cluster topology perception to realize the automatic cutting and parallel computing of the oversized model, and significantly enhance the acceleration of the cluster; the new DNN distributed parallel programming paradigm can achieve low -code algorithm switching and greatly save development time.

In the field of scientific research innovation and application, Shengsi has launched the MindSpore Science series kit for 8 major scientific computing scenarios. It includes the industry's leading data set, basic model, preset high -precision model, and front and rear processing tools to accelerate the development of scientific industry.

For the openness of the industrial ecology, Shengsi is promoting open source and opening up with all walks of life in industry, university, and research. Shengsi MindSpore AI framework has become a technical support for the development of large models. Open source and opening up make the industry and academia develop its own large model based on it. The Shengteng Community and the MindSpore community have been strengthening support for the open source of large models. As of July, the download volume of Shengsi community has exceeded 2 million, and the community contributors have exceeded 5,900.

At present, Huawei Joint scientific research institutions and industry circles, based on the MindSpore AI framework and the strong computing power of Shengteng AI, continuously develop the industrial ecology of large models and industry models, and empower the digitalization and intelligence of thousands of industries.

For example, Based on Shengsi MindSpore, Pengcheng Lab has launched the industry's first 200 billion parameter Chinese pre -training language model Pengcheng. ; Wuhan University has created the world's first remote sensing image intelligent interpretation framework Wuhan .luojianet and the industry's largest remote sensing sample library Wuhan .luojiaSet after embedding the advanced technical characteristics of Shengsi MindSpore, which provides convenience for remote sensing application development.

From root technology innovation and improvement of the performance of large models to the accelerated development of the application of different scientific computing industries, the full process of the large model enables the construction of the system, the open source and openness of the industrial ecology and the bridge connection, based on the Shengteng AI software and hardware collaboration with the coordination of the soft and hardware of Shengteng AI Technological innovation and industrial services help, the road of innovation and industrial landing of large models is becoming more and more spacious, accelerating the practice of intelligent transformation in various industries, and more original technological achievements in different fields will be born in the future.

- END -