[IJCAI2022 Tutorial] Can be optimized by micro -division: integrated structural information into the training process

Source: Specialty

not

This article is about 1,000 words, it is recommended to read for 5 minutes

This tutorial starts with the foundation of micro -optimization, and discusses how to convert optimizations into micro -constructor to build blocks in order to use in larger architectures.

Structural information and domain knowledge are two necessary components of training a good machine learning model to maximize the performance in target applications. This tutorial summarizes how to use optimization as a distinguished construction block to merge important operating information in the application into a machine learning model.

Machine learning models have achieved significant success in many industrial applications and social challenges, including natural language processing, computer vision, time sequence analysis and recommendation system. In order to adapt to different applications, incorporating the structure information and field knowledge in the application into a machine learning model is an important element in the training process. However, it often depends on fine -tuning and characteristic projects, without systematic methods to adapt to various applications. On the other hand, operational science is a method of application -driven. The optimization problem is based on the knowledge and restraint of target applications to export operating solutions. Optimization formulas can capture structure information and field knowledge in the application, but the unable to microlysis and complex operation process of the optimization process makes it difficult to integrate into a machine learning model.

This tutorial starts with the foundation of micro -optimization, and discusses how to convert optimizations into micro -constructor to build blocks in order to use in larger architectures. The direct benefit of micro -optimization is to integrate the structural information and field knowledge in the optimization formula into a machine learning model. The first part of this tutorial covers various applications, which is optimized as a micro unit in the machine learning model to properly handle the operational tasks in strengthening learning, control, optimal transportation and geometry. Experiments show that micro -optimization methods can effectively simulate the operation process than neural networks. The second part of this tutorial focuses on integrated various industrial and social challenges as a distinguished optimization layer into the training pipeline. The integration of this machine learning model and application -driven optimization leads to end -to -end learning. Learning based on decision -making, the training model directly optimizes the performance in the target application. Finally, this tutorial summarizes a series of applications and its computing restrictions that can be micro -optimized, and provides readers with various open directions.

https://guaguakai.github.io/ijcai22-differentiable-optimization/

- END -