Optimization Networks for Integrated Machine Learning

Optimization networks are a new methodology for holisti- cally solving interrelated problems that have been developed with com- binatorial optimization problems in mind. In this contribution we revisit the core principles of optimization networks and demonstrate their suit- ability for solving machine learning problems. We use feature selection in combination with linear model creation as a benchmark application and compare the results of optimization networks to ordinary least squares with optional elastic net regularization. Based on this example we jus- tify the advantages of optimization networks by adapting the network to solve other machine learning problems. Finally, optimization analysis is presented, where optimal input values of a system have to be found to achieve desired output values. Optimization analysis can be divided into three subproblems: model creation to describe the system, model selec- tion to choose the most appropriate one and parameter optimization to obtain the input values. Therefore, optimization networks are an obvious choice for handling optimization analysis tasks.