The Mutual Information Function as a Preprocessing Tool for Predictor Design

In this work we present the application of the mutual information function to the problem of predictor design. From our point of view optimum predictor design depends not just on proper system identification, optimal fitting of system parameters or appropriate methods to learn the system behavior. We argue that it is most important to find the optimal input parameters independent of the system identification or learning task. We apply the mutual information function as a preprocessing tool to select, in a least square sense, the best input parameters apriori. This means that we use the data resulting from the data acquisition process to find the combination of input parameter that is most ''informative'' on the output to be predicted. As we will explain, this task is done using the mutual information function, which serves as base for the upper bound on the prediction gain. As a demonsration example, we use load prediction for power distribution networks.