Application of FPGA to Machine Learning: Cluster Regression

In this thesis, a new approach for a Machine Learning algorithm named Cluster Regression is introduced, which is developed and designed to make an efficient hardware realization on an FPGA. The idea is based on Random Forest regression, which is a very successful machine learning technique, but here the splitting during the learning process is final and greedy. It could be an improvement if the elements chosen for model fitting are re-selectable in terms of to which segment they belong. This new algorithm uses the unsupervised technique K-Means to gain additional information for a supervised regression task. Additionally, within the presented algorithm each region is described by a linear plane instead of the mean. Several simulations with different datasets are done to ensure a good estimation perfor- mance of the digital hardware realization. The results of different versions of the algorithm are compared to selected Machine Learning regressors. As the simulations show, depending on the dataset, the Cluster Regression algorithm outperforms Random Forest regression, Multiple Linear regression, and Gaussian Kernel regression. The hardware design is fully synthesisable on an FPGA and can be adjusted to the needed precision and hyper-parameter set. As a result, the presented work is useful in selected regression tasks. Furthermore, due to the parallelized hardware design, the learning time can be reduced by a multiple depend- ing on the compared sequential computer. Additionally, this calculation time is linearly proportional to the learning data size due to several hardware optimization techniques. The prediction time of new outputs is short and also linearly proportional to the size of the testing data. Concerning future research, this work could help to connect Machine Learning with digital hardware design even more since there is a big potential for the improvement of efficiency and calculation speed.