Towards Deep and Discriminative Canonical Correlation Analysis

We introduce a discriminative extension of Deep Canonical Correlation Analysis (DCCA) for the purpose of multi-view representation learning. The objective of DCCA is to learn two groups of latent features which are highly correlated when projected into the common CCA-space. Repre- sentations learned with DCCA pre-training have proven to be beneficial when used in a subse- quent classification tasks. In this work we tackle exactly the problem of multi-view classification by incorporating a discriminative regularizer on the hidden representations already at train time. Inspired by a deep learning interpretation of Lin- ear Discriminant Analysis (DeepLDA) we de- sign a joint optimization target that encourages the network to learn representations which are not only correlated but also highly discrimina- tive. Preliminary results show that the joint opti- mization of correlation and separation is feasible and helps to enhance the classification power of the learned representations.