Variational networks for joint image reconstruction and classification of tumor immune cell interactions in melanoma tissue sections

Immunotherapy is currently revolutionizing the treatment of cancer. Detailed analyses of tumor immune cell interaction in the tumor microenvironment will facilitate an accurate prediction of a patient’s clinical response. The automatic and reliable pre-screening of histological tissue sections for tumor infiltrating immune cells (TILs) will support the development of TIL-based predictive biomarkers for checkpoint immunotherapy. In this paper, a learning approach for image classification is presented, which allows various pattern inquires for different types of tissue section images. The underlying trainable reaction diffusion model combines classification and denoising. The model is trained using a stochastic generation of training data. The effectiveness of this approach is demonstrated for immunofluorescent and for Hematoxylin and Eosin (H&E) stained melanoma section images. A particular focus is on the classification of TILs in the proximity to melanoma cells in an experimental melanoma mouse model and in human melanoma. This new learning approach for images of melanoma tissue sections will refine the strategy for the practical clinical application of biomarker research.