Deep Learning Methods for Credit Assignment in Reinforcement Learning and Immune Repertoire Classification

The ongoing advances in the field of machine learning (ML) in the past decades continue to have a strong impact on many aspects of our human civilization. This has already lead to the development of technologies that were deemed science fiction not too long ago. While colloquially referred to as artificial intelligence (AI), these new technologies are often relying on algorithms in the field of deep learning (DL). This cumulative thesis aims at developing novel methods to push the field of DL forward. In particular, it focuses on two challenging areas: (i) The delayed reward problem in reinforcement learning (RL). Solutions to this problem could be a crucial factor for the success of RL methods in many real-world settings. (ii) The task of immune repertoire classification as massive multiple instance learning problem in immunology. Successes in immune repertoire classification can improve our knowledge about the interaction of the immune system with diseases and allow for the development of novel medical therapies and applications. Both of these areas share a common trait in terms of methodology: an underlying immensely challenging credit assignment problem. That is, the parts of the input data that determine the outcome, such as the outcome of a game in RL or the immune status of a person, need to be identified and credit needs to be assigned to them accordingly. As a solution to the delayed reward problem, we proposed a novel method return decomposition for delayed rewards (RUDDER) in (Arjona-Medina* et al., 2019). RUDDER utilizes pattern recognition methods to assign credit directly to actions that cause delayed rewards, which drastically simplifies learning for subsequent RL methods. In (Widrich* et al., 2021a,b,c), we furthermore employed our novel continuous modern Hopfield networks (Ramsauer et al., 2020, 2021) for an improved realization of RUDDER. For immune repertoire classification, we proposed a novel method deep repertoire classification (DeepRC) in (Widrich et al., 2020). DeepRC is able to identify those very few immune receptor sequences that are responsible for the immune status of an individual in the huge immune repertoire of sequences while outperforming other existing and novel methods.