Sample Efficient Deep Learning Methods for Reinforcement Learning and Bioimaging

Deep learning (DL) has emerged as a pivotal technology, with applications spanning across various domains. Among the sub-fields of deep learning, supervised learning has paved the way for advancements by leveraging labeled data. Another key area, reinforcement learning (RL), learns optimal behaviors through interaction with an environment. Each area presents unique applications, successes, and challenges that demand extensive datasets.
Sample efficiency emerges as a crucial concern, especially where large datasets or a high number of samples are impractical or expensive to obtain. The works presented in this cumulative thesis aim at improving sample efficiency in two areas, namely RL and bioimaging. In RL, learning algorithms often require a massive quantity of samples to learn effective policies. However, acquiring such a large quantity of samples can be particularly challenging in complex environments or real-world settings. This difficulty is compounded by the credit assignment problem, that is tracing the impact of actions to outcomes.
In the field of bioimaging, supervised learning models depend on extensive and often costly datasets, thus also grappling with the issue of sample efficiency. Each sample’s high cost necessitates maximizing information extraction from each instance. Advanced neural networks capable of processing high-resolution images have thus become essential.
With Align-RUDDER (Patil* et al., 2022) we proposed a novel method to dramatically improve sample efficiency in reinforcement learning. Based on RUDDER (Arjona-Medina* et al., 2019), Align-RUDDER tackles the credit assignment problem by using sequence alignment, a method commonly used in bioinformatics for identifying common sub-sequences within a set of long sequences. In Widrich* et al. (2021a,b,c) we proposed another approach when sequence alignment is not feasible by replacing the LSTM model in RUDDER with modern Hopfield networks (Ramsauer et al., 2021).
For bioimaging, we proposed a method to efficiently learn from the limited amount of data available in Hofmarcher et al. (2019). In addition to a novel neural network architecture we use multi-task learning. By learning from a large number of tasks the model can share representations between tasks, thereby increasing sample efficiency. Furthermore, we compiled and published a large bioimaging dataset with bioactivity labels.