Deep reinforcement learning and creativity

Abstract

Generative artificial intelligence (AI) is among the most exciting developments in computer science over the last decade. In several fields, it is not only complementing but also replacing the creative abilities that were once solely in humans’ hands. However, current generative models are limited by their learning schemes, which merely aim to imitate training data. To develop more creativity-oriented models, new approaches should be considered. Among them, reinforcement learning (RL) represents a promising direction. RL is an inherently learning-by-acting approach and can capture a greater variety of target behaviors, making it ideal for modeling how humans learn to behave creatively. Studying RL together with creativity can be of crucial importance for both fields. This thesis explores whether creativity can enhance the design of RL algorithms and, vice versa, whether RL can help develop more creative generative models. First, we study if dreaming can help RL agents better generalize, as suggested for humans. We leverage generative augmentations to transform predicted trajectories into dream-like experiences for training agents and evaluate generalization capabilities in different scenarios. Then, we develop a new creativity score that quantifies the originality and value of artifacts. We use it as a reward in an RL framework, and we propose to fine-tune pre-trained models toward more creative solutions. We validate our method in two different domains: poetry generation and problem solving. In addition, we present new sampling schemes to better simulate the human creative process by working at the response generation and validation levels. Finally, we conclude with a deep analysis of three main social and practical issues: whether current models are creative and their implications; whether they can be entitled to agency and what happens to human agency when collaborating with them; how copyright laws can manage the complexity of generative AI to protect human- and machine-generated artworks.