Creating Latent Representations of Synthesizer Patches using Variational Autoencoders

Abstract

Digital synthesizers typically feature a user-adjustable parameter space (i.e. the set of user-adjustable parameters) that is used to shape the sound (or timbre) of the instrument. A synthesizer patch is a snapshot of the state of the instrument's parameter space at a given time and is the representation most familiar to synthesizer users. Creating patches can often be repetitive, tedious, and complicated for synthesizers with large parameter spaces. This paper presents the creation and use of latent representations of synthesizer patches generated by training a Variational Autoencoder (VAE) on a library of existing patches. We demonstrate how to generate previously unseen patches by exploring this latent representation via interpolation through the latent space. Using the open-source synthesizer amSynth as a test bed, we evaluate reconstructed patches against a ground truth both, numerically and timbrally, as well as show how generating new patches from the latent space result in diverse yet musically pleasing timbres.