Learning and generalizing stress patterns with a sequence-to-sequence neural network

Abstract

Abstract We present the first application of modern neural networks to the well-studied task of learning word stress systems. We tested our adaptation of a sequence-to-sequence network on the Tesar and Smolensky (2000. Learnability in optimality theory . Cambridge, MA: MIT Press) test set of 124 “languages”, showing that it acquires generalizable representations of stress patterns in a very high proportion of runs. We also show that the neural network can learn lexically specified patterns of stress, something constraint-based approaches to stress acquisition require extra mechanisms to accomplish. And finally we demonstrate that the model, in an agent-based simulation, is biased toward systematic patterns of stress, despite having the expressive power to memorize its training data.