Regularized algorithms for dictionary learning

Abstract

Dictionary learning (DL) for sparse representation is a difficult optimization problem for which several successful algorithms exist, although none can be claimed the best. A common problem is a possible stall in the evolution of the algorithm, due to nearly linearly dependent atoms. The proposed cure was to regularize the error criterion using either the norm of the representations or an atom coherence measure. However, only gradient-based algorithms have been proposed for the regularized problems. We give here regularized versions of Approximate K-SVD and other algorithms related to it and investigate numerically their behavior. The experiments show that the new regularized algorithms are able to reduce the representation error, and thus produce better dictionaries, when the imposed sparsity is not very high.