Detecting object affordances with Convolutional Neural Networks

Abstract

We present a novel and real-time method to detect \nobject affordances from RGB-D images. Our method trains \na deep Convolutional Neural Network (CNN) to learn deep \nfeatures from the input data in an end-to-end manner. The CNN \nhas an encoder-decoder architecture in order to obtain smooth \nlabel predictions. The input data are represented as multiple \nmodalities to let the network learn the features more effectively. \nOur method sets a new benchmark on detecting object affordances, improving the accuracy by 20% in comparison with \nthe state-of-the-art methods that use hand-designed geometric \nfeatures. Furthermore, we apply our detection method on a \nfull-size humanoid robot (WALK-MAN) to demonstrate that \nthe robot is able to perform grasps after efficiently detecting \nthe object affordances.