Multi-modal human-robot interface for interaction with a remotely operating mobile service robot

Abstract

This paper discusses requirements for information exchange between a human operator and a semi-autonomous service robot operating in a remote indoor environment. The resulting multimodal HuMan-Robot Interface (MRI) allows specification of various types of robot activities by use of an advanced system for natural spoken user-independent speech understanding and flexible robot command generation. Visual screen-based monitoring and support of complex operations is achieved by means of an animated three-dimensional environmental model augmented by the image of an onboard CCD camera. Typical features of the MRI are demonstrated through experiments performed with the mobile manipulator ROMAN.