High-performance vision-based tactile sensing with applications in robotic manipulation

Abstract

As robots evolve from executing hard programmed procedures to intelligently interacting with environments and human beings in both industry and domestic scenarios, equipping robots with the ability to perceive the environment has been an urge yet a challenge for developing more collaboratively friendly robots. Among multiple essential artificial sensations for modern robots including vision, sense of touch, auditory sense, artificial sense of touch provides robots with physical interaction signals in a direct, more accurate, and finer fashion plays a critical role in robotic tasks of exploration and manipulation. Vision-based tactile sensors thrive as a competitive branch of tactile sensors for their various superiorities. In this thesis, we illustrate a full spectrum of the development of vision-based tactile sensors including designs, fabrications, signal interpretation methodologies, and system integration, with an emphasis on applications to robotic manipulation and human-robot interaction. As the outcomes, we present both high-performance tactile sensors that are innovative in terms of designs and functionalities, and cutting-edge algorithms on the extraction of tactile information. Specifically, we present an ultra-thin and 3-dimensional force sensible vision-based tactile sensor that draws inspiration from compound eye structure that is existing in arthropod animals' eyes. Conceptual design and theoretical endorsement of the measurement ability are presented. Fabrication methods of essential components including the wafer-level apposition compound eye structure and dense trackable pattern embedded elastomer ( artificial skin layer) are introduced in detail. An optimization-based image stitching and blending framework are proposed for the multiple fields of view (FOVs) generated from the imaging system using the compound eye structure as its lenses. Moreover, to show the potentials of vision-based tactile sensors in human-robot interactions, we present a dense and full-body vision-based tactile sensing augmented robotic arm with design principle, formulation of the perspective projection inside the imaging system for contact information mapping. Afterward, we turn to the topic of tactile information interpretation. A Helmholtz-Hodge decomposition-based contact force estimation method is proposed to recover contact force and torque from a deformation vector field acquired from a vision-based tactile sensor. This method is applicable to a wide range of tactile array sensors. At a higher level in terms of information abstraction, we developed a contact event detection and prediction framework based on dense tactile inputs for dexterous and reactive manipulations. Extensive experiments and evaluations are conducted for each work in this thesis. we conclude this thesis together with a discussion, illustration of limitations, and proposition of future works.