Automated Object Manipulation Using Vision-Based Mobile Robotic System for Construction Applications

Abstract

In the last decade, automated object manipulation for construction applications has received much attention. However, the majority of existing systems are situated in a fixed location. They are mostly static systems surrounded by necessary tools to manipulate objects within their workspace. Mobility is an essential and key challenge for different construction applications, such as material handling and site cleaning. To fill this gap, this paper presents a mobile robotic system capable of vision-based object manipulation for construction applications. This system integrates scene understanding and autonomous navigation with object grasping. To achieve this, two stereo cameras and a robotic arm are mounted on a mobile platform. This integrated system uses a global-to-local control planning strategy to reach the objects of interest (in this study, bricks, wood sticks, and pipes). Then, the scene perception, together with grasp and control planning, enables the system to detect the objects of interest, pick, and place them in a predetermined location depending on the application. The system is implemented and validated in a construction-like environment for pick-and-place activities. The results demonstrate the effectiveness of this fully autonomous system using solely onboard sensing for real-time applications with end-effector positioning accuracy of less than a centimeter.