Generative AI and Robotics: From Large Language Models to Intelligent Human-Robot Interaction and Task Planning

Abstract

The rapid advancement of ChatGPT and other large language models (LLMs) has generated substantial interest in their application within robotics. This review explores the integration of generative AI into robotics, focusing on how LLMs like ChatGPT are enhancing robot intelligence, human-robot interaction, and task planning. LLMs enable improved natural language processing, facilitating more effective communication between humans and robots, while also contributing to robot perception, decision-making, and control across various modalities such as visual, auditory, and tactile inputs. The review examines the potential of LLMs to support seven types of robot intelligence, while addressing the exclusion of intrapersonal intelligence due to ethical concerns surrounding self-awareness and bias. Novel approaches for incorporating ChatGPT into task planning are discussed, particularly the use of state awareness to improve robot autonomy and adaptability. The role of reinforcement learning in optimizing LLM-based robotic systems is also considered, particularly in enhancing decision-making and generating realistic training data. Finally, the review outlines the challenges and future research directions for integrating generative AI into robotics, focusing on overcoming limitations such as outdated knowledge, multi-party interactions, and motion control.