Improving Autonomy and Natural Interaction of Pepper Robot via Large Language Models

Abstract

<title>Abstract</title> The field of Social Robotics is concerned with the development and enhancement of robots as interactive social companions and tools, aimed at aiding humans in a variety of tasks. Despite the ongoing progress, a primary issue encountered is the discrepancy between human instructions and the robot's interpretation and execution of these directives. Often, this is attributed to the deterministic nature of pre-defined programming, resulting in poor performance during tasks that deviate from this programming. This research contributes to this problem and proposes a solution to this predicament by enhancing the autonomous function and interaction of a Pepper robot through the assessment of a Large Language Model (LLMs). By leveraging LLM capabilities, the objective is to create a system allowing the robot to autonomously interpret instructions given in natural language to perform general-purpose tasks. The study involves the comparison of different LLMs proficiency in generating code commands for robotics. The assessment of the quality and efficiency of the produced code will be grounded upon the results of code execution, leveraging diverse strategies and code abstraction tiers. The evaluation methodology combines automated tests along with human evaluations. Our principal contribution encompasses the development of a task-processing system that links natural language instructions to robotic operations. Furthermore, our analysis revealed that precisely 400 out of 720 algorithmically generated tasks successfully passed the automated runtime execution evaluation. Among the LLMs scrutinized, GPT-4 registered the highest success rate in task completion (50.8%).