Self-Recovery Prompting: Promptable General Purpose Service Robot System with Foundation Models and Self-Recovery

概要

A general-purpose service robot (GPSR), which can execute diverse tasks in various environments, requires a system with high generalizability and adaptability to tasks and environments. In this paper, we first developed a top-level GPSR system for worldwide competition (RoboCup@Home 2023) based on multiple foundation models. This system is both generalizable to variations and adaptive by prompting each model. Then, by analyzing the performance of the developed system, we found three types of failure in more realistic GPSR application settings: insufficient information, incorrect plan generation, and plan execution failure. We then propose the self-recovery prompting pipeline, which explores the necessary information and modifies its prompts to recover from failure. We experimentally confirm that the system with the self-recovery mechanism can accomplish tasks by resolving various failure cases. Supplementary videos are available at https://sites.google.com/view/srgpsr .

白坂 翠萌

学部生

松嶋 達也

特任研究員

人間と共生できるような適応的なロボットの開発と，そのようなロボットを作ることにより生命性や知能を構成的に理解することに興味があります．

綱島 颯志

学部生

池田 悠也

修士課程

工学部システム創成学科知能社会システムコース4年

保呂 蒼威

学部生

生駒 創

学部生

辻 知香葉

学部生

和田 輝

学部生

大見謝 恒和

学部生

小武海 大

学部生

松尾 豊

教授

岩澤 有祐

准教授