RoboCup@Home2024 DSPL Application

TRAIL (Tokyo Robotics and AI Lab) is a subgroup of Matsuo Laboratory (Matsuo Lab) at The University of Tokyo. The team was launched in 2020 as a project and was officially organized as a team in 2021. While Matsuo Lab mainly engages in fundamental research on deep learning, especially world models, TRAIL aims to realize accumulated software skills on hardware, namely general-purpose robots, by utilizing robot learning.

Matsuo Lab also provides educational programs to the public (including junior high/high school and university students, and adults). A part of first and second year students at The University of Tokyo have gathered and developed their robot operating skills through these educational programs, and play significant roles in this project, RoboCup@Home 2024 participation.

Approach

To manage helpful tasks in various domestic environments, service robots should be able to execute tasks that cover a wide range of situations and handle possible errors quickly. RoboCup@Home, whose goal is the application of robots to domestic environments, offers tasks that test robots’ abilities to deal with these difficulties.

In the current situation, DNN(Deep Neural Network)-based models, each of which is aimed at a specific task, are often used to get over these difficulties. However, such models need collecting and labeling data, demanding much human and time resources. Therefore, this approach doesn’t seem to fit robot operations in domestic environments.

We suggest a new system the center of which is a model developed by generalizing bases such as Foundation Models through simulators. This system enables robots to deal with unexpected problems only with brief tuning.

We built a robot system that integrates several foundation models (CLIP, Detic, GPT, Sentence BERT, and Whisper). Our robot system won the competition with a perfect score in each category, verifying the system’s effectiveness.

Accomplishments

• RoboCup@Home 2023 DSPL League
  • 3rd place
  • You can see our poster here.
• RoboCup@Home Japan Open 2023 DSPL League
  • 1st place
  • 3rd place at Technical Challenge
    • You can see our presentation video and slides here.
• RoboCup@Home Japan Open 2020 DSPL
  • 1st place at Techical Challenge
  • 2nd place
    • Video(x20)
• World Robot Summit (WRS) 2020 Partner Robot Challenge ¹
  • 2nd place
  • WRS2020 Technical Challenge Paper
  • The system paper is accepted in Advanced Robotics (2022) ²

Current Technology

Contribution

Establishment of a Platform to Collect and Share Data Regarding Service Robots

We propose to establish a platform that integrates and shares data collected all over the world, by teams belonging to the RoboCup@Home community, utilizing our experiences as the AI lab. There is no large-scale data set in the robotics field so far, in contrast to other fields such as image recognition and natural language processing. However, we expect that platform will facilitate and accelerate the development of service robots, by sharing the labor of collecting and labeling data, which sometimes can be troublesome. We also consider the characteristics of the RoboCup@Home community desirable for realizing this platform, for the same robot (HSR) and similar environments (RoboCup@Home) are located all over the world.

Provide data to the Open X-Embodiment Dataset

We at Matsuo Lab’s TRAIL contributed data to the Open X-Embodiment Dataset. RT-X is the largest open-source real robot dataset to date. Until now, Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. The Open X-Embodiment Dataset is an application of these to robotics. It contains 1M+ real robot trajectories spanning 22 robot embodiments, from single robot arms to bi-manual robots and quadrupeds.

https://robotics-transformer-x.github.io/

Educational Programs

Matsuo Lab and TRAIL provide a variety of educational programs to the public.

For example, to provide opportunities to develop real robots for first- and second-year students, who don’t belong to laboratories, TRAIL has offered them an extracurricular course ‘Robot System Tutorial’

Education is important from any perspective, but especially in the field of robotics, it is of vital importance. We believe our programs enable students to smoothly learn a wide range of knowledge needed to develop robots, which is one of the difficulties students who major in robotics confront. Also, by offering such opportunities, we can inherit the know-how we have gained and make our research results used in wider fields, contributing to the long-term development of service robots.

Here are some of the courses we offer:

• Data Science Basic Course
• Deep Learning Basic Course
• World Models Seminar
• Robot System Tutorial

Publications

You can see out publications here.

Team members

You can see our team members here.