Imagine Alexa or Siri were able to control your robot. What do we need to make this possible? We recently published a paper at NeurIPS 2020: “Language-Conditioned Imitation Learning for Robot Manipulation Tasks” that answers this question. In this medium article, we provide a step-by-step guide that allows you to talk to your robot.

The full source code, dataset, and a pre-trained model are available at our GitHub repository https://github.com/ir-lab/LanguagePolicies.

Introduction

How to teach new skills to robots in an efficient way is an ongoing problem in robotics. Especially considering the ever-increasing availability of robots to the general public poses the…