Throughout history, we have successfully integrated various machines into our homes. Dishwashers, laundry machines, stand mixers, and robot vacuums are a few recent examples. However, these machines excel at performing only a single task effectively. The concept of a "generalist machine" in homes - a domestic assistant that can adapt and learn from our needs, all while remaining cost-effective - has long been a goal in robotics that has been steadily pursued for decades. In this work, we initiate a large-scale effort towards this goal by introducing Dobb-E, an affordable yet versatile general-purpose system for learning robotic manipulation within household settings. Dobb-E can learn a new task with only five minutes of a user showing it how to do it, thanks to a demonstration collection tool ("The Stick") we built out of cheap parts and iPhones. We use the Stick to collect 13 hours of data in 22 homes of New York City, and train Home Pretrained Representations (HPR). Then, in a novel home environment, with five minutes of demonstrations and fifteen minutes of adapting the HPR model, we show that Dobb-E can reliably solve the task on the Stretch, a mobile robot readily available on the market. Across roughly 30 days of experimentation in homes of New York City and surrounding areas, we test our system in 10 homes, with a total of 109 tasks in different environments, and finally achieve a success rate of 81%. Beyond success percentages, our experiments reveal a plethora of unique challenges absent or ignored in lab robotics. These range from effects of strong shadows, to variable demonstration quality by non-expert users. With the hope of accelerating research on home robots, and eventually seeing robot butlers in every home, we open-source Dobb-E software stack and models, our data, and our hardware designs at https://dobb-e.com

翻译：纵观历史，我们已成功将多种机器融入家庭。洗碗机、洗衣机、搅拌机和扫地机器人是近期的几个例子。然而，这些机器仅擅长高效执行单一任务。家用“通用型机器”——一种能够适应并学习我们需求、同时保持经济实惠的家居助手——一直是机器人学中持续追求数十年的目标。在本工作中，我们通过引入Dobb-E——一种经济实惠且多功能的通用系统，用于学习家庭环境中的机器人操作——启动了实现这一目标的大规模努力。借助我们使用廉价零部件和iPhone搭建的演示采集工具（“Stick”），Dobb-E仅需用户展示五分钟即可学习新任务。我们利用Stick在纽约市22户家庭中收集了13小时数据，并训练了家庭预训练表示（HPR）。随后，在全新家庭环境中，通过五分钟演示和十五分钟HPR模型适配，我们展示了Dobb-E能在市售移动机器人Stretch上可靠地完成任务。在纽约市及周边地区约30天的家庭实验中，我们在10户家庭中测试了系统，覆盖不同环境下的109项任务，最终成功率达到81%。除成功率外，我们的实验揭示了实验室机器人学中缺失或忽略的诸多独特挑战，包括强阴影的影响、非专业用户的演示质量差异等。为加速家用机器人研究，并最终实现家家户户拥有机器人管家，我们在https://dobb-e.com开源了Dobb-E软件栈与模型、数据及硬件设计。