Future collaborative robots must be capable of finding objects. As such a fundamental skill, we expect object search to eventually become an off-the-shelf capability for any robot, similar to e.g., object detection, SLAM, and motion planning. However, existing approaches either make unrealistic compromises (e.g., reduce the problem from 3D to 2D), resort to ad-hoc, greedy search strategies, or attempt to learn end-to-end policies in simulation that are yet to generalize across real robots and environments. This thesis argues that through using Partially Observable Markov Decision Processes (POMDPs) to model object search while exploiting structures in the human world (e.g., octrees, correlations) and in human-robot interaction (e.g., spatial language), a practical and effective system for generalized object search can be achieved. In support of this argument, I develop methods and systems for (multi-)object search in 3D environments under uncertainty due to limited field of view, occlusion, noisy, unreliable detectors, spatial correlations between objects, and possibly ambiguous spatial language (e.g., "The red car is behind Chase Bank"). Besides evaluation in simulators such as PyGame, AirSim, and AI2-THOR, I design and implement a robot-independent, environment-agnostic system for generalized object search in 3D and deploy it on the Boston Dynamics Spot robot, the Kinova MOVO robot, and the Universal Robots UR5e robotic arm, to perform object search in different environments. The system enables, for example, a Spot robot to find a toy cat hidden underneath a couch in a kitchen area in under one minute. This thesis also broadly surveys the object search literature, proposing taxonomies in object search problem settings, methods and systems.

翻译：未来协作机器人必须能够自主寻找物体。作为一项基础技能，我们期待物体搜索最终能成为任何机器人的即用型能力，如同物体检测、SLAM和运动规划等现有技术。然而，现有方法要么做出不切实际的假设（如将三维问题简化为二维），要么采用临时性的贪婪搜索策略，或试图在仿真中学习端到端策略却难以推广至真实机器人与环境。本文论证：通过采用部分可观测马尔可夫决策过程（POMDPs）对物体搜索进行建模，同时利用人类世界结构（如八叉树、相关性）和人机交互特性（如空间语言），可实现通用物体搜索的实用高效系统。为支持这一论点，本文开发了三维环境下应对多种不确定性（包括视场受限、遮挡、噪声干扰、不可靠检测器、物体间空间关联及模糊空间语言如"红车在Chase银行后面"）的（多）物体搜索方法与系统。除在PyGame、AirSim和AI2-THOR等仿真器中进行评估外，本文设计与实现了与机器人无关、与环境无关的三维通用物体搜索系统，并将其部署于Boston Dynamics Spot机器人、Kinova MOVO机器人和Universal Robots UR5e机械臂上，在不同环境执行物体搜索任务。该系统可使Spot机器人在厨房区域一分钟内找到隐藏沙发下的玩具猫。本文同时广泛综述了物体搜索文献，提出物体搜索问题设置、方法与系统的分类体系。