Target Search and Navigation in Heterogeneous Robot Systems with Deep Reinforcement Learning

Collaborative heterogeneous robot systems can greatly enhance the efficiency of target search and navigation tasks. In this paper, we design a heterogeneous robot system consisting of an unmanned aerial vehicle (UAV) and an unmanned ground vehicle (UGV) for search and rescue missions in unknown environments. The system is able to search for targets and navigate to them in a maze-like mine environment with the policies learned through deep reinforcement learning algorithms. During the training process, if two robots are trained simultaneously, the rewards related to their collaboration may not be properly obtained. Hence, we introduce a multi-stage reinforcement learning framework and a curiosity module to encourage agents to explore unvisited environments. Experiments in simulation environments show that our framework can train the heterogeneous robot system to achieve the search and navigation with unknown target locations while existing baselines may not. The UGV achieves a success rate of 89.1% in the mission within the original environment, and maintains a 67.6% success rate in untrained complex environments.