Voice and Gesture Control

With the increase in popularity of robotic systems and autonomous agents in our everyday lives, human-robot interaction is becoming an increasingly important area of research. Drones are just one of such robotic entities that have increased in popularity, both for consumer use cases and industry applications. Beyond traditional single modality controller interfaces (e.g. joystick or buttons on a screen), there is little advancement for interaction design to enable intuitive human-machine interaction and collaboration. Our work aims to explore the potential of multi-modal interaction for advancing the usability experience of human-robot interaction, specifically drones. We present a prototype system consisting of a head-mounted hand tracker and voice control on a laptop to support gesture and voice recognition which can be used for controlling a drone. We utilize off the shelve components and software APIs for the implementation. Our system can support both gestures and voice commands to enhance interaction design for human-robot collaboration. We conduct a user study to evaluate the usability of multi-modal interaction. Our findings from 15 participants show that people prefer a combination of gestures and voice commands when controlling a drone. We hope our findings and detailed description of our system implementation will help advance interaction design for human-robot collaborations with drones and also can be generalized beyond to other robotic agents.