HRI Design Tools

Overview

Our explorations of the design space for robotic technologies have demonstrated that the interactions afforded by these technologies—such as the robotic arm handing objects over at precisely the right time—can be very powerful but highly complex. Creating such interactions will require designers to perform design tasks that are substantially more complex and will demand methods and tools that empower them in tackling such complexity. In a second thread of research, my group has drawn on formal methods, specifically program analysis and synthesis, to reimagine the design process for robotic technologies as a mixed-initiative interaction where prototyping, simulation, and implementation tasks are performed by the computer and designers can focus on the creative and collaborative aspects of the design process.

In this thread of research, I build a suite of tools we have been building to support the unique and complex task of designing robotics applications for human use in collaboration with researchers in programming languages. My collaborators in this line of work include Aws Albarghouthi, Allison Sauppé, Maya Cakmak, and Rastislav Bodik. Students who are participating in this work include David Porfirio (now at NRL), Laura Stegner, and Andrew Schoen.

Tools

Some of the tools we have built in this thread of research include:

• RoVer — a visual authoring environment that verifies robot programs for social norms and task completeness — Paper, Talk — UIST 2018 Best Paper Award

• Synthé — an authoring system that enables designers to demonstrate social interactions through role play — Paper, Talk

• Figaro — a tangible authoring system to situate human-robot interactions in projected plans of environments — Paper, Talk

• Authr — a visual task authoring environment to translate human work plans into human-robot team workplans — Paper, Short Talk, Full Talk

• Tabula — a multimodal task authoring environment that integrates speech and gesture input and program repair to construct programs from partial input — Paper, Talk (listed under Supplemental Material)

• CoFrame — an environment for programming collaborative robots and for training in robot programming, built on a model of expertise in robotics — Paper, Talk

• Lively — a programming environment to generate configurable, real-time, task-based and communicative or socially-expressive motion for collaborative and social robotics — Paper, Talk (download), Short Video (download) — HRI 2023 Best Systems Paper Award

Videos

This project is supported by NSF awards 1651129 and 1925043.