Using Exploratory Search to Learn Representations for Human Preferences

Nathaniel Dennler Stefanos Nikolaidis Maja Matarić

University of Southern California
HRI 2024

Abstract

Robots that interact with humans must adapt to the different preferences of human users. However, the time and effort needed for non-expert users to specify their preferences for a robot are a barrier to effective robot adaptation. Better representations of user preferences in the form of learned features have the potential to facilitate robot adaptation. In this work, we propose a method to learn representations using Contrastive Learning from Exploratory Actions (CLEA) that leverages data automatically collected from an interactive signal design process to better learn user preferences. We show that using data collected automatically from the design process can aid with learning user preferences compared to purely self-supervised learning.

Robots are expected to operate in unique and diverse contexts. Each context has a set of preferences and expectations. We want to learn representations for robot behaviors that facilitate user-driven customization of the robot in situ.

A user uses an ipad to design signals for a robot

We ground our approach in a signal design task, where 25 participants designed signals for a robot that assists them during an item finding task. Participants designed four signals: idle, searching, has item, and has information. Each signal consisted of three unique modalities: video, audio, and movement.

We leverage the insight that users make exploratory actions when designing robot behaviors. In particular, users look at potentially good options in detail, and disregard options that are not appropriate.

We construct a set of the data that the user explores, and the set of the data that the user ignores. We use these sets to sample contrastive triplets. Two signal components (video, audio, or movement) are draw from the same set and used as the anchor and positive examples. One signal component is sampled from the other set and used as the negative example. We use a contrastive objective to learn representations.

Video Presentation

BibTeX


        @inproceedings{dennler2024exploratory,
          title={Using Exploratory Search to Learn Represerations for Human Preferences},
          author={Dennler, Nathaniel and Nikolaidis, Stefanos and Matari{\'c}, Maja},
          booktitle={Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction},
          year={2024}
        }