The above video shows the interface users used to design signals for a robot. The robot helps users with an item finding task, and users showed diverse preferences and expectations for how the robot should behave. While using this interface, users subtly expressed their preferences for the robot's behavior by exploring different signal options before finally settling on their preferred signal. In this work, we harness this subtle source of information to learn user-aligned representations of robot behaviors. By using features that are more aligned with users, these users can more easily customize the robot to their preferences. Because it is easier to design signals, users are then able to provide even more information on their preferences. The cycle results in continously improving both the robot customization process and the features that facilitate robot learning.
A good feature representation has four criteria: completeness, simplicity, minimality, and explainability. Completeness refers to the ability of the representation to capture all the information needed to predict the user's preferences. Simplicity refers to the ability of the representation to reflect user's preferences with a simple linear model. Minimality refers to the ability of the representation to be concise and not contain redundant information. Explainability refers to the ability of the representation to be readily used with existing explainability techniques, for example "explanation by example".
A simple alternative to CLEA is to learn feature representations directly from data using self-supervised approaches like autoencoders or variational autoencoders. Other options are to use pre-trained models such as X-CLIP or Audio Spectrogram Transformers. We found that using the data directly from users allows us to learn representations that are better than these baselines on all four criteria, highlighting the importance of collecting diverse user data.
To evaluate CLEA, we collected a dataset of user rankings for robot signals. We then trained a model to predict these rankings using the learned representations. We show the results for the four criteria below.
Completeness. We show that CLEA representations are more complete than self-supervised representations in the graph above. We evaluated this using the Test Preference Accuracy (TPA), the ability of a model to predict user choices on a held-out test set. We show that CLEA representations are more complete than self-supervised representations because they achieve a higher TPA.
Simplicity and Minimality. We show that CLEA representations are simpler and more minimal than self-supervised representations in the graph above. We evaluated this using the Area Under the Alignment Curve (AUC Alignment). AUC Alignment measures the ability of a model to predict user choices on a held-out test set using a simple linear model. We learn the linear model by using a bayesian update rule to update the model's weights. We show that CLEA representations are simpler than self-supervised representations because they achieve a higher AUC Alignment across all dimesions and modalities. They are more minimal than self-supervised approaches because they achieve a higher AUC Alignment with fewer dimensions.
