Robot-Wearable Conversation Hand-off for Navigation

Dániel Szabó1 ✉️, Aku Visuri1, Benjamin Tag2, Simo Hosio1
1 University of Oulu
2 The University of New South Wales
Research of Crowd Computing Oulu • More Projects
Teaser image showing the conversation hand-off concept
Figure 1: Usage of the conversation hand-off system presented as a storyboard.
TLDR; We explored if transferring a navigation assistant from a robot to a smartwatch helps. People found it fun and engaging ("teleporting assistant"), but actually preferred just using the watch because it was more efficient.

Abstract

Navigating large and complex indoor environments, such as universities, airports, and hospitals, can be cognitively demanding and requires attention and effort. While mobile applications provide convenient navigation support, they occupy the user's hands and visual attention, limiting natural interaction. In this paper, we explore conversation hand-off as a method for multi-device indoor navigation, where a Conversational Agent (CA) transitions seamlessly from a stationary social robot to a wearable device. We evaluated robot-only, wearable-only, and robot-to-wearable hand-off in a university campus setting using a within-subjects design with N=24 participants. We find that conversation hand-off is experienced as engaging, even though no performance benefits were observed, and most preferred using the wearable-only system. Our findings suggest that the design of such re-embodied assistants should maintain a shared voice and state across embodiments.

Paper (Coming Soon)

Introduction

Indoor navigation aids are crucial for complex environments. We propose a "Hybrid Indoor Navigation System" where a Conversational Agent (CA) can hand-off an ongoing navigation session from a stationary social robot (Pepper) to a wearable device (Apple Watch). This allows users to start their journey with a social interaction and continue with a personal device on the go.

Method

We conducted a within-subjects user study (N=24) in a university campus setting. We compared three conditions:

Photos of the experiment setup
The participant uses the robot (left) and the watch (middle) during the experiment.
Hand-off GUI
The interface for the conversation hand-off.

Results

Preference: 70.8% preferred Wearable-Only, 29.2% preferred Hand-off. No one preferred Robot-Only.

Workload (NASA-TLX): Robot-Only had significantly higher mental demand than the other conditions. Hand-off was seen as engaging and fun ("teleporting assistant").

Trust: There was no significant difference in trust between the conditions.

NASA TLX Results
NASA-TLX subscale results comparing the three conditions.

Conclusion

We found that while the wearable-only system was favored for its convenience, the conversation hand-off was seen as engaging. Flexible triggers for transition and rapid hand-offs are key design considerations for future systems.