Gaze-based interaction for effective tutoring with social robots

Abstract

The central thesis of this work is that effective gaze behavior can help build a shared understanding and mutual awareness between humans and robots, leading to positive outcomes in a tutoring interaction. Gaze behavior is an essential cue for social engagement and coordinated action, principally for tasks that imply human-robot collaboration, such as tutoring. The work presented in this dissertation is a compilation of findings from three empirical studies designed to explore the design space of gaze-based interaction to enrich human-robot interaction in educational settings where robots assume tutor or trainer roles. In the first study, we examined how people perceive and interpret social cues from gaze provided by either a human or a robot tutor during a collaborative tutoring activity. The objective was to investigate whether people can notice and accurately interpret gaze-based cues from a tutor and whether they can accept the cues as help during learning interactions. We incorporated eye-tracking to examine gaze interaction during human-human and human-robot communications. We found that participants noticed the gaze cues from the robot tutor significantly more often than those of the human tutor. Consequently, we found that participants performed better with the robot tutor compared to the human tutor. These initial findings provide design recommendations for gaze-based communications to improve learning performance during human-robot tutoring. Based on the results from the first study, we investigated how to implement gaze-based communication as an efficient help mechanism for robot-child tutoring. The objective was to examine child-robot gaze mechanisms to inform the robot's behavior design as a facilitator of children's task-solving. We carried out simultaneous observations of the child's gaze and the robot to examine the events of mutual gaze and gaze following patterns during the tutoring activity and to assess the impact of different child-robot coordinated gaze patterns on children's behavior and performance. We found that if a robot tutor provides gaze-based support, children perform better during the tutoring activity than when a tutor SUMMARY xv does not offer such cues. We also found that more events of mutual gazing patterns between the child and the robot tutor improve children's awareness of the tutor's intention during the activity leading to better performance. Therefore, we concluded that increasing gaze coordination between the child and the robot can improve performance and build mutual awareness during robot-based educative interventions. In the last study, we investigated the nature and dynamics of gaze-based human-robot interaction (HRI) in tutoring. The objective was to examine intricate patterns of gaze interchanges between a child and a robot during the tutoring activity and to assess the impact of child-robot coordinated gaze on children's behavior and performance. We combined both observational and sequential lag methods to examine the relevant gaze sequences during a collaborative tutoring activity. We found that appropriate sequences and timing of the dyad's gaze behaviors between a child and a robot can lead to effective interactions between a child and a robot tutor. Based on these findings, we concluded that a robot tutor could positively influence the flow of the child's actions if the child interprets the social cues appropriately, improving the task execution and the play experience. This new understanding of the dynamic nature of gaze behavior during child-robot interaction contributes to the design of robot gaze behavior, to build better robot-based interventions in education and therapy settings. Overall, the findings from the user studies contribute to new design guidelines for gaze-based communications to improve learning performances and promote positive human-robot tutoring interactions. In addition to the findings of the user studies, the main contributions of this dissertation include; First, an experimental framework for studying how gaze-based cues of robots can be applied to improve performance and quality of tutoring interaction. The experimental setting allows for simultaneous analyses of humans (adult-child) and the robot's gaze during a collaborative tutoring activity. Second, a coding scheme developed to measure the dynamics of child-robot interaction with an emphasis on coordinated and sequential gaze patterns between children and robots. The third is the use of simultaneous observational and lag-based methods to examine coordinated and interaction sequences of gaze between children and robots, helping unravel the dynamics of child-robot interaction in a tutoring setting. The lag-based methods provide an opportunity to investigate complex gaze sequences that—to our best knowledge—have not been previously explored in robot-based educational backgrounds or other human-robot interactions. The lag methods can be extended to analyze, in-depth, other interactive behaviors during human-robot interaction (HRI)