Adaptive Robot-Assisted Autism Therapy Using a Robot with Hand and Face Tracking

Laura Boccanfuso, Jason M. O'Kane
In Proc. International Conference on Social Robotics 2010.


This research explores interactive games using hand and face tracking with a robot as a tool for autism therapy. The robot is equipped with a head and two arms, each with two degrees of freedom, and a camera. We trained a classifier to detect human hands and subsequently, used this classifier along with a standard face tracker to create two interactive games. In the first game the robot waits for the child to initiate an interaction by raising one or both hands. In the second game, the robot initiates interactions. These games are designed to increase attention, promote turn-taking skills and encourage child-led verbal and non-verbal communication through simple imitative play. This research makes two specific contributions: (1) We present a low-cost robot design which measures and adapts to a child's actions during interactive games and, (2) we train and test a hand detector, based on Haar-like features, which is usable in various kinds of human-robot interactions.


  author       = {Laura Boccanfuso and Jason M. O'Kane},
  title        = {Adaptive Robot-Assisted Autism Therapy Using a Robot with
		 Hand and Face Tracking},
  booktitle    = {Proc. International Conference on Social Robotics},
  year	       = {2010}

O'Kane's home page
O'Kane's publication list

Wed Mar 4 12:52:19 EST 2020