LUDI
About
Contact
Search
Register
Log in
Back
Socially Interactive Robot Sign Language Tutors
Summary
Play Experience by Assistive Technology for play
Devices, services & contexts for play with children with disabilities
Name of Play Experience by AT
Socially Interactive Robot Sign Language Tutors
Type of project
Intervention
Finished research project
Ongoing research project
Summary
(~150 words: target group, aims, kind of activities, Play Experience by AT (devices, services and contexts) used, play experiences and results)
This study presents a socially interactive humanoid robot-assisted system for sign language (SL) tutoring for children with communication impairments by means of imitation-based interaction games. In this study, a five-fingered robot platform Robovie R3 is used to express a set of chosen words in Turkish sign language (TSL) using hand and body movements combined with facial expressions. The robot is able to recognize signs through a RGB-D camera and give vocal, visual and motional (as signs) feedback. The proposed game consists of an introductory phase where participants are introduced to the robot and the signs; an imitation-based learning phase where participants are motivated to imitate the signs demonstrated by the robot, and a test phase where the signs taught in the previous phases are tested within a guessing game. The current research presents the results from the studies with three different test groups. The presented humanoid robot is used as an assistive social companion in the game context using sign language and visual clues to interact with the children. The robot is evaluated according to the participant’s sign recognition ability within different setups. The results indicate that the robot has a significant effect on the sign learning performance of participants.
Play Experience by Assistive technology
Low-tech, high-tech products, services and contexts for play
Description, intended use(s), intended target group(s), manufacturer/developer, reference
The humanoid robot used in the study is a specially modified version of the Robovie R3 robot. The standard R3 platform is 1.08 m tall and weighs 35 kg and has 17 DOF (2*arms*4, neck*3, 2*eyes*2, wheels*2) but the modified version of R3 robot has additional DOF in wrists, and fingers, 29 DOFs in total; and also a LED mouth to imitate and/or express some basic emotions (such as a happy, sad or neutral facial expressions). The robot has five-fingered hands with independent fingers and longer limbs providing the opportunity to implement signs more accurately. The robot has also a small platform on the chest with an ASUS RGB-D camera which is used for gesture recognition. This platform provides also some flexibility on the interaction level and with respect to game scenario e.g. the depth camera attached to it can be easily replaced by a touch pad tablet according to game requirements in future studies.
Picture
Download
Context
The context of use
(home, school, rehab center or other environments)
The presented platform can provide assistance to deaf or hard of hearing children in reinforcing their knowledge on Turkish Sign Language (TSL) by playing different games with the robot. Therefore, the robot may be used in different environments such as homes or classrooms.
Type of Play
Type of play in this play system
LUDI Classification of types of play
Cognitive Dimension
Practice
Symbolic
Constructive
Rule play (including videogames)
Social Dimension
Solitary
Parallel
Associative
Cooperative
Objectives
Objectives related to play according to ICF-CY
International Classification of Functioning, Disability and Health - Children & Youth Version (ICF-CY)
Play for the sake of play
Major life areas - d880 engagement in play
d8800 solitary play
d8801 onlooker play
d8802 parallel play
d8803 shared cooperative play
d8808 engagement in play, other specified
d8809 engagement in play, unspecified
Community social and civic life - d920 recreation and leisure time
d9200 play
Play-like activities
Therapeutic and educational objectives
b1 Mental functions
b2 Sensory functions and pain
b3 Voice and speech functions
b4 Functions of cardiovascular, hematological, immunological and respiratory systems
b5 Functions of digestive, metabolic and endocrine systems
b7 Neuromusculoskeletal and movement related functions
d1 Learning and applying knowledge (learning through symbolic play, learning through pretend play)
d2 General tasks and demands
d3 Communication
d4 Mobility
d7 Interpersonal interactions and relationships
Participant
Participant(s)
Number
1-5
5-10
10-20
>20
Chronological Age
0-3 years
3-6 years
6-12 years
12-18 years
Development Age
0-3 years
3-6 years
6-12 years
12-18 years
LUDI Categories of disabilities
Mental/intellectual impairments:
mild
moderate
severe
profound
Hearing impairments
Partially hearing impaired
Deaf
Visual impairments
Partially sighted
Blind
Communication disorders (speech and language disorders)
Physical impairments
Mild
Moderate
Severe
Autism Spectrum Disorders
Multiple disabilities
Explanation
Explanation on the use of low-tech, high-tech devices, services or contexts
Explanation
No instruction, self-discovery of the participant/subject
Verbal instruction, language and communication fitting to chronological age
Verbal instruction, language and communication is adapted
Visual and/or verbal instruction with AAC (Aumentative and Alternative Communication)
Visual instruction with written language
Visual instruction with pictures or drawings
Modeling by therapist/researcher
Hand over hand: therapist/researcher leads the actions of the participant
Prompting: therapist/researcher touches the participant as a key for further actions
Guided discovery: therapist/researcher coaches the participant so s/he discovers how to use the assistive technology
Modeling by peer
Visual instruction by peer
Verbal instruction by peer
Involvement
Adult: therapist/educator/researcher
Parent or significant others
Peer with disabilities
Peer without disabilities
Role
Non-participatory observer
Participatory observer
Providing instruction
After the instruction, providing supervision during play
Video
Download
Evaluation
Evaluation of objectives and outcome measures
Description of outcome measure(s)
Observation by professional/researcher providing the play experience
Observation by other professional/researcher
Video analysis
Feedback from client/parents/professionals
(validated and reliable) outcome measures like tests, self-reports of client/system, questionnaires
Information about availability of outcome measure: publisher, website, contact person
Participants’ teachers and TSL instructors were interviewed before and after the studies. The children participants have been handed a questionnaire with pictures to evaluate the designed system. A content analysis of the interviews was conducted and the study was evaluated in terms of the signs recognized by the children.
Achievements
Summary of achieved effects
The results of paper-based test and verbal surveys show that the participants do not have any difficulty in learning and recognizing the signs from the Robovie R3 robot. The participants reported that it was easy to understand R3’s signs, since it had five fingers and longer limbs its gestures were comprehensible. Participants also remarked that the Robovie R3 could produce smooth and natural-looking gestures. Overall, the results with the different test groups were encouraging in terms of the humanoid robot’s contribution to the sign language learning process of individuals with different profiles. The results emphasized that the presented interaction game could be played with children suffering from different degrees of hearing impairment and the humanoid robots might be presented as an effective teaching material in the context of improving the interaction ability of children.
References
References to the intervention or research project
Contact Person
Assoc.Prof.Dr. Hatice Kose, hatice.kose@itu.edu.tr
Website
http://humanoid.itu.edu.tr
Publication
Uluer, Pinar; Akalin, Neziha; Kose, Hatice; A New Robotic Platform for Sign Language Tutoring, International Journal of Social Robotics, 7(5), p.571-585, 2015, Springer Netherlands
Keywords
Keywords
Example: Robots, Virtual Games, Mobile Phone, Public Playground, Adapted Toys
Humanoid robots, Interaction games, Non-verbal communication, Sign language tutoring
Play Systems with similar keywords
Motion sensing devices, humanoid and non-humanoid robots for enhancing motor and social skills of children with developmental problems
The Effect of Embodiment in Sign Language Tutoring with Assistive Humanoid Robots
Back
