LUDI
About
Contact
Search
Register
Log in
Back
KROG - Kinect-Robot Interaction and Gaming
Summary
Play Experience by Assistive Technology for play
Devices, services & contexts for play with children with disabilities
Name of Play Experience by AT
KROG - Kinect-Robot Interaction and Gaming
Type of project
Intervention
Finished research project
Ongoing research project
Summary
(~150 words: target group, aims, kind of activities, Play Experience by AT (devices, services and contexts) used, play experiences and results)
Krog project aims to develop a number of games with a Kinect-Robot-Big Screen interaction paradigm. The project is mainly focused on children with autism disorders, althought experiments have been done with children showing different abilities. Main goal of this project is to develop children’s capability of interaction and socialization in order to improve their quality of life. The robot used in the project has been developed to have an acceptable body (shape and physical properties, such as color, softness, robustness size, ...), and interesting abilities. Free interaction with the robot has provided interesting results. Kinect can detect the position of both the robot and people playing in its range and this information can be used to project on the screen elements of the game, so obtaining a mixed virtual-physical iteraction. A few games have been developed and tested, putting in evidence the great possibilities of this paradigm. The project is funded by Politecnico di Milano under the Polisocial grant program (http://www.polisocial.polimi.it/us/home/).
Play Experience by Assistive technology
Low-tech, high-tech products, services and contexts for play
Description, intended use(s), intended target group(s), manufacturer/developer, reference
The robot (Teo) based on Triskarino (http://www.deib.polimi.it/eng/equipments/dettagli/131), an omnidirectional base developed at AIRLab - Politecnico di Milano in several copies for several purposes. It has three omnidierectional wheels 8cm diameter and a diameter of about 40 cm, and can run at up to 1m/sec. It can be easily moved by a child, since omni-wheels do not show resistence, and the base is lightweight (about 2 Kg) and quite stable (most of the weight is on the bottom). On this base a soft body is mounted, covered by a soft cotton fabric, on which velcro elements can be attached (e.g., eyes, mouth, tongue, arms, ...). The body is soft enough to be pleasant to hug, and robust enough to resist punches and kicks. Sensors on the robot include distance sensors, accelerometers, and force sensors on the body. A hat can be added with push buttons (which can be personalized by cards inserted in them) and/or touch/proximity sensors. Actuators include a loudspeaker that can reproduce either pre-recorded or computer-generated sounds, a set of coloured LEDs, and the wheels for movement. The main computation onboard is done with Arduino Mega, and the robot can exploit the wireless communication link with an external computer to interact with Kinect and the screen. A remote control is also available through a standard bllue-tooth joypad, both to move the robot, and to produce a set of pre-programmed emotional patterns to be shown in (possibly automatic) answer to data from sensors and Kinect. Kinect is used to detect the position of both the robot and people playing in its range. A big screen or a projector can be used in games to add elements. The interaction sessions start with a familiarization phase both with the robot (free iteraction) and with its avatar o the screen, to explore the possibilities of interaction. Familiarization with the robot exploits a lot of interesting relationships and can be done both with a single child, or with a group of children also at different times. After familiarization, games can be played. Games can be personalized and designed through a friendly interface on a PC. The games so far implemented exploit a few abilities. In the "Color" game, the child is asked for the color of an object appearing on the screen and has to push a button corresponding to the correct color. In the "Witch says colors" game, there is a mat with six blobs of different colours and the screen is asking to reach a color (different settings are possible, involving relationships between robot and child). In the "Position" game, avatars of both the child and robot are on the screen and they are asked to take relative position. The original intended target group was autistic children with medium-low impairment, but the approach, upon suggestion of the operators has been successfully tested (mainly the free interation part) also with other kind of both physically and cognitively disable people. The project site is http://hoc13.elet.polimi.it/polisocial/
Context
The context of use
(home, school, rehab center or other environments)
Interactive sessions
Type of Play
Type of play in this play system
LUDI Classification of types of play
Cognitive Dimension
Practice
Symbolic
Constructive
Rule play (including videogames)
Social Dimension
Solitary
Parallel
Associative
Cooperative
Objectives
Objectives related to play according to ICF-CY
International Classification of Functioning, Disability and Health - Children & Youth Version (ICF-CY)
Play for the sake of play
Major life areas - d880 engagement in play
d8800 solitary play
d8801 onlooker play
d8802 parallel play
d8803 shared cooperative play
d8808 engagement in play, other specified
d8809 engagement in play, unspecified
Community social and civic life - d920 recreation and leisure time
d9200 play
Play-like activities
Therapeutic and educational objectives
b1 Mental functions
b2 Sensory functions and pain
b3 Voice and speech functions
b4 Functions of cardiovascular, hematological, immunological and respiratory systems
b5 Functions of digestive, metabolic and endocrine systems
b7 Neuromusculoskeletal and movement related functions
d1 Learning and applying knowledge (learning through symbolic play, learning through pretend play)
d2 General tasks and demands
d3 Communication
d4 Mobility
d7 Interpersonal interactions and relationships
Participant
Participant(s)
Number
1-5
5-10
10-20
>20
Chronological Age
0-3 years
3-6 years
6-12 years
12-18 years
Development Age
0-3 years
3-6 years
6-12 years
12-18 years
LUDI Categories of disabilities
Mental/intellectual impairments:
mild
moderate
severe
profound
Hearing impairments
Partially hearing impaired
Deaf
Visual impairments
Partially sighted
Blind
Communication disorders (speech and language disorders)
Physical impairments
Mild
Moderate
Severe
Autism Spectrum Disorders
Multiple disabilities
Explanation
Explanation on the use of low-tech, high-tech devices, services or contexts
Explanation
No instruction, self-discovery of the participant/subject
Verbal instruction, language and communication fitting to chronological age
Verbal instruction, language and communication is adapted
Visual and/or verbal instruction with AAC (Aumentative and Alternative Communication)
Visual instruction with written language
Visual instruction with pictures or drawings
Modeling by therapist/researcher
Hand over hand: therapist/researcher leads the actions of the participant
Prompting: therapist/researcher touches the participant as a key for further actions
Guided discovery: therapist/researcher coaches the participant so s/he discovers how to use the assistive technology
Modeling by peer
Visual instruction by peer
Verbal instruction by peer
Involvement
Adult: therapist/educator/researcher
Parent or significant others
Peer with disabilities
Peer without disabilities
Role
Non-participatory observer
Participatory observer
Providing instruction
After the instruction, providing supervision during play
Achievements
Summary of achieved effects
Analytic results on the first 22 children are still to be elaborated in details. Qualitative feedback is very positive from the operators and therapists. KROG can elicit operational behaviors, social interaction and emotional responses that normally do not occur using other methods, or that require a much longer time to be achieved. For example, an autistic child explicitly called a mate to play with Teo and the screen together: it was the first time he expressed the willingness to play in social mode. A girl with severe Hyperactivity Disorder relaxed in few minutes after meeting Teo, and she was able to concentrate on, and perform, a learning task (the “Colors” game) during the same session. Another autistic girl that never interacted with others invented a new game with Teo (make it following one of the possible add-ons: a tie) when she was alone with it in the familiarizazion phase, and then, when the others came in the arena, took the lead and showed them what was it possible to do. The possibility to develop new games and ad hoc settings is very large, and the tools to do that well accepted by the operators. The robot in itself can be used as a tool driven by the operator to interact with the children without personal involvement, and with a clearly not-human object that can have some personal reactions. Autonomous activity of the robot frees the operator from controlling it, but also freighten the operators that expect that something might happen that could not been recovered in the relationship. It is a question of both reliability of the autonomous reactions, and greater confidence in the still new tool.
Keywords
Keywords
Example: Robots, Virtual Games, Mobile Phone, Public Playground, Adapted Toys
Human-Robot Interaction, Kinect, touchless interaction, robogame, autism.
Back
