©2000 by Lillian Biermann Wehmeyer & Beverly Hardcastle Stanford

CRITIQUING A QUALITATIVE RESEARCH STUDY

Adapted from Gall, Borg, & Gall and Beverly Hardcastle Stanford, Ph.D.

As researchers, we cannot avoid orienting our studies with our beliefs about what knowledge is.

As a consequence, all research is biased. What is important is that the bias or perspective be recognized and incorporated into our evaluations. We need also to be aware of how our theoretical biases are influencing how we evaluate others.

Gall, Borg & Gall appear to subscribe to the positivist orientation toward research. As a consequence, some of their evaluation questions (pp. 609-612) are biased in that direction. While there are positivist qualitative researchers, they are in the minority. Most who engage in qualitative research are interpretive or critical theorists. I am a researcher who subscribes to the latter two perspectives.—Dr. Stanford

Gathering resources on a topic is challenging enough, yet it is only the beginning. One must ask whether the research was well designed and properly executed. Then one must consider whether the reported conclusions are supported by the study. The form on the following pages has been designed for that purpose.

Suggested procedure:

(1) Read the research article. Then complete the NUMBERED items in the SUMMARY column. This gives an overview of the entire study.

Note: If the study is complex, involving several research questions, you may find it easier to complete a separate form for each question, or for each group of closely related questions.

(2) Next, complete the rest of the SUMMARY column. Keep in mind that not all boxes will be applicable to all studies.

(3) Now fill in the CRITIQUE column. Do not just answer “yes” or “no,” but explain and justify your opinion.

(4) Next, review the critique column; then fill in the

EVALUATION

box at the end of the form.

(5) Finally, write a brief SUMMARY, as described.

QUALITATIVE CRITIQUE FORM

Your Names
_. _______

List the article in APA format

RESEARCH STEP	QUESTIONS	SUMMARY	CRITIQUE
Researcher(s)	SUMMARY: Are there multiple researchers, or only one? What information do the researchers present regarding their background and potential biases? CRITIQUE: How might the researchers’ background influence their interpretations of the data? Does that seem to be the case?
#1–Purpose/ Research Question(s)	SUMMARY: What is the purpose or research question(s)? CRITIQUE: Are the researchers trying to prove something they already believe? How do you know? Do the researchers demonstrate undue positive or negative bias in describing the subject of the study? If so, In what way?
Literature Review/ Theoretical Framework	SUMMARY: What previous work had been done leading up to this study? Do the authors present a theoretical orientation for their study? If so, how do they connect it to the study? CRITIQUE: How comprehensive is the literature review section of the report? Justify your answer. Does it include studies that you know to be relevant to the problem? If not, what has been overlooked?
Approach	SUMMARY: Is the study exploratory or confirmatory? How do you know? CRITIQUE: Is the approach appropriate to the purpose or research question(s)? Why or why not? If not, what approach would you recommend and why?
#2–Methodology	SUMMARY: Which of the types of qualitative research is being used (grounded theory or inquiry, historical, case study, etc.)? CRITIQUE: To what extent is the methodology appropriate to the purpose or research question(s)? Explain. If inappropriate, what might have been better?
#3–Participants	SUMMARY: Who are being studied? How were they selected? CRITIQUE: Did the selection procedure result in a case or cases that were particularly interesting? What would you expect the researcher to learn from them about the phenomena of interest? Why do you think so? What improvements would you suggest?
Context	SUMMARY: What is the context or setting of the study? CRITIQUE: What information enables the reader to transfer the findings to other settings? Is it adequate? Can a reader determine whether or not his or her context is similar to that of the study? If not, what information is missing?
#4—Data Sources	SUMMARY: What sources were used to gather data (interviews, questionnaires, artifacts, documents, etc.)? CRITIQUE: Were the data sources appropriate? What might have been more useful? Evaluate the adequacy of the instruments in relation to the research questions.
Trustworthiness of Data Sources	SUMMARY: What evidence is presented to demonstrate the trustworthiness of each data source? Were the data sources triangulated, that is, with multiple sources of data for the same phenomenon? If so, how? CRITIQUE: Are you convinced that the data collected match the data needed to respond to the purpose or research question(s)? …that the data were honest and accurate? To what extent are you convinced—or do you doubt—the trustworthiness of the data? Explain.
Ethics	SUMMARY: Does the author discuss ethical issues? If so, how have they been addressed? CRITIQUE: Do you see any ethical problems in the study? Have ethical issues been appropriately addressed? If so, how? Are any ethical issues not properly taken care of? What might have been done to address these concerns?
Data Collection	SUMMARY: What procedures were used to gather data and in what sequence? What was the intensity of data collection? For instance, over what period of time were observations made? …documents collected? etc. CRITIQUE: How do the authors demonstrate that their data collection procedures were unbiased and complete? …of sufficient intensity, e.g., continuous rather than fragmented? What additional information about procedures might be needed so that others could replicate the study?
#5–Analysis	SUMMARY: How were the data analyzed? What evidence is there of triangulation (for example, multiple persons working independently as they analyze data and then comparing results, multiple theories, multiple methods, or perspectives of multiple disciplines)? CRITIQUE: How clearly is the process of analysis described? Why do you say so? Did the report include a “thick” description (or, in a short article, quotations or other evidence of rich data)? To what extent is the analysis trustworthy? Why or why not?
Themes/Patterns	SUMMARY: Did any themes, patterns, or constructs emerge from the data? If so, what? CRITIQUE: Consider the categories or patterns and the themes or constructs that emerged? How well do they relate to the data? How plausible are they? Are you convinced? Why or why not?
#6–Findings	SUMMARY: What did the researchers learn? …or what insights did they gain? CRITIQUE: Consider the findings, hypotheses, or further questions in logical relationship to the data and analysis. Evaluate the match. Were alternative explanations considered? …needed? If so, what?
Conclusions: Implications and Recommendations	SUMMARY: What do the authors indicate as the significance of the study? What implications for practice are suggested? What hypotheses or questions are recommended for further research? CRITIQUE: How logically were the conclusions derived from the findings? Were statements of generalizability, if made, appropriately qualified? If the study was theory-based, do the conclusions relate to the theory? Is a reader motivated to take action? Explain your response.
Reference List and Appendixes	SUMMARY: Do the authors provide a list of references? …in APA style? Did they choose to provide a resource list for further study? Appendixes? CRITIQUE: If the format is not APA, is it at least consistent? Are all cited references provided? Did the authors look for an appropriate range of information sources as to date and format (books, journals, government documents, etc.)? Provide all necessary detail?
EVALUATION Overall, what do you think of the study? How well was it carried out? How important and useful are the findings? Other comments? Are you persuaded to follow the recommendations? Why or why not?

Write a SUMMARY, probably one to two paragraphs in length, to summarize the article as you might if the study appeared in your literature review for an article, grant application, or dissertation. Be sure to tell the reader who did what, why, where and when, and with whom. Mention the theoretical base (if any), and point out notable strengths and problems (if any).

Using game technology to teach six elementary school children with autism to take
a shower independently
Ya-Shu Kanga and Yao-Jen Changb

aDepartment of Special Education, Chung Yuan Christian University, Chung-Li, Taiwan; bDepartment of Electronic Engineering, Chung Yuan
Christian University, Chung-Li, Taiwan

ABSTRACT
Objective: This paper contributes to research investigating the effectiveness of video game intervention
in shower training of children with autism spectrum disorders. Previous studies have supported the use
of video games in special education.
Methods: We employed the Kinect sensor to gamify shower training. Specifically, a non-concurrent
multiple baseline design was adopted to demonstrate the relation between game-based intervention
and taking a shower independently. Six children in a special education class at a regular elementary
school participated in the experiment.
Results: Data showed that the percentage of correct task steps significantly increased among all six
participants; motivation to engage in training was enhanced as well, thus improving task performance
during the intervention and maintenance phases.
Conclusions: Although the game is a promising and highly accepted training tool for school-use, it
currently remains error-prone, and the requested demand exceeds the support that can be provided by
special education teachers. A more technically robust system, combined with additional attractive
games, will likely result in higher participant motivation and superior task performance.

ARTICLE HISTORY
Received 19 August 201

7

Revised 14 July 2018
Accepted 14 July 2018

KEYWORDS
Autism; Kinect; shower
training; video game

Introduction

Social communication and social interaction include the child’s
engagement in spontaneously reading and correctly interpret-
ing verbal and nonverbal social and emotional cues; engage-
ment in recognizing social and emotional information;
different social behaviors and their consequences in diverse
social tasks (e.g., how to initiate a conversation, how to negoti-
ate needs, how to make group entry).1 People with autism
spectrum disorder (ASD) demonstrate qualitative impairments
in social communication and social interaction across multiple
contexts. One of the most salient concerns of parents of chil-
dren with autism, as well as parents of children with other
disabilities, is whether or not their child will live a safe, pro-
ductive, and independent life. Individuals acquiring indepen-
dence early in life have more potential to thrive in domestic and
vocational settings.2,3 Individuals with ASD may have difficul-
ties in performing daily living tasks.4 Among other daily living
tasks, independent showering/bathing is an essential life skill
for people with ASD. Poor showering and bathing skills can
cause physical discomfort and negatively impact personal
hygiene, self-confidence, social competence, and acceptance,
thereby resulting in dependence on caregivers for assistance.

The difficulties in verbal reasoning and short-term mem-
ory present both pragmatic and pedagogic challenges in
teaching children with ASD to take a shower (among other
living skills) properly through traditional methods such as
verbal instruction and memorization tasks.5 Instructional

strategies that incorporate visual supports (e.g., pictures, in
vivo modeling, video modeling [VM]) are considered evi-
denced-based practices that capitalize on the strengths of
children with ASD.6–16 These technological interventions
can help students with ASD become more independent and
better prepared for adulthood by learning a variety of life
skills. In particular, Piccin, Crippa, Nobile, Hardan, and
Brambilla17 developed VM tools for helping individuals with
ASD in increasing their ability to perform personal hygiene
activities. Hayes and Hosaflook18 developed a mobile system
to support youth with ASD learn about and track healthy
hygiene behaviors to help teens and young adults develop
skills for independent living and employment. Additionally,
a randomized control trial examined the effectiveness of a 3-
week video-modeling brushing intervention delivered to
patients over the Internet.19 Among the various visual sup-
ports available, VM has increased in popularity in the last
three decades20 due to the advancement and availability of
technology, the decreasing cost of producing videos, and its
strategic efficiency in teaching daily living, as well as func-
tional, vocational, academic, and social skills.11 These differ-
ent strategies have generally been used in isolation; for
example, children may use picture prompts, video prompts,
or VM but usually do not adopt these strategies collectively.
Recently, Cihak et al.21 used augmented reality technology, a
new approach that fuses picture prompting and VM to train
elementary students with autism in a chain task.

CONTACT Yao-Jen Chang yjchang@cycu.edu.tw Department of Special Education, Chung Yuan Christian University, Chung-Li, Taiwana
Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/ipdr.

DEVELOPMENTAL NEUROREHABILITATION
2019, VOL. 22, NO. 5, 329–337
https://doi.org/10.1080/17518423.2018.1501778

© 2018 Taylor & Francis

An alternative approach to developing skills in children
with ASD is game-based training, a strategy made possible
by the recent advancements in sensor and human–computer
interaction technology. The gamification of training involves
the child playing a video game that engages him or her in a
target behavior, with the goal of subsequently imitating it in
real life. Gameplay demands focus and attention, motivates
the user to practice, and provides the user with a sense of
achievement, even if the user cannot perform that task in the
real world.22,23 Therefore, gamification has become a new
candidate for an intervention strategy that can successfully
teach a variety of skills to individuals with ASD. Gotsis et al.24

designed a video game intervention for children with ASD,
and their preliminary results showed that games focusing on
social skills training have the potential to improve adaptive
behaviors and quality of life for children with ASD. Dickinson
and Place25 showed that game-based interventions using the
Nintendo game system (Kyoto, Japan) and the “Mario &
Sonic at the Olympics” software package appear to be poten-
tial mechanisms that can improve the social functioning of
children with ASD. Similarly, Cai et al.26 designed a game
called “Virtual Dolphinarium” that showed promising results
in promoting learning and positive behavior among children
with ASD.

In short, previous studies have supported the use of video
games in special education. However, despite the extent of
difficulties related to showering that are faced by individuals
with ASD and the importance of independent showering, no
study has yet addressed how interactive game technology can
be used to teach multistep showering skills to children with
ASD. In addition, teachers in special education schools cannot
train children with ASD to take a shower independently,
unless the school is equipped with showering facilities and
the training program is provided. Nevertheless, even with the
necessary equipment provided, such training programs are
time consuming and labor intensive. Therefore, we developed
an interactive game that turns shower training into a fun
learning experience, in addition to allowing teachers to max-
imize their classroom availability for other educational pur-
poses. For example, instructors in large classrooms may not
have sufficient time for repeated one-on-one instruction or
live modeling. However, children can play the showering
game at school without showering facilities under their tea-
chers’ supervision until their ability to take a shower has
become sufficiently independent.

The motion capture capability of Kinect makes it a candi-
date for implementing assistive technology targeting students
with ASD. For example, the MEBook system used Kinect
sensor to inject self-images into a social narrative game to
teach students with ASD proper greeting behaviors.27 The
Kinect sensor was also used to automatically detect stereoty-
pical motor movements such as hand flapping.28 In this study,
we gamified independent shower training using the Kinect
sensor to motivate children with ASD who had not developed
showering skills. Specifically, we developed a game based on
Kinect’s gesture recognition technology, which has recently
increased in popularity in the video game interaction design
domain. The incorporation of gesture recognition technology
prevents the children from having to wear intrusive body

sensors; additionally, Kinect provides real-time three-dimen-
sional (3D) anatomical landmark position data, and it is
inexpensive, portable, and simple to set up. Studies have also
shown that the Microsoft Kinect and 3D motion analysis
systems have comparable intertrial reliability and excellent
concurrent validity.29–31

Chang, Chen, and Huang32 and Chang, Han, and Tsai33

have leveraged the human gesture recognition capabilities of
Kinect to determine whether a user performed exercises cor-
rectly in physical rehabilitation. Hung, Chang, and Han34 also
used Kinect to recognize actions when they developed a video
game that demonstrated the potential to increase range of
motion for adolescents with cerebral palsy. However, our
study is the first to use Kinect in the development of inter-
active games for children with ASD to encourage engagement
in shower training. The game was designed according to a
showering task analysis, which outlined the task steps
involved in taking a shower and simulated them in the com-
puter game.

The purpose of this study was to examine whether game
technology is an effective way to teach elementary students
with ASD to take a shower independently and to determine
the long-term maintenance effects of using game technology
to facilitate independent showering.

Method

Participants

Four male and two female elementary school children with
ASD (Allen, Bart, Chris, Diane, Emilie, and Fred) participated
in this study. All of the children were enrolled in special
education programs under the autism category, and their cog-
nitive and adaptive functioning fell within the moderate intel-
lectual disability range. Specifically, the children were recruited
based on the following criteria: (a) diagnosis of autism or an
intellectual disability, (b) an Individual Education Plan goal to
improve adaptive behavior related to personal care, (c) no
physical disability that would impede the performance of the
skill, (d) ability to understand the objects in the video game
such as shampoo and showerhead, (e) ability to recognize the
model in the game was controlled by the children’s body, and
(f) agreeing to participate in the study. Table 1 presents a list of
the characteristics of each participant.

The first participant, Allen, was 9 years old and in the third
grade. Although Allen had few verbal communication skills
and understood a limited set of instructions, he was able to
use the Picture Exchange Communication System (PECS) to

Table 1. Children’s characteristics.

Participant Age CTONI-2
VABS-II adaptive behavior

standard scores (%)
VABS-II age
equivalents

Allen 9:3 76 74 4 6:2
Bart 9:9 < 65 58 0.3 3:1

0

Chris 11:6 < 65 61 0.5 5:2 Diane 9:4 75 58 0.3 3:8 Emilie 9:8 < 65 49 < 0.1 2:0 Fred 11:9 < 65 55 0.1 3:8

Note. CTONI-2: Comprehensive Test of Nonverbal Intelligence—Second Edition;
VABS-II: Vineland Adaptive Behavior Scales-II, Second Edition; %: percentile.

330 Y.-S. KANG AND Y.-J. CHANG

communicate. The second participant, Bart, was also 9 years
old and in the third grade. He had complex communication
needs and primarily used a letter board or iPad to commu-
nicate. Chris, the third participant, was 10 years old and in the
fourth grade. Similar to Allen, Chris had limited verbal skills
and used the PECS to communicate. Diane was 9 years old
and in the third grade. She had no verbal communication,
could not understand oral instructions, and had difficulties
with social adaptation. Emilie was 9 years old and in the third
grade. She had few verbal skills and understood a limited set
of instructions; she also had difficulty with hand–eye coordi-
nation. Fred was 11 years old and in the fifth grade. Similar to
Diane, he did not verbally communicate, could not under-
stand oral instructions, and had difficulty with academic
learning. None of the participants had previous experience
with Kinect. The information about the participants was
gathered from the teachers. All six participants were given
pseudonyms to protect their privacy, and informed consent
was provided by the service organization, individual staff
members involved in the study, and the main caregivers on
behalf of the children with ASD. The study was approved by
the Regional Institutional Review Board under approval num-
ber LSHIRB No. 15-007-A2.

Setting

All of the students attended a special education class for part
of their school day to address functional or life skill difficul-
ties. Nine other students with multiple disabilities were also
present in their classroom. During the baseline, game-based
intervention, and maintenance phases, the children took a
shower independently in their own bathroom at home, as
the skill of showering naturally occurred in that location.
The game-based intervention phase occurred only in the
special education classroom. The Kinect sensor requires an
area of 15 ft by 9 ft for motion capture of the player. Within
this area, furniture should be removed. The bathrooms where
the interventions took place were participants’ own bath-
rooms installed with a camera. The shower taking was video
recorded and later examined by the special education teachers
for evaluating correctness according to task analysis.

Materials

The game is called Take a Shower! and was designed accord-
ing to the 25 steps that were identified for taking a shower
(Table 2). The task-fidelity checklist as shown in Table 2 was
developed by the special education teachers that we worked
with. These steps were determined based on the School
Ability Index which was released by the Ministry of
Education for all of the elementary and secondary schools.
The children were guided through each step in the task-
fidelity checklist. The game started by children selecting a
favorite cartoon model by holding the hand over the selected
model on the screen for 3 s. The movements of the model
including undressing, picking up the hand shower, and turn-
ing on/off the water were controlled by the child using the
body. To make the game more fun, a water fairy was used as a
visual cue and moved around each part of a body to guide

children to wet the body accordingly. For example, to prompt
the children to wet the chest, the fairy would stop at the chest
of the cartoon model. Points were scored when children
aimed their handheld showerhead at the water fairy accu-
rately. Similarly, an animated germ demon would mark the
parts of the body of a model that needs cleaning, and a score
would be generated based on how much of the model the
students shampooed using a shampoo wand activated by real
showering motions in front of the Kinect camera. Figure 1
shows a few screenshots including selecting a role, wetting the
body, and applying the shampoo. The reinforcement includes
a cute animal that pops up on the screen to give praise such as
“You are so great!” when a job is well done. In addition, to
make it more fun, the face of the germ demon changes into a
sad expression before disappearance when the part of the
body the germ demon stops at is cleaned.

This study tested the proposed system in an urban special
education class, which receives students with special needs
from kindergarten through elementary school. The Kinect
device was connected to a Lenovo Thinkpad T61 notebook
computer, and the game software developed in-house was
installed with Microsoft Windows 7. The computer had an
audio module, which we used to deliver audio feedback, and
an external 22-in LCD screen that was used for visual inter-
action. The software was coded using the Kinect2Scratch soft-
ware and the Scratch software language environment tools to
form an integrated development platform. The interactive
interface with audio and video feedback was programmed to
reinforce children’s motivation to engage in the training. For
optimal performance of the Kinect sensor, the participants
were required to stand approximately 3 ft in front of the
Kinect module.

Variables and data collection

The independent variable in this study was the use of game-based
intervention, and the dependent variable was the number of
organized task steps for taking a shower which were completed

Table 2. Task-fidelity checklist for taking a shower.

1. Undress
2. Pick up and hold the hand shower
3. Turn on water
4. Wet the body
5. Turn off water
6. Put down the hand shower
7. Squeeze shampoo
8. Apply the shampoo on the left neck
9. Apply the shampoo on the right neck
10. Apply the shampoo on the left arm
11. Apply the shampoo on the right arm
12. Apply the shampoo on the chest
13. Apply the shampoo on the tummy
14. Pick up and hold the hand shower
15. Turn on water
16. Rinse the left neck
17. Rinse the right neck
18. Rinse the left arm
19. Rinse the right arm
20. Rinse the chest
21. Rinse the tummy
22. Turn off water
23. Put down the hand shower
24. Wipe the body with towel
25. Get dressed

DEVELOPMENTAL NEUROREHABILITATION 331

independently. Data were collected through the use of a data sheet
designed to record the task chain. All of the parents agreed to use a
mobile phone to record their children during showering. Video
clips were carefully handled and processed by two special educa-
tion teachers who served as raters in the study. Event recording
procedures were used to record each step as independently per-
formed or incorrectly performed. An independent response was
defined as initiating the first step in the task analysis within 5 s and
completing each step within 20 s without parental assistance. The
number of independent steps completed was divided by the total
number of steps of the task analysis (i.e., 25) and then multiplied
by 100 to calculate the percentage of steps completed indepen-
dently. The final statistics were graphed for visual analysis.

Experimental conditions

A non-concurrent multiple baseline design35 was adopted to
demonstrate the relation between game-based intervention and
the ability to take a shower independently. The experiment com-
prised three phases: (a) a baseline phase, in which at least a few
sessions were performed to collect baseline data on the partici-
pants; (b) an intervention phase, in which the gaming system was
used for shower training; and (c) a maintenance phase, which was
conducted 2 weeks after the intervention was finished and
assessed the continued performance of the participants.

The six children were assigned into two groups due to their
individual school schedules. Allen, Bart, and Chris constituted
Group 1, and Diane, Emilie, and Fred constituted Group 2.
Groups 1 and 2 participated in the experiments during the
first half and the second half of the semester, respectively.

Baseline phase
During the baseline phase, the children took a shower indepen-
dently at home. Three sessions in a week were recorded, although
the children took the shower on a daily basis. Their parents
recorded the shower process but did not provide oral instructions
or other intervention. In other words, the parents did not interfere
with or try to prevent any inaccuracies from occurring in the
participants’ performance. The baseline phase comprised a mini-
mum of three sessions or until stability was achieved.

Intervention phase
For the first session of this phase, students were instructed on
how to operate the Kinect-based training game. They were
first directed to physically turn on the device and start the
game by raising their hands. When students had problems
going through various task steps in the game, the teacher used

gestural or visual prompts. Pictures were used to prompt the
children to stand within the range of the Kinect sensor, stand
upright, or wave hands to start the game. Teachers were
trained on how to prompt correct responding. Each child
was required to finish every step in the game prior to initiat-
ing the intervention phase, and a full game play took 15 min
for each participant. The teacher supervised the children for
the entire duration of the training session. Two sessions were
scheduled for each child every week during the intervention
phase. The participants used cues on the screen to complete
the game play; in addition, picture-based feedback was used to
enrich the training process and increase motivation to pay
attention to the shower process. The teacher did not interfere
with or try to prevent any inaccuracies from occurring in the
participants’ performance. Depending on the score of the
game, the teacher either praised the student or encouraged
her or him to continue. The intervention phase consisted of
21 sessions. The teachers gave praise contingent on how many
steps were completed correctly during game play. The praise
was given when the child made a notable improvement com-
pared to the last time. As students gained mastery of the
showering steps, the cutoff of giving praise was increased.
Teachers did not provide additional instruction to link the
video game and the live showering. During this phase, chil-
dren also took a shower independently at home. Children
played the game by day at school for each session in the
intervention phase and took the shower later when they
came home. During the intervention phase, for each child,
5–6 h elapsed between the showering game and the home
shower. Parents recorded this shower process for performance
assessment but did not provide oral instructions or other
intervention. Parents were particularly instructed not to rein-
force correct showering. A text message was sent to remind
the parents in the morning of the home shower sessions so
that they did not forget the recording. Parents were instructed
to facilitate home showering through a briefing they received
at the school before the experiment began. Their fidelity was
also monitored by the recording.

Maintenance phase
This phase began 4 weeks after the intervention phase to deter-
mine whether the participants maintained the skills that they had
acquired. During this phase, participants did not have the gaming
system but instead participated in the activity directly using the
same task steps as they had during the baseline phase. Parents
recorded their children’s shower procedures for performance
assessment.

Figure 1. Game screenshots including selecting a role, wetting the body, and applying the shampoo.

332 Y.-S. KANG AND Y.-J. CHANG

Interobserver agreement

Reliability observers watched the recorded videos and col-
lected data on the number of steps performed correctly during
at least 40% of the sessions and across all phases of the study
for each participant (range 45–77%). The observers were
special education teachers who were trained to collect data;
we provided them with the task analysis data sheet, verbally
explained the procedures, and answered all questions regard-
ing the process. This training took approximately 25–40 min.
Agreement between the trainer and reliability observers on
the correctly performed steps was calculated on a session-by-
session basis, using the following formula: agreements/(agree-
ments + disagreements) × 100%. The interobserver agreement
range was between 92% and 100%. The resulting agreement
percentages were 95% on average.

Results

This study assessed the effectiveness of the proposed system
with regard to elementary-age children with ASD acquiring
the skills required to successfully take a shower indepen-
dently. The percentage of steps performed independently
for each child is presented in Figure 2. Overall, the children’s
independent performance in terms of task correctness imme-
diately increased when the game intervention was intro-
duced, and all of the participants acquired and maintained
the skills necessary for the task of taking a shower indepen-
dently. During the baseline phase, Allen completed a mean
of 60% of the steps for taking a shower independently;
during the intervention and maintenance phases, his perfor-
mance immediately improved to 89% and then further
increased to 98%, respectively. Bart’s mean percentage of
steps performed independently during the baseline phase
was 28%; however, following the introduction of the inter-
vention, his mean performance immediately ascended to
67%. During the intervention phase, Bart also required ver-
bal, gestural, and one partial physical prompt. Bart main-
tained 82% performance during the maintenance phase.
Chris’s mean percentage of steps performed independently
during the baseline phase was 75%. During the intervention
and maintenance phases, his mean performance increased to
98% and then 100%, respectively. During the baseline phase,
Diane completed a mean of 58% of the steps for taking a
shower independently. During the intervention phase, her
performance immediately improved to 98%, and she demon-
strated 100% independence during the maintenance phase.
Emilie’s mean percentage of steps performed independently
during the baseline phase was 34%. Following the introduc-
tion of the intervention, her mean performance immediately
ascended to 90%; she later further improved to 99% perfor-
mance during the maintenance phase. Finally, Fred’s mean
percentage of steps performed independently during the
baseline phase was 49%. During the intervention phase, his
mean performance immediately increased to 79%; similar to
Bart, Fred also required verbal, gestural, and one partial
physical prompt. Fred successfully maintained 92% of the
steps during the maintenance phase.

Discussion

For the six participants, Allen, Bart, Chris, Diane, Emilie, and
Fred, the rate of improvement in taking a shower indepen-
dently was 62%, 193%, 33%, 72%, 191%, and 88%, respec-
tively. Furthermore, the results indicate that the proposed
system, in conjunction with operant conditioning strategies,
can facilitate the progress of children with ASD taking a
shower independently. Across all the six participants, the
skill maintained well in the absence of the video game. It
might imply that the video game was a great instructional
tool for the children with ASD to learn visually and the 21
sessions of interventions across 11 weeks made a positive
impact on learning to take the shower independently.
However, more future studies are needed to investigate this
issue. The data in the results suggest that these students were
capable of generalizing from the contrived shower simulation
to the real shower under significantly different stimulus con-
ditions. The improvements showed that the game-based inter-
vention under a stimulus condition of being clothed in a
classroom has generalized to a different stimulus condition
of being unclothed in a bathroom.

For interventions that involve visual modeling, participants
watch the training video first and perform the target behavior
later by recalling the content in the training video. However,
the memory retention may not be sufficient to support the
execution of the specific tasks for the population we address.
In contrast, the Kinect-based game intervention enabled par-
ticipants to practice showering while they played the game.
The change of this intervention might hypothetically involve
less mental workload in the improvement of the home shower
sessions.

In the baseline phase, the parents reported that Allen often
forgot to clean the neck and arms; he was also easily distracted
by the bubbles. However, with the introduction of the training
game, the teachers found that he gradually understood that he
had to clean the neck and arms to continue the game (which
he was motivated to do because he loved playing the game).
After repeated practice, Allen also remembered to pay atten-
tion to cleaning his own neck and arms when showering at
home, according to the parents. By contrast, the parents
reported that Bart had great difficulty in taking a shower
independently during the baseline phase. The teachers also
found that he did not adapt to the game well in the early stage
of intervention. With the assistance of the teacher, he was able
to grasp the game after 2 weeks of practice; however, the
teacher found that he could not completely generalize the
skills he learned in the game to his daily life. Nevertheless,
the results showed that Bart showed great progress overall,
when his task performance score in the maintenance phase
was compared with that in the baseline phase. Bart’s parents
reported that he was motivated to improve showering skills
because he always wanted to look clean but often could not;
after the training, he was very happy to finally be successful.
According to the parents, Chris was able to take a shower
independently during the baseline phase. However, the task
step analysis showed that the “washing the neck” step was
often neglected. The teachers found that Chris enjoyed the
game very much, and he was happy when he played the game

DEVELOPMENTAL NEUROREHABILITATION 333

0

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 60 70

0
10
20
30
40
50
60
70
80
90
100

0 10 20 30

0
10
20
30
40
50
60
70
80
90
100
0 10 20 30
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60 70
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60 70

Baseline Intervention Maintenance

Allen

Bart

Chris

Diane

Emilie

Fred

%
o

f
s

t
e

p
s
c

o
m

p
le

t
e

d
i
n

d
e

p
e

n
d

e
n

t
ly

%
o
f
s
t
e
p
s
c
o
m
p
le
t
e
d
i
n
d
e
p
e
n
d
e
n
t
ly
%
o
f
s
t
e
p
s
c
o
m
p
le
t
e
d
i
n
d
e
p
e
n
d
e
n
t
ly
%
o
f
s
t
e
p
s
c
o
m
p
le
t
e
d
i
n
d
e
p
e
n
d
e
n
t
ly
%
o
f
s
t
e
p
s
c
o
m
p
le
t
e
d
i
n
d
e
p
e
n
d
e
n
t
ly
%
o
f
s
t
e
p
s
c
o
m
p
le
t
e
d
i
n
d
e
p
e
n
d
e
n
t
ly

Date

2
/
1

4
/
2

0
1

7

2
/
2

1
/
2

0
1
7
2
/
2

8
/
2

0
1
7

3
/
0

8
/
2
0
1
7

3
/
1

6
/
2

0
1
7

3
/
2

4
/
2
0
1
7

4
/
0

4
/
2
0
1
7

4
/
1

2
/
2
0
1
7
4
/
2

0
/
2

0
1
7
4
/
2
8
/
2
0
1
7

5
/
0

9
/
2

0
1
7

5
/
1

7
/
2

0
1
7

5
/
2

5
/
2
0
1
7

6
/
0

2
/
2
0
1
7
Baseline Intervention Maintenance

Figure 2. Task performance of Allen, Bart, Chris, Diane, Emilie, and Fred.

334 Y.-S. KANG AND Y.-J. CHANG

and even happier when he scored; he even frequently asked
the teacher to give him more time to play the game. The
teacher agreed in the hope that he could improve his task
performance for neck cleaning in the shower. The results
confirmed that he generalized the skill very well.

After a review of her recording by the teachers, it was
revealed that Diane always neglected the neck part during
the baseline phase. The teachers also found that she was less
engaged at the beginning of the intervention phase because
she had difficulty understanding oral instructions; therefore,
the teacher used gestures to help Diane gradually adapt to the
game. With repeated practice, she finally understood that the
neck cleaning was an integral part of showering. The results
showed that she eventually generalized the skill very well and
made tremendous progress during the maintenance phase.
According to the teachers, Emilie liked the shower training
game because she liked water; however, she had difficulty
adapting to the game at the beginning of the intervention
phase. With repeated practice, she eventually handled the
game very well. A review of the video recording revealed
that she often neglected the chest and tummy areas during
the showers, and more training time was thus spent on those
parts. The results showed that she finally acquired the task
skill and maintained her performance in the phase. The final
participant, Fred, often skipped task steps during the baseline
phase, according to the parents. The teachers found that he
had become interested the moment he saw the game for the
first time; thus, he actively participated in the game at every
intervention session. After repeated practice, the results
showed that Fred gradually familiarized himself with the
task steps required to take a shower, and his task performance
score increased significantly in the maintenance phase.

Social validity was evaluated through a survey, which
assessed the participants’ experience and success during the
study. The survey was administered during a brief interview
to each child’s parents, with questions analyzing personal
opinions regarding practical use of the video game. Open-
ended and closed questions examined the parents’ general
thoughts regarding the video game and asked whether the
game had helped their children learn the showering skills.
Other questions assessed whether the game changed the
method by which each participant took the shower indepen-
dently. Parents were also asked whether they would like to
have their children participate in the future and what tasks
would be helpful for their children to learn. Finally, the inter-
view addressed whether they had discussed the video game
with anyone outside of school. One researcher asked each
participant’s parents the questions, and another audio
recorded the verbal responses. Another survey was distributed
to each teacher as a hard copy on which to write responses.
These questions evaluated the participants’ satisfaction and
their perception of whether the video game was acceptable
and useful with students.

The parents thought the video game was very useful and it
had helped their children learn the showering skills effectively.
The game also changed the method by which their children
took the shower independently. Additionally, parents would
like to have their children participate in the future and have
shopping skills and pedestrian safety listed in the game-based

interventions also. Finally, parents discussed outside of school
the video game with their friends who also had children with
ASD. They highly recommended the game and encouraged
their friends to contact the teachers for more information.

According to the teachers, participants enjoyed playing the
video game and found the game to be an enjoyable motivator.
“The game caught their attention, and they said they would
prefer it to one-on-one teaching of the skills,” recorded one
particular teacher. The teachers agreed that the video game
improved independence and make teaching easier. The tea-
chers expressed a wish for more time and resources to create
additional video games for their students. They also found the
video game to be up-to-date with current teaching methods.
Teachers would like to use this method with their other
students to reduce direct verbal prompting and other indivi-
dual instruction. The teachers commented that education
should include more technology where possible.

Showering and bathing skills children acquired during the
intervention can bring physical comfort and positively impact
personal hygiene, self-confidence, social competence, and
acceptance, thereby resulting in a greater level of indepen-
dence. Additionally, the game-based intervention can benefit
children with ASD for the learning of other functional skills
such as shopping, pedestrian safety, and food preparation.

As this and other studies have demonstrated, even simple
tasks can present complex challenges for some individuals with
ASD. Organizing tasks into a sequence of smaller steps or
actions is an evidence-based practice for teaching new skills to
students with severe developmental disabilities.36 Moreover,
having an understanding of all the steps involved for a particu-
lar task can assist in identifying those steps that require extra
training and can help teach tasks in a logical progression.37 The
results of the present study show that the proposed game helped
children with ASD familiarize themselves with all the task steps
involved in taking a shower and, in particular, assisted them
with the task steps where they needed extra practice.

Gaming (e.g., the use of Kinect in this study) is a relatively
new aspect of training. The game has a seeming advantage of
serving an enjoyable motivator during the interventions.
Nevertheless, it has been utilized for skills development for
not only people with ASD but also for other populations such
as those with intellectual disabilities, attention-deficit/hyper-
activity disorder, or cognitive impairments.38–41 Gaming has a
number of advantages compared with traditional learning.
Specifically, this type of training can be directed toward the
development of a particular skill by organizing exercises that
are (or that gradually become) more challenging; is often
perceived as enjoyable and motivating, which increases long-
term adherence; and can in many cases be performed at home
or at a central location, which increases the frequency of
training.25

One key limitation of this study is that the results are based
on only six cases. Therefore, a general conclusion cannot be
extrapolated regarding the efficacy of the proposed system. The
lack of data on errors made during video game playing is also a
limitation of the study. We hope to enhance the game design by
adding an error recognition feature. Future studies should
include experiments that involve more participants with ASD
and focus on further evaluating the game-based training system.

DEVELOPMENTAL NEUROREHABILITATION 335

Additionally, more interactive features should be added to
enhance user experience. The effectiveness of this technology
for people with other types of developmental disabilities should
also be explored. We have started to design more game features
that include the additional task steps needed to clean the lower
limbs and the back. On the other hand, we are considering
modifications that might be made to the task analysis to make
the game more inclusive for children with profound ASD and
make it easier to put on a task analysis. Notably, our research
will continue to investigate if the results herein can be general-
ized to the broader population of children with ASD or even to
children with other disabilities. One of our ongoing research
projects is to train parents and even grandparents to support
learning through the video game.

Conclusions

To conclude, the proposed video game can be used for effec-
tive shower training of children with ASD. The six partici-
pants in this study showed overall improvements in their task
performance following the intervention. Moreover, the train-
ing game was well received by both the six participants and
their special education teacher who used the intervention.
Although the Take a Shower! system is a promising and highly
accepted training tool for school-use, it currently remains
error-prone, and the requested demand exceeds the support
that can be provided by special education teachers. A more
technically robust system, combined with additional attractive
games, would likely result in higher participant motivation
and enhanced task performance. This would subsequently
both reduce the need for parents or caregivers to motivate
their children extrinsically and allow for field researchers to
investigate the effectiveness of the system.

Declaration of interest

The authors declared that they have no financial or nonfinancial conflicts
of interests with respect to the publication of this research article.

Funding

This work was supported by the National Science Council [MOST
106-2221-E-033-011].

References

1. Crick NR, Dodge KA. A review and reformulation of social-
information-processing mechanisms in children’s social adjust-
ment. Psychol Bull. 1994;115:74–101. doi:10.1037/0033-
2909.115.1.74.

2. Pierce KL, Schreibman L. Teaching daily living skills to children
with autism in unsupervised settings through pictorial self-man-
agement. J Appl Behav Anal. 1994;27(3):471–81. doi:10.1901/
jaba.1994.27-471.

3. Shipley-Benamou R, Lutzker JR, Taubman M. Teaching daily
living skills to children with autism through instructional video
modeling. J Posit Behav Interv. 2002;4(3):166–77. doi:10.1177/
10983007020040030501.

4. Carpentieri S, Morgan SB. Adaptive and intellectual functioning
in autistic and nonautistic retarded children. J Autism Dev
Disord. 1996;26:611–20.

5. Quill KA. Instructional considerations for young children with
autism: the rationale for visually cued instruction. J Autism Dev
Disord. 1997;27:697–714.

6. Ayres KM, Langone J. Intervention and instruction with video for
students with autism: a review of the literature. Educ Train Dev
Disabil. 2005;40(2):183–96.

7. Bryan LC, Gast DL. Teaching on-task and on-schedule behaviors
to high-functioning children with autism via picture activity sche-
dules. J Autism Dev Disord. 2000;30:553–67.

8. Hine JF, Wolery M. Using point-of-view video modeling to teach
play to preschoolers with autism. Topics Early Child Spec Educ.
2006;26(2):83–93. doi:10.1177/02711214060260020301.

9. Johnson JW, Blood E, Freeman A, Simmons K. Evaluating the
effectiveness of teacher-implemented video prompting on an
iPod Touch to teach food-preparation skills to high school
students with autism spectrum disorders. Focus Autism Other
Dev Disabl. 2013;28(3):147–58. doi:10.1177/
1088357613476344.

10. Kellems RO, Frandsen K, Hansen B, Gabrielsen T, Clarke B,
Simons K, Clements K. Teaching multi-step math skills to adults
with disabilities via video prompting. Res Dev Disabil.
2016;58:31–44. doi:10.1016/j.ridd.2016.08.013.

11. Kellems RO, Edwards S. Using video modeling and video
prompting to teach core academic content to students with
learning disabilities. Preventing Sch Failure: Altern Educ
Child Youth. 2016;60(3):207–14. doi:10.1080/
1045988X.2015.1067875.

12. King AM, Thomeczek M, Voreis G, Scott V. iPad® use in children
and young adults with Autism Spectrum Disorder: an observa-
tional study. Child Language Teaching Therapy, 30(2). 2014;
159–73. doi:10.1177/0265659013510922.

13. Mechling LC. Assistive technology as a self-management tool for
prompting students with intellectual disabilities to initiate and
complete daily tasks: a literature review. Educ Train Dev Disabil.
2007;42(3):252–69.

14. Mechling LC. Review of twenty-first century portable electro-
nic devices for persons with moderate intellectual disabilities
and autism spectrum disorders. Education and Training in
Autism and Developmental Disabilities, 2011;46(4):479–498.

15. Mechling LC, Ayres KM, Bryant KJ, Foster AL. Continuous video
modeling to assist with completion of multi-step home living
tasks by young adults with moderate intellectual disability. Educ
Train Autism Dev Disabilities. 2014;49(3):368–380.

16. Rayner C, Denholm C, Sigafoos J. Video-based intervention for
individuals with autism: key questions that remain unanswered.
Res Autism Spectr Disord. 2009;3(2):291–303. doi:10.1016/j.
rasd.2008.09.001.

17. Piccin S, Crippa A, Nobile M, Hardan AY, Brambilla P. Video
modeling for the development of personal hygiene skills in youth
with autism spectrum disorder. Epidemiol Psychiatr Sci. 2018
Apr;27(3):127–132.

18. Hayes GR, Hosaflook SW (2013, June). HygieneHelper: pro-
moting awareness and teaching life skills to youth with autism
spectrum disorder. In Proceedings of the 12th International
Conference on Interaction Design and Children (pp. 539–42).
ACM.

19. Popple B, Wall C, Flink L, Powell K, Discepolo K, Keck D,
Mademtzi M, Volkmar F, Shic F. Brief report: remotely delivered
video modeling for improving oral hygiene in children with ASD:
a pilot study. J Autism Dev Disord. 2016;46(8):2791–96.
doi:10.1007/s10803-016-2795-4.

20. McCoy K, Hermansen E. Video modeling for individuals with
autism: a review of model types and effects. Education and
Treatment of Children. 2007;183–213. doi:10.1353/
etc.2007.0029.

21. Cihak DF, Moore EJ, Wright RE, McMahon DD, Gibbons
MM, Smith C. Evaluating augmented reality to complete a
chain task for elementary students with autism. J Spec Educ
Technol. 2016;31(2):99–108. doi:10.1177/0162643416651724.

22. Chang YJ, Kang YS, Huang PC. An augmented reality (AR)-based
vocational task prompting system for people with cognitive
impairments. Res Dev Disabil. 2013;34(10):3049–56. doi:10.1016/
j.ridd.2013.06.026.

336 Y.-S. KANG AND Y.-J. CHANG

23. Chang YJ, Kang YS, Liu FL. A computer-based interactive game
to train persons with cognitive impairments to perform recycling
tasks independently. Res Dev Disabil. 2014;35(12):3672–77.
doi:10.1016/j.ridd.2014.09.009.

24. Gotsis M, Piggot J, Hughes D, Stone W (2010, June). SMART-
games: a video game intervention for children with autism
spectrum disorders. In Proceedings of the 9th International
Conference on Interaction Design and Children (pp. 194–97).
ACM.

25. Dickinson K, Place M. The impact of a computer-based activity
program on the social functioning of children with autistic spec-
trum disorder. Games for Health Journal. 2016;5(3):209–15.
doi:10.1089/g4h.2015.0063.

26. Cai Y, Chia N, Thalmann D, Kee N, Zheng J, Thalmann N.
Design and Development of a Virtual Dolphinarium for
Children With Autism. IEEE Trans Neural Syst Rehabil Eng.
2013;21(2):208–17. doi:10.1109/TNSRE.2013.2240700.

27. Uzuegbunam N, Wong WH, Cheung SCS, Ruble L (2015, June).
MEBook: kinect-based self-modeling intervention for children
with autism. In Multimedia and Expo (ICME), 2015 IEEE
International Conference on (pp. 1–6). IEEE.

28. Gonçalves N, Rodrigues JL, Costa S, Soares F (2012, August).
Preliminary study on determining stereotypical motor move-
ments. In Engineering in Medicine and Biology Society
(EMBC), 2012 Annual International Conference of the IEEE
(pp. 1598–601). IEEE.

29. Clark RA, Pua YH, Fortin K, Ritchie C, Webster KE, Denehy L,
Bryant AL. Validity of the Microsoft Kinect for assessment of pos-
tural control. Gait Posture. 2012;36(3):372–77. doi:10.1016/j.
gaitpost.2012.03.033.

30. Dutta T. Evaluation of the Kinect™ sensor for 3-D kinematic
measurement in the workplace. Appl Ergon. 2012;43(4):645–49.
doi:10.1016/j.apergo.2011.09.011.

31. Galna B, Barry G, Jackson D, Mhiripiri D, Olivier P, Rochester L.
Accuracy of the Microsoft Kinect sensor for measuring movement in

people with Parkinson’s disease. Gait Posture. 2014;39(4):1062–68.
doi:10.1016/j.gaitpost.2014.01.008.

32. Chang YJ, Chen SF, Huang JD. A Kinect-based system for physi-
cal rehabilitation: a pilot study for young adults with motor
disabilities. Res Dev Disabil. 2011;32(6):2566–70. doi:10.1016/j.
ridd.2011.07.002.

33. Chang YJ, Han WY, Tsai YC. A Kinect-based upper limb rehabi-
litation system to assist people with cerebral palsy. Res Dev
Disabil. 2013;34(11):3654–59. doi:10.1016/j.ridd.2013.08.021.

34. Hung JW, Chang YJ, Han WY. Game technology to increase
range of motion for adolescents with cerebral palsy: a feasibility
study. Int J Disabil Hum Dev. 2016;16(3):267–274.

35. Hammond D, Gast DL. Descriptive analysis of single subject research
designs: 1983—2007. Educ Train Autism Dev Disabilities. 2010;45(2):
187-202.

36. Spooner F, Knight V, Browder D, Smith B. Evidence based practices
for teaching academic skills to students with severe. Remedial Spec
Educ. 2012;33:374–87. doi:10.1177/0741932511421634.

37. Cooper JO, Heron TE, Heward WL. Applied behavior analysis.
2nd. Upper Saddle River (NJ): Pearson Merrill Prentice Hall; 2007.

38. Amon KL, Campbell A. Can Children with AD/HD Learn relaxa-
tion and breathing techniques through biofeedback video games?
Aust J Educ Dev Psychol. 2008;8:72–84.

39. Standen PJ, Brown DJ. Virtual reality in the rehabilitation of
people with intellectual disabilities: review. Cyberpsychology
& Behavior. 2005;8(3):272–82. doi:10.1089/cpb.2005.8.272.

40. Tanaka JW, Wolf JM, Klaiman C, Koenig K, Cockburn J,
Herlihy L, Schultz RT. Using computerized games to teach
face recognition skills to children with autism spectrum dis-
order: the Let’s Face It! program. J Psychol Psychiatry.
2010;51(8):944–52. doi:10.1111/j.1469-7610.2010.02258.x.

41. Weiss PL, Bialik P, Kizony R. Virtual reality provides leisure
time opportunities for young adults with physical and intel-
lectual disabilities. CyberPsychology Behav. 2003;6(3):335–42.
doi:10.1089/109493103322011650.

DEVELOPMENTAL NEUROREHABILITATION 337

Copyright of Developmental Neurorehabilitation is the property of Taylor & Francis Ltd and
its content may not be copied or emailed to multiple sites or posted to a listserv without the
copyright holder’s express written permission. However, users may print, download, or email
articles for individual use.