User Centered Design Process to Develop a Multi-modal Family Needs Assessment Tool

Chapter
THE USER-CENTERED DESIGN PROCESS TO
DEVELOPA MULTI-MODAL FAMILY NEEDS
ASSESSMENT TOOL
Arthi Krishnaswami*
CEO/ Founder, RyeCatcher
ABSTRACT
User-Centered design (UCD) is a set of processes utilized in the development of
products, services, experiences, tools, models, frameworks, systems or other artifacts
based on the needs, wants, goals and pain points of the end users.1
UCD uses iterative
cycles of prototype design and development—from low fidelity paper sketches to high
fidelity prototypes—with rounds of end user testing between prototypes to ensure the
products and services are designed with end user goals in mind at every stage [1, 2].
RyeCatcher has utilized a user-centered design process to build a multi-modal family
needs assessment survey tool—a data collection instrument to gather data from parents,
families or caregivers that support one or more student learners.
Keywords: user-center design, technology, data, support
INTRODUCTION
This chapter will begin by examining the set of techniques collectively termed user-
centered design (UCD). It will then use a case study approach to demonstrate the application
of the UCD process in the design and development of the RyeCatcher Family Needs
*
Corresponding Author Address: Arthi Krishnaswami. Email: arthi@ryecatcher.com.
1
For the purposes of this chapter, we will reference and define key components of a set of different design
processes that are collectively termed UCD, including, but not limited to User Experience, Goal-directed
design, Service design, Interaction design, and Human-Computer Interaction Design. As there continue to be
new formats, frameworks and design thinking systems in development, the methods identified in this chapter
should not be considered to be exhaustive, but rather, a broad overview with key elements called out as they
related to the design and development of the Needs Mapper tool in the case study.

Arthi Krishnaswami2
Assessment tool. It will outline the goals of the project, key milestones, user-research and
results, modifications, learnings, challenges, pivots and next steps in design and development.
It will describe the theoretical framework, research base and key references that informed the
initial development of the tool. It will describe the iterative cycles of design and development
during the first school year of its utilization, including findings and recommendations from
two rounds of usability testing, and the set of modifications made based on testing prior to the
second year of implementation of the tool.
This chapter is not a formal articulation of any specific version of the UCD process, but
shows how the mixture of methods that are collectively termed User-Centered Design can be
effectively utilized as a framework for the design of next generation technology tools in the
educational technology and social services sectors. Traditionally, products serving these fields
do not have the financial and design resources of traditional consumer products, but serve
similar size audiences. The UCD process is an effective, lightweight, and flexible process
when employed in the design and development of educational tools and resources, while
maintaining the fidelity of the implementation necessary for student learning.
USER-CENTERED DESIGN
The user-centered design (UCD) process is a framework for product development that
preferences the user’s needs, wants, behaviors, and feedback throughout the design process.
UCD is comprised of a collection of research, design, and product development methods that
are each informed by end user feedback to validate and verify assumptions, and provide
feedback and insight on the product. UCD uses iterative cycles of prototype design and
development—from low fidelity paper sketches to high fidelity prototypes—to build
products, tools, designs, or other artifacts based on the needs, wants, goals and pain points of
the end user. It is important to note that there are numerous variations of the process, and as
with all processes of invention, the product inventor and/or design team will interpret and
adapt the process to the specific needs of the particular inquiry [1, 2].
Adapted from Cooper, Deal/O’leary, Evenson.
Figure 1. The User-centered design process.
The UCD process is used to design and develop products in all media—online systems,
off-line interactive systems, print artifacts, consumer and physical (industrial) products, and
relies upon iterative cycles of prototype development and user testing. The process prioritizes

The User-Centered Design Process to Develop a Multi-Modal Family … 3
getting design artifacts into the hand of end users early and often, in order to reduce time
spent on solutions that do not directly align with the end users’ goals [3]. The process varies
by project or implementation due to time, definition of requirements and other considerations.
“The trick is to use these tools when appropriate and, more importantly, to use them at the
depth appropriate for the immediate problem you’re trying to solve for the business” [4].
Exploratory Research
Exploratory research is an expansive process with the goal of defining the problem and
understating the end users [2]. It is a deep exploration of the problem space that broadens the
possible frame of the design problem to enable effective problem definition. This phase
occurs at the outset of a project, and features the exploration of user goals, scenarios, pain
points, and needs [1, 2, 5]. Designers and researchers, more efficient and effective if they are
the same person(s) [1], review the existing tools, resources, design artifacts, processes, user
needs, and pain points, and identify a problem statement for the project centered around the
user [2, 5, 6].
A number of key activities comprise this phase of UCD process. This list is not intended
to be complete, but offers an overview of key elements from Cooper, Dubberly, Evenson,
O’Leary/Deal, Hanington, IDEO, and others [1-6].
• Competitive analysis
• Field studies and ethnographic observations of end users with existing tools and
technologies,
• Secondary research into related problems, issues, needs, and concerns
• Interviews with subject matter experts
• Participatory design sessions with end users
• Sketching and concept development
• Idea generation
Generative Research and Design
From problem definition, the research and design process turns to idea generation. The
generative design phase is when the designer/researcher synthesizes the users’ goals and
needs into personas, scenarios and requirements, and transforms those building blocks into
the product via sketches, wireframes, and prototypes [5, 1, 2].
This phase is where designer/researcher transforms the raw indeterminate material of the
problem—the users’ goals—into the product. Often there are an innumerable variety of
solutions and problem definitions, and the designer must bring all the pieces together—from
the indeterminate to the determinate. This relationship between determinacy and
indeterminacy is a key characteristic of the problems that are the subject matter of design
thinking [7].
There are a wide variety of artifacts created during this phase of the process. Idea
generation is an individual process and is performed differently by each practitioner. The

Arthi Krishnaswami4
research and design tools and methods are optimized to the specific elements of each design
problem. The wide array of methods and variation has yielded the development of a number
of toolkits, the most commonly referenced is IDEO’s Method Cards.
Some key techniques used in the generative design/research phase of the process are:
• Sketching
• Personas
• Storyboards
• Wireframes
• Prototypes
Generative research and design yields successively higher fidelity prototypes and designs
until the product is developed. In software development processes, later versions of the
prototypes are often developed by engineers, rather than the design team, and may not be
considered part of the design process [1, 2]. While the variability of software design from
manufacturing is important to understand, for the purposes of this chapter, we will utilize a
model that is a synthesis of physical and digital product design. Ultimately, the visual design
team will hand off the product development to the software engineers, who build the product
from the design specification.
Evaluative Research
Once the product is in development, the design and research activities focus on testing
the product in action with real end users. The designer/researcher conducts user testing of the
product—often in the format of usability, or think-aloud, testing. This method is based on
ethnographic methods and is where an end user is asked to perform tasks that simulate real-
world user scenarios, and provide immediate feedback on the product’s content, architecture,
design, functionality, and other aspects [5-7].
The goal of evaluative research is to refine and modify the product in order to optimize it
for the end user. This type of testing can occur at any level of fidelity. It is effective for low-
fidelity sketches, as well as fully functional software. Increasing fidelity enables less directed
testing situations. Modern screen sharing technology has made remote testing a favored
approach for live software to reduce cost of travel, and gain access to end users in their
natural environment in a non-intrusive way [5-7].
Evaluative research findings inform product updates and modifications. If the changes
required, updates to user needs, or systematic modifications are sufficiently large, an
exploratory or generative design research activity may be initiated, and the iterative UCD
cycle would begin again [1, 5-7].

CASE STUDY: FAMILY NEEDS ASSESSMENT
The Problem, Definitions, and Conditions in the Environment
This case study focuses the development of the RyeCatcher Family Needs Mapper. This
tool was designed and developed as a response to needs identified by student support
personnel at a charter network that provides wraparound services for its students and families.
The first step in product design is defining the problem. The 2014-15 school year was the
second year of providing integrated or wraparound services to students in a middle-size
charter network of about 3000 K-6 grade students in a mid-size southern city that serves
primarily students of color, over 90% of whom were receiving free or reduced price lunches,
a common indicator of poverty. To better support students and families in the community, the
Student Support team at the school identified a need for a tool to capture information from
students and families about their needs for supports and services during their initial contact
with the school at registration.
The Question
How might a needs assessment tool be used when students register for school to capture a
picture of the student/household’s situation and begin populating the wraparound process?
After the question was posed by student support team members, the designers worked to
determine the viability and launch plan for the tool. The entire process took approximately 60
days. The decision to create a needs assessment survey happened on the 01 June, and the
product launched to users on the 31 of July.
Why Is UCD the Right Method for This Type of Project
Because of the nature of the data collected, the short timeline for the development of the
tool, and the variety of perspectives necessary to develop the tool, the optimal product
development process was that of user-centered design.
1. Short time span for product design, development and launch. UCD works for
projects of all time spans—day-long data jams and hackathons, to many month or
yearlong projects— but is particularly effective for short-cycle projects, as it uses
iterative product design and development cycles that can be as short as a few hours.
Because this project needed to be complete within a matter of weeks, the time for
research, strategy, data modeling and iteration needed to occur quickly and in
parallel.
2. Low-fidelity sketches to begin the process. In this case, there was no time to spend on
high fidelity prototypes or concepts. The team needed to move from content to
engineering as quickly as possible, and research, sketching, and prototype design
occurred in rapid, iterative sequence over a period of two weeks. The process began

Arthi Krishnaswami6
with a set of paper sketches and moved into engineering within a few days, as soon
as the types of data entry were determined.
3. Limit time on bad ideas. Frequent user research, even for a short project, ensures that
design and development effort are limited to items the user truly needs. On this
project, the entire effort was to respond to an urgent user request, so frequent, almost
daily, user feedback was an integral part of the process.
NEEDS MAPPER UCD PROCESS
Due to the highly compressed nature of the project, and lack of ability to adjust the
deadline—the product had to be ready for registration—the process was condensed into the
minimally necessary steps. The Needs Mapper design process followed the sequence—
research synthesis, expert interviews, paper prototype design, user testing, digital prototype
design and development, launch, user testing, adaptation and updates.
The product team researched, designed and developed the tool in 60 days. It used a short
cycle design process, an amalgam of a variety of UCD methods—goal-directed design, lean
UX, service design, Agile software design, to name a few of the sources—to create content,
design the interface, and develop the software [1, 2, 4]. The process yielded a minimum
viable product in less than 60 days, and the Family Needs Mapper launched to a first-day
audience of over 300 families.
Theoretical Framework, Theory of Change, Research Base
Definitions
Wraparound Services: A school-based wraparound strategy promotes student academic
achievement by developing, securing and coordinating support that targets the student’s non-
academic barriers to learning. It goes beyond typical mental health services by focusing on
relationship building between the student and his/her family [8, 9]. Wraparound services have
been operationalized in a variety of ways—integrated services, community schools, and
others [10, 8, 11, 12]. In this chapter, we will refer to any and all operationalized forms as
wraparound services.
Needs Assessment: Rouda and Kusy (1995) define “needs assessment” as a systematic
exploration of the way things are and the way they should be [13]. Needs assessments can be
delivered on paper, as interviews, digitally and in other forms.
Turnaround School: A turnaround school is one where the academic outcomes of the
student body are well below the norms, and the administration of a public school is “turned
over” to a new entity, often a charter or other alternative school structure. These environments
generally feature a high rate of staff turnover, school culture fragmentation, lack of trust of
the new staff, and transitional students as they decide whether to stay in the school or find an
alternative school. Because of these factors, staff reported that it was hard to get to know
families at the beginning of the school year.

THE RESEARCH BASE FOR THE PRODUCT:
LITERATURE REVIEW
Wraparound works for two reasons—it promotes a connection between needs, services
and supports, and it empowers students and families [14]. Practitioner experience and a sound
research base support that a school-based wraparound strategy, an emerging field of practice,
promotes progress in attendance, math, ELA and reading achievement, among other key
academic indicators [15]. It can be used for students with special needs, or those without
support plans in place [16].
Though research has shown wraparound to be effective at improving student outcomes, it
has a limited established theoretical framework, which yields a high degree of variability in
its operationalized formats and procedural guidelines [17]. This lack of consistency is also
present in the software, services, tools and resources available for the implementation of
wraparound. This gap is particularly great in the area of social and emotional learning.
The Needs Mapper tool was developed to address the needs of the wraparound process,
and the gaps in the existing software, tools and resources, beginning with the initial meeting
with parents. This improvement in engagement and tailoring services to needs, has a profound
impact on reducing no-shows of students and families at initial meetings with providers, a key
issue in the provision of support services by out-of-school providers [14].
The broader RyeCatcher platform has been built to support the “positive change spiral”
brought about by the wraparound process [14], and to address gaps in the education and social
services software space, particularly related to engaging families and students in collaborative
processes to improve retention at services [18, 19]. The overall functionality promotes shared
decision making in order to improve adherence to goals [20]. The behavior planning tools
rely shared goal-setting for interventions which improves outcomes and performance [20, 18,
14]. And the platform supports creating shared expectations between the providers of services
and parents [19, 18, 21].
THE PRODUCT CONCEPT
Based on the research and the needs identified by the end users, the designers developed
the concept of the Family Needs Mapper—a needs assessment survey tool where families
provide information about their needs and the needs of their children in a digital or paper
format. The designers envisioned a survey to be completed by the family or as an intake
interview guide for initial meetings between support staff and families.
The tool would be developed based on a research base from social services, education,
user-centered design and other fields. The tool facilitates the process of getting to know a
student and family, connecting them to appropriate services a quick, easy, and secure process.
This process of alignment promotes a perception of relevance by tailoring services to needs of
family and student [22, 19, 18].
Families and schools would have a starting point for conversations, and positive forward
movement would happen for families in the first few weeks of school—a critical time in the
school year. Supporting and improving engagement promotes retention in services and
supports [22, 19, 18].

Arthi Krishnaswami8
While a survey is simple data collection instrument, the range of data that the team hoped
to collect included sensitive and personal data about student needs, behaviors, skills,
academic areas for improvement, high risk activities, family attributes, home and living
conditions, English as a Second Language (ESL) status, college matriculation patterns, and
family needs. Because the data being collected fell into a wide range of categories, and the
type of information a family disclosed was to be personal and potentially sensitive, the design
of the survey instrument, instructional content, support materials, and reports needed to
reflect the privacy, security, and personal consideration that the information being gathered
required. A lack of consideration for the implications of the data collected, the format of the
tool, the language used in the tool, and the communications around the tool would impede
adoption and yield a sense of uncertainty when using the tool.
Exploratory Research
The team conducted source research synthesis, card sorting, and subject-matter expert
interviews during the research phase. Due to the compressed timeline, the research phase was
confined to a 3-week period. Exploratory and generative research and design occurred in
rapid sequence, with some overlap.
NEEDS ASSESSMENT DATA COLLECTION METHODS
For the Needs Mapper, the goal of the project was to create an online instrument that
mapped to an existing set of paper-based processes, some of which captured over 100 data
points about a student and families’ needs. The project began with a content audit, and review
of comparable tools and resources.
The design team worked with a wraparound services scholar and subject matter expert, to
compile a broad set of sample needs assessment questions and inventories. Over 15 different
needs assessment instruments were reviewed as a part of the process. The source materials
took on a number of formats including a set of needs assessments, student support services
referral process flows and forms, social service intake interview guides, behavior inventories,
and social worker client intake forms.2
The team reviewed paper based tools, online/digital case management platforms, and any
available online or software-based needs assessment tools. Product design relies upon input
from end users, so alongside the research into assessment formats and tools, the designers
conducted interviews with subject matter experts (SMEs)—school leaders, social workers,
student support team members, behavior interventionists, psychologists, researchers,
designers of learning tools, and others—to develop an understanding of the content, context,
and process of needs assessments. Among the questions asked to the subject matter experts,
the team reviewed the initial set of content buckets with each of the subject experts to
2
A sample of selected, representative source materials: School referral forms for parents, student support services;
external agency services, behavior plans, FBA forms, and many others School Needs Assessment tools from
regions and cities with comparable demographic populations. UCLA Mental Health In-School Needs
Assessment Template; Social Services Needs Assessment; Social Services Intake interview questions; ACES
survey.

determine if any needed to be modified, updated or removed from the list. The experts also
provided insight into the relative benefits and drawbacks of the needs assessments references
and formats.
From the source materials, the team developed an inventory of all of the data points
collected on the various assessment forms, and conducted a card sorting exercise—putting
content items into subject specific buckets— to develop a draft set of consolidated content
buckets which the assessment questions and wraparound supports could fall into. The team
developed a content inventory from compilation of source materials and research into the
needs, behaviors, questions, themes, and patterns that could possibly be identified with a
needs assessment instrument. This initial compilation included well over 100 questions, far
too many for the limited time and attention span of a parent on registration day [23].
Initial list of buckets:
• Tutoring and mentoring services 
• Mental health and substance abuse services 
• Case management 
• Parent support and education 
• Transportation and financial support 
• Health services 
• Food, clothing, housing, and employment assistance 
• Individual, group, and community activities and services 
• Gang diversion 
• Extended learning and enrichment 
A major factor in evaluating a student’s needs is asking the right questions in the right
context during student intake. Feedback and insights from the experts enabled reducing the
initial set of questions based on the themes required to explore, the level of personal
information that would be appropriate for this type of initial outreach to a family, time
constraints, and applicability of school-based services to the needs identified.
SYNTHESIS OD DATA
After the source research and interviews with subject matter experts, the team
synthesized findings into a draft set of questions, and a set of design strategies that guided the
design and development of the application.
There were over 30 questions in the initial draft that fell into original set of buckets. As
the survey or interview needed to be completed at the time of registration, the time for
completion was intended to be under 5 minutes, and the target number of questions was to be
in the range of Miller’s cognitive processing magic number of 5+/- 2 [24]. The challenge was
to reduce the number of overall questions, while capturing all of the data required.

Figure 2. Needs Mapper workflow process.

The key buckets that needed to be captured included academics, motivation, family
needs, conditions at home and exposure to Adverse Childhood Experiences [25]. The ACEs
framework has been among the most researched frameworks in wraparound, and has been
used at the statewide level in Washington State to drive its wraparound approach [25]. The
basic theory is that the more ACEs a child is exposed to, academic and socio-emotional issues
increase. While the ACES survey was used as a guiding framework, the Mapper reworked all
of the questions to avoid overly sensitive questions, which the team determined were not
appropriate to include in a needs assessment for Registration Day. The approach was that the
clearer the survey can identify the presence of ACEs in a child’s life (especially more than
one), the more effective interventions and preventative efforts can be.
The team used a variety of guiding questions to create the draft list from the source list of
questions:
• Which data points, at a minimum, need to be collected to gather effective
information?
• How many questions can parent or families be expected to answer within 5 or 10
minutes?
• What information will parents/families feel uncertain about providing?
• How can the question framing, format or language minimize errors,
misinterpretation, or bad data?
• What data can we collect that is based on sound research and tied to positive
outcomes?
• Does this data enable measuring baselines and outcomes?
Upon realizing the challenge of time, question number and data, the team reverse
engineered the questions by compiling a list of all of the required data points, and reverse
engineering the minimum number of questions needed to gather the data. The initial draft list
of questions was reduced to a shorter set of 12 questions prior to beginning the design phase.
These questions captured 60 data points, and could be answered in between 5-10 minutes in
informal testing with subject matter experts.
Beyond the survey questions, the project needed to develop a set of communications for
the school staff that would be administering the survey, and the parents and families that
would be completing the survey. Clear framing of the goals and benefits of the survey would
be important steps to gain buy-in and support for the tool and its usage [18]. The survey was
designed to be entirely opt-in, so the instructions and explanations had to be clear, effective
and promote the value to the parents, families and students of completing the survey.
Finally, as the product launch date neared and communications were to go to families and
school staff, the tool needed a name. The tool was named the Needs Mapper as it was
determined to be descriptive and evocative.

Arthi Krishnaswami12
GENERATIVE RESEARCH, DESIGN AND DEVELOPMENT
The design strategy for the needs assessment application was based on the secondary
research base, and exploratory research activities. The main themes that informed the design
strategy were to:
• enable capturing a lot of data in a clear and simple way,
• optimize usability for a varied set of audiences to minimize data entry errors, and
• design for a limited attention span and question fatigue during a registration process
already filled with forms.
Figure 3. Multiple-choice question.
Enable Capturing a Lot of Data in a Clear and Simple Way
There were two main aspects to implementing this strategy—engineering flexibility and
design simplicity. Because of the volume of data being captured, the system needed to be
designed and engineered to be flexible to support an expanding set of users and use cases.
From an engineering perspective, the needs assessment was to be built as a survey that
exported data into any type of receiving application—data visualization, analysis,

presentation, and others. It was also built to be able to add and remove questions and
responses, modify the data entry mechanisms, and add new data entry mechanisms over time,
as needed. While there was a lot of flexibility from the engineering perspective, the end user
should experience a simple and easy interface while entering the data, or would abandon the
experience.
Once the initial draft of questions was developed, it was determined that there were 3
potential types of data entry—multiple-choice questions, single or multi-select, and frequency
scale questions. Multiple-choice questions do not offer a great deal of design variability so the
end user has little cognitive load when faced with that question type. A slider control was
implemented for the entry of frequency data to present the users with a more fun and
engaging way to enter the data, while being a familiar data-entry mechanism. Users are also
increasingly comfortable with the online form functionality of radio buttons, where the user
must choose only one option, and check-boxes, where multiple selections are possible. The
design of the data entry was to be clear and easy to read, to minimize distractions and visual
load.
Once the data entry mechanisms were identified and form elements designed, the team
reviewed the prototype with the subject matter experts. There were no major issues identified,
and the prototype moved directly to the engineers. The short timeline demanded that design
and engineering occur together in a live application development environment. Because the
interface was simple, this was the most effective and efficient way to develop this application.
Optimize Usability for a Varied Set of Audiences
Parents, families and school staff have varied levels of technical proficiency and reading
comprehension. As such, the system needed to be designed to optimize usability to support
the wide range of end user audiences. There were a number of key objectives from a usability
perspective. The readability of the interface was paramount as some questions had over 15
possible responses, despite the team’s best efforts to minimize the number of questions. The
instructions had to be simple and embedded into the form itself to avoid clicking through
many additional screens. The typography needed to be the appropriate size and style for
legibility and readability on laptop, tablet or phone.
Beyond the visual design of the page, the design of the survey, itself, needed to help to
minimize errors and provide the end user with a feeling of confidence that his/her data has
been captured properly, and to fix any errors. To reduce errors and provide immediate
feedback to the end user, a summary screen that showed all responses in a single list was
designed to be the second to last page of the survey. Users could modify responses to any of
the questions, and to ensure accuracy prior to submission.
To make it simple for a wide variety of audiences, the Needs Mapper was to be
completed by one family-at-a-time. It was to be completed by parents and families who did
not have accounts on the platform, but were given access to a one-time survey workflow. At
the end of the survey, the user was thanked for his/her time and asked to return the computer
to the administrator. The parent/family user was unable to get back to the platform without
knowing the one-time URL that the admin had access to.

Design for a Limited Attention Span and Question Fatigue during a
Registration Process Already Filled with Forms
The final consideration was that users have limited attention spans, and do not like to fill
out forms [23]. To exacerbate this issue, one use case for this form was that it would be filled
out during registration day for school. Registration day is when families come to register the
student for school, and is already filled with many forms to fill out. The families often have to
go to the session during lunch or another short time period, so they have limited time for the
activity. The Needs Mapper was designed to be an optional part of the registration day, so it
had to be short, easy, and engaging, or users would not spend the time.
Anecdotal and consumer research on survey response behavior indicates that asking
single questions is a more conversational tone is an effective method for engaging users in
longer personal interest surveys. As the Mapper fit into this category, it was determined that
each question would be presented individually. This had the additional benefit of reducing
visual distractions and cognitive load for each question. The team leveraged the repetitive
activity of answering the questions by grouping similar types of questions together.
Figure 4. Answer summary page.

The final additions to the survey were a question that asked parents’ families to identify
their own needs, and a page where parents/families could provide email or phone to opt-in to
receive information based on their responses. The parent contact information page would be
displayed after the submission of the survey itself, to allow the end user to submit the Needs
Mapper data anonymously. All design strategies came together in a fully developed
application the day before launch.
LAUNCH: SYSTEM IMPLEMENTATION
Day One: Launch: Rocky Takeoff, Smooth Once Airborne
The Needs Mapper launched on the School District’s registration day, July 29, 2014. The
launch was a success resulting in over 350 submissions on registration day. The critical factor
in the success of the launch was having support—both from the engineers to ensure the tool
was working as intended and on the ground for end users. As it turned out, both forms of
support were necessary for the launch to be a success.
On launch day, the engineers monitored the application by watching the database entries,
user activity, and server logs. Within a few minutes of launch, the engineers identified a data
saving error, which was impacting a few questions in the middle of the survey. On-site
support developed an alternate form of the survey in case the data saving issue could not be
corrected. On-site support also developed a printed version of the form to capture data for the
fields that were not saving. Within 30 minutes, the problem was identified and a fix for the
bug had been deployed. While there was a minimal amount of data loss, the issue was
remedied quickly, and during the remainder of the day over 350 submissions were collected
without error.
Beyond the technical issues, on-site school staff identified that some users were skipping
the questions that had many responses or the ones that had responses more complicated than
the frequency rating scales. Staff performed sensitive observations and support activities,
determined that reading comprehension was the main issue, and decided to conduct intake
interviews using the Mapper as a guide. This practice had the additional unintended benefit of
enabling the support staff to get to know the student and family quickly and personally during
the registration day, which was one of the main purposes and goals of developing the Needs
Mapper. The support staff and on-site team had conducted the interviews with participant
families in between 3-5 minutes. Together, technical support from the software engineers and
on-site support from the implementation team are the two components of a successful product
launch.
METHOD
User-Testing
The team conducted two rounds of think aloud user testing. The first round of testing was
on launch day. The second round was mid-semester to prepare modified version for second

semester new pilots. The first round of testing focused on critical issues—ones that would
impede data collection or usage—and the second on operational and administrative issues.
User Testing 1: Findings and Recommendations/Product Modifications
The team used feedback from lightweight usability sessions on the launch day to make a
limited number of design, format, process, content, and software modifications and
improvements. Limiting the number of changes after each round of testing ensures the
updates and modifications are the most critical, and that the impact of the changes will be
clear [7].
Optional Questions
All of the questions on the Needs Mapper are optional. The survey was designed to be a
non-threatening tool, so requiring any question to be answered, and requiring a name of the
person filling out the survey could create anxiety or impede adoption or question completion.
Thus, the decision was made to have all of the questions be optional. After the initial
implementation of the Mapper, the team conducted an analysis of the response patterns. Most
users answered most questions—over 80% of questions had some response. The questions
with the most missing responses were those that related to attendance and truancy—how
often is your child late, how often does he/she miss school, and for what reasons. During user
testing, researchers probed about why questions were skipped, and the reason given by the
vast majority of participants was fear of disciplinary action for the child. After testing, a set of
simple instructions were added to the beginning of the Mapper that clearly state that the
responses will not be used for disciplinary purposes. The optional indication was also made
visibly stronger—larger type size and in red.
Language
During the first day of the tool being used in schools, the student support teams at the
schools identified a variation in the level of completion of the forms between families. Some
parents completed the survey quickly, but did not spend much time on each individual
question. Other families spent a longer time on the survey. Some families had difficulty with
the technology of the survey. RyeCatcher made a number of on-site adaptations with the help
of the student support team insights. Families had widely varying reading abilities. To avoid
comprehension issues, the survey was used as an interview guide with counselors, social
workers or other school staff conducting interviews. The time for completing the survey went
down from 5 mins. to 3 mins. when used as an interview guide.

Figure 5. Frequency Slider.
Usability
Two major usability difficulties were identified on the first day of usage—the touchpad
mouse was difficult for end users, and the slider scale for frequency responses, while
engaging for end users, was not easy to use as a data entry mechanism. Each issue was
addressed with a modification to the system. A standalone, wired mouse was made available
for every laptop used in the Needs Mapper data collection process, and the frequency scale
slider was replaced with a traditional set of buttons to indicate response.
Paper Version
Some families were not able to complete the survey during an interview or at the
computer station during registration. These families needed to be able to complete the survey
at a later time, so the one-page paper Mapper was developed. It was critical to make the
survey a single page, front-and-back, to avoid extra paper usage and loss of pages during
transit to and from school.

Content
There were 3 major modification to content in the first round of updates—changing
school to college, changing the sequence and presentation of the absent from school question,
and making it clearer that all questions were optional. When the initial reporting of the data
occurred, the team realized that the question about first to attend had been worded first in
family to go to school rather than college. This was fixed as soon as the error was identified.
Data from prior to the fix was removed from further analysis for that one question. Staff at
the schools impacted were made aware of the issue, and worked to correct it manually. The
second change was to change the sequence and presentation of the absent from school
question to be a twostep question rather than a single question with a conditional second step
that made that one question seem different and more serious than all of the others. Due to the
perceived disciplinary association of the question, users reported being worried about the
consequences of answering the question, and did not respond.
CSV Download
The final modification based on launch day usability was adding an aggregate data
download option. Because the engineers were monitoring the data, they had already built a
rudimentary version of a data download. School staff provided some requirements and use
cases for the data from the Mapper, and a download to a csv, comma-separated value file, was
developed. The file contained every response, for every student, with a time and date stamp
and parent/family contact information, if provided.
The school teams were able to download and create reports from the data immediately.
The implementation team developed a summary report from the raw data, and presented it to
school leaders. Feedback on the format was incorporated, and the report was delivered to
school leaders each month during the school year.
User Testing 2: Operational Improvements
The second round of testing occurred in November, and results informed changes that
were designed and developed between semesters. These updates were mostly focused on
making the tool accessible to larger audiences, and as such, were largely operational.
Email to Parents: Individual and in Bulk
While the first version of the Mapper was designed to be completed one-at-a-time, school
users at the existing pilot schools and potential new sites, asked for the tool to be delivered in
flexible formats—by email, to multiple families at the same time, and on mobile and tablet
devices. Each of these updates was developed between semesters and launched to the pilot
audiences. The text of the email to be sent alongside the Mapper was preset with legally
required and introductory text, with the ability to add an additional note in an open text field.

Create New Student or Use an Existing Student
The original survey required that the student already be a part of the platform database or
the student had to be added from a separate section prior to be able to complete a survey. This
additional step impeded access for users, so a simple one page, 8 field form was developed to
enable adding the student to the database. To keep the process simple, a single landing page
enables the user to select from all options—search for and update the record of an existing
student, add a new student, or send an email survey.
Paper Survey Design Improvements
The initial version of the paper survey was completed by over 100 families in its first
month of implementation. Themes emerged when entering data from the forms about the
usability and design affordances of the paper survey.
A number of families did not complete the student name field. It was on the top of the
front of the form where the field label, student name, was built into the instructions by being
on the blank line that the parent was to fill out.
As is clear in the illustration, the label was not prominent or clear. The form was updated
to include a clearly defined box around the student name field, and the field appears on both
sides of the page. Instructions were too light when copied or printed on some printers. In the
update, instructions were called out as white text on a dark background.
The second round of usability testing improved key operational aspects of the Needs
Mapper critical to wider adoption of the application. The major improvements were bulk
email delivery to enable the survey to be shared with larger audiences, simplifying the
creation of new student profiles a part of the Needs Mapping process, and improving the
usability of the paper survey.
“ Student Name ”
Figure 6. Student Name Field.
CONCLUSION
Refinement: What We Learned
The final phase of the user centered design process is incorporating feedback into the
product, refining the product, and beginning the process all over again. New feature requests
are a constant from end users. It is the designer’s role to prioritize the features to maximize
the impact of any change or update.
By the end of the first year of its use, the Mapper had been completed by over 650
families in the pilot district. The first year of implementation surfaced many insights for the
product development team. A number of major themes emerged related to content, design and

functionality of the experience. The high-level findings are summarized below with
references to articles and secondary research that supports the conclusions.
CONTENT
Engagement and Access
The content of the questionnaire was based on research with end users and subject-matter
experts. Beyond a sound research base, engagement and empowerment of parents and
families is critical to student success [26]. Making the tool available in multiple formats to
support access by all families is critical to that engagement and access. To promote alignment
of needs and services, it is important to capture information from families early and often so
the tool may need to be delivered more than once per year [14].
Spanish Language Translation
To support equitable access, the Mapper will be translated into native Spanish. The
translation will be immediately available to the end user, i.e. not a request for alternate
format. The translation, as the English Language version, must be reviewed for readability
and comprehensibility by a linguistic expert. The complexity of some of the language for
non-native speakers may demand additional supports [27].
High School Needs Mapper
The mapper was designed for families to complete on behalf of themselves and their
students. For older students in high school, there is a need to empower self-efficacy in goal-
setting and planning behavior to improve outcomes and adherence to plans [14]. To promote
high school student self-efficacy and engagement, student support teams requested a version
of the Mapper specifically designed for high school students.
Question Modules
End users from schools that were not demographically similar to the pilot district were
invited to provide feedback on the tool. As the questions had been tailored to the specific
needs of an at-risk community, there were some questions that were not applicable in other
settings. Those end users asked for variations in questions based on demographics. The
design solution would be to develop modules of questions for different types of schools that
aligned to grades served, regional patterns, and demographic characteristics.

DESIGN
Paper survey
The design of the paper version of the survey needed to be improved, in particular, the
information hierarchy and prominence/contrast of key elements in the form design needed to
promote the completion of important data, e.g. name, phone, email, format.
Smart Defaults
The ability to customize and modify the Mapper aligns with end user needs and usage
patterns, but too many choices can yield decision paralysis [23]. To support usage and data
integrity, the system should suggest optimal length and format of questions alongside the
ability to modify and adapt the standard survey instrument. The smart default setting of the
survey is optimized for the 3-minute interview, 60 data points, and data entry via paper,
browser-based web site, or phone/tablet application to enable the widest, most equitable
access.
FUNCTIONALITY
Actionable Data
Users wanted to see actionable data displayed instantly upon data entry or completion of
the survey. They also requested user-specific dashboards for families, students, and schools to
promote next steps, and connecting with service providers aligned with needs.
Customization: Choice without Overload
Users requested flexibility in the content of the questionnaire. The solution would be to
enable users to customize the order and sequence of the questions. The design of the interface
would be optimized to minimize choice overload [23]. Question customization options will
enable end user control of total amount of data being collected, and different audiences have
the tolerance and ability to answer different lengths and complexities of surveys. End users
know their audiences and are in the best position to optimize the experience within a
framework of what is recommended.
Self-Service On-Boarding for New Schools
As is the case with many new products, the first iteration relies on manual processing to
implement the tool. Upon interacting with the system, end users requested there be a self-

service on-boarding process for new schools to send the Mapper to families. They wanted the
process to require no support from the Mapper platform provider.
Alternate Formats
Users requested that the tool be delivered in SMS and mobile app to promote utilization
and access by a broader range of end users.
FUTURE RESEARCH: NEXT STEPS IN DESIGN
AND DEVELOPMENT
For the 2015-16 school year, the survey was distributed more broadly, expanding to over
2000 families. The Mapper continued to be developed through a user-centered design process.
Feedback from the field informed every update. Designers worked closely with engineers and
end users to ensure changes, modifications and updates in the second year of design and
development were an extension of features based on requests from usability testing and
observational research throughout the first year of implementation.
Key updates based on first-year user feedback included:
• In-application descriptive reporting for students, cohort, and schools,
• A mobile version for iPhone and Android,
• updates to the content of questions—more subjects, simplifying language, adding
question about format for communication, and
• SMS notifications to improve access and engagement with families.
The Needs Mapper survey tool and associated RyeCatcher functionality supports the
goals, operationalization, and implementation of the wraparound process. The designers will
utilize the UCD process to identify additional user needs, pain points, and opportunities that
can be addressed by the Needs Mapper, and will work with subject-matter experts to align the
product with research from the social services and education fields.
The UCD process allows tools and resources to evolve and change shape, while the core
goals and models stay in place. As evidenced in this chapter, UCD is an excellent process for
gathering large amounts of data, in order to best support at-risk students and families in the
applied educational setting. Continued work with the UCD process is needed in order to
determine its outreach capabilities and to monitor and measure outcomes for users.
ACKNOWLEDGEMENTS
Thank you to Dr. Susan Elswick, Dr. Richard Fulmer, Charlie Firestone, Ashley Deal,
Raelynn O’Leary, Senchel Matthews, Sonnet, Maya, Mom and Dad for reading, listening,
providing feedback, sharing insights, helping with references, and encouragement throughout.

REFERENCES
[1] Cooper, A., Reimann, R., and Cronin, D. (2007) About Face 3: The Essentials of
Interaction Design. Wiley Publishing, Indiana.
[2] Evenson, S. and Dubberly, H. (2009) “Designing for Service: Creating an experience
advantage.” Introduction to Service Engineering. Wiley Publishing, Indiana.
https://www.pbis.org/school/tertiary-level/ wraparound?text-only
[3] R. Buchanan, “Wicked Problems in Design Thinking,” Design Issues 8, no. 2 (1992):
14-19.
[4] Goethelf, Jeff. (2013) Lean UX. O’Reilly Media, Sebastopol, CA, USA.
[5] Hanington, B. and Martin, B. (2012) Universal Methods of Design: 100 Ways to
Research Complex Problems, Develop Innovative Ideas, and Design Effective
Solutions. Rockport Publishers, Beverly, MA.
[6] Hackos, J & Reddish, J. (1998) User and task analysis for interface design. John Wiley
& Sons, Inc. New York, NY, USA.
[7] Krug, S. (2005) Don’t Make Me Think: A Common Sense Approach to the Web. New
Riders Publishing, Thousand Oaks, Ca, USA.
[8] Burchard, J. D., Bruns, E. J., & Burchard, S. N. (2002). The wraparound approach. In
B. J. Burns & K. Hoagwood (Eds.), Community treatment for youth: Evidence-based
interventions for severe emotional and behavioral disorders (pp. 69-90). New York:
Oxford University Press.
[9] Eber, L. (2003). The art and science of wraparound: Completing the continuum of
schoolwide behavioral support. Bloomington, IN: Forum on Education at Indiana
University.
[10] Bruns, E. J., Suter, J. C., Force, M. M., & Burchard, J. D. (2005). Adherence to
wraparound principles and association with outcomes. Journal of Child and Family
Studies, 14, 521–534.
[11] Burns, B. J., Schoenwald, S. K., Burchard, J. D., Faw, L., & Santos, A. B. (2000).
Comprehensive community-based interventions for youth with severe emotional
disorders: Multisystemic therapy and the wraparound process. Journal of Child and
Family Studies, 9, 283-314.
[12] Miles, P., Bruns, E. J., Osher, T. W., Walker, J. S., & National Wraparound Initiative
Advisory Group. (2006). The wraparound process user's guide: A handbook for
families. Portland, OR: National Wraparound Initiative, Research and Training Center
on Family Support and Children’s Mental Health, Portland State University.
[13] The wraparound content is extracted from ‘Chapter 27: Completing the Continuum of
School-wide Positive Behavior Support: Wraparound as a Tertiary-Level Intervention’
by Eber, L., Hyde, K., Rose, J., Breen, K., McDonald, D., & Lewandowski, H. (in
press) in W. Sailor, G. Dunlap, G. Sugai, R. Horner, (Eds.) Handbook of Positive
Behavior.
[14] Walker, J. S. (2008). How, and why, does wraparound work: A theory of change. In E.
J. Bruns & J. S. Walker (Eds.), The resource guide to wraparound. Portland, OR:
National Wraparound Initiative, Research and Training Center for Family Support and
Children’s Mental Health.

[15] Moore, Kristen and Carol Emig. Integrated Student Supports: A Summary of the
Evidence Base for Policymakers. Child Trends (2014).
[16] Scott, T., & Eber, L. (2003). Functional assessment and wraparound as systemic school
processes: Primary, secondary, and tertiary systems examples. Journal of Positive
Behavior Supports, 5, 131–143.
[17] Walker, J. S., & Shutte, K. M. (2004). Practice and process in wraparound teamwork.
Journal of Emotional and Behavioral Disorders, 12(3), 182-192.
[18] Morrissey-Kane, E., & Prinz, R. J. (1999). Engagement in child and adolescent
treatment: The role of parental cognitions. Clinical Child and Family Review, 2, 183-
198.
[19] Kazdin, A. E. (1996). Dropping out of child psychotherapy: Issues for research and
implications for practice. Clinical Child Psychology and Psychiatry, 1(1), 133-156.
[20] Maddux, J. E. (2002). Self-efficacy. In C. R. Snyder & S. J. Lopez (Eds.), Handbook of
positive psychology (pp. 277-287). New York: Oxford University Press.
[21] Spoth, R., & Redmond, C. (2000). Research on family engagement in preventive
interventions: Toward improved use of scientific findings in primary prevention
practice. The Journal of Primary Prevention, 21, 267-284.
[22] Walker, J. S., Bruns, E. J., Rast, J., VanDenBerg, J. D., Osher, T. W., Koroloff, N., et
al. (2004). Phases and activities of the wraparound process. Portland, OR: National
Wraparound Initiative, Research and Training Center on Family Support and
Children’s Mental Health, Portland State University.
[23] Thaler, R. H., & Sunstein, C. R. (2009). Nudge: Improving decisions about health,
wealth, and happiness. Penguin Books, New York City, NY.
[24] Miller, G. A. (1956). “The magical number seven, plus or minus two: Some limits on
our capacity for processing information”. Psychological Review 63 (2): 81–97.
[25] ACEs Study [https://www.cdc.gov/violenceprevention/acestudy/ about_ace.html] Last
updated April 1, 2016.
[26] Kazdin, A. E., Holland, L., & Crowley, M. (1997). Family experience of barriers to
treatment and premature termination from child therapy. Journal of Consulting and
Clinical Psychology, 65, 453-463.
[27] Zarcadoolas, C., Boyer, J., Krishnaswami, L.A., & Rotherberg, A. (2007) “GIS Maps
to Communicate Emergency Preparedness: How Useable Are They for Inner City
Residents?” Journal of Homeland Security and Emergency Management: Vol. 4 : Iss.
3, Article 16. DOI: 10.2202/1547-7355.1367.
RR

User Centered Design Process to Develop a Multi-modal Family Needs Assessment Tool

More Related Content

What's hot (19)

Similar to User Centered Design Process to Develop a Multi-modal Family Needs Assessment Tool (20)

Recently uploaded (20)

User Centered Design Process to Develop a Multi-modal Family Needs Assessment Tool