PT KinectionAn Augmented Reality Based At-Home PT Facilitator Using Xbox Kinect

EDIT 732 – Group 3

Background and Problem

Physical therapy can help patients avoid surgery, reduce chronic pain, prevent permanent damage and recurring injury through rehabilitative exercises and intervention

For best results, most patients are prescribed a home-exercise program in addition to office visits

Noncompliance can negatively impact patient recovery

As many as one third to two thirds of patients are noncompliant with exercises (Sluijs, Kok, and van der Zee, 1993)

Prior Analyses

User Data

Target AudiencePhysical therapy (PT) patients

Wide range of ages

Variety of reasons for therapy Congenital Degenerative Sports-related, etc.

As diverse as the general population

Learner NeedsAny time access

Visual representation of the exercise

Audible instructions

Immediate feedback

Learning/Technology Gaps

No confirmation of proper form

No realistic model of behavior

Lack of sufficient communication methods

Lack of motivation

Summary of Findings

Patients and physical therapists need to improve communication

Physical therapist need a way to hold patients accountable

Patients need immediate feedback

Patients are not motivated to complete their exercises

The target audience is relatively technology savvy

Personas – Patients

Personas – Patients

Personas – Physical Therapists

Objectives and Solution System Performance

Learning Objectives Motivate patients to regularly

practice prescribed at-home exercises

Educate patients between regular home exercise program and better recovery outcomes

Empower patients to be confident in their skill at performing the home exercise program

Improve patient accountability for exercises

Desired Performance of Solution System

Confirm proper form

Display realistic model behavior

Provide unconventional means of communication with PT

Motivate patient

Provide immediate visual and audible feedback

Be accessible any time

Proposed Solution System

Xbox 360 Kinect

A hybrid device that generates AR using a color (RGB) cameraA depth sensorA microphone arrayA tilt sensor

Skeletal tracking

Gesture Recognition

Benefits to Potential Patients

Simplifies communication with PT

Increases patients’ compliance

Motivates and encourages patients

Provides a realistic example of performance

Provides any-time access to resources

Benefits to Physical Therapist

Monitor patients’ progressSimplifies communication Ensures that patients have access to a realistic

example of performanceMotivates patients to perform prescribed

exercisesEncourages patients to overcome barriersEnsures patients’ continued visits to PT office

Interface Design

Scenarios

Flowchart

Wireframes

Message Center

Session Start Screen

Toolbar

Wireframes

Live Sessions Feedback View

CalendarCalendar

Mentor Demonstration

Mentor Demonstration

Note PadNote Pad

User SessionsUser Sessions

LocationLocation

MemoryMemory

SoundsSounds

LanguageLanguage

VisualVisual

NotificationsNotifications

Menu

Message Center

Message Center Message BoxMessage Box

Live HelpLive Help

System & Accessory Instructions

System & Accessory Instructions

PT Kinection Instructions

PT Kinection Instructions HelpHelpBegin a

SessionBegin a Session

SettingsSettings

Site Map

Memory

Location

Sound

Errors/Alarms [On/Off]

Messages/Email/

Smartphone

Language

Notifications

Visual

Push Notifications

Settings

Settings Sub-Menu

Audience Feedback

Preliminary Prototype Website/Survey

Documented FeedbackFeedback Response

Suggestion to reorder menu to “Begin Session”, “Settings”, “Message Center”, and “Help”.

Declined. Menu is in the order shown for a specific reason. We feel that the first thing the user would want to do on this screen is view messages and the last thing he/she would want to do is begin the session.

Suggestion to add additional descriptive information to controls.

Declined. We envision a user introduction that focuses on a description of uses and features. The user will have an option to view the overview anytime.

Suggestion to change color of control text so that it is easier to see against the white background.

Agreed. Text will be changed to a darker color within the defined color palette.

Suggestion to re-design logo. On-hold. Will consider for future versions.

Suggestion to change font color/weight on Exercise Education menu.

Agreed. Will change background to menu so that text is more legible.

Suggestion to add graphics to start screen.

Agreed. Will add collage of Physical Therapy photographs to start screen.

Layout/Visual Design Elements

General Design Notes

Exercise screens generally divided into three areas Mentor area in upper left “Active” work area Toolbar/menu at bottom

Elements can be “hidden” to reduce distractions and expanded when needed

Repeated elements (round buttons, rounded edges, etc) lend themselves toward sense of unity

Rounded edges for approachability, but mindful to not appear too juvenile

Blue tones chosen to indicate health or wellness

Colors differentiate PT Kinection from other Xbox system menu colors (green, red)

San Serif font for easier readability

New User/Select Profile

Start Screen

Session Toolbar

Exercise Demonstration

Correct Performance

Incorrect Performance

The Virtual PT

• Realistic model behavior• Immediate audible feedback• Motivational cues• Accessible anytime

Message Center

Message Center

Note Pad

Help Menu

Settings Menu

Recording Options

Exercise Education

Alerts & Warnings

Design Rationale & GuidelinesLearning Problem Rationale – Design Decision

Exercise is difficult to understand – steps are difficult to remember

• Live exercise demonstration• Immediate feedback• Communication options• Alerts & warnings• Recording options

Lack of communication

• Message center allows PT and patient to communicate directly through program

• Recording options allow PT to view patient’s form and progress

• Live and documented help options

Lack of motivation • Virtual PT provides immediate motivational feedback• Access to model behavior anytime, provides

motivation to complete the task, rather than waiting to speak to the PT

Briefing Report – Determined that there is a problem, defined barriers and drivers, and proposed an augmented reality-based solution system

Needs Analysis – Performed a thorough investigation of the target audience, determined task order, and further outlined the consideration of AR using Xbox Kinect as a solution for identified learning and technology gaps

Personas – Defined potential patients/users and their PT experience and the affects on health, and PT distribution of home exercise prescriptions, and user affects on health

Prototype – Developed a preliminary prototype, gathered comprehensive feedback from potential users, defined changes, and made corrections resulting in a final prototype that addresses the original learning problem, the defined learning objectives, technology and learning gaps, and potential users’ wants and needs

Summary

Project Team

HEATHER STAYROOK (hstayroo@gmu.edu)Mobile Course Interface Template Design Submissions Lead

KATE ORF (korf@gmu.edu) Performance Analysis Team Lead

MELISSA HESS (mhess7@gmu.edu) Persona/User Needs Analysis Team Lead

NOF AL-HAJ (nalhaj@gmu.edu) Needs Analysis Team Lead

SARAH AL-HAJ (alhajsarah@gmail.com) Design Prototype Presentation & Documentation Team Lead

References

Berardini, C. A. (2007, September 20). Xbox 360 and Xbox live facts and stats. Retrieved from http://news.teamxbox.com/xbox/14595/Xbox-360-and-Xbox-Live-Facts-and-Stats/

Reisinger, D. (2010, January 06). Xbox 360 is most-used game console, Nielsen says. Retrieved from http://news.cnet.com/8301-13506_3-10426467-17.html

Swinkels, I. C., Hart, D. L., Deutscher, D., van den Bosch, W. J., Dekker, J., de Bakker, D. H., van den Ende, C.H. (2008). Comparing patient characteristics and treatment processes in patients receiving physical therapy in the United States, Israel and the Netherlands: Cross sectional analyses of data from three clinical databases. BMC Health Services Research, 8(163). Doi: 10.1186/1472-6963-8-163