Date post: | 14-Nov-2014 |
Category: |
Technology |
Upload: | mandidavis |
View: | 956 times |
Download: | 0 times |
User Experience of Interactive Educational Systems
An iPad StudyHuman Factors in Information Design
Design and Usability CenterBentley University
Amanda Davis Vignesh KrubaiDiego Mendes
© Copyright
Agenda
• What We are Researching
• Technology and Methodology
• Qualitative Findings
• Quantitative Analysis
• What Worked Well
• Challenges / Recommendations
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
Education and Tablet Technology
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
• Pilot programs on using tablets in schools and higher education are underway
– NYC, Chicago, Virginia, California, Boston and Internationally– Intended to be used at home and school in place of textbooks
• Benefits include– Light-weight– Variety of applications– Interactive Learning
• Not many studies done to verify if students learn more, faster or better with tablets
What is happening now?
What We Are Researching– Engagement is a key part of learning effectively
• Is the increased use of technology affecting engagement?
– Do new or traditional technology engage students differently?
• Digital Textbook v. Paper Textbook
– Measuring engagement is complex because it involves biological and cognitive processes
– Requires an array of methods to be applied
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
Technology and Research Methods
Pearson Education E-bookPearson Education- One of the most innovative leaders in educationInkling Interactive Digital Textbook Technology- Same book in digital version and paper version- Allows for the same interaction (highlighting, post it notes etc…)
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
Digital Book Paper Book
Research Methods
What methods did we choose to apply?
Combination of different methods:
- Affectiva’s Q Sensor
- SMI Eye tracking
- Product Reaction Cards
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
Q Sensor: Measuring EDA
Q SensorCreated by Affectiva, a company founded by MIT Media Lab faculty and students
What is the Q-Sensor?A wireless biosensor that measures emotional arousal via skin conductance (Electro Dermal Activity).
High arousal = High User Engagement
Low arousal = Low User Engagement
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
Q Sensor: Research TechniqueChallenges in using the Q SensorThe data in the Q Sensor tells you there is some sort of engagement, but not which type of engagement. High engagement could mean:
Frustration Excitement Stress Delight
The Analysis:
1. Calculate average EDA Value
2. Calculate Peaks per minute
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
SMI Eye Tracking: Background
• Created by SMI (SensoMotoric Instruments)
• Advantageous for use on mobile or tablet devices
• Binocular tracking delivers eye tracking data over all distances
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
SMI Eye Tracking: Data Analysis
• Combined with Q Sensor data, SMI gives us the ability to try to pin point engagement
• Ability to analyze areas of interest across participants
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
Product Reaction Cards: Background
– Developed by Joey Benedek and Trish
Miner (Microsoft)
– A set of 118 words
• 60% positive
• 40% negative/neutral
– Easy to administer and easy to analyze
– Does not rely on a questionnaire or
rating scales
– Users don’t need to generate words
themselves
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
Product Reaction Cards: Technique Used
Process:
– Users were asked to circle 10 words from the list of 118 words
– This process was repeated after each object of study:
• Paper Textbook
• Digital Textbook
– We used the response cards to understand and categorize Affectiva’s Q Sensor
EDA/skin conductance data
Challenges:
– Real time reactions not available
– Participant misinterpretation of word meaning
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
What we found:Qualitative Results
Qualitative Findings
Participants• 10 undergraduate students who owned iPads and used the
internet for at least 10 hours per week
Common iPad Activities• Most common: accessing the web, checking email, gaming and
social networking• Additional: eBooks, music, course maintenance, movies and
looking at powerpoints during class
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
Which do you prefer and why?
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
50% of students preferred each
Digital Paper
Versus
What Do You Like About The Paper vs. The Digital Textbook?
Preferences on Paper Textbook
• Concerns about getting tired looking at the iPad screen & getting tired of swiping to flip pages
• Pictures were clear & didn't require enlarging to see
• Preferred for financial reasons: reselling & cheaper to purchase
• Allowed for more freedom and were more comfortable
• More reliable, as not dependent on batteries
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
What Do You Like About The Digital vs. the Paper Textbook?
Prefer Digital Textbook
• Easier to search for content
• Easier navigation
• Ability to access internet to do supplemental research
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
Quantitative Analysis
Quantitative FindingsWe plan to use the following information to help us in understanding user engagement:
- Electro-Dermal Activity Information- Magnitude = a measure of peak height- Area Under the Curve = represents peak volume- Peaks per Minute = is a measure of peak frequency
- Heatmaps of user engagement throughout session
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
Quantitative FindingsWith the addition of SMI eye tracking, we can:
- Pin point engagement areas and moments effectively
- Get the number of fixations on screen and paper textbooks across participants
- Create heat maps to determine points of engagement
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
What Worked Well
What Worked Well• Ability to observe:
– top-down (Cognitive Perception – post moment) - “I felt…”– bottom-up (Q Sensor Spikes – at the moment) - emotional reactions
• Affectiva- Easy to set up - Extremely detailed data about skin temperature changes, skin
conductance
• Use of SMI Glasses - Easy to set up- Eye tracking – precise view of how things worked- Data analysis capabilities are an advantage - Intuitive
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
Challenges
Challenges
- Coordinating the technology during the tests
- Synching issues- Video loses small frames (due to editing)
- Combining the data- 4 cameras (videos)- Affectiva data- SMI Data
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
Recommendations
Other Methodology Applications
• Areas of Application: - User Experience, Usability, Product Design, Market Research, Education Research,
Psychology Research, Visual Perception Research
• Product Design- Use it to find what emotionally engages people
• Usability- Use it to find out what frustrates people
• Prioritization of design features- Find what is more frustrating/exciting for the user
Caution: Labor intensive process to combine three methods (but doable)
© Copyright Diego Mendes Amanda Davis Vignesh Krubai
Questions?
Contact:Diego Mendes – [email protected]
Amanda Davis –[email protected] Vignesh Krubai – [email protected] © Copyright Diego Mendes Amanda Davis Vignesh Krubai
Thank you to our sponsors who are here and excited to meet you!
The Design and Usability Center