Carmen Gerea Valeria Herskovic
November 2015
Measuring User Experience in Latin America: An Exploratory Survey.
• Introduction • Relative work • Our previous related work and motivation • Study • Conclusions
Index
• UX - phenomenon, field of study and field of practice [7] • Challenges of developing HCI and UX communities [8] • Many UX methods [10] • UX - still not clearly defined nor understood [4]
Introduction Some concerns about UX
• Finland, USA, UK, Netherlands, 2008 / 2009: Country of residence and other sociocultural factors could explain some of the variability of responses [5].
• 2014, 97% participants from Europe, North America, Asia. Survey was in English, French and German [3].
• 2014, UX Measurement Attitudes Survey (UXMAS) [4].
Related work at an international level Small participation from LATAM
Our previous related work and motivation [Master´s Degree Project. Nov 2013 - Jan 2014]
Research questions about UX practitioners
RQ1: “what are the most important difficulties they encounter in their UX projects”.
RQ2: “what measurement and validation methods do they use when designing or launching a new system”.
Previous related work and motivation [Master´s Degree Project. Nov 2013 - Jan 2014]
Interview [6] Survey [34]
Research questions about UX practitioners
RQ1: “what are the most important difficulties they encounter in their UX projects”.
RQ2: “what measurement and validation methods do they use when designing or launching a new system”.
Previous related work and motivation [Master´s Degree Project. Nov 203 - Jan 2014]
Findings - Insights
• Lack of a common language between UX professionals and clients or colleagues from other areas of the company.
• Lack of time on a regularly basis: to correctly define requirements, to test at different stages and to communicate with the team.
• Issues of recruiting users for testing: It is difficult to recruit users on a frequent basis in order to assure user testing is part of the process.
• Uncertainty regarding design and development decisions during a system development and after launching.
Study June - July 2015
Structure of the questionnaire Based on adapted UXMAS + new background questions Introduction explaining the interest of the research topic, followed by:
Questions • Background • Measurability of UX • Deep discussion
Online study June - July 2015
112 participated 40 fully responded
Structure of the questionnaire Based on adapted UXMAS + new background questions (32 q) Introduction explaining the interest of the research topic, followed by:
Questions • Background • Measurability of UX • Deep discussion Since then:
146 participated 42 fully responded
Participants Country of origin
6%4%4%
7%
11%67%
Chile Argentina
Peru
Mexico Spain Other*
*Brasil (1), Colombia (2), Costa Rica (1), Nicaragua (1), Venezuela (2)
Our study UXMAS
40% less than 30 years old 6% more than 40 years old
16% 41%
1) UX as a field in LATAM is even younger than in US / Europe 2) More junior and middle managers vs UXMAS (high
executives ?)
Participants Background
Some hypothesis
Our study UXMAS
87% practitioners 13% researchers
51% 59%
1) UXMAS was largely distributed at academic events 2) Lot of LATAM practitioners do not consider themselves as
a researchers
Participants Background
Some hypothesis
Participants Activities associated with UX most commonly done
Activity MentionsPrototyping 74
Interaction design 69Visual design 62
Usability Testing 60Personas / Archetypes 58
Web analytics 56Card sorting 45
Heuristic analysis 44Programming 42
Field research (ethnography) 30Eye-tracking 13No response 14
Future research should address what they say they do versus what they really do.
Measuring UX “What a measure is”…
ActionCompare, evaluate, analyze, validate, monitor, calculate, quantify
Input / OutputParameters, data, results, performance, knowledge
Tools
Processes
36 mentions
11 mentions
3 mentions
3 mentions
Measuring UX
When it should be measured Is important for…
77% during an interaction 72% after an interaction 31% before an interaction
Our study allowed several options. UXMAS allowed just one.
91% design and evaluation
Measuring UX Experiential qualities they mention
Meaningful, personally
encountered events [2]
that are memorized and
used to construct individual mental
references.
“”
Confidence (3) Satisfaction (3) Motivation (2) Efficiency (2)
Entertainment (2)
79% of those who mentioned one think it is measurable
Only 25% could mention a theoretical and
methodological argument. 28% mentioned a practical
one.
Measuring UX Some thoughts
Subjectivism, most of the time measures can have different interpretations.
“Experiential quality
can not be validated scientifically.
“
Designed experiences have to be validated in order to know if
users feel comfortable with the system or they hate it.
“
“
Measuring UX is difficult and
complex.
Measuring UX Practical issues
Our study UXMAS50% Cost of the UX measurement process
26% Time
Lack of a representative number of users for studies
Lack of knowledge and experience in UX measurement
“Lack of knowledge in exploiting feedback of UX for future system development”
Struggling with basic
issues vs more specific
ones?
Conclusions and discussion
Research issues • Length of the questionnaire. • The complexity of the vocabulary had an impact on
(1) Rate of response (2) Content of the response (some aspects considered methodological in UXMAS were mentioned as practical ones in our study)
As an insight, comparability is not always trivial.
Conclusions and discussion
Future work • More participants from several countries • Understand how cultural background and industry or
research maturity may play a role in the state of the art of the practice of UX.
• Search for methods and tools to solve specific issues related to the Latin American context.
• Develop specific education and research programs aimed to take UX field of study and research to a more mature level.
References
Icons from thenounproject.com.
[1] Hassenzahl, M., Tractinsky, N. 2006. User experience - a research agenda, Behaviour & Information Technology, 25:2, 91-97.
[2] Hassenzahl, M. 2014. User Experience and Experience Design. In: Soegaard, Mads and Dam, Rikke Friis (eds.). The Encyclopedia of Human-Computer Interaction, 2nd Ed.. Aarhus, Denmark: The Interaction Design Foundation: https://www.interactiondesign.org/encyclopedia/user_experience_and_experience_d esign.html
[3] Lallemand, C., Gronier, G. and Koenig, V. 2015. User experience: A concept without consensus? Exploring practitioners´s perspectives through an international survey. Computers in Human Behavior 43 (2015), 35-48.
[4] Law, E., van Schaik, P. and Roto, V. 2014. Attitudes towards user experience (UX) measurement. Int. J. Human-Computer Studies 72 (2014), 526-541.
[5] Law, E., Roto, V. and Hassenzahl, M. 2009. Understanding, Scoping and Defining User eXperience: A Survey Approach. In Proceedings of CHI 2009 (Boston, Massachusets, USA, April 7th, 2009).
[6] McGrath, J. Groups: Interaction and Performance. PrenticeHall, 1984.
[7] Roto, V., Law, E., Vermeeren A. and Hoonhout, J. 2011. User experience white paper. Bringing clarity to the concept of user experience: http://www.allaboutux.org/files/UXWhitePaper.pdf (last access, July 2015)
[8] Sánchez, A., Furtado, E. and Vivas, 2014. N. Challenges for Establishing a Latin American Community in HCI/UX, In CSCW Workshop (2014) http://researcher.watson.ibm.com/researcher/files/brropaula/challenges-CSCW-VF-1.pdf (last access, July 2015)
[9] Tokkonen, E., and Saariluoma, P. 2013. How User Experience is Understood? In Proceedings of Science and Information Conference 2013 (London, UK, October 07-09, 2013).
[10] Vermeeren, A., Law, E., Roto, V., Obrist, M., Hoonhout, J., and Väänänen-Vainio-Mattila, K.: User experience evaluation methods: current state and development needs. Proceedings of the 6th Nordic Conference on Human Computer Interaction: Extending Boundaries (Reykjavik, Iceland, 2010), 521-530.
Thank you! Gracias. Merci. Obrigado. Mulțumesc.
Download the paper: dl.acm.org/citation.cfm?id=2824914 Stay in contact: @carmenFR The survey: https://es.surveymonkey.com/r/MLF79CV