Date post: | 17-Feb-2017 |
Category: |
Design |
Upload: | nikolaj-aalbaek-jensen |
View: | 700 times |
Download: | 0 times |
• Why oh why?• Why should we measure usability?
• The boring stuff…• ISO 9241• Statistics 101
• So many choices…• System Usability Scale• Single Usability Metric• Standardized User Experience
Percentile Rank Questionnaire
Holy f… that is a lot of stuff• Heaven and Hell…
• The pros and cons of measuring usability
• Famous last words…• Summary• Recommended reading
Why oh why?Why should we measure usability?
http://www.skai.gr/files/1/jo/slide_225124_963376_free.jpg
• Numbers enables us to communicate UX results in a way that engages C-level management • Provides us with a way to measure the impact of changes
• Great way of introducing UX KPIs in your organisation, which makes it easier to champion your work and increases visibility
• Makes it easier to include usability requirements/benchmark in project specifications and public procurements
Why should we measure usability?
The boring stuff…The foundation
http://www.skai.gr/files/1/jo/slide_225124_963376_free.jpg
ISO 9241 – Ergonomics of human-system interaction
http://toplogos.ru/images/logo-iso.png
"The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use
ISO 9241 – Ergonomics of human-system interaction
ISO 9241 – Ergonomics of human-system interaction
task completion rateressources spent to solve a task subjective user satisfaction
EffectivenesEfficiencySatisfaction
Statistics 101
https://img0.etsystatic.com/100/0/6005823/il_fullxfull.832106216_e8ue.jpg
• Normal distributionhttps://www.youtube.com/watch?v=772_n15Ke9Q
• Standard deviationhttps://www.youtube.com/watch?v=nxKBfcwEulg
• Confidence intervalhttps://www.youtube.com/watch?v=tFWsuO9f74o
Statistics 101
http://kedoff.com.ua/uploads/posts/2014-11/1415786610_1.jpg
So many choices…A selection of methods
QUIS Questionnaire for User Interaction Satisfaction
SUMI Software Usability Measurement InventoryPSSUQ Post-study System Usability QuestionnaireSUS Software Usability ScaleASQ After-Scenario QuestionnaireSEQ Single Ease QuestionSMEQ Subjective Mental Effort QuestionER Expectation RatingsUME Usability Magnitude EstimationWAMMI
Website Analysis and Measurement Inventory
SUPR-Q
Standardized Universal Percentile Rank Questionnaire
CSUQ Computer System Usability QuestionnaireUSE Usefullness, Satisfaction, and Ease of useUMUX Usability Metric for User eXperienceHQ Hedonic QualityACSI American Customer Satisfaction IndexNPS Net Promoter ScoreCxPi Forrester Customer Experience IndexTAM Technology Acceptance ModelSUM Single Usability Metric
Standardized methodsQUIS Questionnaire for User Interaction
SatisfactionSUMI Software Usability Measurement InventoryPSSUQ Post-study System Usability QuestionnaireSUS Software Usability ScaleASQ After-Scenario QuestionnaireSEQ Single Ease QuestionSMEQ Subjective Mental Effort QuestionER Expectation RatingsUME Usability Magnitude EstimationWAMMI
Website Analysis and Measurement Inventory
SUPR-Q
Standardized Universal Percentile Rank Questionnaire
CSUQ Computer System Usability QuestionnaireUSE Usefullness, Satisfaction, and Ease of useUMUX Usability Metric for User eXperienceHQ Hedonic QualityACSI American Customer Satisfaction IndexNPS Net Promoter ScoreCxPi Forrester Customer Experience IndexTAM Technology Acceptance ModelSUM Single Usability Metric
• Created in 1986 by John Brooke
• 10 item post-session questionnaire • 5 negative biased statements• 5 positive biased statements• 5 point Likert scale
• Yields a score between 0 and 100 – average is 68
• Valid results with 12 responses
• The most reviewed and tested method available
• Free to use
System Usability Scale (SUS)
System Usability Scale (SUS)
http://uxpajournal.org/wp-content/uploads/pdf/JUS_Bangor_May2009.pdf
System Usability Scale (SUS)
http://www.measuringu.com/sus.php
Cons• You cannot use responses from individual items, only the total score• You only get an overall score• It cannot tell what or where something is wrong
Pros• Easy to use and administer• Good for tracking how changes impacts the usability• Easy to understand as a KPI• Can be used to assess when you should include users for qualitative feedback
System Usability Scale (SUS)
• SUS - A quick and dirty usability scalehttp://hell.meiert.org/core/pdf/sus.pdf
• Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale
http://uxpajournal.org/wp-content/uploads/pdf/JUS_Bangor_May2009.pdf
• SUS: A Retrospectivehttp://uxpajournal.org/wp-content/uploads/pdf/JUS_Brooke_February_2013.pdf
System Usability Scale (SUS)
One downTwo to go
https://cdn3.vox-cdn.com/thumbor/ST3W8A3Kum8HlwddNHoF07WWrhk=/0x0:1920x1080/1600x900/cdn0.vox-cdn.com/uploads/chorus_image/image/46147742/cute-success-kid-1920x1080.0.0.jpg
• Created in 2005 by Jeff Sauro and Erika Kindlund
• Based on ISO 9241 definition of usability • Task based measurement • Data driven results
• Free to use
Single Usability Metric (SUM)
Single Usability Metric (SUM)
https://www.measuringu.com/papers/HCII2005_sauro_kindlund-V9.pdf
Cons• Requires additional resources to administer• Can (almost) only be administered during usability testing sessions• Requires an unknown, and potentially relatively large, portion of data sets• Requires more knowledge about statistics than SUS
Pros• Task based data• Good at identifying specific problem areas• Closely related to ISO definition of usability• Very little subjective data
Single Usability Metric (SUM)
• A Method to Standardize Usability Metrics Into a Single Score https://www.measuringu.com/papers/p482-sauro.pdf
• Parts of the SUM: a case study of usability benchmarking using the SUM Metric https://autodeskresearch.com/publications/summetric
• 10 Things To Know About The Single Usability Metric (SUM) http://www.measuringu.com/blog/sum.php
Single Usability Metric (SUM)
Just one more
http://cdn.meme.am/images/5188389.jpg
• Created in 2011 by Jeff Sauro
• 13 item questionnaire
• Measures Usability, Trust & Credibility, Appearance, and Loyalty
• You also get both a SUS and a NPS score
• Results are comparable to a large database
• Pricing between $499 and $2999
Standardized User Experience Percentile Rank Questionnaire (SUPR-Q)
http://www.suprq.com/images/suprq-logo.png
Standardized User Experience Percentile Rank Questionnaire (SUPR-Q)
Cons• Full license (access to database) pricing starts at $1999• Is not as thoroughly tested and reviewed as other methods as it is a
relatively new method• Requires a bit more knowledge about statistics than SUM and SUS
Pros• Better insight than SUM and SUS, overall• Also yields a SUS and a NPS score• Results are comparable to large database
Standardized User Experience Percentile Rank Questionnaire (SUPR-Q)
• SUPR-Q: A Comprehensive Measure of the Quality of the Website User Experience
http://uxpajournal.org/supr-q-a-comprehensive-measure-of-the-quality-of-the-website-user-experience/
• The Essential Elements Of A Successful Websitehttp://www.measuringu.com/suprq.php
• The SUPR-Q websitehttp://www.suprq.com/
Standardized User Experience Percentile Rank Questionnaire (SUPR-Q)
http://foto-video-podarok.ru/foto/kostyum-cherta-na-novyij-god-7.jpg
Heaven and Hell…The pros and cons of measuring usability
• It’s just numbers – further insight requires other methods
• Unreliable, if used incorrectly
• Can only tell you if something is wrong, and not why
CONS
• Really good at measuring the impact of changes
• Great for communicating results - especially with C-level management
• Good way of having KPIs
• Cheap way of identifying/validating problem areas
• Great way of including UX requirements in project specifications / public procurements
PROS
Famous last words…
http://img.memecdn.com/famous-last-words_o_1527913.jpg
• Try out different methods and choose the one that works for you
• Use quantitative methods primarily to measure the impact of changes
• Champion UX KPIs in your organisation for improved visibility of your work
• Use benchmarks actively in project specifications
Summary
Recommended reading
www.measuringu.com
visma.dk/webevaluering
@brugervenlig linkedin.com/in/brugervenlig [email protected]
visma.dk/brugervenlighed
Feel free to reach out…