Date post: | 13-Nov-2014 |
Category: |
Documents |
Upload: | slimportfolio |
View: | 2,491 times |
Download: | 0 times |
Web browser usability test
Lim Zhen Ling Samantha
Ng Yan Ting
Wong Tak Wee
Google chrome• Developed by Google• Released in September 2008• Cool features
– Minimalistic design– Rearrange tabs in different
windows by drag and drop– No need to delete history:
Incognito mode– Google Suggest: All-in-one
address bar– No need for download manager:
All-in-one browsing tab– Top 9 web sites: Convenient
access to your recent/favourite web pages
Google chrome• Developed by Google• Released in September 2008• Cool features
– Minimalistic design– Rearrange tabs in different
windows by drag and drop– Privacy: Incognito mode– Google Suggest: All-in-one
search, history and address bar – No need for download manager:
All-in-one browsing tab– Top 9 web sites: Convenient
access to your recent/favourite web pages
– Highlighted textboxes
web browser usability testing
• Objectives • Pre-testing assessment
– Assessment criteria– Cognitive walkthrough– Pilot test
• Tasks selection: Final revision• Procedure• Results • Limitations • Improvements• Conclusion
Our objectives
• Improve usability of Google Chrome web browser for current and future users
• Create positive user experience
Our objectives
• Improve usability of Google Chrome web browser for current and future users
• Create positive user experience
• Objectives of HCI research (Shneiderman, 1987)↓ time of learning↓ rate of errors↑ retention over time↑ user satisfaction
Web browser assessment criteria
• Web performance• Security• User interface design/Usability
Web browser assessment criteria
• Performance• Security• User interface design/Usability
– Intuitiveness – Convenience– Accessibility
Cognitive walkthrough: Intuitiveness
Decreased learning time and visibility of functions
• Consistency• Informative feedback• Direct manipulation of
system• Interface real estate
management• Overall look and feel
Cognitive walkthrough: Intuitiveness
Decreased learning time and visibility of functions• Consistency ×
– With older, more familiar web browsers
Inactive mode
Active mode
√ Minimalist approach overloading of function buttons × violate intuitiveness
Google Chrome
Internet explorer
Firefox
√ Minimalist approach overloading of function buttons × violate intuitiveness
Cognitive walkthrough: Intuitiveness
Decreased learning time and visibility of functions• Consistency
– With older, more familiar web browsers
To open/close bookmarks bar: Right click
√ direct manipulation with the system inconsistent layout and function × low ‘visibility’ violate intuitiveness
Cognitive walkthrough: Intuitiveness
Cognitive walkthrough: Convenience
• Shortcuts• Easy reversal of actions• Reduce short term memory
load• Error prevention and
solving• User locus of control
Cognitive walkthrough: Convenience• Shortcuts• Easy reversal of actions• Reduce short term memory
load ו Error prevention and
solving• User locus of control
Minimalist design reorganisation of functions require learning ST memory load increases
Cognitive walkthrough: Accessibility• Important or commonly
used functions – Find “Help”
Cognitive walkthrough: Accessibility• Important or commonly
used functions
Minimalist design remove redundant coding search time increasesMinimalist design remove redundant coding area of access decreases time needed for selection increases
Cognitive walkthrough: Accessibility• Important or commonly
used functions • Bookmarks bar
Small area of access violate Fitts’Law time needed for selection increases
To close bookmarks bar: Right click
To open bookmarks bar: Right click
Tasks Selection : Final
Cognitive walkthrough pilot testing finalise tasks
Task 1
Go to http://www.hardwarezone.com, and stop the page before it finishes loading.
Press the Esc button on the keyboardClick on the button
× Low visibility
Task 1
Close the bookmarks bar, and after you have done so, show the bookmarks bar again.
Right-click on the bookmarks bar to close right click on the slit to open × Fitt’s Law
Go to “Tools”
Task 2
Task 2
Task 2
Make and export a bookmark
Drag URL address to bookmarks bar go to “Bookmarks manager” go to “Tools” click on “Export bookmarks”
Click on star icon to bookmarkGo to “Tools” click on “Export bookmarks”
× accessibility - muted suggests inactivity; low visibility
Task 3
Task 3
Task 3
Save a web page on to the desktop
Select “Document” “Save page as…” Check for file on desktop
Task 4
Task 4
Internet ExplorerMozilla Firefox
Locate the “Help” option.
Go to “Tools” Select “Help”
Task 5
Task 5
Participant Profile & Procedures
Participant Profile• 6 participants
• 3 males & 3 females• 1st Time/Non-Google
Chrome Users• NTU Undergraduates• 20~26 years old • Basic level of computer
and internet expertise
Tasks identified are generally common functions usedby participant group.
Procedures
• Venue• Tutorial Room 111 and 113• Quiet, controlled environment
Venue
Venue
Equipment
• Acer Aspire 4710• Google Chrome Version 1.0• MORAE 3• Debut Video Capture Software• Sony Cyber-shot Dsc-s600 • www.hardwarezone.com• Pre/Post-test Questionnaires• Video Release Form
Observation Methods• MORAE 3
– Screen Activity, Mouse Clicks• Debut Video Capture Software• Notetaker’s Guide
– Navigation Pathways, Facial Expressions, Success/Failure etc.• Questionnaire/Feedback Form
– Likert Scale– Open-ended
feedback
RESULTS
Findings
Task 1 Task 2 Task 3 Task 4 Task 5
M1 2 2 1 2 2
M2 2 2 0 2 2
M3 2 0 0 2 2
F1 2 0 0 2 2
F2 1 1 2 1 2
F3 1 2 0 2 2
*Success rating: 0 – Not completed, 1 – Completed with difficulty or help, 2 – Easily completed
Success ratings
Findings
• Task 3 > Task 2 > Task 1 > Task 4 > Task 5
• Highest completion rate: Task 5– All participants completed it with ease
• Lowest completion rate: Task 3– Only 2 participants completed it
• Only 1 of them completed it with ease• The other completed with some difficulty or help
FindingsDifficulty ratings
Findings
• Emotional responses provide information about the tasks too– Whether MATCH with self-reported difficulty ratings
AND success rates
• Participants were confused by some of the tasks– Especially Task 2– And also Task 1 and 3
Results analysis
• By:– Task design – how it affects the performance and
success ratings
– User expectations – how user experience and knowledge affects performance and success ratings
Results analysis
• Task design– Certain tasks inherently more complicated –
demands a longer action sequence• E.g. Tasks 2 & 3
– Other tasks are much simpler – shorter action sequence of 1 or 2 steps only – better performance
• E.g. Tasks 1, 4, & 5
Participant ID
Task 1 Task 2 Task 3 Task 4 Task 5
Mean success rating
1.666667 1.166667 0.5 1.833333 2
Results analysis
Mean success ratings
Results analysis
• User expectations– Mental models developed from use of their
familiar web browsers• E.g. Internet Explorer, Mozilla Firefox, Safari
– Mental models would then affect performance of tasks on Google Chrome
• The smaller the discrepancy between Google Chrome and familiar web browser, the better the performance, the lower the difficulty rating
Results analysisParticipants’ familiar web browsers and Difficulty ratings
Difficulty rating
ID Familiar web browser Task 1 Task 2 Task 3 Task 4 Task 5 Mean
M1 Firefox 1 2 4 1 2 2
M2 IE 1 2 3 1 1 1.6
M3 IE 3 5 5 2 2 3.4
F1 Safari 3 5 5 2 3 3.6
F2 Firefox 2 2 1 3 1 1.8
F3 IE 3 4 5 1 1 2.1667
Average – Firefox 1.5 2 2.5 2 1.5 1.9
Average – IE 2.3 3.7 4.3 1.3 1.3 2.6
P.S. Safari is left out of the comparison due to its much lower % of use among web browser users.
Results analysis
• Task 1: Stop a web page from loading– Location of X button
• Firefox - Left• IE and Google Chrome - Right
– Consistency for IE users: similar location• IE users can more readily identify and navigate around
that area for “Stop’ option
Results analysis
• Task 1: Stop a web page from loading– Difference between IE and Google Chrome
• X button doesn’t double up as the load button, coloured in red (redundancy effect)
– IE users perform better?• No!• 2 participants who completed the task with difficulty or
help– 1 used Firefox, 1 used IE
Results analysis
• Task 1: Stop a web page from loading– IE closer to Google Chrome
• IE users SHOULD give lower difficulty ratings• IE users’ average: 2.3• Firefox users’ average: 1.5• Firefox users give lower difficulty ratings INSTEAD!
Results analysis
• Task 1: Stop a web page from loading– IE users:
• NO better performance• NO lower difficulty rating
– Explained by:• Pressing “Esc” button can also stop a page from loading
– Hence IE users who employ that action sequence, instead of clicking on “Stop” button, would not be expected to perform better because of consistency in design layout
Results analysis
• Task 2: Open / close bookmarks bar– Bookmarks bar
• Firefox: right-click on top options toolbar, and check/uncheck “Bookmarks toolbar”
• IE: no bookmarks bar/toolbar!
Results analysis
• Task 2: Open / close bookmarks bar– Firefox users are expected to have an advantage
since of the presence of similar “Bookmarks toolbar”
• No! - 2 users of Firefox and 2 users of IE completed the task
– 1 of the users of Firefox completed it with difficulty or help.
Results analysis
• Task 2: Open / close bookmarks bar– Firefox closer to Google Chrome
• Firefox users SHOULD give lower difficulty rating• Task 2
– Firefox users’ average: 2 – IE users’ average: 3.7
• Firefox users INDEED give lower difficulty ratings
Results analysis
• Task 2: Open / close bookmarks bar– Firefox users:
• Similar performance as IE users• Lower difficulty ratings
– Explained by:• Similar layout design – consistency
– Firefox users recognize Google Chrome’s bookmarks bar and its associated functions more easily
Results analysis
• Task 3: Make and export bookmark– Action sequence
• Firefox– Make bookmark: click on muted star– Export bookmark: go to “Bookmarks” option ”Organize
Bookmarks” ”Import and backup” “Export HTML”
• IE– Make bookmark: click on yellow star with a plus sign “Add
to favourites”– Export bookmark: click on yellow star with a plus sign
“Import and Export…”
Results analysis
• Task 3: Make and export bookmark– Google Chrome: action sequence similar to Firefox’s
• Make bookmark: click on muted star• Export bookmark: go to “Tools” button go to “Bookmarks
manager” highlight/click on bookmark click on “Tools” option select “Export bookmarks”
– Only 2 participants who managed to complete the task • Both Firefox users
Results analysis
• Task 3: Make and export bookmark– Firefox closer to Google Chrome
• Firefox users SHOULD give lower difficulty rating• Task 3
– Firefox users’ average: 2.5 – IE users’ average: 4.3
• Firefox users INDEED give lower difficulty ratings
Results analysis
• Task 3: Make and export bookmark– Firefox users:
• Better performance• Lower difficulty ratings
– Explained by:• Similar action sequence – consistency
– Firefox users hence more familiar with task
What are the things you liked about Google Chrome?
“Sleek look, nice aesthetics..”
“Stylish and simple in design…”
“Looks sleek and tidy…”
“…look sleek and clean..”
“Faster speed…”
What are the things that you did NOT like about Google Chrome?
“Slightly cumbersome in its usability, especially with regards to bookmarks..”
“Too difficult to find various functions…”
“No gadgets, add-ons..”
“…icon not coloured…”
Feedback
• What people liked:– Mainly looks, aesthetics– Level of emotion: Visceral
• What people disliked:– Navigation, usability– Level of emotion: Behavioral
Would you recommend Google Chrome to your friends?
0
0.5
1
1.5
2
2.5
3
Yes No Maybe
Usability criteria
• Intuitiveness– Consistency– Feedback– Overall look and feel
• Convenience:– Shortcuts– Easy reversal of actions – Reduce cognitive load– User locus of control: error prevention and feedback
• Accessibility
Scope and severity ratings
• Scope– Local problems– Global problems
• Severity 1 - prevents completion of a task
2 - causes significant delays in completing a task
3 - causes minor usability problems, but users can complete the task
4 - minor annoyance - does not significantly impact usability, but should be corrected if time allows.
Tasks Usability Problems No. Of participants affected
Severity Scope
2, 3, 4, 5 Participants face difficulty in completing tasks associated with options classified under “Documents” and “Tools”
4 1 Global
2, 3, 4, 5 Classification of task options was not intuitive i.e. participants could not find the export bookmark option and the functions for bookmarks bar under bookmarks manager
4 1 Global
3 Participants could not find the export bookmarks option
4 1 Local
1, 2, 3, 4, 5 Web browser window closes without warning 3 2/4* Global
2 Participants were unaware that bookmarks bar can be closed and opened via right click
6 2 Local
3 Participants could not identify bookmarking option
4 3 Local
1 Participants could not find the stop button without prompt
2 3 Local
2 Participants could not identify the bookmarks bar without prompt
6 3 Local
1 - prevents completion of a task2 - causes significant delays in completing a task3 - causes minor usability problems, but users can complete the task4 - minor annoyance - does not significantly impact usability, but should be corrected if time allows.* Depending on the task user was performing
RECOMMENDATIONS
4 out of 6 of the participants felt that Google Chrome’s minimalist design (larger web pages) improved their web browsing experience
retain its minimalist designImprove on its existing features based on feedback and current usability problems
Reorganisation of options
Reorganisation of options
Direct manipulation of bookmarks bar
Redundancy coding for “Documents” and “Tools” option
Google Chrome
Open these the next time I use Google Chrome
Do not show me this dialog again
Alert window
Bookmarks bar button/options
Stop button
CONFOUNDS AND LIMITATIONS
Limitations/suggested changes
• Participant selection– Small sample size– Population stereotypes
• Asians• University undergraduates
– Not representative of all users
• Practice effect• Testing environment and
procedure– Discrepancy with the real world
• Computer system• Usage conditions
– Presence of test administrator– Obtrusive equipment
• Repeat usability testing– in a different population
– reorder task sequence– Include practice tests prior to
actual testing
– Naturalistic environment
Web browser evaluation
User performance test Feature test Web compatibility test Configuration test Security test
√
CONCLUSION
Google Chrome with 3D
• Version tested: early stage of development
• Minimalist design aimed at the everyday user– Differs from Chromium– Relatively sound product
• Make adjustments to accommodate users accustomed to browser giants IE and Firefox
• Google Chrome team: make adjustments as we go along– Google Chrome Blog– Google Chrome Beta– Google Chrome with 3D