Date post: | 27-Mar-2015 |
Category: |
Documents |
Upload: | samantha-simpson |
View: | 218 times |
Download: | 0 times |
© 2
008
The
Mat
hWor
ks, I
nc.
® ®
Using Instant Messaging to Usability Test an API
Rachel Cobleigh
Donna Cooper
2
® ®
Our task: evaluate the usability of an API
API = Application Programming Interface Used for writing software code Programmers (our users) have different
expectations than point-and-click users
This API was in the design phase, so no live prototype was available
3
® ®
Our product: MATLAB®
4
® ®
Example command-line interaction
5
® ®
Challenge #1: No GUI
6
® ®
Partial Solution: Use the documentation
?
7
® ®
Another solution: Language Norming
Describe a task and ask the participant to suggest an API name
Provide a list of API names and ask the participant to choose the one that best describes a task
Show the participant an API name and ask what they think it does
Provide help documentation and observe adoption of API names
Describe a task and ask the participant to suggest an API name
Provide a list of API names and ask the participant to choose the one that best describes a task
Show the participant an API name and ask what they think it does
8
® ®
How does API testing compare to GUI testing?
What's different Users will need to rely more heavily on documentation Prepare at least a first pass at error and warning messages and
bare-bones help documentation
What’s the same Recruit as usual Prepare a scenario and user tasks to structure the test session Have a developer act as the “computer” Do walkthroughs before the first real usability test
9
® ®
Challenge #2: Realistic API interaction when there’s no live prototype
Paper prototype? Awkward participant experience Difficult to manage test results
10
® ®
Challenge #2 continued...
Word processor? Participant experience still awkward Pre-written “computer” responses could
bias test results Difficult to manage test results
?? Undefined function or variable ‘c’.
Warning: site vector should be arrow Vector with integer elements
??? Error using ==> imshow>preParseInputs at 340IMSHOW expected at least 1 input argument but was called instead with 0 input arguments.
>> d = zeros(0.5)Warning: site vector should be arrow vector with integer elements
d = [ ]
d = zeros(c)?? Undefined function or variable ‘c’.
11
® ®
Solution: Use Instant Messaging (IM) to simulate the command-line interface!
IM interaction closely resembles command-line interaction
Blank screen interface
Text-based input and responses
Immediate interactive responses to input
12
® ®
Example of an IM prototype interaction
13
® ®
Set up with a local participant
14
® ®
Set up with a remote participant
15
® ®
Mechanics of a successful test session
Customize Use copy-and-paste Have responses ready Agree on a protocol Provide help and/or documentation Ensure that “computer” realistically simulates the
command-line
[>>] try looking at other methods[>>] command mostly worked
16
® ®
Observations & tips for success
IM interactions closely resemble command-line interactions
Participants quickly understood what was expected of them because IM conversations “felt like” the command-line interface
Pre-defined text made the IM session flow quickly so that the illusion of really interacting with the software could be maintained
Delays while “computer” is typing are good opportunities to ask participants questions
17
® ®
Benefits of using IM
Improved participant experience
Easier to manage test results
Transcripts are easy to save and disseminate
18
® ®
Limitations of using IM
It's not a perfect simulation
It can have a slightly different look-and-feel
There can be security issues if testing proprietary designs over the internet
Avoid using personal IM accounts: minimize outside interruptions
Hey hon, can u stop for milk on the way home?
19
® ®
Conclusions
We improved the usability of our API
The techniques were simple and effective
20
® ®
Acknowledgements
We are grateful for the input given by the following people at The MathWorks, without whom this work would not have been possible:
Mary Beth Rettger Amy Kidd Jennifer Lymneos Jeff Goodwin Ellen Mangan
21
® ®
Related work
Steven Clarke has published work on API usability testing Clarke, Steven, Measuring API Usability, Dr. Dobb’s Journal
Special Windows/.NET Supplement, May 2004, pp. S6-S9 Focus is put on analyzing test data in terms of the Cognitive
Dimensions framework http://blogs.msdn.com/stevencl/
Testing was done on a live prototype, in an integrated development environment (i.e., a specialized text editor)
Laura Chessman introduced Language Norming to The MathWorks