�1

User%Acceptance%Report

Prepared%by%Christina%Hough%April%6,%2015

Summary%

In order to measure the usability and usefulness of our ideas for the Musicking website, we designed and conducted two tests. One tested the usefulness of four proposed features, using an open-ended, in-person activity to get participants to think about and discuss each one. The other tested the usability of our proposed information architecture by asking participants to navigate a tree in order to complete tasks.

Guided by the results of these tests, we would make the following recommendations: 1. After a new user signs up, give them a brief “tour” of the user dashboard with an

overview of what the main menu categories offer. 2. From the main event page, use questions to guide users to the appropriate Virtual

Event. 3. Consider whether any event categories could be merged. 4. Eliminate acronyms from all menu categories. 5. Prioritize some version of the hour-tracking feature and the Virtual Events

feature.

Methodology!

Tree test: This electronic exercise was designed to test our proposed information architecture. Using Optimal Workshop’s Treejack feature, we reproduced the information architecture in our working wireframes, and described three tasks. Participants were asked to navigate through the tree until they found where they believed they would be able to complete the task. They could expand each top level to view its branches, and return to a previous level in order to explore a different branch. The system tracked their path and the time it took to complete each task. Participants were also invited to provide comments on the tasks at the end of the exercise.

This exercise was completed by a 9 participants. Most had some experience learning music, though a few did not, as we did not strictly control who received the link. It was important to us that anyone be able to navigate the website, including beginning music learners, so prior experience with music was not essential for the purposes of this exercise.

�2

The tasks: 1. You’ve just practiced. You want to record what you practiced, and for how long.

(We wanted them to find “Log a practice.”) 2. You want to participate in an online session where everyone posts a recording and

then gives one another feedback. (We wanted them to find “Virtual WIP workshops.”)

3. You’re planning on buying a guitar, and want to know what kind you should get. (We wanted them to find “Plucked strings” under “Forums.”)

The tree: • Home

• Forums • Piano and other keyboards • Plucked string instruments • Bowed string instruments • Wind instruments • Musical styles • Teaching music • Composing and songwriting

• Groups • Search groups • Create a group

• Virtual Events • Virtual WIP workshops • Virtual open mic nights • Virtual masterclasses

• My Music Timeline • Log a practice • Create a timeline post • Update profile

Sort and discuss proposed features: This open-ended activity was meant to generate discussion and help us gage the level of interest in specific features. We printed

�3

descriptions of four features on cards, and asked participants to order them from most inclined to use to least inclined to use. We then asked them to explain their reasoning.

We approached 4 people who were active in music to some degree, because we wanted them to have some frame of reference for music learning while imagining how they might use these features. Two participants were recreational musicians, one works as a music therapist and has had to learn new instruments as part of her job, and one was a semi-professional musician and composer.

The features: • Discussion forums where a music community can swap advice and suggestions. • A place where you can publicly post a recording of yourself playing and get

feedback from others. • A way to illustrate your musical progress and share it with others. • A way to keep track of how many hours you’re practicing.

Results%and%Analysis!

Tree Test

We wanted to see if participants could find and identify Log a practice, Virtual WIP workshops, and Plucked strings (under Forums). Of the three tasks, participants found the WIP Workshops under Virtual Events the most challenging thing to find in our tree.

Below is a summary and analysis of how each task fared in the test. For the complete results, see the Appendix.

For task #3 (getting advice on which guitar to buy), every participant was able to locate the plucked strings subforum, and all but one reached it by a direct path (ie. they went directly to Forums, then to Plucked strings). However, this task took quite long for people to complete, with the median time being almost 20 seconds.

It seems that the practice of seeking advice in online forums is familiar to most people. We wondered if anyone would struggle with the broader instrument taxonomies; this

�4

seemed not to be the case overall, but the length of time it took to complete this task might indicate some uncertainty over the category Plucked String Instruments. The hesitation may have been because we did not include a subsequent set of options that included guitar specifically (a “guitar” subforum would appear in the final site).

Task #1 (logging a practice session) seemed somewhat less obvious. Out of 9 participants, 1 did not visit Log a Practice under My Music Timeline, instead choosing Virtual WIP Workshops.

It seems that logging a practice session in a section labeled My Music Timeline made sense to most people, though not to everyone. Such ambiguity as there was might be addressed in the initial sign-up process, perhaps by a brief tour. The median time to complete the task was a little over 10 seconds, suggesting that once people found Log a Practice, the meaning of the label was fairly self-evident.

Task #2 (participating in a Virtual Workshop) was successfully completed by only 3 out of 9 participants, and the median time to complete was almost 25 seconds. The correct choice was Virtual WIP Workshops, but other sections people chose were Virtual Open Mic Nights, Virtual Masterclasses, Create a Timeline Post, Composing and Songwriting, and Piano and Other Keyboards.

It seems that the distinctions between the different types of Virtual Events are not clear. However, 6 out of 9 participants did visit Virtual Events to complete this task, and all those that did chose from among the Virtual Events options. The suggests that the connection between the task and the idea of a Virtual Event is somewhat clear, but the fine distinctions are not. In the comments, one participant pointed out that some people might not know what WIP stands for.

Sort and Discuss Proposed Features

The goal of this activity was to provoke discussion and gauge the level of enthusiasm for each feature, as opposed to actually ranking the features against one another.

We were particularly curious whether or not participants would embrace the idea of publicly posting a recording of themselves, and we prompted participants to comment

�5

specifically on the public nature of this feature. Interestingly, only 1 of the 4 participants balked at the thought of using this feature, and it was one who was less advanced in her musical skills. Another participant thought that he would use this feature when we contextualized the feature like this: your recording will be among other recordings at roughly the same skill level. Other participants, when pressed to comment on posting publicly, noted that it was “part of the deal” with that feature, and trusted that commenters would have the good sense and decency to be constructive and not tear them apart.

The idea of recording their hours practiced was very well received, with 3 of the 4 participants ranking this feature as the one they were most likely to use. As one participant noted, “It’s evidence that you’ve actually done something.” When learning an instrument, progress can feel quite slow, and hours practiced are perhaps the easiest and surest way to measure the effort involved.

We were asked a couple times what “illustrate your musical progress” might mean in this context, and when we turned the question back on the participant, 2 decided that it meant recording a polished piece. Another assumed that it followed naturally from “recording hours practiced” and “getting feedback,” but ranked it lower than the first two.

Interestingly, the participants tended to be most dubious about the discussion forums. Many were not sure how reliable such advice would be, and one expressed concern that the advice might not address his specific needs - for example, someone who was not a punk rock fan might not understand the style’s particular aesthetics, and give bad advice in that context.

Recommendations%

Based on the results outlined above, we recommend the following:

1. After a new user signs up, give them a brief “tour” of the user dashboard with an overview of what the main menu categories offer. The tree test results suggest that these categories do bear some relationship to each task in most participants’ minds, but the relationship is not necessarily clear at first glance.

�6

Guiding users through the menu with a brief summary of each section may be enough to familiarize users with features they may not have encountered before, and confirm in the users mind where these features live.

At a minimum, the tour should include the Timeline and the Events sections - the other categories seem to be more self-explanatory. Descriptions could be:

My Music Timeline: This is where you can record your goals, how many hours you’ve practiced, and what you sound like at this moment. As you progress, your Timeline will show you how hard you’ve worked, and how far you’ve come! Virtual Events: Record yourself playing and share it as part of a Virtual Event, or else drop in a see what other people are doing! Events include workshops, masterclasses, open mic nights and competitions.

2. Guide users to the appropriate Virtual Event. The purpose of each individual type of event is not obvious to users at all. One option would be to rename each event; time permitting, it might be interesting to create an activity where participants are given the description, and asked to come up with their own name. If any patterns emerge we can incorporate them into the new names.

Without time for several more rounds of testing, however, one solution is to include a guide on the events page. The main events page could be divided into four sections, each with a heading in the form of a question addressing the user’s goals. For example, “Do you want to get feedback on something that you’re working on?” Once a user arrives at the Events page, this may be enough to clarify the distinctions between different types of events.

3. Consider whether any event categories could be merged. Keeping the number of events categories to a bare minimum would also help ease confusion. On the one hand, if distinct categories can help guide users to the content that best meets their needs, it may be worth having them learn what each category entails. However, if some categories are quite similar - Masterclasses and Workshops, for example - it might be worth merging them and reducing the number of new things the user has to learn.

�7

4. Eliminate acronyms from all menu categories. Specifically, remove “WIP” from the event category, or else write it out as “Work-in-Progress.”

5. Prioritize some version of the hour-tracking feature and the Virtual Events feature. While these might be two of the more difficult features to build, there does seem to be real interest in them.

�8

Appendix(–(Tree(Test(Results( First click:

(

Paths:

( (

Destinations: