1

Peer Interaction Effectively, yet Infrequently, Enables Programmers to Discover New Tools

Emerson Murphy-HillNorth Carolina State University

Gail MurphyUniversity of British Columbia

2

Background

Emerson’s Problem– I was making a bunch of new user

interfaces for refactoring tools– They made it easier for programmers

to restructure their code– But what if people didn’t know of the

tools existed?

3

Background

Gail’s Problem– Was working on a recommender

system called Spyglass that recommended tools in Eclipse

– In a longitudinal study, only 2 out of 7 participants thought that the recommendations were useful

But we really expected that a recommender system would help!

= ?MS Clippy

4

Why we Thought Spyglass Would Work

Spyglass (and Clippy) is similar to recommendations made by peers, which may be effective– Literature suggests it’s similar; recommender system should

“guide and advise an user similar to a knowledgeable colleague or assistant” [Fisher 84]

– Literature suggests learning from peers works “exceedingly effectively” [Lave and Wegner 91]

But we really expected that a recommender system would help!

= ?

But we don’t really know how tool recommendations are made between peers

MS Clippy

5

A bit more about what we were looking for

Peer RecommendationYou discover when a peer is watching you not use a tool when you could be, then she recommends it

Peer ObservationYou discover by watching a peer using a tool that you didn’t know about

We call these both Peer Interaction, one kind of discovery mode

6

Participants

Sent invitations to 62 participants of Open Source Bridge 2009

Asked personal contacts at 7 large companies to distribute

18 participants total

All programmed as part of their jobs, though not necessarily “Programmers”

7

MethodologySemi-structured phone, Skype, or chat interviews

– Gave participants a list of tools, asked participants to pick a tool and tell me how you discovered it

– Gave participants a list of discovery modes, asked participants to pick 2 most effective

– Asked participants to recall instances of peer interaction

– Asked about perceptions of Spyglass

Frequency

Process

Effectiveness

RecSys

8

Frequency

Process

Effectiveness

RecSys

9

FrequencyProcessEffectiveness

Situation Description

Traditional Pair Programming

Two programmers work at the same computer and collaborate to complete the same task.

Happenstance Interaction

One programmer observes another during a chance encounter.

Help Giving A programmer helps a collocated programmer with a task.

Remote Help Giving

A programmer helps a remote programmer with a task.

Remote Pair Programming

Two programmers work at different computers and collaborate to complete a task.

Change Notification

A programmer commits code to version control, and an email is sent to the team about the commit.

Email One programmer observes another’s actions via email.

RecSys

10

FrequencyProcessEffectiveness RecSys

One cause: Learner associates use with context of use

11

FrequencyProcessEffectiveness RecSys

One cause: distributed, asynchronous development.

12

FrequencyProcessEffectiveness RecSys

After we outlined what Spyglass does, several participants immediately mentioned Clippy

“…and Clippy should just go die as far as I’m concerned.”

Or put more politely:

“Honestly; I bet the [recommender system] would have better success rate [than a peer] at recommending things that I would like, but that doesn’t mean that I would trust the [recommender system] more.”

13

Implications

For recommender systems:No matter how accurate, how timely, how politely, or how valuable the recommendation is, programmers don’t trust recommender systems

But what if, rather than foisting recommendations on people, why don’t we just encourage more frequent peer interaction?

14

Making Peer Interaction More Frequent

How can people who work asynchronously and remotely learn from each other?

http://www.flickr.com/photos/novecentino/2339687721/ http://www.flickr.com/photos/totalaldo/2400635097

15

Making Peer Interaction More Frequent

For each developer, keep a local history of: – what tools he uses– a full “movie” of his development history

Compare tool histories between all developersSuggest either:– To Rob: “Your co-worker, Tim, uses ToolX. Would you

like to see an example of him using it yesterday?” or– To Tim: “Your coworker, Rob, doesn’t seem to know

about ToolX. Would you like to share with him an example of you using it?”

http://www.flickr.com/photos/novecentino/2339687721/ http://www.flickr.com/photos/totalaldo/2400635097

Programmers have thousands of useful tools available.

Peer interaction can help programmers effectively discover new tools,but they need support for more frequent interactions.

CONCLUSION16